4. Scenario Writing. Key modifiers were determined based on variables that were thought to be critical in decision making for use of chemoprophylaxis following spine surgery. Modifiers included timing for intervention, type of condition, type of surgery, duration of surgery, presence of spinal cord injury, or other significant risk factors for VTE. From these modifiers, a matrix was constructed of all possible modifier combinations, resulting in 96 scenarios (Table 2). All scenarios were presented for two interventions: 1) initiating pharmacologic prophylaxis within the first 24 hours postoperatively, and 2) initiating pharmacologic prophylaxis between 1-4 days postoperatively. No attempt was made to define the type of pharmacologic intervention; just that chemoprophylaxis was reasonable to do. Scenarios were intended to represent common scenarios that arise during clinical practice. Conflicts of interest are permissible within the Scenario Writing Work Group as long as writers adhere to the NASS Disclosure Policy.

5. Scenario Review. A separate group independently reviewed the scenarios, which is the primary difference from the RAND/UCLA methodology. This additional step provides additional opportunity to ensure optimal development of scenarios. Feedback was given to the Scenario Writing Work Group to consider further refinement and a final draft of scenarios was created. Conflicts of interest are permissible as long as reviewers adhere to the NASS Disclosure Policy.

6. Literature Review Group. Concurrent to scenario writing and review, a literature review was conducted by the CPG committee members on this topic and used by the AUC work group.2-62 Members of the CPG committee are required to be trained in evidence-analysis techniques. Literature review and completion of accompanying evidentiary tables is a critical component to the process. All CPG recommendations were provided to the AUC raters to assist them with informed voting. All references and evidentiary tables can be found on the NASS website, www.spine.org.

7. Rating. A multidisciplinary group of 13 raters were identified representing the specialties of orthopedics, neurosurgery, physical medicine and rehabilitation, hospital medicine, inpatient spine surgery physician assistants, general surgery, and neurology. Members were all considered to be thought leaders in their fields. There were a total of 6 nonoperative specialists and 7 surgeons. Three raters were part of the ongoing CPG development in this topic. The moderator was experienced in AUC development and was not a member of the CPG. Raters were not required to be trained in evidence analysis. They were required to have participated in the NASS disclosure process.

The group was introduced to the scenarios and the rating method on a prerating conference call which was also recorded. All conflicts were disclosed on this call, and for this AUC, no panel member had a conflict relevant to this topic. Scenarios were to be rated based on what was appropriate (reasonable), and not necessarily reflective of the rater’s individual practice. The raters each rated the scenarios independently and anonymously.

A second interactive virtual meeting was then scheduled to discuss the scenarios and participate in a second round of anonymous rating. This started with introductions and relevant disclosures. Deidentified scores from the first round of rating were compiled and presented to the group to facilitate discussion. Scenarios were clarified if needed. Most of the discussion centered around scenarios where more disagreement existed. Consensus was not a goal, and costs were not considered. Raters were directed to consider whether a procedure was reasonable, in general, relative to the scenarios presented.

Each treatment was evaluated on its own without comparison to other scenarios. Raters combined evidence with personal experience and voted on the appropriateness of each scenario using a 1 through 9 scale. The votes were recorded, and the median used to determine the final score in line with RAND/UCLA methodology. Scores of 1 to 3 indicate the procedure is rarely appropriate, 4 to 6 uncertain/may be appropriate, and 7 to 9 appropriate. An option of N/A was included in rating each scenario in the event that it was thought to be clinically implausible. Each treatment was considered independently, resulting in some treatments receiving similar ratings for the same scenarios.

8. Agreement. As previously described in the RAND/UCLA manual, scenarios were characterized as ‘with agreement’ when the interpercentile range adjusted for symmetry (IPRAS) of the ratings was greater than the interpercentile range (IPR) required for disagreement with perfect symmetry. When the IPRAS was less than the IPR, the scenario was characterized as ‘with disagreement’. This method has previously demonstrated higher sensitivity and specificity for identifying disagreement than prior classical methods.

9. Final Rating. All scenarios received a final rating based on the median score and whether agreement existed amongst the raters. The final ratings were as follows: • 1 to 3 = Rarely appropriate with agreement • 4 to 6 = Uncertain/May Be Appropriate or Disagreement • 7 to 9 = Appropriate with Agreement. Independent of the median score, all scenarios with disagreement were assigned a final rating of ‘Uncertain/May Be Appropriate or Disagreement.’