Evidence-Based Clinical Guidelines for Multidisciplinary Spine Care

Diagnosis and Treatment of Adults with Neoplastic Vertebral Fractures

Guideline Development Methodology & Process

Guideline Development Methodology

Through objective evaluation of the evidence and transparency in the process of making recommendations, it is NASS’ goal to develop evidence-based clinical practice guidelines for the diagnosis and treatment of adult patients with various spinal conditions. These guidelines are developed for educational purposes to assist practitioners in their clinical decision-making processes.

Multidisciplinary Collaboration

With the goal of ensuring the best possible care for adult patients suffering from spinal disorders, NASS is committed to multidisciplinary involvement in the process of guideline development. To this end, NASS has ensured that representatives from research, medical, interventional and surgical spine specialties have participated in the development and review of NASS guidelines. To ensure broad-based representation, NASS welcomes input from other societies and specialties.

Evidence Analysis Training of All Guideline Developers

As a condition of participation, all developers completed NASS’ Evidence-Based Medicine Training prior to participating in guideline development. The training includes a series of readings and exercises to prepare guideline developers for systematically evaluating literature and developing evidence-based guidelines. Participants are awarded CME credit upon completion of the course.

Disclosure of Potential Conflicts of Interest

All participants involved in guideline development have disclosed potential conflicts of interest to their colleagues in accordance with NASS’ Disclosure Policy and their potential conflicts have been documented in the Technical Report associated with this guideline. NASS does not restrict involvement in guidelines based on conflicts as long as members provide full disclosure. Individuals with a conflict relevant to the subject matter were asked to recuse themselves from deliberation. Participants have been asked to update their disclosures regularly throughout the guideline development process.

Levels of Evidence and Grades of Recommendation

NASS has adopted standardized levels of evidence and grades of recommendation to assist practitioners in easily understanding the strength of the evidence and recommendations within the guidelines. The levels of evidence range from Level I (high quality randomized controlled trial) to Level V (expert consensus). Grades of recommendation indicate the strength of the recommendations made in the guideline based on the quality of the literature.

Levels of evidence have specific criteria and are assigned to studies prior to developing recommendations. Recommendations are then graded based upon the level of evidence.

Grades of Recommendation for Summaries or Reviews of Studies as Adopted by the North American Spine Society

· A: Good evidence (Level I studies with consistent findings) for or against recommending intervention.

· B: Fair evidence (Level II or III studies with consistent findings) for or against recommending intervention.

· C: Poor quality evidence (Level IV or V studies) for or against recommending intervention.

· I: Insufficient or conflicting evidence not allowing a recommendation for or against intervention.

Guideline recommendations are written utilizing a standard language that indicates the strength of the recommendation. “A” recommendations indicate a test or intervention is “recommended”; “B” recommendations “suggest” a test or intervention and “C” recommendations indicate a test or intervention “may be considered” or “is an option.” “I” or “Insufficient Evidence” statements clearly indicate that “there is insufficient evidence to make a recommendation for or against” a test or intervention. Work group consensus statements clearly state that “in the absence of reliable evidence, it is the work group’s opinion that” a test or intervention may be appropriate.

In evaluating studies as to levels of evidence for this guideline, the study design was interpreted as establishing only a potential level of evidence. As an example, a therapeutic study designed as a randomized controlled trial would be considered a potential Level I study. The study would then be further analyzed as to how well the study design was implemented and significant shortcomings in the execution of the study would be used to downgrade the levels of evidence for the study’s conclusions. In the example cited previously, reasons to downgrade the results of a potential Level I randomized controlled trial to a Level II study would include, among other possibilities: an underpowered study (patient sample too small, variance too high), inadequate randomization or masking of the group assignments and lack of validated outcome measures.

In addition, a number of studies were reviewed several times in answering different questions within this guideline. How a given question was asked might influence how a study was evaluated and interpreted as to its level of evidence in answering that particular question. For example, a randomized controlled trial reviewed to evaluate the difference between the outcomes of cement augmentation versus medical management of patients with neoplastic vertebral fractures might be a well-designed and implemented Level I therapeutic study. This same study, however, might be classified as providing Level II prognostic evidence if the data for the untreated controls were extracted and evaluated prognostically.

Guideline Development Process

Step 1: Recruitment of Guideline Work Group Members

NASS Clinical Practice Guideline Committee members and members of NASS Sections were solicited to participate in the guideline development process. In total, 27 work group members participated in this effort and are listed as authors. Individuals who were not able to substantively contribute to all activities required for authorship, but participated at some extent throughout the process, have been acknowledged as contributors. Guideline authors and contributors are listed in the Preface and disclosures are listed in the Technical Report associated with this document.

The guideline panel consists of 5 work groups to cover the 7 sections in this guideline: Natural History/Cost Effectiveness, Clinical Diagnosis/Medical Treatment, Imaging Diagnosis/Interventional Treatment (which was further divided into two work groups), and Surgical Treatment. Senior and newer NASS Clinical Practice Guideline members were placed in work groups to ensure that groups with newer members were balanced with members who have more guideline development experience. Each work group consisted of 4 to 6 members representing multidisciplinary backgrounds. The guideline panel includes representation from the fields of neurosurgery, orthopedic surgery, physical medicine and rehabilitation, chiropractic care, physical therapy, anesthesiology, research, radiology, and nursing. NASS believes that having multidisciplinary teams involved in the guideline development process helps to minimize inadvertent biases in evaluating the literature and formulating recommendations.

Step 2: Identification of Work Groups

Step 3: Surveying Patients

To seek patient input to help inform the development of clinical questions, NASS circulated an informal, anonymous, and voluntary Survey Monkey poll to better understand patients’ experiences with vertebral fractures. The survey link was circulated through various websites and social media sites, including NASS’ Facebook and Twitter pages, Know Your Back, eHealth Forums, Daily Strength Support Group, Drugs.com Support Group, Consumers United for Evidence-based Healthcare, and numerous Facebook shares (of the survey link) on consumer and physician profiles. The survey was related to any vertebral fracture and results were shared with the Neoplastic Vertebral Compression Fracture Guideline panel to incorporate patients’ perspectives and priorities when developing questions. The survey included the following questions that allowed for check the box and open-ended responses:

  1. Are you completing this survey as a patient who has experienced a spine compression fracture or as a caregiver on behalf of a patient who has experienced a spine compression fracture?
  2. What symptoms made you or the person you care for seek medical attention for a spine compression fracture? (Check all that apply)
  3. Please identify the treatment(s) received for compression fracture (check all that apply)
  4. When considering options, how important are the following outcomes to you or the patient you care for?
  5. Is there anything that you wish your healthcare provider had shared with you before making your decision to receive treatment for your compression fracture?
  6. What questions do you think patients with compression fractures should ask their healthcare professional when seeking diagnosis and treatment options for compression fractures?
  7. Do you have any additional comments?

Framing questions to ask in the guideline is critical to the guideline development process. Trained guideline panelists were asked to submit a list of clinical questions with the patient survey as reference. Members were asked to use the acronym “PICO” when drafting questions if possible. “PICO” serves to guide the development of clinical questions that include all of the necessary components to build a literature search: “P” for the patient/problem; “I” for the intervention or indicator of interest (procedures, therapies, diagnostic tests, exposure, etc.); “C” for comparison and “O” for outcome of interest. The proposed questions were compiled into a master list and circulated to each member for review and comment. The draft clinical question list was then submitted to the NASS Health Policy Council, Research Council, and Advocacy Council Chair for review. The councils submitted additional questions that may be useful for health policy or research purposes.

Step 4: Identification

of Clinical Questions

Step 5: External Review of Clinical Question Protocol

The draft list of clinical questions was made publicly available on the NASS website for a 4-week public comment period from April 15, 2020 to May 15, 2020. Additionally, stakeholders were invited through email solicitations to comment on the draft questions. Based on feedback, several revisions were incorporated in the clinical question list. After the comment period, an updated clinical question list with summarized changes was posted to the NASS website and circulated to all public comment period reviewers.

One of the most crucial elements of evidence analysis is the comprehensive literature search. Thorough assessment of the literature is the basis for the review of existing evidence and the formulation of evidence-based recommendations. In order to ensure a thorough literature search, NASS has instituted a Literature Search Protocol (Appendix A) which has been followed to identify literature for evaluation in guideline development. In keeping with the Literature Search Protocol, work group members have identified appropriate search terms and parameters to direct the literature search. Specific search strategies, including search terms, parameters and databases searched, are documented in the Technical Report associated with this document. The guideline definition and inclusion/exclusion criteria are outlined in the introduction. Methods were determined and agreed upon a priori and not changed after the literature search.

Step 6: Identification of Search Terms and Parameters

Step 7: Completion of the Literature Search

Once each work group identified search terms/parameters, the literature search was implemented by a medical/research librarian at the American Academy of Orthopaedic Surgeons (AAOS), consistent with the Literature Search Protocol. Following these protocols ensures that NASS recommendations (1) are based on a thorough review of relevant literature; (2) are truly based on a uniform, comprehensive search strategy; and (3) represent the current best research evidence available. NASS maintains a search history in Endnote for future use or reference.

Using Rayyan1, at least 2 work group members independently reviewed all abstracts yielded from the literature search and subsequently the full-text stage to identify the literature they would review in order to address the clinical questions, in accordance with the Literature Search Protocol (Appendix A). Discrepancies were discussed until consensus was reached.

Step 8: Review of Search Results/ Identification of Literature to Review

Step 9: Evidence Analysis

Members independently developed evidentiary tables summarizing study conclusions, identifying strengths and weaknesses (risk of bias) and assigning levels of evidence. In order to systematically control for potential biases, two or more work group members have reviewed each article selected and independently assigned levels of evidence to the literature using the NASS levels of evidence. Any discrepancies in scoring were resolved by discussion by two or more reviewers, and if necessary, arbitration with a larger group including work group authors, section chairs, and project lead. Final ratings are completed at a final meeting or web conference of section workgroup members including the section chair and a guideline co-chair. Many articles have external funding sources (eg, government, research philanthropy, industry) and are described in the individual manuscript. When easily identifiable, industry funding was noted in the guideline article summaries. As a final step in the evidence analysis process, members have identified and documented gaps in the evidence to educate guideline readers about where evidence is lacking and help guide further needed research.

Work groups held virtual meetings to discuss the evidence-based answers to the clinical questions, the grades of recommendations and the incorporation of expert consensus. Expert consensus was incorporated only where Level I-IV evidence is insufficient and the work group has deemed that a recommendation is warranted. Transparency in the incorporation of consensus is crucial and all consensus-based recommendations made in this guideline very clearly indicate that Level I-IV evidence is insufficient to support a recommendation and that the recommendation is based only on expert consensus.

Consensus Development Process

For recommendations with a consensus grading, voting was conducted using a modification of the nominal group technique in which each work group member independently and anonymously ranked a recommendation on a scale ranging from 1 (“extremely inappropriate”) to 9 (“extremely appropriate”). Consensus was obtained when at least 80% of work group members ranked the recommendation as 7, 8, or 9. When the 80% threshold was not attained, up to three rounds of discussion and voting were held to resolve disagreements. If disagreements were not resolved after these rounds, no recommendation was adopted. After the recommendations were established, work group members developed the guideline content, addressing the literature supporting the recommendations.

Recommendation Development

In order to capture all relevant literature on this topic, the questions were formed to cover a broad population and did not necessarily focus on subcategorization (e.g., patients with or without mechanical instability, or patients with or without severe epidural spinal cord compression). After the literature search was complete, a subcategory recommendation would be developed if there was enough evidence available to support. If the literature did not allow subcategorization, a general recommendation was made whenever possible.

Step 10: Formulation of Evidence-Based Recommendations & Incorporation of Expert Consensus

Step 11: Internal Review of Draft Guideline

Guideline sections were reviewed by the section work groups that developed them. The full guideline draft was then reviewed by the entire author group. Revisions to recommendations were considered only when substantiated by a preponderance of appropriate level evidence.

In order to assess clarity and perception of quality, members of NASS’ Clinical Practice Guidelines Committee who were not involved in the development of this guideline were invited to review the draft guideline. These external reviewers were given the opportunity to complete the Appraisal of Guidelines for Research & Evaluation II (AGREE II) instrument and provide open-ended comments. Additionally, a public comment period was open from November 1, 2023 to November 30, 2023. Members of the public, including but not limited to subject matter experts and societies, were eligible to complete a form with rating scales and open-ended questions in order to gather feedback on draft recommendations, assess readability, content, and feasibility (eg, facilitators and barriers to implementation), and improve the overall quality of the guideline. Revisions to recommendations were considered only when substantiated by a preponderance of appropriate level evidence.

Step 12: External Review of Draft Guideline

Step 13: Submission for Board Approval

After any evidence-based revisions were incorporated, the drafts were prepared for review and approval from the NASS Evidence Analysis & Research Council, followed by the NASS Board of Directors. Edits and revisions to recommendations and any other content were considered for incorporation only when substantiated by a preponderance of appropriate level evidence.

Following NASS Board approval, the guideline was slated for publication. No revisions were made after submission for publication, but comments have been and will be saved for the next iteration.

Step 14: Submission for Publication

Step 15: Review and Revision Process

The guideline recommendations will be reviewed 3-5 years after publication, or according the NASS’ topic selection and revision protocol. An EBM-trained multidisciplinary team will review and, when necessary, revise or rescind the guideline.

References

1. Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan — a web and mobile app for systematic reviews. Syst Rev. 2016 Dec 5;5(1):210. DOI: 10.1186/s13643-016-0384-4.

2. Brouwers M, Kho ME, Browman GP, Cluzeau F, feder G, Fervers B, Hanna S, Makarski J on behalf of the AGREE Next Steps Consortium. AGREE II: Advancing guideline development, reporting and evaluation in healthcare. Can Med Assoc J. Dec 2010, 182:E839-842; doi: 10.1503/cmaj.090449