Abstract
We previously conducted an exploration of the trustworthiness of a group of clinical trials of cognitive-behavioral therapy and exercise in spinal pain. We identified multiple concerns in 8 trials, judging them untrustworthy. In this study, we systematically explored the impact of these trials (“index trials”) on results, conclusions, and recommendations of systematic reviews and clinical practice guidelines (CPGs). We conducted forward citation tracking using Google Scholar and the citationchaser tool, searched the Guidelines International Network library and National Institute of Health and Care Excellence archive to June 2022 to identify systematic reviews and CPGs. We explored how index trials impacted their findings. Where reviews presented meta-analyses, we extracted or conducted sensitivity analyses for the outcomes of pain and disability, to explore how the exclusion of index trials affected effect estimates. We developed and applied an ’Impact Index’ to categorize the extent to which index studies impacted their results. We included 32 unique reviews and 10 CPGs. None directly raised concerns regarding the veracity of the trials. Across meta-analyses (55 comparisons), the removal of index trials reduced effect sizes by a median of 58% (Inter Quartlie Range (IQR) 40–74). 85% of comparisons were classified as highly, 3% as moderately, and 11% as minimally impacted. Nine out of 10 reviews conducting narrative synthesis drew positive conclusions regarding the intervention tested. Nine out of 10 CPGs made positive recommendations for the intervention(s) evaluated. This cohort of trials, with concerns regarding trustworthiness, has substantially impacted the results of systematic reviews and guideline recommendations. Perspective: We found that a group of trials of CBT for spinal pain with concerns relating to their trustworthiness has had substantial impacts on the analyses and conclusions of systematic reviews and clinical practice guidelines. This highlights the need for a greater focus on the trustworthiness of studies in evidence appraisal. Pre-registration: Our protocol was preregistered on the Open Science Framework: https://osf.io/m92ax/
| Original language | English |
|---|---|
| Pages (from-to) | 2103-2130 |
| Number of pages | 28 |
| Journal | Journal of Pain |
| Volume | 24 |
| Issue number | 12 |
| Early online date | 13 Jul 2023 |
| DOIs | |
| Publication status | Published - 31 Dec 2023 |
Data Availability Statement
The full data underpinning the analysis of the impact of studies on published meta-analyses are available via Figshare: https://doi.org/10.17633/rd.brunel.21427995.v1Funding
This study was not supported by any external funding. Michael Wewege was supported by a Postgraduate Scholarship from the National Health and Medical Research Council of Australia and a PhD Supplementary Scholarship from Neuroscience Research Australia. Christopher Eccleston undertook consultancy for Orion Pharma and Reckitt on the psychology of pain. He is a member of the UK Pain Parliamentary Campaign Group with a focus on raising awareness in government on the needs of pain patients. Amanda Williams undertook consultancy for Reckitt on the psychology of pain. NOC was a member of the Guideline Development Group for the 2016 NICE guidance for low back pain and Sciatica (NG59) which is included in this study. AW, CE, EF, and LH were authors on one included review. 5 They were not involved in any data extraction or analysis related to that review for this project.
Keywords
- Clinical practice guidelines
- Clinical trials
- Spinal pain
- Systematic reviews
- Trustworthiness
ASJC Scopus subject areas
- Clinical Neurology
- Neurology
- Anesthesiology and Pain Medicine