Kexin Fu
4/25/2023 Journal Club
Systematic Review and Meta-analysis
Method Introduction
CONTENT
1
Method Application
Systematic Review and Meta-analysis Reading, understanding, interpreting (Sebastian, etc., 2018)
Opioids for Chronic Noncancer Pain
A Systematic Review and Meta-analysis (Jason, etc., 2018)
2
Method Introduction
Scheidt S, Vavken P, Jacobs C, Koob S, Cucchi D, Kaup E, Wirtz DC, Wimmer MD. Systematic Reviews and Meta-analyses. Z Orthop Unfall. 2019 Aug; 157(4):392-399.
1
1. Clearly Defined Research Question And Search Algorithm
2. Blinded Selection Of Relevant Papers By Trained Persons
3. Quality Assessment Of The Studies
4. Evaluation And Summary Of The Evidence
5. Interpretation And Determination Of The Overall Effect
Background of Review：We need reviews
Why we need reviews?
keep up-to-date about the field of research and knowledge
Identify research gaps and generating hypotheses
Pressure to promote academic career in life science
Find safer, better, reproducible treatment
save and time and money to research on same topics
Increasing publication require compressed and objective review
......
Background of Review: Two types of reviews
Narrative Review (NR)
Systematic Review (SR)
A general summary of the current state of publications on a topic.
Features :
original, narrative, more subjective,
help shine a light on specific questions,
less sophisticated, less time-consuming
require experience and knowledge,
limited evidence level
A review of the evidence on a clearly formulated question that uses systematic and explicit methods to identify, select and critically appraise relevant primary research, and to extract and analyze data from the studies that are included in the review.
Features :
updated, more objective, standardized, planned and reproducible, reliable, larger sample size to offset statistically weakness, high evidence level, evidence-based
Background of Review：Systematic Review (SR) and Meta-analysis
“Meta-analyses are the further development of SRs in the sense that they provide a quantitative and mathematical-statistical summary of individual studies.”
Meta-analysis is often used in conjunction with systematic reviews to synthesize the findings of multiple studies on a specific topic.
Systematic review provides a thorough and systematic summary of all the available literature on a topic
Meta-analysis provides a quantitative summary of the results of multiple studies.
Methods for SR and meta-analysis：Overview
Clearly Defined Research Question And Search Algorithm
Blinded Selection Of Relevant Papers By Trained Persons
Quality Assessment Of The Studies
Evaluation And Summary Of The Evidence
Interpretation And Determination Of The Overall Effect
1. Clearly Defined Research Question And Search Algorithm
Defined Research Question
P Adults (age 18 years and above), both sexes, all ethnicity, all nationality
C Placebo, Or No Intervention, or Anxiolytics Or Traditional Approaches, or Drug Based Approaches, or Other Cognitive Behavioural Therapy
O Anxiety Symptom Scores, or Generalised Anxiety
I Mindfulness Meditation
T 1 year
Goal: Explore if minduflness meditation is effective for anxiety
Question Among Adults, compared with all other approaches, what is the effectiveness of Mindfulness Meditation for the relief of Anxiety?
1. Clearly Defined Research Question And Search Algorithm
Search Algorithm
Boolean Logic: “AND”, “OR”, “NOT”
Fuzzy logic: “NEAR”, “WITHIN 5 Words”
Specific Symbols: “*” , “$” （truncation)
Title/Abstract/Full text...
CINAHL, EMBASE, PubMed, MEDLINE...
Always RCTs?
2. Blinded Selection Of Relevant Papers By Trained Persons
2.1 Eligibility
2.2 Exclusion criteria
2.3 Independent screening
2.4 Full text review
3. Quality Assessment Of The Studies
Newcastle-Ottawa Scale
The selection of the study populations
The comparability of the subgroup
The ascertainment of the therapeutic effect
https://www.ohri.ca/programs/clinical_epidemiology/oxford.asp
Cochrane Risk
In the field of evidence-based medicine, the Cochrane Risk of Bias (RoB) is a tool used to assess the quality of randomized controlled trials (RCTs) and other types of study designs. It involves evaluating the risk of various biases, such as selection bias, performance bias, detection bias, attrition bias, and reporting bias, in the design, conduct, and reporting of a study. The Cochrane RoB tool is commonly used by systematic reviewers and meta-analysts to assess the internal validity of studies included in their reviews, and to determine the overall quality of evidence.
https://sites.google.com/site/riskofbiastool/welcome/rob-2-0-tool?authuser=0
4. Evaluation And Summary Of The Evidence：Forest Plot
4. Evaluation And Summary Of The Evidence: Test for heterogeneity
common effect
within-study error
variance of effect
4. Evaluation And Summary Of The Evidence: Effect Models
“fixed effects model” and “random effects model”
The first model assumes a constant therapeutic effect （studies included are homogeneous) in the meta-analysis, whereby the second model presupposes that there exists other confounders （studies included lack homogeneous) and that the result of the analysis will therefore, due to the generally wider confidence interval, be more conservative.
“fixed effects model”
for single study included, the effect is:
weighted effect is:
# of studies
“random effects model”
for single study included, the effect is:
within-study error
mean of all true effects
between-study error
The difference between common effect and mean of all true effects is the variance of each single study effect
estimated between-group variance
DerSimonian & Laird Method:
weighted effect is:
Grading of Recommendations, Assessment, Development and Evaluation (GRADE)
GRADE provides a transparent and systematic approach to evaluating evidence, considering factors such as study design, risk of bias, inconsistency, indirectness, imprecision, and publication bias, to determine the overall quality or certainty of the evidence. The system categorizes evidence into four levels: high, moderate, low, or very low quality. The quality of evidence is determined based on the cumulative risk of bias and other factors, with higher quality evidence being considered more reliable and informative.
https://gdt.gradepro.org/
4. Evaluation And Summary Of The Evidence: Evidence Evaluation
5. Interpretation And Determination Of The Overall Effect
Funnel Plot: Accessment of publication bias
Method Application
2
1. Clearly Defined Research Question And Search Algorithm
2. Blinded Selection Of Relevant Papers By Trained Persons
3. Quality Assessment Of The Studies
4. Evaluation And Summary Of The Evidence
5. Interpretation And Determination Of The Overall Effect
Busse JW, Wang L, Kamaleldin M, Craigie S, Riva JJ, Montoya L, Mulla SM, Lopes LC, Vogel N, Chen E, Kirmayr K, De Oliveira K, Olivieri L, Kaushal A, Chaparro LE, Oyberman I, Agarwal A, Couban R, Tsoi L, Lam T, Vandvik PO, Hsu S, Bala MM, Schandelmaier S, Scheidecker A, Ebrahim S, Ashoorion V, Rehman Y, Hong PJ, Ross S, Johnston BC, Kunz R, Sun X, Buckley N, Sessler DI, Guyatt GH. Opioids for Chronic Noncancer Pain: A Systematic Review and Meta-analysis. JAMA. 2018 Dec 18;320(23):2448-2460. doi: 10.1001/jama.2018.18472.
Introduction：Opioids for Chronic Noncancer Pain (CNCP)
Chronic Noncancer Pain (CNCP)
Any painful condition that persists for ≥3 months that is not associated with a diagnosis of cancer. The National Center for Health Statistics estimates that 25% of the US population experiences CNCP, Chronic pain not caused by cancer is the primary cause of healthcare resource consumption and disability during adult working years.
Opioid
Opioids are a class of drugs that derive from, or mimic, natural substances found in the opium poppy plant. Opioids work in the brain to produce a variety of effects, including pain relief, and some people use opioids because of the euphoria (“high”) they can produce. Opioids include the 1）illegal drug heroin, 2）synthetic opioids such as fentanyl, and 3）pain relievers available legally by prescription, such as oxycodone (OxyContin®), hydrocodone (Vicodin®), codeine, morphine, and many others. Opioid drugs can cause addiction, also known as opioid use disorder (OUD).
According to the VAS test, non-steroidal drugs(preferred) + weak opioid analgesics, such as tramadol, were recommended for pain assessment at pain level 4-7. Strong opioids, such as morphine and fentanyl, are recommended over 7 points
Introduction：Opioids for Chronic Noncancer Pain (CNCP)
Importance
Harms and benefits of opioids for chronic noncancer pain remain unclear.
Social effect
50 million adults in the United States were living with chronic noncancer pain,many of whom were prescribed opioid medications. The side effect of opioid mediacation include diversion, addiction, overdose, and death. Studies have found a strong correlation between US states with high drug-poisoning mortality and those with high opioid consumption. Opioid overdose is now the leading cause of unintentional death in the USA, having recently overtaken motor vehicle-related fatalities.
Limitation of recent systematic review
1.no eligible RCTs included
2.no latest studies included
3.no test for interaction (valid subgroup analysis) was reported
4.only English articles included
Introduction：Opioids for Chronic Noncancer Pain (CNCP)
Objective
To systematically review randomized clinical trials (RCTs) of opioids for chronic noncancer pain.
Question
Is the use of opioids to treat chronic noncancer pain associated with greater benefits or harms compared with placebo and alternative analgesics?
Method and Result：Overview
Clearly Defined Research Question And Search Algorithm
Quality Assessment Of The Studies
Interpretation And Determination Of The Overall Effect
Blinded Selection Of Relevant Papers By Trained Persons
Evaluation And Summary Of The Evidence
STEP 1
STEP 2
STEP 3
STEP 4
STEP 5
Method and Result：Clearly Defined Research Question And Search Algorithm
Question
Search Strategy
Eligibility (Inclusion & Exclusion Criteria)
Method and Result：Blinded Selection Of Relevant Papers By Trained Persons
Eligibility
Study Selection
Data collection
Method and Result：Blinded Selection Of Relevant Papers By Trained Persons
Eligibility
Study Selection
Data collection
Method
Result
（Agreement between reviewers at the full-text review stage (κ = 0.78)
Method and Result：Blinded Selection Of Relevant Papers By Trained Persons
Eligibility
Study Selection
Data collection
Method
Result
MAIN OUTCOMES AND MEASURES
- emotional functioning to the 100-point SF-36 men_x0002_tal component score;
- role functioning to the 100-point SF-36 subscale for role limitations due to physical problems;
- social functioning to the 100-point SF-36 subscale for social functioning;
- sleep to the SF-36 sleep quality 100-mm VAS
Method and Result：Blinded Selection Of Relevant Papers By Trained Persons
Eligibility
Study Selection
Data collection
Method and Result：Quality Assessment Of The Studies
Method
Result
All trials were at risk of bias for at least 1 of the following domains; however, 51 (53%) adequately generated their randomization sequence, 48 (50%) adequately concealed allocation, 84 (88%) blinded patients, 84 (88%) blinded caregivers, 83 (87%) blinded data collectors, 82 (85%) blinded outcome assessors, and 6 (6%) included a blinded data analyst. There were 73 trials (76%) with frequent (≥20%) missing outcome data
Method and Result：Quality Assessment Of The Studies
Method and Result：Evaluation And Summary Of The Evidence
Forest Plots
Subgroup analysis
Evidence Evaluation
Method
pain relief: 1.0 cm (VAS)
physical functioning and emotional functioning: 5 points (SF-36)
role functioning and social functioning: 10 points (SF-36)
sleep quality: 10 mm (VAS)
Method and Result：Evaluation And Summary Of The Evidence
Forest Plots
Subgroup analysis
Evidence Evaluation
Method
Method and Result：Evaluation And Summary Of The Evidence
Forest Plots
Subgroup analysis
Evidence Evaluation
Method
Result
Method and Result：Evaluation And Summary Of The Evidence ( vs Placebo)
Pain Relief
P = 0.04 for interaction
Method and Result：Evaluation And Summary Of The Evidence ( vs Placebo)
Pain Relief
Result
Although the difference did not reach the minimally important difference of 1 cm, opioids were associated with pain relief compared with placebo (weighted mean difference, −0.79 cm [95% CI, −0.90 to −0.68 cm] on a 10-cm VAS for pain, P < .001; modeled risk difference for achieving the minimally important difference, 13.6% [95% CI, 11.8% to 15.4%]). Studies with longer follow-up reported less pain relief (eFigures 1 and 2 in Supplement 2; P = .04 for interaction).
High-quality evidence from 42 RCTs that followed up patients for 3 months or longer (16617 patients). Opioids were associated with pain relief compared with placebo (weighted mean difference,−0.69 cm[95%CI,−0.82 to −0.56cm] on a 10-cm VAS for pain, P<.001, not reach the minimum important difference of 1 cm with acceptable heterogeneity modeled risk difference for achieving the minimally important difference, 11.9% [95% CI, 9.7% to 14.1%]
Method and Result：Evaluation And Summary Of The Evidence ( vs Placebo)
Physical Functioning
Result
High-quality evidence from 51RCTs (15754 patients) showed opioids were associated with a small improvement in physical functioning compared with placebo, but did not meet the criterion for the minimally important difference (weighted mean difference, 2.04 points [95% CI, 1.41-2.68 points] on the 100-point SF-36 physical component score, P < .001 with acceptable heterogeneity; minimally important difference, 5 points; modeled risk difference for achieving the minimally important difference, 8.5% [95% CI, 5.9%-11.2%]
Method and Result：Evaluation And Summary Of The Evidence ( vs Placebo)
Emotional Functioning
Result
High quality evidence from 23 RCTs (8962 patients). Opioids were not significantly associated with emotional functioning compared with placebo (weighted mean difference, 0.14 points [95% CI, −0.58 to 0.86 points] on the 100-point SF-36 mental component score, P = .70) reporting actual change scores indicated that opioids were not associated with emotional functioning (weighted mean difference, −0.44 points [95% CI, −1.09 to 0.20 points] on the 100-point SF-36 mental component score, P = .53).
Method and Result：Evaluation And Summary Of The Evidence ( vs Placebo)
Role Functioning
Result
Opioids were associated with improved role functioning compared with placebo (weighted mean difference, 2.80 points [95% CI, 0.99 to 4.61 points] on the 100-point SF-36 subscale for role limitations due to physical problems, P < .001); however, the association was smaller in studies with reported vs converted change scores (eFigure 9 in Supplement 2; P = .007 for interaction). When restricted to trials reporting actual change, high-quality evidence from 16 RCTs (5329 patients) demonstrated no association of opioids on role functioning compared with placebo (weighted mean difference, 0.87 points [95% CI, −0.54 to 2.28 points] on the 100-point SF-36 subscale for role limitations due to physical problems, P = .23; Table).
Method and Result：Evaluation And Summary Of The Evidence ( vs Placebo)
Social Functioning
Result
High-quality evidence from 29 RCTs (7623 patients) showed an association of opioids with improved social functioning compared with placebo but did not meet the minimally important difference criterion (weighted mean difference, 1.58 points [95% CI, 0.45-2.70 points] on the 100-point SF-36 subscale for social functioning, P = .006;minimally important difference, 10 points; modeled risk difference for achieving the minimally important difference, 2.6% [95% CI, 0.7%-4.5%];
Method and Result：Evaluation And Summary Of The Evidence ( vs Placebo)
Sleep Quality
Result
Opioids were associated with improved sleep quality compared with placebo (weighted mean difference, 4.56 mm [95% CI, 2.88-6.24 mm] on the SF-36 sleep quality 100-mm VAS, P < .001; minimally important difference, 10 mm; modeled risk difference for achieving the minimally important difference, 5.9% [95% CI, 3.7%-8.1%]; however, the association was smaller in studies with longer follow-up ( P = .03 for interaction). High-quality evidence from 15 RCTs (6585 patients) with follow-up lasting 3 months or longer found that opioids were associated with a small improvement in sleep quality compared with placebo but did not meet the criterion for the minimally important difference (weighted mean difference, 3.42 mm [95% CI, 1.58-5.26 mm] on the SF-36 sleep quality 100-mm VAS, P < .001; modeled risk difference for the minimally important difference, 5.9% [95% CI, 2.8%-9.1%]).
P = .03 for interaction
Method and Result：Evaluation And Summary Of The Evidence ( vs Placebo)
Vomiting
Result
Opioids were associated with an increased incidence of vomiting; however, this association was less in the 18 enrichment RCTs (5961 patients) compared with placebo (RR, 2.50 [95% CI, 1.89-3.30], P < .001; risk difference, 3.6% [95% CI, 2.1%- 5.4%]) than in 33 nonenrichment RCTs (11 268 patients) compared with placebo (RR, 4.12 [95% CI, 3.34-5.07], P < .001; risk difference, 7.1% [95% CI, 5.4%-9.3%]; P = .007 for interaction). At least 20 RCTs reported each of the following adverse events: nausea, constipation, dizziness, drowsiness, headache, pruritus, and dry mouth. Except for headache, opioid use was associated with a higher incidence of these adverse events compared with placebo.
Method and Result：Evaluation And Summary Of The Evidence ( vs Active Comparators)
Method and Result：Evaluation And Summary Of The Evidence ( vs Active Comparators)
Opioids vs Nonsteroidal Anti-Inflammatory Drugs Result:
Moderate-quality evidence from 9 RCTs (1431 patients) showed no difference in the association of opioids vs nonsteroidal antiinflammatory drugs for pain relief (weighted mean difference, −0.60 cm [95% CI, −1.54 to 0.34 cm] on the 10-cm VAS for pain, P = .21). Moderate-quality evidence from 7 RCTs (1311 patients) suggested no difference in physical functioning between opioids and nonsteroidal anti-inflammatory drugs (weightedmean difference, −0.90 points [95% CI, −2.69 to 0.89 points] on the 100-point SF-36 physical component score, P = .33). High-quality evidence from 5 RCTs (2632 patients) showed an association of opioids with vomiting compared with nonsteroidal anti-inflammatory drugs (RR, 4.71 [95% CI, 2.92 to 7.60], P < .001; risk difference, 6.3% [95% CI, 3.2% to 11.1%]
Other Results summary:
Compared with tricyclic antidepressants drug (low-quality; low-quality)
pain:WMD,−0.13cm[95%CI,−0.99to0.74cm];Physical functioning: WMD,−5.31points [95%CI,−13.77 to 3.14points]
Compared with anticonvulsants drug (moderate-quality; low-quality)
pain:WMD,−0.90cm[95%CI,−1.65to−0.14cm]; physical functioning: WMD,0.45points [95%CI,−5.77 to 6.66points]
Compared with synthetic cannabinoids (low-quality; low-quality)
pain:WMD, −0.13 cm [95% CI, −1.04 to 0.77 cm]; physical functioning: WMD, −1.2 points [95% CI,−4.50 to 2.10points]
Compared with usual care (low-quality; low-quality)
pain:WMD, −2.06 cm[95% CI,−2.65 to −1.48 cm]
Method and Result：Interpretation And Determination Of The Overall Effecte
Funnel Plots
Subgroup analysis
Method
assessed publication bias by visual assessment of funnel plot asymmetry and by using the results from the Begg test
Result
Discussion
Conclusion
Method and Result：Interpretation And Determination Of The Overall Effecte
Funnel Plots
Sensitivity test
Discussion
Conclusion
Method
conducted a post hoc sensitivity analysis using the Hartung-Knapp-Sidik-Jonkman method
Result
Method and Result：Interpretation And Determination Of The Overall Effecte
Funnel Plots
Sensitivity test
Discussion
Conclusion
Method and Result：Interpretation And Determination Of The Overall Effecte
Funnel Plots
Subgroup analysis
Discussion
Conclusion
Conclusion and relevant
In this meta-analysis of RCTs of patients with chronic noncancer pain, evidence from high-quality studies showed that opioid use was associated with statistically significant but small improvements in pain and physical functioning, and increased risk of vomiting compared with placebo. Comparisons of opioids with nonopioid alternatives suggested that the benefit for pain and functioning may be similar, although the evidence was from studies of only low to moderate quality.
Meaning
Opioids may provide benefit for chronic noncancer pain, but the magnitude is likely to be small.
Method and Result：Interpretation And Determination Of The Overall Effecte
Funnel Plots
Subgroup analysis
Strength
Limitation
Discussion
Conclusion
Limitation (continue)
Method and Result：Interpretation And Determination Of The Overall Effecte
Funnel Plots
Subgroup analysis
Discussion
Conclusion
Reference
[1] Scheidt S, Vavken P, Jacobs C, Koob S, Cucchi D, Kaup E, Wirtz DC, Wimmer MD. Systematic Reviews and Meta-analyses. Z Orthop Unfall. 2019 Aug;157(4):392-399. English, German. doi: 10.1055/a-0751-3156.
[2] Busse JW, Wang L, Kamaleldin M, Craigie S, Riva JJ, Montoya L, Mulla SM, Lopes LC, Vogel N, Chen E, Kirmayr K, De Oliveira K, Olivieri L, Kaushal A, Chaparro LE, Oyberman I, Agarwal A, Couban R, Tsoi L, Lam T, Vandvik PO, Hsu S, Bala MM, Schandelmaier S, Scheidecker A, Ebrahim S, Ashoorion V, Rehman Y, Hong PJ, Ross S, Johnston BC, Kunz R, Sun X, Buckley N, Sessler DI, Guyatt GH. Opioids for Chronic Noncancer Pain: A Systematic Review and Meta-analysis. JAMA. 2018 Dec 18;320(23):2448-2460. doi: 10.1001/jama.2018.18472.
[3] Arindam Basu. How to conduct meta-analysis: A Basic Tutorial. PeerJ Preprints. 2017 May. https://doi.org/10.7287/peerj.preprints.2978v1
[4] Undertaking Systematic Reviews of Research on Effectiveness. CRD’s Guidance for those Carrying Out or Commissioning Reviews. CRD Report Number 4 (2nd Edition). NHS Centre for Reviews and Dissemination, University of York. March 2001.
[5] Counsell C. Formulating questions and locating primary studies for in_x0002_clusion in systematic reviews. Ann Intern Med 1997; 127: 380–387.
[6] Dowell D, Haegerich TM, Chou R. CDC guideline for prescribing opioids for chronic pain—United States, 2016. MMWR Recomm Rep. 2016;65(1):1-49. doi:10.15585/mmwr.rr6501e1
[7] Nielsen S, Degenhardt L, Hoban B, Gisev N. A synthesis of oral morphine equivalents (OME) for opioid utilisation studies. Pharmacoepidemiol Drug Saf. 2016;25(6):733-737. doi:10.1002/pds.3945
[8] Tendal B, Nüesch E, Higgins JP, Jüni P, Gøtzsche PC. Multiplicity of data in trial reports and the reliability of meta-analyses: empirical study. BMJ. 2011;343:d4829. doi:10.1136/bmj.d4829
[9] Breivik H, Ljosaa TM, Stengaard-Pedersen K, et al. A 6-months, randomised, placebo-controlled evaluation of efficacy and tolerability of a low-dose 7-day buprenorphine transdermal patch in osteoarthritis patients naïve to potent opioids.
Q & A
Kexin Fu
4/25/2023 Journal Club
Thank You