����������������M.S.S. in Health Economics�HE 604: Public Health and Epidemiology (PHE)�Systematic review

Dr. Aninda Nishat Moitry

MBBS, MPH, MSc, FRSPH

03 April 2023

Overview of presentation

• Systematic review
• Meta-analysis

​

​

​

​

What is a Systematic Review?

• In basic terms, a systematic review is a protocol-driven, comprehensive literature review, usually designed to answer a specific clinical question
• A review of the evidence on a clearly formulated question that uses systematic and explicit methods to identify, select and critically appraise relevant primary research, and to extract and analyze data from the studies that are included in the review (Cochrane collaboration)

​

…and meta-analysis?

Statistical combination of >= 2 studies to produce single estimate of effect of exposure

​

meta-analysis is simply the statistical combination of findings; it should not be considered as a type of review

​

Why are systematic reviews necessary

​

• Large volume of published material makes it impractical for an individual person to remain up to date on a variety of common conditions
• This is further complicated when individual studies report conflicting conclusions, a problem that is prevalent when small samples and retrospective designs are used

​

Systematic vs. Narrative reviews

Systematic review

• Scientific approach to a review article
• Criteria determined at outset
• Comprehensive search for relevant articles
• Explicit methods of appraisal and synthesis
• Meta-analysis may be used to combine data

​

Literature review

• Depend on authors’ inclination
• Author gets to pick any criteria
• Search any databases
• Methods not usually specified
• Narrative summary
• Easily doable and less time consuming
• Usually not replicable

Systematic Review in context

(Systematic review) ≠ (Meta–analysis)

• Meta-analysis is the statistical part of a systematic review
• Meta-analysis is always part of a systematic review
• Systematic review does not always have a meta-analytic part

*IPD= individual participant data

Systematic reviews

Meta-analyses

IPD*

Time Commitment?

• On average, systematic reviews require at least 12 months of preparation

Estimated timeline…

    Month                   Activity -

           1 – 2                   Preparation of protocol

           3 – 8                   Searches for published and unpublished studies

           2 – 3                   Pilot test of eligibility criteria

           3 – 8                   Inclusion assessments

           3                         Pilot test of ‘Risk of bias’ assessment

           3 – 10                 Validity assessments

           3                        Pilot test of data collection

           3 – 10                 Data collection

           3 – 10                 Data entry

           5 – 11                 Follow up of missing information

           8 – 10                 Analysis

           1 – 11                 Preparation of review report

           12 –                    Keeping the review up-to-date

​

Minimum Parameters…

1) At least two team members working on the review, ideally three team members.

• Team members needed in order to prevent bias.
• At least one member should ideally be a research librarian

​

2) Clearly defined question (PICO)

​

3) Pre-Specified Inclusion and Exclusion Criteria

• A systematic review is composed of only primary studies

​

4) Comprehensive Search of Literature with Clear Methods/Search Strategy

• Need to at least search

1) Databases of Published Literature

2) Grey Literature - degree dependent on question

3) Reference List of Key Articles

4) Cited Reference Search of Key Articles

​

Minimum Parameters…(Cont.)

5) Critical Evaluation of Included Articles/Studies

​

6) Quantitative or Qualitative Data Synthesis

​

7) Evidence Based Conclusions

​

​

Part of Criteria Come From:

​

Cook, D. J., Mulrow, C. D., & Haynes, R. B. (1997). Systematic reviews: synthesis of best evidence for

clinical decisions. Annals of internal medicine,126(5), 376-380.

​

Resources required for systematic reviewing

Can be time consuming

Team science (to reduce bias)

Bibliographic software (e.g. Endnote)

Statistical software (if appropriate)

Before You Begin: Ask?

• Do I have a clearly defined clinical question with established inclusion and exclusion criteria?

​

• Do I have a team of at least three people assembled?

​

• Do I have time to go through as many search results as we might find?

​

• Do I have resources to get foreign-language articles appropriately translated?

​

• Do I have the statistical resources to analyze and pool data?

​

Key elements of a systematic review

• Structured, systematic process involving several steps :
• Identify the problem
• Formulate the question
• Plan the review
• Comprehensive search
• Unbiased selection and abstraction process
• Critical appraisal of data
• Synthesis of data (may include meta-analysis)
• Interpretation of results

​

Step 1

Identification of problem

Problem Identification

​

• Literature review
• Expert consultation

​

Step 2

Formulate research question

FINER criteria for research question

Feasible

Interesting

Novel

Ethical

Relevant

​

Hulley S, et al. 2001 Designing Clinical Research

Patient:

Disease or condition

Stage, severity

Demographic characteristics (age, gender, etc.)

Intervention (or “Exposure”):

Type of intervention or exposure

Dose, duration, timing, route, etc.

Comparison:

Absence of risk or treatment

Placebo or alternative therapy

Outcome:

Risk or protective

Dichotomous or continuous

Type: mortality, morbidity, quality of life, etc.

​

Components of a research question (PICOT)

​

Type of Study:

RCTs

Cohort

Case-control

Cross-sectional

All

PICOT

Formulation of an etiology question

Is smoking a risk factor for lung cancer?

Are people who smoke regularly at a greater risk of developing lung cancer as compared to those who do not smoke?

Exposure

Outcome

Exposure

Patient

Outcome

Comparison

+ cohort & case-control studies

Formulation of a diagnosis question

Is MRI a good screening test for breast cancer?

Is MRI a more sensitive and specific test in diagnosing breast cancer as compared to mammography among

high risk women?

Test (intervention)

Outcome

Outcome

Comparison

Test (intervention)

Step 3

Develop review protocol

Plan the Review

​

• Prepare a protocol
• Register the protocol

​

Protocol

Background

Objectives

Pre-determined selection criteria

Planned search strategy

Planned data abstraction

Proposed method of synthesis of findings

Establishment of an advisory group

​

Step 3

Initiate search strategy

Where to locate studies

• Pubmed/ Medline
• CINAHL
• ISI Web of Science
• EMBASE
• PsychINFO
• Cochrane Review
• SCOPUS

​

​

​

Additional sources to identify studies for systematic reviews�

Reference lists of retrieved articles

Manual searching of relevant publications

Experts in the field

Corresponding or first authors of published

studies identified for the systematic review

Issues to consider

Publication bias

Search bias

Pubmed citation example

Title: Interaction between 5-HTTLPR genotype, stressful life events and depression

​

Search terms:

Life stress

Life event

Depression

Depress

Serotonin transporter

5-HTTLPR

Interaction

Moderation

​

Risch et al. JAMA 2010

Step 4

Apply inclusion /exclusion criteria

Inclusion/exclusion criteria

P - Population

I - Intervention

C - Comparison (if necessary)

O - Outcome

T - Type of study (if necessary)

​

Subject headings OR Textwords

To find studies using all of the PICO elements:

P and I and C and O (and T)

Exclusion criteria

Keep log of excluded studies

Note reasons for exclusion

Have eligibility checked by more than one reviewer

Develop strategy to resolve disagreements

Search strategy example

Risch et al. JAMA 2010

Step 5

Quality appraisal

Principles of quality appraisal

Quantitative studies

Internal Validity

allocation bias, confounding, attrition, statistical

analysis, intervention integrity, withdrawals and

dropouts

​

External Validity (generalizability or applicability)

Are there quality appraisal tools?

Step 6

Data abstraction

Data abstraction

​

Design and pilot data abstraction form

​

Consider >1 reviewer

​

Consider blinding of observers to authors, institutions and journals

​

Data abstraction elements

publication details

study design

population details (n, characteristics)

intervention details

setting

outcomes and findings

​

Summary of study characteristics - example

Risch et al. JAMA 2009

Data abstraction examples

http://www.cochrane.org

​

Step 7

Analysis

Synthesizing the Evidence

NARRATIVE SYNTHESIS

primary studies explained qualitatively and summarized

​

​

META-ANALYSIS

findings summarized and then combined statistically

Is there heterogeneity?

No

Yes

Meta-analysis

Narrative synthesis

Deal with heterogeneity?

(e.g. subgroup analysis)

Step 8

Interpret findings

Interpretation of results

​

Strength of the evidence

Explanations of effectiveness

Applicability

Trade-offs between benefits and harms

Implications for practice

​

Strength of the evidence

• Describe the overall strength of the evidence
• The quality of the evidence
• The size and consistency of the results (particularly important in population based studies where a small effect at the community level may have much significance than the effect of comparable size at the individual level)
• Describe biases and limitations of the review process (e.g. difficulties in locating articles, inability to translate non-English articles)

Thank You