1 of 29

A Brief Overview of �Systematic Reviews

Professor Nicole Williams

CUNY School of Medicine Medical Library

2 of 29

Learning Outcomes

Define the purpose and outcomes of systematic review projects

1

Understand the role of systematic reviews in the context of other forms of evidence-based research synthesis

2

Examine systematic review requirements and best practices

3

Identify library resources and services that support systematic reviews

4

3 of 29

What is a systematic review?

4 of 29

Systematic Reviews are the highest level of evidence synthesis.

  • The robust process and documented methodology work to eliminate bias and ensure full transparency in the data collection process, which make systematic reviews ideal for answering questions relating to healthcare and medicine.

5 of 29

The Evidentiary Pyramid in Evidence-Based Medicine

6 of 29

Good systematic reviews are:

Rigorous

Unbiased

Transparent

Reproducible

7 of 29

Reviews in Comparison

​

Systematic Reviews

Narrative Literature Reviews

Review Question

  • Clearly defined, focused, and answerable clinical question
  • PICO Questions: Patient/Population/Problem, Intervention, Comparison, Outcome
  • Can be general topic or specific question

Search Process

  • Perform searches across multiple databases
  • Attempt to local all relevant studies
  • Systematic, comprehensive, reproducible
  • Document the search line by line
  • Perform searches of one or more databases
  • No attempt to be exhaustive or locate all relevant studies

Inclusion and Exclusion Criteria

  • Explicit, predetermined descriptions of what types of studies are to be included to minimize selection bias
  • Usually do not describe why certain studies are included and others excluded

Number of Authors

Three or more

One or more

Time

Long (6 months – Multiple years)

Shorter (One day – a few months)

Value

Connects practicing clinicians to high quality evidence. Supports Evidence Based Practice (EBP) and healthcare decision making.

Provides summary of literature on a topic

8 of 29

SR methodology is defined by standards.

  • Cochrane: The Cochrane Handbook for Systematic Reviews of Interventions

​

  • Institute of Medicine (US) Committee on Standards for Systematic Reviews of Comparative Effectiveness Research
  • PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses

​

  • National Academy of Medicine (formerly IOM): Finding What Works in Health Care: Standards for Systematic Reviews

​

9 of 29

PRISMA (2020) Flow Diagram

10 of 29

PRISMA �(2020) �Checklist

11 of 29

PRISMA �(2020) �Checklist

12 of 29

Systematic Reviews and the Clinical Researcher

Use of Primary Studies

  • The data and studies published by clinical researchers form the backbone of systematic review research
  • Knowledge of the process can enhance the discoverability and impact of your work

Evidence Synthesis Research

  • As an author and researcher, you may wish to synthesize existing research to validate or discover “answers” to clinical questions via a validated methodological approach.

13 of 29

14 of 29

Developing a Research Topic

  • Questions should be
    • Well-defined
    • Answerable
    • Searchable

​

  • Consider using PICO or another standardized structure
  • Check for already published literature
    • PubMed
    • MEDLINE
    • Embase
    • Cochrane Library
    • PROSPERO

15 of 29

Protocol Formation

  • The Protocol acts as the “rule book” for the review. It establishes the methodology and eliminates variation, bias, and “scope creep” later in the process. 
    • Protocols are essential to the SR process
  • Completed protocols should be registered or otherwise published in such a way that other researchers in the field can find the protocol - reducing the risk of a group attempting a review on the same topic.

​

16 of 29

Developing Inclusion & Exclusion Criteria

Inclusion Example

  • Articles related to disease, intervention, or phenomenon of interest
  • Articles in English
  • No geographic or publishing date restrictions

Exclusion Example

  • Articles not related to research question
  • Review articles
  • Clinical Trials
  • Grey literature

17 of 29

Searching for Studies

  • Database search plan
    • At least 3 databases with expert search competency (such as your institutional librarians)
    • Other sources
  • Retrieval > Relevance
  • Comprehensive, reproducible, exhaustive
  • Ample documentation and tracking of results

18 of 29

Example Search Strategy

Concept: Cannabidiol use and surgery outcomes in adolescents

MEDLINE/OVID

​

('cannabis'/exp OR cannabis OR 'cannabidiol'/exp OR cannabidiol OR 'cannabinoid'/exp OR cannabinoid OR 'marijuana'/exp OR marijuana OR thc) AND ('pain management'/exp OR 'pain management' OR 'analgesia'/exp OR analgesi* OR 'analgesic'/exp) AND (preoperative OR perioperative OR postoperative OR 'surgery'/exp OR surg*) AND ('young adult*' OR 'adolescent'/exp OR adolescen* OR teenage*)

PubMed

("Cannabis"[Mesh] OR marijuana OR cannabi* OR tetrahydrocannabinol) AND (pain OR "pain management" OR "Analgesia"[Mesh] OR analgesi*) AND ( "General Surgery"[Mesh] OR "Surgical Procedures, Operative"[Mesh] OR "surgery" [Subheading] OR surg* OR perioperative OR postoperative OR preoperative OR operati*) AND ("Young Adult"[Mesh] OR "adolescent"[Mesh] OR teenage* OR youth OR juvenile)

Embase

('cannabis' OR 'cannabis'/exp OR cannabis OR 'cannabidiol'/exp OR 'cannabidiol' OR cannabi* OR 'cannabinoid'/exp OR 'cannabinoid' OR 'marijuana' OR 'marijuana'/exp OR marijuana OR 'tetrahydrocannabinol'/exp OR tetrahydrocannabinol OR thc) AND ('pain'/exp OR pain OR 'pain management'/exp OR 'pain management' OR 'analgesia'/exp OR 'analgesia' OR analgesi* OR 'analgesic'/exp OR 'analgesic') AND (operation* OR 'surgery'/exp OR 'surgery' OR surg*) AND ('adolescent'/exp OR 'adolescent' OR adolescen* OR teenage* OR 'young adult*')

19 of 29

Managing References: EndNote 20

Embase, Medline, CINAHL, Scopus

Articles from databases

EndNote 20

Remove duplicates, retrieve full text, export to screening tool, + more

20 of 29

Article screening and selection

Usually completed in 2 stages

Title/abstract screening

Full text screening

Two independent (blinded) reviewers, with third as tie breaker

Log is kept with data from each stage of screening with number of articles excluded (with reasons)

Tools to assist:

Rayyan

Microsoft Excel

EndNote

21 of 29

Data Extraction

Data is extracted from the included studies for analysis, either qualitative or quantitative

Standardized forms for extraction are created for use by the research team

Tools: dependent on the comfort level of the research team

Microsoft Excel

Microsoft Word

More: http://systematicreviewtools.com/index.php

22 of 29

Quality Assessment

Method 1: Utilize existing quality assessment tool, depending on RQ and SR design

​

Method 2: Create or modify customized QA tool from existing guidelines or criteria

Each study that meets the inclusion criteria is assessed for quality

Additional studies may be excluded as a result of this process

Must be documented

23 of 29

Quality Assessment Tool Examples

GRADE

    • Grading of Recommendations Assessment, Development and Evaluation
    • Used for intervention studies, various research methods included
    • https://www.gradeworkinggroup.org/

CASP

    • Critical Appraisal Skills Programme
    • Offers training workshops and free checklists for quantitative and qualitative study types
    • https://casp-uk.net/

24 of 29

Discussion and Results

Applicability of results

Strength of existing evidence

Benefits, costs, and tradeoffs

Limitations

Implications for future research

25 of 29

Publishing a systematic review

Check your targeted journal(s) for specific author requirements pertaining to systematic reviews

      • Protocol documentation
      • Summary of findings
      • Search methodology
      • PRISMA charts, appendices, data tables and presentation

26 of 29

How can librarians help you with a systematic review?

27 of 29

Librarians and your systematic review

As consultants: Provide background information and resources on the systematic review process

1

Recommend search databases, protocol registration platforms, and citation management software

2

Suggest edits for your search strategy

3

As co-authors: take a more hands-on review in the process, including designing the search strategy and writing the methodology section

4

28 of 29

Relevant Databases

29 of 29

Do you have any questions? ��Please email us at library@med.cuny.edu