Systematic Reviews Workshop #1 �Setting yourself up for success
Instructors: Zahra Premji & Alix Hayden
(UVic) (UofC)
CC BY-NC-ND 4.0
Territory Acknowledgement
2
Calgary
Victoria
We acknowledge and respect the lək̓ʷəŋən peoples on whose traditional territory the university stands and the Songhees, Esquimalt and W̱SÁNEĆ peoples whose historical relationships with the land continue to this day.
I would like to take this opportunity to acknowledge the traditional territories of the people of the Treaty 7 region in Southern Alberta, which includes the Blackfoot Confederacy (comprising the Siksika, Piikani, and Kainai First Nations), as well as the Tsuut’ina First Nation, and the Stoney Nakoda (including the Chiniki, Bearspaw, and Wesley First Nations). The City of Calgary is also home to Métis Nation of Alberta, Region 3.
Introductions
Who are we?
Who are you?
3
Workshop objectives
By the end of this workshop, attendees will be able to
4
CC BY-NC-ND 4.0
Workshop 1 summary
5
What is knowledge/evidence synthesis | | |
Systematic versus scoping review | | |
1. Defining your question | | A. Creating a focused researchable question |
| | B. Pre-scan of the literature |
| | |
2. Creating your protocol/plan | | A. PRISMA –P |
| | B. Choosing the information sources |
| | C. Developing inclusion/exclusion criteria |
| | |
3. Data collection | | A. Pre-scan of the literature and locating seed papers |
| | B. Keywords/textwords and Subject Headings |
| | C. Mining seed papers for keywords and subject headings |
What is knowledge/�evidence synthesis?
6
Synthesis is
“the contextualization and integration of research findings of individual research studies within the larger body of knowledge on the topic. A synthesis must be reproducible and transparent in its methods, using quantitative and/or qualitative methods.”
(Canadian Institutes of Health Research: http://www.cihr-irsc.gc.ca/e/29418.html#4.1)
A systematic review is
“A review of a clearly formulated question that uses systematic and explicit methods to identify, select, and critically appraise relevant research, and to collect and analyse data from the studies that are included in the review.”
(Cochrane: https://community.cochrane.org/glossary#letter-S)
Typical steps of a knowledge synthesis project
to
7
1. Determine the review question |
2. Create the protocol/plan |
3. Data collection |
4. Study selection |
5. Data extraction |
6. Critical appraisal |
7. Synthesize (narrative or meta-analysis) |
8. Write manuscript (using reporting standards) |
Competencies required
It Takes a Team!
Average # is 5 authors
8
Borah, R., Brown, A. W., Capers, P. L., & Kaiser, K. A. (2017). Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry. BMJ open, 7(2), e012545.
Time to Complete
9
Borah, R., Brown, A. W., Capers, P. L., & Kaiser, K. A. (2017). Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry. BMJ open, 7(2), e012545.
Start Your Review
Publish Your Review
Why Does it Take So Long?
Each review is different
10
Guiding documents
11
A review can follow PRISMA and still lack comprehensiveness
Conducting/methodological guides
Reporting standards
Tells you how to conduct your review ☺
Tells you how to write your review ☺
CC BY-NC-ND 4.0
So why conduct a review?
13
Families of Reviews
14
Mixed Methods Reviews
Sutton, A., Clowes, M., Preston, L., & Booth, A. (2019). Meeting the review family: exploring review types and associated information retrieval requirements. Health Information & Libraries Journal, 36(3), 202-222.
Purpose Specific Reviews
Qualitative Reviews
Traditional Reviews
Systematic Reviews
Rapid Reviews
Systematic vs Scoping
Systematic Review
may be undertaken to confirm or refute whether or not current practice is based on relevant evidence, to establish the quality of that evidence, and to address any uncertainty or variation in practice that may be occurring.
Scoping Review
to determine the scope or coverage of a body of literature on a given topic and give clear indication of the volume of literature and studies available as well as an overview (broad or detailed) of its focus.
15
Munn, Z., Peters, M. D., Stern, C., Tufanaru, C., McArthur, A., & Aromataris, E. (2018). Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC medical research methodology, 18(1), 143.
Traditional vs Systematic vs Scoping
16
Munn, Z., Peters, M. D., Stern, C., Tufanaru, C., McArthur, A., & Aromataris, E. (2018). Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC medical research methodology, 18(1), 143.
Which One: Systematic Review or Traditional Review?
Systematic Review
Traditional (Narrative) Review
17
Greenhalgh, T., Thorne, S., & Malterud, K. (2018). Time to challenge the spurious hierarchy of systematic over narrative reviews?.
European journal of clinical investigation, 48(6).
“The narrative review is not a poor cousin of the systematic review but a different and potentially complementary form of scholarship.”
18
What is knowledge/evidence synthesis | | | |
Systematic versus scoping review | | | |
1. Defining your question | | A. Creating a focused researchable question | |
| | B. Pre-scan of the literature | |
2. Creating your protocol/plan | | | |
3. Data collection | | | |
Question Frameworks
19
Question formulation using PICO
20
P
Population/Problem
I
Intervention (or exposure or factor)
C
Comparison
O
Outcomes
Patient, population, problem, disease |
Intervention, exposure, prognostic factor |
Comparison (such as gold standard, alternate intervention, no exposure) |
Outcome of interest (such as efficacy, incidence, prevalence, sensitivity, or other measurable outcome) |
Original reference for PICO: Richardson, W.S., Wilson, M.C., Nishikawa, J. and Hayward, R.S.A. (1995), “The well‐built clinical question: a key to evidence based decisions”, ACP Journal Club, Vol. 123 No. 3, pp. A12‐A13.
PICO: Examples
Humor in romantic relationships: A meta‐analysis
The goal of the present investigation is to identify the direction and strength of the association between humor and relationship satisfaction in romantic relationships.
Hall, J. A. (2017). Humor in romantic relationships: A meta‐analysis. Personal Relationships, 24(2), 306-322.
P – individual in a romantic relationship
I/E – Humor
C – none
O – relationship satisfaction
21
PICO: Examples
Paw preferences in cats and dogs: Meta-analysis
…determine the existence and extent of population level paw preferences in domesticated cats and dogs.
Ocklenburg, S., Isparta, S., Peterburs, J., & Papadatou-Pastou, M. (2019). Paw preferences in cats and dogs: Meta-analysis. Laterality: Asymmetries of Body, Brain and Cognition, 1-31.
P – domesticated cats and dogs
I/E – paw preference
C – none
O – existence and extent
22
PICO: Examples
Laughter-inducing therapies: Systematic review and meta-analysis
This research aims to broadly describe the field of laughter-inducing therapies, and to estimate their effect on mental and physical health for a broad range of populations and conditions.
van der Wal, C. N., & Kok, R. N. (2019). Laughter-inducing therapies: Systematic review and meta-analysis. Social Science & Medicine.
P – Any individuals
I/E – laughter-inducing therapy
C – other intervention, or no intervention
O- mental and physical health outcomes
23
24
PICO Activity - example
Initial Question: Is pet therapy effective?
Answerable research question: In post-secondary students, is pet therapy an effective intervention for alleviating stress?
P (population) – Post-secondary students
I/E (intervention) – Pet Therapy
C (Comparison) – none
O (Outcome) – Stress reduction
CC BY-NC-ND 4.0
25
PICO Activity
Initial Question: Is social media harmful
Answerable research question:
Complete the activity on the Google form, individually.
We will share your PICO or research questions with the class
P (population) –
I/E (intervention) –
C (Comparison) –
O (Outcome) –
CC BY-NC-ND 4.0
26
What is knowledge/evidence synthesis | | | |
Systematic versus scoping review | | | |
1. Defining your question | | A. Creating a focused researchable question | |
| | B. Pre-scan of the literature | |
2. Creating your protocol/plan | | | |
3. Data collection | | | |
Pre-scan of the literature
27
CC BY-NC-ND 4.0
https://graduateresearcherspace.files.wordpress.com/2015/10/litrevgoogle.gif
28
Pre-scan of the literature
29
Hybholt, L., Buus, N., Erlangsen, A., & Lauge Berring, L. (2018). Older adults bereaved by suicide: a systematic literature search identifying zero studies. Archives of suicide research, 1-6.
30
What is knowledge/evidence synthesis | | |
Systematic versus scoping review | | |
1. Defining your question | | |
2. Creating your protocol/plan | | A. PRISMA –P |
| | B. Choosing the information sources |
| | C. Developing inclusion/exclusion criteria |
3. Data collection | | |
What is a Protocol?
“In the context of systematic reviews and meta-analyses, a protocol is a document that presents an explicit plan for a systematic review. The protocol details the rationale and a priori methodological and analytical approach of the review.”
(Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, Shekelle P, Stewart LA. Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4(1):1. doi: 10.1186/2046-4053-4-1)
31
Protocols
Why: promotes consistent conduct, accountability, research integrity, transparency, aid in decision-making
When: A priori
Where: Should be published (Prospero, some journals, Open science framework, or institutional repository).
Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, Shekelle P, Stewart LA. Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4(1):1. doi: 10.1186/2046-4053-4-1
32
PRISMA-P (Preferred Reporting Items for Systematic Reviews and Meta-Analysis Protocols)
33
http://www.prisma-statement.org/documents/PRISMA-P-checklist.doc
34
What is knowledge/evidence synthesis | | |
Systematic versus scoping review | | |
1. Defining your question | | |
2. Creating your protocol/plan | | A. PRISMA –P |
| | B. Choosing the information sources |
| | C. Developing inclusion/exclusion criteria |
3. Data collection | | |
Choosing the Information Sources
35
What databases to search
AND
What are some of the limits we might use in database searches
36
What is knowledge/evidence synthesis | | |
Systematic versus scoping review | | |
1. Defining your question | | |
2. Creating your protocol/plan | | A. PRISMA –P |
| | B. Choosing the information sources |
| | C. Developing inclusion/exclusion criteria |
3. Data collection | | |
Inclusion / Exclusion Criteria
Inclusion Criteria
Exclusion Criteria
37
Inclusion / Exclusion Criteria
38
Common Inclusion / Exclusion Criteria
39
Common Inclusion / Exclusion Criteria
40
Ocklenburg, S., Isparta, S., Peterburs, J., & Papadatou-Pastou, M. (2019). Paw preferences in cats and dogs: Meta-analysis. Laterality: Asymmetries of Body, Brain and Cognition, 1-31.
41
Time for a break
42
What is knowledge/evidence synthesis | | |
Systematic versus scoping review | | |
1. Defining your question | | |
2. Creating your protocol/plan | | |
3. Data collection | | A. Pre-scan of the literature and locating seed papers |
| | B. Keywords/textwords and Subject Headings |
| | C. Mining seed papers for keywords and subject headings |
Locating Seed/Known Articles
What: Quick search of P and I/E (in Google Scholar, or multi-disciplinary database)
Goal: Locate 3-5 articles that meet our inclusion criteria.
Aim for a “diverse” set
43
CC BY-NC-ND 4.0
Searchable Concepts
Not all concepts are searchable. Not all concepts need to be searched.
44
Searchable Concepts
Humor in romantic relationships
P – individual in a romantic relationship
I/E – Humor
C – none
O – relationship satisfaction
Hall, J. A. (2017). Humor in romantic relationships: A meta‐analysis. Personal Relationships, 24(2), 306-322
Paw preferences in cats and dogs
P – domesticated cats and dogs
I/E – paw preference
C – none
O – existence and prevalence
Ocklenburg, S., Isparta, S., Peterburs, J., & Papadatou-Pastou, M. (2019). Paw preferences in cats and dogs: Meta-analysis. Laterality: Asymmetries of Body, Brain and Cognition, 1-31.
Laughter-inducing therapies:
P – Any individuals
I/E – laughter-inducing therapy
C – other or no intervention
O- mental and physical health outcomes
van der Wal, C. N., & Kok, R. N. (2019). Laughter-inducing therapies: Systematic review and meta-analysis. Social Science & Medicine.
45
46
What is knowledge/evidence synthesis | | |
Systematic versus scoping review | | |
1. Defining your question | | |
2. Creating your protocol/plan | | |
3. Data collection: Planning | | A. Pre-scan of the literature and locating seed papers |
| | B. Keywords/textwords and Subject Headings |
| | C. Mining seed papers for keywords and subject headings |
Textwords/Keywords
47
CC BY-NC-ND 4.0
Subject Headings
48
CC BY-NC-ND 4.0
Subject Headings versus textwords/Keywords
Subject Headings
Textwords/Keywords
49
CC BY-NC-ND 4.0
50
What is knowledge/evidence synthesis | | |
Systematic versus scoping review | | |
1. Defining your question | | |
2. Creating your protocol/plan | | |
3. Data collection: Planning | | A. Pre-scan of the literature and locating seed papers |
| | B. Keywords/textwords and Subject Headings |
| | C. Mining seed papers for keywords and subject headings |
Mining seed papers
51
Subject heading for pet therapy concept
Textwords for pet therapy concept
52
Mining Seed Papers - Activity
Keywords/textwords, and subject headings can be found by analyzing database records
Activity
Look up 2 seed papers in each database and analyze the article record.
Add 1) relevant subject headings and 2) textwords to the appropriate columns of the table in the following Googledoc:
CC BY-NC-ND 4.0
Planning Your Review
53
The Government of Alberta is the copyright holder and
the Health Knowledge Network is the creator of this material
Research question
Eligibility criteria
Information sources
Searchable concepts
Protocol completed. Study begins
Textwords and subject headings. i.e. database search strategies
Summary
54
First: We identified a PICO and research question & decided on information sources
Second: We generated keywords and identified searchable concepts
Third: We identified some subject headings in 2 different databases
Workshop #2: We will use the keywords and subject headings to build a comprehensive search strategy
CC BY-NC-ND 4.0
THANKS!
Any questions?
Zahra Premji
University of Victoria
Alix Hayden
University of Calgary
55
Presentation template by SlidesCarnival