1 of 40

Formulating a

research question

2 of 40

Instructors

2

Natasha Bishop

Evidence Synthesis Information Scientist

Virginia Tech

Elizabeth DiGiovine

Research and Evidence Synthesis Librarian

Cornell University

3 of 40

Significance �of Research Questions to Evidence Synthesis

3

4 of 40

The research

question is the foundation of

evidence synthesis

4

Research Question

Establishes focus

Communicates purpose

Provides boundaries for decision-making

Guides the rest of the review

5 of 40

The research

question is the foundation of

evidence synthesis

5

Research Question

Establishes focus

Communicates purpose

Provides boundaries for decision-making

Guides the rest of the review

Eligibility Criteria

6 of 40

The research

question is the foundation of

evidence synthesis

6

Research Question

Eligibility Criteria

Search

Establishes focus

Communicates purpose

Provides boundaries for decision-making

Guides the rest of the review

7 of 40

The research

question is the foundation of

evidence synthesis

7

Research Question

Eligibility Criteria

Search

Data collection

Establishes focus

Communicates purpose

Provides boundaries for decision-making

Guides the rest of the review

8 of 40

The research

question is the foundation of

evidence synthesis

8

Research Question

Eligibility Criteria

Search

Data collection

Synthesis

Establishes focus

Communicates purpose

Provides boundaries for decision-making

Guides the rest of the review

9 of 40

The research

question is the foundation of

evidence synthesis

9

Research Question

Eligibility Criteria

Search

Data collection

Synthesis

Conclusion

Establishes focus

Communicates purpose

Provides boundaries for decision-making

Guides the rest of the review

10 of 40

Consequences of a poorly written

research question...

Too large to be feasible

Too narrow to find relevant literature

Lack of focus/direction

Does not address a knowledge gap

10

11 of 40

The �Role of �the Librarian

11

12 of 40

The Librarian’s Role

Let’s get your house in order together

Get your house in order.

Research consult

Evidence synthesis meeting

So how can we, as librarians, prevent some of these issues? And why should we be actively involved?

Why should we be actively involved?

We want to make positive contribution to the information landscape
We want to help our colleagues/researchers/patrons be successful
We also hold stake in the success of the review. An evidence synthesis consultation should not be treated as a typical research consultation because you might start as a consultant but later become a co-author which is a significant multi-year time commitment. Not to mention your name will be associated with the project, so you will want to ensure that the review is successful, useful, and of high quality.

So what exactly is out role?

Start a conversation to help guide the structure of the question:

Ask researchers what their goals are for the review. Does this relate to the question?

Do they hope to influence policy? Do they want to inform a future study?

Ask researchers who the stakeholders and users are for this particular review. Have the stakeholders and users been engaged and consulted? Who has been involved in the conversation?

Does the research question roughly follow a framework? In other words, can you identify two to three key concepts? Is it searchable and answerable?
Can the researchers supply at least three primary seed articles that address this research question?
Does the question “pass” preliminary/exploratory searching?

Have there been other evidence syntheses that have answered this question? Roughly how much research exists? WE can answer these questions!!

13 of 40

The Librarian’s Role

Collaborator

Workshop question drafts
Conduct exploratory searches
Adjust the question’s scope

So how can we, as librarians, prevent some of these issues? And why should we be actively involved?

Why should we be actively involved?

We want to make positive contribution to the information landscape
We want to help our colleagues/researchers/patrons be successful
We also hold stake in the success of the review. An evidence synthesis consultation should not be treated as a typical research consultation because you might start as a consultant but later become a co-author which is a significant multi-year time commitment. Not to mention your name will be associated with the project, so you will want to ensure that the review is successful, useful, and of high quality.

So what exactly is out role?

Start a conversation to help guide the structure of the question:

Ask researchers what their goals are for the review. Does this relate to the question?

Do they hope to influence policy? Do they want to inform a future study?

Ask researchers who the stakeholders and users are for this particular review. Have the stakeholders and users been engaged and consulted? Who has been involved in the conversation?

Does the research question roughly follow a framework? In other words, can you identify two to three key concepts? Is it searchable and answerable?
Can the researchers supply at least three primary seed articles that address this research question?
Does the question “pass” preliminary/exploratory searching?

Have there been other evidence syntheses that have answered this question? Roughly how much research exists? WE can answer these questions!!

14 of 40

The Librarian’s Role

Collaborator

Workshop question drafts
Conduct exploratory searches
Adjust the question’s scope

Sounding board

Sometimes people just need to talk through ideas :)

So how can we, as librarians, prevent some of these issues? And why should we be actively involved?

Why should we be actively involved?

We want to make positive contribution to the information landscape
We want to help our colleagues/researchers/patrons be successful
We also hold stake in the success of the review. An evidence synthesis consultation should not be treated as a typical research consultation because you might start as a consultant but later become a co-author which is a significant multi-year time commitment. Not to mention your name will be associated with the project, so you will want to ensure that the review is successful, useful, and of high quality.

So what exactly is out role?

Start a conversation to help guide the structure of the question:

Ask researchers what their goals are for the review. Does this relate to the question?

Do they hope to influence policy? Do they want to inform a future study?

Ask researchers who the stakeholders and users are for this particular review. Have the stakeholders and users been engaged and consulted? Who has been involved in the conversation?

Does the research question roughly follow a framework? In other words, can you identify two to three key concepts? Is it searchable and answerable?
Can the researchers supply at least three primary seed articles that address this research question?
Does the question “pass” preliminary/exploratory searching?

Have there been other evidence syntheses that have answered this question? Roughly how much research exists? WE can answer these questions!!

15 of 40

Qualities of a �well-formulated�research question

15

16 of 40

A FINER research question

F

I

N

E

R

16

- Feasible

- Interesting

- Novel

- Ethical

- Relevant

Cochrane handbook, chapter two

17 of 40

A FINER research question

F

I

N

E

R

17

- Feasible

Cochrane handbook, chapter two

18 of 40

A FINER research question

F

I

N

E

R

18

- Feasible

- Interesting

Cochrane handbook, chapter two

19 of 40

A FINER research question

F

I

N

E

R

19

- Feasible

- Interesting

- Novel

Cochrane handbook, chapter two

20 of 40

A FINER research question

F

I

N

E

R

20

- Feasible

- Interesting

- Novel

- Ethical

Cochrane handbook, chapter two

21 of 40

A FINER research question

F

I

N

E

R

21

- Feasible

- Interesting

- Novel

- Ethical

- Relevant

Cochrane handbook, chapter two

22 of 40

A FINER research question

F

I

N

E

R

22

- Feasible

- Interesting

- Novel

- Ethical

- Relevant

Cochrane handbook, chapter two

23 of 40

Aligning questions with goals

Sometimes it’s helpful to have a team clarify what goal(s) they are seeking to accomplish with their review.

23

Will the question as formulated help them reach those goals?

24 of 40

Assessing the information landscape

24

Exploratory Searching

Make sure prospective question hasn’t already been answered
Check protocol registries

25 of 40

Too broad

What are the benefits of drinking tea?�

Too narrow

What are the impacts of drinking matcha tea every Sunday on cardiovascular health in women aged 30-32?�

Just right

What are the impacts of green tea on blood pressure?

Appropriate Scope

Does the question adhere to the Goldilocks Principle?

26 of 40

Lumping

Expanding the definition of a concept to include other domains.

Splitting

Breaking a concept down into smaller subsets and selecting one to use.

26

Adjusting the question's scope

27 of 40

Managing Russian Doll questions

Question within a question within a question…

What are the economic, environmental, and health impacts of producing, marketing, and consuming processed foods?

27

28 of 40

Frameworks

28

29 of 40

Frameworks

29

Stay Tuned!

30 of 40

The Review Type: Scoping vs. Systematic

30

31 of 40

Systematic Review

What does the evidence say?

Focused research question with narrow parameters

What evidence exists?

Broad research question that assesses the state of the literature

31

The research question informs the review type

Scoping Review

32 of 40

Systematic Review

Compares, evaluates, and synthesizes evidence to assess...

Constructs
Rates/trends
Associations
Predictions
Cause and effect

Explores breadth of the literature

Maps the evidence

Identifies knowledge gaps

Clarifies key concepts and/or definitions

May act as precursor to a systematic review

32

The review types have distinct objectives

Scoping Review

33 of 40

Systematic Review

“What is the impact of care

farming on quality of life,

depression and anxiety, on

a range of service user

groups?”

Murray, J., Wickramasekera, N., Elings, M., Bragg, R., Brennan, C., Richardson, Z., Wright, J., Llorente, M. G., Cade, J., Shickle, D., Tubeuf, S., & Elsey, H. (2019). The impact of care farms on quality of life, depression and anxiety among different population groups: A systematic review. Campbell Systematic Reviews, 15(4). https://doi.org/10.1002/cl2.1061

"How is care farming defined in the literature? What are the primary features of a care farm?"

33

Example Question

Scoping Review

34 of 40

"Practice is the

best of all instructors."�� -Publilius Syrus

35 of 40

Systematic Review

"What is the strength of the effect of [Mindfulness-Based Programs] on school adjustment outcomes overall and individually?"

Mettler J, Khoury B, Zito S, Sadowski I, Heath NL. Mindfulness-based programs and school adjustment: A systematic review and meta-analysis. J Sch Psychol. 2023 Apr;97:43-62. doi: 10.1016/j.jsp.2022.10.007. Epub 2023 Jan 25. PMID: 36914366.

35

Scoping Review

36 of 40

Systematic or Scoping?

“What are the current applications of drones in disaster management, specifically mass disaster?”

36

37 of 40

Systematic or Scoping?

“What are the current applications of drones in disaster management, specifically mass disaster?”

Mohd Daud SMS, Mohd Yusof MYP, Heo CC, Khoo LS, Chainchel Singh MK, Mahmood MS, Nawawi H. Applications of drone in disaster management: A scoping review. Sci Justice. 2022 Jan;62(1):30-42. doi: 10.1016/j.scijus.2021.11.002. Epub 2021 Nov 14. PMID: 35033326.

37

38 of 40

Systematic or Scoping?

“How effective are created or restored freshwater wetlands for nitrogen and phosphorus removal?”

38

39 of 40

Systematic or Scoping?

“How effective are created or restored freshwater wetlands for nitrogen and phosphorus removal?”

Land M, Granéli W, Grimvall A, Hoffmann CC, Mitsch WJ, Tonderski KS, Verhoeven JT. How effective are created or restored freshwater wetlands for nitrogen and phosphorus removal? A systematic review. Environmental Evidence. 2016 Dec;5:1-26.https://doi.org/10.1186/s13750-016-0060-0

39

40 of 40

Questions?

40