1 of 25

Adapting systematic review methods for graduate students or researchers working solo

Matt Kibbee

2 of 25

What do you think?

Can a graduate student who is not part of a research team conduct a full systematic review?

​

YES

NO

(use the Zoom reaction or to vote)

​

3 of 25

What do you think?

Can a graduate student who is not part of a research team conduct a full systematic review?

​

YES, if

NO, but

For a recent discussion of this question, see: Farhad. (2022, July 18). Can I conduct a Systematic Review as my Master’s dissertation or PhD thesis? Yes, It Depends! Medium. https://farhadinfo.medium.com/can-i-conduct-a-systematic-review-as-my-masters-dissertation-or-phd-thesis-yes-it-depends-d2429fd2e20f

4 of 25

Graduate Students Are Evidence Synthesis Frequent Flyers

  • Graduate students frequently seek out help with evidence synthesis, yet they can often lack:
    • Sufficient time
    • Research partners
    • Financial resources
    • Knowledge of evidence-synthesis procedures

5 of 25

Why do students pursue evidence synthesis projects?

​

    • Their advisor told them to.
    • They believe it will be “easier” than primary research.
    • They want to be more methodical about their literature reviews.
    • They want to publish their literature review.

6 of 25

Challenges

Keep in mind:

  • Researchers working with limited resources can often require more help throughout the process.
  • Academic ethics may limit the degree of your involvement, especially if the project is part of a thesis.

7 of 25

Discussion!

  • What is your experience working with graduate students, undergraduates, or solo researchers who are interested in evidence synthesis?

​

  • What challenges have you experienced? What successes?

8 of 25

An adapted toolkit:

You or your team’s toolkit can be adapted to guide students who don’t have the resources for a full systematic review.

  • Intake form
  • Initial consultation
  • Protocol template
  • Documentation / Resource lists / Videos
  • Libguides
  • Email Templates (for FAQs)
  • Examples of published, high-quality (non-systematic) literature reviews

Adapted from: Wafford, Q. E., & O’Dwyer, L. C. (2021). Adopting a toolkit to manage time, resources, and expectations in the systematic review process: A case report. Journal of the Medical Library Association, 109(4). https://doi.org/10.5195/jmla.2021.1221

9 of 25

Intake Form

  • Whether or not you are working as part of a team, you should decide your policies regarding graduate students, undergraduates, and solo researchers.

​

  • A good intake form can help set expectations for the patron and protect your and your team’s time.

10 of 25

Intake Form:

University of Minnesota Request Form https://www.lib.umn.edu/services/systematic-reviews-request

11 of 25

The Initial Consultation

When helping students decide what sort of review to pursue, be honest about the requirements but help them brainstorm possibilities:

  • Several stages of a typical evidence synthesis project require 2–3 people;
  • Many reviews can take a year or longer;
  • It’s difficult to predict the number of included studies you will end up with. It could be zero; it could be more than 100. Most reviews require a flexible timeline.

​

“Do you have colleagues or classmates you can recruit to help with screening/extraction?”

“Let’s sketch out a rough timeline and see if that works for your schedule.”

“Even really well organized reviews can take longer than expected. Would you be willing to keep working on this after you graduate?”

12 of 25

The Initial Consultation

Have prepared language to help guide you and your team if the researcher isn’t grasping the process or is making unrealistic demands on your time.

Suggested language from Cornell’s Initial Consultation Guide:

"because you're working with such a tight timeline..."

or "for this type of project..."

  • "...I think having us work with you as a consultant will be the best fit. We're more than happy to look at a draft of your protocol, give you feedback on your search strategy, or provide any other help you need. Also working with us this way will get you the help you need faster, because your emails to our help inbox go to our team…”

​

13 of 25

Even if a systematic review isn’t right for the researcher, you can emphasize the many alternatives.

​

Cornell’s Methodology Decision Tree:

https://cornell.app.box.com/s/tfgvuicvsn9s58g7c0akxh0cmcuifbbo/file/573873815952

14 of 25

Protocol:

“Structured Literature Review”

A structured literature review adapts systematic review steps so that researchers can conduct their literature reviews in a more methodical, transparent manner.

Ideal for students who:

    • Want to be more methodical about their literature review
    • Want to learn more about the systematic review process
    • Don’t need to find everything
    • Have limited time/resources
    • Want to publish their literature review

​

Cornell’s Planning Worksheet for a Structured Literature Review: https://osf.io/ve8gc

​

​

​

15 of 25

Protocol: A Structured Literature Review

The worksheet guides students through a truncated version of the systematic review process:

  1. Write out review question and identify concepts
  2. List synonyms
  3. Determine sources for evidence gathering
  4. Write search strategy for each database
  5. Determine inclusion and exclusion criteria
  6. Perform search and use citation manager
  7. Appraise qualities of studies
  8. Extract data
  9. Write up methods

​

​

​

​

16 of 25

Protocol: A Structured Literature Review

Each step briefly describes what’s required and links to additional resources.

​

​

​

​

​

17 of 25

Documentation:

Research Logs

Your worksheet can link to documents that provide additional guidance or remind researchers of best practices.

​

​

​

​

​

​

From Cornell’s “Generic Research Log”: https://cornell.app.box.com/file/723624407710?s=vf6sitssfe5bpmf9b490olllteroxvhh

18 of 25

Resources: Automation

  • Automated tools can be a huge help for a researcher working alone.
  • For example, SR-Accelerator (https://sr-accelerator.com) is a free resource can help with:
    • Search translation
    • Deduplication
    • Writing the methods section
  • ChatGPT and other LLMs
    • Keyword generation
    • Search writing
    • Translation

​

​

​

​

Remember:

it’s easy to overwhelm students who are new to the process, so keep things curated and clear!

19 of 25

“Can I just call it a systematic review?”

No!

20 of 25

“Can I just call it a systematic review?”

Journals increasingly have explicit standards to which an author must adhere in order to call their project a “systematic review.”

​

From Journal of the Society for Social Work and Research : Author Guidelines:

​

https://www.journals.uchicago.edu/journals/jsswr/instruct#systematic-reviews

21 of 25

Consider: “Systematized Review”

  • Outside of the major review types, naming conventions are not standardized.

​

  • “Systematized” may appeal to students who really want that “system-” in their title.

For a definition of systemized review and examples, see: Grant, M. J., & Booth, A. (2009). A typology of reviews: An analysis of 14 review types and associated methodologies. Health Information and Libraries Journal, 26(2), 91–108. https://doi.org/10.1111/j.1471-1842.2009.00848.x

​

22 of 25

Remember:

Working with graduate students is often a hugely rewarding process.

​

Make their first impression of evidence synthesis a good one!

23 of 25

Questions & Conversation

Feel free to contact me at mk282@cornell.edu if you have any questions!

24 of 25

Readings

Boland, A., Cherry, M. G., & Dickson, R. (Eds.). (2017). Doing a systematic review: A student’s guide (2nd edition.). SAGE.

​

Farhad. (2022, July 18). Can I conduct a Systematic Review as my Master’s dissertation or PhD thesis? Yes, It Depends! Medium. https://farhadinfo.medium.com/can-i-conduct-a-systematic-review-as-my-masters-dissertation-or-phd-thesis-yes-it-depends-d2429fd2e20f

​

Grant, M. J., & Booth, A. (2009). A typology of reviews: An analysis of 14 review types and associated methodologies. Health Information and Libraries Journal, 26(2), 91–108. https://doi.org/10.1111/j.1471-1842.2009.00848.x

​

Wafford, Q. E., & O’Dwyer, L. C. (2021). Adopting a toolkit to manage time, resources, and expectations in the systematic review process: A case report. Journal of the Medical Library Association, 109(4). https://doi.org/10.5195/jmla.2021.1221

​

Wissinger, C. L. (2018). Is there a place for undergraduate and graduate students in the systematic review process? Journal of the Medical Library Association : JMLA, 106(2), 248–250. https://doi.org/10.5195/jmla.2018.387

​

​

25 of 25

Resources

​

​

​

​