1 of 7

Producing rapid evidence profiles

Kerry Waddell, PhD

Senior Scientific Lead, McMaster Health Forum

24 February 2025

mcmasterforum.org

2 of 7

Overview of contextualized evidence syntheses

Goal

Timeline

Type of product

Scope

Profile evidence and experiences quickly

1-5 days

Rapid evidence profile

Living evidence profile (updated at regular intervals)

Example products:

REP; LEP

Key insights from highly relevant evidence syntheses (and sometimes primary studies where no evidence syntheses exist) identified from targeted searches of bibliographic databases

Experiences from relevant jurisdictions identified from targeted searches of websites and documents

Synthesizes evidence and experiences quickly

2-10 weeks

Rapid syntheses

Living evidence syntheses (updated at regular intervals)

Example products:

RS; LES

Synthesis of findings (rather than a profile of key insights) from all relevant evidence syntheses and primary studies identified (with scope depending on the timeline specified by the requester)

Analysis of experiences from relevant jurisdictions (with number of jurisdictions and depth of analysis depending on the timeline specified by the requester)

Comprehensively synthesize evidence (global public goods)

2.5+ months

Full evidence synthesis (or enhancements to previous versions)

Evidence synthesis with interpretive and/or qualitative designs or statistical analysis

Example product: Full evidence synthesis

Enhanced synthesis methods in terms of: scope (e.g., number of sources); methods (e.g., risk-of-bias assessments); outputs (e.g., living evidence synthesis with enhancements incorporated over time)

3 of 7

Overview of contextualized evidence syntheses

Goal

Timeline

Type of product

Methodological ’short-cuts’

Profile evidence and experiences quickly

1-5 days

Rapid evidence profile

Living evidence profile (updated at regular intervals)

Example products:

REP; LEP

  • Focused on identifying existing evidence syntheses that are relevant to the topic
  • Search 2-3 databases with time restrictions of 5-10 years
  • Duplicate review of small subset, then single review for inclusion
  • Single data extraction
  • Narrative profile of findings

Synthesizes evidence and experiences quickly

2-10 weeks

Rapid syntheses

Living evidence syntheses (updated at regular intervals)

Example products:

RS; LES

  • Focused on identifying existing evidence syntheses and single studies that are recent and context-relevant
  • Search 2-3 databases with time restrictions of 5-10 years
  • Duplicate review of small subset, then single review of inclusion
  • Single data extraction
  • Narrative synthesis of findings

Comprehensively synthesize evidence (global public goods)

2.5+ months

Full evidence synthesis (or enhancements to previous versions)

Evidence synthesis with interpretive and/or qualitative designs or statistical analysis

Example product: Full evidence synthesis

  • Synthesis follows transparent methods for comprehensively synthesized evidence, few, if any, short-cuts are taken

4 of 7

Flow chart for ultra-rapid evidence support

Individual or unit identifies a question for which they want evidence support, and hosts internal meetings to frame and provide additional context for the question to be asked

Individual or unit submits the request – either using the request form or using the questions in it as prompts – to the designated focal point in their department

Requestor and MHF work together to book a 30-minute scoping call ASAP

MHF prepares a one-page document outlining what can likely be provided based on the submitted question

MHF prepares and then submits its completed product by the agreed date and time for review

Focal points coordinate the review of the completed product and return the edits to MHF within two weeks (or other agreed upon timeframe)

[if needed] Requester and MHF book a 30-minute follow-up call to review any feedback on the submitted document and to scope any additional work that needs to be completed

Identify a question

Submit request internally

Prepare an outline

Book a scoping call

Participate in scoping call

Submit product

Review product

Provide feedback

MHF facilitates the scoping call where the document is reviewed, and circulates a revised document to confirm next steps after the call

5 of 7

Internal chart flow chart for contextualized evidence syntheses

Request comes in and program leads review request form

Program leads develop protocol and organizing framework based off request form

Protocol is provided for feedback to the whole writing team at weekly huddle

Protocol is finalized following scoping call and coordinator is assigned to the product

Comments from the team on the protocol are included

Program lead and coordinator develop and run search strategy and review results

Scoping call held with requestor

Any challenges with data extraction are resolved by the coordinator and program lead

Team collectively undertakes data extraction and submits to MS Teams channel

Coordinator pulls together submissions and synthesizes data extraction into narrative report

Report is reviewed by program lead prior to submission to requestor

6 of 7

Overview of team involved in REP

Title

Description of role

Program leads

Scientific director at the Forum and senior scientific lead for evidence support help to run the program and are involved in the scoping discussions for all contextualized evidence syntheses

Coordinators

Responsible for leading and coordinating work on individual projects. At any one time the forum has 2-3 coordinators each leading different work. These individuals are ultimately responsible for synthesizing and writing up the final product as well as overseeing research assistants.

Research assistants

Responsible for conducting data extraction and support coordinators in developing any tables or write ups needed for the final report. At any given time, there are about seven to twelve research assistants working on a range of products.

Operational/logistics lead

Responsible for leading the administrative process of contextualized evidence syntheses including ensuring contracts are in place and the team is meeting set deadlines.

Formatting and copy editors

Responsible for formatting final documents and undertaking copy edit before posting.

7 of 7

Key elements that allow for rapid timelines

Set processes and templates for each type of product

Flexible staff arrangements that support a team to ‘jump’ on a project

Contracting and administrative processes set in advance

Network of subject matter experts that can advise on frameworks and included evidence

Network of other evidence synthesis teams that support a distributive approach