1 of 16

ISEE LAB Instructional Modules: Conducting Risk of Bias (RoB) Assessment and Intro to ROBIVS

PRESENTED BY Ozi Obiamalu

DATE 17 MAR 2026

2 of 16

Risk of Bias

is defined as systematic flaws or limitations in the design, conduct, or analysis of a primary study that distort the results.

Risk of Bias Assessment

addresses the degree to which these methods minimized flaws in summary estimates and conclusions.

2

3 of 16

Tool Selection Based on Study Design

Systematic reviews : use ROBIS for reviews focused on interventions

Randomized Controlled Trials: Cochrane Risk of Bias Tool

Observational Studies: use US Preventive Services Task Force

3

4 of 16

The Tool Box

CASP - CASP CHECKLIST�ROBIS - ROBIS CHECKLIST

MMAT - MMAT CHECKLIST

4

Multiple ROB for multiple study designs

primarily assess how well the authors or researchers conducted their study according to the standards of conducting this particular study design to reduce the bias

ROB relies on questions cover the elements in the PICO framework (questions related to P, I, O, completeness/assessment of the data, whether the results were accurately derived). The way the questions are written are tailored to the unique study designs ( so it varies from qualitative to quantitative).

ROB tool also assesses how they went about answering the research question and whether their methodology was sufficient to answer the question. �- The MMAT explains study designs, they developed ways so that we can assess the quality in each study design. Learn more about the study designs, ROB assessment tool following the MMAT references if you are conducting a systematic review or scoping review you can use it to assess the bias in individual studies. you can develop your own ROB assessment tool. ��

5 of 16

The Tool Box

CASP - CASP CHECKLIST�ROBIS - ROBIS CHECKLIST

MMAT - MMAT CHECKLIST

5

Multiple ROB for multiple study designs

primarily assess how well the authors or researchers conducted their study according to the standards of conducting this particular study design to reduce the bias

ROB relies on questions cover the elements in the PICO framework (questions related to P, I, O, completeness/assessment of the data, whether the results were accurately derived). The way the questions are written are tailored to the unique study designs ( so it varies from qualitative to quantitative).

ROB tool also assesses how they went about answering the research question and whether their methodology was sufficient to answer the question. �- The MMAT explains study designs, they developed ways so that we can assess the quality in each study design. Learn more about the study designs, ROB assessment tool following the MMAT references if you are conducting a systematic review or scoping review you can use it to assess the bias in individual studies. you can develop your own ROB assessment tool. ��

6 of 16

6

7 of 16

Part 1: Assessing Relevance

7

8 of 16

Part 1a: Determine the Match

8

9 of 16

Part 2: Identify Concerns with the Review Process

9

10 of 16

Domain-Based Explanation

Study Eligibility Criteria: ensures criteria were pre-specified and unambiguous
Identification and Selection of Studies: evaluates search sensitivity
Data Collection and Study Appraisal: checks for rigor in extraction and primary study assessment
Synthesis and Findings: examines if data combination was appropriate

10

11 of 16

Domain 3: Duplicate Assessment and Conflict Resolution

11

12 of 16

Part 2: Guide for Low vs. High Concern Ratings

Assessment Area	Low Concern / Yes (Example)	High Concern / No (Example)
Objectives	Adhered to pre-defined objectives and unambiguous criteria.	Lack of pre-specified objectives or post-hoc changes to criteria.
Search Strategy	High sensitivity of search; used multiple databases.	Limited search; likely missed relevant contemporaneous research.
Data Synthesis	Appropriate data combination; accounted for heterogeneity.	Inappropriate data combination; failure to account for heterogeneity.
Appraisal	Rigorous primary study assessment by at least two reviewers.	Failure to address or identify biases in primary studies.

12

13 of 16

Module 3: Judging Risk of Bias

13

14 of 16

Module 4: Visualizing Bias

14

15 of 16

Visualizing Bias in R

Data Input 2. Tool Selection 3. Plot Generation

15

16 of 16

THANK YOU FOR WATCHING!��Q&A�

“

16