Term/Definition Attribution(s)

ROB1 (https://www.bmj.com/content/343/bmj.d5928) Source Content

ROB2 (https://sites.google.com/site/riskofbiastool/welcome/rob-2-0-tool/current-version-of-rob-2?authuser=0) Source Content

ROBINS-I (https://drive.google.com/file/d/0B7IQVI0kum0kWldlU1BzRGxnclE/view) Source Content

Newcastle (http://www.ohri.ca/programs/clinical_epidemiology/oxford.asp) Source Content

ROBIS (https://www.bristol.ac.uk/population-health-sciences/projects/robis/) Source Content

COKA (http://build.fhir.org/valueset-evidence-classifier-code.html) Source Content

QUIPS (https://www.researchgate.net/publication/235658860_Assessing_Bias_in_Studies_of_Prognostic_Factors) Source Content

PROBAST (https://www.acpjournals.org/doi/10.7326/M18-1376) Source Content

QUADAS-2 (http://www.bristol.ac.uk/population-health-sciences/projects/quadas/quadas-2/) Source Content

MMAT (http://mixedmethodsappraisaltoolpublic.pbworks.com/w/file/fetch/127916259/MMAT_2018_criteria-manual_2018-08-01_ENG.pdf) Source Content

Cochrane ROB expression of DTA Reviews (https://methods.cochrane.org/sites/methods.cochrane.org.sdt/files/public/uploads/ch09_Oct09.pdf) Source Content

MASTER scale (https://doi.org/10.1016/j.jclinepi.2021.01.012)

Jadad RCT book () Source Content

see PROBAST: 1.1 Were appropriate data sources used, e.g., cohort, randomized controlled trial, or nested case–control study data? Prognostic model studies. Prognostic model studies are at low ROB when based on a prospective longitudinal cohort design, where methods tend to be defined and consistently applied for participant inclusion and exclusion criteria, predictor assessment, and outcome determination across a predefined follow-up (1). Using prespecified and consistent methods ensures that participant data are systematically and validly recorded. Model development and validation studies have higher potential for ROB when participant data are from existing sources, such as existing cohort studies or routine care registries, because data are often collected for a purpose other than development, validation, or updating of prediction models, and are also often without a protocol. ... A cohort design (including RCT or proper registry data) or a nested case-control or case-cohort design (with proper adjustment of the baseline risk/hazard in the analysis) is[should be] used to generate absolute risk predictions..... Diagnostic model studies. Diagnostic models predict the presence or absence of an outcome (target disease) at the same time point as the index tests or predictors are measured (Box 2). Accordingly, the design with the lowest ROB for diagnostic model studies is a cross-sectional study where a group (cohort) of participants is selected on the basis of certain symptoms or signs that make them suspected of having the target condition of interest. Subsequently, the predictors (index tests) and outcome (disease presence or absence) according to the reference standard are measured in all participants (https://www.acpjournals.org/doi/full/10.7326/M18-1377?journalCode=aim)

McDonagh M, Peterson K, Raina P, et al. Avoiding Bias in Selecting Studies. 2013 Feb 20. In: Methods Guide for Effectiveness and Comparative Effectiveness Reviews [Internet]. Rockville (MD): Agency for Healthcare Research and Quality (US); 2008-. Available from: https://www.ncbi.nlm.nih.gov/books/NBK126701/

A situation in which the effect or association between an exposure or outcome is distorted by another variable. (from Glasser, 2014, Essentials of Clinical Research)

Risk of bias arising from the randomization process (section header)

Bias due to confounding (section header)

Allocation bias = Systematic difference in how participants are assigned to comparison groups in a clinical trial. (https://catalogofbias.org/biases/allocation-bias/)

If successfully accomplished, randomization avoids an influence of either known or unknown prognostic factors (factors that predict the outcome, such as severity of illness or presence of comorbidities) on intervention group assignment. This means that, on average, the intervention groups have the same prognosis before the start of intervention. If prognostic factors influence the intervention group to which participants are assigned then the estimated effect of intervention will be biased by ‘confounding’, which occurs when there are common causes of intervention group assignment and outcome. Confounding is an important potential cause of bias in intervention effect estimates from observational studies, because treatment decisions in routine care are often influenced by prognostic factors.

Recognized Difference with Potential for Confounding, Recognized confounding difference

1.3 Did baseline differences between intervention groups suggest a problem with the randomization process?

1.1 Is there potential for confounding of the effect of intervention in this study?

25. Key baseline characteristics/prognostic indicators for the study were comparable across groups

Such differences could be the administration of additional interventions that are inconsistent with the study protocol, or non-adherence by the interventionalists or study participants to their assigned intervention. Such differences may occur based on assignment to intervention or may occur due to adherence to intervention.

CoB: Performance bias = Systematic differences in the care provided to members of different study groups other than the intervention under investigation (https://catalogofbias.org/biases/performance-bias/)

ROBINS-I = Bias due to deviations from intended intervention

MASTER scale (https://doi.org/10.1016/j.jclinepi.2021.01.012): 17. Care was delivered equally to all participants

Bias due to deviations from intended intervention (section header)

8. Exposure variations/treatment deviations were less than 20%

4.1. Were there deviations from the intended intervention beyond what would be expected in usual practice?

MASTER scale (https://doi.org/10.1016/j.jclinepi.2021.01.012): 8. Exposure variations/treatment deviations were less than 20%

CoB: Compliance bias = Participants compliant with an intervention differ in some way from those not compliant which can systematically affect the outcome of interest. (https://catalogofbias.org/biases/compliance-bias/)

4.4. Was the intervention implemented successfully for most participants?

CoB: Compliance bias = Participants compliant with an intervention differ in some way from those not compliant which can systematically affect the outcome of interest. (https://catalogofbias.org/biases/compliance-bias/)

4.5. Did study participants adhere to the assigned intervention regimen?

MASTER scale (https://doi.org/10.1016/j.jclinepi.2021.01.012): 17. Care was delivered equally to all participants

Outcome data is completely accounted for each main outcome, including attrition and exclusions from the analysis, the reasons for attrition or exclusions, and any re-inclusions in analyses for the review (Higgins JP, Altman DG, Gøtzsche PC, Jüni P, Moher D, Oxman AD, Savovic J, Schulz KF, Weeks L, Sterne JA; Cochrane Bias Methods Group; Cochrane Statistical Methods Group. The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ. 2011 Oct 18;343:d5928. doi: 10.1136/bmj.d5928. PMID: 22008217; PMCID: PMC3196245.)

Errors in measuring of participants’ outcome variables arise when the measured values do not equal the true or underlying values. Such errors can bias estimates of intervention effect from a randomized trial. These errors are often referred to as measurement error (for continuous outcomes), misclassification (for dichotomous or categorical outcomes) or under-ascertainment/over-ascertainment (for events). Errors in measurement may be differential or non-differential in relation to intervention assignment.
• Differential measurement errors are related to intervention assignment. Such errors are systematically different between experimental and comparator intervention groups, and are less likely when outcome assessors are blinded to intervention assignment.
• Non-differential measurement errors are unrelated to intervention assignment.

Bias in measurement of outcomes, Bias in classification of intervention

PROPOSED DEFINITION: A bias due to the selection or application of statistical analysis methods.
PROBAST = Domain 4 (Analysis) covers potential sources of bias in the statistical analysis methods. It assesses aspects related to the choice of analysis method and whether key statistical considerations (for example, missing data) were correctly addressed. Use of inappropriate analysis methods or omission of important statistical considerations increases the potential for bias in the estimated predictive performance of a model.
QUIPS: The Statistical Analysis and Reporting domain addresses the appropriateness of the study’s statistical analysis and completeness of reporting. It helps the assessor judge whether results are likely to be spurious or biased because of analysis or reporting. To make this judgment, the assessor considers the data presented to determine the adequacy of the analytic strategy and model-building process and investigates concerns about selective reporting. Selective reporting is an important issue in prognostic factor reviews because studies commonly report only factors positively associated with outcomes. A study would be considered to have low risk of bias if the statistical analysis is appropriate for the data, statistical assumptions are satisfied, and all primary outcomes are reported.

PROPOSED DEFINITION: A bias due to distortions in the selection of or representation of information in study results or research findings.
CoB: Reporting biases = A systematic distortion that arises from the selective disclosure or withholding of information by parties involved in the design, conduct, analysis, or dissemination of a study or research findings (https://catalogofbias.org/biases/reporting-biases/) also notes: The Dictionary of Epidemiology defines reporting bias as the “selective revelation or suppression of information (e.g., about past medical history, smoking, sexual experiences) or of study results.”
The Cochrane Handbook states it arises “when the dissemination of research findings is influenced by the nature and direction of results.”
The James Lind Library states “biased reporting of research occurs when the direction or statistical significance of results influence whether and how research is reported.”
QUIPS: The Statistical Analysis and Reporting domain addresses the appropriateness of the study’s statistical analysis and completeness of reporting. It helps the assessor judge whether results are likely to be spurious or biased because of analysis or reporting. To make this judgment, the assessor considers the data presented to determine the adequacy of the analytic strategy and model-building process and investigates concerns about selective reporting. Selective reporting is an important issue in prognostic factor reviews because studies commonly report only factors positively associated with outcomes. A study would be considered to have low risk of bias if the statistical analysis is appropriate for the data, statistical assumptions are satisfied, and all primary outcomes are reported.
ROB2 = This domain addresses bias that arises because the reported result is selected (based on its direction, magnitude or statistical significance) from among multiple intervention effect estimates that were calculated by the trial investigators. We call this bias in selection of the reported result. Consideration of risk of bias requires distinction between:
• An outcome domain. This is a state or endpoint of interest, irrespective of how it is measured (e.g. severity
of depression);
• An outcome measurement. This is a specific way in which an outcome domain is measured (e.g. measurement of depression using the Hamilton rating scale 6 weeks after starting intervention); and
• An outcome analysis. This is a specific result obtained by analysing one or more outcome measurements (e.g. the difference in mean change in Hamilton rating scale scores from baseline to 6 weeks between experimental and comparator groups). This domain does not address bias due to selective non-reporting (or incomplete reporting) of outcome domains that were measured and analysed by the trial investigators (115). For example, deaths of trial participants may be recorded by the trialists, but the reports of the trial might contain no mortality data, or state only that the intervention effect estimate for mortality was not statistically significant. Such bias puts the result of a synthesis at risk because results are omitted based on their direction, magnitude or statistical significance. It should therefore be addressed at the review level, as part of an integrated assessment of the risk of reporting bias (116).
ROBINS-I = Bias in selection of the reported result