1 of 26

GRADE Ontology Project

GRADE Ontology Working Group

Saphia Mokrane, Joanne Dehnbostel

GIN, Geneva, September 18th, 2025

2 of 26

Plan

The GRADE Approach, an evolving approach
The GRADE Ontology Working Group
How to participate in the GRADE Ontology Working Group

3 of 26

Ontology: Definition

Computer science:

Any formal representation of a set of concepts within a domain, and the relationships between those concepts (Gruber, 1995; Whetzel et al., 2011).

Examples:

SNOMED - recording patient clinical information (Stearns et al., 2001)
Mesh - terms used by the US National Library of Medicine for indexing health research (Lowe and Barnett, 1994).

4 of 26

The GRADE Approach

Grading of Recommendations Assessment, Development and Evaluation

A structured framework

for the assessment of the certainty of a body of evidence by outcome and across outcomes.

for explicit processes and transparent judgements to moving from evidence to a decision using EtD frameworks - recommendation.

5 of 26

The GRADE Approach, an evolving approach …

… to meet the needs of systematic review authors, guideline developers and other users. (https://www.gradeworkinggroup.org/)

How to apply the GRADE approach to qualitative evidence?	Development of the CERQual-Project
How to apply the GRADE approach to recommendations without direct evidence?	Development of Good Practice Statement and Consensus-Based Recommendation
How does the GRADE approach take into account the Equity in the development of a Recommendation?	Improvement of the EtD framework
How could the availability of the more recent evidence influence the rating of evidence in the GRADE approach ?	Availability v/s Accessibility : Publication Bias, Reporting Bias (one of the Risk of Bias) => Dissemination Bias

6 of 26

The GRADE Book

https://book.gradepro.org/

7 of 26

8 of 26

Why is the GRADE Ontology important for the GRADE Approach?

The GRADE approach

Already widely accepted framework
Commonly used by producers of SR and GL developers
Computerized into multiple applications such as ISoF, GRADEpro or MAGICapp
Lack of a formalized terminology standard within GRADE

Potential inconsistent application or understanding of the GRADE terminology
Limited with efficiency with which the results of its use can be communicated.
Development of the GRADE Ontology is necessary.

Potential benefits for creators and users of systematic reviews and guidelines, including educators and learners & Tool developers

9 of 26

The GRADE Ontology Project (P. Whaley, B. Alper)

Objective:

To develop and maintain a standardized, computable vocabulary of terms, an ontology, necessary for the expression of GRADE certainty-of-evidence and evidence-to-decision judgements.

Means:

GRADE Ontology Working Group
HEvKA platform
Support GGG

10 of 26

Weekly open vote

(GRADE ontology working group)

Weekly meeting

(GRADE ontology working group)

Feedback from the GGG - Open vote

GRADE ontology working group & GGG

100% “Yes” vote

YES/NO votes

Comments

« We believe an exacting, iterated consensus process is essential for ensuring the utility and acceptability of the GRADE Ontology . »

Discussion on any dissenting comments or votes, and modified terms reopened for vote.

GRADE Ontology #1

100% “Yes” vote

The voting process

P. Whaley et al. / Journal of Clinical Epidemiology 187 (2025) 111921

11 of 26

The GRADE project groups

12 of 26

13 of 26

P. Whaley et al. / Journal of Clinical Epidemiology 187 (2025) 111921

14 of 26

How To Get Involved:

The GRADE Ontology Project is a part of the

Health Evidence Knowledge Accelerator

15 of 26

Evolution of the Health Evidence Knowledge Accelerator (HEvKA)

Guidelines International Network GIN Tech Meeting-suggestion to achieve interoperability for Evidence Ecosystem

2017

HL7 EBMonFHIR Project created-meetings 1x per week

2018

Covid-19 put pressure on the evidence system-Covid Knowledge Accelerator (COKA)-Meetings 12x per week

2020

Scientific Knowledge Accelerator Foundation

Non-Profit created to support the effort. SKAF subsidized poster printing for this conference.

2022

COKA became Health Evidence Knowledge Accelerator (HEvKA) to widen our focus -

11 meetings/week

2023-2025

16 of 26

HEvKA Today

11 Virtual Meetings Every Week

17 of 26

Review of Our Process

a 13 step HEvKA/SEVCO term definition protocol (Alper et al),
virtual weekly meeting of an international group of GRADE experts

to curate terms from published GRADE guidances,
to establish preferred and alternative terms for included concepts,
to define the concepts,
to write guidance for application.

Voting is supported on the FEvIR Platform at https://fevir.net
discussion of any dissenting comments or votes until agreement is reached.
A globally agreed approach across a diverse community is required to create a comprehensive consensus

18 of 26

To Join the Project go to https://fevir.net/resources/Composition/111563

Log in to the FEvIR Platform (can use Google)

email must be added to your profile

Press the “Join Voting Group” Button

19 of 26

Scroll to Project Actions to View or Vote

Click View to see the developing GRADE Ontology Draft

20 of 26

Terms are considered properties or values, use the left navigation bar to find a term of interest

21 of 26

Outcome Importance, For Example

From here you can comment or vote on this term

22 of 26

Or you can go to the MyBallot App on the FEvIR Platform to vote for all open terms at once. There is currently only one term open for voting. https://fevir.net/myballot

23 of 26

Results so far:

87 Total Terms

21 Approved by GRADE Ontology Working Group

8 Endorsed by GGG (13 in the process)

At the time of the GES last year: 24 countries, 23 unique term editors, 25 unique voters, 42 unique meeting participants, 78 total unique participants

24 of 26

Conclusions

Conclusions: We have successfully migrated the SEVCO/ HEvKA terminology development process to another domain and achieved a remarkable level of consensus. Feasibility and stability of the process have been demonstrated. That success will lead to coherence across GRADE reports and their computability allowing results of the GRADE process to be widely disseminated.

This project is open to everyone, please join the project group using the following link: https://fevir.net/resources/Project/111563.

Reference: Alper BS et al, COVID-19 Knowledge Accelerator (COKA) Initiative. Making science computable: Developing code systems for statistics, study design, and risk of bias. J Biomed Inform. 2021 Mar;115:103685.