�� � �����Using technology to make the Evidence Ecosystem more efficient and effective

Global collaborations for interoperability

​

Scientific Knowledge Accelerator Foundation (SKAF)

Using evidence. Improving lives.

Making Evidence Computable

Making the Evidence Ecosystem More Efficient and Effective

Declaration of Conflict of interest: Brian Alper�

I have a financial interest with the following organisation(s) that could be perceived as a direct or indirect conflict of interest in the context or content of this presentation:

• Computable Publishing LLC

I have an affiliation with the following organisation(s) that could be perceived as a direct or indirect conflict of interest in the context or content of this presentation:

• Scientific Knowledge Accelerator Foundation, GIN, GIN Tech, GRADE Working Group

​

​

Location Knowledge – Societal Evolution

Guidelines and Scientific Knowledge

Familiar, conceptually organizing much of our workflow

Sharable Value Unit

Physical object, a relatively large unit for sharing many knowledge bits in one container

Current PLATFORM for dissemination

Sharable Value Unit

Digital object (like a PDF), a relatively large unit for sharing many knowledge bits in one container

Many specific software tools, but each tool limited to local execution

Sharable Value Unit

Small digital object (micro-content), but within the constraints of the executable environment

Widely interactive, interoperable, integrated possibilities – PLATFORM of the near future

Sharable Value Unit

Small digital object, enabling contextualized selection, customizable presentation, and reusable dissemination

Produce guidance

Disseminate guidance

to policy makers, clinicians and patients

Implement guidance and decision support

Synthesize evidence

Produce evidence

Evaluate and

improve practice

Image adapted from MAGIC Foundation

Trustworthy

evidence

Coordination and

support

Common

Methodology

Culture for sharing and innovation

Tools

and

platforms

Digitally

structured

data

​

Global standards

The Digital and Trustworthy

Evidence Ecosystem

Produce guidance

​

Disseminate guidance

Implement guidance and decision support

Synthesize evidence

Produce evidence

Evaluate and improve practice

What are we creating?

Trustworthy

evidence

Coordination and

support

Common

Methodology

Culture for sharing and innovation

Tools

and

platforms

Digitally

structured

data

​

Global standards

The Digital and Trustworthy

Evidence Ecosystem

Produce guidance

Guidelines, Decision Aids, Clinical Decision Support

​

Disseminate guidance

Implement guidance and decision support

Synthesize evidence

Systematic Reviews

Produce evidence

Research Studies

Evaluate and improve practice

What are we creating?

Trustworthy

evidence

Coordination and

support

Common

Methodology

Culture for sharing and innovation

Tools

and

platforms

Digitally

structured

data

​

Global standards

The Digital and Trustworthy

Evidence Ecosystem

MEDLINE

ClinicalTrials.gov

PICO Portal

McMaster University Evidence Alerts

GRADEpro

MAGICapp

DynaMed

MEDLINE

PICO Portal

McMaster University Evidence Alerts

MAGICapp

DynaMed

Standard Structured Form

GRADEpro

ClinicalTrials.gov

Finding and Accessing Evidence

Faster and More Accurate with Interoperable Evidence

Declaration of Conflict of interest: Joanne Dehnbostel�

I have a financial interest with the following organisation(s) that could be perceived as a direct or indirect conflict of interest in the context or content of this presentation:

• Computable Publishing LLC

I have an affiliation with the following organisation(s) that could be perceived as a direct or indirect conflict of interest in the context or content of this presentation:

• Scientific Knowledge Accelerator Foundation, GIN, GIN Tech, GRADE Working Group

​

​

Finding different types of evidence on a common search

• Supported by common approach to classification structure
• Current examples show:
• Summary of Findings derived from GRADEpro
• Research study protocol (and results) derived from ClinicalTrials.gov
• Systematic review publication derived from PubMed (MEDLINE)
• Likely available in 2025:
• Cochrane reviews – with Cochrane RevMan FHIR export
• Systematic review datasets - with SRDR+ FHIR export

Appraising and Synthesizing Evidence

Faster and More Accurate with Interoperable Knowledge

Declaration of Conflict of interest: Karen Robinson�

I have a financial interest with the following organisation(s) that could be perceived as a direct or indirect conflict of interest in the context or content of this presentation:

• None

I have an affiliation with the following organisation(s) that could be perceived as a direct or indirect conflict of interest in the context or content of this presentation:

• Scientific Knowledge Accelerator Foundation, PICO Portal

​

Using AI for data extraction and risk of bias

Increase accuracy and efficiency :

• Identify relevant text in articles/documents
• About 15-20 seconds to answer 30 RoB or extraction questions on one PDF; process 3-4 PDFs / minute for 30 questions
• Replace a reviewer

​

​

an

From tables

From figures

Interoperability

Online facilitation of risk of bias assessment

• RoBAT from FEvIR
• access the RoBAT by opening any Citation, Evidence, or Composition Resource on the FEvIR Platform, then clicking the RoBAT button in the left navigation menu.�

Accessing RoBAT

Codes

Interoperability

Developing Guidance

Faster and More Accurate with Interoperable Data

Declaration of Conflict of interest: Linn Brandt�

I have a financial interest with the following organisation(s) that could be perceived as a direct or indirect conflict of interest in the context or content of this presentation:

• MAGIC Evidence Ecosystem Foundation

I have an affiliation with the following organisation(s) that could be perceived as a direct or indirect conflict of interest in the context or content of this presentation:

• GIN, GIN Tech, GRADE Working Group

​

​

Contextualizing and Implementing Guidance

Faster and More Accurate with Interoperable Knowledge

Declaration of Conflict of interest: Ilkka Kunnamo�

I have a financial interest with the following organisation(s) that could be perceived as a direct or indirect conflict of interest in the context or content of this presentation:

• Duodecim Publishing Company

I have an affiliation with the following organisation(s) that could be perceived as a direct or indirect conflict of interest in the context or content of this presentation:

• Scientific Knowledge Accelerator Foundation, GIN, GIN Tech, GRADE Working Group

​

​

Reusing evidence resources

• Effective collaboration is based on sharing of evidence resources
• Recommendations and Summary of Findings (SOF) tables are available from guideline development tools, e.g.
• MAGICApp https://app.magicapp.org/#/guidelines has 288 public guidelines with 5982 recommendations and 4838 PICOs/SOF tables
• GRADEPro has 589 public recommendations and evidence profiles (424 in English) https://guidelines.gradepro.org/search and a large number of guidelines on organizations’ websites
• Separating computable data from user interfaces will make possible to display evidence in both uniform and tailored formats, adapt and contextualize content.

​

Adapting recommendations

Rationale for change

New wording for recommendation

Adaptation report and full version history of all changes

https://fevir.net/resources/Composition/277058

​

Summary of findings converted into computable format

Rating relative importance of outcomes

Calculating net effect and its 95 % CI

Increased risk for gastrointestinal bleeding makes net effect negative

Confidence interval is wide

Baseline risk estimate in patients with Padua Risk Score 4 or higher https://www.jthjournal.org/article/S1538-7836(22)06692-2/fulltext

2.48%

5.4%

-1.78%

-1.8%

0.53

0.27

0.22

Increasing baseline risk to correspond patients who have Padua Risk Score 4 or higher makes net effect positive (confidence interval not calculable).

Baseline risk estimate in patients with Padua Risk Score 4 or higher https://www.jthjournal.org/article/S1538-7836(22)06692-2/fulltext

2.48%

5.4%

-1.78%

-1.8%

0.53

0.27

0.22

Increasing baseline risk to correspond patients who have Padua Risk Score 4 or higher makes net effect positive (confidence interval not calculable).

Need for adaptation of the recommendation

Developing decision support rules

• A typical clinical decision support rule checks if a patient is eligible to an intervention by comparing patient data with evidence related to the P(atient) and I(ntervention and C(comparator) elements and triggers a reminder if an intervention which the patient has not received would improve patient-important O(utcomes).
• Eligibility to an intervention (and recommendation to use intervention) is based on
• inclusion criteria (e.g. diabetes and cardiovascular or renal disease)
• exlusion criteria (e.g. earlier adverse effects of SGLT-2 inhibitors or SGLT-2 inhibitors already in use)
• Patient data received in FHIR format from the electronic health record system (EHR) can be automatically mapped to evidence in FHIR format and find patients in whom both the inclusion and exclusion criteria match -> rules engines may become redundant in the future.
• Requires that codes from different coding systems (such as ICD-10, ICPC-2, SNOMED CT) are mapped to concepts that are shared between the EHR and evidence. The code mappings can be made available in FHIR format.

Example of study eligibility criteria: https://fevir.net/resources/Group/170443

​

Population health

• Question: what are the health impacts of blood pressure lowering and LDL cholesterol lowering on diabetic patients in my practice?
• Sources of evidence:

​

​

​

​

​

Evidence from systematic reviews converted into FHIR format

Outcomes of blood pressure lowering by 5 mmHg (irrespective of baseline level)

Obtaining blood pressure levels from a practice population of 1900 diabetics: data from electronic health records (EHR)

Target according to guideline

< 135/85

49 % of the population

at target

Professionals received reminders of people not meeting target for a decision support system integrated with EHR

​

The Population was automatically screened for high(est) blood pressure values using a population dashboard using EHR data, and people were actively contacted

51 % not

at target

Changes in average blood pressure and LDL cholesterol in 5 years in a practice population of 1900 diabetics: data from electronic health records (EHR)

• Systolic blood pressure 5.8 mmHg lower
• LDL cholesterol 0.56 mmol/l lower

​

• Estimated average risks for next 5 years at baseline according to cardiovascular risk calculator:
• Myocardial infarction 5.2 %
• Stroke 4.0 %

​

Number of events avoided by lowering blood pressure and LDL cholesterol in 5 years in a practice population of 1900 diabetics: calculated from observed changes and in data from electronic health records (EHR) and effect estimates from a contextualized SOF table

LDL cholesterol Blood pressure

Myocardial infarction 17 9

Stroke 8 12

Heart failure 8

Total 25 29

All events, total 54

​

​

Calculating net benefit using importance of outcomes

LDL cholesterol Blood pressure Importance of avoiding

Myocardial infarction 17 9 40 %

Stroke 8 12 70 %

Heart failure 8 60 %

Death 100 %

Total 25 29

All events, total 54

​

Importance-adjusted 12 16

(death-equivalents)

​

Informing Evidence to Decision framework for new interventions in the future

• When evidence of a new intervention, or potentially practice-changing evidence on old inverventions becomes available, the population impact (potential health benefit, need of resources) could be immediately available for different populations and on national level
• Information on population impact could be used in the EtD framework when developing recommendations on the use of the intervention

Collaborating on Projects and Using Resources Efficiently

Easier with Computable Resources

Declaration of Conflict of interest: Khalid Shahin�

I have a financial interest with the following organisation(s) that could be perceived as a direct or indirect conflict of interest in the context or content of this presentation:

• Computable Publishing LLC

I have an affiliation with the following organisation(s) that could be perceived as a direct or indirect conflict of interest in the context or content of this presentation:

• Scientific Knowledge Accelerator Foundation, GIN, GIN Tech, GRADE Working Group

​

​

Audience Discussion

What are your opportunities?

What are your concerns?

Audience Participation

• We are excited by what technology will enable for our day-to-day experience in 2025!
• Can you see it?
• Can you share it?
• What are your opportunities to use technology to make things better next year?
• What are your concerns?

EBMonFHIR

Extending the Fast Healthcare Interoperability Resources (FHIR) standard for health data exchange to provide FHIR Resources for clinical research (evidence) and recommendations for clinical care (clinical practice guidance)

HL7® FHIR®

• HL7 (Health Level 7) is a patient health standards organization for interoperability
• Their v2 specification is used in over 35 countries.
• In the United States v2 is used with 95% of hospital systems
• FHIR (Fast Healthcare Interoperability Resources) is the new standard developed by HL7
• FHIR is much more detailed compared to HL7 v2 and many hospital systems are upgrading to this standard

HL7® Work Groups

• Biomedical Research and Regulation (BR&R)

​

• Clinical Decision Support (CDS)

​

• Clinical Quality Information (CQI)

​

EBMonFHIR

• Evidence Based Medicine for FHIR (EBMonFHIR)
• We developed resources in the FHIR standard for storing clinical research data and results
• Sponsored by the CDS, CQI, and BR&R Work Groups
• EvidenceVariable and Group resources describes the components of a study Population, Intervention, Comparison, and Outcome (PICO)
• Evidence Resource captures the results and statistics of a study’s findings
• Many profiles developed for more specific use cases
• The structure is still being developed and improved upon

A portion of the Evidence Resource structure

Ideas from Khalid

•  Speaker will introduce Organization, its objectives, projects contributing to evidence interoperability, and how to get involved

​

• Introduce HL7; project of CDS/BRR/CQI WGs; EBMonFHIR IG
• Weekly EBMonFHIR IG meetings; weekly CDS WG meetings; Connectathons 3x/year

​

How to Join EBMonFHIR Development

• We meet multiple times a week and at the HL7 Connectathons
• EBMonFHIR Implementation Guide Meetings
• Thursdays 8 am US Eastern Time/2pm Central European Time
• CDS Work Group Meetings
• Thursdays 12pm US Eastern Time/6pm Central European Time
• HL7 Connectathons
• 3 Times a year, one virtual and two in-person meetings

Health Evidence Knowledge Accelerator (HEvKA)

An open virtual group working on developing universal standards for computable expression of health evidence and coordinating group and consortial efforts to advance health evidence knowledge identification, evaluation, and dissemination

Evolution of the Health Evidence Knowledge Accelerator (HEvKA)

HEvKA Today

12 Virtual Meetings Every Week

Scientific Evidence Code System (SEVCO)

Reference: Alper BS et al,  COVID-19 Knowledge Accelerator (COKA) Initiative. Making science computable: Developing code systems for statistics, study design, and risk of bias. J Biomed Inform. 2021 Mar;115:103685

You can join us!

Please join us for further presentations at GES 2024

GRADE Working Group

From evidence to recommendations – transparent and sensible

Why have a GRADE Ontology?

• Explicit definitions of the terms used in the GRADE approach
• A terminology standard like SNOMED-CT or ICD-10 or MeSH
• Interoperability of data systems that use the GRADE approach
• Computer-assisted support for using the GRADE approach
• Computer-assisted support for search based on GRADE concepts
• E.g. search for reports with ‘moderate’ or ‘high’ certainty of evidence

​

Please join us for further presentations at GES 2024

HEvKA Today

12 Virtual Meetings Every Week

Guidelines International Network (GIN)

The global network supporting evidence-based guideline development and implementation

GIN

​

Vision – Trustworthy and accessible guidance for better health.

​

Mission – To lead, strengthen and support collaboration and work within the guideline development, adaptation, and implementation community.

​

Workgroups

• Adaptation
• AI
• Implementation
• Updating
• Regional groups, like GIN-NA
• GIN-Tech

​

​

https://g-i-n.net/get-involved/working-groups

GIN Tech

The core aim of the working group is to aid GIN members in sharing data from the various digital tools and providing a forum for members to discuss the best way to use the tools that are available.

Please join us for further presentations at GES 2024

HEvKA Today

12 Virtual Meetings Every Week

Making Evidence Computable

balper@computablepublishing.com

Fevir.net

Using evidence. Improving lives.