1 of 6

�� How to Measure the Rate of Knowledge Transfer from Clinical Trials to Clinical Practice Guidance

Dehnbostel J ¹, Alper B ¹, Shahin K¹, Robinson K ^1,2, Couban R ^1,3

1 Scientific Knowledge Accelerator Foundation, Franklin NC, USA�2 Johns Hopkins University, Baltimore MD, United States�3 McMaster University , Hamilton Ontario, Canada

Using evidence. Improving lives.

Detailed Outline for Activities/Interaction Plans

7 minutes: Brian Alper (Facilitator) will introduce the Evidence Ecosystem and the potential impact of a common standard for computable evidence

GPS print to digital to executable to computable 4 square intro
Evidence Ecosystem image – around the circle – many handoffs
7 parts – 42 conversions vs. 14 conversions (calculate the number if 100 parts)

7 minutes each (total 35 minutes) - Speaker will introduce how technology can improve the experience in 2025

Joanne Dehnbostel - finding and accessing evidence

Combining more things and more types of things to be searched – e.g. combine MEDLINE search and ClinicalTrials.gov search on FEvIR Platform for search terms that show both (demo of search term that leads to results with MEDLINE Citations, MAGICapp Guideline/Recommendation/Summary of Findings, GRADEpro Recommendation/Summary of Findings, ClinicalTrials.gov ResearchStudy) – e.g. rare cancer or some focused clinical question that is easily shown in both MAGICapp and GRADEpro, e.g. remdesivir for COVID
Show the search and the different types of things found
Discuss how commonality in classification and commonality in basic structure for representing a knowledge artifact makes this possible
Note how Cochrane reviews can be included with Cochrane Revman FHIR export

Karen Robinson - appraising and synthesizing evidence

Using AI for data extraction and risk of bias with PICO Portal
Using RoBAT (ROB1 profile) for an example RCT

Linn Brandt - developing guidance

New possibilities for Translation, using LLC, Automatic vs. manual - what is good enough, what is scary and potentially bad.
- Using the same coding systems to allow users to group things and find stuff, - how do we decide, agree. What are the most important stuff.
- How we experience users are levraging capabilities of multiple systems, including very basic ones, and why it is difficult to overcome that.- and probably will be like that also going forward.
…Maybe others have other ideas.

Ilkka Kunnamo - contextualizing and implementing guidance

Start with a MAGICapp recommendation
Adapt the MAGICapp recommendation, show the adapted recommendation and the Adaptation Report
Modify the MAGICapp summary of findings by adding relative importance of outcomes, show Summary of Net Effect
Further developments that will be ready in 2025 - show mockup of Population Net Benefit in Duodecim tools

Khalid Shahin - collaborating on projects and using resources efficiently

Show an example of obtaining a guideline from MAGICapp, adapting a recommendation from that guideline, and comparing adapted recommendations from multiple guideline panel members
Technology improves by facilitating precise, exact changes; by re-using much detail from other systems
Technology improves by facilitating precise, exact combinations of the focal knowledge units

20 minutes: Active audience discussion responding to speakers and sharing opportunities and concerns

4 minutes each (total 20 minutes) - Speaker will introduce Organization, its objectives, projects contributing to evidence interoperability, and how to get involved

1. Khalid Shahin - EBM on Fast Healthcare Interoperability Resources (EBMonFHIR)

Introduce HL7; project of CDS/BRR/CQI WGs; EBMonFHIR IG
Weekly EBMonFHIR IG meetings; weekly CDS WG meetings; Connectathons 3x/year

5. Joanne Dehnbostel - Health Evidence Knowledge Accelerator (HEvKA)

Introduce HEvKA and full meeting schedule
Note Measuring the Rate later in the conference
Introduce SEVCO and highlight its meetings

3. Ilkka Kunnamo - GRADE Working Group

Introduce GRADE Ontology, note poster later in the conference

2. Karen Robinson - Guidelines International Network (GIN)

Note related WGs (Accelerated, Adaptation, AI, Updating, …)

4. Brian Alper - GINTech (a GIN work group)

GINTech page https://connect.g-i-n.net/groups/5f99577864bb59785dba7f1e/details
The group that triggered EBMonFHIR
Thursday meeting every 6 weeks
HEvKA Making Guidelines Computable WG meeting weekly Wednesdays
Poster “Making Guidelines Computable” P29 tomorrow

8 minutes: open discussion

�

2 of 6

Declaration of Conflict of Interest: Joanne Dehnbostel

I have a financial interest with the following organisation that could be perceived as a direct or indirect conflict of interest in the context or content of this presentation:

Employed by Computable Publishing LLC

I have an affiliation with the following organisations that could be perceived as a direct or indirect conflict of interest in the context or content of this presentation:

Scientific Knowledge Accelerator Foundation
Guidelines International Network (GIN)
GIN Tech
GRADE Working Group
Health Evidence Knowledge Accelerator (HEvKA)

3 of 6

They say it takes 17 years to get from evidence to practice. Is that still true? How can we measure this Today?

We have been working to create a data standard to improve the efficiency of evidence data exchange, and we need a way to measure whether or not we are shortening this gap.

The Method:

1) Identify The Evidence

Create Reproducible Search Strategy for Randomized Controlled Trials (RCTs) using Scopus
Download List of articles in RIS Form
Upload this list to FEvIR Platform using RIS converter

We often hear that it takes 17 years to get from evidence to practice. But we do not have an accurate way to measure this important time gap in the modern era.

We owe it to the health of patients to find the best available and most current evidence when making critical decisions.

The Scientific Knowledge Accelerator Foundation is developing measures of the rate of scientific knowledge transfer.

It hasn’t taken 17 years to do this study yet, but the original idea started in a GINTech meeting in 2017, the Evidence Based Medicine on FHIR Health level 7 working group was created in 2018

We all know what happened in 2020 making this project even more important. So we created the Covid-19 Knowledge Accelerator to focus on a single topic. In 2022 we created a non profit organization to help to support our efforts and called it the Scientific Evidence Knowledge Accelerator. In 2023 we felt comfortable retiring the Covid-19 knowledge accelerator which became the Health Evidence Knowledge accelerator to cover a wide range of topics.

here we are in 2024 with a protocol and a software tool ready to do the work.

4 of 6

When Was the Original Work First Published?- Start the Clock!

5 of 6

Find the Date First Published for Citing Articles that match our Inclusion Criteria and�Stop the Clock!

2) Find Citing Articles

Download list of citing articles

Upload to the FEvIR Platform using RIS converter

Apply eligibility criteria

Systematically Derived
Intended to guide clinical practice
Research results of index article are part of the results of citing article
Record the date of the citing article
Adjudicate results--Stop the clock!

6 of 6

Conclusions/Future Directions

We now have a way of measuring the rate of scientific knowledge transfer for the specific purpose of guiding clinical practice.
A software tool to facilitate this type of investigation is freely available at https://FEvIR.net/RADAR.
You can be involved in this project. Use the QR code or the link below to find out how to join us! https://fevir.net/206346

Questions?