1 of 43

WGM 2022-05-11

Evidence-Based Medicine (EBMonFHIR) and COVID-19 Knowledge Accelerator (COKA)

Brian Alper: balper@computablepublishing.com

https://docs.google.com/presentation/d/1YDAJ3Wx_psjnjBzw1lUm5ItbI-IQlnOq

© 2019+ Health Level Seven ® International. Licensed under Creative Commons Attribution 4.0 International HL7, Health Level Seven, FHIR and the FHIR flame logo are registered trademarks of Health Level Seven International. Reg. U.S. TM Office.

2 of 43

Agenda

EvidenceVariable Resource for Eligibility Criteria
Active Projects Update

Scientific Evidence Code System (SEVCO)
EBM Decision Map Profile (CPGonEBMonFHIR IG)
ArtifactAssessment

COVID-19 Knowledge Accelerator

© 2019 Health Level Seven ® International. Licensed under Creative Commons Attribution 4.0 International HL7, Health Level Seven, FHIR and the FHIR flame logo are registered trademarks of Health Level Seven International. Reg. U.S. TM Office.

2

3 of 43

EvidenceVariable Resource for Eligibility Criteria

© 2019 Health Level Seven ® International. Licensed under Creative Commons Attribution 4.0 International HL7, Health Level Seven, FHIR and the FHIR flame logo are registered trademarks of Health Level Seven International. Reg. U.S. TM Office.

3

4 of 43

Connectathon Track Participants

Brian Alper (Computable Publishing LLC)
Gustav Vella (Healex)
Khalid Shahin (Computable Publishing LLC)
Joanne Dehnbostel (Computable Publishing LLC)
Lorenz Rosenau (Universitätsklinikum Schleswig-Holstein / Universität zu Lübeck )
Martin Doege (Healex)
Gregor Lichtener (Universitätsmedizin Greifswald)

Harold Lehmann (John Hopkins)
Robert Dingwell (MITRE Corp)
Henrich Krämer (BI X GmbH)
Rana Cook (NTT Data Services)
Bret Eathorne (GE Healtcare)
Darell Woelk (Green Room Technologies)
Michelle N Dardis (Director, Department of Quality Measurement)
Matt Bishop (Open City Labs)
Alexander Zautke (Firely)

4

5 of 43

Why Eligibility Criteria?

Scenarios where machine-assisted matching patients to structured ‘eligibility criteria’ is valuable:

Clinical trials – recruitment: finding the research subjects
Clinical trials – finding available treatments/trials from the patient perspective
Research site – determine recruitment potential for selecting research sites
Clinical Guidelines – finding relevant recommendations for the patient
Clinical Guidelines – finding eligible population for implementation
Evidence/Decision Support – applying relevant evidence and decision support to the patient
Quality Measurement – finding matching patients for performance measures

5

6 of 43

Eligibility Criteria: an array of characteristics

Each characteristic is either an Inclusion Criterion (must be met to be eligible) or an Exclusion Criterion (must not be met to be eligible)
Characteristics may be defined in many ways, including:

The type of characteristic “Y” has a value of “X”
The characteristic is defined by referencing a definition
The characteristic is a specific way of combining other characteristics

Each characteristic may be qualified (modified) by:

How it is measured or determined
Timing of occurrence
Offset compared to a reference point

The number of variations and permutations leads to high complexity

�

6

7 of 43

Choosing how to define a characteristic

EvidenceVariable.characteristic supports 7 approaches to characteristic definition. For each characteristic, you must choose 1 and only 1 of:

characteristic.definitionCanonical
characteristic.definitionId
characteristic.definitionReference
characteristic.definitionCodeableConcept
characteristic.defByTypeAndValue
characteristic.defByCombination
characteristic.definitionExpression

7

8 of 43

Choosing how to define a characteristic

EvidenceVariable.characteristic supports 7 approaches to characteristic definition. For each characteristic, you must choose 1 and only 1 of:

characteristic.definitionCanonical
characteristic.definitionId
characteristic.definitionReference
characteristic.definitionCodeableConcept
characteristic.defByTypeAndValue
characteristic.defByCombination
characteristic.definitionExpression

8

9 of 43

Defining a characteristic with definitionCanonical

When to use this approach:

When the characteristic is already defined by a specific URI/URL

Why? allows re-use without rebuilding the characteristic

How to do it:

Provide the URI/URL value as the value for characteristic.definitionCanonical

9

10 of 43

Choosing how to define a characteristic

EvidenceVariable.characteristic supports 7 approaches to characteristic definition. For each characteristic, you must choose 1 and only 1 of:

characteristic.definitionCanonical
characteristic.definitionId
characteristic.definitionReference
characteristic.definitionCodeableConcept
characteristic.defByTypeAndValue
characteristic.defByCombination
characteristic.definitionExpression

10

11 of 43

Defining a characteristic with definitionId

When to use this approach:

When the characteristic is already defined elsewhere in the EvidenceVariable Resource

Why? allows re-use without rebuilding the characteristic

How to do it:

1) insert a value (short string) in characteristic.linkId in the characteristic element where it is defined
2) provide this value as the value for characteristic.definitionId

11

12 of 43

Choosing how to define a characteristic

EvidenceVariable.characteristic supports 7 approaches to characteristic definition. For each characteristic, you must choose 1 and only 1 of:

characteristic.definitionCanonical
characteristic.definitionId
characteristic.definitionReference
characteristic.definitionCodeableConcept
characteristic.defByTypeAndValue
characteristic.defByCombination
characteristic.definitionExpression

12

13 of 43

Defining a characteristic with definitionReference

When to use this approach:

when the characteristic is already defined with a Resource (but not a canonical resource with direct URL)

Why? allows re-use without rebuilding the characteristic

How to do it (use any of):

use characteristic.definitionReference.reference to provide the absolute or relative path to the resource
use characteristic.definitionReference.identifier to reference the resource by identifier when unable to directly reference it
use characteristic.definitionReference.display to provide a text narrative to identify the resource

13

14 of 43

Choosing how to define a characteristic

EvidenceVariable.characteristic supports 7 approaches to characteristic definition. For each characteristic, you must choose 1 and only 1 of:

characteristic.definitionCanonical
characteristic.definitionId
characteristic.definitionReference
characteristic.definitionCodeableConcept
characteristic.defByTypeAndValue
characteristic.defByCombination
characteristic.definitionExpression

14

15 of 43

Defining a characteristic with definitionCodeableConcept

When to use this approach:

when the characteristic is defined with a single code or text description (e.g. “good performance status” as a single concept, as opposed to combining a code for “good” and a code for “performance status”)

Why? allows re-use without rebuilding the characteristic

How to do it:

use characteristic.definitionCodeableConcept.coding when providing term(s) from a code system as the value
use characteristic.definitionCodeableConcept.text when providing a plain text alternative to define the characteristic

15

16 of 43

Choosing how to define a characteristic

EvidenceVariable.characteristic supports 7 approaches to characteristic definition. For each characteristic, you must choose 1 and only 1 of:

characteristic.definitionCanonical
characteristic.definitionId
characteristic.definitionReference
characteristic.definitionCodeableConcept
characteristic.defByTypeAndValue
characteristic.defByCombination
characteristic.definitionExpression

16

17 of 43

Defining a characteristic with defByTypeAndValue

When to use this approach:

when the characteristic is defined with 2 concepts, one to describe what is observed and one for the acceptable value(s) for the observation (e.g. type defined by “performance status” and value defined by “good”)

How to do it:

use characteristic.defByTypeAndValue.type[x] to describe what is observed

typeCodeableConcept, typeReference, or typeId

use characteristic.defByTypeAndValue.value[x] to describe the acceptable value[s] matching the type:

valueCodeableConcept, valueBoolean, valueQuantity, valueRange, valueReference, or valueId

17

18 of 43

Choosing how to define a characteristic

EvidenceVariable.characteristic supports 7 approaches to characteristic definition. For each characteristic, you must choose 1 and only 1 of:

characteristic.definitionCanonical
characteristic.definitionId
characteristic.definitionReference
characteristic.definitionCodeableConcept
characteristic.defByTypeAndValue
characteristic.defByCombination
characteristic.definitionExpression

18

19 of 43

Defining a characteristic with defByCombination

When to use this approach:

when the characteristic is defined as a combination of, or a subset of, characteristics

How to do it:

use characteristic.defByCombination.code to define how to combine characteristics (all-of, any-of, at-least, at-most)
use characteristic.defByCombination.threshold if using at-least or at-most for the code
use characteristic.defByCombination.characteristic to contain all the characteristics in this combination

19

20 of 43

Choosing how to define a characteristic

EvidenceVariable.characteristic supports 7 approaches to characteristic definition. For each characteristic, you must choose 1 and only 1 of:

characteristic.definitionCanonical
characteristic.definitionId
characteristic.definitionReference
characteristic.definitionCodeableConcept
characteristic.defByTypeAndValue
characteristic.defByCombination
characteristic.definitionExpression

20

21 of 43

Defining a characteristic with definitionExpression

When to use this approach:

when defining the characteristic with a formula for calculation, a complex logical combination, or direct expression for machine use

How to do it:

use characteristic.definitionExpression.description for a plain text way to define the logical expression
use characteristic.definitionExpression.language to define the formal expression language (e.g. CQL)
use characteristic.definitionExpression.expression to define with the formal expression language

21

22 of 43

Notable achievements

Introduction and understanding of many EvidenceVariable.characteristic elements, including 7 different definition element choices
Documentation of “when to use” and “how to use” for 7 different definition element choices and 6 different value element choices
two FHIR change requests for EvidenceVariable

add a characteristic.defByCombination.code = “except-subset”
add a characteristic.instances[x] 0..1 Quantity | Range

Modeling of many characteristics of different forms of definition, many examples
EvidenceVariable Resource created by a third party (German Medical Informatics Initiative) was viewable on FEvIR Platform
FEvIR Platform has 4 examples https://fevir.net/resources/Project/32444 and shows 70%-80% of the functionality
Confirmation of need for and general acceptance of Eligibility Criteria in EvidenceVariable

22

23 of 43

Scientific Evidence Code System (SEVCO)

© 2019 Health Level Seven ® International. Licensed under Creative Commons Attribution 4.0 International HL7, Health Level Seven, FHIR and the FHIR flame logo are registered trademarks of Health Level Seven International. Reg. U.S. TM Office.

23

24 of 43

Here is a screenshot of CodeSystem Viewer showing a code system that we are developing called the Scientific Evidence Code System. This instance is titled “DRAFT ONLY (Not published for current use)” as we are using this CodeSystem Resource to develop the code system.

The left navigation shows this is a hierarchical code system, and we are looking at a specific term. The ‘Phase 3 trial’ is a draft term. The preferred term is phase 3 trial and there are 4 alternative terms. You can see a definition for the term, a comment for application, and various parent terms in a hierarchical tree to see where this term fits. Then you can see a number of other properties, three of which are shown in this screenshot. You can see how we are using the properties to track the provenance of the term development. Any disagreement from the expert working group leads to continued discussion and revision. When we reach 100% agreement, we will add an “Expert Working Group Agreement” property and remove the “(draft)” marker from the term.

This CodeSystem Resource is openly available. Anyone is welcome to view this system and can add a comment for any term. Just use the Comment button.

25 of 43

26 of 43

27 of 43

28 of 43

FEvIR Platform: Terminology Tools You Can Use

Scientific Evidence Code System

Progress-to-date as of May 11, 2022.

	# Terms	# Approved
Study Design	61	54
Risk of Bias	260	104
Statistics	235	34
SEVCO (total)	561	192

34%

(was 22% in January 2022)

29 of 43

EBM Decision Map Profile (EBMonFHIR IG)

© 2019 Health Level Seven ® International. Licensed under Creative Commons Attribution 4.0 International HL7, Health Level Seven, FHIR and the FHIR flame logo are registered trademarks of Health Level Seven International. Reg. U.S. TM Office.

29

30 of 43

ClinicalPracticeGuideline Profile of ImplementationGuide

Complete report and mapping of a guideline
ref RecommendationPlan Profile of PlanDefinition
ref GuidelineCitation Profile of Citation

30

31 of 43

RecommendationPlan Profile of PlanDefinition

Description of a recommended intervention in a guideline
ref RecommendationAction Profile of ActivityDefinition
ref RecommendationEligibilityCriteria Profile of EvidenceVariable
ref RecommendationJustification Profile of ArtifactAssessment
ref RecommendationCitation Profile of Citation

31

32 of 43

RecommendationJustification Profile of ArtifactAssessment

Evidence, justification, and ratings of evidence-to-decision framework judgments for a recommendation
ref EvidenceReport Profile of Composition
ref NetEffect Profile of EvidenceVariable
ref NetEffectEstimate Profile of Evidence
ref CertaintyOfEvidence Profile of ArtifactAssessment

32

33 of 43

EvidenceReport Profile of Composition

ref StudyCitation Profile of Citation
ref StudyOutcomeEvidence Profile of Evidence
ref OutcomeEvidenceSynthesis Profile of Evidence
ref CertaintyOfEvidence Profile of ArtifactAssessment

33

34 of 43

OutcomeEvidenceSynthesis Profile of Evidence

Evidence (statistics) generated from a meta-analysis or systematic review for a single clinical question (PICO)
ref StudyCitation Profile of Citation
ref StudyOutcomeEvidence Profile of Evidence
ref StudyGroupGroup Profile of Group
ref CertaintyOfEvidence Profile of ArtifactAssessment
ref StudyEligibilityCriteria Profile of EvidenceVariable
ref EvidenceDataset Profile of EvidenceVariable

34

35 of 43

StudyOutcomeEvidence Profile of Evidence

Evidence (statistics) generated from a single study for a single clinical question (PICO)
ref StudyCitation Profile of Citation
ref StudyGroup Profile of Group
ref RiskOfBias Profile of ArtifactAssessment
ref StudyEligibilityCriteria Profile of EvidenceVariable
ref InterventionDefinition Profile of EvidenceVariable
ref OutcomeDefinition Profile of EvidenceVariable

35

36 of 43

Putting It All Together

9 Resources profiled
21 Profiles

36

37 of 43

ArtifactAssessment

© 2019 Health Level Seven ® International. Licensed under Creative Commons Attribution 4.0 International HL7, Health Level Seven, FHIR and the FHIR flame logo are registered trademarks of Health Level Seven International. Reg. U.S. TM Office.

37

38 of 43

38

39 of 43

Actual Uses of ArtifactAssesment include

Comments on SEVCO terms
Votes on SEVCO terms
Risk of Bias Assessments (RoBAT)
Recommendation Justification
MeSH Headings and Keywords (with qualifier codings)
Third-party classifications for Citations

© 2019 Health Level Seven ® International. Licensed under Creative Commons Attribution 4.0 International HL7, Health Level Seven, FHIR and the FHIR flame logo are registered trademarks of Health Level Seven International. Reg. U.S. TM Office.

39

40 of 43

COVID-19 KNOWLEDGE ACCELERATOR INTRODUCTION

© 2019 Health Level Seven ® International. Licensed under Creative Commons Attribution 4.0 International HL7, Health Level Seven, FHIR and the FHIR flame logo are registered trademarks of Health Level Seven International. Reg. U.S. TM Office.

40

41 of 43

COVID-19 Knowledge Accelerator (COKA)

A virtual organization – no cost, no contracts
Open, transparent – all meetings
Engages multiple global collaborative groups, organizations, and individuals
A way for any to volunteer to accelerate development and implementation of standards for evidence exchange, especially with attention to COVID-19

© 2019 Health Level Seven ® International. Licensed under Creative Commons Attribution 4.0 International HL7, Health Level Seven, FHIR and the FHIR flame logo are registered trademarks of Health Level Seven International. Reg. U.S. TM Office.

41

42 of 43

COKA has 11 active working groups

Project Management
Eligibility Criteria WG
Research Design WG
Statistic Standard and Terminology WG
Risk of Bias Terminology and Tooling WG
Common Metadata Framework WG
Computable EBM Tools Development WG
Usability Research WG
EBM Decision Map Profile WG
Knowledge Ecosystem Liaison WG
Communications WG CDS EBM sub-WG

© 2019 Health Level Seven ® International. Licensed under Creative Commons Attribution 4.0 International HL7, Health Level Seven, FHIR and the FHIR flame logo are registered trademarks of Health Level Seven International. Reg. U.S. TM Office.

42

43 of 43

43