1 of 49

WGM 2022-05-13

Evidence-Based Medicine (EBMonFHIR) and COVID-19 Knowledge Accelerator (COKA)

Brian Alper: balper@computablepublishing.com

https://docs.google.com/presentation/d/1arLXwZpT4RBlRl2C7KFVpZyYMmGG5RaN

© 2019+ Health Level Seven ® International. Licensed under Creative Commons Attribution 4.0 International HL7, Health Level Seven, FHIR and the FHIR flame logo are registered trademarks of Health Level Seven International. Reg. U.S. TM Office.

2 of 49

Agenda

  • EvidenceVariable Resource
    • Eligibility Criteria
    • Jira trackers – for voting
  • Active Projects Update
    • Scientific Evidence Code System (SEVCO)
    • EBMonFHIR IG
      • EBM Decision Map Profile (CPGonEBMonFHIR IG)
    • ArtifactAssessment
  • COVID-19 Knowledge Accelerator

​

© 2019 Health Level Seven ® International. Licensed under Creative Commons Attribution 4.0 International HL7, Health Level Seven, FHIR and the FHIR flame logo are registered trademarks of Health Level Seven International. Reg. U.S. TM Office.

2

3 of 49

EvidenceVariable Resource for Eligibility Criteria

© 2019 Health Level Seven ® International. Licensed under Creative Commons Attribution 4.0 International HL7, Health Level Seven, FHIR and the FHIR flame logo are registered trademarks of Health Level Seven International. Reg. U.S. TM Office.

3

 

4 of 49

Connectathon Track Participants

  • Brian Alper (Computable Publishing LLC)
  • Gustav Vella (Healex)
  • Khalid Shahin (Computable Publishing LLC)
  • Joanne Dehnbostel (Computable Publishing LLC)
  • Lorenz Rosenau (Universitätsklinikum Schleswig-Holstein / Universität zu Lübeck )
  • Martin Doege (Healex)
  • Gregor Lichtener (Universitätsmedizin Greifswald)

​

  • Harold Lehmann (John Hopkins)
  • Robert Dingwell (MITRE Corp)
  • Henrich Krämer (BI X GmbH)
  • Rana Cook (NTT Data Services)
  • Bret Eathorne (GE Healtcare)
  • Darell Woelk (Green Room Technologies)
  • Michelle N Dardis (Director, Department of Quality Measurement)
  • Matt Bishop (Open City Labs)
  • Alexander Zautke (Firely)

​

4

5 of 49

Why Eligibility Criteria?

Scenarios where machine-assisted matching patients to structured ‘eligibility criteria’ is valuable:

  • Clinical trials – recruitment: finding the research subjects
  • Clinical trials – finding available treatments/trials from the patient perspective
  • Research site – determine recruitment potential for selecting research sites
  • Clinical Guidelines – finding relevant recommendations for the patient
  • Clinical Guidelines – finding eligible population for implementation
  • Evidence/Decision Support – applying relevant evidence and decision support to the patient
  • Quality Measurement – finding matching patients for performance measures

5

6 of 49

Eligibility Criteria: an array of characteristics

  • Each characteristic is either an Inclusion Criterion (must be met to be eligible) or an Exclusion Criterion (must not be met to be eligible)
  • Characteristics may be defined in many ways, including:
    • The type of characteristic “Y” has a value of “X”
    • The characteristic is defined by referencing a definition
    • The characteristic is a specific way of combining other characteristics
  • Each characteristic may be qualified (modified) by:
    • How it is measured or determined
    • Timing of occurrence
    • Offset compared to a reference point
  • The number of variations and permutations leads to high complexity

�

6

7 of 49

Choosing how to define a characteristic

EvidenceVariable.characteristic supports 7 approaches to characteristic definition. For each characteristic, you must choose 1 and only 1 of:

  • characteristic.definitionCanonical
  • characteristic.definitionId
  • characteristic.definitionReference
  • characteristic.definitionCodeableConcept
  • characteristic.defByTypeAndValue
  • characteristic.defByCombination
  • characteristic.definitionExpression

​

7

8 of 49

Choosing how to define a characteristic

EvidenceVariable.characteristic supports 7 approaches to characteristic definition. For each characteristic, you must choose 1 and only 1 of:

  • characteristic.definitionCanonical
  • characteristic.definitionId
  • characteristic.definitionReference
  • characteristic.definitionCodeableConcept
  • characteristic.defByTypeAndValue
  • characteristic.defByCombination
  • characteristic.definitionExpression

8

9 of 49

Defining a characteristic with definitionCanonical

When to use this approach:

  • When the characteristic is already defined by a specific URI/URL

Why? allows re-use without rebuilding the characteristic

How to do it:

  • Provide the URI/URL value as the value for characteristic.definitionCanonical

9

10 of 49

Choosing how to define a characteristic

EvidenceVariable.characteristic supports 7 approaches to characteristic definition. For each characteristic, you must choose 1 and only 1 of:

  • characteristic.definitionCanonical
  • characteristic.definitionId
  • characteristic.definitionReference
  • characteristic.definitionCodeableConcept
  • characteristic.defByTypeAndValue
  • characteristic.defByCombination
  • characteristic.definitionExpression

10

11 of 49

Defining a characteristic with definitionId

When to use this approach:

  • When the characteristic is already defined elsewhere in the EvidenceVariable Resource

Why? allows re-use without rebuilding the characteristic

How to do it:

  • 1) insert a value (short string) in characteristic.linkId in the characteristic element where it is defined
  • 2) provide this value as the value for characteristic.definitionId

​

11

12 of 49

Choosing how to define a characteristic

EvidenceVariable.characteristic supports 7 approaches to characteristic definition. For each characteristic, you must choose 1 and only 1 of:

  • characteristic.definitionCanonical
  • characteristic.definitionId
  • characteristic.definitionReference
  • characteristic.definitionCodeableConcept
  • characteristic.defByTypeAndValue
  • characteristic.defByCombination
  • characteristic.definitionExpression

12

13 of 49

Defining a characteristic with definitionReference

When to use this approach:

  • when the characteristic is already defined with a Resource (but not a canonical resource with direct URL)

Why? allows re-use without rebuilding the characteristic

How to do it (use any of):

  • use characteristic.definitionReference.reference to provide the absolute or relative path to the resource
  • use characteristic.definitionReference.identifier to reference the resource by identifier when unable to directly reference it
  • use characteristic.definitionReference.display to provide a text narrative to identify the resource

13

14 of 49

Choosing how to define a characteristic

EvidenceVariable.characteristic supports 7 approaches to characteristic definition. For each characteristic, you must choose 1 and only 1 of:

  • characteristic.definitionCanonical
  • characteristic.definitionId
  • characteristic.definitionReference
  • characteristic.definitionCodeableConcept
  • characteristic.defByTypeAndValue
  • characteristic.defByCombination
  • characteristic.definitionExpression

14

15 of 49

Defining a characteristic with definitionCodeableConcept

When to use this approach:

  • when the characteristic is defined with a single code or text description (e.g. “good performance status” as a single concept, as opposed to combining a code for “good” and a code for “performance status”)

Why? allows re-use without rebuilding the characteristic

How to do it:

  • use characteristic.definitionCodeableConcept.coding when providing term(s) from a code system as the value
  • use characteristic.definitionCodeableConcept.text when providing a plain text alternative to define the characteristic

​

15

16 of 49

Choosing how to define a characteristic

EvidenceVariable.characteristic supports 7 approaches to characteristic definition. For each characteristic, you must choose 1 and only 1 of:

  • characteristic.definitionCanonical
  • characteristic.definitionId
  • characteristic.definitionReference
  • characteristic.definitionCodeableConcept
  • characteristic.defByTypeAndValue
  • characteristic.defByCombination
  • characteristic.definitionExpression

16

17 of 49

Defining a characteristic with defByTypeAndValue

When to use this approach:

  • when the characteristic is defined with 2 concepts, one to describe what is observed and one for the acceptable value(s) for the observation (e.g. type defined by “performance status” and value defined by “good”)

How to do it:

  • use characteristic.defByTypeAndValue.type[x] to describe what is observed
    • typeCodeableConcept, typeReference, or typeId
  • use characteristic.defByTypeAndValue.value[x] to describe the acceptable value[s] matching the type:
    • valueCodeableConcept, valueBoolean, valueQuantity, valueRange, valueReference, or valueId

​

17

18 of 49

Choosing how to define a characteristic

EvidenceVariable.characteristic supports 7 approaches to characteristic definition. For each characteristic, you must choose 1 and only 1 of:

  • characteristic.definitionCanonical
  • characteristic.definitionId
  • characteristic.definitionReference
  • characteristic.definitionCodeableConcept
  • characteristic.defByTypeAndValue
  • characteristic.defByCombination
  • characteristic.definitionExpression

18

19 of 49

Defining a characteristic with defByCombination

When to use this approach:

  • when the characteristic is defined as a combination of, or a subset of, characteristics

How to do it:

  • use characteristic.defByCombination.code to define how to combine characteristics (all-of, any-of, at-least, at-most)
  • use characteristic.defByCombination.threshold if using at-least or at-most for the code
  • use characteristic.defByCombination.characteristic to contain all the characteristics in this combination

19

20 of 49

Choosing how to define a characteristic

EvidenceVariable.characteristic supports 7 approaches to characteristic definition. For each characteristic, you must choose 1 and only 1 of:

  • characteristic.definitionCanonical
  • characteristic.definitionId
  • characteristic.definitionReference
  • characteristic.definitionCodeableConcept
  • characteristic.defByTypeAndValue
  • characteristic.defByCombination
  • characteristic.definitionExpression

20

21 of 49

Defining a characteristic with definitionExpression

When to use this approach:

  • when defining the characteristic with a formula for calculation, a complex logical combination, or direct expression for machine use

How to do it:

  • use characteristic.definitionExpression.description for a plain text way to define the logical expression
  • use characteristic.definitionExpression.language to define the formal expression language (e.g. CQL)
  • use characteristic.definitionExpression.expression to define with the formal expression language

21

 

 

 

22 of 49

Notable achievements

  1. Introduction and understanding of many EvidenceVariable.characteristic elements, including 7 different definition element choices
  2. Documentation of “when to use” and “how to use” for 7 different definition element choices and 6 different value element choices
  3. two FHIR change requests for EvidenceVariable
    1. add a characteristic.defByCombination.code = “except-subset”
    2. add a characteristic.instances[x] 0..1 Quantity | Range
  4. Modeling of many characteristics of different forms of definition, many examples
  5. EvidenceVariable Resource created by a third party (German Medical Informatics Initiative) was viewable on FEvIR Platform
  6. FEvIR Platform has 4 examples https://fevir.net/resources/Project/32444 and shows 70%-80% of the functionality
  7. Confirmation of need for and general acceptance of Eligibility Criteria in EvidenceVariable

​

22

23 of 49

Jira trackers

© 2019 Health Level Seven ® International. Licensed under Creative Commons Attribution 4.0 International HL7, Health Level Seven, FHIR and the FHIR flame logo are registered trademarks of Health Level Seven International. Reg. U.S. TM Office.

23

24 of 49

Add except-subset to CharacteristicCombination code set for EvidenceVariable

  • https://jira.hl7.org/browse/FHIR-37338
  • Example: Known history of chronic liver disease (except for NAFLD/NASH) https://fevir.net/resources/EvidenceVariable/32960
    • defByCombination
      • code = except-subset
      • characteristic
        • Exclude = false
        • defByTypeAndValue -- valueCodeableConcept.coding for Chronic liver disease (disorder)
      • characteristic
        • Exclude = true
        • defByTypeAndValue -- valueCodeableConcept.coding for Chronic nonalcoholic liver disease (disorder)

​

​

24

25 of 49

Add EvidenceVariable.characteristic.instances[x]

  • https://jira.hl7.org/browse/FHIR-37346
  • instances[x] 0..1 Quantity | Range
  • Example: Severe depression requiring > 2 medications https://fevir.net/resources/EvidenceVariable/32960
    • defByTypeAndValue
      • typeCodeableConcept.coding.code: MedicationRequest
      • valueCodeableConcept.text: reason for MedicationRequest includes Depression (SNOMED 35489007)
    • instancesQuantity
      • comparator: >
      • value: 2

25

26 of 49

Change EvidenceVariable.characteristic to 7 definition… elements

https://jira.hl7.org/browse/FHIR-37356

​

  • definition[x]
    • definitionReference
    • definitionCanonical
    • definitionCodeableConcept
    • definitionExpression
    • definitionId
  • defByTypeAndValue
  • defByCombination
  • definitionReference
  • definitionCanonical
  • definitionCodeableConcept
  • definitionExpression
  • definitionId
  • definitionByTypeAndValue
  • definitionByCombination

​

26

27 of 49

Improve the documentation for EvidenceVariable

  • https://jira.hl7.org/browse/FHIR-36800
  • No proposed disposition yet
  • Mondays 1-2 pm Eastern is now COKA Eligibility Criteria Working Group

​

27

28 of 49

Scientific Evidence Code System (SEVCO)

© 2019 Health Level Seven ® International. Licensed under Creative Commons Attribution 4.0 International HL7, Health Level Seven, FHIR and the FHIR flame logo are registered trademarks of Health Level Seven International. Reg. U.S. TM Office.

28

29 of 49

30 of 49

31 of 49

32 of 49

33 of 49

FEvIR Platform: Terminology Tools You Can Use

Scientific Evidence Code System

​

Progress-to-date as of May 13, 2022.

​

# Terms

# Approved

Study Design

61

54

Risk of Bias

260

110

Statistics

235

39

SEVCO (total)

561

203

36%

​

(was 22% in January 2022)

34 of 49

EBM Decision Map Profile (EBMonFHIR IG)

© 2019 Health Level Seven ® International. Licensed under Creative Commons Attribution 4.0 International HL7, Health Level Seven, FHIR and the FHIR flame logo are registered trademarks of Health Level Seven International. Reg. U.S. TM Office.

34

35 of 49

ClinicalPracticeGuideline Profile of ImplementationGuide

  • Complete report and mapping of a guideline
  • ref RecommendationPlan Profile of PlanDefinition
  • ref GuidelineCitation Profile of Citation

35

36 of 49

RecommendationPlan Profile of PlanDefinition

  • Description of a recommended intervention in a guideline
  • ref RecommendationAction Profile of ActivityDefinition
  • ref RecommendationEligibilityCriteria Profile of EvidenceVariable
  • ref RecommendationJustification Profile of ArtifactAssessment
  • ref RecommendationCitation Profile of Citation

​

36

37 of 49

RecommendationJustification Profile of ArtifactAssessment

  • Evidence, justification, and ratings of evidence-to-decision framework judgments for a recommendation
  • ref EvidenceReport Profile of Composition
  • ref NetEffect Profile of EvidenceVariable
  • ref NetEffectEstimate Profile of Evidence
  • ref CertaintyOfEvidence Profile of ArtifactAssessment

​

37

38 of 49

EvidenceReport Profile of Composition

  • ref StudyCitation Profile of Citation
  • ref StudyOutcomeEvidence Profile of Evidence
  • ref OutcomeEvidenceSynthesis Profile of Evidence
  • ref CertaintyOfEvidence Profile of ArtifactAssessment

​

​

38

39 of 49

OutcomeEvidenceSynthesis Profile of Evidence

  • Evidence (statistics) generated from a meta-analysis or systematic review for a single clinical question (PICO)
  • ref StudyCitation Profile of Citation
  • ref StudyGroupGroup Profile of Groupref StudyEligibilityCriteria Profile of EvidenceVariable
  • ref EvidenceDataset Profile of EvidenceVariable
  • ref CertaintyOfEvidence Profile of ArtifactAssessment

​

​

​

39

40 of 49

EvidenceDataset Profile of EvidenceVariable

  • ref StudyOutcomeEvidence Profile of Evidence

​

​

40

41 of 49

StudyOutcomeEvidence Profile of Evidence

  • Evidence (statistics) generated from a single study for a single clinical question (PICO)
  • ref StudyCitation Profile of Citation
  • ref StudyGroup Profile of Group
  • ref StudyEligibilityCriteria Profile of EvidenceVariable
  • ref InterventionDefinition Profile of EvidenceVariable
  • ref OutcomeDefinition Profile of EvidenceVariable
  • ref RiskOfBias Profile of ArtifactAssessment

​

​

​

​

41

42 of 49

Putting It All Together

  • 9 Resources profiled
  • 21 Profiles
  • Thursdays 10-11 am Eastern is now
    • COKA EBM Decision Map Profile Working Group
    • (a CDS EBM sub-WG)

​

​

​

​

42

43 of 49

ArtifactAssessment

© 2019 Health Level Seven ® International. Licensed under Creative Commons Attribution 4.0 International HL7, Health Level Seven, FHIR and the FHIR flame logo are registered trademarks of Health Level Seven International. Reg. U.S. TM Office.

43

44 of 49

44

45 of 49

Actual Uses of ArtifactAssesment include

  • Comments on SEVCO terms
  • Votes on SEVCO terms
  • Risk of Bias Assessments (RoBAT)
  • Recommendation Justification
  • MeSH Headings and Keywords (with qualifier codings)
  • Third-party classifications for Citations
  • Examples:
    • Comment, Rating Or Classifier (CROC) Library https://fevir.net/resources/Project/46046

© 2019 Health Level Seven ® International. Licensed under Creative Commons Attribution 4.0 International HL7, Health Level Seven, FHIR and the FHIR flame logo are registered trademarks of Health Level Seven International. Reg. U.S. TM Office.

45

46 of 49

COVID-19 KNOWLEDGE ACCELERATOR INTRODUCTION

© 2019 Health Level Seven ® International. Licensed under Creative Commons Attribution 4.0 International HL7, Health Level Seven, FHIR and the FHIR flame logo are registered trademarks of Health Level Seven International. Reg. U.S. TM Office.

46

47 of 49

COVID-19 Knowledge Accelerator (COKA)

  • A virtual organization – no cost, no contracts
  • Open, transparent – all meetings
  • Engages multiple global collaborative groups, organizations, and individuals
  • A way for any to volunteer to accelerate development and implementation of standards for evidence exchange, especially with attention to COVID-19

​

© 2019 Health Level Seven ® International. Licensed under Creative Commons Attribution 4.0 International HL7, Health Level Seven, FHIR and the FHIR flame logo are registered trademarks of Health Level Seven International. Reg. U.S. TM Office.

47

48 of 49

COKA has 11 active working groups

  • Project Management
  • Eligibility Criteria WG
  • Research Design WG
  • Statistic Standard and Terminology WG
  • Risk of Bias Terminology and Tooling WG
  • Common Metadata Framework WG
  • Computable EBM Tools Development WG
  • Usability Research WG
  • EBM Decision Map Profile WG
  • Knowledge Ecosystem Liaison WG
  • Communications WG CDS EBM sub-WG

© 2019 Health Level Seven ® International. Licensed under Creative Commons Attribution 4.0 International HL7, Health Level Seven, FHIR and the FHIR flame logo are registered trademarks of Health Level Seven International. Reg. U.S. TM Office.

48

49 of 49

49