Code System Development for Computable Expression of �Scientific Knowledge

WCO-2020

Authors: Brian S. Alper, Joanne Dehnbostel,

Vignesh Subbian, Muhammad Afzal

COVID-19 Knowledge Accelerator (COKA)

Acknowledgement: Bhagvan Kommadi, Khalid Shahin

​

Who sets the standards?

Health Level Seven International (HL7®) is a standards development organization that maintains standards and solutions for global health data interoperability.

Who sets the standards?

Health Level Seven International (HL7®) is a standards development organization that maintains standards and solutions for global health data interoperability.

Fast Healthcare Interoperability Resources (FHIR®) is an HL7 standard

​

Who sets the standards?

Health Level Seven International (HL7®) is a standards development organization that maintains standards and solutions for global health data interoperability.

Fast Healthcare Interoperability Resources (FHIR®) is an HL7 standard

​

EBMonFHIR (Evidence Based Medicine on FHIR) is an HL7 project to extend FHIR to research findings and guidance.

Who sets the standards?

Health Level Seven International (HL7®) is a standards development organization that maintains standards and solutions for global health data interoperability.

Fast Healthcare Interoperability Resources (FHIR®) is an HL7 standard

​

EBMonFHIR is an HL7 project to extend FHIR to research findings and guidance.

COVID-19 Knowledge Accelerator (COKA) is an open virtual organization developing standards for machine-interpretable expression of scientific evidence. COKA is a massive acceleration of an EBMonFHIR project to extend the Fast Healthcare Interoperability Resources (FHIR) standard to the expression of scientific evidence (variable definitions, statistics, and certainty of findings). COKA is facilitating implementation of evidence-related FHIR resources across efforts to identify, evaluate and report COVID-19 evidence.

COKA

Code Systems Needed

• COKA has 10 active working groups developing 6 FHIR Resources for evidence expression.
• More than 30 code systems needed where there is not a current scaled fit-for-purpose system.
• The Code Systems will enable researchers and other information specialists to create, search, interpret, report, display and disseminate concepts more efficiently across and between systems.
• We selected FOUR code systems for initial development.

​

​

​

​

​

Alper BS, Dehnbostel J, Robinson K, Subbian V, Afzal M, Soares A, Kunnamo I, Shahin K, Lehmann H, Fedorowicz Z, McClure R, Thabane L. For the COVID-19 Knowledge Accelerator (COKA) Initiative. Code System Development Protocol. Created September 1, 2020. Last revised September 17, 2020. Registered at: https://osf.io/3akjv. Available at: https://tinyurl.com/CodeSystemDevelopmentProtocol

​

Statistic Type

Statistic Model

Study Design

Risk of Bias

1

2

3a

4

3b

5

6

Agreement on concept properties

Yes

7~9

No

Most concepts from single source?

10

11

12~13

Yes

Process for Code System Development�Iterate throughout all steps as needed

No

Step 1. Assemble an expert working group

​

​

57 diverse participants from

26 countries across

6 continents

Statistic Type

​

​

Step 2. Identify the Tools

• StatisticType code system defined by the FHIR project
• StatisticAttributeEstimateType code system defined by the FHIR project
• ObservationMethodAggregate value set from HL7 V3 ObservationMethod code system
• Cochrane Review Manager (RevMan)

• StatisticModelCode code system defined by the FHIR project
• StatisticModelMethod code system defined by the FHIR project

Statistic Model

Risk of Bias

S

​

• StatisticCertaintySubcomponentType code system defined by the FHIR project
• StatisticCertaintySubcomponentRating code system defined by the FHIR project
• Cochrane Collaboration’s Risk of Bias tool (ROB-1)
• Revised Risk of Bias Tool (ROB-2)
• Risk Of Bias In Non-randomised Studies - of Interventions (ROBINS-I)
• Newcastle-Ottawa Scale for non-randomized studies
• Risk of Bias in Systematic Reviews (ROBIS) … also QUIPS, QUADAS-2, PROBAST, and more…

​

​

• StudyType code system defined by the FHIR project
• ResearchStudyPhase code system defined by the FHIR project
• MEDLINE MeSH Headings for Study Characteristics [Publication Type]
• ClinicalTrials.gov study type classifiers
• ResearchStudy-StudyDesign code system used in the database of Genotypes and Phenotypes (dbGaP)

​

Study Design

Step 3. Draft Terms

Number of Terms Drafted

Code System Categorical Organization

Statistic Type Code System

Univariate

Ratios

Difference

Correlation

Dispersion

Statistical Distribution

Statistic Model Code System

Determination of Relationship

Adjustment of Variables

Pooling

Variance

Study Design Code System

Exposure Introduction

Assignment

Context

Comparator Design Definition

Data Collection Timing

Analysis Approach

Study Goal

Clinical Research Regulatory Subsets

Risk of Bias Code System

Participant Selection

Comparator Selection

Performance

Detection

Analysis

Reporting

Attrition

Synthesis

Study Selection

and more….

Step 4. Search for Ontologies

• 25 ontologies identified so far.
• 23 ontologies are freely available without restrictions.

Step 5. Map Ontologies

Term Identification

Search for a term

Term not found

2a

1

Term found

Enter NOT FOUND

Enter preferred term

Enter alternative terms

Enter term definition

a

b

c

2b

1

2

3a

4

3b

5

6

Agreement on concept properties

Yes

7~9

No

Most concepts from single source?

10

11

12~13

Yes

Process for Code System Development�Iterate throughout all steps as needed

No

Join an Expert Working Group for one of these Code System Development efforts: https://tinyurl.com/InviteToEWG

​

Join the COVID-19 Knowledge Accelerator: https://tinyurl.com/coka2020

​

Questions? Email balper@computablepublishing.com

​