MCBK Trust and Policy WG���Co-Chairs:�Josh Richardson, PhD, MS, MLIS, FAMIA�Tony Solomonides, PhD, MSc, FAMIA��February 29, 2024

Trust and Policy Working Group Charter

• Understand the current CBK landscape as it relates to trust, governance of CBK, and policies for CBK.
• Identify roles and opportunities for the MCBK community in promoting transparency and trust.
• Define attributes of and potentially develop prototypes for CBK “product information labels” that would promote transparency of trust.
• Develop model governance structures.
• Make recommendations for measuring and evaluating trust and transparency of CBK artifacts, implementation, and evaluation.�

2

2023 MCBK T&P WG Activities

3

Considerations when Representing Trust in CBK

Jamie McCusker

Director of Data Operations at Rensselaer Polytechnic Institute and co-chair of the MCBK Technical Standards and Infrastructure Workgroup

​

Josh Richardson

Research Scientist at RTI International and co-chair of the MCBK Trust and Policy Workgroup

​

February 15, 2023

Purpose

• To present the outcomes of MCBK Technical Standards & Infrastructure discussions and articulate the principles necessary to support a technical infrastructure for applying CBK at scale, and with it, the larger learning cycle infrastructure.
• To identify infrastructure characteristics that can lead to the dissemination of trustworthy, open knowledge objects within an LHS.
• To present the FAIR + T(rust) O(penness) framework

Questions We Discussed

• What are any CBK-MM governance needs that the T&P should tackle?
• Charter
• SOP
• RACI
• What would help the CBK-MM effort achieve its goals while also not being (too) interruptive?
• What specifically from W3 governance should CBK-MM reference or mirror?
• What’s feasible given the volunteer basis of the T&P?
• Other?

Presented AMIA 2023 Poster

• Concept maps of an ontology to arrive at an interchange language for CBK metadata
• Minimum information to structure, describe, and access CBK resources
• Targeted key constituencies from knowledge creators to knowledge publishers and managers, to knowledge end users

How do CBK, CDS, and GenAI relate?

8

create

apply

monitor

• The evidence-based medicine (EBM) movement was a reaction to the weaknesses of “expert opinion”.

​

• It’s more important to see what the data point to than to sift through mutually contradictory expert assertions.

​

• Did we lose anything in this shift in focus?

​

• Why is trust such a major issue in AI?

�

Why develop a CBK “nutrition label”

• Increase transparency of how knowledge is/was created and maintained
• Promote trust among multiple stakeholders: developers, implementers, end users, etc.
• Enable effective regulation of knowledge
• Better protect intellectual property

​

Forthcoming mandate for a “nutrition label”?

https://fedscoop.com/hhs-health-it-division-carving-out-artificial-intelligence-niche/

​

Previous Work on Dataset “Nutrition Label”

https://datanutrition.org/labels/v3/?id=ee3d5314-e1b5-4fc2-a370-ae511a2696ca

​

Sendak et al.’s “Model Facts Label” for AI/ML

https://www.nature.com/articles/s41746-020-0253-3

​

Dr. Robert Reid Webinar Presentation

Met with MCBK T&P UK (December 2023)

Events:

• Launch meeting October 2019
• Participation in MCBK sessions at MIE May 2023, MedInfo July 2023, MCBK NA events & UK national meetings, bimonthly NHSE-NICE “Maximising use of evidence and knowledge” discussions
• First ever global knowledge collaborathon (“hackday”) with Health Data Research UK, 2021
• Two computable guideline hackdays with NICE, 2022 & 2023
• Workshop on regulation of knowledge objects, London, February 2023
• Annual conference 2023
• Workshop on eForms and GP templates at FCI conference, Manchester Nov ‘23

​

​

Publications:

• “Mobilising computable biomedical knowledge: UK perspectives” – 7-article special issue of BMJ Health & Care Informatics, 2020
• PS Blog posts about Computable Knowledge & LHS on HDRUK website
• Decision Support Policy White Paper for British Computer Society (27 press mentions)
• Articles in LHS Journal Oct. 2023 MCBK special issue on regulation and NICE project

​

https://docs.google.com/presentation/d/1pl-C6WEU0dbXGnuLoZsNkvvORCo6ezAduH_RYX3aceU/edit?usp=sharing

Accomplishments

• Understand the current CBK landscape as it relates to trust, governance of CBK, and policies for CBK.
• Reid presentation
• Identify roles and opportunities for the MCBK community in promoting transparency and trust.
• Partnering with the CBK-MM initiative
• Define attributes of and potentially develop prototypes for CBK “product information labels” that would promote transparency of trust.
• Discussed characteristics of CBK label design
• Develop model governance structures.
• Partnering with the CBK-MM initiative
• Make recommendations for measuring and evaluating trust and transparency of CBK artifacts, implementation, and evaluation.
• TBD

MCBK Trust & Policy WG Meetings

• All are welcome to join!
• Meets 1-hour monthly
• 3^rd Wednesday of each month
• 12-1pm EST
• We value ideas around discussion, presentation, and joint efforts
• Co-chairs
• Josh Richardson: jrichardson@rti.org
• Tony Solomonides: tony.solomonides@gmail.com

18