Generalist medical AI

Towards interstellar health

Michael Moor, MD PhD

NASA SMD AI Workshop, Huntsville AL

March 25, 2024

@Michael_D_Moor

​

Mission:

2

Develop medical

AI systems that are

​

​

​

​

​

​

and enable:

Early diagnosis

Personalized

therapies

• flexible
• reusable
• knowledgeable
• reliable

3

No domain knowledge

Fixed data modality

Adaptability

Reasoning with medical knowledge:

Flexible multimodality

Narrow applicability

Challenges:

We need generalist medical AI!

4

Generalist medical AI

Moor et al., Nature (2023)

GMAI = Medical FM + the 3 defining capabilities:

i) adaptability, ii) reasoning w/ med. knowledge, iii) flex. MM

5

How could GMAI look like?

Bedside decision support:

Moor et al., Nature (2023)

Interactive radiology reports:

Augmented procedures:

Roadmap

Unlocking generalist capabilities:

6

🚀 GMAI in space?

Causality: predicting personalized response to unseen drugs

Multimodality: flexibly integrating modalities in medical AI

NeurIPS 2023, Spotlight (~3%)

7

Cure the disease

​

​

​

​

No effect

​

​

​

​

​

Harm the patient

Pre-treatment

Covariates

(X)

Treatment

(W)

Outcome

(Y)

Motivation

8

We train a single model across thousands of drugs & combinations across 4.5 billion health claims from ~30 million patients.

CaML: causal meta-learning

[1] Chandak, Payal, Kexin Huang, and Marinka Zitnik. "Building a knowledge graph to enable precision medicine." Scientific Data 10.1 (2023): 67.

Prior knowledge from a biomedical knowledge graph [1]

Part I: Take-aways & Outlook

• We introduced a challenging problem: to zero-shot predict personalized effects of new treatments.

​

• Our method, CaML unlocks the capability to estimate these personalized treatment effects zero-shot and shows compelling performance on massive health claims data.

​

• Preliminary outlook: we find that GPT-style causal foundation models lead to drastic improvements.

​

​

​

​

​

9

Roadmap

Unlocking generalist capabilities:

10

Causality: predicting personalized response to unseen drugs

Multimodality: flexibly integrating modalities in medical AI

🚀 GMAI in space?

Background: doing multimodality the “old way”

11

“Patient fell and hit her head.”

0.95

Risk of intracranial

Bleeding =

12

​

​

​

​

ML4H 2023

Med-Flamingo

Roadmap

Unlocking generalist capabilities:

13

Causality: predicting personalized response to unseen drugs

Multimodality: flexibly integrating modalities in medical AI

🚀 GMAI in space?

14

GMAI in space?

Sample size

Compute

Monitoring modalities

Noise, Latency, Corruption

Pre-training:

Terrestrial data,

Historic mission data,

Simulation data

​

Refinement:

Multi-modal fusion,

Longitudinal data

Acknowledgements

15

The SNAP group, Stanford CS

​

​

​

​

Jure Leskovec

​

​

​

​

Karsten Borgwardt

​

Hamed Nilforoshan

Michihiro Yasunaga

Yusuf Roohani

Shirley Wu

Qian Huang

Yash Dalmia

​

​

Pranav Rajpurkar

​

​

​

​

​

Eric Topol

​

Harlan Krumholz

Oishi Banerjee

Anja Surina

Sara Oblak

Yining Chen

And more!

Thank you for your attention!

16

Appendix

17

GMAI in deep space:

• Local inference, model needs to fit into mobile computing device

​

• Large-scale training & distillation on earth to update “edge” models

​

• Continued training / learning on flight:
• compute-efficient model adaptation with rich monitoring and interventional data (mobile labs)

​

• We need more resource-efficient computing: e.g. biocomputers,

DNA storage etc.

​

18

Potential of GMAI in deep space:

• Could enable longer-term missions (every medical specialists on board with OOD generalization capabilities)

​

• Embodied GMAI agents pot. superior to (sporadic) telemedicine

​

• GMAI research agents could unlock ground-independent, interdisciplinary on-board research

​

​

​

​

​