1 of 56

BEYOND ACCURACY, �MORE THAN A MODEL:�New Trends in AI for Neuroimaging Research

Qingyu Zhao, Ph.D

qiz4006@med.cornell.edu

Weill Cornell Medicine, Cornell University

Machine Intelligence in NeuroImaging (MINI) Lab

https://mini-cornell.github.io/�

2 of 56

Weill Cornell Medical College
Cornell University (Ithaca)
Cornell Tech

Q. Zhao, qiz4006@med.cornell.edu

2

Cornell Tech

Cornell University

3 of 56

Q. Zhao, qiz4006@med.cornell.edu

3

AI Trends

Healthcare

Needs

Design AI with Purpose

4 of 56

Q. Zhao, qiz4006@med.cornell.edu

4

AI Trends

Medical Data

5 of 56

Neuroimaging in the Stone Age

Q. Zhao, qiz4006@med.cornell.edu

5

Cut open & measure

1^st “Neuroimaging” Technique (1880)

Human Circulation Balance: non-invasively measure the redistribution of blood during emotional and intellectual activity

6 of 56

Neuroimaging Nowadays

Q. Zhao, qiz4006@med.cornell.edu

6

7 of 56

Structural MRI

Machine Learning for Neuroimaging - Autumn 2024

7

wired.com

8 of 56

Functional MRI

Machine Learning for Neuroimaging – Autumn 2024 – Session 11

8

An active brain region consumes more oxygen and causes more blood flow.

. . .

Start

2 sec

12 min

time

A

B

BOLD signal at A

BOLD signal at B

time

Functional Connectome

9 of 56

Q. Zhao, qiz4006@med.cornell.edu

9

Brain-Behavior Mapping

Developmental Neuroscience

Preclinical Neuroimaging

Clinical Neuroimaging

Neuropsychiatry

Lesion Segmentation & Detection

Diagnosis / Prognosis

10 of 56

Brain mapping through neuroimaging studies:

Understand brain mechanisms underlying neurological symptoms
Objective diagnosis tools
Prevention/intervention programs
Screening / prognostic tools
Predict treatment response

Q. Zhao, qiz4006@med.cornell.edu

10

Neuroimaging

Risk Assessment

Intervention

Therapy

Neural Mechanisms

Treatment Response

Diagnosis

Brain Mapping / Brain-Wide Association Study (BWAS)

Behavior

Cognition

11 of 56

What should be the trends?

Q. Zhao, qiz4006@med.cornell.edu

11

Traditional ML

Deep Learning

Self-Supervised Learning

Hypothesis Testing

12 of 56

12

Data Processing

……

Hypothesis test

…

Measurements

Participants

Hypothesis test

…

Data Collection

Easy to interpret

Control

Patients

Brain Mapping

Population-level inference

Univariate analysis

13 of 56

13

……

…

Measurements

Participants

Machine �Learning

Model

Healthy

Diseased

Label Participant

Subject-level inference

Model

Interpretation

Data Processing

Data Collection

Brain-Based Predictive Modeling

14 of 56

Q. Zhao, qiz4006@med.cornell.edu

14

T1 MRI

.

Atlas registered to each subject

Segmentation

.

Backhausen, et al. Neuropsychology Review, 2022

Feature Extraction

Brain-Based Predictive Modeling

15 of 56

Q. Zhao, qiz4006@med.cornell.edu

15

		…

		…

		…

		…

y

1

0

1

.

0

X

.

Feature Vectors

T1 MRI

.

Atlas registered to each subject

Segmentation

.

Support Vector Machine

Random Forest

LASSO

…

Brain-Based Predictive Modeling

16 of 56

Q. Zhao, qiz4006@med.cornell.edu

16

Mean BOLD signal at region A

Mean BOLD signal at region B

Functional Connectivity

y

1

0

1

.

0

X

.

Flattening

Brain-Based Predictive Modeling

17 of 56

17

……

…

Measurements

Participants

Machine �Learning

Model

Healthy

Diseased

Label Participant

Model

Interpretation

Data Processing

Data Collection

Brain-Based Predictive Modeling

18 of 56

18

Deep�Learning

Model

Healthy

Diseased

Label Participant

Subject-level inference

Model

interpretation

Data Collection

Brain-Based Predictive Modeling

19 of 56

Q. Zhao, qiz4006@med.cornell.edu

19

Deep Convolutional Networks

20 of 56

Q. Zhao, qiz4006@med.cornell.edu

20

Deep Convolutional Networks

Esmaeilzadeh et al. End-To-End Alzheimer’s Disease Diagnosis and Biomarker Identification, MLMI 2018

MCI

Control

Alzheimer’s

21 of 56

Q. Zhao, qiz4006@med.cornell.edu

21

Vision Transformers (ViT)

Dosovitskiy, et al. ICLR 2021

22 of 56

Q. Zhao, qiz4006@med.cornell.edu

22

Vision Transformers (ViT)

Singla, et al. Multiple Instance Neuroimage Transformer, Pred. Intel. Med. 2022

23 of 56

Q. Zhao, qiz4006@med.cornell.edu

23

Image-based Convolution

Graph Convolution

Connectivity Matrix

Graph Representation

Graph Convolution for fMRI Analysis

24 of 56

Q. Zhao, qiz4006@med.cornell.edu

24

Graph Convolution for fMRI Analysis

Li et al., BrainGNN, Medical Image Analysis 2022

25 of 56

Q. Zhao, qiz4006@med.cornell.edu

25

T

4D BOLD

Spatio-Temporal Graph Convolution

time

Spatial Graph

Healthy

Patients

fMRI is 4D signal !

Spatio-Temporal Graph Convolution

Gadgil et al. Spatio-Temporal Graph Convolution for Resting-State fMRI Analysis, MICCAI 2020

26 of 56

Transformers for Functional MRI

Q. Zhao, qiz4006@med.cornell.edu

26

Bedel, et al. MedIA 2023

27 of 56

Q. Zhao, qiz4006@med.cornell.edu

27

Stanford HAI 2025 AI index report

# of ML Studies for Alcohol Use Disorder between 2015-2025

“New Trends” in AI ?

28 of 56

Beyond Accuracy, More than a Model

Q. Zhao, qiz4006@med.cornell.edu

28

HIV Patients

Health Controls

ML Grind

Best Accuracy

So What?

For many brain disorders, neuroimaging is NOT used for diagnosis and treatment settings

29 of 56

Q. Zhao, qiz4006@med.cornell.edu

29

Heterogeneity

Generalizability

Comorbidity

Multi-modality (Non-Imaging)

Can I build this?

Loss function
Accuracy
Baselines
Ablations
…

Will it matter?

Clinical relevance
Scientific relevance
Generalizability
Transparency
…

Design AI with Purpose

30 of 56

Small Sample Sizes, Limited Generalizability

NIH Primary Aim: Establish reproducibility
Big data in Neuroimaging is not “big” enough.
Traditional R01s recruit ~100 participants.

Q. Zhao, qiz4006@med.cornell.edu

30

1,500 scientists lift the lid on reproducibility

31 of 56

Pre-train model on massive unlabeled data
Using learned representations for downstream tasks

Q. Zhao, qiz4006@med.cornell.edu

31

Encoder

g

Healthy

Patients

		…

Pre-train (massive unlabeled data)

Self-Supervised Learning

Downstream classification (small labeled data)

32 of 56

Q. Zhao, qiz4006@med.cornell.edu

32

Latent Space

Self-Supervised Learning

Contrastive Learning

33 of 56

LSSL: Longitudinal Self-Supervised Learning

Longitudinal data should follow coherent aging trajectories in the latent space

Q. Zhao, qiz4006@med.cornell.edu

33

Encoder

Learned

Representation Space

g

Zhao* et al. Longitudinal Self-Supervised Learning, Medical Image Analysis, 2021

Kim, Learning-based inference of longitudinal image changes: Applications in embryo development, wound healing, and aging brain, PNAS 2025

time

Aging

34 of 56

Unbiased Brain Age Estimate

Q. Zhao, qiz4006@med.cornell.edu

34

time

62.9 65.0 71.3 75.3 82.1 86.1 92.5

Zhao et al., Longitudinal Self-Supervised Learning, Medical Image Analysis, 2021

Longitudinal Data

LSSL

Health Controls

Chronological age = 80

Brain-age Estimate

Downstream Tasks

35 of 56

He, et al. Masked Autoencoders Are Scalable Vision Learners. CVPR 2021

19

Masked Autoencoders

36 of 56

Q. Zhao, qiz4006@med.cornell.edu

36

Masked Autoencoders for Functional MRI

Milecki et al., NERVE: Network-Aware Representations of Brain Functional Connectivity via Masked AutoEncoders, Under Review

37 of 56

Q. Zhao, qiz4006@med.cornell.edu

37

Functional Connectivity Matrix

Masked Autoencoders for Functional MRI

BrainLM, Brain-JEPA, …

BrainMASS

Caro, et al. BrainLM, ICLR 2024

38 of 56

Q. Zhao, qiz4006@med.cornell.edu

38

Accuracy-Centric Evaluation

Yang, et al. BrainMass, TMI 2024

39 of 56

Q. Zhao, qiz4006@med.cornell.edu

39

100%

50%

90%

80%

70%

60%

Aggarwal et al. Diagnostic accuracy of deep learning in medical imaging: A systematic review and meta-analysis. NPJ Digit Med. 2021

Alzheimer’s

Psychosis

Substance use disorder

ADHD

Prediction Accuracy

Random guessing

Neuroimaging-based Prediction in Mental Health

40 of 56

Q. Zhao, qiz4006@med.cornell.edu

40

Neuroimaging-based Prediction in Mental Health

Fluid Intelligence Regression

Classification Accuracy

Varoquaux, Cross-validation failure: Small sample sizes lead to large error bars

Marek et al., Nature 2022

41 of 56

Population Heterogeneity

Q. Zhao, qiz4006@med.cornell.edu

41

Marquand et al. Understanding Heterogeneity in Clinical Cohorts Using Normative Models: Beyond Case-Control Studies, Bio. Psyc. 2016

42 of 56

Population Heterogeneity

Small studies tend to be homogeneous;

In larger studies, external factors can influence brain-behavior mapping

Sex differences
Community environment
Education
Socioeconomic status
Parental monitoring
…

Q. Zhao, qiz4006@med.cornell.edu

42

Brain

Behavior

Confounder

Moderator

43 of 56

Unbiased, Fair, Invariant, Confounder-Free Learning

Q. Zhao, qiz4006@med.cornell.edu

43

Input

Output

Association

Diagnosis

Confounder

Confounders affect the relationship between input and output variables (e.g., diagnosis).
Improper modeling of those relationships often results in spurious and biased associations.
Widely studied in traditional statistical analysis but overlooked in deep learning applications

Zhao*, Adeli* et al. Nature Communications 2020

44 of 56

Q. Zhao, qiz4006@med.cornell.edu

44

Adversarial learning of features F that are

Predictive of target of interest
Unpredictive of confounder / bias variables

Brain

Diagnosis

Confounder

Remove direct link

Zhao*, Adeli* et al. Nature Communications 2020

Shah et al, Confounder-Free Continual Learning via Recursive Feature Normalization, ICML 2025

Unbiased, Fair, Invariant Confounder-Free Learning

45 of 56

Failure of the “one-size-fits-all” paradigm

Q. Zhao, qiz4006@med.cornell.edu

45

Universal Model ?

Greene et al., Brain–phenotype models fail for individuals who defy sample stereotypes, Nature 2022

Brain

Behavior

Confounder

Moderator

46 of 56

Heterogeneous ML

Q. Zhao, qiz4006@med.cornell.edu

46

External Factors

(demographic, social, environmental, etc)

Brain

Predictive

Moderation

Behavior

Zhao et al. The Transition From Homogeneous to Heterogeneous Machine Learning in Neuropsychiatric Research, Biological GOS 2024

Population-Specific Models

Hypernetwork-based Model

47 of 56

Subtypes of Brain-Behavior Association

Q. Zhao, qiz4006@med.cornell.edu

47

Sex Difference

Salience

Sensorimotor

Cingulo-Parietal

Cortical Connectivity

Connectivity to Sub-cortical Regions

Adverse Childhood Experiences (ACE)

Basal Ganglia

Milecki et al., Heterogeneous brain functional mapping of childhood psychiatric symptoms, SfN 2024 Press Highlight

Milecki et al., Regularized CCA Identifies Sex-Specific Brain-Behavior Associations in Adolescent Psychopathology Trans. Psyc. 2025

Sensorimotor

48 of 56

Data-Driven Sample Weighting

Q. Zhao, qiz4006@med.cornell.edu

48

Paschali, et al., Spectral Graph Sample Weighting for Interpretable Sub-cohort Analysis in Predictive Models for Neuroimaging, Predict Intell Med. 2024

Sample weights reveal predictability of sub-cohorts

49 of 56

Heterogeneity - Comorbidity

Individuals diagnosed with the same psychiatric disorder can present vastly different sets of symptoms

Two patients diagnosed with Major Depressive Disorder (MDD) could share no overlapping symptoms, yet both meet diagnostic criteria.
Diagnosis Labels do not reflect underlying neurobiology: brain differences are often small, inconsistent, or non-replicable.

Multi-Organ Comorbidity

Q. Zhao, qiz4006@med.cornell.edu

49

50 of 56

Modeling Heterogeneity/Comorbidity

Better phenotyping
Individualized analysis
Multi-modal data: imaging, genetics, behavior, and environment

Q. Zhao, qiz4006@med.cornell.edu

50

Jiang, …, Zhu, Nat Comm 2025

51 of 56

Role of Neuroimaging in Multi-Modal Analysis

Neuroimaging measures tend to have

Smaller effect sizes
Colinearity with other predictors

Q. Zhao, qiz4006@med.cornell.edu

51

Green,…,Squeglia, Predictors of Substance Use Initiation by Early Adolescence, Am J Psychiatry, 2024

52 of 56

Multi-Modal Data Fusion

Q. Zhao, qiz4006@med.cornell.edu

52

�

ML Model Assumption

Brain

Genetic

Environmental

Cognitive

Behavioral

Hazardous Drinking

53 of 56

Q. Zhao, qiz4006@med.cornell.edu

53

Brain

Hazardous Drinking

Environmental, Behavioral, Cognitive

Prediction based on brain-behavior coupling

Brain

Genetic

Environmental

Cognitive

Behavioral

Hazardous Drinking

Multi-Modal Data Fusion

54 of 56

Q. Zhao, qiz4006@med.cornell.edu

54

Cognitive & Motor Measures

Functional Connectivity

Canonical Correlation of Brain and Behavior

CCA

Graph Neural Network

Wang, et al. Trans. Psyc. 2025

55 of 56

Q. Zhao, qiz4006@med.cornell.edu

55

Heterogeneity

Generalizability

Comorbidity

Multi-modality (Non-Imaging)

Can I build this?

Loss function
Accuracy
Baselines
Ablations
…

Will it matter?

Clinical relevance
Scientific relevance
Generalizability
Transparency
…

Design AI with Purpose

56 of 56

https://mini-cornell.github.io/�

56

MINI

Machine intelligence In neuroimaging

Camila González

Yixin Wang

Edith Sullivan

Adolf Pfefferbaum

Kilian Pohl

Ehsan Adeli

Amy Kuceyeski

Heejong Kim

Mert Sabuncu