Exploring Hidden Markov Models in Human Functional Magnetic Resonance Imaging Data With Applications to the Locus Coeruleus Circuit

Committee Members:

Dr. Xiaoping Hu, Co-Chairperson

Dr. Megan A.K. Peters, Co-Chairperson

Dr. Aaron Seitz

​

August 19^th, 2021

Sana Hussain

Dissertation Defense Presentation

Department of Bioengineering

​

Overview

Overview

LC Circuit

Hidden Markov Models

1

Project Goal 1: Characterize different hidden Markov models subtypes to help make informed decisions about which one should be used in future investigations.

​

Project Goal 2: Apply a hidden Markov model to a functional magnetic resonance imaging dataset focusing on the locus coeruleus to examine its relationship with attention.

Introduction to Hidden Markov Models

Overview

LC Circuit

Hidden Markov Models

2

• Brain states are patterns of activation levels or connectivity strengths between networks in the brain.
• Hidden Markov models (HMMs) can identify latent brain states from functional magnetic resonance imaging (fMRI) data.

​

HMM

• Forward Algorithm
• Viterbi Algorithm
• Baum-Welch Algorithm

Input

Output

• Mean State Patterns
• Activation or Connectivity
• Covariance Matrices
• Viterbi Path
• Transition Probability Matrix

Goal: Characterize different HMM subtypes to help make informed decisions in future investigations

Focus on 3 HMM subtypes

ROI

Can be a variety of measures

Blood Oxygen Level Dependent

Subtype 1: Activation-Based HMM (AB HMM)

Overview

LC Circuit

Hidden Markov Models

3

……………………………………

Step 1: Extract BOLD Signal From ROIs

X TRs per

network

Step 2: Concatenate Across Subjects

Step 3: Input Into HMM & Obtain Outputs

TIME

BOLD

Repeat for All ROIs

…..

ROIs

(X)*(Subjects)

• Mean State Patterns

​

​

• Covariance Matrices

​

​

• Viterbi Path

​

​

• Transition Probability Matrix

# ROIs

# states

# states

# states

# ROIs

# ROIs

X # states

Subtype 2: Summed Functional Connectivity HMM (SFC HMM)

Overview

LC Circuit

Hidden Markov Models

4

Step 1: Sliding Window Analysis 🡪 Pearson Correlate All ROIs Using Time Window Δt & Move Over 1 Time Point

Calhoun 2014

Δt

Subtype 2: Summed Functional Connectivity HMM (SFC HMM)

Overview

LC Circuit

Hidden Markov Models

4

Step 1: Sliding Window Analysis 🡪 Pearson Correlate All ROIs Using Time Window Δt & Move Over 1 Time Point

ROIs

ROIs

……………………………………

Step 2: Sum Each ROI x ROI Matrix Over 1 Dimension To Obtain Node-Wise Connectivity Vector

……………………………………….

ROI

Step 3: Concatenate Across Subjects and Input Into HMM

X – Δt Time Windows

X – Δt Time Windows

Step 4: Obtain Connectivity State Patterns

Calhoun 2014; Ou et al. 2014

Subtype 3: Full Functional Connectivity HMM (FFC HMM)

Overview

LC Circuit

Hidden Markov Models

5

Step 1: Sliding Window Analysis 🡪 Pearson Correlate All ROIs Using Time Window Δt & Move Over 1 Time Point

ROIs

ROIs

……………………………………

X – Δt Time Windows

Calhoun 2014

Step 2: Acquire All Values From Lower (or Upper) Triangle

………………………………….

X – Δt Time Windows

• Mean State Patterns

​

​

• Covariance Matrices

​

​

# states

# states

# states

X # states

• Viterbi Path

​

​

• Transition Probability Matrix

Step 3: Concatenate Across Subjects and Input Into HMM

Reshape into ROI x ROI Matrix

ROIs

ROIs

Viterbi Averaging Connectivity States

Overview

LC Circuit

Hidden Markov Models

6

• AB HMM and FFC HMM directly output states but SFC HMM requires manual calculation.
• Employ method of SFC HMM state acquisition (averaging according to Viterbi path) for all state types.
• Examine only connectivity states but can draw the same conclusions when examining activation states.

S1

S1

S3

S3

S5

S4

Viterbi Averaging Connectivity States (cont.)

Overview

LC Circuit

Hidden Markov Models

6

Extract and Average ROI x ROI Matrices Corresponding to AB HMM Viterbi Path

……………………………………

Average

ROIs

ROIs

……………………………………

S1

S4

S3

S5

S5

S2

Goal: Determine similarity of “Viterbi Averaged” states with each subtype’s direct output

S1

Average

ROIs

ROIs

S2

Average

ROIs

ROIs

S3

Average

ROIs

ROIs

S5

Average

ROIs

ROIs

S4

AB HMM Viterbi Path

From Sliding Window Correlation

Overarching Questions and Dataset

Overview

LC Circuit

Hidden Markov Models

7

Overarching Questions

1. What kinds of questions does each model type answer?
2. When is it useful to employ each model type?

fMRI Dataset and Networks

• Human Connectome Project (HCP) Unrelated 100
• 99 subjects
• 14.4 minutes of resting state

​

• Networks
• Default Mode Network (DMN)
• Fronto-Parietal Control Network (FPCN)
• Dorsal Attention Network (DAN)
• Salience Network (SN)

Van Essen et. al 2013

Deshpande et. al 2011

Raichle 2011

9 ROIs

7 ROIs

7 ROIs

9 ROIs

Total = 29 ROIs

Model Order = Number of latent HMM states that used in an investigation, i.e., model order 4 means 4 states were used.

Brain states: combinations of activation levels or connectivity strengths between networks’ ROIs

RAICAR-Based Method

Overview

LC Circuit

Hidden Markov Models

8

Yang et al. 2010

Chen et al. 2016

Ranking and Averaging Independent Component Analysis by Reproducibility

Run HMM for a specified model order x3 with different initializations

Match states across runs via Pearson correlations

Pearson correlate all pairs of states within a group and average

Sort averaged correlations from largest to smallest

Plot sorted correlations against model order

Repeat for model orders 3 – 15

Run 1

Run 2

Run 3

R² = X₁

R² = X₂

R² = X₃

avg = Y₁

1

# states

stability

Ex. Model Order = 3

Model Order = Number of latent HMM states that used in an investigation, i.e., model order 4 means 4 states were used.

Y₁

Y₂

Y₃

Model Order Determination

Overview

LC Circuit

Hidden Markov Models

9

8 States for AB HMM

8 States for SFC HMM

For better comparison with SFC HMM, assign FFC HMM to have 8 states.

AB HMM

SFC HMM

threshold = 0.9

Connectivity State Patterns

Overview

LC Circuit

Hidden Markov Models

10

ROI

ROI

ROI

ROI

ROI

ROI

ROI

ROI

ROI

ROI

S1_FFC

S2_FFC

S3_FFC

S4_FFC

S5_FFC

S6_FFC

S7_FFC

S8_FFC

S1_SFC

S2_SFC

S3_SFC

S4_SFC

S5_SFC

S6_SFC

S7_SFC

S8_SFC

-0.3

0.3

-0.2

-0.1

0

0.1

0.2

-0.3

0.3

-0.2

-0.1

0

0.1

0.2

Connectivity State Patterns

Overview

LC Circuit

Hidden Markov Models

11

AB HMM Covariance Matrices

AB HMM Connectivity Matrices

Convert Covariances to Pearson Correlations

S1_SFC

S2_SFC

S3_SFC

S4_SFC

S5_SFC

S6_SFC

S7_SFC

S8_SFC

S1_FFC

S2_FFC

S3_FFC

S4_FFC

S5_FFC

S6_FFC

S7_FFC

S8_FFC

AB Viterbi-Based S1

AB VB S2

AB VB S3

AB VB S4

AB VB S5

AB VB S6

AB VB S7

AB VB S8

Viterbi Paths

Overview

LC Circuit

Hidden Markov Models

12

1

8

2

3

4

5

6

7

29 ROIs

ROIs

ROIs

ROIs

ROIs

29

• Windowed analysis 🡪 SFC & FFC HMMs have Δt fewer data points than AB HMM
• Pearson correlation 🡪 Data is scaled to be -1 ≤ R² ≤ 1
• Sliding window moved over 1 time point 🡪 Autocorrelation

Conclusions About HMM Comparisons

Overview

LC Circuit

Hidden Markov Models

13

• AB HMM, SFC HMM, and FFC HMM provide different information about an fMRI dataset
• Choosing which one to use in an investigation depends on the topic being studied

Activation-Based (AB) HMM

• Favorable temporal resolution
• Examine fMRI temporal dynamics
• Did not produce similar connectivity patterns as SFC or FFC HMM
• Each HMM subtype is distinct in identifying spatial patterns for their inputs
• S1_AB ↔ Covariance-Based Connectivity S1_AB
• Examine activation and connectivity states in conjunction

Summed Functional Connectivity (SFC) HMM

• Recognizes changes in general connectedness between ROIs
• Outputs different state patterns than FFC HMM
• Reduced smoothing compared to FFC HMM
• Better to examine connectivity temporal dynamics

Full Functional Connectivity (FFC) HMM

• Recognizes changes in specific connections between ROIs
• Smoothing could have occurred because of model order choice or number of R² values fit

Overview

Overview

LC Circuit

Hidden Markov Models

Project Goal 1: Characterize different hidden Markov models subtypes to help make informed decisions about which one should be used in future investigations.

​

Project Goal 2: Apply a hidden Markov model to a functional magnetic resonance imaging dataset focusing on the locus coeruleus to examine its relationship with attention.

Locus Coeruleus (LC)

Overview

LC Circuit

Hidden Markov Models

14

• Involved in stress, the sleep-wake cycle, and attention
• Compromised LC is associated with aging, Parkinson’s disease, and Alzheimer's disease
• Employ HMM to examine changes in latent brain states behavior before and after LC activity up-regulation
• Use a squeezing task to noninvasively up-regulate LC activity

attention

Experimental Paradigm

Overview

LC Circuit

Hidden Markov Models

15

PostAr

N = 30 Subjects

2 Sessions/Conditions:

• Active 🡪 Squeeze a squeeze-ball at maximum grip strength
• Sham 🡪 Refrain from squeezing

Networks

• Default Mode Network (DMN)
• Fronto-Parietal Control Network (FPCN)
• Dorsal Attention Network (DAN)
• Salience Network (SN)
• Locus Coeruleus (LC)

9 ROIs

7 ROIs

7 ROIs

9 ROIs

Total = 31 ROIs

2 ROIs

Road Map

Overview

LC Circuit

Hidden Markov Models

RM

Fractional Occupancy

• Proportion of time spent in a state
• Examine during active and sham
• Correlate with LC MTC

Transition Probability

• Probability of transitioning btw states
• Examine during active and sham
• Correlate with LC MTC

Pupil Dilation

• Explore in relation to state transitions
• Examine during active and sham
• Correlate with LC MTC

Pupil dilation is a proxy measure of LC activity

LC Magnetization Transfer Contrast (MTC) quantifies LC structure

Determine Model Type

Determine Model Order

Interpret States

Measure temporal dynamics

Compare measures across 2 conditions

AB HMM

Model Order Determination

Overview

LC Circuit

Hidden Markov Models

16

Choose 5 States

State Patterns and Viterbi Paths

Overview

LC Circuit

Hidden Markov Models

17

1

2

3

4

5

S1

DMN-dominant

S2

S3

S4

S5

ATT-dominant

Whole Brain Activation

Whole Brain Deactivation

Squeeze/Arousal

DMN

FPCN

DAN

SN

LC

Road Map

Overview

LC Circuit

Hidden Markov Models

RM

Fractional Occupancy

• Proportion of time spent in a state
• Examine during active and sham
• Correlate with LC MTC

Transition Probability

• Probability of transitioning btw states
• Examine during active and sham
• Correlate with LC MTC

Pupil Dilation

• Explore in relation to state transitions
• Examine during active and sham
• Correlate with LC MTC

Pupil dilation is a proxy measure of LC activity

LC magnetization transfer contrast (MTC) quantifies LC structure

Determine Model Type

Determine Model Order

Interpret States

AB HMM

5 States

S1 → DMN-dominant

S2 → ATT-dominant

S3 → Whole Brain Activation

S4 → Squeeze/Arousal

S5 → Whole Brain Deactivation

18

Active

Sham

State

Fractional Occupancy

Overview

LC Circuit

Hidden Markov Models

2 (condition) x 2 (block) x 5 (state) RM ANOVA

✔

✔

🗶

18

Fractional Occupancy (FO)

Overview

LC Circuit

Hidden Markov Models

p = 0.8065

t = -0.2472

p = 4.3502e-06

t = 5.6353

p =1.1644e-04

t = -4.4504

p = 0.0038

t = -3.1422

p = 0.2290

t = -1.2289

*

**

**

*

*

**

**

*

**

**

**

* 0.05 < p < 0.1

** p ≤ 0.05

DMN-dominant

ATT-dominant

Whole Brain Activation

Squeeze/Arousal

Whole Brain Deactivation

Magnetization Transfer Contrast quantifies LC structure

RS0

PostAr

Switching Rate

Overview

LC Circuit

Hidden Markov Models

p = 0.5453

t = 0.6120

19

Transition Probabilities

Overview

LC Circuit

Hidden Markov Models

RTPM = Relative to RS0 Transition Probability Matrix

S1 → DMN-dominant

S2 → ATT-dominant

S3 → Whole Brain Activation

​

​

S4 → Squeeze/Arousal

S5 → Whole Brain Deactivation

* 0.05 < p < 0.1

** p ≤ 0.05

**

**

*

**

20

To

To

To

TPM = Transition Probability Matrix

21

Pupil Dilation & Transition Probabilities

Overview

LC Circuit

Hidden Markov Models

subtract pupil size from these TRs

S1

S1

S1

S3

S3

S3

switch

21

Pupil Dilation & Transitions

Overview

LC Circuit

Hidden Markov Models

To

To

To

22

Conclusions About LC Project

Overview

LC Circuit

Hidden Markov Models

• Fractional Occupancy showed no difference as a function of the handgrip task
• TPMs/RTPMs showed potential differences between active and sham sessions
• Possible Reasons:
1. Handgrip task may not have elicited a strong enough LC response
• Cold pressor (Schwabe and Schächinger 2018)
• Electrical pulses (Oyarzún et al. 2012)
• Jarring sounds (Redondo et al. 2008)
2. Analyses were not sensitive enough to changes caused by handgrip task
• May require further investigation into sensitivity of these measures
• Perhaps they can capture additional dynamics when another stressor is used

​

• RTPMs showed potential evidence of norepinephrine depletion and attentional reset

​

• We were unable to establish a relationship between LC MTC and Fractional Occupancy and State Transitions
• All subjects were healthy young adults
• LC neuronal loss is reduced in younger subjects (Manaye et al. 1995, Zucca et al. 2006, Shibata et al. 2006)

​

• Only a handful of subjects survived our pupil dilation exclusion criterion
• Increase the number of subjects scanned
• Increase total scan duration

23

Acknowledgements

Overview

LC Circuit

Hidden Markov Models

Committee:

Dr. Xiaoping Hu Dr. Megan A. K. Peters Dr. Aaron Seitz

LC Project:

Mahsa Alizadeh Shalchy

Kimia Yaghoubi

Isaac Menchaca

Lab Members:

Kaiqing Chen

Zhenhai Zhang

Kaiming Li

Abby Barlow

Alex Reardon

Lebo Wang

Queenie Xu

Jason Langley

Xu (Jerry) Chen

Chelsea Evelyn

Shaida Abachi

Vanessa Ceja

Mehdi Orouji

Olenka Graham Castaneda

Supplemental: Network ROI MNI Coordinates

Overview

LC Circuit

Hidden Markov Models

Supplemental: ED-Based Method

Overview

LC Circuit

Hidden Markov Models

Euclidean Distance

Run HMM for a specified model order x3 with different initializations

Permute state orderings from two realizations

Average all acquired Euclidean distances

Plot averaged Euclidean distances against model order

Repeat for model orders 3 – 15

Repeat x100 and for all realizations

Uniquely match states across the two realizations using the smallest Euclidean distance

Realization 1

Realization 2

Realization 3

ED₁

ED₂

ED₃

model order

mean Euclidean distance

Supplemental: AB HMM Model Order

Overview

LC Circuit

Hidden Markov Models

RAICAR-Based Stability Analysis

ED-Based Stability Analysis

15 States

14 States

13 States

12 States

11 States

10 States

9 States

8 States

7 States

6 States

5 States

4 States

3 States

Stability

Stability

Model Order

Mean Euclidean Distance

15

14

13

11

12

10

9

8

7

6

5

4

3

2.5

2

1.5

0.5

1

0

8 States for AB HMM

Supplemental: SFC HMM Model Order

Overview

LC Circuit

Hidden Markov Models

15 States

14 States

13 States

12 States

11 States

10 States

9 States

8 States

7 States

6 States

5 States

4 States

3 States

Stability

Stability

Model Order

Mean Euclidean Distance

15

14

13

11

12

10

9

8

7

6

5

4

3

1

2

3

4

5

6

7

8

8 States for SFC HMM

RAICAR-Based Stability Analysis

ED-Based Stability Analysis

Supplemental: FFC HMM Model Order

Overview

LC Circuit

Hidden Markov Models

For better comparison with SFC HMM, assign FFC HMM to have 8 states

RAICAR-Based Stability Analysis

ED-Based Stability Analysis

8 States

9 States

Stability

Stability

Model Order

8

9

Mean Euclidean Distance

Supplemental: Viterbi Averaging Activation States

Overview

LC Circuit

Hidden Markov Models

AB HMM Viterbi Path

Average

S1

……………………………………

S1

S1

S4

S4

S3

S3

S3

S5

S5

S5

S2

……………………………………

Repeat for all ROIs

ROIs

Repeat for All States

# state x # ROI Matrix

TIME

BOLD

BOLD Signal Time Series

S1

S2

S3

S4

S5

Repeat Procedure for

• SFC HMM Viterbi Path
• FFC HMM Viterbi Path

Supplemental: HCP Activation State Patterns

Overview

LC Circuit

Hidden Markov Models

R² = 0.9992

Supplemental: HCP Activation State Patterns Pt. 2

Overview

LC Circuit

Hidden Markov Models

Supplemental: SFC States

Overview

LC Circuit

Hidden Markov Models

Supplemental: FFC States

Overview

LC Circuit

Hidden Markov Models

Supplemental: Miscellaneous Connectivity States

Overview

LC Circuit

Hidden Markov Models

Supplemental: ED Between Connectivity States

Overview

LC Circuit

Hidden Markov Models

Supplemental: ED Between Connectivity States Pt. 2

Overview

LC Circuit

Hidden Markov Models

Supplemental: SFC States for Different Δt

Overview

LC Circuit

Hidden Markov Models

Supplemental: SFC States for Different Δt

Overview

LC Circuit

Hidden Markov Models

Supplemental: Model Subtype TPM and FOC

Overview

LC Circuit

Hidden Markov Models

Supplemental: LC Activation State Patterns

Overview

LC Circuit

Hidden Markov Models

Supplemental: LC Model Order

Overview

LC Circuit

Hidden Markov Models

RAICAR-Based Stability Analysis

ED-Based Stability Analysis

Model Order

Stability

Stability

Mean Euclidean Distance

15 States

14 States

13 States

12 States

11 States

10 States

9 States

8 States

7 States

6 States

5 States

4 States

3 States

4

3.5

3

2

1

0

2.5

1.5

0.5

3

15

14

13

12

11

10

9

8

7

6

5

4

Choose 5 States

Supplemental: Fano Factor

Overview

LC Circuit

Hidden Markov Models

Supplemental: Fano Factor Pt. 2

Overview

LC Circuit

Hidden Markov Models

Supplemental: Average Duration

Overview

LC Circuit

Hidden Markov Models

*

Active

Sham

*

*

2 (condition) x 2 (block) x 5 (state) RM ANOVA

✔

✔

🗶

* = 0.05 < p < 0.1

** = p ≤ 0.05

Supplemental: Average Duration Pt. 2

Overview

LC Circuit

Hidden Markov Models

p = 0.0472

t = 2.0756

p = 0.5684

t = -0.5772

p = 0.0024

t = -3.3320

p = 0.0110

t = -2.7234

p = 0.9657

t = 0.0434

*

**

*

**

**

**

**

**

**

*

**

* = 0.05 < p < 0.1

** = p ≤ 0.05

S1 → DMN-dominant

S2 → ATT-dominant

S3 → Whole Brain Activation

S4 → Squeeze/Arousal

S5 → Whole Brain Deactivation

Supplemental: Pupil Dilation Switching Rate

Overview

LC Circuit

Hidden Markov Models

subtract pupil size from these TRs

S1

S1

S1

S3

S3

S3

switch

p = 0.4569

t = -0.7553

Supplemental: RTPM and LC MTC

Overview

LC Circuit

Hidden Markov Models

† = 0.05 < p < 0.1

* = p ≤ 0.05