Reproducibility in Human NeuroImaging: Lessons from the Human Connectome Project

​

David Van Essen and Matt Glasser

Washington University in St Louis

​

​

ReproNim Webinar Series

October 1, 2021

​

Biological Finding: MRI-based human cortical parcellations are reproducible!

PART 1 (David)

• What is HCP Style Neuroimaging (vs traditional neuroimaging) and what benefits does it provide?
• Spoiler Alert: there are MAJOR benefits in the ReproNim context!

PART 2 (Matt)

• Making a reproducible multi-modal parcellation
• Other examples of reproducible HCP-Style Neuroimaging

​

1) WU-Minn-Ox HCP (2010-2016) acquired data on brain structure, function, and connectivity in healthy adults (twins + sibs) Improved scanners, pulse sequences

• Multimodal imaging (4 hr total)
• 1100 subjects scanned at 3T; ~200 scanned at 7T; ~100 with MEG
• Extensive behavioral, demographic data; genotyping

2) and analyzed the data

• Improved HCP preprocessing pipelines
• Better alignment, atlases, and visualization (Connectome Workbench)
• Advanced analyses (connectivity/behavior; cortical parcellations)

3) and shared the data, plus methods and tools

• ConnectomeDB + BALSA databases (>27 PetaBytes shared!)
• >18,000 investigators accepted HCP Data Use Terms
• >1,500 publications citing HCP data

​

3

The HCP-style Neuroimaging Paradigm

Seven core tenets (Glasser et al. Nature Neuroscience, 2016)

• Collect lots of multimodal imaging data.
• Maximize resolution, data quality (e.g., multiband fMRI, dMRI)

3) Minimize distortion and blurring of each subject’s data

4) Respect geometry of brain structures (‘CIFTI grayordinates’).

5) Align data precisely across individuals and across studies.

6) Analyze results using an accurate brain parcellation.

• Freely share the data (including publication-related data).

Visual

Stimulation

Visual

Fixation

Difference

Image

Positron Emission Tomography: Function without Structure

Subject 1

Noisy, blurry data!

Subject 1

Subject 5

Early 1990’s: MRI reveals structure and function!

MNI 152 Average MRI

• Linear volumetric registration to a fuzzy ‘template’
• Nonlinear registration to a sharper template
• Better intersubject alignment
• Does not respect cortical surface topology
• What to do with fMRI signals?
• Noisy
• (initially) required large (~4mm) voxels?

Traditional Volume-based Analysis

• volume-registration to standard MNI brain space
• Smooth (i.e. blur) to reduce misalignments
• voxel-wise statistical analysis
• Task fMRI Analysis
• Resting State Analysis
• Structural Image Analysis (e.g. voxel-based morphometry)
• Threshold the statistical map
• Show voxels having a ‘significant’ effect
• Identify clusters of significant voxels

Data from the Sheet-like Cerebral Cortex Is More Easily Analyzed and Visualized on Surface Models

• Surface models: initially manual & tedious (Van Essen & Maunsell 1980)
• Mid-1990’s: fMRI + computerized cortical cartography (Sereno et al 1995)

• Surface-based analysis:
• Common for visual cortex
• Helps in parcellation & functional analyses
• BUT - traditional volume-based analysis remains dominant to this day!!

s

Sereno et al (1995) Science

3x3x4mm 1.5T fMRI on the surface with no smoothing

• Standard fMRI: 2.5 – 4 mm voxels; blurring across compartmental boundaries
• HCP 3T: 2.0 mm voxel size, 1 frame/0.7 s; ideally fMRI should be <2.6mm
• HCP 7T: 1.6 mm voxel size (2x smaller voxel volume), 1 frame/1.0 s

Cortical Thickness vs Image Resolution

High Resolution

Low Resolution

Glasser et al (2016) Nature Neuroscience

Blurring across folds depends on voxel size

• Blurring across cortical folds
• Cannot tell if activation is on one side or the other of a fold
• Two sides of a fold may be very different (e.g. sensory and motor cortex)
• Low resolution data still benefits from HCP-Style Analysis

Glasser et al (2013) Neuroimage

Estimated ‘magnitude of leakage’ across cortical folds

Geometric Image Distortion

• Echo Planar Images (EPI): used for fMRI, diffusion, perfusion
• EPI: stereotyped distortion patterns
• Air in sinuses, mastoids -> distortions in nearby brain
• Distortion correction:
• Field map enables accurate correction
• Critical for HCP-Style analyses

​

Surface-based Registration: Getting from One Subject’s 3D Space to Another, Accurately

1. Define 2D white and pial surfaces in 3D
2. Average white and pial to make unbiased midthickness
3. Inflate midthickness surface to a sphere
4. Deform in 2D across the sphere to match the individual to the atlas folding pattern
5. Deflate to group average midthickness surface
6. Inflate back to sphere
7. Deform across the sphere to match new individual folding pattern
8. Deflate from sphere to individual surface
9. Back to 3D of new subject
10. Poor match for direct 3D to 3D

​

Surface-based Registration Improves Cortical Spatial Localization

• Easier to align cortical areas along the 2D cortical sheet across subjects than trying to align both cortical areas and cortical folds in the 3D volume
• Easier to preserve spatial relationships (areal borders) on the surface
• e.g. area 2 is only on the anterior bank of the post central sulcus

Van Essen et al 2012

Probabilistic cytoarchitectonic areas (Zilles and Amunts group) registered on the surface by Fischl et al (2008)

fMRI activations: better for Folding-based Surface Registration than Volume-based

• Volume Registered:
• 2mm FWHM volume smoothed
• 4mm FWHM volume smoothed
• Surface Registered:
• 2mm FWHM surface smoothed
• 4mm FWHM surface smoothed
• Percent improvement in task fMRI statistical maps over 2mm volume registered
• 2mm surface smoothed better than 4mm volume smoothed

• Not a novel finding (across many modalities): Fischl et al 1999, Fischl 2008, Anticevic et al., 2008, Van Essen et al., 2012, Frost and Goebel, 2012, Tucholka et al., 2012, Smith et al 2013, etc…

CIFTI “Grayordinates”

For gray-matter analyses (e.g., fMRI):

• Cortex plus subcortical gray matter only
• Appropriate geometric models
• Cerebral cortex: surfaces, vertices
• Subcortical: volumes, voxels
• Cerebellar cortex: voxels (for now)

​

• Intersubject alignment:
• Cortex: surface registration
• Subcortical: nonlinear volume-based
• Atlas-based subcortical parcellation

​

• Spatial smoothing in grayordinates space
• No blurring outside cortical ribbon or between subcortical parcels

​

• “dense” grayordinate analyses:
• 91k x 91k grayordinates = 30 GB

(vs 90 GB full volume)

Glasser et al (2013) Neuroimage: “HCP Pipelines”

Folding-based Surface Alignment Is Often Blurry

• Best when folds are consistent along with areal boundaries
• V1 and early somatosensory, motor areas
• Most other areas have significant misalignment
• e.g. around MT+

Max Overlap 100%

Max Overlap 50%

From Fischl et al 2008

Completely Misaligned

Why Does This Occur?

• High folding variability over most of cortex
• Areal boundaries vary relative to folds
• Only a few regions are consistent across subjects

How to Accurately Align Cortical Areas

• Take Area MT as an example:
• Not a volume-based registration (max overlap 46%)
• Not a folding-based surface registration (max overlap 57%)
• An areal-feature-based surface registration (max overlap 100%)
• Areal features are things like architecture, function, connectivity, topography, or the areas themselves

MSMAll Areal Feature-based Registration: Aligning Most Cortical Areas Across Most Subjects

• MSMAll: starts with folding-only (MSMSulc)
• uses Resting State Networks, Myelin Maps, and Resting State Visuotopic Maps
• Recommended: use HCP files with ‘_MSMAll’ in their names
• MSMSulc files (lacking ‘_MSMAll’): useful for legacy analyses
• MSMSulc and MSMAll versions: available in HCP Pipelines repositories https://github.com/Washington-University/Pipelines:
• New is the publicly available code; Old was used for the Main HCP,
• Standard FreeSurfer’s functional alignment barely beats that of a rigid rotation of the spherical surface and has lots of distortion as it overfits folding patterns
• MSMSulc’s much gentler folding registration has better functional alignment with much less distortion
• MSMAll was optimized to maximize functional alignment while keeping distortion less than FreeSurfer

Cluster Mass Improvement Over FreeSurfer (FS)

Distortion

Imaging Fast (TR<1s) Helps Remove Artifacts and Noise from fMRI Data

• Functional Connectivity: correlate a seed timeseries with all other timeseries
• Vulnerable to non-random artifacts (produces bias)
• Remove artifacts from subject moving
• Remove artifacts from subject respiratory physiology
• Remove random measurement noise
• Not all artifact-reduction methods are equal!
• Objectives: reduce noise and preserve signal
• Activation studies benefit too
• If possible, acquire “Multi-band” EPI (multiple slices excited per TR)

Glasser et al (2016) Nature Neuroscience

Glasser et al (2018; 2019) Neuroimage

Reproducibility of HCP data after careful preprocessing: Study Design

• Two groups of 210 subjects (‘210P’ and ‘210V’); no families in common
• Use later for Parcellation (P) and statistical Validation (V)
• Compare three major categories of information:
• Architectural (myelin, thickness, folding)
• Functional (task fMRI contrast)
• Connectivity (resting state networks and dense connectomes)
• Compute Pearson correlation between dense spatial maps of the two groups for each measure
• Use minimally smoothed or unsmoothed data; HCP minimal preprocessing pipelines; registered with area feature-based registration; ‘dedrifted’ (to avoid cumulative bias)
• Analysis from Glasser et al (2016) Nature

Structural Reproducibility: Group Surfaces, Folding, and Architectural Measures

• 210P Group Surface + Folding
• Left Midthickness
• Left Inflated
• Left Flat
• 210V Group Surface + Folding
• Folding Quantitative Comparison
• Sulc Folding Map r=0.996
• Curvature Folding Map r=0.979
• Folding patterns blurry in many regions after MSMAll registration relative to an individual
• poor correlation between folds and areal boundaries (e.g. cognitive areas)
• Remaining sharp folding patterns: regions where folds and areas are well correlated (e.g. early sensory areas)
• Architectural Quantitative Comparison
• Myelin (r=0.998)
• Thickness (r=0.994)
• Group myelin maps have reproducible fine spatial detail
• Related to function in sensory cortex

F

F

UL

UL

Functional Reproducibility: tfMRI

• Cleaned Contrast Beta Maps Scaled from 0.75% to -0.75% BOLD from 210P for Left Inflated, Right Inflated, and Flattened Surfaces
• And 210V, most Reproducible Contrast (Relational vs Baseline, r=0.995)
• Median Reproducible Contrast (Story vs Baseline, r=0.987)
• Least reproducible contrast (Tools category – Average categories, r=0.967), excluding outlier
• Overall reproducibility of all task contrasts for surface only and all grayordinates (little bit lower)
• Again reproducible fine spatial detail

Functional Reproducibility: tfMRI

• Cleaned Contrast Beta Maps Scaled from 0.75% to -0.75% BOLD from 210P for Left Inflated, Right Inflated, and Flattened Surfaces
• And 210V, most Reproducible Contrast (Relational vs Baseline, r=0.995)
• Median Reproducible Contrast (Story vs Baseline, r=0.987)
• Least reproducible contrast (Tools category – Average categories, r=0.967), excluding outlier
• Overall reproducibility of all task contrasts for surface only and all grayordinates (little bit lower)
• Again reproducible fine spatial detail

Connectivity Reproducibility: rfMRI

• Reproducible Group Resting State Network (RSN) maps (median r=0.99)
• Language Network
• Its right hemisphere homologue
• Reproducible dense functional connectivity (dFC) maps (median r=0.99)
• Default mode network
• Visual network
• Across all RSNs and most dFC seed grayordinates reproducibility is high
• fMRI blind spot seeds are outliers

Biological Finding: MRI-based human cortical parcellations are reproducible!

PART 1 (David)

• What is HCP Style Neuroimaging (vs traditional neuroimaging) and what benefits does it provide?
• Spoiler Alert: there are MAJOR benefits in the ReproNim context!

PART 2 (Matt)

• Making a reproducible multi-modal parcellation
• Other examples of reproducible HCP-Style Neuroimaging

​

What Do We Want in a Cortical Parcellation?

• A cortical parcellation ideally should have four qualities:
1. Be based on hundreds of precisely aligned subjects, representing the typical areal arrangement of the studied population
2. Reflect complementary and converging evidence from multiple modalities across the whole cerebral cortex
3. Reflect the existing terminology used in the neuroanatomical literature
4. Be possible to automatically replicate the parcellation in individual subjects based on multi-modal areal fingerprints
• The HCP’s multi-modal cortical parcellation v1.0 has these four qualities

Glasser et al (2016) Nature Neuroscience

How Might One Parcellate the Cortex?

• Most extant parcellations were generated with only a one of these modalities
• Architecture, Function, Connectivity, and Topography
• The HCP measured all four
• Gradients (derivatives with respect to space) are an objective measure of locations where a modality is changing rapidly
• These represent potential areal boundaries
• Gradients provide a common measure to compare across modalities

?

How Might One Parcellate the Cortex?

• What makes a gradient convincing as an areal boundary?
• Agreement in spatial location of a putative boundary between two or more independent modalities
• Statistically robust and significant evidence of multi-modal differences across the boundary
• Presence in both hemispheres
• Not associated with known imaging artifact
• Prior literature evidence for the boundary
• Last, relate the spatial relationships of areal boundaries to existing parcellations in the neuroanatomical literature to identify areas or describe new ones

?

Architectonic 🡪 Myelin 🡪 Gradients

• To define cortical areal borders, look where myelin content changes
• The spatial gradient objectively shows where the transition in myelin content occurs
• The local maximum of the gradient is the most likely location of a potential areal border
• Some transitions are larger than others, but transitions that occur in multiple modalities are especially interesting as areal border candidates

Light

Heavy

Low

High

Architectonic 🡪 Thickness 🡪 Gradients

• Cortical Thickness is another modality that gives us architectural information
• Sharp transitions in cortical thickness also give us some areal boundary candidates
• Curvature is regressed out of thickness maps to reduce folding effects (thicker on gyri, thinner on sulci)

​

Low

High

Function 🡪 task fMRI 🡪 STORY vs REST 🡪 Gradients

• Positive areas have more activity during the task relative whereas negative areas have more activity during resting
• tfMRI contrast beta maps (i.e. effect size maps) produce gradients just like the architectonic maps

Low

High

-

+

Connectivity 🡪 Resting State fMRI 🡪 Gradients

Low

High

-

+

• Positive areas are functionally connected (correlated)
• Gradient tells us where functional connectivity changes across the cortex and by how much
• Stepping across a strong gradient leads to a dramatic change in functional connectivity
• Note that areas that activate together are often functionally connected

What about Topography?

• Not enough time to discuss in detail today
• Possible to map out visuotopy with retinotopic tasks
• Also possible to map out visuotopy with resting state fMRI
• In either case, topography enables mapping of visual areas whereas connectivity would tend to split and combine visual areas according to topographic gradients
• Also can be used to map subareas of somatosensory and motor cortex

Multi-modal Parcellation: Putting It All Together for One Cortical Area

• A strip of lightly myelinated cortex between the FEFs and Premotor Eye Field
• Gradients define most likely areal boundaries
• Has unique task activity in the STORY vs Resting contrast
• Task fMRI gradients line up with myelin gradients
• Has a unique functional connectivity pattern with respect to its neighbors
• Resting state gradients line up with the myelin and task gradients
• Multiple independent modalities (architecture, function, and connectivity) agree on area
• The last step in parcellation is to identify the area with respect to the literature
• Here the area largely corresponds to 55b in the Hopf (1956) myeloarchitectonic parcellation

Seed

Seed

Multimodal Cortical Parcellation: Methods

• The multi-modal parcellation was constructed from 210(P) subjects
• Borders were defined using gradients in group average
• Architecture (myelin maps and thickness with curvature regressed out)
• Function (86 task fMRI contrast maps from 7 tasks)
• Connectivity (Resting state functional connectivity)
• Topography (Visuotopic resting state functional connectivity)
• Semi-automated parcellation approach:
• Neuroanatomists (MG and DVE) chose modalities to use for each areal border and drew initial boundary
• Algorithm optimized the boundary to best follow the multi-modal gradients
• Areas were identified with reference to the prior neuroanatomical literature
• We attempted to keep the same area names when possible

Multimodal Cortical Parcellation: Predictions

• Qualitative Predictions based on monkeys and partial human parcellation (Van Essen et al 2012):
• 150-200 human cortical areas per hemisphere
• Wide variability in areal size and shape across cortex
• Will be examples of inter-areal heterogeneity (e.g. early sensory-motor topographies)

​

Multimodal Cortical Parcellation: Borders

• Qualitative Results:
• 180 Areas (97 new, 83 existing) per hemisphere (all areas found in both hemispheres)
• Wide variability in areal size and shape across cortex
• Accurate parcellation of previously troublesome but, well understood areas like early visual and sensory motor cortex
• Some Areas contain topographic subareas (e.g. the motor and somatosensory areas)

​

Multimodal Cortical Parcellation: Colors

■ Auditory

■ Sensori-motor

■ Visual

■ Task-Negative (Dark)

■ Task-Positive (Bright)

Core groups of areas are pure colors, areas with shared connectivity are mixed colors

Parcellated Analyses

Dense Myelin Map

Light

Heavy

Parcellated Analyses

Parcellated Myelin Map

Light

Heavy

Parcellated Analyses �

Full Correlation Functional Connectome (PGi)

Partial Correlation Functional Connectome (PGi)

(CIFTI .pconn.nii)

Group Z

Group Z

20

-20

20

-20

Group Z

Group Z

tfMRI Working Memory (2BK-0BK)

tfMRI Language (STORY)

20

-20

20

-20

Parcellated Analyses

(CIFTI .pscalar.nii)

Case Study: Sharing Extensively Analyzed Data

• Glasser et al., 2016 Nature comprised over 50 figures and over 200 data files
• “Scene files” in Connectome Workbench were used to save figures and then upload them to BALSA (https://balsa.wustl.edu) together with the data files
• A link is provided in the paper to download the data for each figure

Download a fully labeled HCP_MMP1.0 parcellation:

https://balsa.wustl.edu/sceneFile/Zvk4

Reproducibility of the HCP’s multi-modal parcellation

• How to replicate the group average parcellation in individual subjects?
• Automatically?
• Even atypical subjects?
• How to replicate the group average parcellation in a new group of subjects?
• We developed a machine learning areal classifier.

Reproducibility of Areal Classifications at the Individual Subject Level

• Some Test-Retest HCP Subjects were completely rescanned through the entire HCP protocol and Pipelines a second time
• Typical Subject Run 1
• Typical Subject Run 2
• Split Subject
• Shifted Subject
• Not only does the classifier reproducibly identify area 55b, the atypical areal topologies are stable in time

Original Group Parcellation and Individual Regularized Areal MPMs

• Subject 1
• Subject 2
• Subject 3

Group Parcellation

Individual Parcellation

Comparison of Individual Subject Areal Detection Rates in 210P and 210V Groups

• 210V has very similar areal detection rates as 210P despite not having been used in the parcellation or classifier training
• 96.6% of areas detected (210V)
• 96.4% without task fMRI (210V)
• The areal classifier will work for studies without the HCP’s specialized task fMRI battery
• Need only T1w, T2w, Field Map, and fMRI to use it

210P

210V

Reproducing the Multi-modal Parcellation Using Only Areal Fingerprints: Probabilistic Maps

• The trained classifier was applied to the 210P and 210V datasets to generate individual subject parcellations
• These parcellations’ areas were averaged across subjects to produce probabilistic areas
• The probabilistic maps are very similar across the two groups

V1

M1

55b

46

210P

210V

Reproducing the Multi-modal Parcellation Using Only Areal Fingerprints: MPMs

• Group maximum probability maps were computed for both groups from probabilistic maps
• The areal boundary grayordinates displayed
• Blue for 210P
• Red for 210V
• Purple for both
• The boundaries are in very high agreement
• Correlation of these parcellations is 0.965
• This is in line with the dense map reproducibilities we saw earlier

What Happens If You Try To Compare Traditionally Processed Results to the New Brain Map?

• Process the individual area definitions According to
• 1) The HCP’s Approach (areal feature-based surface alignment, no smoothing)
• 2) Traditional Approaches (volume-based alignment +/- volume smoothing)
• Only the HCP’s approach reproduces the map of most cortical areas

Coalson, Van Essen, and Glasser (2018) Proceedings of the National Academy of Sciences (PNAS)

Uncertainty

Strongest Label

Yellow is not greymatter

Original Map

Reproducible Visual Neuroscience—Retinotopic Visual Cortical Maps

• Compared independent retinotopic fMRI processed using a folding-based alignment approach to HCP 7T fMRI data aligned with areal features
• Very similar patterns, but the HCP’s data have sharper group maps and tighter population receptive fields

HCP

MRC Vis

Himmelberg et al., BioRxiv

Reproducible Cognitive Neuroscience—A Domain General Cognitive Core

• Used the HCP’s task fMRI data and brain map to define a domain general cognitive core
• Replicated the findings on their own subjects scanned and analyzed using the HCP’s approach to brain imaging in both visual and auditory tasks

Assem et al., (2020) Cerebral Cortex

Assem et al., BioRxiv

HCP

MRC Vis

MRC Aud

Putting It All Together: A Better, HCP-Style, Approach to Brain Imaging Research

• We propose the following principles for acquisition, analysis, and sharing of brain imaging data:
• 1) Acquire high quality, high resolution multi-modal data
• 2) Avoid blurring brain imaging data or lossy temporal denoising
• 3) Use 2D surfaces for cortex, 3D volumes for subcortical nuclei
• 4) Accurately align brain areas across people
• 5) Use an accurate, functionally relevant map of the brain
• 6) Freely share brain imaging results and data in publicly available databases, not just peak 3D coordinates
• Comparison of traditional processing (volume alignment, smoothing) with HCP-Style processing
• Like the difference between ground and space telescopes

Glasser et al (2016) Nature Neuroscience