Multiplexed Tissue Imaging: Tools and Approaches

I2K 2024, October 24^th 2024

SciLifeLab BioImage Informatics Facility

Multiplexed Tissue Imaging: Tools and Approaches © 2024 by The SciLifeLab BioImage Informatics Facility is licensed under CC BY-SA 4.0

Workshop information

Goals:

• Conceptual overview of common tools & approaches (theory)
• First steps with PIPEX, TissUUmaps, spatial analysis (hands-on)

Target audience:�Novices in multiplexed imaging data processing / analysis

Questions welcome after theory / during hands-on session

​

Workshop material:�https://github.com/BIIFSweden/I2K2024-MTIWorkshop

2

Agenda

Theory session:

• Introduction
• Image processing
• Toolkits & pipelines
• Spatial single-cell analysis

Hands-on session:

PIPEX, TissUUmaps, SpatialData

3

SciLifeLab BioImage Informatics Unit (BIIF)

4

Support, Training, and Tools for Image Data Analysis.

​

State-of-the-art analysis on image data using computer vision, machine learning, and bioinformatics.

​

Help to deploy computational methods.

Contact: biif@scilifelab.se

Jan Ellenberg, SciLifeLab director since July 2024

Open positions @ BIIF

www.scilifelab.se/units/bioimage-informatics/

SciLifeLab BIIF-based

• BioImage Informatician, Uppsala site (permanent)
• Research Software Engineer, Stockholm-Solna site (1 year for now)

EMBL-Heidelberg (2 years) + SciLifeLab BIIF-based (up to a year)�Postdoc within the ARISE2 program (3 years): �“Developing scalable and reusable analysis pipelines for spatial omics”�Together with Christian Tischer (EMBL)

5

Introduction

Multiplexed tissue imaging

6

Motivation

Conventional fluorescence microscopy:

• Biological sample (tissue, cell culture, …)
• Antibodies + fluorophores (IF), DAPI
• Optical microscopy readout

​

Spectral overlap�→ limited # targets

7

Image source: https://imagej.net/plugins/lumos-spectral-unmixing

Multiplexed tissue imaging

Workshop focus:

• Tissue sections
• Antibody-based
• Two-dimensional
• Single time point

Many destructive,�some non-destructive

Targeted, but often analyzed “as if untargeted”

8

Image source: de Souza, Zhao & Bodenmiller. Multiplex protein imaging in tumour biology. Nat Rev Cancer (2024).

Multiplexed tissue imaging data

One-shot readout

Mass spectrometry-based

Single CYX image (50+ channels)

No image co-registration needed

Iterative readout

Fluorophores / oligonucleotides

Several (C)YX images (iterations)

Iterations may require co-registration

9

Single 2D image stack, one channel per target

Many (mostly proprietary) file formats

OME Bio-Formats → (OME-)TIFF

OME-NGFF: https://ngff.openmicroscopy.org

Motivation

Conventional fluorescence microscopy

Structures within biological samples are made visible by

• Fluorescent stains (e.g. DAPI) or tags (e.g. GFP)
• Antibodies + fluorophores (immunofluorescence, IF)

Optical microscopy readout recorded by camera/detector

Spectral overlap�→ limited # targets

10

Image source: https://imagej.net/plugins/lumos-spectral-unmixing

Multiplexed tissue imaging

One-shot readout

Mass spectrometry-based

Single CYX image (50+ channels)

No image co-registration needed

Iterative readout

Fluorophores / oligonucleotides

Several (C)YX images (iterations)

Iterations may require

co-registration

11

Image source: de Souza, Zhao & Bodenmiller. Multiplex protein imaging in tumour biology. Nat Rev Cancer (2024).

Image processing

Stitching & registration, segmentation, quantification

12

Overview

13

Image visualization & quality control

Stitching & registration

Combine tiles & co-register channels

• Often done by instrument software (can be imprecise)
• Co-registration only necessary for iterative readouts

​

Frequently used:

ASHLAR (Muhlich et al., 2022)�Combined stitching & registration�of multiplexed tissue imaging data

14

Image source: Muhlich et al. Stitching and registering highly multiplexed�whole-slide images of tissues and tumors using ASHLAR. Bioinformatics (2022).

Image segmentation

Segmentation of nuclei, cells, regions, … in (dense) tissue sections

Perfect 2D segmentation unattainable* → aim for “good enough”!

Classical algorithms (e.g. Watershed):

– Cannot leverage multi-channel information

Supervised pixel classification + classical algorithms�(e.g. random forests/Ilastik + CellProfiler’s IdentifyPrimaryObjects)

+ Can leverage multiple channels

~ Tailored to dataset (how to re-use?)� – Requires training data (manual annotations)

15

Image source: Zhang et al.Segmentation of overlapping cells in cervical smears based on spatial relationship and Overlapping Translucency Light Transmission Model. Pattern Recognition (2016).

Image segmentation - classical approaches

Nuclei segmentation

• intensity threshold
• watershed

​

Problems:

• dense regions
• variations in nuclei size

​

Cell segmentation

• seeded watershed starting from nuclei
• supervised pixel classification + watershed on probabilities

Problems:

Signal distributed over multi-channel → channel aggregation

No clear cell borders

other (e.g. vessel, tumor regions)

• supervised pixel classification

​

Problems:

Tailored to dataset (how to re-use?)

Requires training data (manual annotations)

16

Nuclei segmentation + expansion may be sufficient!

Deep learning-based image segmentation

Pre-trained models (e.g., Cellpose, Mesmer, StarDist)

+ Large and diverse training data → high accuracy

+ Can be fine-tuned to new annotations (transfer learning)�– Cannot leverage multi-channel → channel aggregation

– Only work for intended task (e.g. nuclei segmentation)

Models trained from scratch (e.g., U-Nets, Mask R-CNNs)�→ require large training data; cf. supervised pixel classification

Novel approaches:�Channel-invariant methods (e.g. InstanSeg/ChannelNet)�Foundation models (e.g. vision transformers, Segment Anything)

17

Cell quantification

Relies on accurate segmentation; often scripted (e.g. scikit-image regionprops)

Cell intensities: aggregate pixel values (summary statistic)

• Distributional statistics (e.g., mean, SD):�sensitive to outliers, e.g. “hot pixels”
• Positional statistics (e.g., max, median, percentile):�sensitive to subcellular spatial “coverage”

Nuclei segmentation + expansion may be sufficient!

Cell morphology: size, shape, …�2D imaging of thin tissue sections → cells not fully captured!

18

Image source: https://x.com/zanotellivrt/status/1187299455822901248

Cell quantification

Relies on accurate segmentation; often scripted (e.g. scikit-image regionprops)

Cell intensities: aggregate pixel values (summary statistic)

• mean, SD, etc. (distributional statistics)�– sensitive to outliers, e.g. “hot pixels”
• max, median, percentile, etc. (positional statistics):�+ robust to outliers�– sensitive to subcellular spatial “coverage”

​

Cell morphology: size, shape, …�2D imaging of thin tissue sections → cells not fully captured!

19

Image source: https://x.com/zanotellivrt/status/1187299455822901248

Image visualization

20

Image source: https://napari.org/0.5.0/guides/layers.html

Image source: https://imagej.net/ij/plugins/multi-channel-helper/index.html

Image source: https://qupath.readthedocs.io�/en/0.5/docs/tutorials/multiplex_analysis.html

Multiplexed imaging and TissUUmaps

21

https://tissuumaps.scilifelab.se/multiplex.tmap?path=private/Christophe/multiplex_demo

Toolkits & pipelines

MCMICRO, Steinbock, PIPEX

22

Overview

Terminology:

• Tool: focus on single task (see previous section)
• Toolkit: multiple streamlined tools → stepwise execution
• Pipeline: defined order/workflow of steps → end-to-end execution

Toolkits & pipelines “make tools interact” and enable batch processing

Selected toolkits/pipelines:

• MCMICRO
• Steinbock
• PIPEX

23

MCMICRO

Schapiro, Sokolov & Yapp et al. MCMICRO: a scalable, modular image-processing pipeline for multiplexed tissue imaging. Nature Methods, 2022.

https://mcmicro.org

Nextflow pipeline�Configuration file → “end-to-end” execution

Technology-agnostic, many modules (containers)

Primarily developed at Harvard Medical School / Heidelberg University�(nf-core/mcmicro is work in progress)

24

MCMICRO

25

Image source: Schapiro, Sokolov & Yapp et al. MCMICRO: a scalable, modular image-processing pipeline for multiplexed tissue imaging. Nature Methods (2022).

Steinbock

Windhager et al. An end-to-end workflow for multiplexed image processing and analysis. Nature Protocols, 2023.

https://bodenmillergroup.github.io/steinbock/

Single container / Python package�Command-line interface → “stepwise” execution

Imaging Mass Cytometry (IMC) focus, multi-channel TIFF input

R/Bioconductor imcRtools package support, various export formats

Maintained by the Bodenmiller Lab (ETH / University of Zurich)

https://www.youtube.com/watch?v=CiyFPS9ig8o&t=6s

26

Steinbock

27

Image source: Windhager et al. An end-to-end workflow for multiplexed image processing and analysis. Nature Protocols (2023).

PIPEX

“A collection of commands, scripts and utilities to transform CODEX images into valuable analysis items” (public GitHub repository, manuscript in preparation)

https://github.com/CellProfiling/pipex/

Python scripts → “end-to-end” execution�Graphical user interface for configuration

CODEX/PhenoCycler focus, several supported input formats

“Standard analysis” (e.g. unsupervised clustering) report

Maintained by the Lundberg Lab (Stanford University)

28

Comparison

29

Spatial single-cell analysis

Tools & approaches

30

Overview

31

Image visualization & quality control

Overview

Field of active research!

“Traditional” single-cell analysis (cf. scRNA-seq) + spatial analysis

R:�Seurat�Bioconductor SingleCellExperiment/SpatialExperiment�https://bioconductor.org/books/release/OSCA/ (non-spatial)

Python:�scverse anndata/SpatialData

32

Single-cell data transformation

Transform skewed cell intensity distributions�(e.g. for visualization, thresholding):

• log(1 + x)
• asinh(x / c)

Dimensionality reduction:

• PCA (linear)
• t-SNE (nonlinear)�https://distill.pub/2016/misread-tsne/
• UMAP (nonlinear)�https://pair-code.github.io/understanding-umap/

Do NOT cluster on nonlinear embeddings!

33

Image source: https://github.com/maxentile/advanced-ml-project/issues/2

Image source: https://bodenmillergroup.github.io/IMCDataAnalysis/single-cell-visualization.html

Batch effect correction

Remove between-sample variation, but retain within-sample variation�(e.g. to facilitate cell phenotyping)

Frequently used (cell level):

• Canonical correlation analysis,�e.g. implemented in Seurat
• Iterative clustering + PCA,�e.g. implemented in Harmony
• Mutual nearest neighbors,�e.g. implemented in Batchelor

On pixel level (i.e. before segmentation) or on cell level (i.e. after quantification)?

34

Image source: https://www.10xgenomics.com/analysis-guides/introduction-batch-effect-correction

Cell phenotyping

Assign a cell type (?) to each cell

Frequently used:

• Channelwise thresholding (“gating”)
• Unsupervised clustering, e.g. PhenoGraph, FlowSOM
• Supervised classification, e.g. random forests (requires cell labels)
• Probabilistic methods, e.g. Astir, Garnett (require cell type definition)

Often, a combined approach is most effective, e.g.

• Thresholding on selected channels → coarse cell types
• Unsupervised clustering within coarse cell types → novel subpopulations

35

Spatial cell graphs

Nodes represent cells in space

Edges connect cells in spatial proximity

Frequently used:

• Distance thresholding
• k-nearest neighbors (kNN)
• Delaunay triangulation

36

Spatial single-cell analysis

• Community detection (topology only), e.g. Leiden/Louvain algorithms
• Interaction analysis (topology, cell types)�Spatial permutation tests wrt. pairwise cell type interactions
• Cellular neighborhood analysis (topology, cell types/intensities)�Neighborhood aggregation of cell types/intensities + clustering in feature space
• Spatial context analysis (topology, cellular neighborhoods)�Neighborhood aggregation of cellular neighborhoods + top-n grouping
• Patch detection, graph neural networks, …

See also: https://bodenmillergroup.github.io/IMCDataAnalysis/performing-spatial-analysis.html

37

Hands-on session

PIPEX, TissUUmaps, SpatialData

38

Workshop material:�https://github.com/BIIFSweden/I2K2024-MTIWorkshop