1 of 108

The SpatialData framework: from raw data to interactive exploration (workshop introduction)

BIOINFO 2024, Gyeongju

LucaMarconato

Luca Marconato (Oliver Stegle lab)

LucaMarconato2

Workshop

notebooks

Documentation

notebooks

scverse/spatialdata

October 22nd, 2024

lmarconato

Joint work with Giovanni Palla, Kevin Yamauchi, Tim Treis, Wouter Vierdag and more

2 of 108

The SpatialData framework: from raw data to interactive exploration (workshop introduction)

BIOINFO 2024, Gyeongju

LucaMarconato

Luca Marconato (Oliver Stegle lab)

LucaMarconato2

Workshop

notebooks

Documentation

notebooks

scverse/spatialdata

October 22nd, 2024

workshop

docs

lmarconato

Joint work with Giovanni Palla, Kevin Yamauchi, Tim Treis, Wouter Vierdag and more

3 of 108

Spatial organizations are crucial to explaining biological mechanisms

Oral Pole

Drosophila developing embryos show spatial pattern of gene expression

4 of 108

Spatial organizations are crucial to explaining biological mechanisms

Oral Pole

Tumor microenvironments are organized in complex spatial structures

Lymphocytes (red) infiltrating in beyond the tumor boundaries (yellow) are linked with better prognosis

Adapted from Saltz et al. (Cell Reports, 2018)

Adapted from Balkwill et al. (J. Cell Sci, 2018)

5 of 108

Spatial organizations are crucial to explaining biological mechanisms

Oral Pole

Spatial pattern of expression at the subcellular level

Protein expressed near the boundary (green) vs within the cell (purple)

6 of 108

Spatial omics data could be, in principle, grouped into three classes

IMC

Multi-channel images (“Raster”)

MALDI

Microscopy

Introduction 1/

MERFISH

Cartana ISS

Single molecule

Annotated regions

Visium

ST

Slide-seq

Tomo-seq

7 of 108

Example 1/3: Visium 10x technology

55 µm

Visium capture

regions (“spots”)

Visium

Introduction 1/

Adapted from Moor et al. (ScienceDirect, 2017)

Permeabilization

Sequencing

8 of 108

Example 1/3: storage of a Visium dataset

embeddings (matrices)

expression (AnnData)

cell2location estimates (matrices)

clusters (matrices)

image (png)

expression data (matrices)

scaling factors (json)

positions (matrices)

large images (png, 40k x 40k pixels)

Introduction 1/

“raw” data processed data

◼︎

◼︎

9 of 108

Example 2/3: the smFISH and MERFISH techniques

Haimovich et al. (Bio-protocol, 2018)

Introduction 1/

Adapted from Crosetto et al. (Nat. Rev. Genet, 2014)

Adapted from Xia et al. (Biophysics and Computational Biology, 2019)

10 of 108

Example 2/3: storage of a MERFISH dataset

Allen: VISp from SpaceTx SpaceJam data (Github)

Introduction 1/

single cell spatial data (matrices, dataframe)

single molecule spatial data (matrices, dataframe)

boundaries (polygons)

11 of 108

Example 3/3: imaging mass cytometry (IMC) / Hyperion

Aboral Pole

Oral Pole

Adapted from Bodenmiller et al. (Cell Systems, 2018)

Simultaneous staining for:

  • RNA
  • Proteins

Introduction 1/

12 of 108

Example 3/3: storage of an IMC dataset

Aboral Pole

pairwise cell distances (matrices/graphs)

boundaries (polygon)

channel information (dataframe)

embeddings (matrices)

Another IMC dataset (1 slide)

channel information (dataframe)

multi-channel images of pixel-level expression (tensor)

multi-channel images with cell masks (tensor)

single cell spatial data (matrices, locations)

multi-channel images of pixel-level expression (tensor)

Jackson, Fisher et al. (Nature,2020)

Introduction 1/

“raw” data processed data

◼︎

◼︎

13 of 108

SpatialData unifies the representation of spatial omics across technologies

Xenium

Resolution: single-molecule

Up to 5K genes

a.

Visium

SpatialData

Resolution: 55µm

Transcriptome-wide

Visium HD

Resolution: 2µm, 8µm, 16µm, …

Transcriptome-wide

  • Non-goal: not an analysis library
  • Interoperable infrastructure
  • Simple read/write
  • Flexible representation
  • Object manipulation

Preview Data: FFPE Human Lung Cancer with Xenium Multimodal Cell Segmentation

Visium HD Spatial Gene Expression Library, Mouse Small Intestine (FFPE)

Data: https://www.10xgenomics.com/products/xenium-in-situ/preview-dataset-human-breast

Data at different scales

14 of 108

SpatialData unifies the representation of spatial omics across technologies

a.

Visium

Resolution: 55µm

Transcriptome-wide

Data: https://www.10xgenomics.com/products/xenium-in-situ/preview-dataset-human-breast

Data at different scales

15 of 108

SpatialData unifies the representation of spatial omics across technologies

Xenium

Resolution: single-molecule

Up to 5K genes

a.

Visium

Resolution: 55µm

Transcriptome-wide

Preview Data: FFPE Human Lung Cancer with Xenium Multimodal Cell Segmentation

Data: https://www.10xgenomics.com/products/xenium-in-situ/preview-dataset-human-breast

Data at different scales

16 of 108

SpatialData unifies the representation of spatial omics across technologies

Xenium

Resolution: single-molecule

Up to 5K genes

a.

Visium

Resolution: 55µm

Transcriptome-wide

Visium HD

Resolution: 2µm

Preview Data: FFPE Human Lung Cancer with Xenium Multimodal Cell Segmentation

Visium HD Spatial Gene Expression Library, Mouse Small Intestine (FFPE)

Data: https://www.10xgenomics.com/products/xenium-in-situ/preview-dataset-human-breast

Data at different scales

17 of 108

SpatialData unifies the representation of spatial omics across technologies

Xenium

Resolution: single-molecule

Up to 5K genes

a.

Visium

Resolution: 55µm

Transcriptome-wide

Visium HD

Resolution: 2µm, 8µm, 16µm, …

Transcriptome-wide

Preview Data: FFPE Human Lung Cancer with Xenium Multimodal Cell Segmentation

Visium HD Spatial Gene Expression Library, Mouse Small Intestine (FFPE)

Data: https://www.10xgenomics.com/products/xenium-in-situ/preview-dataset-human-breast

Data at different scales

18 of 108

SpatialData unifies the representation of spatial omics across technologies

Xenium

Resolution: single-molecule

Up to 5K genes

a.

Visium

SpatialData

Resolution: 55µm

Transcriptome-wide

Visium HD

Resolution: 2µm, 8µm, 16µm, …

Transcriptome-wide

Preview Data: FFPE Human Lung Cancer with Xenium Multimodal Cell Segmentation

Visium HD Spatial Gene Expression Library, Mouse Small Intestine (FFPE)

Data: https://www.10xgenomics.com/products/xenium-in-situ/preview-dataset-human-breast

Data at different scales

19 of 108

SpatialData unifies the representation of spatial omics across technologies

Xenium

Resolution: single-molecule

Up to 5K genes

a.

Visium

SpatialData

Resolution: 55µm

Transcriptome-wide

Visium HD

Resolution: 2µm, 8µm, 16µm, …

Transcriptome-wide

  • Simple read/write

Preview Data: FFPE Human Lung Cancer with Xenium Multimodal Cell Segmentation

Visium HD Spatial Gene Expression Library, Mouse Small Intestine (FFPE)

Data: https://www.10xgenomics.com/products/xenium-in-situ/preview-dataset-human-breast

Data at different scales

20 of 108

SpatialData unifies the representation of spatial omics across technologies

Xenium

Resolution: single-molecule

Up to 5K genes

a.

Visium

SpatialData

Resolution: 55µm

Transcriptome-wide

Visium HD

Resolution: 2µm, 8µm, 16µm, …

Transcriptome-wide

  • Simple read/write
  • Flexible representation

Preview Data: FFPE Human Lung Cancer with Xenium Multimodal Cell Segmentation

Visium HD Spatial Gene Expression Library, Mouse Small Intestine (FFPE)

Data: https://www.10xgenomics.com/products/xenium-in-situ/preview-dataset-human-breast

Data at different scales

21 of 108

SpatialData unifies the representation of spatial omics across technologies

Xenium

Resolution: single-molecule

Up to 5K genes

a.

Visium

SpatialData

Resolution: 55µm

Transcriptome-wide

Visium HD

Resolution: 2µm, 8µm, 16µm, …

Transcriptome-wide

  • Simple read/write
  • Flexible representation
  • Object manipulation

Preview Data: FFPE Human Lung Cancer with Xenium Multimodal Cell Segmentation

Visium HD Spatial Gene Expression Library, Mouse Small Intestine (FFPE)

Data: https://www.10xgenomics.com/products/xenium-in-situ/preview-dataset-human-breast

Data at different scales

22 of 108

SpatialData unifies the representation of spatial omics across technologies

Xenium

Resolution: single-molecule

Up to 5K genes

a.

Visium

SpatialData

Resolution: 55µm

Transcriptome-wide

Visium HD

Resolution: 2µm, 8µm, 16µm, …

Transcriptome-wide

  • Non-goal: not an analysis library
  • Simple read/write
  • Flexible representation
  • Object manipulation

Preview Data: FFPE Human Lung Cancer with Xenium Multimodal Cell Segmentation

Visium HD Spatial Gene Expression Library, Mouse Small Intestine (FFPE)

Data: https://www.10xgenomics.com/products/xenium-in-situ/preview-dataset-human-breast

Data at different scales

23 of 108

SpatialData unifies the representation of spatial omics across technologies

Xenium

Resolution: single-molecule

Up to 5K genes

a.

Visium

SpatialData

Resolution: 55µm

Transcriptome-wide

Visium HD

Resolution: 2µm, 8µm, 16µm, …

Transcriptome-wide

  • Non-goal: not an analysis library
  • Interoperable infrastructure
  • Simple read/write
  • Flexible representation
  • Object manipulation

Preview Data: FFPE Human Lung Cancer with Xenium Multimodal Cell Segmentation

Visium HD Spatial Gene Expression Library, Mouse Small Intestine (FFPE)

Data: https://www.10xgenomics.com/products/xenium-in-situ/preview-dataset-human-breast

Data at different scales

24 of 108

SpatialData unifies the representation of spatial omics across technologies

a.

Reading the data from disk:

Xenium

Visium

Visium HD

Preview Data: FFPE Human Lung Cancer with Xenium Multimodal Cell Segmentation

Visium HD Spatial Gene Expression Library, Mouse Small Intestine (FFPE)

Data: https://www.10xgenomics.com/products/xenium-in-situ/preview-dataset-human-breast

Data at different scales

cells

approximation

nuclei

2µm, 8µm, 16µm bins

transcripts

workshop

workshop

workshop

25 of 108

Data is represented with a modular combination of reusable elements

Microscopy images, …

Segmentation mask

Raster geometries

26 of 108

Data is represented with a modular combination of reusable elements

Microscopy images, …

Segmentation mask

Raster geometries

Vector geometries

Transcripts locations

Cells, ROIs,

27 of 108

Data is represented with a modular combination of reusable elements

Microscopy images, …

Segmentation mask

Raster geometries

Vector geometries

Transcripts locations

Cells, ROIs,

Gene expression, cell types, …

Annotations

Annotates

28 of 108

SpatialData unifies the representation of spatial omics across technologies

cells

a.

Visium

Visium HD

Xenium

approximation

nuclei

2µm, 8µm, 16µm bins

transcripts

Preview Data: FFPE Human Lung Cancer with Xenium Multimodal Cell Segmentation

Visium HD Spatial Gene Expression Library, Mouse Small Intestine (FFPE)

Data: https://www.10xgenomics.com/products/xenium-in-situ/preview-dataset-human-breast

Data at different scales

29 of 108

SpatialData unifies the representation of spatial omics across technologies

a.

Visium

Visium HD

Xenium

Preview Data: FFPE Human Lung Cancer with Xenium Multimodal Cell Segmentation

Visium HD Spatial Gene Expression Library, Mouse Small Intestine (FFPE)

Data: https://www.10xgenomics.com/products/xenium-in-situ/preview-dataset-human-breast

Data at different scales

30 of 108

SpatialData unifies the representation of spatial omics across technologies

a.

Visium

Visium HD

Xenium

Preview Data: FFPE Human Lung Cancer with Xenium Multimodal Cell Segmentation

Visium HD Spatial Gene Expression Library, Mouse Small Intestine (FFPE)

Data: https://www.10xgenomics.com/products/xenium-in-situ/preview-dataset-human-breast

Data at different scales

31 of 108

SpatialData unifies the representation of spatial omics across technologies

a.

Visium

Visium HD

Xenium

Preview Data: FFPE Human Lung Cancer with Xenium Multimodal Cell Segmentation

Visium HD Spatial Gene Expression Library, Mouse Small Intestine (FFPE)

Data: https://www.10xgenomics.com/products/xenium-in-situ/preview-dataset-human-breast

Data at different scales

32 of 108

SpatialData unifies the representation of spatial omics across technologies

a.

Visium

Visium HD

Xenium

Preview Data: FFPE Human Lung Cancer with Xenium Multimodal Cell Segmentation

Visium HD Spatial Gene Expression Library, Mouse Small Intestine (FFPE)

Data: https://www.10xgenomics.com/products/xenium-in-situ/preview-dataset-human-breast

Data at different scales

transcripts

33 of 108

SpatialData unifies the representation of spatial omics across technologies

a.

Visium

Visium HD

Xenium

Preview Data: FFPE Human Lung Cancer with Xenium Multimodal Cell Segmentation

Visium HD Spatial Gene Expression Library, Mouse Small Intestine (FFPE)

Data: https://www.10xgenomics.com/products/xenium-in-situ/preview-dataset-human-breast

Data at different scales

cells

transcripts

34 of 108

SpatialData unifies the representation of spatial omics across technologies

a.

Visium

Visium HD

Xenium

Preview Data: FFPE Human Lung Cancer with Xenium Multimodal Cell Segmentation

Visium HD Spatial Gene Expression Library, Mouse Small Intestine (FFPE)

Data: https://www.10xgenomics.com/products/xenium-in-situ/preview-dataset-human-breast

Data at different scales

cells

nuclei

transcripts

35 of 108

SpatialData unifies the representation of spatial omics across technologies

a.

Visium

Visium HD

Xenium

Preview Data: FFPE Human Lung Cancer with Xenium Multimodal Cell Segmentation

Visium HD Spatial Gene Expression Library, Mouse Small Intestine (FFPE)

Data: https://www.10xgenomics.com/products/xenium-in-situ/preview-dataset-human-breast

Data at different scales

cells

approximation

nuclei

transcripts

36 of 108

SpatialData unifies the representation of spatial omics across technologies

a.

Visium

Visium HD

Xenium

Preview Data: FFPE Human Lung Cancer with Xenium Multimodal Cell Segmentation

Visium HD Spatial Gene Expression Library, Mouse Small Intestine (FFPE)

Data: https://www.10xgenomics.com/products/xenium-in-situ/preview-dataset-human-breast

Data at different scales

cells

approximation

nuclei

transcripts

37 of 108

SpatialData unifies the representation of spatial omics across technologies

a.

Visium

Visium HD

Xenium

Preview Data: FFPE Human Lung Cancer with Xenium Multimodal Cell Segmentation

Visium HD Spatial Gene Expression Library, Mouse Small Intestine (FFPE)

Data: https://www.10xgenomics.com/products/xenium-in-situ/preview-dataset-human-breast

Data at different scales

cells

approximation

nuclei

transcripts

2µm, 8µm, 16µm bins

38 of 108

SpatialData unifies the representation of spatial omics across technologies

a.

Visium

Visium HD

Xenium

Preview Data: FFPE Human Lung Cancer with Xenium Multimodal Cell Segmentation

Visium HD Spatial Gene Expression Library, Mouse Small Intestine (FFPE)

Data: https://www.10xgenomics.com/products/xenium-in-situ/preview-dataset-human-breast

Data at different scales

workshop

39 of 108

Coordinate transformations enable alignment to common coordinate systems

SpatialData object

{“name" : “pixel-space",

"axes" : [

{"name": "j", "type": "space", "discrete": true },

{"name": "i", "type": "space", "discrete": true }]}

pixel space

{"name" : "pixels-to-micrometers",

"type" : "affine",

"values" : [

[0.89, -0.45, 1.00],

[0.45, 0.89, 2.00],

[0.00, 0.00, 1.00]],

"input_space" : "pixel-space",

"output_space": "physical-micrometers"}

transformation

{"name" : “physical-micrometers",

"axes" : [

{"name": "y", "type": "space", "unit": “micrometer"},

{"name": "x", "type": "space", "unit": “micrometer”}]}

physical space

Read/write

docs

docs

docs

Specification

Disk format

(extension)

workshop

40 of 108

Coordinate transformations enable alignment to common coordinate systems

{“name" : “pixel-space",

"axes" : [

{"name": "j", "type": "space", "discrete": true },

{"name": "i", "type": "space", "discrete": true }]}

pixel space

41 of 108

Coordinate transformations enable alignment to common coordinate systems

{“name" : “pixel-space",

"axes" : [

{"name": "j", "type": "space", "discrete": true },

{"name": "i", "type": "space", "discrete": true }]}

pixel space

{"name" : “physical-micrometers",

"axes" : [

{"name": "y", "type": "space", "unit": “micrometer"},

{"name": "x", "type": "space", "unit": “micrometer”}]}

physical space

42 of 108

Coordinate transformations enable alignment to common coordinate systems

{“name" : “pixel-space",

"axes" : [

{"name": "j", "type": "space", "discrete": true },

{"name": "i", "type": "space", "discrete": true }]}

pixel space

{"name" : "pixels-to-micrometers",

"type" : "affine",

"values" : [

[0.89, -0.45, 1.00],

[0.45, 0.89, 2.00],

[0.00, 0.00, 1.00]],

"input_space" : "pixel-space",

"output_space": "physical-micrometers"}

transformation

{"name" : “physical-micrometers",

"axes" : [

{"name": "y", "type": "space", "unit": “micrometer"},

{"name": "x", "type": "space", "unit": “micrometer”}]}

physical space

43 of 108

Coordinate transformations enable alignment to common coordinate systems

docs

{“name" : “pixel-space",

"axes" : [

{"name": "j", "type": "space", "discrete": true },

{"name": "i", "type": "space", "discrete": true }]}

pixel space

{"name" : "pixels-to-micrometers",

"type" : "affine",

"values" : [

[0.89, -0.45, 1.00],

[0.45, 0.89, 2.00],

[0.00, 0.00, 1.00]],

"input_space" : "pixel-space",

"output_space": "physical-micrometers"}

transformation

{"name" : “physical-micrometers",

"axes" : [

{"name": "y", "type": "space", "unit": “micrometer"},

{"name": "x", "type": "space", "unit": “micrometer”}]}

physical space

44 of 108

Coordinate transformations enable alignment to common coordinate systems

docs

{“name" : “pixel-space",

"axes" : [

{"name": "j", "type": "space", "discrete": true },

{"name": "i", "type": "space", "discrete": true }]}

pixel space

{"name" : "pixels-to-micrometers",

"type" : "affine",

"values" : [

[0.89, -0.45, 1.00],

[0.45, 0.89, 2.00],

[0.00, 0.00, 1.00]],

"input_space" : "pixel-space",

"output_space": "physical-micrometers"}

transformation

{"name" : “physical-micrometers",

"axes" : [

{"name": "y", "type": "space", "unit": “micrometer"},

{"name": "x", "type": "space", "unit": “micrometer”}]}

physical space

docs

45 of 108

Coordinate transformations enable alignment to common coordinate systems

docs

{“name" : “pixel-space",

"axes" : [

{"name": "j", "type": "space", "discrete": true },

{"name": "i", "type": "space", "discrete": true }]}

pixel space

{"name" : "pixels-to-micrometers",

"type" : "affine",

"values" : [

[0.89, -0.45, 1.00],

[0.45, 0.89, 2.00],

[0.00, 0.00, 1.00]],

"input_space" : "pixel-space",

"output_space": "physical-micrometers"}

transformation

{"name" : “physical-micrometers",

"axes" : [

{"name": "y", "type": "space", "unit": “micrometer"},

{"name": "x", "type": "space", "unit": “micrometer”}]}

physical space

docs

docs

46 of 108

Coordinate transformations enable alignment to common coordinate systems

docs

SpatialData object

{“name" : “pixel-space",

"axes" : [

{"name": "j", "type": "space", "discrete": true },

{"name": "i", "type": "space", "discrete": true }]}

pixel space

{"name" : "pixels-to-micrometers",

"type" : "affine",

"values" : [

[0.89, -0.45, 1.00],

[0.45, 0.89, 2.00],

[0.00, 0.00, 1.00]],

"input_space" : "pixel-space",

"output_space": "physical-micrometers"}

transformation

{"name" : “physical-micrometers",

"axes" : [

{"name": "y", "type": "space", "unit": “micrometer"},

{"name": "x", "type": "space", "unit": “micrometer”}]}

physical space

docs

docs

47 of 108

Coordinate transformations enable alignment to common coordinate systems

docs

SpatialData object

{“name" : “pixel-space",

"axes" : [

{"name": "j", "type": "space", "discrete": true },

{"name": "i", "type": "space", "discrete": true }]}

pixel space

{"name" : "pixels-to-micrometers",

"type" : "affine",

"values" : [

[0.89, -0.45, 1.00],

[0.45, 0.89, 2.00],

[0.00, 0.00, 1.00]],

"input_space" : "pixel-space",

"output_space": "physical-micrometers"}

transformation

{"name" : “physical-micrometers",

"axes" : [

{"name": "y", "type": "space", "unit": “micrometer"},

{"name": "x", "type": "space", "unit": “micrometer”}]}

physical space

Read/write

docs

docs

Specification

Disk format

(extension)

48 of 108

Coordinate transformations enable alignment to common coordinate systems

docs

SpatialData object

{“name" : “pixel-space",

"axes" : [

{"name": "j", "type": "space", "discrete": true },

{"name": "i", "type": "space", "discrete": true }]}

pixel space

{"name" : "pixels-to-micrometers",

"type" : "affine",

"values" : [

[0.89, -0.45, 1.00],

[0.45, 0.89, 2.00],

[0.00, 0.00, 1.00]],

"input_space" : "pixel-space",

"output_space": "physical-micrometers"}

transformation

{"name" : “physical-micrometers",

"axes" : [

{"name": "y", "type": "space", "unit": “micrometer"},

{"name": "x", "type": "space", "unit": “micrometer”}]}

physical space

Read/write

docs

docs

Specification

Disk format

(extension)

workshop

49 of 108

Coordinate transformations enable alignment to common coordinate systems

docs

SpatialData object

{“name" : “pixel-space",

"axes" : [

{"name": "j", "type": "space", "discrete": true },

{"name": "i", "type": "space", "discrete": true }]}

pixel space

{"name" : "pixels-to-micrometers",

"type" : "affine",

"values" : [

[0.89, -0.45, 1.00],

[0.45, 0.89, 2.00],

[0.00, 0.00, 1.00]],

"input_space" : "pixel-space",

"output_space": "physical-micrometers"}

transformation

{"name" : “physical-micrometers",

"axes" : [

{"name": "y", "type": "space", "unit": “micrometer"},

{"name": "x", "type": "space", "unit": “micrometer”}]}

physical space

Read/write

docs

docs

Specification

Disk format

(extension)

workshop

50 of 108

Interactive visualization and annotation with napari-spatialdata

workshop

Marcela Toth

Rahul B. Shrestha

51 of 108

Interactive visualization and annotation with napari-spatialdata

Visium HD Spatial Gene Expression Library, Mouse Small Intestine (FFPE)

Shift + E

Save annotations*

workshop

*after user feedback we are revisiting the UX

Marcela Toth

Rahul B. Shrestha

52 of 108

Interactive visualization and annotation with napari-spatialdata

workshop

Marcela Toth

Rahul B. Shrestha

53 of 108

Interactive visualization and annotation with napari-spatialdata

Visium HD Spatial Gene Expression Library, Mouse Small Intestine (FFPE)

workshop

Marcela Toth

Rahul B. Shrestha

54 of 108

Interactive visualization and annotation with napari-spatialdata

Visium HD Spatial Gene Expression Library, Mouse Small Intestine (FFPE)

Shift + E

Save annotations*

workshop

*after user feedback we are revisiting the UX

Marcela Toth

Rahul B. Shrestha

55 of 108

Pathology annotation with napari-spatialdata

56 of 108

Pathology annotation with napari-spatialdata

57 of 108

Interactive visualization and annotations for custom embeddings

Kasia Kedziora

58 of 108

Interactive visualization and annotations for custom embeddings

Kasia Kedziora

59 of 108

Interactive visualization and annotations for custom embeddings

Kasia Kedziora

60 of 108

Static, composable visualization with spatialdata-plot

workshop

workshop

Harald Vöringer

Sonja Stockhaus

61 of 108

Static, composable visualization with spatialdata-plot

workshop

Harald Vöringer

Sonja Stockhaus

62 of 108

Static, composable visualization with spatialdata-plot

workshop

workshop

Harald Vöringer

Sonja Stockhaus

63 of 108

The modular structure of SpatialData objects integrates into existing analysis workflows

workshop

workshop

bento-tools

64 of 108

The modular structure of SpatialData objects integrates into existing analysis workflows

65 of 108

The modular structure of SpatialData objects integrates into existing analysis workflows

workshop

66 of 108

The modular structure of SpatialData objects integrates into existing analysis workflows

workshop

workshop

67 of 108

The modular structure of SpatialData objects integrates into existing analysis workflows

workshop

workshop

bento-tools

68 of 108

Generalized, reusable operations are defined for SpatialData objects

Spatial queries

Deep learning interface

Spatial aggregations

Vectorize

Rasterize

Coordinate transformations

Documentation:

e.g. rotate an image

e.g. crop the data

e.g. create image tiles around cells

e.g. count transcripts inside cells

docs

docs

docs

docs

docs

workshop

docs

workshop

69 of 108

Generalized, reusable operations are defined for SpatialData objects

Coordinate transformations

Documentation:

e.g. rotate an image

docs

70 of 108

Generalized, reusable operations are defined for SpatialData objects

Spatial queries

Coordinate transformations

Documentation:

e.g. rotate an image

e.g. crop the data

docs

docs

workshop

71 of 108

Generalized, reusable operations are defined for SpatialData objects

Spatial queries

Deep learning interface

Coordinate transformations

Documentation:

e.g. rotate an image

e.g. crop the data

e.g. create image tiles around cells

docs

docs

docs

workshop

72 of 108

Generalized, reusable operations are defined for SpatialData objects

Spatial queries

Deep learning interface

Spatial aggregations

Coordinate transformations

Documentation:

e.g. rotate an image

e.g. crop the data

e.g. create image tiles around cells

e.g. count transcripts inside cells

docs

docs

docs

workshop

docs

workshop

73 of 108

Generalized, reusable operations are defined for SpatialData objects

Spatial queries

Deep learning interface

Spatial aggregations

Vectorize

Rasterize

Coordinate transformations

Documentation:

e.g. rotate an image

e.g. crop the data

e.g. create image tiles around cells

e.g. count transcripts inside cells

docs

docs

docs

docs

docs

workshop

docs

workshop

74 of 108

Aligning multiple modalities with a landmark-based workflow

Before alignment (raw data)

After alignment

docs

75 of 108

Aligning multiple modalities with a landmark-based workflow

Before alignment (raw data)

76 of 108

Aligning multiple modalities with a landmark-based workflow

Before alignment (raw data)

77 of 108

Aligning multiple modalities with a landmark-based workflow

Before alignment (raw data)

After alignment

78 of 108

Aligning multiple modalities with a landmark-based workflow

Before alignment (raw data)

After alignment

docs

79 of 108

Spatial query APIs allow for cropping the data based on regions of interest

docs

workshop

80 of 108

Spatial query APIs allow for cropping the data based on regions of interest

81 of 108

Spatial query APIs allow for cropping the data based on regions of interest

82 of 108

Spatial query APIs allow for cropping the data based on regions of interest

docs

workshop

83 of 108

Aggregation example: computing cell types fractions within regions of interest

Xenium cell types

Manually annotated ROI

Immune infiltration

Cell type fraction

Collaborator: Elyas Heidari

docs

workshop

84 of 108

Aggregation example: computing cell types fractions within regions of interest

Xenium cell types

Manually annotated ROI

Immune infiltration

Collaborator: Elyas Heidari

85 of 108

Aggregation example: computing cell types fractions within regions of interest

Xenium cell types

Manually annotated ROI

Immune infiltration

Cell type fraction

Collaborator: Elyas Heidari

86 of 108

Aggregation example: computing cell types fractions within regions of interest

Xenium cell types

Manually annotated ROI

Immune infiltration

Cell type fraction

Collaborator: Elyas Heidari

docs

workshop

87 of 108

We provide a unified API covering all the aggregation operations

88 of 108

We provide a unified API covering all the aggregation operations

89 of 108

We provide a unified API covering all the aggregation operations

90 of 108

We provide a unified API covering all the aggregation operations

91 of 108

We provide a unified API covering all the aggregation operations

92 of 108

Visium HD bins are in a grid, allowing us to represent them as an image

a.

  • Visium HD data has millions of bins
  • Instead of storing them as vector data, images would be more performant

Visium HD

2µm, 8µm, 16µm bins

Visium HD Spatial Gene Expression Library, Mouse Small Intestine (FFPE)

93 of 108

Visium HD bins are in a grid, allowing us to represent them as an image

a.

  • Visium HD data has millions of bins
  • Instead of storing them as vector data, images would be more performant

Visium HD

2µm, 8µm, 16µm bins

Visium HD Spatial Gene Expression Library, Mouse Small Intestine (FFPE)

> rasterization is needed

94 of 108

Visium HD bins are in a grid, allowing us to represent them as an image

a.

  • Visium HD data has millions of bins
  • Instead of storing them as vector data, images would be more performant

Visium HD

2µm, 8µm, 16µm bins

Visium HD Spatial Gene Expression Library, Mouse Small Intestine (FFPE)

(x1, y1)

(x2, y2)

Rasterization

+

Bins as vector data

Bins as rasterized data

Gene 1

Gene 2

Gene 3

> rasterization is needed

1

1

2

2

3

4

4

4

5

95 of 108

Naive rasterization not possible. Solution: on-demand rasterization for binned data

Rasterization is already supported in SpatialData

Visium HD data is composed of small bins of 2x2 microns

Challenge:

About 6M bins and 30K genes per bin would represent 180GB as a dense image

> Solution: on-demand rasterization via rasterize_bins()

Visium HD Spatial Gene Expression Library, Mouse Small Intestine (FFPE)

96 of 108

Naive rasterization not possible. Solution: on-demand rasterization for binned data

Rasterization is already supported in SpatialData

Visium HD data is composed of small bins of 2x2 microns

Challenge:

About 6M bins and 30K genes per bin would represent 180GB as a dense image

> Solution: on-demand rasterization via rasterize_bins()

Visium HD Spatial Gene Expression Library, Mouse Small Intestine (FFPE)

97 of 108

Naive rasterization not possible. Solution: on-demand rasterization for binned data

Challenge:

About 6M bins and 30K genes per bin would represent 180GB as a dense image

> Solution: on-demand rasterization via rasterize_bins()

Visium HD data is composed of small bins of 2x2 microns

rasterize_bins() in action! Displayed on Napari

Visium HD Spatial Gene Expression Library, Mouse Small Intestine (FFPE)

98 of 108

Naive rasterization not possible. Solution: on-demand rasterization for binned data

Challenge:

About 6M bins and 30K genes per bin would represent 180GB as a dense image

> Solution: on-demand rasterization via rasterize_bins()

Visium HD data is composed of small bins of 2x2 microns

rasterize_bins() in action! Displayed on Napari

Visium HD Spatial Gene Expression Library, Mouse Small Intestine (FFPE)

99 of 108

rasterize_bins() preserves spatial alignment

1a. the pixel size is different for the various bins (2µm, 8µm, …)

2. For multi-slides runs, it is crucial to have a coordinate-system-aware rasterization.

NB: this image is showing Visium slides, not Visium HD

Figure adapted from Erikson et al. (Nat. 2022), Marconato et al. (Nat. Met. 2024)

> rasterize_bins() handles all of this

1b. the image origin should be aligned with the vector data

1c. The Visium HD bins are slightly rotated:

Not a problem for alignment, but may lead to Moiré patterns if rasterized

(x1, y1)

(x2, y2)

Small bins as vector data

2 microns bins as rasterized data

Gene 1

Gene 2

Gene 3

1

1

2

2

3

4

4

4

5

(x1, y1)

(x2, y2)

8 microns bins as vector data

100 of 108

100

Challenge:

  • Visium HD: about 6M bins and 30K genes require 300MB in sparse format, but 180GB as a dense image.
  • Stereo-seq: up to 70M bins, 2.1TB as a dense image

Solution:

  • on-demand rasterization via SpatialData.rasterize_bins()
  • The resulting images preserves spatial alignment (pixel sizes, rotations, etc.)

Performant representation of “bins”-based technologies

Storage of sparse data is convenient, but images are preferred for visualization. E.g. 6M bins = slow to visualize; 6M pixels = very fast

Visium HD data is composed of small bins of 2x2 microns

Figure adapted from the Visium HD paper [1]

Other popular binned technologies:

  • Stereo-seq
  • Open-ST

(x1, y1)

(x2, y2)

Rasterization

+

Bins as vector data

Bins as rasterized data

Gene 1

Gene 2

Gene 3

1

1

2

2

3

4

4

4

5

101 of 108

The scverse consortium pursues long-term software maintenance

First scverse conference (September 2024)

scverse hackathon (Innsbruck 2022)

scverse hackathon (Heidelberg 2023)

Contributing upstream simplifies long-term maintenance (EuroSciPy, August 2024)

Core developers and contributors

2025 plan

  • Paris hackathon
  • Bay Area hackathon

Get involved!

(both for academia and industry)

102 of 108

The scverse consortium pursues long-term software maintenance

Core developers and contributors

103 of 108

The scverse consortium pursues long-term software maintenance

Contributing upstream simplifies long-term maintenance (EuroSciPy, August 2024)

Core developers and contributors

104 of 108

The scverse consortium pursues long-term software maintenance

scverse hackathon (Innsbruck 2022)

Contributing upstream simplifies long-term maintenance (EuroSciPy, August 2024)

Core developers and contributors

105 of 108

The scverse consortium pursues long-term software maintenance

scverse hackathon (Innsbruck 2022)

scverse hackathon (Heidelberg 2023)

Contributing upstream simplifies long-term maintenance (EuroSciPy, August 2024)

Core developers and contributors

106 of 108

The scverse consortium pursues long-term software maintenance

First scverse conference (September 2024)

scverse hackathon (Innsbruck 2022)

scverse hackathon (Heidelberg 2023)

Contributing upstream simplifies long-term maintenance (EuroSciPy, August 2024)

Core developers and contributors

107 of 108

The scverse consortium pursues long-term software maintenance

First scverse conference (September 2024)

scverse hackathon (Innsbruck 2022)

scverse hackathon (Heidelberg 2023)

Contributing upstream simplifies long-term maintenance (EuroSciPy, August 2024)

Core developers and contributors

2025 plan

  • Paris hackathon
  • Bay Area hackathon

Get involved!

(both for academia and industry)

108 of 108

Conclusions and acknowledgements

Wolfgang Huber

Josh Moore (OME, GerBi)

Kevin Yamauchi (ETH)

Yvan Saeys (UGhent)

DKFZ

Ilia Kats

Tobias Graf

Moritz Gerstung

Elyas Heidari

  • established interoperable format for

spatial omics based on OME-NGFF

Isaac Virshup

Josh Moore

Kevin Yamauchi

Luca Marconato

EMBL

Oliver Stegle

Luca Marconato

Sinem Saka

Wouter-Michiel Vierdag

Wolfgang Huber

Harald Vöheringer

Constantin Ahlmann-Eltze

Mike Smith

Helena Crowell (CNAG)

Nils Eiling (UZH)

Will Moore (OME, UDundee)

Quentin Blampey (UParis Saclay)

Florian Wünnemann (UKHD)

Mark Keller (HMS)

10x Genomics team

CZI cellxgene team

Funded by

Isaac Virshup

Josh Moore

Kevin Yamauchi

Giovanni

Palla

Lotte Pollaris (UGhent)

Benjamin Rombaut (UGhent)

Arne Defauw (VIB)

Grzegorz Bokota (UW)

Christian Tischer (EMBL)

Andreas S. Eisenbarth (EMBL)

Omer Bayraktar (Sanger)

Tong Li (Sanger)

Ilan Gold (Helmholtz)

Giovanni Palla

Kevin Yamauchi

Isaac Virshup

First authors are underlined

Helmholtz Munich

Fabian Theis

Giovanni Palla

Isaac Virshup

Tim Treis

Sonja Stockhaus

Laurens Lehner

Marcella Toth

Rahul Shrestha

  • in-memory multimodal representation
  • processing, visualization
  • scales to large datasets

Wouter-Michiel Vierdag

Tim Treis

Marcela Toth

Rahul B. Shrestha

Harald Vöringer

Sonja Stockhaus

Laurens Lehner

Quentin

Blampey

Sinem Saka

Josh Moore

Nature Methods, 2024

Elyas Heidari

Luca

Marconato

Moritz

Gerstung

Fabian Theis

Oliver

Stegle

Benjamin Rombaut

Lotte

Pollaris

Yvan Saeys

Kasia Kedziora