Noise2Stack: �Improving Image Restoration by Learning from Volumetric Data
Mikhail Papkov, Kenny Roberts, Lee Ann Madissoon, Jarrod Shilts, Omer Bayraktar, Dmytro Fishman, Kaupo Palo, and Leopold Parts
MLMIR 2021
1
01.10.2021
Agenda
2
Motivation
3
No data is absolutely clean
...especially in biology:
4
Noise is problematic
...for both manual annotation and existing automated pipelines (such as pre-trained neural networks), because
5
Garbage in
Garbage out
But we can do better
Segmentation
Data
6
MRI dataset
7
Noise generation
8
FFT
Sample
10%
IFFT
Post-processing opportunity
All the frequencies that are present in the image are there for sure (because of the frequency sampling procedure) — copy-paste them to the output spectrum
9
Ctrl-C Ctrl-V
Input spectrum
Denoised spectrum
Microscopy
10
Denoising with neural networks
11
U-Net
An encoder-decoder architecture with skip connections, widely used for denoising and segmentation tasks (hyperparameters could vary)
Different configurations for MRI and microscopy from the literature
12
U-Net
The exact architecture is heavier
13
Noise2Clean
Use clean (noise-free, high-resolution) images as training targets (ground truth).
Problem: clean ground truth is a rare thing in biomedical imaging.
14
Input
Output
Target
MSE
Noise2Noise
Use independent noisy acquisitions of the same image as both input and target
Problem: many existing datasets do not have a second copy
15
Input
Output
Target
MSE
Noise2Noise
For MSE loss the minimum is found in expected value of each pixel
16
Input
Output 1
Output 1
Output N
Target
MSE
Training
Noise2Noise as a baseline
Previous works show that Noise2Noise performs on par with traditional denoising (Noise2Clean) and outperforms existing self-supervised methods (e.g., Noise2Void)
Self-supervised methods does not work with structured noise
17
Noise2Stack
18
Noise2Stack
In Noise2Noise every plane in a stack is treated independently on its position
19
Noise2Stack
In Noise2Stack we sample neighbors along with the target plane (copy-supervised mode) or without it (self-supervised mode)
20
Results
21
Metrics
22
MRI
23
MRI
24
Input
Noise2Noise baseline
Noise2Stack (self-supervised)
Noise2Stack (copy-supervised)
Ground truth
Stack
Microscopy
25
Downstream segmentation
26
Plane denoising
In MRI, middle planes are harder to denoise (less background, symmetry point).
In microscopy, marginal planes are noisier.
27
Conclusion
28
Conclusion
29
Acknowledgements
This work was funded by PerkinElmer Inc. (VLTAT19682) and Wellcome Trust (206194). We thank High Performance Computing Center of the Institute of Computer Science at the University of Tartu for the provided computing power.
30
31
Plane similarity in volumetric images
Neighboring planes have a lot in common
Can this shared information be used for denoising?
32