UncertaINR: Uncertainty Quantification

of End-to-End Implicit Neural Representations

for Computed Tomography

Francisca Vasconcelos*

Oxford & UC Berkeley

Bobby He*

Oxford

Nalini Singh

MIT

Published in the Transactions on Machine Learning Research (April 2023).

Yee Whye Teh

Oxford

* denotes equal author contribution

Problem Motivation,

Background, and Setup

2

Problem Setting: Computed Tomography (CT)

Measurement

Image Reconstruction

(A non-trivial inverse mapping...)

3

Image Copyright Massachusetts Medical Society.

Problem Motivation

• CT and other radiation-based imaging platforms are commonly used in medical imaging
• 70+ million CT scans are performed in the US annually¹
• Imaging tradeoff: more measurements improve image reconstruction quality, but expose the patient to increased radiation
• an estimated ~29,000 current/future cancer cases result from CT scans performed in the US in 2007²

• Additionally, uncertainty estimates can be used by doctors as additional information about reconstruction quality and to improve measurement collection.

4

⟹ Algorithm goal: achieve highest possible image reconstruction quality with as few possible measurements.

⟹ Algorithm goal: achieve calibrated uncertainty quantification of the reconstructed image.

¹ González et al. Projected cancer risks from computed tomographic scans performed in the United States in 2007. Archives of internal medicine (2009).

² Smith-Bindman et al. Radiation Dose Associated with Common Computed Tomography Examinations and the Associated Lifetime Attributable Risk of Cancer. Archives of Internal Medicine (2009).

CT Image Reconstruction with Uncertainty

5

Reconstruction

Algorithm

CT Image Reconstruction with Uncertainty

6

Reconstruction

Algorithm

?

Implicit Neural Representations (INRs)

7

Encoding CT Image Reconstruction in End-to-End INRs

8

CT Image Reconstruction with Uncertainty

9

Reconstruction

Algorithm

✓

CT Image Reconstruction with Uncertainty

10

Reconstruction

Algorithm

✓

?

Uncertainty Quantification of End-to-End INRs (UncertaINR)

11

?

INR Uncertainty Quantification Methods

12

Experimental Methods, �Results, and Conclusions

13

Performance Metrics

Peak Signal-to-Noise Ratio (PSNR)

& Signal-to-Noise Ratio (SNR)

Negative Log Likelihood

(NLL)

Expected Calibration Error

(ECE)

Measures reconstruction accuracy.

Measures reconstruction accuracy and calibration.

Measures reconstruction calibration.

vs

14

Experiment #1:

Shepp-Logan Ablation Study

15

Shepp-Logan Data Set

Artificially generated 2D Shepp-Logan data phantoms.

16

Baselines

Implemented standard medical imaging reconstruction techniques:

FBP - Filtered Backprojection

CGLS - Conjugate Gradient Least Squares

EM - Expectation Maximization

SIRT - Simultaneous Iterative Recon. Tech.

SART - Simultaneous Algebraic Recon. Tech.

Classical reconstruction techniques struggle in very low measurement regimes.

17

Experiments

Performed first large-scale study of hyperparameters for INRs with uncertainty, in very low data regimes.

Hyperparameters considered:

Model - width, depth, activation function, RFF frequency (Ω₀)

BBB - KL factor, prior standard deviation

MCD - probability of dropout

Note: computationally not feasible to fully hyper-tune HMC.

18

5 and 20 views

Results

MCD significantly outperformed BBB.

Deep ensembles of 5-10 MCD base learners achieved the best overall performance.

UINR significantly outperformed classical reconstruction approaches.

Table 1.

Results

20-View uncertainty INR reconstruction output.

20

Experiment #2:

AAPM Model Assessment

21

AAPM Data Set

Abdominal CT scan images, provided by the Mayo Clinic for the �American Association of Physicists in Medicine (AAPM) 2016 Low-Dose CT Grand Challenge.

22

Reconstruction Accuracy Baselines

COIL-Enhanced Approaches:

• COIL with FBP*
• COIL with FISTA-TV*
• COIL with FBP-Unet*
• COIL with GM-RED*

Standard Medical Imaging Reconstruction:

• FBP - Filtered Backprojection
• CGLS - Conjugate Gradient Least Squares
• EM - Expectation Maximization
• SIRT - Simultaneous Iterative Recon. Tech.
• SART - Simultaneous Algebraic Recon. Tech.
• FISTA-TV* - Fast Iterative Shrinkage-Thresholding Algo. with TV-Regularization

SOTA Deep-Learning Approaches:

• FBP-Unet*
• GM-RED*

23

* denotes results taken from CoIL: Coordinate-based Internal Learning for Tomographic Imaging. 2021 IEEE Transactions on Computational Imaging.

Note: These methods require substantially more training data than UINR, so we do not expect to perform as well as these methods

Experiments

Reconstruction-Only Methods:

• Implicit Neural Representation (INR)

​

Reconstruction with Uncertainty:

• GOP with HMC
• DE of INRs
• UINR with MCD
• DE of UINRs with MCD
• HMC UINR

​

• Grid-of-Pixels (GOP)

24

Results

MCD generally outperformed HMC.

Despite the unfair data comparison, UINR is competitive with SOTA approaches.

UINRs significantly outperformed classical recon.

UINR outperformed GOP recon. and uncertainty.

Table 2.

UINRs outperformed standard INRs.

DEs of MCD UINRs performed the best, especially in the low-view setting.

Results

26

Concluding Remarks - Project Summary

• Introduced UncertaINR: a Bayesian end-to-end INR framework.
• Performed first study of uncertainty quantification for INRs.
• Large ablation studies.
• Found MCD (simple, efficient) outperforms ‘gold-standard’ HMC (computationally intensive).
• UINR achieves:
• better reconstruction performance than classical medical reconstruction techniques
• competitive performance with SOTA NNs (require substantially more training data)
• better uncertainty quantification than voxel-based approaches.

Concluding Remarks - Broader Impact

• CT image reconstruction requiring less data reduces patient exposure to radiation.
• Uncertainty can be used to inform doctor diagnoses (think variance images).
• Uncertainty can be used perform informed measurement collection (think active learning).

Thanks for watching!

UncertaINR: Uncertainty Quantification of End-to-End

Implicit Neural Representations for Computed Tomography

Published in the Transactions on Machine Learning Research (April 2023)

Make sure to check out the paper and project Github! :)

Paper: https://arxiv.org/abs/2202.10847

Github: https://github.com/bobby-he/uncertainr

OpenReview: https://openreview.net/forum?id=jdGMBgYvfX

29