Uncertainty
1
Objective
2
Understand the importance of uncertainty modelling
Be aware of possible sources of uncertainty
Know of applications in which cases modelling of uncertainty can be useful
Know of classical ways of assessing uncertainty
Understand the needs for sampling methods and approximations
Why need for uncertainty assessment
3
Knowing limits of prediction
Likelihood to be wrong
Course of action
When to abstain from using result
Applications:
- Diagnostic -> impact on treatment
- Segmentation -> Surgical planning
Example diagnosis course
4
https://pubmed.ncbi.nlm.nih.gov/37821686/
Labelling uncertainty
5
Artefacts and uncertainty
6
https://arxiv.org/pdf/2109.02413.pdf
Different types of uncertainty
7
Aleatoric
Epistemic
Subtypes of aleatoric uncertainty
8
Homoscedastic
Heteroscedastic
Ensembling
9
Bootstrap / Bootstrap aggregation
Take N samples with replacement
Evaluate statistics of interest for these N samples / train model for these N samples
Do this k times
Average across the k trials / Calculate variance / confidence interval of estimate across k trials
Typical bagging algorithm:
Random Forests!!
Modeling / sensing of uncertainty
10
Considering model parameters as samples of a distribution and evaluating output according to these samples
Assessing model variability – different outputs based on different inputs
Assessing risk associated with predicted probability
Considering different models and sampling outputs from these models
Bayesian modelling
11
Model parameters are considered as random variables
Data helps refining the parameter distribution
New data distribution given training data
Analytically difficult
Bayesian linear regression
12
https://www.inf.ed.ac.uk/teaching/courses/mlpr/2016/notes/w7b_gaussian_processes.html
Bayesian regression and uncertainty in diffusion MRI
13
https://www.sciencedirect.com/science/article/pii/S1053811918302696?casa_token=dQVirkrBE8wAAAAA:AOdtRHpC0Ed8XaNtEohLYgjDmTDH7jdhYcFcVw07kdt6EnrpP-zKOpChWq6Jw9HZGAxtfXY1No0
Residual variance to assess uncertainty of prediction
Used for weighting samples (subject and voxel)
From Bayesian regression to Gaussian processes
14
Gaussian processes
15
Distribution over functions
Assumption of continuity
If x close to x’, f(x) close to f(x’)
Infinite multivariate Gaussian distribution over function values
Entirely defined by mean and covariance
Covariance symmetric definite positive
Gaussian processes
16
f(x)
x
x1 x2
x3
f1
f
2
f3
x
x1 x2
x3
f(x)
f3
f2 f1
New sample prediction / multiple samples
17
Crucial choice of covariance matrix
18
Possibility to optimize using training data
19
Gaussian processes and associated uncertainty in medical imaging
20
Age prediction and identification of outliers
Uncertainty as out-of-distribution identifier
https://www.sciencedirect.com/science/article/pii/S1053811918302854?casa_token=7Tpl3XhCVakAAAAA:8hTlEetvojqdfEZfPCwmwpzTft6BsqGDeld99hA98N9CYSlPvUOtdaTM6xSfh-nkkihmDHvBXHY
Gaussian process – Prediction of future
21
https://link.springer.com/chapter/10.1007/978-3-319-19992-4_49
Variational inference
22
Evidence lower bound - ELBO
Use of factorization for simplification
Sampling
23
Used for large multi-dimensional, complex to model distribution.
Create approximation of distribution to be used for generation of integral statistics (mean, variance)
No need for knowledge of normalization factor
Ex: Metropolitan Hastings
Summary
24