1 of 15

Xun Huan

Mechanical Engineering

University of Michigan

xhuan@umich.edu

https://uq.engine.umich.edu

A Brief Introduction to

Uncertainty Quantification

Day 2 Workshop:

Climate Change Impacts On Flood Risks and Decision Making

2 of 15

How much can we rely on predictions and forecasts?

What is the probability that this will happen? Or something different?
What are the resulting effects on floods, and the danger to our infrastructure and personnel?
How can these results be conveyed in an understandable and useful manner?
How do we decide what to do? (Imagine, having around 60 hours lead time at a site in Texas)

3 of 15

Uncertainty quantification

This talk will focus on uncertainty quantification (UQ)
UQ seeks to capture the probability of different possible outcomes
Uncertainty arises from:

Lack of knowledge (epistemic uncertainty)
‘Natural’ variability (aleatoric uncertainty)

Crucial for assessing risk, and for decision-making

4 of 15

“… learning is achieved, not by mere theoretical speculation on the one hand, nor by the undirected accumulation of practical facts on the other, but rather by a motivated iteration between theory and practice…”

Learning is an iterative (never-ending) process

— George E. P. Box

Box. “Statistics and Science”, J. American Statistical Association, 71(356):791-799, 1976.

5 of 15

Experiments and modeling in engineering and science

Argonne National Lab

Collab with K. Sienko; human subject experiments

doi: 10.1186/s12984-017-0339-6

Electrolyte reservoirs

Cell

Pump

NCLS

CLS

Collab with D. Kwabi; design experiments for battery capacity fade

Experiments

Models

6 of 15

A broad view of uncertainty quantification (UQ)

Theory

Model

Data

Product, Design, Decision

Prediction

Experiment

Uncertainty Propagation

Optimization Under Uncertainty

Optimal Experimental Design

Uncertainty “Reduction”

(Inference, Estimation, Calibration, Training, Data Assimilation)

Many possible sources of uncertainty:

Model inputs and parameters
Data targets/labels (quality, quantity, informativeness)

Model form/structure (physics, equations, geometry)
Usage/users, human factors
Other assumptions and uncaptured relationships

7 of 15

“Uncertainty is everywhere and you cannot escape from it.”

Uncertainty quantification research

— Dennis V. Lindley

Lindley. Understanding Uncertainty, 2014.

Uncertainty quantification (UQ) focuses on systematic approaches to bridge together data and models, and allows us to:

characterize
incorporate
propagate
reduce

uncertainty in complex engineering systems

How much can we `trust’ a prediction, and how to improve it?

How do we make decisions in the presence of uncertainty?

8 of 15

Uncertainty propagation and sensitivity analysis

9 of 15

Uncertainty reduction through Bayesian inference

Bayes’ rule:

(Bayesian Inference)

Observation Data

Prior

Likelihood

Posterior

Marginal likelihood

Thomas Bayes

10 of 15

Uncertainty reduction and robust design optimization

11 of 15

Optimal experimental design

12 of 15

Real-time detection of rotorcraft icing using DNNs

DNN Model

Acoustic Measurements

Performance

Metrics

13 of 15

UQ for machine learning

UQ for machine learning models:

Treat ML as the “Model” in the UQ flowchart, and perform UQ tasks for it
Example: Bayesian neural networks

Challenge: extremely high-dimensional (millions)

[e.g., RNNs, CNNs]

Techniques: mean-field variational inference, Stein variational gradient descent, etc.

Deep Neural Network

Bayesian Neural Network

Bayesian Inference

14 of 15