1 of 89

Data Driven Modeling for Climate/Weather Research

Will Chapman

Climate and Global Dynamics Lab

NSF - National Center for Atmospheric Research

Dec. 09, 2024

2 of 89

Who am I?

Will Chapman - Project Scientist at NCAR in the Climate and Global Dynamics Lab (AMP / CAM)

Background:

  • UCSD undergraduate (Environmental Engineering).
  • Stanford (M.S.) civil engineering
  • Scripps Institution of Oceanography (Climate Science, Ph.D.)
  • NCAR’s ASP postdoctoral Fellowship program*
  • Multiscale Machine Learning in Coupled Earth System Modeling (M2lines) postdoctoral Fellow*
  • NCAR Project Scientist

​

Email: wchapman@ucar.edu

Website: willychap.github.io

Github: https://github.com/WillyChap

​

​

*concurrent

​

What I work on:

Currently:

  • Hybrid Modeling & Machine Learning: coupling our global climate models and ‘machine learned’ physics parameterization adjustments
  • Supermodelling: In a supermodel individual models exchange state information as they run, influencing each other’s behavior.
  • Climate/Weather model emulation

Previously:

  • NWP forecast post-processing (Machine Learning)
  • Forecast uncertainty development/quantification
  • Signal-to-Noise at subseasonal to seasonal time scales
  • The dynamics of the Madden-Julian Oscillation & ENSO teleconnections

2

3 of 89

“Introduce the DCMIP participants to basic Machine Learning ideas and how machine learning is used in atmospheric science and modeling (e.g.: for post processing, in hybrid physics-ML models, full emulators, for process understanding, etc). Basically, this should be a high-level overview that should introduce various ML techniques, terminology, ML workflows, and applications.”

What am I talking about for the next hour:

3

4 of 89

Quick Poll

5 of 89

What is Machine Learning?

6 of 89

What is Machine Learning?

Machine Learning: Any optimized/optimizable statistical linking function that maps an input space to a desired (or targeted) output.

7 of 89

Input Vector (x)

 

Output Vector (y)

 

8 of 89

Input Vector (x)

 

Output Vector (y)

 

Some Function

9 of 89

 

Input Vector (x)

 

Output Vector (y)

 

10 of 89

 

Input Vector (x)

 

Output Vector (y)

 

 

11 of 89

 

Input Vector (x)

 

Output Vector (y)

 

 

 

 

12 of 89

 

Input Vector (x)

 

Output Vector (y)

 

 

 

 

*garbage

13 of 89

 

Input Vector (x)

 

Output Vector (y)

 

 

 

 

14 of 89

 

Input Vector (x)

 

Output Vector (y)

 

 

 

 

 

15 of 89

 

 

Input Vector (x)

 

Output Vector (y)

 

 

 

 

 

GOAL:

16 of 89

 

Input Vector (x)

 

Output Vector (y)

 

 

 

 

How?:

How?:

 

17 of 89

 

Input Vector (x)

 

Output Vector (y)

 

 

 

 

 

How?:

 

*less garbage

18 of 89

AI, Machine Learning, Deep Learning

Linear regression

Decision Tree

Neural Network

Chatbots

ARTIFICIAL INTELLIGENCE

Programs with the ability to mimic human intelligence

MACHINE LEARNING

Any optimized/optimizable statistical linking function that maps an input space to a desired (or targeted) output.

DEEP LEARNING

Artificial Neural Networks

(Machine learning on steroids)

19 of 89

Explosion of Machine Learning in Atm./Ocean Science

Fig. 4.

Number of AI presentations at the AMS AI conferences through time including a large increase starting in 2018. Non-AI-focused presentations are not included. The 2021 conference was impacted by the COVID-19 pandemic.

This interest in DL for ES problems that skyrocketed in the mid-2010s led to the AI Conference hosting dedicated DL sessions that covered the wide variety of applications where it has been applied. One remarkable aspect of DL adoption is how quickly some highly complex methods were widely adopted across the research community after being introduced the previous year.

20 of 89

  1. equations of motion
  2. equations of state
  3. thermodynamic equation
  4. mass balance equation
  5. water balance equation

For a continuous medium consisting of an ideal gas, (or mixture of ideal gases) these equations are derived from first principles and are certain

Fundamental set of equations:

  1. Discretize the Atm/Ocn into grid boxes (defines the smallest scales that can be resolved)
  2. Properties are considered uniform w/in each box
  3. Equations are marched forward in time.

Quick Primer on Traditional Prediction of Weather / Climate

21 of 89

How Large of a Box?

  • Computing Power
  • Domain size?
  • Length of forecasting window

‘Dynamics’ Step

‘Physics’ Step

time 1

time 2

~fluid flow + thermo

~subgrid scales

Practically, how is this done?

22 of 89

State of the Art Climate Model (NCAR’s CAM6):

Challenges:

Practically, how is this done?

  1. Computationally expensive ( This scales with resolution and ensemble size)
  2. O(1million) of lines of code
    1. Unbelievable amount of expertise required to “understand” a model.
  3. Only observationally constrained by data assimilation (+ mean forcing)
  4. Physics parameterizations contain known errors
  5. Numerical Approximations

23 of 89

Challenges:

  • FourCastNet (Kurth et al 2022)
  • Pangu Weather (Bi et al. 2022)
  • GraphCast (Lam et al 2022)
  • DLWP (Weyn et al 2020,2021)

1.

2.

3/4.

Why is ML well posed to address these challenges?

  1. Computationally expensive ( This scales with resolution and ensemble size)
  2. O(1million) of lines of code
    1. High amount of expertise required to “understand” a model.
  3. Observationally constrained by data assimilation (+ forcing, some parameterizations)
  4. Physics parameterizations contain known errors
  5. Numerical Approximations

24 of 89

Nonlinear Mappings

https://www.stefanmesken.info/machine%20learning/tensorflow-without-tears/

  1. Computationally expensive ( This scales with resolution and ensemble size)
  2. O(1million) of lines of code
    1. High amount of expertise required to “understand” a model.
  3. Observationally constrained by data assimilation (+ forcing, some parameterizations)
  4. Physics parameterizations contain known errors
  5. Numerical Approximations

Challenges:

Why is ML well posed to address these challenges?

25 of 89

1. Model Replacement

4. Hybrid Modeling

  • What has the algorithm learned about the system?
  • XAI and Interpretable Machine Learning
  • How can ML compliment our current Numerical Weather Prediction & General Circulation models in online runs?
    • Improved dynamics or parameterizations
    • Improved data assimilation

2. Scientific Discovery

https://www.ecmwf.int/

Mayer et al 2021

Chapman et al 2019

3. Post-Model Run / Data Improvement

  • Model / Observation Downscaling
  • Post-processing (Improved performance, interpretation)
  • Communication / Risk Assessment
  • Mimicking the equations of motion in the atmosphere through pure ML

Courtesy of Laure Zanna

Four Areas of Machine Learning in Weather/Climate

Four Areas of Machine Learning in Weather/Climate

25

26 of 89

1. Model Replacement

4. Hybrid Modeling

  • What has the algorithm learned about the system?
  • XAI and Interpretable Machine Learning
  • How can ML compliment our current Numerical Weather Prediction & General Circulation models in online runs?
    • Improved dynamics or parameterizations
    • Improved data assimilation

2. Scientific Discovery

https://www.ecmwf.int/

Mayer et al 2021

Chapman et al 2019

3. Post-Model Run / Data Improvement

  • Model / Observation Downscaling
  • Post-processing (Improved performance, interpretation)
  • Communication / Risk Assessment
  • Mimicking the equations of motion in the atmosphere through pure ML

Courtesy of Laure Zanna

Four Areas of Machine Learning in Weather/Climate

Four Areas of Machine Learning in Weather/Climate

26

27 of 89

1. Model Replacement

4. Hybrid Modeling

  • What has the algorithm learned about the system?
  • XAI and Interpretable Machine Learning
  • How can ML compliment our current Numerical Weather Prediction & General Circulation models in online runs?
    • Improved dynamics or parameterizations
    • Improved data assimilation

2. Scientific Discovery

https://www.ecmwf.int/

Mayer et al 2021

Chapman et al 2019

3. Post-Model Run / Data Improvement

  • Model / Observation Downscaling
  • Post-processing (Improved performance, interpretation)
  • Communication / Risk Assessment
  • Mimicking the equations of motion in the atmosphere through pure ML

Courtesy of Laure Zanna

Four Areas of Machine Learning in Weather/Climate

27

28 of 89

1. Model Replacement

4. Hybrid Modeling

  • What has the algorithm learned about the system?
  • XAI and Interpretable Machine Learning
  • How can ML compliment our current Numerical Weather Prediction & General Circulation models in online runs?
    • Improved dynamics or parameterizations
    • Improved data assimilation

2. Scientific Discovery

https://www.ecmwf.int/

Mayer et al 2021

Chapman et al 2019

3. Post-Model Run / Data Improvement

  • Model / Observation Downscaling
  • Post-processing (Improved performance, interpretation)
  • Communication / Risk Assessment
  • Mimicking the equations of motion in the atmosphere through pure ML

Courtesy of Laure Zanna

Four Areas of Machine Learning in Weather/Climate

Four Areas of Machine Learning in Weather/Climate

28

29 of 89

1. Model Replacement

4. Hybrid Modeling

  • What has the algorithm learned about the system?
  • XAI and Interpretable Machine Learning
  • How can ML compliment our current Numerical Weather Prediction & General Circulation models in online runs?
    • Improved dynamics or parameterizations
    • Improved data assimilation

2. Scientific Discovery

https://www.ecmwf.int/

Mayer et al 2021

Chapman et al 2019

3. Post-Model Run / Data Improvement

  • Model / Observation Downscaling
  • Post-processing (Improved performance, interpretation)
  • Communication / Risk Assessment
  • Mimicking the equations of motion in the atmosphere through pure ML

Courtesy of Laure Zanna

Four Areas of Machine Learning in Weather/Climate

Four Areas of Machine Learning in Weather/Climate

29

30 of 89

1. Model Replacement

4. Hybrid Modeling

  • What has the algorithm learned about the system?
  • XAI and Interpretable Machine Learning
  • How can ML compliment our current Numerical Weather Prediction & General Circulation models in online runs?
    • Improved dynamics or parameterizations
    • Improved data assimilation

2. Scientific Discovery

https://www.ecmwf.int/

Mayer et al 2021

Chapman et al 2019

3. Post-Model Run / Data Improvement

  • Model / Observation Downscaling
  • Post-processing (Improved performance, interpretation)
  • Communication / Risk Assessment
  • Mimicking the equations of motion in the atmosphere through pure ML

Courtesy of Laure Zanna

Four Areas of Machine Learning in Weather/Climate

Four Areas of Machine Learning in Weather/Climate

30

31 of 89

1. Model Replacement

4. Hybrid Modeling

  • What has the algorithm learned about the system?
  • XAI and Interpretable Machine Learning
  • How can ML compliment our current Numerical Weather Prediction & General Circulation models in online runs?
    • Improved dynamics or parameterizations
    • Improved data assimilation

2. Scientific Discovery

https://www.ecmwf.int/

Mayer et al 2021

Chapman et al 2019

3. Post-Model Run / Data Improvement

  • Model / Observation Downscaling
  • Post-processing (Improved performance, interpretation)
  • Communication / Risk Assessment
  • Mimicking the equations of motion in the atmosphere through pure ML

Courtesy of Laure Zanna

Four Areas of Machine Learning in Weather/Climate

Four Areas of Machine Learning in Weather/Climate

EXCITING!!!!

EXCITING!!!!

EXCITING!!!!

Old & Boring!!!

31

32 of 89

1. Model Replacement

4. Hybrid Modeling

  • What has the algorithm learned about the system?
  • XAI and Interpretable Machine Learning
  • How can ML compliment our current Numerical Weather Prediction & General Circulation models in online runs?
    • Improved dynamics or parameterizations
    • Improved data assimilation

2. Scientific Discovery

https://www.ecmwf.int/

Mayer et al 2021

Chapman et al 2019

3. Post-Model Run / Data Improvement

  • Model / Observation Downscaling
  • Post-processing (Improved performance, interpretation)
  • Communication / Risk Assessment
  • Mimicking the equations of motion in the atmosphere through pure ML

Courtesy of Laure Zanna

Four Areas of Machine Learning in Weather/Climate

Four Areas of Machine Learning in Weather/Climate

🤠Wild-West🐴

🤠Wild-West🐴

👨🏼‍🔬Rigorous👩🏽‍🔬

🤠Wild-West🐴

32

33 of 89

1. Model Replacement

4. Hybrid Modeling

  • What has the algorithm learned about the system?
  • XAI and Interpretable Machine Learning
  • How can ML compliment our current Numerical Weather Prediction & General Circulation models in online runs?
    • Improved dynamics or parameterizations
    • Improved data assimilation

2. Scientific Discovery

https://www.ecmwf.int/

Mayer et al 2021

Chapman et al 2019

3. Post-Model Run / Data Improvement

  • Model / Observation Downscaling
  • Post-processing (Improved performance, interpretation)
  • Communication / Risk Assessment
  • Mimicking the equations of motion in the atmosphere through pure ML

Courtesy of Laure Zanna

Four Areas of Machine Learning in Weather/Climate

Four Areas of Machine Learning in Weather/Climate

33

34 of 89

X1

time 2

time 3

time N

F(X1)

…

Here F is a LARGE neural network and the model is

”rolled out” in an autoregressive framework:

​

​

If X0 is the model state at time zero then:

​

Xn = F(... F(F(F(F(X0)))

F(X2)

F(XN)

Model Replacement

35 of 89

X1

time 2

time 3

time N

…

Here F is a neural network and the model is

used as a “one shot” predictor:

​

​

​

Xn = F(X0)

Model Replacement

F(X0)

36 of 89

Motivation:

“Physics based models are slow, resource intensive, don’t handle subgrid parameterizations well, and thus are poor at representing upscale error growth so they are inaccurate. Machine learning models derive their dynamics from observations and thus can more accurately represent weather prediction.”

– intentionally inflammatory Machine Learning person.

37 of 89

Motivation: Electricity costs for one year of forecasts1

Based on experiments on HoReKa by J. Quinting, for electricity costs of 50 Cents/kWh, 40 ensemble members, 2 model runs per day for 365 days. – Courtesy of Sebastian Lerch, images DALLE

Physics based Ensemble:

~440,000 €

AI Ensemble:

38 of 89

Motivation: Electricity costs for one year of forecasts1

Based on experiments on HoReKa by J. Quinting, for electricity costs of 50 Cents/kWh, 40 ensemble members, 2 model runs per day for 365 days. – Courtesy of Sebastian Lerch, images DALLE

Physics based Ensemble:

~440,000 €

AI Ensemble:

~200 €

0.04% of energy cost

39 of 89

Motivation: Electricity costs for one year of forecasts1

Based on experiments on HoReKa by J. Quinting, for electricity costs of 50 Cents/kWh, 40 ensemble members, 2 model runs per day for 365 days. – Courtesy of Sebastian Lerch, images DALLE

Physics based Ensemble:

~440,000 €

AI Ensemble:

~200 €

Training ~10,000 €

40 of 89

NCAR’s Weather ML Model (CREDIT-WXformer)

41 of 89

NCAR’s Weather ML Model (CREDIT-WXformer)

42 of 89

Figure 3: An example of 2m temperature forecasts and a corresponding SYNOP observation. (a) Evolution plots showing forecasts valid at Sodankylä (Finland) on 22 February 2022: the 15-day ensemble forecast in the form of the ensemble mean and quantile forecasts (box-plots showing the 5%, 25%, 75%, and 95%), the 10-day PGW and IFS forecasts. (b) PGW forecast at day 2, (c) IFS forecast at day 2, (d) PGW forecast at day 6, (e) IFS forecast at day 6 for Europe with the location of the Sodankylä SYNOP station indicated with a red cross.

ECMWF - Model Replacement

42

43 of 89

Hakim et al. 2023

Idealized extratropical cyclone development over the North Pacific ocean (mean sea level pressure contours every 2 hPa; 850 hPa specific humidity anomalies in colorfill, g/kg)

Model Replacement

Does it know “physics”?

43

44 of 89

  • Conservation laws are not respected (~, yet).
  • The full state space is not being modeled (yet).
  • Long Range Stability
  • Model Smoothing
  • They are not suited to hypothesis testing.
  • A major stigma still exists, and there is no true field of Weather/Climate Data Science ( / it’s very nascent)
    • AI models behave differently from models based on physics, so understanding their predictions requires specialized training.

Challenges with Pure ML Models

Model Replacement

44

45 of 89

      �

  • AI models underestimate kinetic energy at longer wave numbers 
  • Energy loss occurs early but seems to stabilize around 5 days
  • 1-hour model does not lose energy between 1 and 5 days
  • Consistent with other deterministic AI NWP models

​

  1. AI models underestimate kinetic energy at longer wave numbers
  2. Energy loss occurs early but seems to stabilize around 5 days

Current Problems – Model Smoothing

46 of 89

Summary

  1. Surprising and tremendous progress in AI-based weather models over the past ~3 years.
  2. Possibly an unprecedented disruption to the evolution of weather prediction.
  3. AI weather models are here to stay (climate?)
    • But can’t yet fully replace full weather modeling systems
  4. Rapid progress, but many open questions remain
    • Extreme events
    • Changing climate
    • Forecast uncertainty
  5. Private industry has driven a lot of this progress
  6. Weather services are investing strongly

​

47 of 89

1. Model Replacement

4. Hybrid Modeling

  • What has the algorithm learned about the system?
  • XAI and Interpretable Machine Learning
  • How can ML compliment our current Numerical Weather Prediction & General Circulation models in online runs?
    • Improved dynamics or parameterizations
    • Improved data assimilation

2. Scientific Discovery

https://www.ecmwf.int/

Mayer et al 2021

Chapman et al 2019

3. Post-Model Run / Data Improvement

  • Model / Observation Downscaling
  • Post-processing (Improved performance, interpretation)
  • Communication / Risk Assessment
  • Mimicking the equations of motion in the atmosphere through pure ML

Courtesy of Laure Zanna

Four Areas of Machine Learning in Weather/Climate

Four Areas of Machine Learning in Weather/Climate

47

48 of 89

1. Model Replacement

4. Hybrid Modeling

  • What has the algorithm learned about the system?
  • XAI and Interpretable Machine Learning
  • How can ML compliment our current Numerical Weather Prediction & General Circulation models in online runs?
    • Improved dynamics or parameterizations
    • Improved data assimilation

2. Scientific Discovery

https://www.ecmwf.int/

Mayer et al 2021

Chapman et al 2019

3. Post-Model Run / Data Improvement

  • Model / Observation Downscaling
  • Post-processing (Improved performance, interpretation)
  • Communication / Risk Assessment
  • Mimicking the equations of motion in the atmosphere through pure ML

Courtesy of Laure Zanna

Four Areas of Machine Learning in Weather/Climate

Four Areas of Machine Learning in Weather/Climate

48

49 of 89

Develop data-informed, interpretable & generalizable subgrid physics models (ocean, ice, atm)

https://m2lines.github.io/

​

Hybrid ML Modeling Vision (m2lines)

Slide Courtesy of Laure Zanna

49

50 of 89

Develop data-informed, interpretable & generalizable subgrid physics models (ocean, ice, atm)

https://m2lines.github.io/

​

Hybrid ML Modeling Vision (m2lines)

Produce error corrections derived from observational products for climate components

Slide Courtesy of Laure Zanna

50

51 of 89

Slide Courtesy of Laure Zanna

‘Physics’ Step

~subgrid scales

Sources of Error in Climate Models

51

52 of 89

Slide Courtesy of Laure Zanna

‘Dynamics’ Step

~fluid flow + thermo

‘Physics’ Step

~subgrid scales

Sources of Error in Climate Models

52

53 of 89

  1. What is truth?
    1. Hi-resolution model?
    2. Observations?

​

​

  • Online Implementation (Language Barrier)
    • Python-Fortran Interface
    • Nonlocality in distributed HPC
      1. Stencils
    • GPU/CPU complexity
      • communication, performance, interface

IPCC (AR4 WG 1 Chapter 1 page 113 Fig. 1.4).

Challenges in Hybrid Modeling

53

54 of 89

A Review of ESM Hybrid Work

Link To Github

54

55 of 89

Develop data-informed, interpretable & generalizable subgrid physics models (ocean, ice, atm)

https://m2lines.github.io/

​

Hybrid ML Modeling Vision (m2lines)

Slide Courtesy of Laure Zanna

55

56 of 89

Importance of the Ocean Boundary Layer

56

57 of 89

Learning Vertical Mixing in the Ocean BL

Courtesy of Aakash Sane

Sane et al. 2023 JAMES

57

58 of 89

Learning Vertical Mixing in the Ocean BL

Courtesy of Aakash Sane

Sane et al. 2023 JAMES

58

59 of 89

ANN trained from advanced turbulence model

    • Implemented into MOM6 for OMIP simulations (1958-2022)
    • Adds physics within parameterization and contributes to reduce upper ocean biases

Key Features

    • Compact network -> Was coded in Fortran w/ minimal overhead (~5%)
    • Local space/time network: No additional stencil/storage requirements

Courtesy of Aakash Sane

Sane et al. 2023 JAMES

Improving Ocean Biases

59

60 of 89

https://m2lines.github.io/L96_demo/intro.html

https://github.com/m2lines/L96_demo

​

“The goal of this book is to conceptualize the problems associated with climate models within a simple and computationally accessible framework. We will introduce the readers to climate modeling by using a simple tool, the [Lorenz, 1995] (L96) two-timescale model. We discuss the numerical aspects of the L96 model, the approximate representation of subgrid processes (known as parameterizations or closures), and simple data assimilation problems (a data-model fusion method). We will then use the L96 results to demonstrate how to learn subgrid parameterizations from data with machine learning, and then test the parameterizations offline (apriori) and online (aposteriori), with a focus on the interpretability of the results.”

  • Balwada et al 2023

Co-Authors and I have a Book on Hybrid Modeling

60

61 of 89

1. Model Replacement

4. Hybrid Modeling

  • What has the algorithm learned about the system?
  • XAI and Interpretable Machine Learning
  • How can ML compliment our current Numerical Weather Prediction & General Circulation models in online runs?
    • Improved dynamics or parameterizations
    • Improved data assimilation

2. Scientific Discovery

https://www.ecmwf.int/

Mayer et al 2021

Chapman et al 2019

3. Post-Model Run / Data Improvement

  • Model / Observation Downscaling
  • Post-processing (Improved performance, interpretation)
  • Communication / Risk Assessment
  • Mimicking the equations of motion in the atmosphere through pure ML

Courtesy of Laure Zanna

Four Areas of Machine Learning in Weather/Climate

Four Areas of Machine Learning in Weather/Climate

61

62 of 89

1. Model Replacement

4. Hybrid Modeling

  • What has the algorithm learned about the system?
  • XAI and Interpretable Machine Learning
  • How can ML compliment our current Numerical Weather Prediction & General Circulation models in online runs?
    • Improved dynamics or parameterizations
    • Improved data assimilation

2. Scientific Discovery

https://www.ecmwf.int/

Mayer et al 2021

Chapman et al 2019

3. Post-Model Run / Data Improvement

  • Model / Observation Downscaling
  • Post-processing (Improved performance, interpretation)
  • Communication / Risk Assessment
  • Mimicking the equations of motion in the atmosphere through pure ML

Courtesy of Laure Zanna

Four Areas of Machine Learning in Weather/Climate

62

63 of 89

Earth System

Model

Using your neural net to be an earth science detective.

​

The best use cases are around VERY intentional question design.

63

64 of 89

64

65 of 89

65

66 of 89

Montavon et al. (2017), Pattern Recognition; Montavon et al. (2018), Digital Signal Processing

​

Prediction: CAT ✓

Layerwise Relevance Propagation

Slide courtesy of KJ Mayer

66

67 of 89

where the network looked to determine it was a “cat”

Montavon et al. (2017), Pattern Recognition; Montavon et al. (2018), Digital Signal Processing

​

Prediction: CAT ✓

Prediction: CAT ✓

Back propagate ‘Relevance’

Layerwise Relevance Propagation

Slide courtesy of KJ Mayer

67

68 of 89

Example use of LRP

Figure 3 from Lapuschkin et al. (2019)

​

Q: is there a horse in the image?

A: yes.

Q: why does the NN think that?

Slide courtesy of KJ Mayer

68

69 of 89

Example use of LRP

Figure 3 from Lapuschkin et al. (2019)

​

Q: is there a horse in the image?

A: yes.

Q: why does the NN think that?

​

A: COPYRIGHT!?

relevant region for the correct prediction

Slide courtesy of KJ Mayer

69

70 of 89

OUTGOING LONGWAVE RADIATION

(OLR)

X

LRP HEATMAP

X

MAYER & BARNES 2021

Slide courtesy of KJ Mayer

70

71 of 89

What are relevant physical structures of OLR in the tropics

for prediction over the North Atlantic?

71

72 of 89

LRP map (shading) highlights OLR anomalies (contours) that looks like MJO phases 3-4

Positive Predictions

+

MAYER & BARNES 2021

Slide courtesy of KJ Mayer

72

73 of 89

MAYER & BARNES 2021

NEGATive Predictions

LRP map (shading) highlights OLR anomalies (contours) that looks like MJO phases 7-8

–

Slide courtesy of KJ Mayer

73

74 of 89

1. Model Replacement

4. Hybrid Modeling

  • What has the algorithm learned about the system?
  • XAI and Interpretable Machine Learning
  • How can ML compliment our current Numerical Weather Prediction & General Circulation models in online runs?
    • Improved dynamics or parameterizations
    • Improved data assimilation

2. Scientific Discovery

https://www.ecmwf.int/

Mayer et al 2021

Chapman et al 2019

3. Post-Model Run / Data Improvement

  • Model / Observation Downscaling
  • Post-processing (Improved performance, interpretation)
  • Communication / Risk Assessment
  • Mimicking the equations of motion in the atmosphere through pure ML

Courtesy of Laure Zanna

Four Areas of Machine Learning in Weather/Climate

Four Areas of Machine Learning in Weather/Climate

74

75 of 89

1. Model Replacement

4. Hybrid Modeling

  • What has the algorithm learned about the system?
  • XAI and Interpretable Machine Learning
  • How can ML compliment our current Numerical Weather Prediction & General Circulation models in online runs?
    • Improved dynamics or parameterizations
    • Improved data assimilation

2. Scientific Discovery

https://www.ecmwf.int/

Mayer et al 2021

Chapman et al 2019

3. Post-Model Run / Data Improvement

  • Model / Observation Downscaling
  • Post-processing (Improved performance, interpretation)
  • Communication / Risk Assessment
  • Mimicking the equations of motion in the atmosphere through pure ML

Courtesy of Laure Zanna

Four Areas of Machine Learning in Weather/Climate

Four Areas of Machine Learning in Weather/Climate

75

76 of 89

Ob.

Fcst.

76

77 of 89

Ob.

Fcst.

77

78 of 89

Ob.

Fcst.

Bias

78

79 of 89

Ob.

Fcst.

79

80 of 89

“It was obvious [NWP] was the way of the future. It was also obvious it would be a long time before “real weather” was forecast by the models, as they concentrated on the upper atmosphere. It was also obvious this was the place for statistics to play a role. The idea of Model Output Statistics (MOS) was quickly born.”

The Earliest ML Adopter!

80

81 of 89

“It was obvious [NWP] was the way of the future. It was also obvious it would be a long time before “real weather” was forecast by the models, as they concentrated on the upper atmosphere. It was also obvious this was the place for statistics to play a role. The idea of Model Output Statistics (MOS) was quickly born.”

The Earliest ML Adopter!

Old & Boring!!!

👨🏼‍🔬& Rigorous👩🏽‍🔬

81

82 of 89

This paradigm hasn’t changed in 50 years, the models have just gotten more complicated (and probabilistic), and the statistics to evaluate a forecast have advanced!

82

83 of 89

83

84 of 89

Generative AI and postprocessing?

The new frontier of model post-processing is leveraging generative AI + physical constraints to generate new, unbiased, probabilistic forecasts conditioned on a current forecast state.

​

To the right is work done by Tim Higgins (CU Boulder) to generate forecasts of atmospheric rivers from pure noise.

84

85 of 89

Model Replacement

​

  • Kurth, Thorsten, et al. "Fourcastnet: Accelerating global high-resolution weather forecasting using adaptive fourier neural operators." Proceedings of the Platform for Advanced Scientific Computing Conference. 2023.
  • Lam, Remi, et al. "GraphCast: Learning skillful medium-range global weather forecasting." arXiv preprint arXiv:2212.12794 (2022).
  • Bi, Kaifeng, et al. "Pangu-weather: A 3d high-resolution model for fast and accurate global weather forecast." arXiv preprint arXiv:2211.02556 (2022).
  • Rasp, Stephan, et al. "WeatherBench: a benchmark data set for data‐driven weather forecasting." Journal of Advances in Modeling Earth Systems 12.11 (2020): e2020MS002203.
  • Rasp, Stephan, and Nils Thuerey. "Data‐driven medium‐range weather prediction with a resnet pretrained on climate simulations: A new model for weatherbench." Journal of Advances in Modeling Earth Systems 13.2 (2021): e2020MS002405.
  • Weyn, Jonathan A., Dale R. Durran, and Rich Caruana. "Can machines learn to predict weather? Using deep learning to predict gridded 500‐hPa geopotential height from historical weather data." Journal of Advances in Modeling Earth Systems 11.8 (2019): 2680-2693.
  • Weyn, Jonathan A., et al. "Sub‐seasonal forecasting with a large ensemble of deep‐learning weather prediction models." Journal of Advances in Modeling Earth Systems 13.7 (2021): e2021MS002502.
  • Hakim, Gregory J., and Sanjit Masanam. "Dynamical Tests of a Deep-Learning Weather Prediction Model." arXiv preprint arXiv:2309.10867 (2023).
  • Ben-Bouallegue, Zied, et al. "The rise of data-driven weather forecasting." arXiv preprint arXiv:2307.10128 (2023).
  • Ham, Yoo-Geun, Jeong-Hwan Kim, and Jing-Jia Luo. "Deep learning for multi-year ENSO forecasts." Nature 573.7775 (2019): 568-572.

​

86 of 89

Hybrid Modeling

​

  • Watt‐Meyer, Oliver, et al. "Correcting weather and climate models by machine learning nudged historical simulations." Geophysical Research Letters 48.15 (2021): e2021GL092555.
  • Bretherton, Christopher S., et al. "Correcting coarse‐grid weather and climate models by machine learning from global storm‐resolving simulations." Journal of Advances in Modeling Earth Systems 14.2 (2022): e2021MS002794.
  • Brenowitz, Noah D., et al. "Machine learning climate model dynamics: Offline versus online performance." arXiv preprint arXiv:2011.03081 (2020).
  • Chen, Tse‐Chun, et al. "Correcting Systematic and State‐Dependent Errors in the NOAA FV3‐GFS Using Neural Networks." Journal of Advances in Modeling Earth Systems 14.11 (2022).
  • Yuval, Janni, and Paul A. O’Gorman. "Stable machine-learning parameterization of subgrid processes for climate modeling at a range of resolutions." Nature communications 11.1 (2020): 3295.
  • Yuval, Janni, Paul A. O'Gorman, and Chris N. Hill. "Use of neural networks for stable, accurate and physically consistent parameterization of subgrid atmospheric processes with good performance at reduced precision." Geophysical Research Letters 48.6 (2021): e2020GL091363.
  • Bolton, Thomas, and Laure Zanna. "Applications of deep learning to ocean data inference and subgrid parameterization." Journal of Advances in Modeling Earth Systems 11.1 (2019): 376-399.
  • Ross, Andrew, et al. "Benchmarking of machine learning ocean subgrid parameterizations in an idealized model." Journal of Advances in Modeling Earth Systems 15.1 (2023): e2022MS003258.
  • Chapman, William E., and Judith Berner. "Benefits of Deterministic and Stochastic Tendency Adjustments in a Climate Model." arXiv preprint arXiv:2308.15295 (2023).
  • Gregory, William, et al. "Deep learning of systematic sea ice model errors from data assimilation increments." Journal of Advances in Modeling Earth Systems 15.10 (2023): e2023MS003757.
  • Sane, A., Reichl, B. G., Adcroft, A., & Zanna, L. (2023). Parameterizing Vertical Mixing Coefficients in the Ocean Surface Boundary Layer using Neural Networks. arXiv preprint arXiv:2306.09045.

87 of 89

Scientific Discovery

​

  • Mayer, Kirsten J., and Elizabeth A. Barnes. "Subseasonal forecasts of opportunity identified by an explainable neural network." Geophysical Research Letters 48.10 (2021): e2020GL092092.
  • Mayer, Kirsten J., and Elizabeth A. Barnes. "Subseasonal midlatitude prediction skill following quasi-biennial oscillation and Madden–Julian Oscillation activity." Weather and Climate Dynamics 1.1 (2020): 247-259.
  • Barnes, Elizabeth A., et al. "Identifying opportunities for skillful weather prediction with interpretable neural networks." arXiv preprint arXiv:2012.07830 (2020).
  • Mayer, Kirsten J., and Elizabeth A. Barnes. "Quantifying the effect of climate change on midlatitude subseasonal prediction skill provided by the tropics." Geophysical Research Letters 49.14 (2022): e2022GL098663.
  • Gordon, Emily M., and Elizabeth A. Barnes. "Incorporating Uncertainty Into a Regression Neural Network Enables Identification of Decadal State‐Dependent Predictability in CESM2." Geophysical Research Letters 49.15 (2022): e2022GL098635.
  • Toms, Benjamin A., Elizabeth A. Barnes, and Imme Ebert‐Uphoff. "Physically interpretable neural networks for the geosciences: Applications to earth system variability." Journal of Advances in Modeling Earth Systems 12.9 (2020): e2019MS002002.
  • Barnes, E. A., Toms, B., Hurrell, J. W., Ebert‐Uphoff, I., Anderson, C., & Anderson, D. (2020). Indicator patterns of forced change learned by an artificial neural network. Journal of Advances in Modeling Earth Systems, 12(9), e2020MS002195.
  • Lagerquist, Ryan, Amy McGovern, and David John Gagne II. "Deep learning for spatially explicit prediction of synoptic-scale fronts." Weather and Forecasting 34.4 (2019): 1137-1160.

​

88 of 89

Post-Model Run / Data Improvement

​

  • Rasp, Stephan, and Sebastian Lerch. "Neural networks for postprocessing ensemble weather forecasts." Monthly Weather Review 146.11 (2018): 3885-3900.
  • Chapman, W. E., et al. "Improving atmospheric river forecasts with machine learning." Geophysical Research Letters 46.17-18 (2019): 10627-10635.
  • Haupt, S. E., Chapman, W., Adams, S. V., Kirkwood, C., Hosking, J. S., Robinson, N. H., ... & Subramanian, A. C. (2021). Towards implementing artificial intelligence post-processing in weather and climate: proposed actions from the Oxford 2019 workshop. Philosophical Transactions of the Royal Society A, 379(2194), 20200091.
  • Chapman, William E., et al. "Probabilistic predictions from deterministic atmospheric river forecasts with deep learning." Monthly Weather Review 150.1 (2022): 215-234.
  • Glahn, Harry R., and Dale A. Lowry. "The use of model output statistics (MOS) in objective weather forecasting." Journal of Applied Meteorology and Climatology 11.8 (1972): 1203-1211.
  • Schulz, Benedikt, and Sebastian Lerch. "Machine learning methods for postprocessing ensemble forecasts of wind gusts: A systematic comparison." Monthly Weather Review 150.1 (2022): 235-257.
  • Vannitsem, Stéphane, et al. "Statistical postprocessing for weather forecasts–review, challenges and avenues in a big data world." Bulletin of the American Meteorological Society (2020): 1-44.
  • Scheuerer, Michael. "Probabilistic quantitative precipitation forecasting using ensemble model output statistics." Quarterly Journal of the Royal Meteorological Society 140.680 (2014): 1086-1096.
  • McGovern, Amy, et al. "Using artificial intelligence to improve real-time decision-making for high-impact weather." Bulletin of the American Meteorological Society 98.10 (2017): 2073-2090.
  • Gagne, David John, et al. "Storm-based probabilistic hail forecasting with machine learning applied to convection-allowing ensembles." Weather and forecasting 32.5 (2017): 1819-1840.
  • Schreck, John S., et al. "Evidential Deep Learning: Enhancing Predictive Uncertainty Estimation for Earth System Science Applications." arXiv preprint arXiv:2309.13207 (2023).

​

89 of 89

Questions?