Prospects and Frontiers in Weather and Climate Part II

Machine Learning

Travis A. O’Brien

Indiana University

with contributions from Ankur Mahesh

Recap of where the field is right now

From Will Chapman’s DCMIP 2025 talk

Recap of where the field is right now

From Dima Kochkov’s DCMIP 2025 talk

Recap of where the field is right now

From David Hall’s DCMIP 2025 talk

ML is revolutionizing our field

New capabilities allow us to ask new questions

• Huge ensembles allow us to potentially ask questions that were previously out of reach�
• Differentiability brings the power of complicated adjoint models to the masses

ML is revolutionizing our field | Huge Ensembles

On August 23, 2023, Kansas City had an extreme heatwave, with 35°C air temperature, 56% relative humidity, and a heat index of 43°C.

​

The 10-day IFS ensemble forecasts predicted warmer than average temperatures, but no members captured the combined magnitude of surface heat and humidity.

​

Huge Ensembles of Neural Network Simulations (HENS) samples the tails of the forecast distribution and is able to capture the magnitude of the event.

Mahesh, Ankur, et al. "Huge ensembles part I: Design of Ensemble Weather Forecasts Using Spherical Fourier Neural Operators." arXiv preprint arXiv:2408.03100 (2024).

Mahesh, Ankur, et al. "Huge Ensembles Part II: Properties of a Huge Ensemble of Hindcasts Generated with Spherical Fourier Neural Operators." arXiv preprint arXiv:2408.01581 (2024).

ML is revolutionizing our field | Differentiability

• Adjoint models allow investigation of model forecasts to small changes in initial conditions
• Adjoint models are horrendously complicated to write
• Few NWP models have adjoint models
• Differentiability property of ML models brings the power of adjoint models to the masses

ML is revolutionizing our field | Differentiability

Ways that ML-based ESMs might be used

Ullrich, P. A. et al. Recommendations for Comprehensive and Independent Evaluation of Machine Learning‐Based Earth System Models. Journal of Geophysical Research: Machine Learning and Computation 2, (2025). doi: 10.1029/2024JH000496

Barriers in the near-term

• Compute capabilities:�running and training large models require large (expensive) hardware
• Learning curve for physical scientists to learn ML is large�(but dropping, and the field is changing rapidly: new journals, new resources, etc.)
• Trust

Why do we trust dynamical models?

How can we trust ML models? | Hierarchical Testing

How can we trust ML models? Component-level understanding

O’loughlin, R. J., Li, D., Neale, R. & O’brien, T. A. Moving beyond post hoc explainable artificial intelligence: a perspective paper on lessons learned from dynamical climate modeling. Geosci. Model Dev 18, 787–802 (2025). doi: 10.5194/gmd-18-787-2025

My predictions for the next 1-3 years

• Stepping toward operational ML S2S models
• More interpretable models
• More weather/climate-focused curricula�(online learning materials)
• Smaller models?
• Fully ML reanalyses

3-5 years

• Major advances in S2S capabilities???
• Increasing prevalence of hybrid dynamical/ML models
• Emergence of Earth system foundation models
• LLMs fused with ESMs

5+ years

• Dynamical models will still be around�and will still be essential
• O(1 km) scale climate scenarios generated by ML models
• Range of climate sensitivity reduced???

Resources for learning more about ML

1. Practical Deep Learning for Coders - 1: Getting started
2. Transformers (how LLMs work) explained visually | DL5
3. Attention in transformers, step-by-step | DL6
4. Let's build GPT: from scratch, in code, spelled out.
5. Let's build the GPT Tokenizer