1 of 46

AI2ES

Site-wide presentation on

CSU’s Team* and Activities

* Includes several NOAA folks.

Wed, March 3, 2021

2 of 46

Overview

Introducing the CSU team�(includes folks from NOAA-GSL, NOAA-STAR)�
Tropical Cyclones �(AI2ES Use Case #3)�
Subseasonal to seasonal (S2S) predictions �(AI2ES Use Case #4)�
AI methods for weather and climate�
Outreach

3 of 46

Part 1:

Introducing CSU’s AI2ES team

4 of 46

Mentoring

Outreach

Imme Ebert-Uphoff

(CIRA / ECE)

Elizabeth Barnes

(Atmospheric Science)

Chuck Anderson

(Computer Science)

Matt Rogers

(CIRA)

Kate Musgrave

(CIRA)

Post-doc

(CIRA)

Grad student

(computer science)

Grad student

(ATS)

Jebb Stewart

(NOAA-GSL, NCAI)

Christina Kumler

(NOAA-GSL / CIRES)

Tropical Cyclones

Subseasonal-to-

Seasonal (S2S)

Who we are

* Original proposal

In red: �senior folks

Trustworthy AI

5 of 46

Mentoring

Trustworthy AI

Outreach

Imme Ebert-Uphoff

(CIRA / ECE)

Elizabeth Barnes

(Atmospheric Science)

Chuck Anderson

(Computer Science)

Matt Rogers

(CIRA)

Kate Musgrave

(TC group lead - CIRA)

Jason Stock

Grad student - computer science

Kathy Haynes

(CIRA)

Chris Slocum

(NOAA-STAR @ CIRA)

Grad student

(ATS)

Jebb Stewart

(NOAA-GSL, NCAI)

Christina Kumler

(NOAA-GSL / CIRES)

Tropical Cyclones

Subseasonal-to-

Seasonal (S2S)

Ryan Lagerquist

(CIRA @ NOAA-GSL)

Antonis Mamalakis

(ATS / CIRA)

Post-doc

(CIRA)

Who we are now�- we have grown!

In red: funded

by AI2ES

6 of 46

Mentoring

Trustworthy AI

Outreach

Imme Ebert-Uphoff

(CIRA / ECE)

Elizabeth Barnes

(Atmospheric Science)

Chuck Anderson

(Computer Science)

Matt Rogers

(CIRA)

Kate Musgrave

(TC group lead - CIRA)

Jason Stock

Grad student - computer science

Kathy Haynes

(CIRA)

Chris Slocum

(NOAA-STAR @ CIRA)

Grad student

(ATS)

Tropical Cyclones

Subseasonal-to-

Seasonal (S2S)

Ryan Lagerquist

(CIRA @ NOAA-GSL)

Antonis Mamalakis

(ATS / CIRA)

Lander Ver Hoef

(GRA, Math)

Henry Adams

(Math)

Topological Data An.

Emily King

(Math)

Harmonic Analysis

Newest addition: math folks!

Post-doc

(CIRA)

Jebb Stewart

(NOAA-GSL, NCAI)

Christina Kumler

(NOAA-GSL / CIRES)

7 of 46

Where we are

CSU FOOTHILLS Campus:

In Fort Collins, CO

CIRA = �Cooperative Institute for Research in the Atmosphere

8 of 46

CSU FOOTHILLS Campus:

In Fort Collins, CO

Imme Ebert-Uphoff

(CIRA / ECE)

Elizabeth Barnes

(Atmospheric Science)

Matt Rogers

(CIRA)

Post-doc (TBD)

(CIRA)

Kathy Haynes

(CIRA)

Chris Slocum

(NOAA-STAR @ CIRA)

Grad student (TBD)

(ATS)

Where we are

Kate Musgrave

(CIRA)

Antonios Mamalakis

(postdoc)

9 of 46

Who or What is CIRA?�

CIRA = Cooperative Institute for Research in the Atmosphere
Research department within college of Engineering at CSU, partnering with Department of Atmospheric Science.
Primary funding from NOAA and NASA grants
Headquarters - and largest group - located at CSU (Fort Collins)
CIRA staff also integrated in many NOAA line offices, including:

Hosting NOAA RAMMB group (NESDIS),
Global Systems Lab (Boulder, CO),
National Hurricane Center (Miami, FL)
National Weather Service (in KS and MD)�

Working with satellite imagery and NWP models to �infer state of earth system, including

Tropical cyclones,
Cloud properties, vertical profiles,
Detection of convective initiation, winter weather,
Detection of fire, snow and dust,
Data assimilation for NWP,
Forecaster training.

Real-time satellite data feed of �GOES-16 and GOES-17 data.

Many of the above are available as public satellite products.� Check out SLIDER at https://rammb-slider.cira.colostate.edu/ �Switch “sector” from “Full disk” to “Conus”, then choose a “Product” to view different satellite products, e.g., from GOES-16, in real time.

10 of 46

CSU MAIN Campus

In Fort Collins, CO
4 miles EAST of Foothills campus

Chuck Anderson

(Computer Science)

Jason Stock

Grad student - computer science

Lander Ver Hoef

(MATH)

Henry Adams

(Math, Topological Data Analysis)

Emily King

(Math, Harmonic Analysis)

Where we are

Computer Science Building

Mathematics Building

11 of 46

NOAA-GSL

In Boulder, CO
66 miles South of Fort Collins

GSL

Global Systems Laboratory

Jebb Stewart

(NOAA-GSL, NCAI)

Christina Kumler

(NOAA-GSL / CIRES)

Ryan Lagerquist

(CIRA @ NOAA-GSL)

Where we are

David Skaggs Research Center

CSU

GSL

12 of 46

NOAA-GSL team in Boulder, Co

Jebb Stewart

- NOAA-GSL

- NAIEC/NCAI

- Sample ML Research:

ML for TCs
Region of interest in satellite imagery
Detect convection from satellite imagery

Christina Kumler

- NOAA-GSL

- CIRES (CU Boulder)

- Sample ML Research:

ML for TCs
Region of interest in satellite imagery
Fire Radiative Power

Ryan Lagerquist

- CIRA (@ NOAA-GSL)

- Postdoctoral Fellow

- Sample ML Research:

ML emulation of radiative transfer (for NWP)
Forecast convection from satellite imagery
ML for TCs

13 of 46

Imme Ebert-Uphoff

Lead of CSU’s AI2ES team
Research Professor – Electrical & Computer Engineering
Machine Learning Lead - Cooperative Institute for � Research in the Atmosphere (CIRA)

Sample research interests:

Image-to-Image translation for weather applications using NNs
Explainable AI – What did my neural network learn?
Physics-driven design of ML:

Feature engineering,
Creative cost functions,
Physical constraints in loss.

Exploring new mathematical tools: Topological data analysis for detecting texture, shapes, in satellite imagery.

Recent ML applications:

Generating synthetic MRMS imagery (radar) from GOES satellite imagery,
Detecting convection from satellite imagery,
Speeding up radiative transfer in NWP using ML,
Improving vertical profiles,
ML for tropical cyclones.

Mica & Imme

14 of 46

Part 2:

Tropical Cyclones (AI2ES Use case #3)�

Leads:

Kate Musgrave (TC expertise)
Imme Ebert-Uphoff (ML expertise)

15 of 46

Imme Ebert-Uphoff

(ML expertise)

Kate Musgrave

(TC expertise)

Kathy Haynes

(CIRA)

Chris Slocum

(NOAA-STAR @ CIRA)

Christina Kumler

(NOAA-GSL / CIRES)

Ryan Lagerquist

(CIRA @ NOAA-GSL)

Post-doc

(CIRA)

ML for Tropical Cyclones @ CSU

Jebb Stewart

(NOAA-GSL, NCAI)

In red: Leads

16 of 46

Kate Musgrave

Lead of Tropical Cyclone group at CIRA
Research Scientist
Dual BS in computer engineering and quantitative science (2003), PhD in atmospheric science (2011)

TC activities at CIRA include:

TC structure and evolution
Statistical-dynamical models for TCs
Dynamical model post-processing and evaluation
Satellite-based TC products
Research-to-operations transition of TC products and guidance
ML for TCs

17 of 46

Motivation - Background on TCs

Both the analysis and prediction of TC intensity and intensity change are dependent on the convective structure
IR from geostationary satellites gives us consistent spatial and temporal coverage, but upper-level cirrus obscures the underlying convection
Microwave from polar orbiters gives us important information about the convective structure, but does not have the spatial and temporal resolution
Generating synthetic microwave from geostationary satellite data can fill the gaps in coverage

18 of 46

Synthetic MW from GOES-R

Random Forest

1000 trees
Regression
Pixel-based

Input

GOES-R L1b & L2 products

10-minute full disk
16-channel ABI
Channel differences
Channel ratios
Geometric Thickness

Trained on 2018 AMSR-2

Atlantic TC overpasses

Limitations

Daytime only
Does not handle regimes

outside training: cold SST,

mountainous terrain

Chris Slocum

(NOAA-STAR @ CIRA)

John Knaff

(NOAA-RAMMB @ CIRA)

From Slocum and Knaff 2020, AMS Annual Meeting

19 of 46

Proposed AI2ES work for TCs

Improve AI algorithms to generate synthetic MW imagery�
Develop AI algorithms to predict TC temporal evolution and rapid intensification�
Extract scientific insights about TCs

20 of 46

Next steps

Kathy Haynes (at least 6 months on TCs):

Funded by other sources.
Starting work on using NNs to generate synthetic MW.

Ryan Lagerquist (side project):

Funded by other sources.
Starting work on identifying & predicting TC intensity and rapid intensification.

Postdoc (AI2ES funding for 5 years):

Interviews starting this week.�

Post-doc

(CIRA)

21 of 46

Part 3

�Subseasonal-to-Seasonal (S2S)�(AI2ES Use Case #4)

Lead: Elizabeth Barnes

22 of 46

Prof. Elizabeth Barnes

Dual B.S. in mathematics and physics (2007); PhD in atmospheric science (2012)�
Joined CSU Atmospheric Science faculty in 2013�
Teach ATS graduate courses in atmospheric dynamics and data science (including ML)�
I have been actively collaborating with Prof. Imme Ebert-Uphoff and Prof. Chuck Anderson for 4+ years�
I have been having an absolute blast walking the line between atmospheric/climate scientist and data scientist

Prof. Elizabeth A. Barnes

eabarnes@rams.colostate.edu

Barnes Group website

me

23 of 46

Subseasonal-to-Seasonal (S2S)

Bridging the Gap

23

NOAA CPO image

text from Earth System Prediction Capability Office

and National Academy Report

Prof. Elizabeth A. Barnes

eabarnes@rams.colostate.edu

Barnes Group website

24 of 46

Subseasonal-to-Seasonal (S2S)

Bridging the Gap

24

NOAA CPO image

text from Earth System Prediction Capability Office

and National Academy Report

Prof. Elizabeth A. Barnes

eabarnes@rams.colostate.edu

Barnes Group website

25 of 46

Forecasts of Opportunity

certain conditions lead to more predictable behaviour than others

Beyond the weather timescale we must look for specific states of the earth system, i.e. “opportunities”, that lead to enhanced predictable behavior.�

25

See Mariotti et al. (2020) and also Albers and Newman (2019)

Prof. Elizabeth A. Barnes

eabarnes@rams.colostate.edu

Barnes Group website

Colorado State University

26 of 46

Finding forecasts of opportunity

When? Under what conditions do we have skillful forecasts of opportunity?�
Why? Where is this predictability coming from?�
How do we leverage these opportunities?

26

Prof. Elizabeth A. Barnes

eabarnes@rams.colostate.edu

Barnes Group website

Colorado State University

27 of 46

Stratospheric polar vortex

Stratospheric sudden warmings provide extended predictability of midlatitude surface weather out ~60 days.

27

See e.g. Domeisen et al. (2019, 2020)

Prof. Elizabeth A. Barnes

eabarnes@rams.colostate.edu

Barnes Group website

Colorado State University

28 of 46

Tropical activity

The Madden-Julian oscillation (MJO) provides extended predictability of midlatitude weather out 4+ weeks.

28

Prof. Elizabeth A. Barnes

eabarnes@rams.colostate.edu

Barnes Group website

Colorado State University

29 of 46

Future S2S efforts under AI2ES

Identifying and understanding forecasts of opportunity

XAI (e.g. LRP)
Abstention approaches (classification and regression)�

Improving S2S accuracy with empirical methods

Seeing how far forecasts of opportunity will take us
Comparisons with state-of-the-art dynamical models
Exploring hybrid dynamical-empirical prediction methods (e.g. transfer learning)

Prof. Elizabeth A. Barnes

eabarnes@rams.colostate.edu

Barnes Group website

https://depositphotos.com/portfolio-1472772.html

30 of 46

Part 4:

AI methods for weather and climate

Leads:

Chuck Anderson
Elizabeth Barnes
Imme Ebert-Uphoff�

31 of 46

Imme Ebert-Uphoff

(CIRA / ECE)

Elizabeth Barnes

(Atmospheric Science)

Chuck Anderson

(Computer Science)

Jason Stock

Grad student - computer science

Kathy Haynes

(CIRA)

Grad student

(ATS)

Ryan Lagerquist

(CIRA @ NOAA-GSL)

Antonis Mamalakis

(ATS / CIRA)

Lander Ver Hoef

(GRA - Math)

Henry Adams

(Math)

Topological Data An.

Emily King

(Math)

Harmonic Analysis

AI for weather & climate @ CSU

Jebb Stewart

(NOAA-GSL, NCAI)

Christina Kumler

(NOAA-GSL / CIRES)

In red:

Funded by AI2ES

Kate Musgrave

(TC group lead - CIRA)

32 of 46

Explainable AI from Simple to Complex

Chuck Anderson

Colorado State University
Professor, Computer Science
Research in neural networks (30+ years)
www.cs.colostate.edu/~anderson

Jason Stock

Colorado State University
Ph.D. Candidate, Computer Science
Research in machine learning using data from radiosondes, satellite imagery, and surface measurements
www.linkedin.com/in/jason-stock/

33 of 46

Seek simple explanations before proceeding to the complex.

Start with simple models first,

then progress to more complex ones, as long as they are supported by the data.

More complex model trained to fix errors of previous model.

Explanations/interpretations increase in detail.

Explainable AI from Simple to Complex

Linear

1 Hidden Layer

2 Hidden Layers

.

+

Input

Samples

Cascade Network Structure

34 of 46

Based on Viewing Forced Climate Patterns Through an AI Lens, Barnes, et al., 2019.

Linear

1 Hidden Layer

2 Hidden Layers

+

temperature

year

What did these 7 and 8 layer neural networks discover in the global temperature data?

linear model

.

35 of 46

Abstention Networks�A network that can say “IDK”

Why This Is Interesting: Unlike many commercial applications, we do not expect ML to be able to predict everything all of the time in the earth system (even if we had enough data).

The Idea: We expect certain earth system conditions to be more predictable than others, so we want to leverage these opportunities for skillful forecasts

Goal: Develop a neural network architecture that can say “IDK” (I don’t know).

Utility: This methodology is easy to implement and may be useful for a wide range of applications

Progress: We have a working methodology that improves accuracy over more standard post-training approaches, working on applying it to regression now

% of samples the network predicts �(e.g. does not say IDK)

baseline approach

our abstention network

Prof. Elizabeth A. Barnes

eabarnes@rams.colostate.edu

Barnes Group website

Barnes and Barnes (in prep)

classification accuracy

36 of 46

XAI via �Attribution Maps

Layerwise Relevance Propagation (LRP)

One of many XAI methods for neural networks to attribute different features of the input to the ultimate output for a single sample
While many methods exist, thus far we have found LRP to be incredibly useful and intuitive, however, research is underway to explore other methods too (see future slide)

Pr(cat)

LRP

of 1 sample

Prediction

of 1 sample

Pr(cat)

Montavon et al. (2017), Pattern Recognition

Prof. Elizabeth A. Barnes

eabarnes@rams.colostate.edu

Barnes Group website

37 of 46

Benefits of LRP for scientific research

Identifying problematic strategies (i.e. right answer for the wrong reasons)
Designing the machine learning methodology
Building trust
Discovering new science!

When our machine learning method is capable of making a correct prediction we can explore why

Prof. Elizabeth A. Barnes

eabarnes@rams.colostate.edu

Barnes Group website

38 of 46

LRP for �Scientific Discovery

Subseasonal-to-seasonal (S2S) prediction: �Learning climate states that lead to more predictable behaviour than others on S2S timescales

Network Task: Train a neural network to ingest daily maps of outgoing longwave radiation (OLR) to predict the sign of the subseasonal circulation anomalies over the North Atlantic 22 days in advance�

Interpretable AI Approach: Learn tropical patterns of variability that lead to enhanced predictability of midlatitude weather on subseasonal timescales

38

Mayer and Barnes (under review)

Kirsten Mayer

Cluster 1

N = 127

Cluster 2

N = 48

LRP: OLR patterns that lead to accurate & �confident Z500<0 predictions over the North Atlantic

daily maps of OLR

Prof. Elizabeth A. Barnes

eabarnes@rams.colostate.edu

Barnes Group website

39 of 46

Other examples of scientific discovery with LRP

39

Zack Labe

Indicator patterns of �forced change�

Learn non-linear, time-evolving patterns of forced change in climate simulations and observations, e.g. Barnes et al. (2020, JAMES)

Jamin Rader

Decadal predictability

Emily�Gordon

Benjamin�Toms

Learning climate states that lead to more predictable behaviour than others on decadal timescales, e.g. Toms, Barnes & Hurrell (in review)

Benjamin�Toms

Kirsten Mayer

Zane�Martin

Exploring MJO physics with implications for prediction, scientific mechanisms, and basic theory, e.g. Toms et al. (under review)

MJO dynamics, predictability & teleconnections

Prof. Elizabeth A. Barnes

eabarnes@rams.colostate.edu

Barnes Group website

40 of 46

Explainable AI: What did my neural network learn?

Which XAI method is best for weather/climate applications?
What are the pros and cons for each?
Generate benchmark with ground truth for attribution.�

Benchmark for evaluation of attribution heatmaps in weather and climate applications

Work in progress: to be submitted to special issue of IEEE Transactions on Neural Networks and Learning Systems (March 12, 2021).

Imme Ebert-Uphoff

Elizabeth Barnes

Antonis Mamalakis

(ATS / CIRA)

Imme Ebert-Uphoff, CSU

41 of 46

Using Explainable AI for satellite applications

Sample application:

Generate synthetic MW imagery from GOES imagery.

Once a neural network is learned, post-analysis:

What did my �neural network learn?

Details: I. Ebert-Uphoff and K.A. Hilburn, �Evaluation, Tuning and Interpretation of Neural Networks for Meteorological Applications, �Bulletin of the American Meteorological Society (BAMS), Aug 2020.

Explainable AI – Example:�Using Layer-Wise Relevance Propagation (LRP) to identify strategies a neural network (NN) has learned.

GOES channels (input to NN)

Apply LRP to find out!

Where is NN looking?

Result: NN mainly looking at

Cold cloud tops,
Cloud boundaries,
Lightning.

Imme Ebert-Uphoff, CSU

Kyle Hilburn

(CIRA)

NN strategies expressed in meteorological terms!

42 of 46

Physics-based features for satellite imagery:

How can Topological Data Analysis (TDA) and Harmonic Analysis help for analysis of satellite imagery?
Underutilized tools that can yield physically interpretable image features, which can then be used in (simpler & more intuitive) ML algorithms.

Lander Ver Hoef

(GRA - Math)

Henry Adams

(Math)

Topological Data An.

Emily King

(Math)

Harmonic Analysis

Imme Ebert-Uphoff

New topic -

Just starting.

Satellite imagery

(GOES-16)

Image in topological space (persistence bar code)

Details: �Lander Ver Hoef et al., Topological Data Analysis for Identifying Convection in GOES-R Imagery, �AMS 101st Annual meeting, 20th Conference on Artificial Intelligence for Environmental Science, Jan 2021.

Imme Ebert-Uphoff, CSU

43 of 46

Physics-based features for satellite imagery (new topic, just starting)

Lander Ver Hoef

(GRA - Math)

Tip:

We converted AI2ES Y1 travel funding (unused due to COVID) into GRA summer funding for Lander.
You need to ask for permission, but easy to get permission if “scope of research is unchanged” by budget change.

Imme Ebert-Uphoff, CSU

44 of 46

Part 5:

AI2ES Outreach Activities at CSU

Lead: Matt Rogers

45 of 46

CIRA Outreach Program

Research associate at CIRA, background in satellite remote sensing and education/outreach programs
Education/Public Outreach lead for NASA CloudSat mission (2005-2011), developed and trained CloudSat Education Network, a national and global network of schools trained to make surface cloud observations coincident with satellite overpasses.
At CIRA, outreach focus on bringing NOAA and CSU research to four primary audiences (K-12, community science, professional development, and public affairs).

�Current outreach activities at CSU/CIRA include:�

Fire/flood/drought resiliency programs for mountain communities.
K-5 weather standards professional development for educators.
Media and classroom training on satellite visualization tools.

Key Goal: Authentic Subject Matter Expert involvement!

46 of 46

Upcoming AI2ES outreach activities

Coordinate with TAMUCC and Del Mar further - AI4K12 good start!

Need to recruit, but need product with which to recruit!

From CSU AI research over spring/summer, develop pilot outreach activities, aligned with standards as possible, for deployment in Fall 2021:�

AI basics (methods, tools, including ethics!)
Publicly available datasets key - CIRA to provide as able
Aligned with AI2ES research products�

Engage and interact with K-12 educators (coding clubs, other groups?) and see how products evolve based on needs - then evolve!