ENSO forecast skill in a changing climate�Can past changes in ENSO forecast skill be attributed to climate change?

Jiale Lou ¹, Matt Newman ², Andrew Hoell ², and Andrew Wittenberg ³

​

¹ Atmospheric and Oceanic Sciences Program, Princeton University, Princeton, NJ, USA.

² Physical Sciences Laboratory, NOAA, Boulder, CO, USA.

³ Geophysical Fluid Dynamics Laboratory, NOAA, Princeton, NJ, USA.

Credit: NOAA climate.gov

Snapshots from AMS2024 programs

Backward model-analog technique

What is ‘model-analog’ and how it works?

Forecast spread

1. Finding the best matches (“analogs”) to observed anomaly in pre-existing climate model simulations

Here: match observed tropical sea surface temperature (SST) and/or sea surface height (SSH) anomalies; equally weighted (looking at ML techniques to improve this).

2. Making the model-analog forecasts

3. Evaluating the model-analog forecast skill

Without re-running the climate models, the subsequent model evolutions directly form the suite of model-analog forecast ensembles.

Validating the forecasts by assessing forecast skill metrics.

• Simple, fast, cheap, explainable
• Does not require full global 3D data assimilation to “initialize” forecasts.
• Benchmarking the operational climate predictions.

​

• No initialization shock

The initial states come from the model space directly. No sudden adjustment from the internal model state to better match the provided initial (assimilated) conditions.

​

• Turn all the pre-existing climate models into forecast model

Taking advantage of large and lengthy climate simulations; expanding climate reforecasts (hindcasts); making forecasts for all the model variables.

​

• Verification and training data are separate

no need to subset the already short verification datasets to construct the empirical models.

Why model-analogs? (Merits of the model-analog technique)

Ding et al. 2018; 2019

Month 6 hindcast skill, 1982-2009

SST

precipitation

Operational

NMME

hindcasts

NMME

analogs

CMIP5

analogs

NMME model-analogs for tropical Indo-Pacific based on control runs of the same NMME models used for assimilation-initialized hindcasts (NCAR CESM1/CCSM4, GFDL CM2.1/ FLOR)

Ding et al (2019, GRL)

Examples: Model-analog forecast skill matches/exceeds traditional hindcasts.

Q1: Does ENSO predictability vary from decade to decade?

​

Q2: Have these changes been attributed to climate change?

​

Recipe for model-analog hindcasts:

• Library: Using 25 CMIP6 pre-industrial control simulations (fixed 1850)
• Results shown here use “best” 9 models
• Verification: Seven SST reanalysis datasets.
• Define anomalies relative to preceding 30-yr mean (“fair-sliding” in Risbey et al. 2021)
• Also use SST datasets that go back to late 1800’s (“SST-only analogs”)
• Evolution of skill is similar for both SST and SLP (not used to choose analogs)

SEAS5-20C (Weisheimer et al. 2022) was only run 2x yearly, however…��AC skill, determined for sliding 30-yr windows (centered by year, with increasing lead time)

7

Lou, Newman, and Hoell (2023)

Model-analog ENSO skill is comparable to ECMWF SEAS5-20C hindcasts

Apples-to-apples comparison:

​

Same verification dataset;

Same verification period;

Same initialization months.

8

Q1: Does ENSO predictability vary from decade to decade?

YES. ENSO forecast skill underwent multi-decadal variations with the minimum skill in the middle of 20^th century.

ROC score = 1 Perfect score

ROC score < 0.5 No skill

Figure: Predictive relative operating characteristic (ROC) area evolution for (a) La Niña condition and (b) El Niño condition based on NINO3.4 time series over the 30-year moving hindcast windows.

​

ROC score: hit rate vs. false alarm rate

La Niña being more predictable than El Niño is recent.

Lou, Newman, and Hoell (2023)

9

ENSO forecast skill corresponds well with the variance change

Figure: same as previous slide.

​

Figure: 30-yr moving variance of NINO3.4 index based on seven observations.

Question: Can we attribute the past multi-decadal variation of ENSO forecast skill to climate change vs. internal variability?

AGU Annual Meeting • matt.newman@noaa.gov • 15 December 2023

Q1: Does ENSO predictability vary from decade to decade? Yes.

​

Q2: Have these skill variations been attributed to climate change?

​

Experiment 1: “Real-world” experiment

CESM2 piControl vs. CESM2 historical large ensembles (100 members)

(Fixed vs. realistic external forcing)

​

Experiment 2: “Perfect-model” experiment

Evaluate how “perfect-model” skill depends on external forcing vs. noise realizations (note: CESM2 ENSO too strong, too far to west)

Real-world experiment: CESM2 piControl vs. CESM2-LENS

Better second-year ENSO forecast skill with historical forcing

Verification period: 1871-2020 (150 yrs)

1871

2020

Skill difference:

CESM2 piControl minus CESM2LENS

​

Blue shading 🡪 LENS better

Initial months

Lead time [months]

Initial months

Ensemble-mean ENSO amplitude (red line) increased in CESM2LENS, mainly for “Year 2” leads, albeit with substantial decadal variation for different ensemble members (gray lines)

12

Lou, Newman, Hoell, and Wittenberg (2023, in prep)

ENSO amplitude (30-yr moving window)

ENSO AC skill (30-yr moving window)

Perfect-model experiment: Ensemble-mean ENSO forecast skill has slowly increased since the late 1800’s.

“ENSO has high internal variability and single realizations of a model can produce very different results to the ensemble mean response. Internal variability explains 90% of the CMIP5 spread of historical ENSO SST changes” (Maher et al. 2018; GRL)

CESM2LENS (100 ensemble members)

Mean ENSO skill determined from only a subset of ensemble members with “realistic” multi-decadal ENSO amplitude variations (colored lines) better matches observed skill variations

13

Lou, Newman, Hoell, and Wittenberg (2023, in prep)

ENSO amplitude (30-yr moving)

ENSO AC skill (30-yr moving)

Perfect-model experiment: Runs with ENSO variations similar to observed show U-shaped skill evolution.

Model-analogs provide a cheap and easy way of making the long hindcast datasets.��ENSO forecast skill underwent multi-decadal variations since the late 1800s.��ENSO forecast skill corresponds with ENSO amplitudes.��Substantial past variation in ENSO skill was primarily driven by internal ENSO variability, although there may have been a small overall increase due to climate change, especially for longer forecast leads.��Some ENSO events were predictable two or more years ahead, particularly over the last few decades but also in the late 1800’s/early 1900’s period.

Conclusions:

• All real-time ENSO forecasts and hindcasts datasets at https://www.psl.noaa.gov/forecasts/seasonal/

https://psl.noaa.gov/forecasts/seasonal/

20^th century ENSO hindcasts available at

Lou, J., M. Newman, and A. Hoell, 2023: Multi-decadal variation of ENSO forecast skill since the late 1800s. npj Climate and Atmos. Sci., 6, 89, doi: 10.1038/s41612-023-00417-z.

​

Model-analog code for downloading: https://zenodo.org/records/8070768

Thanks to Yan Wang for updating the webpage.

Thanks to Ho-Husan Wei for validating the code.