1 of 20

The CCPP Framework�A Tool for Unifying Physics Across Earth System Models for Operations, Research, and Development

Michael Kavulich Jr.^1,2, Dom Heinzeller^2,3,*, Grant Firl^2,4, Ligia Bernardet^2,3, Man Zhang^2,3, and Steve Goldhaber^1,**

1 NATIONAL CENTER FOR ATMOSPHERIC RESEARCH, BOULDER, CO, UNITED STATES

2 DEVELOPMENTAL TESTBED CENTER, BOULDER, CO, UNITED STATES

3 NOAA GLOBAL SYSTEMS LABORATORY, UNIVERSITY OF COLORADO/CIRES, BOULDER, CO, UNITED STATES

4 NOAA, OAR/GSL, BOULDER, UNITED STATES

* CURRENT AFFILIATION: UNIVERSITY CORPORATION FOR ATMOSPHERIC RESEARCH COMMUNITY PROGRAMS (UCP); JOINT CENTER FOR SATELLITE DATA ASSIMILATION

** CURRENT AFFILIATION: ESM CONSULTANTS LLC

AGU Fall Meeting; December 14, 2022

Hi, my name is Michael Kavulich, I’m an associate scientist from the National Center for Atmospheric Research in the Developmental Testbed Center, and I’m here to talk to you today about the CCPP Framework: A Tool for Unifying Physics Across Earth System Models for Operations, Research, and Development. I owe my co-authors a big thank you for their help putting this presentation together, as well as their work on the CCPP Framework and its documentation over the years that have made this presentation possible. And the co-authors listed here are just a few of the many contributors to the CCPP and its framework over the past several years, and you can find links to further information on the final slide.

Now to be clear, “Physics” in this context is not just traditional physics such as cloud microphysics, PBL schemes, etc. This can be any parameterization or other process that runs in between dynamic time steps of an earth system model, including chemical, aerosol, and other processes.

2 of 20

Why are we here?

“simulating aerosol particles, cloud microphysical and macrophysical properties, and gas, aerosol, and cloud chemistry…in the context of real atmospheric-scale climate and air quality applications is complicated by differences in their implementation within host models…and complexities introduced by the ongoing maintenance and development of institutional host models.”

Current development of chemistry and physics schemes comes with downsides

Specifics of one model may mask deeper physical insights (e.g. compensating errors or biases)
Scientific and software development from one host model has difficulty reaching others, even if nominally the “same” scheme
Independent development in different models means duplication of work
Even if originally implemented identically, code can diverge, and keeping code synced between different models takes significant effort

2

[Title]

Well, to paraphrase a quote from the session abstract: Physics, chemistry, and other schemes are typically developed as a part of or closely with a specific model, and this comes with downsides:

The dynamics, numerics, or other specifics of one model may prevent (or at least mask) deeper physical insights; for example, compensating errors or biases giving the “right answer for the wrong reasons”
Scientific and software development from one host model has difficulty reaching others, even if nominally the “same” scheme. This is not only because of the reasons mentioned in the first bullet, but due to the sheer time and effort required for duplicating code in a different software environment
Along the same lines: independent development in different models can mean duplication of work, and accordingly the potential for further errors
Over time, even if originally implemented identically (which is not common), code can diverge, and it takes significant effort to keep code synced between different host models

3 of 20

History and mission of the Common Community Physics Package (CCPP)

These problems have a large overlap with those that the CCPP was developed to address:

Unifying physics schemes across host models/dycores
Enabling smoother research-to-operations (R2O) and operations-to-research (O2R) for model parameterizations across the wider scientific community

The first version of the CCPP was developed by the Developmental Testbed Center (NCAR/NOAA collaboration) from 2016-2017, and v1 was released in early 2018. The latest version is v6.0, released in June 2022.

Designed to be interoperable between any host model that has incorporated the CCPP Framework
Fortran-based design to match most scientific code bases
Designed with computational efficiency and parallelism in mind, specifically to support operational efforts

3

These problems have a large overlap with those that the Common Community Physics Package (CCPP) was developed to address. It was developed to

Unify physics schemes across host models and dycores, and to
Enable smoother research-to-operations (R2O) and operations-to-research (O2R) for model parameterizations across the wider scientific community

The first version of the CCPP was developed by the Developmental Testbed Center (an NCAR/NOAA collaboration) from 2016-2017, and version 1 was released to the public in April 2018. The latest released version is v6.0 in June 2022, in conjunction with the release of the UFS Short Range Weather Application v2.0 release; all of which are open source and available on GitHub.

Through the CCPP, physics schemes are designed to be interoperable between any host model that has incorporated the CCPP Framework
The CCPP has a Fortran-based design to match most scientific code bases (though in theory the framework is generic enough that it could be expanded for use with other languages)
And all of this was designed with computational efficiency and parallelism in mind, specifically to support operational efforts for which these considerations are paramount

4 of 20

CCPP components

CCPP Physics

https://github.com/NCAR/ccpp-physics
Library of CCPP-compliant, model-agnostic schemes

Schemes are organized into Suites defined by Suite Definition Files (SDFs)
Within each SDF, schemes are organized in groups that will be called together by the host model

CCPP Framework

https://github.com/NCAR/ccpp-framework
Generalized model-agnostic software framework for connecting one or more sets of physical parameterizations with a host application

4

The CCPP broadly consists of two parts. The first is the CCPP Physics, which is the code repository for the physics schemes that are currently part of the CCPP. This part is beyond the scope of this talk, but for context I’ll give a few details. Individual schemes are written as Fortran subroutines, ideally quite similar to how the developer originally wrote them (and often these were developed in collaboration with the original developers). These schemes are written independently, but when used with a host model they are organized into Suites defined by simple xml-format Suite Definition Files (SDFs). Within each SDF, schemes are organized in “groups” that will be called together by the host model (for example: radiation, fast_physics, stochastics, etc.).

Aside from the CCPP Physics part, we have the CCPP Framework, which is what this talk will focus on. The CCPP Framework is the software framework that connects these schemes to the various host models and to each other.

This diagram shows the various parts of the CCPP in an example host model; the CCPP Framework is roughly represented by the three boxes on the right. I will get into more details about how this works in later slides.

5 of 20

The CCPP Framework

The main acts of the CCPP framework happen prior to model compilation:

ccpp_prebuild.py

Reads SDF(s) to know which physics to include in which groups
Reads metadata from individual schemes and hosts
Uses this information to build software caps for communication between schemes and host model
Generates makefile information for compiling software caps and physics subroutines within host model’s build system

INTEGRATING PHYSICS WITH HOST MODELS

5

The main action of the CCPP framework happens at build time. A python script (ccpp_prebuild.py) is called prior to compiling a host model (usually as part of the Cmake step of a host model's Cmake build).
The script is given one or more Suite Definition Files detailing the physics suite or suites that the host model will be built with
The script is also given a config file, which contains metadata and other definitions needed by the framework to create the software caps for the host model build
It matches variables provided with variables requested to ensure that the host model and schemes are compatible
Finally it uses all this information to autogenerate software caps for suites and groups, a static API file, and makefile information for compiling physics and caps within host’s build system

6 of 20

The CCPP Framework

Important glue that allows physics to be independent of host models (and each other):

CCPP maintains a list of “Standard Names” (github.com/ESCOMP/CCPPStandardNames) based on CF conventions that ensures no ambiguity of variables when communicating between schemes and hosts

Arbitrarily extendable; if a new variable is needed, it can be added!

Scheme-level metadata files ensure that the appropriate variables are available for communication between all independent schemes

Does not require completely re-writing existing schemes; translation of variable names, units, dimensions, etc. is handled by the framework so long as the appropriate metadata is provided

Host-level metadata files ensure that appropriate variables and data types are available for communication with all schemes

STANDARDS AND METADATA

6

7 of 20

The CCPP Framework

CCPP Prebuild debug mode

CCPP framework automatically adds checks for variable allocations and dimensions to the software caps, ensuring that these are all correct prior to executing a scheme

Variable tracking tool allows developers to explore physics suites to see how variables are used within each scheme
More details in the CCPP Technical Document

https://ccpp-techdoc.readthedocs.io/en/latest/

CCPP Single-Column Model (SCM)

designed for testing and development of CCPP-compliant schemes and suites
Can test new schemes or modifications of existing schemes without running a full atmospheric model
uses fixed input data from field studies

DEVELOPMENT TOOLS

7

8 of 20

Host models incorporating CCPP Framework

A number of atmospheric models make use of the CCPP Framework

CCPP Single-Column Model (SCM)
Unified Forecast System (UFS), MRW (global) and SRW (regional)
Naval Research Lab NEPTUNE model

Future projects to incorporate CCPP Framework into more models

Hurricane Analysis and Forecast System (HAFS)
Community Earth System Model (CESM)
Model for Prediction Across Scales (MPAS)

8

9 of 20

An example of chemistry and aerosols in CCPP

NOAA Global Systems Lab (GSL) has developed an experimental implementation of the GEFS for aerosol predictions (GEFS-Aerosols) with CCPP (CCPP-Chem)
Previously deployed using the NUOPC model coupling layer, the CCPP version of this global chemistry implementation precludes the need for a custom chemistry driver, and allows for commingling of the chemical and physical schemes in more physically realistic ways
Experimental realtime runs: https://fim.noaa.gov/FV3CCPPchem/

Based on older FV3 dycore, efforts underway to update system to latest UFS MRW (prototype 8) for future operational implementation in the GEFS

9

Mostly emissions

Radiation, Land Surface may include biogenic emissions

radiation

Land Surface Model

PBL & tracer vertical diffusion

Convection & convective wet deposition

Microphysics

sea salt emission

Dust emission

fire emission and plume-rise module

DMS emission

Aerosol settling (dust, sea salt, pp25 and pp10)

dry deposition

Large-scale wet deposition

GOCART chemistry

Aerosols optical calculation

Met. Prep and anthro. emission

Tracer transport, wet deposition

chemistry is GOCART, could be GEOS-Chem, CMAQ,...

radiation

Land Surface Model

PBL & tracer vertical diffusion

Convection & convective wet deposition

Microphysics

sea salt emission

Dust emission

fire emission and plume-rise module

DMS emission

Aerosol settling (dust, sea salt, pp25 and pp10)

dry deposition

Large-scale wet deposition

GOCART chemistry

Aerosols optical calculation

Met. Prep and anthro. emission

Physics

GSDChem chemical driver

NUOPC

version

CCPP version

Figure courtesy of Kate Zhang, NOAA GSL/CSL. Full citation on final slide

10 of 20

Ongoing/future work in the CCPP Framework

Integrating CCPP with CESM, MPAS, and other NCAR models
Calculating variable tendencies automatically for ease of development and efficiency
Piggybacking

Run dycore once with multiple suites running independently

Single-precision physics (32-bit)

Currently being implemented by NRL for the NEPTUNE model

UFS dynamic tendencies in SCM

Using dynamic output from a UFS model run to drive the dynamics of the SCM

10

11 of 20

Support and collaboration

Michael J. Kavulich, Jr.

kavulich@ucar.edu

NATIONAL CENTER FOR ATMOSPHERIC RESEARCH, �DEVELOPMENTAL TESTBED CENTER

CCPP/SCM Scientific and Technical Documentation:

https://dtcenter.org/community-code/common-community-physics-package-ccpp/documentation

Support and discussion on GitHub Discussions

https://github.com/NCAR/ccpp-framework/discussions

Still in the planning phase: CCPP Visioning Workshop: we want community input for future design decisions and development efforts!

11

Thank you for your attention!

Questions?

12 of 20

Bonus slides

EVERYONE LOVES BONUS SLIDES RIGHT?

12

13 of 20

CCPP v6 supported suites

13

Type	Operational	Developmental
Suite Name	GFS_v16	GFS_v17_p8	HRRR	RRFS_v1beta	WoFS	RAP
Host	SRW, SCM	SCM	SRW, SCM	SRW, SCM	SRW,SCM	SCM
Microphysics	GFDL	Thompson	Thompson	Thompson	NSSL	Thompson
PBL	TKE EDMF	TKE EDMF	MYNN	MYNN	MYNN	MYNN
Surface Layer	GFS	GFS	MYNN	MYNN	MYNN	GFS
Deep Convection	saSAS	saSAS+CA	—	—	—	Grell-Freitas
Shallow Convection	saMF	saMF	—	—	—	Grell-Freitas
Radiation	RRTMG	RRTMG	RRTMG	RRTMG	RRTMG	RRTMG
Gravity Wave Drag	uGWP	GSL-Drag	GSL-Drag	uGWP	uGWP	GSL-Drag
Land Surface	Noah	Noah-MP	RUC	Noah-MP	Noah-MP	RUC
Ozone	NRL 2015	NRL 2015	NRL 2015	NRL 2015	NRL 2015	NRL 2015
H2O	NRL	NRL	NRL	NRL	NRL	NRL
Aerosols	GOCART	GOCART	GOCART	OPAC	OPAC	GOCART

14 of 20

Basic code structure

14

module myscheme�� implicit none� contains

!> \section arg_table_myscheme_init Argument Table

!! \htmlinclude myscheme_init.html�!!

subroutine myscheme_init (errmsg, errflg)

character(len=*), intent(out) :: errmsg� integer, intent(out) :: errflg

...� end subroutine myscheme_init��!> \section arg_table_myscheme_run Argument Table

!! \htmlinclude myscheme_run.html�!!

subroutine myscheme_run(ni, psfc, errmsg, errflg)� integer, intent(in) :: ni� real, intent(inout) :: psfc(:)� character(len=*), intent(out) :: errmsg� integer, intent(out) :: errflg� ...� end subroutine myscheme_run�end module myscheme myscheme.F90

“Hook” for CCPP metadata

15 of 20

CCPP scheme metadata

15

[ccpp-table-properties]

name = myscheme

type = scheme

dependencies = other_file.F90

[ccpp-arg-table]� name = myscheme_run� type = scheme

[stress]

standard_name = surface_wind_stress

long_name = surface wind stress

units = m2 s-2

dimensions = (horizontal_loop_extent)

type = real

kind = kind_phys

intent = in

...

myscheme.meta

Start of new metadata “table”

type = [scheme, module, DDT, host]

name of attached subroutine/module

16 of 20

CCPP scheme metadata

16

the key (standard name) by which this data is known in the CCPP

more descriptive name if standard name is not sufficient

note the format; possibility of automatic unit conversion among schemes and between host

name of variable in subroutine

[ccpp-table-properties]

name = myscheme

type = scheme

dependencies = other_file.F90

[ccpp-arg-table]� name = myscheme_run� type = scheme

[stress]

standard_name = surface_wind_stress

long_name = surface wind stress

units = m2 s-2

dimensions = (horizontal_loop_extent)

type = real

kind = kind_phys

intent = in

...

myscheme.meta

17 of 20

CCPP scheme metadata

17

standard names of array dimensions;

() for scalar;

can specify start:end for dimension (default is 1)

FORTRAN intrinsic type or DDT name

precision or character length

FORTRAN argument intent

[ccpp-table-properties]

name = myscheme

type = scheme

dependencies = other_file.F90

[ccpp-arg-table]� name = myscheme_run� type = scheme

[stress]

standard_name = surface_wind_stress

long_name = surface wind stress

units = m2 s-2

dimensions = (horizontal_loop_extent)

type = real

kind = kind_phys

intent = in

...

myscheme.meta

18 of 20

CCPP scheme metadata

18

Applies to entire scheme; dependencies attribute allows compiling only those files that are necessary for a given list of suites

[ccpp-table-properties]

name = myscheme

type = scheme

dependencies = other_file.F90

[ccpp-arg-table]� name = myscheme_run� type = scheme

[stress]

standard_name = surface_wind_stress

long_name = surface wind stress

units = m2 s-2

dimensions = (horizontal_loop_extent)

type = real

kind = kind_phys

intent = in

...

myscheme.meta

19 of 20

CCPP Host metadata

Most of the host metadata is in FV3/ccpp/data/GFS_typedefs.meta FV3/ccpp/data/CCPP_typedefs.meta
Other files also have metadata to help define DDTs or provide other variables to the physics (e.g. machine.F)
Differences compared to scheme metadata:

Uses type = DDT or module
Intent metadata attributes are not used
Variables can have active attribute:

active = logical expression
Since host models may conditionally allocate memory, the logical expression uses CCPP standard names and represents when the given variable is allocated for use in physics:

e.g., active = (flag_diagnostics_3D)

19

20 of 20

CCPP API calls

Physics initialization, running, and finalization

ccpp_physics_init

calls the init stage of all schemes in the suite (in SDF order)

ccpp_physics_timestep_init

calls the timestep_init stage of of the entire suite at once or just one group

ccpp_physics_run

can call the run phase of the entire suite at once or just one group

ccpp_physics_timestep_finalize

calls the timestep_finalize stage of entire suite at once or just one group

ccpp_physics_finalize

deallocates memory and/or any other run-once finalization work

20

Autogenerated in ccpp_static_api.F90