CCPP History

Cheryl Craig, Courtney Peverley,�Dustin Swales, Dom Heinzeller

May 2026

Before CCPP, there was IPD

Interoperable Physics Driver written by P. Tripp (EMC) and R. Benson (GFDL) for the NGGPS “dycore” bake off.

​

​

IPD refactored old GCM physics drivers and made GFS physics accessible to others: interoperable physics!

FV3

Gaps in IPD interoperability

IPD did not increase flexibility for adding or reorganizing physical parameterizations.

IPD drivers were maintained by hand, 10k+ lines long.

No information (metadata) on variables needed to run a parameterization: Which quantity exactly? What units?

CCPP: improve interoperability through metadata and auto-generated interfaces based on suite definition.

ccpp-prebuild v0: dynamic loading

Suite definition file

All schemes compiled�into dynamic library

Open libexmaple.so

Load and run scheme test

BUILD

RUN

2017

Original design: Tim Brown (NOAA/CIRES), Dave Gill (MMM)

ccpp-prebuild metadata in code

CCPP hooks + doxygen

metadata + doxygen

looong lines and many rows

ccpp-scm: physics R&D + CCPP sandbox

2017

Early adoption in FV3GFS, NEPTUNE

NEPTUNE: Dom sat down with Alex Reinecke (NRL) for half a day to implement a dynamic CCPP “helloworld” suite. No visibility of code (restricted access).�Alex and his team took it from there.

​

FV3GFS: Incremental approach termed “hybrid CCPP”. Schemes were one by one called from CCPP instead of IPD (no CCPP group/suite caps - needed dynamic CCPP).

2018-ish

Issues with dynamic loading

��

1. Developers and debugging tools can’t see final code
2. Data is retrieved from memory by location (symbol)
3. IPD used blocked data structures for performance;�all N blocks were combined for init/final phases.
4. CCPP couldn’t do this, thus had to run init N times
5. This and dynamic loading made it SLOOOOOOOW

​

​

Host code�(Fortran)

CCPP API�(C)

Scheme code�(Fortran)

ccpp-prebuild v3: static caps

​

ccpp-prebuild v3 was the first real “code generator”; it generated Fortran caps for groups and suites. This was only possible after a full adoption of CCPP in the UFS.

Side plot: the UFS-CCPP�acceptance drama (aka�bit-for-bit challenge)

2019

Metadata updates

Unwieldy metadata tables in Fortran code were replaced with accompanying metadata files. The metadata parser used by ccpp-prebuild was retired in favor of the new capgen metadata parser (see later in this talk).

2019

ccpp-prebuild static: pros & cons

​

1. Static build alleviated all issues identified�with dynamic CCPP, ran faster, used less memory
2. Enabled optional arguments, debugging features (allocation and size checks), and unit conversions
3. No runtime flexibility: suites defined at build�time. Addressed by building caps for multiple suites

�Static ccpp-prebuild v4 replaced IPD in the UFS in 2020

ccpp-prebuild v4-v7 features 1/2

​

optional argument

allocation/size check

unit conversion

ccpp-prebuild v4-v7 features 2/2

​

​

Subcycling of schemes to call at�higher frequency, do iterations.

�Replaced blocked data structures�(IPD-style) with chunked arrays.

​

foo%bar(1:ncolums)

foo(1:nblocks)

foo(1:nblocks)%� bar(1:blocksize)

vs

ccpp-prebuild releases, operations

CCPP v6 published in GMD and BAMS!

Capgen motivation

Prebuild has some limitations…

• Hard to extend
• No mechanism for handling tracers/constituents, presenting challenges for physics-to-chemistry coupling
• Host model needs to allocate interstitial physics variables, which is burdensome for development
• Limited error checking
• Unwieldy in-line metadata (discussed in prior slide), addressed by moving to capgen’s metadata parser

Capgen is a “clean sheet” design to address these limitations.

capgen features

Fortran parser

In prebuild, only the metadata files are parsed. There is no mechanism to ensure that the fortran arguments match what is in the metadata. Bad! ��Capgen parses the fortran files and compares it to the metadata for consistency.

This is done at build-time, catching potential run-time errors sooner.

Fortran/metadata checking can also be performed independently (for example, as part of a Github action)

Facilitate Scheme Portability

Prebuild has support for unit-conversions, allowing for the host and schemes to maintain their own unit conventions.

Capgen extends this approach to do a range of “variable transformations”

• Unit conversions (e.g., m -> microns)
• Type conversions (e.g., R4 -> R8)
• Vertical reordering (e.g., top-down -> bottom-up)
• Re-indexing (e.g., [i,k]->[k,i]) *Not yet implemented

Introduce “Suite variables”

Some schemes require fields output from other schemes.

In prebuild, these needed to be allocated by the host (e.g., CCPP_typedefs in UFS).

In capgen, there is no need to do this. Instead, capgen will define these fields as module variables within the suite cap.

This greatly reduces the overhead in the host.

Introduce “register” phase

In addition to the “init”, “timestep_init”, “run”, and “timestep_final” phases that exist in prebuild, capgen introduces a new register phase. This phase is to be (optionally) be called before “init”.

• This allows physics schemes to set variables before the host model grid is initialized

Add built-in constituents handling

The constituents/tracer object is known to and handled by the framework, allowing schemes to:

• Request the full object or individual tracer(s) via metadata (build-time)
• Register tracer(s) at run-time during the register phase
• Update the tracer tendencies via a identically-indexed tendency array
• Get important metadata for tracers (molar mass, diagnostic name)

without any intervention needed by the host model (meaning that schemes that use the constituent object are portable to other CCPP-enabled host models)

CCPP Timeline

v0 dynamic

v3 static

v6

GMD Vol 16, issue 8

BAMS Vol 105, issue 8

capgen

prebuild

NEPTUNE v1.6

v7

v4

v5

development begins

constituents added

testing revamped

CAM4 physics

in CAM-SIMA

HAFS v1

HAFS v2/v2.1

GEFS-Aerosols v12.3

capgen challenges

Challenges with Capgen…

Capgen is currently

• Vastly more powerful than prebuild, but it is an extremely complicated code base that is close to its limits of its extensibility. This is prior to meeting�all of NOAA/NRL’s operational needs.�
• Much slower than prebuild. Tests in the SCM are ~20x slower when writing the suite caps for all of NOAA operational SDFs.

Challenges with Capgen…

Capgen was

• Not designed with the same operational considerations as prebuild (e.g., Performance and NCO requirements)

​

• A great deal of time was spent trying to overcome these challenges, ultimately requiring a refactoring of capgen, yielding a more complicated codebase. Issues remain.

​

Challenges with Capgen…

Capgen has been under development for ~7 years!

• Primary developer of Capgen left NCAR 3(?) years ago.�
• We continued to engage, develop, and push forward�with Capgen development, but recently we lost this collaboration with the primary developer.

Considerations going forward

How can we best leverage the pros of both the capgen and prebuild designs?

​

As we look towards capgen version 2.0 (?), what do we want to simplify and/or clean up? What features can be removed?

​

How can we best improve on capgen while minimizing impacts to participant host models?