CCPP suite naming

Justification for renaming

CCPP Version 7

• DTC is working on the joint CCPP/SCM release, planned for June 2024
• We currently have three suites that we would like to add as “officially supported” for the new release:
• GFS_v17_HR3, the suite used for the GFS_v17 HR3 prototype
• A suite identical to GFS_v17_HR3, but with RRTMGP rather than RRTMG
• The RRFS_v1 suite
• In this planning, we have had to re-visit the topic of naming: what do we call these new suites?

What are the current guidelines for naming CCPP Suites?

�������

• …None?
• It seems like, regardless of the outcome of this discussion, we should have some written guidelines on how to name a CCPP Suite

CCPP Suite names: The problem

• Okay, so we don’t have explicit rules. But we do have conventions that seem to have fallen around the following standard:

​

�

suite_FV3_GFS_v16.xml

suite_FV3_GFS_v17_p8_mynn.xml

suite_FV3_HAFS_v1_thompson_tedmf_gfdlsf.xml

suite_FV3_RAP_noah_sfcdiff_cires_ugwp.xml

suite_FV3_RRFS_v1beta.xml

​

​

CCPP Suite names: The problem

• Okay, so we don’t have explicit rules. But we do have conventions that seem to have fallen around the following standard:

​

�

suite_FV3_GFS_v16.xml

suite_FV3_GFS_v17_p8_mynn.xml

suite_FV3_HAFS_v1_thompson_tedmf_gfdlsf.xml

suite_FV3_RAP_noah_sfcdiff_cires_ugwp.xml

suite_FV3_RRFS_v1beta.xml

​

​

…Every part of this de-facto standard has problems!

Duplicate information

Extra words and letters of unclear meaning

References to specific operational/prototype products

“.xml” is fine

CCPP Suite names: The problem

• What is the problem with duplicate information in suite names?��suite_FV3_GFS_v17_p8_mynn.xml
• Not all bad:
• At a glance, you can tell that this is a CCPP suite file
• Including “FV3” in the filename indicates what dycore it was intended for
• In practice, there shouldn’t be any confusion if this info is removed
• Inspecting the contents of the file makes it obvious �(First tag is “<suite name=”)
• Suite files for each dycore (fv3atm, SCM, NEPTUNE) are stored in their respective repositories
• Bad outweighs good:
• Makes filenames longer than needed
• There is a hard limit of 37 characters; this is needed due to character limitations when creating caps for some compilers
• Can be very misleading when comparing across repositories (see next slide)

​

​

CCPP Suite names: The problem

• What is the problem with duplicate information in suite names?
• Further problems: “identical” named suites�are not identical across dycores!
• suite_SCM_HRRR.xml and �suite_FV3_HRRR.xml should theoretically�be identical aside from potential dycore-�specific interstitials, but in reality there are �many differences!
• This problem appears in many different suites

​

​

​

CCPP Suite names: The problem

• What is the problem with referencing operational/prototype products in suite names?

Problem 1: suite_FV3_RAP_noah_sfcdiff_cires_ugwp.xml

• RAP is a WRF-based product! It will not have the same schemes, and it will not call them in the same order as in the UFS
• Even if all the same schemes are called in the same order (highly doubtful), the schemes used are not CCPP schemes, and the source code can be very divergent.

​

​

CCPP Suite names: The problem

• What is the problem with referencing operational/prototype products in suite names?

Problem 2: suite_FV3_GFS_v16.xml

• Similar problem as 1: GFS v16 does not use CCPP
• Ignoring that issue, by the time a product becomes operational, the suite file will likely have changed from the version in the UFS Weather Model implementation
• GFS version 16 went operational in March 2021; by May 2021 the “GFS_v16” suite already had a new scheme in it, and there had been no official release in the meantime
• If a suite undergoes changes during pre-operational testing, we can’t just count on changing the suite file in the weather model to match, because release timelines do not correspond with operational timelines, even if we wanted them to!
• Even if the suite is exactly the same, the source code for the individual schemes will be different!
• Will be addressed at a later date…

​

​

CCPP Suite names: The problem

• What is the problem with referencing operational/prototype products in suite names?

Problem 3: suite_FV3_GFS_v17_p8.xml

• Similar problem as 2: this suite is not the same as that in GFS v17, prototype 8
• Even worse, the average user is not familiar with operational naming conventions; they will just see “GFS_v17” and assume that’s what it is

​

​

CCPP Suite names: The problem

• What is the problem with additional details?
• suite_FV3_GFS_v17_p8_mynn.xml; this logically should be the same as suite_FV3_GFS_v17_p8 but with MYNN presumably added
• Is it MYNN surface or MYNN-EDMF, or both? (it’s the second one)
• What is the connection of a modified prototype of the GFS_v17 suite to the final product? Are the results even meaningfully similar?
• suite_FV3_HAFS_v1_thompson_tedmf_gfdlsf.xml
• There isn’t actually a suite_FV3_HAFS_v1.xml suite, so what is this difference conveying?
• Both HAFS A and B already use TEDMF, but neither use GFDL sfc…so what do these extra words in the name mean?
• At what point does a suite have so many differences that it shouldn’t be considered similar to the original?
• If further changes are made, will we just keep extending the filename indefinitely?

​

CCPP Suite names: The problem

• Hypothetical Scenario 1A: I am attending a conference, presenting my results on experiments with SRW v3.0.
• I say I used CCPP suite FV3_GFS_v17
• This conveys the following meaning to the audience: “I used the same physics as GFS version 17”
• Incorrect assumption, due to differences in physics code and potentially even different schemes
• What I really should say is I used the CCPP suite similar to the operational physics suite for GFS version 17, but with some differences
• I used this hash of the weather model from this date, which is (list differences from operational version)
• Currently, pretty much no one does this
• Someone responsible for the actual GFS v17 is in attendance, and takes issue with your characterization of your physics as the same as GFS v17
• Sad face emoji

CCPP Suite names: The problem (summarized)

• Current CCPP Suite names have a number of problems
• They are bloated with duplicate information
• They try to convey too much information about the operational product they are/were meant to emulate
• It’s not just that the information conveyed is incomplete, it’s actively misleading
• They are appended with ambiguous keywords

​

CCPP Suite names: The solution

• Why should the name of the suite attempt to convey so much information about its contents?
• Let’s just use meaningless/nonsense names
• Initial proposal: bird names (bluebird.xml, robin.xml, seagull.xml, etc.)
• This gives pronouncable, unique, easy-to-remember names
• Avoids misconceptions due to misleadingly named suites
• Puts onus on user to understand a suite’s contents rather than relying on implications from potentially misleading filename
• Puts onus on writers/presenters to describe a suite’s contents rather than relying on implications from potentially misleading filename
• Avoids any ambiguity due to arbitrary suffixes
• Removes concerns about filename length

​

CCPP Suite names: The solution

• Hypothetical Scenario 1B: I am attending a conference, presenting my results on experiments with SRW v3.0.
• I say I used CCPP suite “jackdaw.xml”
• This conveys little meaning to the audience, which forces me to clarify more
• So I say I used the CCPP suite similar to the operational physics suite for GFS version 17, but with some differences
• The suite uses this list of schemes
• I used this hash of the weather model from this date, which is (list differences from operational version)
• Compared to Scenario A, there is no confusion due to unspoken assumptions
• Happy face emoji

Potential rebuttals to the solution

• What is wrong with the current proposal? An operational suite prior to operation gets the prefix “pre_”, the prefix is removed once that operational suite is implemented, and once development moves on the prefix becomes “post_”
• This does not solve problems mentioned earlier:
• Community release timelines and operational implementation timelines will rarely (if ever) line up
• This also creates new problems!
• Renaming suite files is very disruptive to workflows! It ruins back-compatibility, requires lots of documentation changes, and introduces ambiguity when trying to replicate results.
• If the operational product changes suites, do we change our suite file to match?
• If we do, then we have two versions of the same suite floating around: not good!
• If we don’t, then we have an ambiguous suite file: not good!
• Naming is ambiguous: what is “post_GFS_v16”? How is it different from the operational implementation? Are those differences meaningful?
• What to do about suites not tied to an operational development cycle?

Potential rebuttals to the solution

• If suite names don’t reference an operational product, I have no idea which suites are good for which applications
• As it currently stands, the references to operational products are misleading: you might think that you are using “GFS_v16” physics, but your physics through CCPP are actually completely different to those used in the operational product
• This is a problem that can only be solved with documentation, and taking the lazy solution (just slap the suite named “GFS” in there) prevents users from actually thinking about what suite is best for their experiment.
• If you actually want to replicate the physics of an operational product, look up the documentation of that product to find out which suite was used
• Our CCPP documentation already does this!

​

Potential rebuttals to the solution

• If suite names don’t reference an operational product, I have no idea which suites are good for which applications
• Additionally, the application that created a suite is not necessarily a good indicator of what applications it should be used for
• Example: a suite used for a regional, high-resolution prototype such as RRFS may or may not also be appropriate for global applications, or hurricane applications, or S2S applications.
• This is another problem that should be solved with documentation, not lazy implications from filenames

The key to making this work: documentation

• We need suite-specific documentation in each repository
• Descriptions of the suites as a list of schemes
• Intended applications
• E.g. “Suite ‘Puffin’ is intended for use on large-scale grids (continental-to-global scales) at resolutions coarser than 3 km, in tropical and mid-latitude weather regimes”
• Ideally, include further justification for these intended criteria (e.g., it was tested and vetted in this study, or is similar to this operational product)

CCPP Suites: The solution (summarized)

• Going forward, all new suites will follow these criteria
• If bird names are exhausted (unlikely), or a change is desired for some other reason, a new category could be used (e.g. trees, cities, elements, etc.)
• Code will be introduced to map old suite names to new
• Documentation should be increased/expanded with not only descriptions of the suites as a list of schemes, but the intended applications
• Operational products should advertise and document which suite they use
• This way people can emulate operational products if they want, with the appropriate caveats

Action items

• Documentation needs to be expanded
• CCPP scientific documentation should include more details about the intended applications for each suite
• CCPP technical documentation should include these guidelines for suite naming
• CCPP Framework
• Remove the requirement that suite files start with literal string “suite_”
• Add support for human-readable “description” field in Suite XML files
• Code in framework that will allow the host to specify a mapping from old suite files to new
• Future capabilities: A tool to more easily show the differences between two suites

​

Questions? Complaints?

• There is no perfect solution, but I hope I have demonstrated that this proposal is significantly better than the status quo.

​

• New suites for CCPP v7:
• magpie.xml (currently: SCM_GFS_v17_HR3)
• pheasant.xml (currently: SCM_GFS_v17_HR3 + RRTMGP)
• owl.xml (currently: RRFS_v1)
• These names were suggested by ChatGPT

Bonus

• Slides
• Go
• After
• This
• Slide