Facility Testing of E4S at NERSC��2023 ECP Community BOF Days�February 15, 2023, 3:00 pm Eastern time�

Shahzeb Siddiqui

HPC Consultant

Lawrence Berkeley National Laboratory

​

1

Our Testing Approach

• How do we decide what packages to test?
• Determine what is installed in the NERSC E4S stack by running spack find and identify possible test candidates
• Where do test come from?
• spack test
• E4S Testsuite
• Standalone test
• How do we monitor test?
• CDASH
• Meet Biweekly to go through test failures, open issues and CDASH results
• What do we do with test failures?
• First we report issues to https://github.com/buildtesters/buildtest-nersc/issues to error is captured
• Issues related to E4S Testsuite or Spack Test go to their respective projects
• Do we continuously run the same static test?
• Yes
• Test are run until we officially end support for E4S stack, then we remove test from the repository. See https://docs.nersc.gov/applications/e4s/#e4s-support-timeline for more details

2

Buildtest at NERSC

• What is it?
• A git repository using buildtest to run tests primarily focused on testing facility deployments of E4S
• Where is it?
• Upstream Repo: https://software.nersc.gov/NERSC/buildtest-nersc
• Mirror Repo: https://github.com/buildtesters/buildtest-nersc
• How does it work?
• Leverages Gitlab CI/CD using scheduled pipelines to run a subset of tests
• Uses a custom gitlab runner (Jacamar) to run tests via a single user account
• All test are run using the devel branch of buildtest
• Where do the test get published?
• CDASH at https://my.cdash.org/index.php?project=buildtest-nersc

3

CDASH

• Buildtest can upload test results to CDASH
• CDASH produces color coded charts, timelines, and links to test output
• Comprehensible presentation of test results posted to the web for users and developers

4

CI/CD Pipeline

5

Spack Test running hdf5

https://my.cdash.org/test/71797437

6

How to get involved

• If you would like to have your package tested in the NERSC E4S Facility stack, please open an issue in the buildtest-nersc project.
• You can join the #buildtest-nersc channel in buildtest slack if you need help contributing test.
• We have biweekly meetings to go over the buildtest-nersc project, including open issues, and test results in CDASH. If you need assistance, please contact us and we can share link to the meeting.
• Need help with contribution? Please see CONTRIBUTING.md

7

References

Buildtest

• Documentation: https://buildtest.rtfd.io/
• Source Code: https://github.com/buildtesters/buildtest
• Slack: http://hpcbuildtest.slack.com/
• Buildtest at NERSC: https://github.com/buildtesters/buildtest-nersc
• CDASH for buildtest-nersc: https://my.cdash.org/index.php?project=buildtest-nersc

E4S

• E4S Home Page: https://e4s.io
• E4S Docs: https://e4s.rtfd.io/
• E4S Testsuite: https://github.com/E4S-Project/testsuite

​

​

​

Spack

• Documentation https://spack.rtfd.io
• Source Code: https://github.com/spack/spack

GitLab

• GitLab Runner Docs: https://docs.gitlab.com/runner/
• GitLab CI/CD: https://docs.gitlab.com/ee/ci/

Jacamar CI

• Documentation: https://ecp-ci.gitlab.io/
• Source Code: https://gitlab.com/ecp-ci/jacamar-ci

​

8

ACKNOWLEDGEMENT

This research was supported by the Exascale Computing Project (17-SC-20-SC), a collaborative effort of two U.S. Department of Energy organizations (Office of Science and the National Nuclear Security Administration) responsible for the planning and preparation of a capable exascale ecosystem, including software, applications, hardware, advanced system engineering and early testbed platforms, in support of the nation’s exascale computing imperative.

9