1 of 7

Software Update

Brian Lin

Area Coordinators Meeting - April 21, 2021

2 of 7

Release Series

  • OSG 3.6 released!
    • Major highlights: dropping GridFTP/GSI, consolidating Gratia probes, transitioning to a rolling release, container tags that closely track OSG Yum repositories
    • Lots of Python 3 work: osg-configure, gratia-probe, HTCondor-CE, osg-build
    • Next steps: release XRootD and GlideinWMS
    • Created per-release series “upcoming” Yum repositories: 3.5-upcoming contains software with GSI + SciToken support
    • Added SciToken-based job submission tests (also added to 3.5-upcoming)
  • Concerns
    • SciToken support not yet there in the rest of the infrastructure; a handful of instances of sites updating/asking about updating to 3.6
    • GSI-based job submission tests missing from 3.5-upcoming

2

3 of 7

Enterprise Linux 8

  • Software Team attended a few HEP meetings about their OS plans: RPM is still a safe bet
  • WN software (CVMFS, client) is well-supported today
  • Built and tested GridFTP
  • HTCondor-CE 5.1.0 and HTCondor 9.0.0 releases upstream will open the floodgates for releases of the other EL8 packages
  • Concern: RHEL academic licensing is being discussed as an option by CERN/FNAL

3

4 of 7

Compute Federation

  • Compute Entrypoints:
    • Software along with Ops/Facilitation demonstrated end-to-end job submission to Bridges2 and Expanse with pilot usage counted against user allocations
    • SIUE: running a self-hosted CE in front of their Ubuntu 20 cluster with CVMFS!
  • Backfill containers
    • Admins of topology registered “execution endpoints” may now request tokens so they can immediately start contributing: https://os-registry.opensciencegrid.org/
    • 2.86M core hours over the past 30 days (GRACC)
    • Images now rebuilt at least weekly to follow Software container policies
    • Working on adding CVMFS support directly into the container
  • Concern: The certificate/DNS-free CE work project is still effort starved

4

5 of 7

Containers

  • Hosted 6 hackathons to work on Kubernetes/container related tasks and to educate internal staff/collaborators on Kubernetes and the GitOps model used on the Tiger cluster at Morgridge
  • Deployed an EL8 Koji RPM building worker on Tiger (necessary for EL8 builds, thanks to UChicago for holding us over until now!); working on a spare instance to test patches and more frequent updates
  • Assisted Ops with transitions of Topology and YUM repo services to Tiger
  • Starting to see more adoption of containers, especially for Frontier Squids: ATLAS is starting to use a federated operational model for ATLAS T2 site Squids
  • Next steps: develop internal tooling/strategies to produce release candidate images, improve ease of kicking off manual rebuilds, and add the ability to inspect images for important underlying software versions

5

6 of 7

Other

  • Software made major contributions to HTCondor
    • Provided development, coordination, testing, and documentation effort for HTCondor-CE 5.1.0
    • Assisting with improving build and release processes, such as improved GitHub integration and Ubuntu/daily build containerization efforts
  • Working with CMS and Operations to standardize Topology, Hosted CE, and Factory configuration for HPCs
  • Assisted the XRootD team with drafting a version support policy
  • Continued to provide technical support in campus engagements (GP-ARGO, UTC, Villanova)
  • More tool/workflow development for various PATh metrics

6

7 of 7

7