1 of 11

State of ARC

Maiken Pedersen�On behalf of the ARC development team

NDGF AHM 2021-1

2 of 11

ARC development focus last years

3 of 11

Release strategy

  • Scheduled ~bi-monthly releases
  • All commits except backward incompatible changes are committed directly to master
  • No clear distinction between a bugfix and minor release (change to previous)
    • We inform in the release notes when there are changes other than trivial bugfixes
    • All scheduled releases are numbered X.Y.0 (e.g. 6.11.0)
  • If an emergency/express release is necessary due to an important security patch or a blocker bug – release is numbered X.Y.Z where Z != 0 (e.g. 6.10.1)
  • No “golden release” → no patching of earlier releases (except security patches of last major release, see later)

  • Backward incompatible changes are released in a major release (e.g. 7.0.0)
    • Major releases occur between 1-3 years

  • Only nordugrid-arc released with an ARC release – no more release bundle (no docs, nagios, gangliarc)
    • 🡪 Release name changed from year.month.u.# to ARC 6.0.0
    • ARC nagios plugin still present in nordugrid repo, but follows its own release cycle, maintaned by Petter Urkedal (NBI). https://source.coderefinery.org/nordugrid/arc-nagios-plugins

4 of 11

ARC on GitLab and GitHub

  • https://source.coderefinery.org/nordugrid/arc
  • Mirrored at https://github.com/nordugrid/arc
    • We can receive pull-requests here, they are then automatically copied to Gitlab for further processing
  • Nightly builds for all supported platforms on http://builds.nordugrid.org/nordugrid-arc/master/
  • Getting the released packages

5 of 11

Ongoing developments for ARC 6�APEL ActiveMQ 🡪 ARGO MS

  • EGI running  campaign of migration from ActiveMQ message brokers to ARGO messaging service → APEL publisher switches to ARGO
  • ARC changes are ready for testing: https://source.coderefinery.org/nordugrid/arc/-/merge_requests/1168
    • Scheduled for release before APEL moves to ARGO which is end of May
    • Need testing done well before that!
      • If not we will have to ask for another extension

6 of 11

Ongoing developments for ARC 6-7�Tokens

  • Ongoing work
  • Prototype in place since ARC 6.6: http://www.nordugrid.org/documents/arc6/misc/oidc_tokens.html
  • Workshop in June for sorting out requirements
    • We will attend
    • Hopefully delegation requirements will be fixed then

7 of 11

Ongoing developments for ARC 6�REST interface

  • Implemented in 6.9.0
    • Simplification of WS interfaces - skips all SOAP related things , HTTP based REST
    • Can be used with simple clients based on something like curl for example
    • Easy to integrate into third-party clients which do not want to adopt ARC library
  • Currently being tested
    • Tested and working in aCT
    • A couple of minor client bugs which will be fixed in 6.12
  • http://www.nordugrid.org/documents/arc6/tech/rest/rest.html

8 of 11

ARC 7

  • No dedicated effort as of yet → release probably not before 2022
  • Issue for tracking plans or wishes for release: https://source.coderefinery.org/nordugrid/arc/-/issues/84

9 of 11

Development plans for ARC 7 (or 7++)

Following the technological development

  • Support of token-based security layer
  • Going towards container orchestrators
    • So far via RTE’s – jobs can be run in docker or singularity containers
    • We support docker for htcondor
    • Singularity
  • NEW ARC deployment mechanisms: ARC installation via containers

  • DATA subsystem scalability improvements
  • Controldir scalability improvements
  • Phasing out of gridftp/ldap
    • HTCondor support for REST there will be no need for them anymore

  • Dream?
    • LRMS-less ARC: an ARC CE that can be deployed directly on top of set of dedicated resources
    • Autonomous ARC?

  • Need client-developer!

  • NeIC Open Call 2021 – hope for ARC+Galaxy development project
    • Submit jobs to HPC via Galaxy and ARC

Always ongoing: improving code-base for scalability and stability

10 of 11

Wishes for ARC 7?

11 of 11

Conclusion

  • Day-to-day business for ARC
  • Not yet much focus on ARC 7, but will come…
  • Crossing fingers for NeIC project!