​

Parallel Sessions on SANPC for Experiments

​

Discussion Highlights

Discussion Points

• Lessons Learned: Software Maintenance, Sustainability, and Stewardship
• Lessons Learned: Common Scientific Software
• Lessons Learned: Data and Analysis Preservation
• Opportunities: Novel Approaches
• Lessons Learned: Partnership Between Host Labs and Experiments
• Lessons Learned: Development and Retention of the Workforce
• Lessons Learned and Opportunities: Experiment-Theory Workflows
• Wishlist: Support by Funding Agencies

2

Key Points for the White Paper

• Ensure Success for Software & Computing in NP:
• Encourage partnership between hosting institutions and experiments, between facilities, and between experiments.
• Data and analysis and knowledge preservation, enabled by institutional memory, driven by workforce development.
• Foster Software R&D:
• Incentives for software development and innovation.
• Verification and validation as integral part of the development process.
• Testbeds and cross-cutting efforts for adapting novel approaches and technologies.
• Workforce Stewardship:
• Career support, career path, career opportunities essential for building work force.
• Training essential.
• Funding of Computing within projects/experiments
• Recognize software & computing and continuous R&D and as vital aspects of projects.
• Support cross-cutting initiatives:
• Expanded support for cross-cutting initiatives between experiment-theory in NHEP.
• Data Physics: Qualitatively New Distincion
• New career title/job description: Data Physicist
• New funding path within FAs for the new type (DP) proposals.

3

​

Parallel Sessions on SANPC for Experiments

​

Discussion Summary

Lessons Learned: Software Maintenance, Sustainability, and Stewardship

• Not separate development and operations.
• Scale of the collaboration / project defines possible scope.
• Adapt widely-used software, avoid not-invented here syndrome.
• Key: training as part software development and operations, understand how and for which tasks software can be used.
• Key: verification and validation as integral part of the development process
• Reproducibility
• Best practices for software development and deployment
• Key: Modularity and Portability
• Key: incentives for software development (recognition, career paths)
• Incorporate new tools (also from outside community)
• Impact of out of the domain (?) deprecated software
• Question about reusability

5

Lessons Learned: Common Scientific Software

The role of common scientific software in experiments:

• Common software capture know-how in the community.

​

​

Keys to successful software projects in experiments:

• Software needs to be findable.
• Idea: Catalogue of software.
• Incentive for software developer, experiments, and management.
• Tutorials on how to use software and how to develop.
• Identify needs and gaps.

​

6

Lessons Learned: Data and Analysis Preservation

• Capture discussion.
• Open Data:
• Means not just preserving data as published, but also supporting data and analysis
• Ability to “replay” an analysis or redo it using modern approaches (ALEPH & H1 case study)
• Mandated by 2022 OSTP Memo, NP guidelines being drafted by Michael Cooke (DOE-NP), but mandate currently poorly articulated
• User facilities (thinking FRIB) provide most of an analysis chain for users. Once data is taken and user leaves site, we must make sure the people with the data have the ability to archive their full analysis chain, including their own special purpose parts so that their analysis can be reconstituted by others
• Smaller experiments (e.g. single PI, university based, think ARUNA labs) will need significant help to meet new requirements
• Key for reproducibility
• ​

7

Lessons Learned: Data and Analysis Preservation

• Traditional Stewards:
• Traditional “stewards” for storing experimental data in the long term are HEPData (high energy heavy-ion data included), and EXFOR (NNDC is US lead) (low energy neutron-on-target data). There are no stewards for intermediate energies/intermediate system data (such as JLab, FRIB heavy-ion data). There are many user communities who need such data. See graphic below.
• These stewards do not �yet have ability to �preserve all required �by in-coming Open Data �mandates
• Data Management Plans �on proposals need�“Enforcement mechanism”?
• ​

8

Lessons Learned: Data and Analysis Preservation

• Lessons from ALEPH & H1 effort (see slides attached for more details)
• Foresight from the collaborations for the data preservation
• Incredible support from members in the collaboration on the technical aspects and knowledge
• Many bright young students who dug into the data collected before they were born
• Reproduction of published physics results using identical event selections
• Development of data-driven checks to understand the data
• Ability to rerun key software is crucial
• Not easy to gain control of stored information — more lower-level information will be useful
• Suggestion to do some “user test” while experiments are still ongoing

9

Incentives

Tooling and infrastructure

10

Opportunities: Novel Approaches and Technologies

• Streaming Readout:
• Testbeds.
• AI/ML:
• Reproducibility.
• Interpretability. Physis-driven.
• Heterogeneous Computing:
• Cross-cutting with other fields.
• Re-use existing tools.
• Other novel approaches, e.g., quantum computing:
• IRI-ABA.
• R&D Funding:
• The SBIR program can provide supplemental funding for a company to tackle the high-risk aspects of a new project, who can then generalize that work to apply to other labs. This reduces the duplication of effort, and frees lab developers up to take on more interesting problems. Subsequent purchases by labs (called Phase III) are eligible to be easy no-bid sole-source contracts, and are encouraged by the DOE.

​

11

Lessons Learned: Partnership Between Hosting Institutions and Experiments and Between Facilities and Between Experiment

• Many universities have small nuclear physics experimental programs in-house (e.g. ARUNA labs) and do not have the scale and access that large labs (JLab, LBNL, BNL, …) to software engineering talent and time.
• User-centered approach: Understand requirements and address them.
• Small collaborations should be contacted soon after acceptance of proposal to start addressing the software aspect of the experimental campaigns.
• Software engineers from the host institution should seek the “software lead” of the small collaboration to continue work up to and after the experiment runs.
• It would be prudent to ask for a graduate student/post doc assistant to focus on this task.
• Core group of research software developers not adequate to support ongoing facility operations and software R&D.
• Needed to provide continuity across experiments at a facility (particularly small to medium scale experiments)
• Address institutional and funding barriers to support such a software team
• Experiments not at facilities want to be able to efficiently access best practices / community-wide resources; also would benefit from access to sw engineering e.g. via allocation of laboratory staff resources

12

Lessons Learned: Partnership Between Host Labs and Experiments

• Links to discussions about common software and IRI-ABA.

13

Lessons Learned: Development and Retention of the Workforce

• Recognition, compensation.
• Career support:
• Career path.
• Career opportunities / freedom.
• Work to change the workplace culture towards software development as a scientist - need to change the narrative from work on software making the individual not a scientist
• Workplace flexibility
• Pipeline / interaction with junior scientists
• Project software engineers (&software-focused scientists) can be isolated from collaboration junior scientists by the project/science divide
• Missed opportunity to train next generation
• Working with junior people can enhance job satisfaction & recognition (jr. people write papers)
• Recognize role of software infrastructure experts in mentoring trainees
• Project $ on training?
• Software needs to be commented with a user guide that is written for the perspective of summer students/early graduate students. For a good reference - the Mona/Lisa collaboration (FRIB) guide software

14

Wishlist: Support by Funding Agencies

• Support for a group of research software developers to support ongoing science operations and software R&D
• take advantage of incredible new developments (e.g. ML/AI) to minimize time to science
• Recognize software & computing as vital aspects of projects / experiments and provide software & computing funding
• “There is no science without analysis.”
• Encourage the acknowledgement of analysis methodology within the proposals submitted and how that analysis will be funded to completion
• Recognize value of regular meetings of NP to discuss software
• Support of data and analysis preservation
• Home for computing research with crossover HEP/NP applications
• Proposals for data physicist (not pure exp/th) need a landing spot at the funding agencies
• Recognize diverging/diversifying skillsets: there are few universal software experts. There are separate roles for algorithm experts / performance computing / software engineering. A job that demands every skill will leave out specialists who can excel at one.

15

Lessons Learned and Opportunities: Experiment-Theory Research Workflows

• Common use cases in experiment and theory:
• MC event generators
• Extraction of QCFs
• Opportunities for common use cases:
• Create a library of codes that are used to predict the outcome of physics experiments. Helpful for sourcing models for proposals etc.
• Encourage work between theorists/experimentalists outside of your own collaboration
• Common frameworks between experiment and theory:
• Opportunities?
• Where do the proposals go?

16

Key Points for the White Paper / Follow up

• Follow up from the workshop: survey of the software used in NP
• Gather information on the software being used; purpose also on cross-communication between the projects / experiments to understand the commonalities and differences better
• Training for development of sustainable software?
• Use of COLLABS
• Particular software packages/framework trainings?
• Provided career paths for scientific software developers and support roles
• Particularly at Facilities to provide long-term continuity and sustained development
• ​

17