Introduction to BNL Physics Dept’s new

Nuclear and Particle Physics Software (NPPS) Group

Torre Wenaus

NPPS Group Meeting 1

May 8, 2019

https://indico.bnl.gov/event/6124/

Background

• Discussions over the last 2 years on how to consolidate NP and HEP software activities at BNL into a common effort that strengthens both
• Discussions converged this year on a plan to create the NPPS group
• Consolidate strong software teams in PAS and STAR
• Continue and grow the successes of HEP and NP software at BNL in PAS and STAR and elsewhere in the Department
• Plus… that word… synergy… 1 + 1 > 2
• Support and build on the growing list of experiments & programs with existing or potential S&C roles for BNL
• STAR, EIC, sPHENIX, ATLAS, Belle II, DUNE, LQCD theory, Astro, ...
• Complement and leverage CSI with a comprehensive software group in Physics
• RACF successfully consolidated facilities across NP and HEP two decades ago, it’s time we did it in software!
• Have been developing and iterating plans with experiments and management and experiments since early Feb, yesterday we arrived at ‘ready’

2

T. Wenaus May 2019

Mandate

• Consolidate software efforts across HEP and NP into one group, a consortium of experts on NPP software, infrastructure, services and tools, sharing knowledge and coordinating efforts
• Top priority: serve experiment operational needs, with continuity of support and responsibilities when creating the NPPS group. Don’t break things!
• NPPS members are experiment collaborators, reporting along experiment reporting lines
• Cannot emphasise this enough. Constantly raised by experiment leads & others.
• Perform development and R&D with near and long term objectives aligned with the Department’s strategic plan and vision
• Strength in S&C is part of BNL’s vision for technical and intellectual leadership in key NPP experiments

​

3

T. Wenaus May 2019

Mandate 2

• Foster the engagement of BNL with wider NPP S&C community & projects
• The group as a consolidation of strong experimental software teams in both nuclear and particle physics, is unique among US Laboratories
• Seek opportunities to draw on this strength and past successes and expand its reach for the benefit of US and international nuclear and particle physics programs
• The focus of the group on common, cross-cutting projects will allow it to present a strong case for support in an environment that strongly favors common cross-cutting efforts over ones focused on single experiments
• Be proactive in seeking funding via LDRD, program development funds, DOE ASCR, DOE HEP Computing, and DOE NP, with an emphasis on common projects and leveraging the group’s cross-cutting scope & expertise

​

4

T. Wenaus May 2019

Membership

• Decided through consultation between experiments, individuals, Department Chair, NPPS
• Initial membership 19 people
• Involvement from beyond the membership, beginning with the leadership team and extending to the technical teams/forums
• We’ll engage people from RACF, Omega, EDG, PHENIX, STAR, CSI, …
• Group structured so that non-members can collaborate fully and effectively
• Make group boundaries transparent for those working together on projects
• Foster bottom-up organization of working teams
• Hence the technical team structure of the group
• This group began as a top-down mandate of the management but we’re building it as a bottom-up collaboration around technical topics in support of our experiments

5

You will have seen the FAQ

T. Wenaus May 2019

Membership - who we are

19 people from

• 4 groups
• ~6 experiments
• 2 sites

with diverse expertise and operations responsibilities in their experiments

6

T. Wenaus May 2019

Leadership team

• Group is managed by a leadership team consisting of
• Group leader answerable to the Department Chair for the group’s program
• Works in close collaboration with the rest of the leadership team in all aspects of planning and managing the group’s activities and overseeing its personnel
• 60% of Torre’s time (rest is US ATLAS, mostly HL-LHC computing R&D)
• Leadership team members represent the principal experiments and programs of the group
• Not symmetric: some have substantial-to-moderate size software teams (ATLAS, STAR, Belle II), some far from it (sPHENIX, DUNE, EIC)
• Representation on leadership team reflects this asymmetry: sPHENIX, DUNE, EIC reps are liaisons and not NPPS members
• Leadership team membership (bold = NPPS member)
• ATLAS - Alexei Klimentov
• Belle II - Paul Laycock
• DUNE - Brett Viren
• EIC - Alexander Kiselev
• sPHENIX - Chris Pinkenburg
• STAR - Gene Van Buren

7

T. Wenaus May 2019

Leadership team responsibilities

• Group leader shares and delegates responsibilities within the leadership team, engaging the team in deciding the division of responsibilities
• Setting plans, goals and priorities for the group is the collective responsibility of the leadership team
• Leadership team members
• serve as liaison between the group and their respective experimental/scientific communities
• oversee technical team activities in their domain
• serve (not exclusively) as point of contact between the group and funding agencies
• identify funding opportunities and take the lead in developing proposals
• The leadership team conducts the day to day work of managing the group via regular (initially weekly) management meetings and ongoing discussions, email etc.

8

T. Wenaus May 2019

Contact leads and liaisons

• As Hong said in the announcement, close coordination of the NPPS group with the rest of the Department is essential for this model to succeed
• To that end, lead contacts / liaisons are in the process of being established to stakeholders inside and outside Physics
• In addition to the experiments/science areas represented on the leadership team:
• SDCC/RACF
• LSST and other astroparticle
• LQCD - HEP and NP
• CSI
• Computer Science and Mathematics
• Computational Science Lab
• Center for Data-Driven Discovery
• More as required

​

9

T. Wenaus May 2019

Technical teams

• NPPS members will be on one or more technical teams
• Technical teams carry out the work of the group
• Tech team leaders (or co-leaders) have lead responsibility for the team’s activities, deliverables, vision
• Identify opportunities for projects, common efforts and develop the means of pursuing them
• Developing the means = acquiring/identifying the effort, without overburdening existing people
• Reporting lines for experiment deliverables remain with the experiment
• Again, the tenet of the group: continuity of experiment responsibilities & deliverables
• Starting point for team members is continuity of experiment responsibilities
• Initially the tech teams effectively operate more like discussion forums
• Commonality begins with discussion & communication, extends to real common effort when the projects and the manpower are identified
• Technical team leaders will over time share managerial responsibilities with the NPPS leadership, decided case by case based e.g. on experience
• Performance evaluations will initially be done by the group leader, in close consultation with others (in and outside NPPS) overseeing the person’s work

10

T. Wenaus May 2019

The technical teams

• The teams:
• Simulation (e.g. STAR, sPHENIX, EIC simu; ATLAS fast simu on HPCs)
• Core software & advanced algorithms (e.g. accelerator-ready algorithms & data models)
• Reconstruction (e.g. sharing TPC expertise)
• Databases (e.g. sharing conditions DB expertise/software)
• Workflow and workload management (PanDA and its integrations e.g. Pegasus)
• Distributed data management (Rucio for Belle II, DUNE, sPHENIX?, STAR?)
• Software support (e.g. continuous integration, software stack support, supporting production/analysis/dev, NPPS-supported services on RACF/EC2 provisioned platforms)
• User interfaces, monitoring & analytics (e.g. commonality among UIs, analytics tools applied to experiment needs)
• The teams grow as commonality and common projects grow
• Day 1: their function is to foster communication & collaboration around a topical area: basically a discussion forum
• Day N: they graduate to a technical team when projects+effort emerge

11

T. Wenaus May 2019

Forums

• In a few areas, NPPS will not host much effort, but should work as a facilitator and catalyst for communication and collaboration
• In such areas NPPS will host a forum rather than a technical team
• Good forum candidates identified so far:
• Streaming DAQ and real-time analysis
• Great interest and activity across many experiments
• TDAQ/online is outside NPPS scope but much online/offline commonality in software
• Collaborative tools, websites, documentation and training
• A very common request in discussions is for NPPS to please help in this area!
• NPPS can add its voice, focus discussion, and work directly when supported to do so
• meeting/agenda tools, single sign-on, GitHub Pages, wikis, Drupal, Sharepoint,
• documentation tools, best practices
• training tools and programs
• Will organize this forum jointly with SDCC/RACF

… and to be included somewhere, not yet discussed: data preservation

12

T. Wenaus May 2019

Coordination and communication

• Coordination with stakeholders is via a coordination team consisting of NPPS leadership, experiment/stakeholder contacts, and technical team leaders
• Meetings:
• Every week, a general open group meeting with standing agenda
• Top-of-the meeting items & news, tech team highlights, focus topic
• Open to all; reserve items of interest only to NPPS members to end of the meeting
• Every month a broadened open meeting of the coordination team on broader planning topics
• Group meeting notes maintained and posted
• Technical teams organize their own meetings as needed
• Monthly meeting between Physics management and NPPS, SDCC leaders
• Will organize a seminar series on S&C topics within or of interest to HENP
• Joint with SDCC/RACF
• Web presence on an open source model in github (propose: bnlnppsoftware)
• Group calendar (bnlnppsoftware@gmail.com) to share events of interest to NPPS
• Any leadership team member can add items, happy to add others as well

13

T. Wenaus May 2019

Weekly meeting

Preferred slot: Wed 10 am, 1 hour limit (followed by 1hr leadership team meeting)

Or: same arrangement Fri 10am

Can we agree now on Wed 10 (or failing that Fri 10)?

​

14

T. Wenaus May 2019

Administrative support

• The Department is in the process of hiring a new administrative support person for the group
• Likely to be shared with another group
• Office will be 1-220 (Torre’s old office)
• Until then, Linda and Rachel have kindly agreed to continue to support us
• So for now, your admin support does not change

15

T. Wenaus May 2019

Early priority activities: NP

• EIC
• Important to increase the visibility and participation of BNL in EIC software
• Expand the impact of our busy EIC software experts by giving them help
• JLab is much more visible, even though BNL is strong in the software
• Working on ways to increase BNL visibility, leverage our software strengths
• Take advantage of tutorials to show our stuff
• Trieste software workshop, BNL detector R&D meeting, Paris EIC UG workshop, …
• Representation in coordinating software activities (consortium, software WG)
• Torre will join Markus Diefenthaler (JLab) and Andrea Bressan (INFN) as EIC UG SW WG co-convener
• Joint (with JLab) seminar series, BNL organizing the next “Future of NP S&C” workshop, ...
• 2019 LDRD: requesting 1 postdoc x3yrs on putting usable practical software in the hands of users
• sPHENIX
• As documented in the review last spring, much to do, much opportunity to draw on BNL expertise to help
• Consolidate STAR, sPHENIX TPC expertise
• Use BNL strengths in distributed computing to bootstrap sPHENIX into a distributed model as needed
• Common interests in streaming DAQ, real time analysis
• Need for a computing model; sPHENIX has just established a computing task force
• 2019 LDRD: requesting 1 postdoc x3yrs on sPHENIX offline software including Exascale (MT safe, high concurrency, memory requirements, efficient parallel I/O)

16

Waiting on whether we’re approved to proceed

with 2 postdoc x 3 year LDRD proposal

T. Wenaus May 2019

Early priority activities: Computing model WG

• Request from Elke: please help with software costing for an eRHIC detector, as input to eRHIC detector costing (complete in the summer)
• One of several current or coming areas in which we need to contribute to computing model like activities
• DUNE is giving attention to a computing model
• sPHENIX needs to develop a computing model
• ATLAS will soon launch (with WLCG) development of HL-LHC computing model
• An immediate priority is to establish a computing model WG to address these
• Membership right now == the NPPS leadership team
• Looking to add people, inside NPPS and outside (e.g. RACF)
• Not everyone will have equal involvement in all activities but the WG will be a repository of the common knowledge and expertise that will contribute to all

17

T. Wenaus May 2019

Early priority activities: DUNE

• This is a key moment for DUNE S&C: DUNE is actively establishing who does what in S&C
• This is the time for BNL to step forward with effort as well as expertise to secure responsibilities
• Leverage our capabilities to secure responsibilities which can then later be funded by DUNE ops
• Department seemingly receptive to using program development funds to bootstrap effort if responsibilities can be agreed with DUNE & FNAL
• Discussions underway, negotiating in our areas of strength
• Wire-Cell (Brett’s TPC reco)
• Distributed computing
• Conditions DB
• And we should be involved in Data Model/Computing Model development
• Will participate in an Aug 2019 3 day workshop on DUNE computing model development
• Proposing to host it here

18

T. Wenaus May 2019

More early priority activities

• Collaborative tools
• Improvements in this area requested by many
• Some work underway for EIC (funded!)
• NPPS role mainly fostering communication, catalyst, voice for experiments, with much overlap with SDCC/RACF
• Will run a forum on this jointly with RACF
• Organize a meeting soon addressing experiment/stakeholder needs & priorities, and tool/infrastructure status and timeline
• Background is a draft look at collaborative tool needs put together ~2 years ago (Tom Throwe, Jerome Lauret)
• HPCs as an accessible accelerator-enabled resource
• ATLAS-CSI project developing ML and GPU-access services for HPCs (e.g. Summit)
• Has application to all interested in using ML and/or GPUs at significant scale
• e.g. DUNE GPU project now involved
• Real-time data analysis
• Already two meetings/workshops this year; launch the forum to carry this forward

19

T. Wenaus May 2019

Planning and oversight

• Annually the NPPS leadership in consultation with coordination team develops and proposes the plan for the following fiscal year
• Software, infrastructure, tools and services that need to be developed and supported
• New R&D projects which may benefit all or specific experiments
• Corresponding FTE and budget requirements and the proposed share across experiments are also developed
• Discussed and approved with all experiments, funds distributed accordingly
• NPPS FTEs that are funded directly by each experiment’s Operation funds are defined and agreed in the annual plan
• Shared needs across experiments should be defined in writing and budgeted
• e.g. if BELLE-II needs 30% of a person from STAR to work on database, and STAR agrees, the BELLE-II program should pay for the 30% FTE
• Reviews initiated by NPPS or Department leadership as needed

20

T. Wenaus May 2019

Seeking support for new effort

• Realizing NPPS objectives requires new effort
• Cannot add common project mandates to 100% committed people
• Department has gone to the trouble of creating the group -- should now bootstrap us into being able to leverage it in key strategic areas!
• Submitted a wish list for NPPS priorities as an LDRD 2019 abstract
• While recognizing that not all topics will necessarily be appropriate to LDRD
• e.g. program development funds
• Got a departmental ‘yes’ on sPHENIX and EIC, waiting to see if we’re asked to submit a full proposal
• Will look for others, external and internal, over time
• Experiments
• Agency funding opportunities

21

T. Wenaus May 2019

NPPS and STAR (presented to the STAR management)

• First and foremost message from STAR: continuity, don’t break things
• Top priority: serve experiment operational needs, with continuity of support and responsibilities
• What STAR can do to help make the changes as smooth as possible?
• Mission of the NPPS group is to gain the trust of STAR in reliably delivering on expectations
• STAR making the expectations clear will help
• Formality of that is up to STAR; requirements, deliverables, etc.
• The best insurance it will work well is it’s on the same model as up to now: STAR people in NPPS are STAR people just as they have been in the STAR group, participating in STAR meetings, answerable to STAR reporting lines and deliverables
• What we can hope/expect from the new group with regards to computing in STAR
• Frankly, early NPPS priorities for effort over and above ‘continuity’ identified in discussing and planning the group are in other areas: sPHENIX and EIC in NP, DUNE in HEP
• Strongest message from STAR is “don’t perturb us”; continuity is our initial focus

22

T. Wenaus May 2019

NPPS and PAS (presented to PAS)

• Not a whole lot changes on May 6 for former PAS members, continuity is foremost!
• Getting the tech teams -- er, forums -- going will broaden our awareness of software in the department, opportunities for commonality, sharing effort and knowledge, etc.

23

T. Wenaus May 2019

Ramping up the group

• Foremost message: continuity in one’s workday, no change to responsibilities
• Leadership team up and running since late March
• Pursuing opportunities for getting in new effort
• LDRD: sPHENIX and EIC
• DUNE: working on a case for program development funds
• Let’s collectively look for more
• Establish group infrastructure: meetings, mailing lists, calendar, web presence, github etc
• Volunteers to help appreciated
• Set the landscape, get to know one another and the opportunities
• Surveying the experiments, and then the technical areas, in the first meetings of the group
• Activate forums and (initially forum-like) technical teams, with priority to those aligned with our early priorities, and begin organizing initial activities
• Pursue the opportunities and mandates we’re already aware of
• Getting our EIC software out there to users
• Computing model & related for EIC, DUNE, ATLAS

24

T. Wenaus May 2019

Message to all

​

​

​

Always ready to discuss questions, concerns, ideas!

25

T. Wenaus May 2019