1 of 4

Vera C. Rubin Observatory

Data Management/System Performance

Data Facilities meeting

William O'Mullane & Leanne Guy - 13 Feb 2025

Wrap up

Vera C. Rubin Observatory | Lyon DF meeting | 11-02-2025

Acronyms & Glossary

2 of 4

Great meeting - thanks everyone for participating

Day 3

  • Good discussion on DP2 and DR1
  • Lots of thoughts on documentation
  • GPDR - panic - or be happy
  • Archive of RAWs at FrDF - zip files
    • In one year we will have as many files as LHC for 15 years

​

Day1

  • Nice intro from each data facility
  • Lot of discussion about data wrangling - we may need a chief wrangler ..
  • Discusses risks, mitigations and planning for facilities
    • Including dark data and much more

Day2

  • CM-service/daemon and Panda -
  • Sasquatch - RSP
  • We reviewed outstanding construction items
  • Zipfiles came up a lot

​

2

Kudos to people who stayed online for much of this from unfortunate timezones - Tim, Leanne, Steve, Michelle .. Matthew, Stellios I think in fortunate zone

Vera C. Rubin Observatory | Lyon DF meeting | 11-02-2025

Acronyms & Glossary

3 of 4

Actions ?

  1. Add items to the risk register? Third party software like rucio, PanDA -wil/adam
  2. Get wranglers in a group and empower someone the lead it - Adam
  3. Arrange a meeting (Square/TSS/ Sandrine?) to rationalize EFD schema , decide what do we keep for 10 years from summit - Frossie/wil
  4. Figure out who does what monitoring for DFs - what from application level - multisite alerts - bring in Zhaoyu Ken Herner .. - Fabio/Adam/George
  5. Clarify with IDACS about pre release and after DP1, usingto verify them for DP2 - Leanne/wil ?
  6. Formalize a set of metrics for verifying processing is the same at each site - also formalize tracts to be processed at all sites to gather those metrics. Metrics or image comparison? Identify contacts UK Fr - - Jen
    1. Decide who writes the tools ? - Yusra
  7. Define and decide if we need/have a RubinOne/Multione network — for USA-EU. IDACs? -Cristian NET-team
    • Data transfer tests - greg is sending ks of files - tell multione they have index for lsst
  8. Figure out how to have bot accounts at IN2P3 - Fabio
  9. Write up the cm-service arch in a techo note - colin
  10. Sort out TAPE SE and putting outputs on TAPE at SLAC - KT and Andy
  11. @Brian Yanny to make a ticket to test zip out in finalJob in one of our test campaigns (DM-48915)
  12. Update sizing DMTN-135 after OR5 - Adam/wil
  13. Write up and bring DR scope options to OPS-EXEC - Leanne
  14. Decide end point plan how to verify for Qserv with Fritz/Colin/Gregory/Leanne/frossie - wil
  15. Do we need a Customizeable GPDR statement for DAC sign up - other things from that session - Frossie

3

Vera C. Rubin Observatory | Lyon DF meeting | 11-02-2025

Acronyms & Glossary

4 of 4

BIG THANKS

  • Thanks to IN2P3 for hosting us !
    • Many people have been busy organising - Fabio, Gabriele, Quentin
    • Meeting rooms Dinner Lunches Breaks - all were excellent.

​

4

Vera C. Rubin Observatory | Lyon DF meeting | 11-02-2025

Acronyms & Glossary