1 of 29

Preparing users for the next generation of data intensive astronomy

Alex Clarke - Operations Data Scientist

alex.clarke@skao.int

Ugur Yilmaz - DevOps Engineer

ugur.yilmaz@skao.int

Collaborations Workshop - 3 May 2023

Slide /

2 of 29

Outline

  • How to do Astronomy?
  • Square Kilometer Array Observatory (SKAO)
  • Processing Data at a Scale
  • SAFe Way to Develop and Collaborate Socially Distributed Software
  • Open and Sustainable Science in the Era of Big Data

Q&A: Talking preferred, alternative: https://app.sli.do/event/74yBfmatPgA59tfQKiu3AS

at slido.com with the code #2270043

2

Slide /

3 of 29

Slido

  • joining at slido.com with the code
  • #2270043

3

Slide /

4 of 29

The traditional way to do astronomy

4

Edwin Hubble (Image credit: NASA)

Slide /

5 of 29

Observation and Data Lifecycle (example based on Atacama Large Millimeter/submillimeter Array)

5

Observation Planning

The first step is to plan the observation. This involves choosing the target, the observing mode, and the observing time.

Observation execution

Once the observation is planned, it is executed by the ALMA staff. The data is received in real time and stored on disk.

Data processing

The data is then processed by the ALMA staff. This involves calibration and imaging

Data access

The data is then made available to the public through the ALMA data archive (using CASA)

Data archiving

The processed data is then archived in the ALMA archive. A distributed data store around the world

05

01

02

03

04

Several weeks/ months

3-10 EB, 1EB = 1TB

Data rate is around 30Gb/s or Streaming 6 HD movies every second

Slide /

6 of 29

Two telescopes, one observatory

6

SKA Low

131072 antennas in the Western Australian desert

SKA mid

197 dishes in South Africa

Slide /

7 of 29

Data Processing in SKA

  • SKA ~ 100 TB

  • SKA ~ 700 PB x12 times LHC/CERN

  • SKA ~ 250 TFLOPS: This is top 3 in the current fastest supercomputers (ref. 2022)

7

https://www.bing.com/images/create/a-diagram-of-the-machine-learning-approach-for-est/6451ad7d4e204e69806c6670514360ca?id=uwRjEMFgPsFIxI3Nv8BqLQ%3d%3d&view=detailv2&idpp=genimg&idpclose=1&FORM=SYDBIC

Slide /

8 of 29

How does SKA data get to the users?

8

SKA MID

SKA LOW

8 Tb/s

9 Tb/s

100 Gb/s

SKA Regional Centres

100 Gb/s

20 Tb/s

2 Pb/s

Central Signal Processor

Central Signal Processor

Science data processor

Science data processor

Slide /

9 of 29

SKA Regional Centres (SRCs)

9

Interoperability

Heterogeneous SKA data from different SRCs and other observatories

Support to Science Community

Support community on SKA data use, SRC services use, Training, Project Impact Dissemination

Visualization

Advanced visualizers for SKA data and data from other observatories

Science Enabling Applications

Analysis Tools, Notebooks,

Workflows execution

Machine Learning, etc

Distributed Data Processing

Computing capabilities provided by the SRCNet to allow data processing

Data Discovery

Discovery of SKA data from the SRCNet, local or remote, transparently to the user

Data Management

Dissemination of Data to SRCs and Distributed Data Storage

Slide /

10 of 29

SKA data model: Discovery and re-use

All SKA Data Products will (in time) become public

Following from the example given by Hubble and SDSS, this will be the biggest generator of science

10

Slide /

11 of 29

SKA Software Vision and Strategic Objectives

It’s hard to do this all from ground up!

We are a global collaboration committed to building and delivering a world-class software and computing ecosystem, enabling the SKAO community to achieve transformational science.

Our culture of innovation, shared governance, and respect for diversity creates mutual value for all.

11

Slide /

12 of 29

ARTs

12

Slide /

Slide /

13 of 29

Software Quality & Delivery: SKAO Context

  • We are a highly distributed and diverse project - different:
    • Time zones
    • Cultures
    • Developer experiences
    • Subject domains
    • Delivery timeframes
    • Large varied codebase
    • Small central team enabling this

“How do we make sense of this so that we can deliver to a consistent level of quality, and efficiently use resources available, on time, and on budget”

13

28 Agile teams

> 200 people

Slide /

14 of 29

SAFe way to develop and collaborate

  • Right now:
    • All parts of Telescope Software, data processing ~ Stream Aligned Teams
    • Infrastructure, Tooling ~ Enabling & Platform
  • Analogy for future:
    • Scientists, Institutions ~ Stream Aligned
    • Telescope ~ Enabling & Platform

14

Slide /

15 of 29

Look at what Qualities needed? -

they influence approach

  • Portability/Scalability - Containerisation, and consistent processes
  • Security - Controlled delivery of scanned and signed artefacts (logistics & supply chain)
  • Reliability - reproducible, automated testing
  • Maintainability/Extensibility - open source, and open and consistent standards for everything
  • Availability - open access to code and processes
  • Sustainability - open access tools, and most common industry standards, with the lowest practical common denominator

15

Slide /

16 of 29

How can the SKAO promote open and reproducible science

  • We want to make it easy for users to cite the workflows used to obtain their data

  • We are running data challenges that give users example of the challenges they will face (large data volumes that require more efficient processing techniques)

  • In these data challenges we are championing teams that produce results in a reproducible and sustainable way

16

Slide /

17 of 29

Data challenge reproducibility awards

  • We came up with some criteria to rank teams in terms of how reproducible their workflows were.

  • By no means a complete list, but we wanted to encourage them to think about the things they could do to improve the reproducibility and sustainability of their work.

17

Reproducibility Awards Gold/Silver/Bronze scheme: https://sdc2.astronomers.skatelescope.org/sdc2-challenge/reproducibility-awards

Slide /

18 of 29

Maximising science output at regional centres

Users will bring their code to the data at Science Regional Centres

We want reproducibility and sustainability build into these tools at the Regional Centres

    • Science enabling applications at the ready
    • Data discovery easy and intuitive
    • Supporting the scientific community
    • Data management and distributed processing

These things make it easier for users to take care of their workflows, and ultimately be more productive

As an observatory we have a responsibility to our users to make their lives as easy as possible, but understanding how our approach will influence users is not trivial.

18

Slide /

19 of 29

Thank you!

Q & A

We recognise and acknowledge the Indigenous peoples and cultures that have traditionally lived on the lands on which our facilities are located.

Slide /

20 of 29

Slido

  • joining at slido.com with the code
  • #2270043

20

Slide /

21 of 29

User notes 2

Mini-workshop or demo session abstract

The Square Kilometre Array Observatory (SKAO) is a global radio telescope that aims to transform our understanding of the Universe, and deliver benefits to society through global collaboration and innovation. Thousands of radio antennas and dishes will be spread across Australia and South Africa, allowing astronomers to observe the universe in unprecedented detail from 2028.

The SKAO is an Intergovernmental Organisation currently involving 16 countries with the headquarters based at Jodrell Bank in Cheshire. The distributed nature of this large project presents a challenge, with over 25 teams deploying hardware and software spread amongst 100 organisations around the world. We operate under a Scaled Agile Framework (SAFe) approach, which promotes workflow patterns that enable a large number of teams to collaborate and deliver work towards a large common goal.

The SKAO will deliver data at an unprecedented volume and rate, where it will mostly not be feasible for users to download data to their personal or even institutional computing facilities. In anticipation of this we are working on tools that will enable scientists to explore and process data remotely at dedicated facilities no matter where they live around the world. A key driver for science output is ensuring that data products and software pipelines are easily discoverable, explorable, and understandable. Achieving this depends on us deploying user friendly, flexible and efficient services that allow scientists to maximise their output from the data, and so a key challenge for us is ensuring that our software and operations teams are working together effectively towards these goals.

In this talk we will discuss how our different teams work together, and how we are implementing software pipelines and data discovery tools that will maximise scientific output. The SKAO is committed to ensuring the long-term sustainability of our software, data and systems, and we will discuss how we are encouraging users to align with our values, making science inclusive and reproducible. We want these themes built directly into the data management systems, software pipelines, and compute resources so it is easier for users to take care of their workflows, and ultimately lead to being more confident and productive. As an observatory we have a responsibility to our users to make their lives as easy as possible, but understanding how our approach will influence users is not trivial. At the end of our talk we are keen to gather some ideas from you on how organisations can make users' lives easier and encourage healthy working behaviours.

Mini-workshop or demo session audience

People don't need any prior knowledge about astronomy, science or software. This talk will present how we are building the observatory, how users will be affected by a shift in scientific workflows, and how we can make it easier for them to work in a sustainable and reproducible way.

Mini-workshop or demo session theme(s)

Technical development (e.g. relating to software sustainability, software products and digital tools, infrastructure and documentation, software development skills, and training)

Mini-workshop or demo session description

Alex and Ugur will give a joint presentation as discussed in the abstract, and we will be leaving time for a brief discussion as we would love to hear the thoughts of others on these themes and bring that feedback back to SKAO. We hope that showing people how the SKAO will operate will inspire people on how to design systems for big data workflows. We want to create a research ecosystem system which takes care of its users, so that it is easier for them to explore and analyse data in a reproducible and sustainable way. One of the themes of our talk will be how data and software tools influence peoples workflows - if it's hard to install and clunky to use, users are less likely to use it and even less likely to bother making a piece of work reproducible, ultimately making their work harder in the long run. We want to highlight how an organisation like SKAO can make a big impact on encouraging healthy working behaviour, creating more efficient and happier users. We don't see any barriers to remote participants engaging with our session. We might use a shared goolge doc, or Sli.do to collect questions, thoughts, and feedback throughout the session.

We saw potential for this to be a 25 minute talk followed by a 20+ minute discussion around ways organisations influence their users workflows and how they can make life easier for their users (emphasis on organisations that serve data/software/compute resources), as we are keen on gathering ideas around this. But we have currently decided on cutting this down to a 20+10 session for efficiency. However, if you think there is scope/desire for this to be 45-60 minute session we would be happy to plan for that, but we thought we would initially be conservative and not use up time that could be used for other sessions. Happy to discuss more if there is interest in a longer session from us!

Any special requirements or other information you would like to include about your proposed session?

We are interested in using Sli.do. We will check it out and see if we can use it to get feedback and thoughts throughout our talk and discussion.

Scheduled

CW23 Day 2: Wednesday, 3 May 2023 from 10:00 - 10:30 BST (09:00 - 09:30 UTC; 30 minutes)

21

Slide /

22 of 29

Requirements

The purpose of

the requirements specifications

is to

capture stakeholders needs.

Requirement Specifications

are managed throughout the life of the project

to create a consensus

of what should be delivered as a system.

22

Slide /

23 of 29

Data Flow

23

Slide /

24 of 29

Proposal Observation Flow

OSO is the suite of software tools that support the entire science process from proposal submission to data-product delivery.

24

Slide /

25 of 29

Observatory Science Operations Tools

Provide a science-oriented user interface for the following functions:

  • Science proposals submitted from the community for peer assessment.
  • Successful proposals become projects.
  • Projects get divided into:
    • Scheduling Blocks (SBs) for observing, and
    • Processing Blocks (PBs) for data processing.
  • SBs planned for execution over periods of days/weeks/months, assigned to subarrays.
  • Subarray is a subset of telescope’s resources designed to achieve specific science objectives.
  • SBs are executed on the assigned subarrays, producing raw data.
  • Data processing forms calibrated data-cubes (and other results) from the raw data.

OSO tools provide easy to use science (and operations) oriented interfaces for using these functions, with aim to hide, as much as possible, the complexity of the telescopes.

25

Slide /

26 of 29

Observatory Science and Operations Tools (2 of 2)

  • Proposal Submission Tool allows community investigators to submit proposals to use the SKA telescopes.
  • Proposal Management Tool allows staff and reviewers to triage, assess and rank proposals.
  • Observation Design Tool creates Scheduling Blocks (SBs): the unit of observing for a science project.
  • Observation Planning Tool is used to create plans for prioritised observing across all projects for any period (days, weeks or months).
  • Observation Scheduling Tool selects the “best” SB to observe next from the prioritised plan.
  • Observation Execution Tool executes the SB, translates science-oriented configurations to top level telescope-oriented configurations and feeds that to the TMC.
  • Project Tracking Tool tracks the progress of science projects and their SBs and PBs through the system, for staff and community investigators.
  • Shift Log Tool records operator shift information for later interrogation.
  • Observation Data Archive stores all information about projects, SBs and progress.

26

Slide /

27 of 29

Observation Operations

27

Slide /

28 of 29

Functional Observation

28

Slide /

29 of 29

Standardisation … for everything

Throughout the Software Development Life Cycle:

    • Common project structure per framework/artefact type
    • Common code standards
    • Common documentation standards
    • Consistent tooling - reliable/predictable behaviour
    • Reference templates for all tasks:
      • Linting, building, testing, publishing etc.
    • Reference templates for all CI/CD pipelines
    • Reference implementation of Dev Env
    • Centrally managed artefact delivery - gated
    • Automated checking for consistency
    • Target platform API

29

Slide /