1 of 18

Science Platforms and the IVOA

The SKA Regional Centres Network Use Case

​

​

Jesús Salgado

SKA Regional Centres Network Architect

​

And the SRCNet members

​

​

​

​

​

​

​

​

​

​

​

2 of 18

3 of 18

4 of 18

5 of 18

  • ~ 600 PB/year of Scientific Data
  • 16 countries involved
  • Up to 100 FTEs during development phase

​

  • Collaboration agreements with CERN, GEANT, CTAO
  • Collaborations with CNRS, Vera Rubin and others

​

5

SKA Regional Centres (SRC) Network in Numbers

6 of 18

SKA Regional Centre Capabilities Blueprint

Interoperability

Heterogeneous SKA data from different SRCs and other observatories

Visualization

Advanced visualizers for SKA data and data from other observatories

Science Enabling Applications

Analysis Tools, Notebooks,

Workflows execution

Machine Learning, etc

Distributed Data Processing

Computing capabilities provided by the SRCNet to allow data processing

Data Discovery

​

Discovery of SKA data from the SRCNet, local or remote, transparently to the user

Data Management

Dissemination of Data to SRCs and Distributed Data Storage

Support to Science Community

Support community on SKA data use, SRC services use, Training, Project Impact Dissemination

7 of 18

SRC Network global capabilities

​

8 of 18

SRCNet Principles

Use of Standards

Build SKA science archive around FAIR and IVOA standards

​

Data Management

Avoid unnecessary

duplication and transfers

Roughly 5-10 million dollars per year in new data, for one copy

​

Collaboration and Reproducibility

Science Reproducibility at the level of workflows is essential as data should not be downloaded

1

3

2

By:

.com

9 of 18

The IVOA Context

Slide /

10 of 18

InterOperability and

Federation

Federated Authentication

and Distributed Processing

Platforms interconnected

Data Lakes

Science Enabling Applications

Astropy and Astroquery

Notebooks

Users environments

Discovery And Access Services

Cone Search

SSAP, SIAP

TAP

​

Democratic Science and AI

Harmonisation

Transparent Data Access

Combined Computing Resources

11 of 18

Science platforms

Authentication

Interactive Analysis

Big Terminal

Batch

Authorisation

Metadata

Computing

Data

Orchestrators

SW Repository

12 of 18

Science Platforms Interoperability

​

Authentication

Interactive Analysis

Big Terminal

Batch

Authorisation

Metadata

Computing

Data

Authentication

Interactive Analysis

Big Terminal

Batch

Authorisation

Metadata

Computing

Data

Federated Authentication

API

API

Federated Data Lake

Service

Service

Orchestrators

Orchestrators

SW Repository

SW Repository

Interoperable/federated SW Repositories

13 of 18

Some possible data mesh services

Data Type

Operation

Input

Output

Any Type

Get Stream

ID

Input Stream

Data Cube

Cut-out

ra, dec, size, resolution

Data Cube

Data Cube

Get Spectra

ra, dec, size

Spectrum

Data Cube

Get Time Series

ra, dec, size

Time Series

Data Cube

Get Slice

wavelength

Image

Image

Change Resolution

ra, dec, size, resolution

Image (FITS to HiPS)

Image

Source Extraction

ID, algorithm params

Source Catalogue

Spectrum native

Convert to VO

ID

Spectrum VO

Source Catalogue

Similar Source

Source ID

Source Catalogue

14 of 18

The problem of the formats

Spectrum

Mission 1

Looking for

Spectrum VO

(greatest common divisor)

Spectrum

Format 1

Spectrum

Format 2

Final Spectrum VO Format

(lowest common denominator)

Translation

Server

Translation

Server

Native

VO

15 of 18

Execution Planner

Execution

Planner

Science

Platform I

Science

Platform II

Science

Platform III

Science

Platform IV

16 of 18

Solving the Topology

Execution

Planner

Information

System

17 of 18

Summary

  • IVOA provides discovery and access protocols for most of the astronomical data
    • Standards, Integration with scripting languages, Easy publication and collaboration environments
  • Many astronomical use cases are enabled due to IVOA standards

​

  • Possible “interoperable science platform” new phase with:
    • Federated Authentication Protocols
    • Improved data access
    • Remote operations
    • (Simplified) federated execution
      • Execution planner
      • Topologies
      • Software characterisation

PROMOTE

NEW?

EXTEND

NEW API

COMPLETE

STANDARD

18 of 18

Thanks for your attention

Slide /