1 of 8

ESAP architecture brainstorm

How it will fit in with the surrounding technologies

2 of 8

Useful links

Deliverable 5.2 (Zheng et al.)
ESAP architecture document (Nico)
Notes on the Jupyter (Hub and Notebook) API exploration (Stelios, James for IDA WG)
User interface design document (possibly a good starting point for refining the scope of the MVP): (Stelios)
WP5 ESAP Focus Months
Stelios overview slides here

3 of 8

non-interactive

ESAP

WP3 OSSR workflows/containers repo

IAM

JupyterHub service

Data provenance

Rucio jupyterlab extension

Usable service by all vs deploy-your-own

DIRAC service

ESAP gateway

Data discovery

Compute discovery

Workflow discovery

IVOA

Rucio

notebooks

containers

cvmfs

User

Sqlite DB

(holding user cart for eg)

Batch cluster

X509 auth

IAM auth

Bare-metal VM

IAM auth

interactive

4 of 8

What might a Jupyter-based MVP use case look like...

Caveats/limitations:

Type of jupyterhub server will be fixed

To start with, Jupyterhub server may have to be one that has rucio tooling baked in
The EVN Jupyterlab environment already spawns a modified kernel - we need CASA (and other tools) available for our notebooks

Compute facilities not polled dynamically

User flow	Corresponding API requirements/other requirements �(green exists, red to do, blue workarounds)
User logs in to ESAP via IAM	IAM integration with ESAP [done]
User selects IVOA data, and a Rucio DID	IVOA, Rucio search functionality [done]
User selects a Jupyterhub compute facility	Hard coded Jupyterhub deployment(s) to point to. These deployments may have to be task specific, if unable to select environments to spawn [workaround].
User selects a notebook via a WP3 OSSR repo	Query and select a notebook from a hard-coded list of options [workaround] Query OSSR repo (where/what would this be?) [to do]
User sent to chosen Jupyterhub service with data/metadata about the data they have chosen (IVOA data, and/or rucio DID), and either the chosen notebook itself or a query to get the notebook.	IAM login persisted (token saved in browser? If token is client-specific, may have to log in again) [workaround] Create user server, and send payload to the server via Jupyter notebook API [todo]
User downloads the data (and notebook if needed) and runs workflow	User environment would need a handy script to ready the environment for the user [todo] to recognise rucio to allow a file to be downloaded via DID rucio credentials [todo] Have any compute specific apt-based packages preinstalled that the notebook would need (conda packages can be installed on the fly) [todo]

5 of 8

What might an Jupyter-based, MVP bring-compute-to-data, use case look like...

User flow	Corresponding API requirements/other requirements �(green exists, red to do, blue workarounds)
User logs in to ESAP via IAM	IAM integration with ESAP [done]
User selects IVOA data, and a Rucio DID	IVOA, Rucio search functionality [done]
User selects a notebook via a WP3 OSSR repo	Query OSSR repo (where/what would this be?) [to do]
Jupyterhub is chosen that is close to the data selected	ESAP queries Rucio API for where replicas of DID exist. [todo] Options: (MVP) Static mapping between RSEs and Jupyterhub deployments [workaround] (Ideal) Deploy externally accessible Jupyterhub instance at the data location (On-demand compute provision model, in this case a medium-lived instance, to allow WIP persistence)
User sent to chosen Jupyterhub service with metadata about the data they have chosen (archive specific query that has been built, and/or rucio DID), and either the chosen notebook itself or a query to get the notebook.	IAM login persisted (token saved in browser?) [workaround] Create user server, and send payload (data, metadata, notebook) to the server via Jupyter notebook API [todo]
User downloads the data (and notebook if needed) and runs workflow	User environment would need a handy script to ready the environment for the user [todo] to recognise rucio to allow a file to be downloaded via DID rucio credentials [todo] Have any compute specific apt-based packages preinstalled that the notebook would need (conda packages can be installed on the fly) [todo]

6 of 8

Bringing compute to the data related thoughts

Very simply...

Build a throwaway very simple JHub deployment just so we have two to choose from (min: just need a reasonably sized VM to bring up a smaller JHub service)
ESAP has rucio service account to query RSEs and their distances
We have a RSEs to compute platform mapping that is either static or takes into account distances as defined on RSEs.

Slightly more advanced...

A setup where a JupyterHub service is deployed at the compute site near data on demand, in response to ESAP user request

Maybe a pre-running Jupyterhub service isn’t what we want in that case.
Would littlest jhub work better? https://tljh.jupyter.org/en/latest/topic/whentouse.html#topic-whentouse
If multi-purpose JHub services aren’t able to do what we require we may be tied to more modular instances anyway.

Would be good to know about LSST JHub architecture. Was it monolithic or did compute/data resources span multiple sites?
Does Notebook Aspect stage data in environment, and if so how?

7 of 8

Other thoughts/ideas/questions

What happens after a user runs their workflow? �Uploading data to the DL from a jupyter notebook has been complicated in the past (need for x509 credentials, gfal issues with ubuntu base image)
BinderHub �Perhaps? https://binderhub.readthedocs.io/en/latest/zero-to-binderhub/setup-binderhub.html#connect-binderhub-and-jupyterhub

8 of 8

Actions (19/04/21)

Hard coded compute facilities and their characteristics in the ESAP DB