1 of 43

The de.NBI Cloud and LS AAI

​

​

​

​

ELIXIR AAI Engagement Call X

October 7th 2024

​

Nils Hoffmann, de.NBI Cloud Governance

2 of 43

de.NBI – German Network for Bioinformatics Infrastructure

​

​

​

​

de.NBI consortium

  • 24 partners
  • 8 service centers
  • national German node in ELIXIR
  • since 2022 sustained funding via German federal budget to FZJ within Helmholtz Society

de.NBI mission

  • Provision of comprehensive first-class bioinformatics tools & services to users in basic and applied life sciences research
  • Bioinformatics training in Germany and Europe through a wide range of workshops and courses
  • Providing cloud computing resources for academia in Germany
  • Transfer of expertise between academia and industry in our Industrial Forum
  • Cooperation of the German bioinformatics community with international bioinformatics networks

​

The consortium is organised into eight thematic Service Units.

​

3 of 43

de.NBI Cloud Overview

Perun

4 of 43

de.NBI Cloud

Application Process and Project Types

5 of 43

de.NBI Cloud - A Solution for (almost) Every Use Case

​

Infrastructure-, Platform- and Workflows- as-a-Service

  • High configurability, infrastructure virtualization
  • API access, e.g. for use with Terraform or Ansible
  • Any software of the cloud ecosystem�
  • High configurability, service and container orchestration
  • API access, e.g. for use with kubectl
  • Any containerized software / service, Helm charts�
  • Beginner-friendly, preconfigured Research Envs
  • LS AAI guarded interactive browser sessions
  • “One-click” solution for setting up your own cluster�
  • Fully established bioinformatics tools and workflows, maintained by the community
  • Point-and-click GUI for composition of bioinformatics workflows
  • Interactive tours and comprehensive training library��

�

​

6 of 43

Project application workflow

  • principal investigator of a German university or research institution applies for cloud resources by proposing a project and describing required resources through the de.NBI Portal, accepts usage policy
  • the project is reviewed by a scientific committee
  • after approval of the application the project is created in the de.NBI Cloud Portal
  • project resources are allocated at one of the cloud sites

​

​

Perun

7 of 43

de.NBI Cloud Federation Concept

  • Requirements
    • Authentication ?
    • Authorization ?
    • Project Coordination ?

Portal

8 of 43

de.NBI Cloud Federation Concept

  • Requirements
    • Authentication ? Home Institution Account & Life Science AAI
    • Authorization ? Perun API Access
    • Project Coordination ? Propagate to cloud site

Portal

API Access

Propagate

In production since 2017

9 of 43

AAI and Perun Ecosystem

  • Infrastructure that offloads the burden of authentication
  • Provides authorization capabilities
  • Gives you a unified login experience
  • Identity Management system (IDM)
    • Manage user accounts, identities
    • Categorize users
    • Manage relations
    • Represent services
    • Handle projects as groups, custom group attributes / resources
    • Allows to connect different accounts to one identity
  • Access Management system (AM)
    • SAML, OIDC, …
    • Proxy the authentication via EduGAIN, Google, ORCID, LinkedIn, …
    • Last resort: username and password (LS Hostel account)

​

10 of 43

de.NBI Cloud Portal Access and Authorization Workflow

de.NBI Cloud Portal

1

2

Authenticate via OIDC / SSO

Project�Management

1. Single Sign On with LS AAI

  • Easy access using the home institution credentials
  • Use of OpenID Connect
  • Administration features (e.g: usage policy, AUP)
  • Receive user attributes like
    • Name
    • User affiliations
    • LS ID
    • Call to Perun API in the name of the user

2. Further requests can be made in the name of the user by using the Access Token

11 of 43

de.NBI Cloud Portal Access and Authorization Workflow

de.NBI Cloud Portal

1

2

3

Authenticate via OIDC / SSO

Perun API

Access

Project�Management

3. OpenID Connect Access Token allows to use Perun as a database and set attributes (e.g SSH Keys)

  • Enables the implementation of different views

12 of 43

de.NBI Cloud Portal Access and Authorization Workflow

de.NBI Cloud Portal

1

2

3

4

Authenticate via OIDC / SSO

Perun API

Access

Project�Management

4.1 Perun propagates database information via ssh or https to cloud locations.

  • e.g project quotas, ssh keys, project members

4.2 Perun Keystone Adapter saves Perun information in OpenStack

  • User LS ID is set into Keystone
  • Perun propagates on every update

​

(e.g. Bielefeld, Giessen, … etc)

https://github.com/deNBI/perunKeystoneAdapter

13 of 43

de.NBI Cloud Portal Access and Authorization Workflow

de.NBI Cloud Portal

1

2

3

4

5

Authenticate via OIDC / SSO

Perun API

Access

Project�Management

Authenticate via

OIDC or SAML

4.1 Perun propagates database information via ssh or https to cloud locations.

  • e.g project quotas, ssh keys, project members

4.2 Perun Keystone Adapter saves Perun information in OpenStack

  • User LS ID is set into Keystone
  • Perun propagates on every update

​

(e.g. Bielefeld, Giessen, … etc)

https://github.com/deNBI/perunKeystoneAdapter

14 of 43

Project application workflow

  • principal investigator of a German university or research institution applies for cloud resources by proposing a project and describing required resources through the de.NBI Portal
  • the project is reviewed by a scientific committee
  • after approval of the application the project is created in the de.NBI Cloud Portal
  • project resources are allocated at one of the cloud sites
  • project members can be added by the PI

​

​

Perun

15 of 43

de.NBI Cloud User Roles in Perun

Access Committee Perun Virtual Organisation Manager

Administrator Perun Facility Manager

Principal Investigator Perun Project Admin

Project Member Perun Project Member

  • Decides which projects should �run in the de.NBI Cloud
  • Maintains the cloud location
  • Announces downtimes (via Cloud Portal)
  • Applies for a project
  • Manages project members
  • Responsible for all actions in the de.NBI Cloud
  • Access to cloud resources
  • May be promoted to have project admin permissions

16 of 43

de.NBI Cloud Entity Model in Perun

Access to the de.NBI Cloud Portal

Perun

de.NBI Cloud Virtual Organisation

17 of 43

de.NBI Cloud Entity Model in Perun

Access to the de.NBI Cloud Portal

Perun

de.NBI Cloud Virtual Organisation

de.NBI Cloud Facility Giessen

de.NBI Cloud Facility Bielefeld

18 of 43

de.NBI Cloud Entity Model in Perun

Perun

de.NBI Cloud Virtual Organisation

OpenStack

Resource

Access to the de.NBI Cloud Portal

Groups are propagated

Groups are not propagated

de.NBI Cloud Facility Giessen

de.NBI Cloud Facility Bielefeld

Openstack Resource

SimpleVM Resource

19 of 43

de.NBI Cloud Entity Model in Perun

Group 3

Access to the de.NBI Cloud Portal

Groups are propagated

Groups are not propagated

Perun

de.NBI Cloud Virtual Organisation

de.NBI Cloud Facility Giessen

de.NBI Cloud Facility Bielefeld

Openstack Resource

OpenStack

Resource

SimpleVM Resource

Group 1

RAM: 4 GB

Cores: 2

Group 2

RAM: 16 GB

Cores: 38

Group 4

RAM: 240 GB

Cores: 28

RAM: 2 GB

Cores: 1

20 of 43

de.NBI Cloud Entity Model in Perun

Access to the de.NBI Cloud Portal

Groups are propagated

Groups are not propagated

Access to a project hosted on a specific cloud site

Perun

de.NBI Cloud Virtual Organisation

de.NBI Cloud Facility Giessen

de.NBI Cloud Facility Bielefeld

Openstack Resource

OpenStack

Resource

SimpleVM Resource

Group 1

RAM: 4 GB

Cores: 2

Group 2

RAM: 16 GB

Cores: 38

Group 3

Group 4

RAM: 240 GB

Cores: 28

RAM: 2 GB

Cores: 1

21 of 43

de.NBI Cloud Entity Model in Perun

Perun

de.NBI Cloud Virtual Organisation

de.NBI Cloud Facility Giessen

de.NBI Cloud Facility Bielefeld

Openstack Resource

OpenStack

Resource

Access to the de.NBI Cloud Portal

SimpleVM Resource

Groups are propagated

Groups are not propagated

Perun API Access

Group 1

Access to a project hosted on a specific cloud site

de.NBI Cloud Portal

RAM: 4 GB

Cores: 2

Group 2

RAM: 16 GB

Cores: 38

Group 3

Group 4

RAM: 240 GB

Cores: 28

RAM: 2 GB

Cores: 1

22 of 43

de.NBI Cloud

SimpleVM - Customized self-service VMs or with predefined research environments

https://simplevm.denbi.de/wiki

23 of 43

SimpleVM - Simplified Access to Virtual Machines

​

​

Self Service VMs and Research Environments

  • Beginner-friendly, preconfigured & custom research environments
  • Integrated with LifeScience AAI for authentication and authorization
  • Easy VM, volume and user management
  • Browser and SSH access
  • based on OpenStack, Ansible and Packer, recipes at https://github.com/deNBI/resenvs

IDEs

Data Science Notebooks

Remote Desktop

Custom VREs

+

Community Curated

24 of 43

SimpleVM Federation & Deployment

  • Deployed across four de.NBI Cloud sites with central project and VM management dashboard
  • Deployed at Bielefeld University for Data Science Infrastructure
  • Pluggable AAI (e.g. local University IdP or LifeScience AAI) via Keycloak and local DB
  • Microservice Architecture
    • SimpleVM Portal
    • SimpleVM API Gateway & Reverse Proxy
    • SimpleVM OpenStack Client (per site)
  • VM access via Gateway and single IP
  • Better usage & allocation of GPU resources
  • Intrusion Prevention using SNORT

25 of 43

SimpleVM Portal and VM Access Workflow

SimpleVM Portal

1

2

3

6

Authenticate via OIDC / SSO

VM �Management

Authenticate via

OIDC / SSO

Task / Actions Queue

SimpleVM API

Access

SimpleVM

Reverse Proxy

4

VM Actions

7

Access VM via�Browser

Keycloak

SimpleVM OpenStack Client

5

Execute VM Tasks on OpenStack Project

26 of 43

SimpleVM Portal - Create Instance - Flavor Selection

https://simplevm.denbi.de/wiki

27 of 43

SimpleVM Portal - Resenvs and Volume Management

https://simplevm.denbi.de/wiki

28 of 43

SimpleVM Portal - VM Access Management

https://simplevm.denbi.de/wiki

29 of 43

SimpleVM Features: Workshop & Cluster Module

  • Workshop module to define VMs using base images, virtual research environments, or snapshots
  • Add participants with an existing Life Science AAI account to your project
  • Send VM access details to each participant individually
  • Launch VM instances for your participants
  • “One-click” solution for starting more complex cluster setups, e.g. for Nextflow
  • Connect multiple machines to one SimpleVM Cluster
  • Job scheduling mechanism (SLURM)
  • Easy automatic scale-up and scale-down
  • Currently in beta development, available on request

​

​

30 of 43

de.NBI Cloud

The Cloud in Numbers

31 of 43

Largest life science cloud in Germany and one of the leading European academic clouds in life sciences

​

Focus on reference data�data and storage via�different file storage protocols

redundant SANs

Focus on compute power �specialized hardware (GPU, FPGA)

high-memory instances

SSD accelerated ephemeral storage

Computing�Hardware

Storage�capacity

Storage

Capacity�38 PB

SSD / Flash�330 TB

~56,000

up to 4 TB

~100 PB

720 TB

~520 GPUs

de.NBI Cloud Federation

32 of 43

de.NBI Cloud Federation - Projects and Users

  • > 3,500 registered users
  • > 1,200 projects registered
    • OpenStack > 700
      • Workshop 37
    • SimpleVM > 500
      • Workshop 59
      • Cluster 29

numbers as of Sep. 12th 2024

> 1000’s of users of our services:

Perun

33 of 43

Training and Community Building

​

Training Announcements at https://www.denbi.de/de-nbi-events or @denbiOffice, Registration: https://events.denbi.de/

Beginners

  • Introduction to VM basics: Linux, command line
  • Using SimpleVM with Research Environments for applied bioinformatics use-cases

Intermediate

  • Bioinformatics workflows: BioConda, BioContainers, CWL, Nextflow, Snakemake

Advanced

  • IaaS / PaaS courses for: OpenStack, Kubernetes
  • Infrastructure automation: Ansible, Terraform
  • Grid computing in the cloud: BiBiGrid / SLURM

Courses:

47 Participants

60 Participants

75 Participants

67 Participants

ONLINE

2020

2021

2018

2019

2022

2023

20 Participants

Jülich

2024

34 of 43

Citations

This work was supported by the BMBF-funded de.NBI Cloud within the German Network for Bioinformatics Infrastructure (de.NBI) (031A537B, 031A533A, 031A538A, 031A533B, 031A535A, 031A537C, 031A534A, 031A532B).

> 1000 publications since 2017 reference de.NBI Cloud

numbers as of Sep. 12th 2024

35 of 43

Testimonials

36 of 43

de.NBI Cloud

​

National, European and International Collaboration

37 of 43

Compute-Related Involvement of ELIXIR-DE with ELIXIR

​

​

de.NBI Cloud Participation in ELIXIR Platforms

ELIXIR Compute Platform

  • Identity and access management
  • ELIXIR-DE Platform Coordination
  • Data Integration for Compute
  • ELIXIR Hybrid Cloud Eco-system
  • Deploying Reproducible Containers and Workflows across Cloud Environments

ELIXIR Tools Platform

  • Packaging, Containerisation and Deployment

EOSC Focus Group

  • EOSC Association Technical Interop TF

https://elixir-europe.org/

38 of 43

de.NBI Cloud Cooperation & Involvement

​

International Cooperation with / support of

​

  • ELIXIR-EXCELERATE
  • EOSC Life (Hybrid Cloud), EOSC Pilot, EOSC Entrust
  • H2020
  • GA4GH
  • EOSC HuB, EGI, GÉANT, Human Cell Atlas (upcoming)
  • ICGC Mirror, MII, Cancer Genome Collaboratory

​

​

Federated German compute and storage cloud ready for all scientific projects in life sciences

  • Central data backbone and catalyst for German and international initiatives with German coordination

International and European

​

Page 38

EOSC Marketplace

39 of 43

EOSC ENTRUST

​

​

  • Goal: Create a European network of TRUSTed research environments
  • Create Blueprints, Best Practice Guidelines and Software Components for sensitive data processing
  • 34 European Partners
  • Multiple European Infrastructures: �CESSDA, ECRIN, HDR UK, ELIXIR, EUDAT
  • de.NBI Cloud / ELIXIR-DE participates in TRE Provider Forum (WP10,11,12)

European Initiatives

40 of 43

de.NBI Cloud Cooperation & Involvement

National Initiatives

​

​

National Resarch Data Infrastructures (NFDI)

  • GHGA
  • NFDI4Microbiota
  • NFDI4Biodiversity
  • DataPLANT

NFDI Base Services Initative

  • Federated multi-cloud infrastructure�for all NFDIs across scientific domains

GAIA-X

  • HEALTH-X dataLOFT�H. Wagener, BIH/Charité

41 of 43

de.KCD - German Competence Center Cloud Technologies for Data Management and Processing

​

​

Collaboration with multiple RDM initiatives & NFDIs & other DCCs

​

Part of the Data competence centers for science German Recovery and Resilience Plan (DARP) and BMBF’s Research data action plan

Project Administration: VDI/VDE Innovation und Technik GmbH

​

https://datenkompetenz.cloud

42 of 43

de.NBI Cloud

Acknowledgements

43 of 43

Acknowledgements

EMBL

Peer Bork

Jan Korbel

Tobias Rausch

​

Heidelberg University

Rob Russel

Nick Kepper

Gießen University

Alexander Goesmann

Burkhard Linke

Marius Dieckmann

Frank Förster

Sebastian Beyvers

Bielefeld University

Jens Stoye

Volker Tölle�Stefan Albaum�Björn Fischer�

Freiburg University

Rolf Backofen

Björn Grüning

Jan Leendertse

Marc Herbstritt

Tübingen University

Oliver Kohlbacher

Jens Krüger

Johannes Werner

Mohamad Chehab

Fabian Wannenmacher

Amir Baleghi

Fabian Paz

DKFZ

Ivo Buchhalter

Martin Lang

Philip Kensche

Jordi Pujol

Bastian Beyer

Berlin Institute of Health

Roland Eils

Jürgen Eils

Martin Braun

Stefan Schneider

Valentin Schneider-Lunitz

Sven Twardziok

Harald Wagener

FZ Jülich @ Bielefeld

Peter Belmann�Christian Henke�Nils Hoffmann�Jan Krüger�Qiqi Mok �Grace Florensia�Viktor Rudko�Alexander Sczyrba�Alex Walender�David Weinholz�

https://cloud.denbi.de