1 of 34

Orchestra: �A cloud platform for hands-on computational workshop environments��

Sean Davis, MD, PhD

December 2, 2021

https://seandavi.github.io

@seandavis12

https://bit.ly/31qGUN8

2 of 34

Driving Use Case

  • Delivery of hands-on workshop materials in a live coding style
  • Reproducible, reusable workshop materials
  • Portable, ready-to-use environment and deployment
  • Scalable infrastructure, ideally no local compute needs and little network bandwidth

3 of 34

History of Bioconductor Workshop Infrastructure

Standalone Workshop Packages

    • Until 2015

Amazon Machine Images

    • 2016-2019

Orchestra and Docker

    • 2020-Present

4 of 34

Building blocks and flow

    • Github Repo
    • Testing

Materials

    • Build
    • Host

Container

    • Kubernetes
    • Docker

Infrastructure

5 of 34

Workshop materials

6 of 34

Workflow packages

  • Standard R packages
  • Installable, testable
  • Can leverage standard R and Bioconductor tooling for building and distribution
  • Benefits from social coding
  • Best-of-breed tooling for publication, including Rmarkdown and pkgdown

7 of 34

Standardizing and Formalizing

  • Guarantee basic level of functionality for workshops
  • Interactive and standalone use
  • Allow for community contribution and improvement
  • Incorporate social contract for pedagogical best practices
  • Easy reuse or repurposing of materials and cloud infrastructure.

8 of 34

Template workshop materials

  • GitHub based
  • Uses GitHub Actions to automate
    • Building
    • Testing
    • Container creation with Docker
    • Website creation pkgdown

​

https://github.com/seandavi/BuildABiocWorkshop

9 of 34

Template workshop materials

  • GitHub based
  • Uses GitHub Actions to automate
    • Building
    • Testing
    • Container creation with Docker
    • Website creation pkgdown

​

https://seandavi.github.io/BuildABiocWorkshop/

10 of 34

Templates for workshop materials

11 of 34

Model pedagogical practices

  • Pre-requisites
  • Learning goals and objectives
  • Clear description of covered material
  • Approximate schedule and time expectations
  • How to participate (if hands-on, state as much)

12 of 34

Containerization

13 of 34

A basic Dockerfile

https://github.com/seandavi/BuildABiocWorkshop/blob/master/Dockerfile

14 of 34

And then build

Basic docker build

Github actions

Cloud build processors (AWS, GCP, Azure)

Other CI services

15 of 34

Infrastructure

16 of 34

https://platform9.com/wp-content/uploads/2019/05/Virtualization-vs-containerization-kubernetes-in-enterprise-1.jpg

17 of 34

https://csharpcorner-mindcrackerinc.netdna-ssl.com/article/getting-started-with-kubernetes-part2/Images/1.png

18 of 34

https://cloudstackgroup.com/wp-content/uploads/2021/04/Untitled-design-5-1320x744.png

19 of 34

Kubernetes vs Docker

# Run a container

docker run -d --restart=always -e DOMAIN=cluster

--name nginx-app -p 80:80 nginx

​

kubectl create deployment --image=nginx nginx-app

​

# List running pods/containers

docker ps -a

​

kubectl get pods

# Attach to running container

docker attach 55c103fa1296

​

kubectl attach -it nginx-app-5jyvm

​

# logs

docker logs -f a9e

​

kubectl logs -f nginx-app-zibvs

20 of 34

Single Orchestra Workshop Architecture

21 of 34

Deployment

22 of 34

Single Orchestra Workshop Architecture

23 of 34

Service

24 of 34

Single Orchestra Workshop Architecture

25 of 34

Ingress

26 of 34

Full Orchestra Architecture

27 of 34

Orchestra platform Components

28 of 34

Workshop Lifecycle

29 of 34

Workshop�Monitoring

30 of 34

Running cluster

31 of 34

Walkthrough

32 of 34

Choose a workshop

33 of 34

Orchestra is Just CODE

34 of 34

Proceed to Code Review