Open Infrastructure in the�Cloud with JupyterHub

@choldgraf

Chris Holdgraf, UC Berkeley and Project Jupyter

bit.ly/jupyterhub-sdss-2019

you???

@choldgraf

A bit about me then...

Cognitive Neuroscience

Open Source

@choldgraf

A bit about me now...

Research and Open Source

Education and Open Source

Jupyter @ Berkeley

a community of people and an ecosystem of open tools and standards for interactive computing

create things that are language-agnostic and modular. Empower people to use other open tools.

For example: the Jupyter Notebook

The Jupyter Notebook is a stack of modular, open tools

You

Your awesome report

package ecosystem

Notebook document specification

Jupyter server protocol

Interactive Kernels

Notebook

interfaces

How does the ☁️ fit into this?

(some) data science should be taught to everyone

(no, really)

Here’s what this means at Berkeley...

How can Jupyter connect people with computation?

Build infrastructure tools that are workflow- and platform-agnostic. Give people control over resources, freedom to deploy what and where they wish.

(in the ☁️)

What is JupyterHub?

Host pre-configured data science environments�on shared infrastructure

jupyter.org/hub

My fancy machine in the cloud

myhub.org

myhub.org

myhub.org

environments

myhub.org

interfaces

environments

AUTHENTICATION

myhub.org

interfaces

environments

JupyterHub distributions

A pre-configured JupyterHub setup with sensible defaults and lots of documentation, fit for many use-cases

The Littlest JupyterHub�tljh.jupyter.org

JupyterHub on Kubernetes�z2jh.jupyter.org

☁️

​

💻

�

Scalable in both users and in resources�

Uses Docker for environment management

�Agnostic to the provider and�hardware configuration

Zero to JupyterHub for Kubernetes

z2jh.jupyter.org

�

�

The littlest JupyterHub

Deploy JupyterHub on a �single virtual machine�

Faster, lightweight setup�and administration�

More easily created�and destroyed

​

tljh.jupyter.org

�

�

JupyterHub in�the wild

✨

✨

education and training 🎓

datahub.berkeley.edu

�

inferentialthinking.com

​

​

nbgitpuller - one-click interactive content

jupyterhub.github.io/nbgitpuller

Chris Is Trying A Live Demo

Hopefully he doesn’t embarrass himself too badly.

data8.org/sp19/

Data 8 is...

jupyter book

github

gofer grader

jupyter notebook

nbgitpuller

scipy stack

large-scale�science 🌎☁️

pangeo.io

• A JupyterHub + BinderHub managing an open stack of tools for geospatial analysis
• Utilize and improve pre-existing tools, rather than build new ones. Push improvements upstream.
• Add value with customization and configuration
• Provide access to high-performance hardware

The Pangeo pattern

pangeo.io

pangeo.io

Pangeo is...

zarr

jupyter widgets

dask

jupyter

lab

xarray

scipy stack

big data

Open and Interactive collaboration 🤝

openhumans.org

https://zenodo.org/record/2697185#.XNV2a_DJe5Y

Chris Is Trying A Live Demo

Hopefully he doesn’t embarrass himself too badly.

exploratory.openhumans.org

OpenHumans is...

open data

R stack

exploratory

jupyter

notebook

community

scipy stack

• JupyterHub connects users with interactive environments on shared infrastructure using open tools
• JupyterHub distributions are opinionated deployments of a JupyterHub for a specific scale or purpose.
• The Littlest JupyterHub - Deploy on a single VM 💻
• Zero to JupyterHub for Kubernetes - Deploy in the ☁️
• JupyterHub has been used for
• Large-scale education (data8.org)
• High-performance 🌎 analysis in the ☁️(pangeo.io)
• Collaborative community analytics 🤝 (openhumans.org)

In summary

Get involved with Jupyter

@choldgraf

jupyterhub-team-compass.readthedocs.io�discourse.jupyter.org

• All of these projects are open source, run by open communities
• Jupyter is a place where *anybody* can participate
• If you’d like to get involved: