Kubernetes for Grad Students

Romil Bhardwaj

04/29/2022 - PS2

If you’re following along, make sure to install the prerequisites @ https://github.com/romilbhardwaj/kube-tutorial

The grad student research toolkit

• You are a grad student
• You find a problem
• You come up with a solution
• You implement your solution using some tools from your toolkit

This tutorial wants you to consider adding Kubernetes and Docker to your toolkit

Tutorial outline

• A simple webapp
• Containerizing the webapp with Docker
• Running the container across nodes in a cluster with Kubernetes
• Running Ray on Kubernetes and autoscaling Ray
• Autoscaling your Kubernetes cluster with a mix of spot and on-demand instances

A simple web app

• Let’s create a simple webserver in python that returns the hostname for every GET request
• Dependencies are installed in the local environment

Romil’s Laptop

Web App

Packages

3.8

Challenge 1 - Dependency Isolation

• What if I want to run legacy code that requires python 2.7 on my laptop?
• My current environment supports only py 3.8 for my web app

Romil’s Laptop

Web App

Packages

3.8

Py 2.7 App

Challenge 2 - Portability

• What if I want to run my code on some other machine?
• Do I install dependencies again? Version mismatches?

Romil’s Laptop

Web App

Packages

3.8

Ion’s Laptop

Web App

Packages

​

????

Solution - wrap dependencies and code into containers!

• Package code and dependencies into a single portable unit
• Provides portability and isolation

Romil’s Laptop

Container

Web App

Packages

3.8

More formally…

A container is a standard unit of software that packages up code and all its dependencies so the application runs quickly and reliably from one computing environment to another.

Containers

Virtual Machines

https://www.sdxcentral.com/cloud/containers/definitions/containers-vs-vms/

Containers with Docker

• Docker is a container runtime that allows you to create and run containers
• A container is run from a container image, which is the isolated filesystem that contains necessary code and dependencies
• Let’s look at some code…
• Creating an image
• Running a container
• Debugging your container

Publishing your container images

• This container was built and run locally on your laptop
• How do you replicate the same environment on other machines without having to rebuild?
• Solution - push your built image to a remote container registry!
• Most popular - DockerHub, can setup your own private registry on AWS, GCP or Azure.
• Let’s push our image…

Romil’s Laptop

Container

Web App

Packages

3.8

Container Registry

Distributed apps with docker?

• What if I want to run multiple containers on a cluster?
• Port handling on even one node is hard

​

Node

Container 1

Container 2

Port 8000

Port 8000

Port 8000

Distributed apps with docker?

• What if I want to run multiple containers on a cluster?
• Port handling on even one node is hard
• Managing docker daemons across nodes?

Node 2

Container 2

Docker Daemon

Node 1

Container 1

Docker Daemon

❓

❓

Distributed apps with docker?

• What if I want to run multiple containers on a cluster?
• Port handling on even one node is hard
• Managing docker daemons across nodes?
• Service discovery?
• Failure recovery?
• Scheduling?
• Shared storage?
• Secrets management?
• Autoscaling?

Managing containers in distributed environments is hard!

Enter Kubernetes

https://kubernetes.io/

Kubernetes Features (the important ones for grad students)

Kubernetes (and container) features that everyone takes for granted

• Portability - Move your code to any cluster, any cloud
• Resource isolation - CPU/Memory limits (if set) are real limits
• Environment isolation - install whatever in your sandbox, not bound to a fixed environment
• Generality - not just python code. It’s a container to yourself.
• Multitenancy - Isolation enables multitenancy, create quotas etc.
• Huge community - everything you would need is already out there

The history behind Kubernetes

Who’s using k8s?

(according to cncf.io)

https://www.cncf.io/reports/cncf-annual-survey-2021/

Who’s really using kubernetes?

The kubernetes fractal of complexity

• Kubernetes, like other big systems, gets more complex as you dive deeper

Don’t stare at the k8s abyss for too long!

• However, K8s abstractions are great
• You can accomplish majority of your grad student goals by learning only high level Kubernetes abstractions

Kubernetes 101 - kind

• KinD - Kubernetes in Docker
• What the ….
• Great tool to run Kubernetes on your laptop
• Creates multiple docker containers and makes them act as Kubernetes nodes
• Stitches a Kubernetes cluster between containers and gives you a sandbox to experiment with k8s
• kind create cluster
• Creates a cluster while you sit back and relax

Tutorial - Set up kind

Kubernetes 101 - Pods

• Pods are the smallest deployable units of computing that you can create and manage in Kubernetes
• A group of one or more containers with shared storage and network resources, and a specification for how to run the containers.

apiVersion: v1

kind: Pod

metadata:

name: nginx

spec:

containers:

- name: nginx

image: nginx:1.14.2

ports:

- containerPort: 80

Kubernetes 101 - Deployment

• A Deployment provides declarative updates for Pods - think of it as a pod manager
• You specify the number of pod replicas you want, the deployment will ensure that state is achieved (in the face of node or pod failures)
• You almost never create pods by themselves - always use a deployment to create pods

# deployment.spec

spec:

replicas: 3

selector:

matchLabels:

app: MyApp

template:

metadata:

labels:

app: MyApp

spec:

containers:

- name: nginx

image: nginx:1.14.2

ports:

- containerPort: 80

Deployment

Pod 1

{app: MyApp}

Pod 2

{app: MyApp}

Kubernetes 101 - kubectl cheatsheet

• kubectl is a command line tool to interact with your kubernetes cluster
• kubectl apply -f <myobject.yaml>
• “Applies” the object YAML to the kubernetes cluster (create object if not exist)
• kubectl get <pods/deployments/...>
• To get the list of kubernetes objects
• kubectl describe <pods/deployments/...> <obj_name>
• To get detailed information about an object
• kubectl exec -it <pod_name> /bin/bash
• Get a shell into a pod
• kubectl logs -f <pod_name>
• Stream STDOUT/STDERR logs of the pod
• kubectl proxy
• Starts a proxy server for you to peek inside your k8s cluster’s network

Tutorial - Writing our first Deployment YAMLs

Tutorial - Deploying our first Deployment YAMLs

Tutorial - SSHing and getting logs

Tutorial - accessing through Pod IP

Kubernetes 101 - Services

• Service is an abstract way to expose an application running on a set of Pods as a network service (IP + Port)
• Why do we have services? Pods can be destroyed and created anytime - how do you keep a track of their IPs?
• Each service gets an IP which can map to many pods

apiVersion: v1

kind: Service

metadata:

name: my-service

spec:

selector:

app: MyApp

ports:

- protocol: TCP

port: 80

targetPort: 9376

Tutorial - Writing and deploying a service object

Tutorial - Accessing the app via service IP and dns name

Tutorial - scaling deployments

Tutorial - deleting pods

Tutorial - Dashboard

Tutorial - Ray on Kubernetes

Tutorial - EKS Spot Autoscaling

Unsolicited advice

• Never try to run/maintain a kubernetes cluster on baremetal
• Always use Rancher/EKS/GKE/AKS
• Don’t reinvent the wheel - K8s features you need are likely already out there
• Invest time in writing container images for your projects - saves time and headaches later

Law of the Instrument: “If all you have is a hammer, then everything looks like a nail”

Use the right tools for the job

https://k8s.af/

Thanks

Congratulations! Instead of imperatively walking on a rake, and it hitting you in your face, you can now declaratively specify the state of the rake the moment it is embedding itself in your cheekbone.

​

Source