1 of 27

Virtualization, Containers, Orchestration, Architecture

2 of 27

Quick about me

  • Graduated Spring 2018
  • Former OCF General Manager and Decal Administrator
  • Work with Apache Kafka and more recently Kubernetes at Yelp
  • All thoughts and strong opinions in this presentation are my own
  • Follow me on Twitter: @baisang_

3 of 27

A warning

  • I have incredibly diverged from how I used to give this talk
    • I want to focus on the big picture, since I think you are all smart enough to follow basic tutorials
  • I hope it will be for the better
  • If not please see my old slides

4 of 27

This lecture

  • First half will be high level talk about why people are krazy for kubernetes
  • Second half will be an interactive web architecture workshop
  • Scratch time -- ask me anything until about 8:50 or so (career, tech, whatever)

5 of 27

Why won’t infra people shut up about containers and Kubernetes

​

6 of 27

How are most web applications made?

Database

Some social media website

Client

Gimme a post

Here’s a post

Client

A post

7 of 27

Monoliths

Database

Some social media website

Client

Gimme a post

Here’s a post

Client

A post

Photo storage

Russian Ads

8 of 27

Microservices have exploded in popularity

9 of 27

Microservices

  • Small, sharp tools
  • Perform limited set of functions as a black box
  • Communicate with each other via APIs and messages
  • Extremely important as companies scale
  • Example: separate services for consumer-facing and business-facing pages

10 of 27

11 of 27

How do we deploy these services?

  • Using our own physical hardware
    • No
  • Using Cloud Provider Virtual Machines
    • Inefficient - if we run one service instance per VM
    • Complicated - if we try running multiple instances (of different types) per VM

12 of 27

Containers

13 of 27

Have we solved our problem?

  • Containers allow us to package our applications in an isolated way
  • We can now bin-pack, fit more than 1 container on a single machine without fears
  • But how do we coordinate this across all of our machines?

14 of 27

True microservice deployment requires orchestration

15 of 27

Container Orchestrators

  • Container Orchestrators help us make sure we have the right amount of containers we want, that they are on the proper machines, restart them when they die to handle failure, etc.
  • Use Distributed Systems Magic™ to recover from failures; multiple masters, distributed key/value stores, and more
  • Examples: Kubernetes, Mesos+Marathon, Docker Swarm

16 of 27

17 of 27

What has this allowed us to do?

  • Easy to create, deploy, and scale new microservices efficiently
  • Container orchestrator does the hard work, software engineers can focus their time on coding instead of deploying

​

18 of 27

Architecture and Design

19 of 27

Don’t read ahead! This part is meant to be interactive

20 of 27

Basic File storage service

Database

File request handler

Client

File download request

File

Client

File upload

21 of 27

Divide into separate services

Database

File upload handler

Client

File download request

File

Client

File upload

File download handler

Problem: both servers for file upload/download are hitting 100% CPU load. What can we do?

22 of 27

Improvement 1: More handlers

Database

File upload handler

File upload handler

  • If one of the handlers fails, now have more!
  • Helps distribute load amongst handlers - scale out
  • New problem: Database disk I/O is maxing out, what do we do now?

Upload

Download

File download handler

File download handler

Upload LB

Download LB

23 of 27

Improvement 2: Caching!

Database

File upload handler

File upload handler

  • Use Redis or Memcached (keep data in memory, fast)
  • Keep caches as close to the client as possible - locality
  • Another problem to fix: what if database goes down?

Upload

Download

File download handler

File download handler

Upload LB

Download LB

Cache

Cache

24 of 27

Improvement 3: Replication

Database

File upload handler

File upload handler

  • Same idea as having a backup
  • Can use replica for reads if low latency
  • Problems, again: Your app is now on front page of Reddit and the disk that your database uses is full

Upload

Download

File download handler

File download handler

Upload LB

Download LB

Cache

replica

Cache

25 of 27

Improvement 4: Partition

DB 1

File upload handler

File upload handler

  • Split database into multiple machines
  • Need to route requests to correct DB
  • More complexity so that’s kinda bad
  • Now we can scale out application servers layer and data layer

Upload

Download

File download handler

File download handler

Upload LB

Download LB

Cache

Rep 1

DB 2

Rep 2

Cache

26 of 27

More reading

​

27 of 27

Questions