1
9 Steps to Awesome with Kubernetes
Burr Sutter (burrsutter.com)
burrsutter@gmail.com
@burrsutter - bit.ly/9stepsawesome
Burr Sutter (burrsutter.com)
@burrsutter - bit.ly/9stepsawesome
Tutorials
https://dn.dev/learn (top-level index)
https://dn.dev/containers-tutorial
https://dn.dev/kube-tutorial (our focus today)
https://dn.dev/openshift-tutorial
https://dn.dev/knative-tutorial
https://dn.dev/quarkus-tutorial
Learning Resources
@burrsutter - bit.ly/9stepsawesome
Watch at your own pace (pause button)
@burrsutter - bit.ly/9stepsawesome
3 classes on O'Reilly Live Training
Kubernetes Basics
(9 Steps to Awesome)
Istio
(Enter the Service Mesh)
Knative
(Kubernetes Serverless)
https://www.oreilly.com/live-training/
Control-F and search for "burr"
@burrsutter - bit.ly/9stepsawesome
Setup
https://redhat-scholars.github.io/kubernetes-tutorial/kubernetes-tutorial/installation.html
Tips on the Virtualization Drivers�https://docs.okd.io/latest/minishift/getting-started/setting-up-virtualization-environment.html
@burrsutter - bit.ly/9stepsawesome
Your Journey to Awesomeness
Self-Service,
On-Demand,
Elastic Infrastructure
Automation
CI & CD
Deployment Pipeline
Advanced
Deployment
Techniques
Microservices
Re-Org to
DevOps
@burrsutter - bit.ly/9stepsawesome
MVP
@burrsutter - bit.ly/9stepsawesome
Continuous Delivery, Deployment, Improvement
Agility
@burrsutter - bit.ly/9stepsawesome
3 Month Deployment Cycle
3 Months BEFORE you gain Feedback and Learn
@burrsutter - bit.ly/9stepsawesome
Love Thy Mono
Old School New School
@burrsutter - bit.ly/9stepsawesome
9 Steps
0 - Introduction�1 - Installation & Getting Started�2 - Building Images, Running Containers�3 - Logs�4 - oc/kubectl exec magic�5 - Configuration & Environment�6 - Service discovery & load-balancing�7 - Live & Ready�8 - Rolling updates, Canaries, Blue/Green�9 - Debugging Databases�Bonus Items
@burrsutter - bit.ly/9stepsawesome
Step 0: Introduction
@burrsutter - bit.ly/9stepsawesome
MyApp.war has been tested with the following
On my Windows 7 desktop
JDK 1.8.43
Wildfly 9
Configuration:
Datasource: MySQLDS
Tested with: mysql-connector-java-5.1.31-bin.jar
Production Environment
Red Hat Enterprise Linux 6.2
JRE 1.7.3
WebSphere 8.5.5
Oracle 9
@burrsutter - bit.ly/9stepsawesome
Dockerfile
Container Guest OS
Java App Server
custom configuration
FROM centos/wildfly�
�COPY xyz.xml /opt/myserver/configuration/�
COPY mysql-connector-java-5.1.31-bin.jar /opt/wildfly/standalone/deployments/�COPY mysql-sample-ds.xml /opt/wildfly/standalone/deployments/�
�COPY myapp/target/your.war /opt/wildfly/standalone/deployments/
your.war
dependencies
@burrsutter - bit.ly/9stepsawesome
DevOps Challenges for Multiple Containers
Node
Node
Node
Node
Node
Logger
Node
@burrsutter - bit.ly/9stepsawesome
A Challenge
Server Hardware
Operating System
Application Server
.war or .ear
Java Virtual Machine
Custom Configuration
Linux Kernel Version & Distribution
Java 1.6.6_45 or Java 1.7.0_67
Weblogic 10.x.y, Tomcat 6.x.y, JBoss EAP 6.x.y
JDBC driver, datasource, JMS queue, users
Have you ever had “/” vs “\” break your app? Or perhaps needed a unique version of a JDBC driver? Or had a datasource with a slightly misspelled JNDI name? Or received a patch for the JVM or app server that broke your code?
Containerize
Your
App
@burrsutter - bit.ly/9stepsawesome
@burrsutter - bit.ly/9stepsawesome
Meet Kubernetes
@burrsutter - bit.ly/9stepsawesome
Key Capabilities
@burrsutter - bit.ly/9stepsawesome
@burrsutter - bit.ly/9stepsawesome
Pods
A group of whales is commonly referred to as a pod and a pod usually consists a group of whales that have bonded together either because of biological reasons or through friendships developed between two or more whales.
In many cases a typical whale pod consists of anywhere from 2 to 30 whales or more.*
*http://www.whalefacts.org/what-is-a-group-of-whales-called/
@burrsutter - bit.ly/9stepsawesome
Kubernetes Terms
Label
@burrsutter - bit.ly/9stepsawesome
Kubernetes Cluster - Nodes
Master
Node
Node
Istio�proxy
Node
Node
Node
Node
Dev
Ops
api
scheduler
controllers
kubelet
kubelet
kubelet
kubelet
kubelet
kubelet
@burrsutter
Kubernetes Cluster - Declarative
Master
Node
Node
Istio�proxy
Node
Node
Node
Node
Dev
Ops
api
scheduler
controllers
kubelet
kubelet
kubelet
kubelet
kubelet
kubelet
image: repo/mytomcat:v1
replicas: 4
@burrsutter
Kubernetes Cluster - 4 Tomcats
Master
Node
Node
Istio�proxy
Node
Node
Node
Node
Dev
Ops
api
scheduler
controllers
kubelet
kubelet
kubelet
kubelet
kubelet
kubelet
@burrsutter
Kubernetes Cluster - Pod Fail
Master
Node
Node
Istio�proxy
Node
Node
Node
Node
Dev
Ops
api
scheduler
controllers
kubelet
kubelet
kubelet
kubelet
kubelet
kubelet
X
@burrsutter
Kubernetes Cluster - Correcting
Master
Node
Node
Istio�proxy
Node
Node
Node
Node
Dev
Ops
api
scheduler
controllers
kubelet
kubelet
kubelet
kubelet
kubelet
kubelet
@burrsutter
Kubernetes Cluster - Node Fail
Master
Node
Node
Istio�proxy
Node
Node
Node
Node
Dev
Ops
api
scheduler
controllers
kubelet
kubelet
kubelet
kubelet
kubelet
kubelet
X
@burrsutter
Kubernetes Cluster
Master
Node
Node
Istio�proxy
Node
Node
Node
Dev
Ops
api
scheduler
controllers
kubelet
kubelet
kubelet
kubelet
kubelet
@burrsutter
Kubernetes Cluster - New Node
Master
Node
Node
Istio�proxy
Node
Node
Node
Dev
Ops
api
scheduler
controllers
kubelet
kubelet
kubelet
kubelet
kubelet
Node
kubelet
@burrsutter
Labels
App: Cool
Env: Dev
Version: 1.0
App: Cool
Env: Prod
Version: 1.0
App: Cool
Env: Dev
Version: 2.0
App: Cool
Env: Prod
Version: 2.0
Node
Logger
Node
Node
Node
@burrsutter - bit.ly/9stepsawesome
Labels App:Cool
App: Cool
Env: Dev
Version: 1.0
App: Cool
Env: Prod
Version: 1.0
App: Cool
Env: Dev
Version: 2.0
App: Cool
Env: Prod
Version: 2.0
Node
Logger
Node
Node
Node
@burrsutter - bit.ly/9stepsawesome
Labels Env:Dev
App: Cool
Env: Dev
Version: 1.0
App: Cool
Env: Prod
Version: 1.0
App: Cool
Env: Dev
Version: 2.0
App: Cool
Env: Prod
Version: 2.0
Node
Logger
Node
Node
Node
@burrsutter - bit.ly/9stepsawesome
Labels Env:Prod
App: Cool
Env: Dev
Version: 1.0
App: Cool
Env: Prod
Version: 1.0
App: Cool
Env: Dev
Version: 2.0
App: Cool
Env: Prod
Version: 2.0
Node
Logger
Node
Node
Node
@burrsutter - bit.ly/9stepsawesome
Labels Version:1.0
App: Cool
Env: Dev
Version: 1.0
App: Cool
Env: Prod
Version: 1.0
App: Cool
Env: Dev
Version: 2.0
App: Cool
Env: Prod
Version: 2.0
Node
Logger
Node
Node
Node
@burrsutter - bit.ly/9stepsawesome
https://dzone.com/articles/kubernetes-lifecycle-of-a-pod
https://blog.heptio.com/core-kubernetes-jazz-improv-over-orchestration-a7903ea92ca
@burrsutter - bit.ly/9stepsawesome
@burrsutter - bit.ly/9stepsawesome
kubectl commands
kubectl get namespaces
kubectl get pods -n mynamespace
kubectl run myvertx --image=burr/myvertx:v1 --port=8080
kubectl logs myvertx-kk605
kubectl expose deployment --port=8080 myvertx --type=LoadBalancer
kubectl scale deployment myvertx --replicas=3
kubectl set image deployment/myvertx myvertx=burr/myvertx:v2
@burrsutter - bit.ly/9stepsawesome
Microservices == Distributed Computing
Pod
Container
JVM
Service A
Pod
Container
JVM
Service B
Pod
Container
JVM
Service C
@burrsutter - bit.ly/9stepsawesome
java -jar myapp.jar
Spring Boot
spring.io/projects/�spring-boot
Spring API (@RestController)
‘Starter’ POMs: �start.spring.io
@burrsutter - bit.ly/9stepsawesome
Quarkus: Supersonic, Subatomic Java
Kubernetes Native
@burrsutter - bit.ly/9stepsawesome
History of Microservices
@burrsutter - bit.ly/9stepsawesome
Continuous
Integration
via XP
1999
AWS EC2
2006
DropWizard
May 2011
Agile
Manifesto
Feb
2001
NETFLIX
to AWS
2010
Ribbon
March� 2012
Hystrix
March 2012
Eureka
July 2012
Java EE6
2009
DevOps
2009
June 2011
2
0
1
5
@burrsutter - bit.ly/9stepsawesome
2015�
Launch 1000+
Containers
Audience
Claims a Container
@burrsutter - bit.ly/9stepsawesome
MyService
Tracing
API
Discovery
Invocation
Resilience
Pipeline
Authentication
Logging
Elasticity
Monitoring
Polyglot Microservices Platform
@burrsutter - bit.ly/9stepsawesome
9 Steps
@burrsutter - bit.ly/9stepsawesome
Step 1: Installation
@burrsutter - bit.ly/9stepsawesome
Lots of Options
Step 1: Installation
@burrsutter - bit.ly/9stepsawesome
Tools
@burrsutter
Localhost Development
Hosted Kubernetes Cluster
Kubernetes Options
@burrsutter
Step 1 Demo
https://redhat-scholars.github.io/kubernetes-tutorial/kubernetes-tutorial/installation.html
@burrsutter - bit.ly/9stepsawesome
Step 2: Building Images
@burrsutter - bit.ly/9stepsawesome
Running your App on Kubernetes
@burrsutter - bit.ly/9stepsawesome
Step 2: Building Images
Options Include:
@burrsutter - bit.ly/9stepsawesome
Managing Yaml
Options Include:
@burrsutter - bit.ly/9stepsawesome
Beyond Managing Yaml - Operators
Phase I
Phase II
Phase III
Phase IV
Phase V
Installation
Provisioning
Configuration
Management
Updates
Patches and Minors
Full Lifecycle
App and Storage (backup, failure, recovery)
Deep Insights
Metrics, alerts, log processing, workload analysis
Auto Pilot
Auto tuning, auto scaling, abnormality detection
Ansible
Go
@burrsutter - bit.ly/9stepsawesome
@burrsutter - bit.ly/9stepsawesome
FROM fabric8/java-jboss-openjdk8-jdk:1.4.0�ENV JAVA_APP_DIR=/deployments�EXPOSE 8080 8778 9779�COPY target/my.jar /deployments/
https://github.com/fabric8io-images/java/tree/master/images/jboss/openjdk8/jdk
https://github.com/fabric8io-images/java/tree/master/images/jboss/openjdk8/jdk
Dockerfile for Java projects
@burrsutter - bit.ly/9stepsawesome
minikube(docker-env) or minishift(docker-env)�docker build -t burr/myimage:v1 .�docker run -it -p 8080:8080 burr/myimage:v1�curl $(minishift ip):8080�# now run it on Kubernetes�kubectl run myapp --image burr/myimage:v1 --port 8080�kubectl expose deployment --port=8080 myapp --type=LoadBalancer�oc expose service myapp�curl myapp-stuff.$(minishift ip).nip.io�# scale up�kubectl scale --replicas=2 deploy/myapp�# create an updated image�docker build -t burr/myimage:v2 .�# rollout update�kubectl set image deployment/myapp myapp=burr/myimage:v2
docker build, kubectl run
@burrsutter - bit.ly/9stepsawesome
https://maven.fabric8.io/#fabric8:setup (no Dockerfile, Deployment.yml)
oc new-project stuff (or kubectl create namespace)
mvn clean compile package
mvn io.fabric8:fabric8-maven-plugin:3.5.40:setup
mvn fabric8:deploy
Fabric8 Maven Plugin http://jshift.io/
@burrsutter - bit.ly/9stepsawesome
Slides: bit.ly/javadockerfail�Recording from JBCNConf 2017
docker run -m 100MB openjdk:8u121 java -XshowSettings:vm -version
docker run -m 100MB openjdk:8u131 java -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap -XshowSettings:vm -version
Do NOT Java + Docker == FAIL
@burrsutter - bit.ly/9stepsawesome
Step 2 Demo
@burrsutter - bit.ly/9stepsawesome
Step 3: logs
@burrsutter - bit.ly/9stepsawesome
System.out.println("Java, where am I?");�Or console.log("Node logs");
kubectl get pods�kubectl logs microspringboot1-2-nz8f8 �kubectl logs microspringboot1-2-nz8f8 -p # last failed pod
OR stern (brew install stern)�https://github.com/wercker/stern�stern myboot
OR kail (https://github.com/boz/kail)�kail -d myboot
logs
@burrsutter
Step 3 Demo
@burrsutter - bit.ly/9stepsawesome
Step 4: oc or kubectl exec
@burrsutter - bit.ly/9stepsawesome
"ssh" into your containers and explore
kubectl get pods --namespace=microworld�kubectl exec -it --namespace=microworld $POD cat /sys/fs/cgroup/memory/memory.limit_in_bytes�Or�kubectl exec -it --namespace=microworld microspringboot1-2-nz8f8 /bin/bash�ps -ef | grep java�Note: the following apply if using the fabric8 generated image, otherwise consult your Dockerfile�java -version�javac -version�# now find that fat jar�find / -name *.jar �cd /deployments (based on use of the fabric8 maven plugin)�ls�exit
Step 4: oc or kubectl exec
@burrsutter - bit.ly/9stepsawesome
Step 4 Demo
https://github.com/burrsutter/9stepsawesome/blob/master/4_kubectl_exec.adoc
Video on Resource Limits/Constraints
@burrsutter - bit.ly/9stepsawesome
Step 5: env and configmaps
@burrsutter - bit.ly/9stepsawesome
An app’s config is everything that is likely to vary between deploys (staging, production, developer environments, etc). 12 Factor Apps
kubectl set env deployment/myboot DBCONN="jdbc:sqlserver://45.91.12.123:1443;user=MyUserName;password=*****;"
kubectl create cm my-config --from-env-file=config/some.properties
Step 5: env vars & configmaps
@burrsutter - bit.ly/9stepsawesome
some.properties
GREETING=jambo�LOVE=Amour
Partial deployment.yml
spec:
containers:
- name: myboot
image: 9stepsawesome/myboot:v1
ports:
- containerPort: 8080
envFrom:
- configMapRef:
name: my-config
Partial .java
@Autowired
private Environment environment;
String greeting = environment.getProperty("GREETING","Default");
String love = environment.getProperty("LOVE","Default");
ConfigMaps
kubectl create cm my-config --from-env-file=config/some.properties
@burrsutter
Step 5 Demo
@burrsutter - bit.ly/9stepsawesome
Step 6: service discovery & load-balancing
@burrsutter - bit.ly/9stepsawesome
String url = "http://producer:8080/";
ResponseEntity<String> response = restTemplate.getForEntity(url, String.class);
Step 6: Service Discovery
@burrsutter - bit.ly/9stepsawesome
@burrsutter - bit.ly/9stepsawesome
Step 7: Live and Ready
@burrsutter - bit.ly/9stepsawesome
Step 7: Live and Ready
resources:
requests:
memory: "300Mi"
cpu: "250m" # 1/4 core
limits:
memory: "400Mi"
cpu: "1000m" # 1 core
livenessProbe:
httpGet:
port: 8080
path: /
initialDelaySeconds: 10
periodSeconds: 5
timeoutSeconds: 2
readinessProbe:
httpGet:
path: /health
port: 8080
initialDelaySeconds: 10
periodSeconds: 3
kubectl create -f Deployment.yml
@burrsutter - bit.ly/9stepsawesome
Step 7 Demo
@burrsutter - bit.ly/9stepsawesome
Step 8: Rolling Updates, Blue/Green�Canary
@burrsutter - bit.ly/9stepsawesome
Blue/Green Deployment
@burrsutter - bit.ly/9stepsawesome
Blue/Green Deployment
DEVELOPMENT
QA
STAGING
PRODUCTION
ROUTER
USERS
BUILD
SCM
@burrsutter - bit.ly/9stepsawesome
Blue/Green Deployment
DEVELOPMENT
QA
STAGING
PRODUCTION
ROUTER
USERS
BUILD
SCM
@burrsutter - bit.ly/9stepsawesome
Blue/Green Deployment
DEVELOPMENT
QA
STAGING
PRODUCTION
ROUTER
USERS
BUILD
SCM
@burrsutter - bit.ly/9stepsawesome
Blue/Green Deployment
DEVELOPMENT
QA
STAGING
PRODUCTION
ROUTER
USERS
BUILD
SCM
@burrsutter - bit.ly/9stepsawesome
Blue/Green Deployment
DEVELOPMENT
QA
STAGING
PRODUCTION
ROUTER
USERS
BUILD
SCM
@burrsutter - bit.ly/9stepsawesome
Blue/Green Deployment
DEVELOPMENT
QA
STAGING
PRODUCTION
ROUTER
USERS
SCM
@burrsutter - bit.ly/9stepsawesome
Blue/Green Deployment
DEVELOPMENT
QA
STAGING
PRODUCTION
ROUTER
USERS
SCM
x
@burrsutter - bit.ly/9stepsawesome
Blue/Green Deployment
DEVELOPMENT
QA
STAGING
PRODUCTION
ROUTER
USERS
SCM
@burrsutter - bit.ly/9stepsawesome
Step 8 Demo
https://redhat-scholars.github.io/kubernetes-tutorial/kubernetes-tutorial/blue-green.html
@burrsutter - bit.ly/9stepsawesome
New Step 9: Databases
@burrsutter - bit.ly/9stepsawesome
https://github.com/burrsutter/9stepsawesome/blob/master/9_databases.adoc
Step 9: Databases
@burrsutter - bit.ly/9stepsawesome
Bonus: Istio
@burrsutter - bit.ly/9stepsawesome
@burrsutter - bit.ly/9stepsawesome
Code Independent
Next Generation - Service Mesh
@burrsutter - bit.ly/9stepsawesome
What is Knative?
"Kubernetes-based platform to build, deploy, and manage modern serverless workloads."
"Essential base primitives for all"
"Knative provides a set of middleware components that are essential to build modern, source-centric, and container-based applications that can run anywhere: on premises, in the cloud, or even in a third-party data center"
@burrsutter - bit.ly/9stepsawesome
Apache Kafka on Kubernetes & OpenShift
@burrsutter - bit.ly/9stepsawesome
Free
eBooks/
Resources
@burrsutter - bit.ly/9stepsawesome
dn.dev/knative-cookbook
@burrsutter - bit.ly/9stepsawesome
https://dn.dev/k8s-patterns
@burrsutter - bit.ly/9stepsawesome
bit.ly/mono2microdb
@burrsutter - bit.ly/9stepsawesome
https://dn.dev/istiobook
@burrsutter - bit.ly/9stepsawesome
Backup Content
@burrsutter - bit.ly/9stepsawesome
@burrsutter - bit.ly/9stepsawesome
| Kubernetes | OpenShift |
Multi-container, multi-host scheduling | ✅ | ✅ |
Self-service provisioning | ✅ | ✅ |
Service discovery | ✅ | ✅ |
Persistent Storage | ✅ | ✅ |
Multi-tenancy | | ✅ |
Networking (SDN) | | ✅ |
Image Registry | | ✅ |
Image Build Tools | | ✅ |
Metrics | | ✅ |
Log Aggregation | | ✅ |
Ingress | | ✅ |
UX: Console, Service Catalog, "oc" | | ✅ |
Secured by Default | | ✅ |
@burrsutter - bit.ly/9stepsawesome
Kubernetes
Kube Native
App
Helmed
App
Kubernetes Ingress
Kubernetes CLI
Kubernetes API
@burrsutter - bit.ly/9stepsawesome
RHEL
Ansible/Terraform/Operators
OVN
Kubernetes
Kube Native
App
OpenShift
Specialized Java Services
---
Any Language�Any Framework
OpenShift Native
App
Operated/Helmed
App
RHT Storage
Quay
Telemetry
(EFK/Profana)
Admin Console
Kubernetes Ingress
OpenShift Routes
Kubernetes CLI
OpenShift CLI
Kubernetes API
OpenShift API
Che
Templates/S2I/BC
OpenShift Mesh/Istio
Knative
@burrsutter - bit.ly/9stepsawesome
Strangler Strategy
@burrsutter - bit.ly/9stepsawesome
@burrsutter - bit.ly/9stepsawesome
@burrsutter - bit.ly/9stepsawesome
Application on Whiteboard
@burrsutter - bit.ly/9stepsawesome
@burrsutter - bit.ly/9stepsawesome
Whiteboard had 3 Tiers
@burrsutter - bit.ly/9stepsawesome
@burrsutter - bit.ly/9stepsawesome
UI, Logic, Data
@burrsutter - bit.ly/9stepsawesome
@burrsutter - bit.ly/9stepsawesome
Clean Architecture
@burrsutter - bit.ly/9stepsawesome
@burrsutter - bit.ly/9stepsawesome
Real Life: Non-Majestic Monolith
@burrsutter - bit.ly/9stepsawesome
@burrsutter - bit.ly/9stepsawesome
Large Team
@burrsutter - bit.ly/9stepsawesome
@burrsutter - bit.ly/9stepsawesome
Strangle
@burrsutter - bit.ly/9stepsawesome
@burrsutter - bit.ly/9stepsawesome
Strangle Hug
@burrsutter - bit.ly/9stepsawesome
@burrsutter - bit.ly/9stepsawesome
Friend Hug
@burrsutter - bit.ly/9stepsawesome
@burrsutter - bit.ly/9stepsawesome
Family Hug
@burrsutter - bit.ly/9stepsawesome
@burrsutter - bit.ly/9stepsawesome
Bear Hug
@burrsutter - bit.ly/9stepsawesome
@burrsutter - bit.ly/9stepsawesome
"Refactor"
@burrsutter - bit.ly/9stepsawesome
@burrsutter - bit.ly/9stepsawesome
Squeeze
@burrsutter - bit.ly/9stepsawesome
@burrsutter - bit.ly/9stepsawesome
Streamline
@burrsutter - bit.ly/9stepsawesome
@burrsutter - bit.ly/9stepsawesome
Embrace Others
@burrsutter - bit.ly/9stepsawesome
@burrsutter - bit.ly/9stepsawesome
Recording from Devoxx BE 2018: https://www.youtube.com/watch?v=ZpbXSdzp_vo
"excellent stuff.. worth 3 hours spending to watch this video.. Really loved it. Will give a complete picture on Kubernetes and how it works" - Pradeep
@burrsutter - bit.ly/9stepsawesome
http://www.g-pop.net/fistoffury.htm
@burrsutter - bit.ly/9stepsawesome
@burrsutter - bit.ly/9stepsawesome