1 of 23

​

Expanding Horizons

Leveraging Advanced Deployment Strategies in Multi-Cluster Kubernetes Environments

​

​

​

Francisco Perea Rodríguez

Eduardo Bonilla Rodríguez

2 of 23

Agenda

Introduction

Why?

Concepts: Deep Dive

So… How to?

Demo

Conclusions

01

02

03

04

05

06

3 of 23

Introduction

4 of 23

Introduction

Consultant at Red Hat

fperearodriguez

Francisco Perea Rodríguez

Senior Customer Success Engineer at Solo.io

Eduardo Bonilla Rodríguez

​

​

edubonifs

5 of 23

Why?

6 of 23

Why?

Cloud Provider

On-Prem

AZ1

AZ2

Cluster1

AZ1

AZ2

Cluster2

App-1

App-1

App-2

App-2

Helloworld

Helloworld

V1

V1

V2

V2

V2

V2

V1

V1

V2

V2

V1

V1

Management cluster

V2

7 of 23

Concepts:

Deep Dive

8 of 23

Kubernetes

Also known as K8s, is an open-source system for automating deployment, scaling, and management of containerized applications.

Designed on the same principles that allow multiple companies to run billions of containers a week, Kubernetes can scale without increasing your operations team.

Whether testing locally or running a global enterprise, Kubernetes flexibility grows with you to deliver your applications consistently and easily no matter how complex your need is.

Kubernetes is open source giving you the freedom to take advantage of on-premises, hybrid, or public cloud infrastructure, letting you effortlessly move workloads to where it matters to you.

9 of 23

Hub Cluster + Workload Clusters

  • Hub Cluster
    • Central Management
    • Monitoring
    • Applications Deployment from Argo
  • Workload Clusters
    • HA
    • Rollouts
    • Applications Networking using Istio

​

10 of 23

Istio

11 of 23

ArgoCD

Argo CD is a declarative GitOps continuous delivery tool for Kubernetes.

It gives us the ability to manage multiple applications in a multi cluster environment

12 of 23

Argo Rollouts

  • Argo Rollouts is a Kubernetes controller and set of CRDs which provide advanced deployment capabilities such as:
    • Blue-Green update strategy
    • Canary update strategy
    • Automated rollbacks and promotions
    • Metric provider integration
    • Service Mesh integration: Istio

13 of 23

Monitoring: Federated Thanos

Prometheus

Prometheus

Thanos Sidecar

Thanos Sidecar

14 of 23

So…

How To?

15 of 23

Top-level architecture

Hub Cluster

Workload cluster 1

App

App

Workload cluster 2

App

App

16 of 23

Istio: Multi-Primary, Multi-Network

mesh1

network1

network2

Workload cluster 1

Workload cluster 2

Service

Service

17 of 23

ArgoCD

Hub Cluster

App

App

Workload Cluster

App

apiVersion: argoproj.io/v1alpha1

kind: Application

metadata:

name: rollouts-app-of-apps

namespace: argocd

spec:

destination:

namespace: argocd

server: https://kubernetes.default.svc

project: default

source:

path: argocd-resources/rollouts/

repoURL: https://github.com/fperearodriguez/multicluster-canary.git

targetRevision: kvm-k8s

syncPolicy:

automated:

prune: true

selfHeal: true

apiVersion: argoproj.io/v1alpha1

kind: ApplicationSet

metadata:

name: rollouts-appset

namespace: argocd

spec:

generators:

- clusters:

selector:

matchLabels:

workloadcluster: "true"

…

apiVersion: argoproj.io/v1alpha1

kind: ApplicationSet

metadata:

name: helloworld-appset

namespace: argocd

spec:

generators:

- matrix:

generators:

- git:

directories:

- path: argocd-resources/rollouts/helloworld/*

…

18 of 23

Prometheus & Thanos

Hub Cluster

Workload cluster 1

Workload cluster 2

PodMonitor��envoy-stats-monitor

ServiceMonitor��istio-component-monitor

PodMonitor��envoy-stats-monitor

ServiceMonitor��istio-component-monitor

19 of 23

apiVersion: argoproj.io/v1alpha1

kind: Rollout

metadata:

name: helloworld

namespace: helloworld

spec:

replicas: 1

selector:

matchLabels:

app: helloworld

service: helloworld

strategy:

…

template:

metadata:

labels:

app: helloworld

service: helloworld

version: v2

spec:

containers:

- image: docker.io/istio/examples-helloworld-v2

name: helloworld

ports:

- containerPort: 5000

name: http

protocol: TCP

resources: {}

serviceAccountName: helloworld

apiVersion: argoproj.io/v1alpha1

kind: AnalysisTemplate

metadata:

name: istio-success-rate

namespace: helloworld

spec:

args:

- name: service

- name: namespace

metrics:

- failureLimit: 3

initialDelay: 15s

interval: 20s

name: success-rate

provider:

prometheus:

address: http://<thanos-endpoint>:9090

query: |

sum(irate(istio_requests_total{

reporter="source",

destination_service=~"helloworld-canary.helloworld.svc.cluster.local",

response_code!~"5.*"}[40s])

) / sum(irate(istio_requests_total{

reporter="source",

destination_service=~"helloworld-canary.helloworld.svc.cluster.local"}[40s])

)

successCondition: result[0] >= 0.90

apiVersion: argoproj.io/v1alpha1

kind: Rollout

metadata:

name: helloworld

namespace: helloworld

spec:

replicas: 1

selector:

matchLabels:

app: helloworld

service: helloworld

strategy:

canary:

analysis:

args:

- name: service

value: helloworld-canary

- name: namespace

valueFrom:

fieldRef:

fieldPath: metadata.namespace

startingStep: 1

templates:

- templateName: istio-success-rate

canaryService: helloworld-canary

stableService: helloworld

steps:

- setWeight: 20

- pause:

duration: 20s

- setWeight: 40

…

Argo Rollouts

Hub Cluster

Workload cluster 1

Workload cluster 2

helloworld-rollout�V1

helloworld-rollout�V1

pod�V1

svc�stable

pod�V1

svc�stable

analysisTemplate

analysisTemplate

helloworld-rollout�V2

helloworld-rollout�V2

pod�V2

pod�V2

istio_requests_total

svc�canary

svc�canary

Virtual Service

Virtual Service

pod�V2

pod�V1

pod�V2

pod�V1

20 of 23

Demo

21 of 23

Demo

22 of 23

Conclusions

23 of 23

​

Blogs:

​

Github repos:

Conclusions