1 of 18

GitOps: yea or nay?

May 4, 2021

DevOpsDays Seattle 2021�Ricardo Castro

2 of 18

Agenda

What is GitOps
What can GitOps do for me
GitOps CD Software Agents
Demo
Extras

2

@mccricardo

https://mccricardo.com

3 of 18

Story time

Scenario: some random day. The sun is shining. Life is good.

Ricardo: I'm going to make a tricky change. There's the possibility of a system wipeout.

Joe: Are you sure? Let me take a look!

Joe: Looks ok. Good to go!

Ricardo: So… Here’s the thing… Remember that Kubernetes cluster? It’s gone...

<CHAOS>

3

@mccricardo

https://mccricardo.com

4 of 18

4

@mccricardo

https://mccricardo.com

5 of 18

Story time, Part II

Ricardo: Thankfully we have everything in Terraform. Let’s relaunch the cluster.

Joe: Cool, it’s up! Let’ figure out what needs to be launched.

Joe: <Smirk> Fortunately we have a pipeline for everything.

Ricardo: Pipeline Alpha-X was deactivated? Any idea why? Shall we ask team Alpha?

Ricardo: Also Pipeline Beta-Y was successful but the application does not work. Any idea why?

Joe: <Thinking> I think team Beta had some manual changes there...

5

@mccricardo

https://mccricardo.com

6 of 18

What is GitOps

Principles

The entire system described declarative
The canonical desired system state is versioned in Git
Approved changes that can be automatically applied to the system
Software agents to ensure correctness and alert on divergence

6

@mccricardo

https://mccricardo.com

#1. The entire system described declaratively.

Kubernetes is just one example of many modern cloud native tools that are “declarative” and that can be treated as code. Declarative means that configuration is guaranteed by a set of facts instead of by a set of instructions. With your application’s declarations versioned in Git, you have a single source of truth. Your apps can then be easily deployed and rolled back to and from Kubernetes. And even more importantly, when disaster strikes, your cluster’s infrastructure can also be dependably and quickly reproduced.

#2. The canonical desired system state versioned in Git.

With the declaration of your system stored in a version control system, and serving as your canonical source of truth, you have a single place from which everything is derived and driven. This trivializes rollbacks; where you can use a `Git revert` to go back to your previous application state. With Git’s excellent security guarantees, you can also use your SSH key to sign commits that enforce strong security guarantees about the authorship and provenance of your code.

#3. Approved changes that can be automatically applied to the system.

Once you have the declared state kept in Git, the next step is to allow any changes to that state to be automatically applied to your system. What's significant about this is that you don't need cluster credentials to make a change to your system. With GitOps, there is a segregated environment of which the state definition lives outside. This allows you to separate what you do and how you're going to do it.

#4. Software agents to ensure correctness and alert on divergence.

Once the state of your system is declared and kept under version control, software agents can inform you whenever reality doesn’t match your expectations. The use of agents also ensures that your entire system is self-healing. And by self-healing, we don’t just mean when nodes or pods fail—those are handled by Kubernetes—but in a broader sense, like in the case of human error. In this case, software agents act as the feedback and control loop for your operations.

7 of 18

What GitOps can do for me

Increased Productivity
Enhanced Developer Experience
Improved Stability
Higher Reliability
Consistency and Standardization
Stronger Security Guarantees
Easier Compliance and Auditing

7

@mccricardo

https://mccricardo.com

The benefits of applying GitOps best practices are far reaching and provide:

Increased Productivity �Continuous deployment automation with an integrated feedback control loop speeds up Mean Time to Deployment. Your team can ship 30-100 times more changes per day, increasing overall development output 2-3 times.
Enhanced Developer Experience �Push code and not containers. Developers can use familiar tools like Git to manage updates and features to Kubernetes more rapidly without having to know the internal of Kubernetes. Newly on-boarded developers can get quickly up to speed and be productive within days instead of months.
Improved Stability�When you use Git workflows to manage your cluster, you automatically gain a convenient audit log of all cluster changes outside of Kubernetes. An audit trail of who did what, and when to your cluster can be used to meet SOC 2 compliance and ensure stability.
Higher Reliability�With Git’s capability to revert/rollback and fork, you gain stable and reproducible rollbacks. Because your entire system is described in Git, you also have a single source of truth from which to recover after a meltdown, reducing your meantime to recovery (MTTR) from hours to minutes.
Consistency and Standardization�Because GitOps provides one model for making infrastructure, apps and Kubernetes add-on changes, you have consistent end-to-end workflows across your entire organization. Not only are your continuous integration and continuous deployment pipelines all driven by pull request, but your operations tasks are also fully reproducible through Git.
Stronger Security Guarantees�Git’s strong correctness and security guarantees, backed by the strong cryptography used to track and manage changes, as well as the ability to sign changes to prove authorship and origin is key to a secure definition of the desired state of the cluster.
Since changes are tracked and logged in a secure manner, compliance and auditing are made trivial. The use of comparison tools like kubediff, terradiff and ansiblediff also allow you to compare a trusted definition of the state of the cluster with the actual running cluster, ensuring that the tracked and auditable changes match reality.

8 of 18

GitOps CD Software Agents

Image source: https://fluxcd.io/

8

@mccricardo

https://mccricardo.com

9 of 18

GitOps CD Software Agents

Flux CD

“Flux is a tool for keeping Kubernetes clusters in sync with sources of configuration (like Git repositories), and automating updates to configuration when there is new code to deploy.”

Image source: https://toolkit.fluxcd.io/

9

@mccricardo

https://mccricardo.com

10 of 18

GitOps CD Software Agents

Argo CD

“Argo CD is a declarative, GitOps continuous delivery tool for Kubernetes.”

Image source: https://argoproj.github.io/argo-cd/

10

@mccricardo

https://mccricardo.com

11 of 18

GitOps CD Software Agents

Jenkins X

“Jenkins X provides pipeline automation, built-in GitOps, and preview environments to help teams collaborate and accelerate their software delivery at any scale.”

Image source: https://jenkins-x.io/about/

11

@mccricardo

https://mccricardo.com

12 of 18

GitOps CD Software Agents

Image source: https://www.weave.works/blog/gitops-high-velocity-cicd-for-kubernetes

12

@mccricardo

https://mccricardo.com

13 of 18

Demo

Flux CD

DefinedCrowd® confidential

@mccricardo

https://mccricardo.com

14 of 18

Extras

What about secrets?

14

@mccricardo

https://mccricardo.com

15 of 18

Extras

GitOps vs IaC

“One of the main differences between IaC and GitOps is the use of immutable containers as deployable artefacts that can be converged on by a suitable orchestration tool, for example Kubernetes. Your desired state is kept in source control. This isn’t always the case with some of the infrastructure as code tools. IaC implementations vary, and sometimes the source of truth is split between a git repo, and a database, and sometimes spread between a weakly-linked union of multiple git repos.”

“While Infrastructure as Code tools provide a way to manage infrastructure, they don’t manage the entire Cloud Native stack. GitOps on the other hand, allows you to manage the whole platform.”

15

@mccricardo

https://mccricardo.com

16 of 18

Extras

Push vs Pull

Push

Pull

16

@mccricardo

https://mccricardo.com

17 of 18

Thank you

About me

Senior Site Reliability Engineer at DefinedCrowd (We’re hiring)
DevOps Porto meetup co-organizer
DevOpsDays Portugal conference co-organizer
Strong believer in culture and teamwork
Open source passionate (former Mozilla contributor)
Taekwondo amateur
Metal lover
Where to find me:

17

@mccricardo

https://mccricardo.com

18 of 18

Links

@mccricardo

https://mccricardo.com