1 of 55

CI/CD in a

Cloud Native World

OSCON 2019

2 of 55

Christie Wilson

Engineer @ Google

Tekton Co-Creator

MY CAT

ME

3 of 55

Cloud Native CI/CD:

Failing Without Fear

4 of 55

Rockstars

Heroes

Ninjas

Janitors

5 of 55

Can’t have success without failure

6 of 55

Cloud Native CI/CD

can make failure easier

7 of 55

What we’re gonna talk about

CI (and CD)
How Cloud Native makes things harder
But it’s worth it: Cloud Native CI

Serverless
Specs and standards
Infrastructure agnostic
Reusable components
Config as code

8 of 55

CI/CD is about making failure safe

9 of 55

What is CI/CD?

10 of 55

What is CI?

Continuous Integration

Literally integrating code together continually

11 of 55

Back in my day

Worked on SVN branches for weeks or months
Integrated them together
No/few tests

What we were doing before
SVN branch
5 ppl merge
No tests, or no automation

To understand continuous integration, you have to understand what we were doing before we had CI.

At my first few jobs, every time I developed a feature, I’d work on an SVN branch, by myself, for weeks or sometimes months.

When I was done, I’d submit the change and deal with any conflicts.

I remember one particularly tense afternoon, sitting with 2 colleagues and staring at conflicts until my eyes hurt because something like 5 of us were trying to submit huge feature branches at the same time.

And there are very few tests. I only learned about the idea of a test baer the end of my time at my first job, and even then I didn’t write them.

At my second job, there were tests, but they had to be run manually and there was no automation at all.

12 of 55

Failing was hard!

13 of 55

CI = Assembly Line?

14 of 55

CI = Assembly Line?

15 of 55

We become better creators by failing

16 of 55

Shift Left

Fail earlier!

There WILL be defects

Test earlier!
Catch failures earlier!

17 of 55

What we’re gonna talk about

CI (and CD)
How Cloud Native makes things harder
But it’s worth it: Cloud Native CI

Serverless
Specs and standards
Infrastructure agnostic
Reusable components
Config as code

18 of 55

What is Cloud Native?

}

Microservices in containers

Images / Containers

Dynamically orchestrated

Optimized resource utilization

}

Kubernetes

19 of 55

Moar complicated

20 of 55

2008 complexity

1st arch
Perl, MVC, 2 endpoints
Release not part of my life
Production servers
Kleenex box

By way of proving that to you, let me tell you about the architecture of the first software system I worked on.

It was all written in Perl. It used the Model, View, Controller pattern and applications I wrote had to connect to a grand total of 2 endpoints:

One was a server that stored and handled user data.

The other was a MySQL database, which I could instantiate on my own machine.

I don’t even know how releasing the software worked. That was not a part of my life at all.

What _was_ a part of my life was dealing with production issues by connecting to live servers.

A coworker with a cold once dropped a kleenex box on his keyboard and dropped a live table in our prod database.

It was scary, and it was risky, but it wasn’t really that complicated

21 of 55

Cloud Native complexity

CN = risk removed, moved, more complex
Microservices, many lang
Built into images, registry
Deploying = layers of config
Templating to make simpler, can’t point at prod config
Microservices find each other

Now what do cloud native applications look like?

A lot of that risk has been removed, or at least moved, but there is way more complexity for developers to deal with.

One application can be made up of many microservices, written in a variety of languages.

Those microservices are built by something, and published to one or more registries.

Deploying these microservices, if you’re using kubernetes, means creating layer upon layer of configuration, involving abstract concepts like Deployments, Services, and Pods, which you need to learn about and understand.

And managing all that configuration often involves multiple levels of templating to try to add reuse (ironically with the goal of making things simpler).

This means it can be very hard to point at one config file and say: this is the config file for foobar service running in production.

Not to mention that these microservices have to find and connect to each other.

I could go on!

22 of 55

Cloud Native complexity

There is a lot to grok here and it’s constantly changing!

Start with a webserver
End up with Kubernetes + Istio + Knative + Helm + Spinnaker

23 of 55

What we’re gonna talk about

CI (and CD)
How Cloud Native makes things harder
But it’s worth it: Cloud Native CI

Serverless
Specs and standards
Infrastructure agnostic
Reusable components
Config as code

24 of 55

Cloud Native CI

(Definition by Christie)

Serverless
Specs and standards
Infrastructure agnostic
Reusable components
Config as code

25 of 55

What we’re gonna talk about

CI (and CD)
How Cloud Native makes things harder
But it’s worth it: Cloud Native CI

Serverless
Specs and standards
Infrastructure agnostic
Reusable components
Config as code

26 of 55

Serverless CI

Scaling up and down resources as needed
You specify what you want to run, freed of the much of responsibility to manage the underlying resources

27 of 55

Serverless CI

Don’t need to waste resources
No need for monolithic execution

28 of 55

What we’re gonna talk about

CI (and CD)
How Cloud Native makes things harder
But it’s worth it: Cloud Native CI

Serverless
Specs and standards
Infrastructure agnostic
Reusable components
Config as code

29 of 55

Specs & Standards

Doesn’t have to be perfect, just good enough & widely used

Part of what makes k8s so cool is that it’s a spec we can all agree on
Using this = composable, reusable, extensible

30 of 55

Specs & Standards

Infinite extensibility!

Any k8s resource can be manipulated (e.g. with controllers, admission webhooks)
Anything built on k8s can be manipulated with k8s tools

31 of 55

What we’re gonna talk about

CI (and CD)
How Cloud Native makes things harder
But it’s worth it: Cloud Native CI

Serverless
Specs and standards
Infrastructure agnostic
Reusable components
Config as code

32 of 55

Use your own infra

Infrastructure agnostic

A kubernetes deployment is a kubernetes deployment
If you can deploy to prod kube, it should be possible to deploy to your kube
Use the same config!

33 of 55

Use your own infra

Shift left

The sooner you fail the better!
Perform testing earlier in your lifecycle
Why not do it before the code is pushed!

34 of 55

Use your own infra

Parity

Make the complexity of kubernetes worth it!
Pre-kube, it was nearly impossible for me to setup a production like environment on my own machine

35 of 55

What we’re gonna talk about

CI (and CD)
How Cloud Native makes things harder
But it’s worth it: Cloud Native CI

Serverless
Specs and standards
Infrastructure agnostic
Reusable components
Config as code

36 of 55

Reusable components

(Kube types, CRDs…)

37 of 55

Reusable components

Do we need another Slack notification plugin?

Write it once, use it again and and again
Kubernetes is itself a building block, that enables the creation of more building blocks

38 of 55

What we’re gonna talk about

CI (and CD)
How Cloud Native makes things harder
But it’s worth it: Cloud Native CI

Serverless
Specs and standards
Infrastructure agnostic
Reusable components
Config as code

39 of 55

Config as Code

(even the yaml)

40 of 55

Learning & Debugging

Fail easily!

As our systems get more complicated, we need to be able to learn faster
We need to be able to look at what the system are doing

41 of 55

Debugging = Learning

42 of 55

The faster you can learn,

the faster you can deliver value

43 of 55

44 of 55

How do we debug?

By poking and prodding the thing
Looking at what it’s doing
Changing it, seeing what happens

How?
Look, read, understand
Investigate, make changes, play
Lightning talk = trash panda
Learn by digging through all the data available
= how i start
I use your project, docs then CI, Dockerfiles, scripts, all super valuable info on how to interact with it

And how do we debug?

We do it by looking at the thing, by reading it, by trying to understand it.

By investigating it.

By making little changes and seeing what happens.

By playing with it.

I once went to a lightning talk where the speaker advocated for being a trash panda

By which he meant learning by digging through all the data available to you.

This is the main way that I learn when I start on something new.

If I want to try to use your project, I immediately start looking for CI, for Dockerfiles, for scripts, for anything I can use to figure out how to actually use the thing myself.

45 of 55

Cloud Native CI

needs to make debugging easy

46 of 55

Config as Code

makes debugging easier

47 of 55

Software development is a practice,

not a performance

Software we think we should be making

Software we actually make

Nailed it!

48 of 55

To get better,

you need to practice

at the edge of your ability

49 of 55

If you’re at the edge,

sometimes you’re gonna fail

50 of 55

Failing is great!

Failing is part of succeeding!

51 of 55

Our CI should make failing easy

52 of 55

Tekton is Cloud Native CI

If you liked that, you’re gonna love Tekton!

(Unless you don’t like yaml…)

Tekton is a Cloud Native CI building block!
github.com/tektoncd/pipeline

53 of 55

54 of 55

Sources

Richard Montañez, inventor of flaming hot cheetos: www.cnbc.com/2018/03/27/a-janitor-invented-flamin-hot-cheetos-and-became-a-pepsico-exec.html
What is CI: www.thoughtworks.com/continuous-integration
What is serverless: martinfowler.com/articles/serverless.html
Building a Debugging Mindset: www.slideshare.net/InfoQ/building-a-debugging-mindset
Learning is the most important skill: twitter.com/ASpittel/status/1101165138361479169
Tekton: github.com/tektoncd/pipeline

55 of 55

Thanks!

Software we think we should be making

Software we actually make

Nailed it!