HOW GOOD IS
OUR CODE?

Dan Kohn

Executive Director, CNCF

1

I’m gonna start with a few quick updates on our event and the Cloud Native Computing Foundation, which runs it.

Cloud Native Computing Foundation

  • Non-profit, part of the Linux Foundation; founded Dec 2015
  • Platinum members:

Incubating

Sandbox

Service Mesh

Storage

Service Discovery

Distributed Tracing

Software Update Spec

Storage

Security

Identity Spec

Identity

Policy

Graduated

Tooling

Package Management

Registry

Orchestration

Monitoring

Networking

API

Service Mesh

Logging

Remote Procedure Call

Distributed Tracing API

Container Runtime

Container Runtime

Metrics Spec

Messaging

Distributed K/V

Monitoring

Serverless

© 2018 Cloud Native Computing Foundation

2

TODAY THE LINUX FOUNDATION IS MUCH MORE THAN LINUX

3

We are helping global privacy and security through a program to encrypt the entire internet.

Security

Networking

We are creating ecosystems around networking to improve agility in the evolving software-defined datacenter.

Cloud

We are creating a portability layer for the cloud, driving de facto standards and developing the orchestration layer for all clouds.

Automotive

We are creating the platform for infotainment in the auto industry that can be expanded into instrument clusters and telematics systems.

Blockchain

We are creating a permanent, secure distributed ledger that makes it easier to create cost-efficient, decentralized business networks.

We are regularly adding projects; for the most up-to-date listing of all projects visit tlfprojects.org

Web

We are providing the application development framework for next generation web, mobile, serverless, and IoT applications.

3

KubeCon + CloudNativeCon

  • China
    • Shanghai: November 13-15, 2018
    • Sponsorships open
  • North America
    • Seattle: December 10-13, 2018
    • Sponsorships open
  • 2019
    • Barcelona: May 20-23, 2019
    • Shanghai: June 26-28, 2019
    • San Diego: November 18-21, 2019

Cloud Native Computing Foundation

4

KubeCon + CloudNativeCon Attendees

© 2018 Cloud Native Computing Foundation

5

HOW GOOD IS
OUR CODE?

6

7

7

A year ago we were in Berlin and this graffiti was the backdrop for the keynote stage

8

8

[Read. Emphasis:] Which of these is the most important? [pause]

Let’s watch a video that’s helped me understand my priorities.

9

After raising and burning through $120 M of venture capital, Juicero collapsed last September. The former CEO Doug Evans then went on a 10-day cleanse [pause], drinking nothing but Live Water. What is live water, you ask? “unfiltered, untreated, unsterilized spring water”

10

“I haven’t tasted tap water in a long time….” Doug Evans said. “You have to be agile and tactile, and be available to experiment. Literally, you have to carry bottles of water through the dark.”

10

So, let’s look at the “live water” of software development:

11

11

Let’s take this one step further

12

12

How many of us use SQLite? [pause for hands] Trick question: it’s built into iOS, Android, Chrome and Firefox, so we all do

OUR SOFTWARE IS NOT AS GOOD AS SQLITE

Developed mainly by one highly-regarded developer, Richard Hipp

100% branch test coverage

Millions of test cases

1,000 times as much test code as product code

https://www.sqlite.org/testing.html

13

Very little software has as positive a reputation as SQLite does. And then, American Fuzzy Lop came along

14

14

No, not this American Fuzzy Lop

AMERICAN FUZZY LOP

15

Software fuzzer built by Michał Zalewski that uses genetic algorithms to find bugs

15

SQLITE STILL HAS BUGS!

When Zalewski ran American Fuzzy Lop against SQLite, he found 22 bugs(!!!) in 30 minutes of work

Note that SQLite quickly fixed all of the bugs and incorporated AFL into their release process

But our code is not as good as SQLite’s!

16

And the problem is *bigger* than that.

HOW BIG IS OUR APP?

17

It’s not enough to just look at the surface area of the code that *we’ve* written, as our app is potentially vulnerable to all the software it depends on, and transitively to all the software that that software depends on as well. [pause] I did the calculations on an app that CNCF has been building called the interactive landscape:

LINUX

LINUX

17 M SLOCs

18

Let me give a shout out here to David Wheeler, the author of SLOCcount, which is the tool we used. [pause] Linux has 17 M source lines of code or SLOCs.

DEPLOYMENT PLATFORM

KUBERNETES
35 M SLOCs

19

Amazingly, K8s is now double Linux's SLOC count, at 35 M. Now for all of this software, we're not using every line of code for *our* installation, but *somebody* is for theirs.

FRAMEWORK

NODE.JS
12.3 M SLOCs

20

Node.js is over 70% of the size of Linux

3RD PARTY LIBRARIES

3RD PARTY LIBRARIES
2.5 M SLOCs

21

We depend on a set of third-party libraries like Webpack and Babel

OUR CODE IS ONLY 40 K SLOCS

OUR CODE


INTERACTIVE LANDSCAPE
40 K SLOCs

22

So, the NPM modules our code relies on are 63 times more code than what we wrote directly.

OUR APPLICATION SOFTWARE STACK

23

Here's a chart of those SLOC counts. Unfortunately, every part of our software stack is a potential vector for vulnerabilities. And, I left one part out of this pie chart.

OUR CODE IS <0.1% OF OUR SOFTWARE STACK

24

Our code is less than 0.1% of all the code we depend on

ALL OF THIS CODE IS VULNERABLE

NODE.JS
12.3 M SLOCs

KUBERNETES
35 M SLOCs

LINUX

17 M SLOCs

3RD PARTY LIBRARIES
2.5 M SLOCs

OUR CODE
40 K SLOCs

25

We're not the only ones working with this software stack. Unfortunately, there are black hats looking for vulnerabilities in every part of our stack, whether or not it is open source

SPECTRE & MELTDOWN IN LINUX

DivideConcept

NODE.JS
12.3 M SLOCs

3RD PARTY LIBRARIES
2.5 M SLOCs

OUR CODE
40 K SLOCs

KUBERNETES
35 M SLOCs

LINUX

17 M SLOCs

26

These are just a few examples

DivideConcept in Webpack

SPECTRE & MELTDOWN IN LINUX

Heartbleed

DivideConcept

NODE.JS
12.3 M SLOCs

3RD PARTY LIBRARIES
2.5 M SLOCs

OUR CODE
40 K SLOCs

KUBERNETES
35 M SLOCs

LINUX

17 M SLOCs

27

Heartbleed in Node.js, via its use of OpenSSL

SPECTRE & MELTDOWN IN LINUX

Heartbleed

subPath

DivideConcept

NODE.JS
12.3 M SLOCs

3RD PARTY LIBRARIES
2.5 M SLOCs

OUR CODE
40 K SLOCs

KUBERNETES
35 M SLOCs

LINUX

17 M SLOCs

28

subPath in Kubernetes

SPECTRE & MELTDOWN IN LINUX

Spectre and Meltdown

Heartbleed

subPath

DivideConcept

NODE.JS
12.3 M SLOCs

3RD PARTY LIBRARIES
2.5 M SLOCs

OUR CODE
40 K SLOCs

KUBERNETES
35 M SLOCs

LINUX

17 M SLOCs

29

Spectre and Meltdown in Linux. Now, stable kernel maintainer Greg KH… underlying hardware. Linux’s job to ameliorate the issues.

The power of open source is the ability to leverage thousands of other developers that are finding bugs and making fixes to the software we depend on

30

But a software patch does not help until we have deployed it into production

31

How can we have the confidence that that deployment won’t break anything?

32

The Answer is
CONTINUOUS INTEGRATION (CI)

But a software patch does not help until we have deployed it into production

How can we have the confidence that that deployment won’t break anything?

The power of open source is the ability to leverage thousands of other developers that are finding bugs and making fixes to the software we depend on

33

Continuous Integration is simply running our tests every time we change our software.

WHAT KIND
OF TESTS SHOULD CI RUN?

34

WHAT KIND OF TESTS SHOULD CI RUN?

Unit testing of individual portions of our source code in isolation?

35

WHAT KIND OF TESTS SHOULD CI RUN?

Integration testing, where we work with external systems like a database?

Unit testing of individual portions of our source code in isolation?

36

WHAT KIND OF TESTS SHOULD CI RUN?

Regression testing, where we add a test after a failure

Unit testing of individual portions of our source code in isolation?

Integration testing, where we work with external systems like a database?

37

WHAT KIND OF TESTS SHOULD CI RUN?

Smoke testing,
also known as build verification testing?

Unit testing of individual portions of our source code in isolation?

Integration testing, where we work with external systems like a database?

Regression testing, where we add a test after a failure?

38

What kind of tests should our CI system run?

WHAT KIND OF TESTS SHOULD CI RUN?

Unit testing of individual portions of our source code in isolation?

All of the above.

Integration testing, where we work with external systems like a database?

Regression testing, where we add a test after a failure?

Smoke testing,
also known as build verification testing?

39

And, in fact, there are others kinds of testing as well, such as the fuzzing I mentioned earlier.

But we can get started with *just* a smoke test.

That’s the idea of turning on a machine, and seeing whether smoke comes out.

40

40

No matter how antiquated the code base we’ve inherited, we can write some smoke tests that confirm that at least “the happy path” is functioning correctly. And then, we can run those smoke tests on every commit.

HOW GOOD IS OUR CODE?

41

So we return to our original questions. First:

HOW GOOD IS OUR CODE?

NOT GOOD ENOUGH

We need to build in the systems and processes that enable us to continuously improve it

42

43

43

And second: which of these is the most important? My answer is that a different part of the cloud native architecture is the most valuable: continuous integration: CI

44

44

Or, to quote Kelsey Hightower. [Read] Let’s take this one step further

CONTINUOUS INTEGRATION (CI)

But what is testing?

Continuous integration (CI) just means constant testing

45

46

46

Testing is like science

TESTING IS LIKE SCIENCE

We have a hypothesis
of what we believe our
code should do, but we don’t know for sure until we
test against objective reality

47

Karl Popper
defined science as

BEING TESTABLE AND FALSIFIABLE

48

48

What do Continuous Integration, Science, and Entrepreneurship all have in common?

49

49

What do Continuous Integration, Science, and Entrepreneurship all have in common?

They each require comparing an idealized conception to the often brutal truth of objective reality

50

50

Which brings us full circle to Live Water, and back to Doug Evans and Juicero. Because no matter what you believe, no scientific test is going to demonstrate any validity to the health claims of “live water”. And even if you succeed in raising $120 M, your company will fail if customers feel defrauded by your product. And no matter how great we might think our code seems to be, if it doesn’t pass a smoke test, we can’t deploy it. [pause]

HOW DOES CONTINUOUS INTEGRATION FIT INTO THE CLOUD NATIVE JOURNEY?

51

51

If CI is the most important step, where does it fit in the cloud native journey?

Cloud Native Trail Map

Trail Map: l.cncf.io

© 2018 Cloud Native Computing Foundation

52

It comes 2nd. [pause] Even though it generates the most value, we recommend that enterprises first containerize their apps, as containerized CI is both easier and more future-proof than doing CI before containerization. This, BTW, is the new cloud native trail map that is available on our website and as a printed handout at our at our booth. And what CI does CNCF recommend?

© 2018 Cloud Native Computing Foundation

53

© 2018 Cloud Native Computing Foundation

54

54

And we can see some of the options. But to conclude, let me show you a quick demo of the interactive landscape application I was describing before. It is designed to let you explore the cloud native ecosystem in depth. And please feel free to try it yourself right now on your phone.

PLEASE TRY THE INTERACTIVE LANDSCAPE NOW:

l.cncf.io

55

55

I hope you find it useful as you continue your exploration of the cloud native ecosystem.

How Good Is Our Code? - Google Slides