1 of 36

Using GoLang Concurrency & Asynchronous Processing with CI/CD

@sudhindraRao

2 of 36

Sudhindra Rao

  • 20 years in Software

  • Process Control Engineer for a few years

  • Rubyist since 2006

  • Golang Skeptic -> Embracer

  • Trainer

  • Developer of class from time to time

  • Current Status: Dev Manager @JFrog

(Uses Spaces not Tabs)

Follow me: @sudhindraRao

sudhindraRao.com

3 of 36

  • Today’s slides are posted
  • Video link will soon be posted
  • Rate My Talk
  • Enter the raffle to win 1 of 10 Go Shirts and Liquid Software Combos (winners will be selected in 3 business days & contacted by email)

https://jfrog.co/GoLangNYCJune

4 of 36

Concurrency and Memento the Movie

  • Events happen out of order
  • The main character has short term memory loss
  • He takes pictures and makes notes
  • Has a lot of tattoos to remind him of events
  • He is in search of the antagonist who caused him pain(John G.)
  • In the end the story comes together and when you unwind you start to understand what really happened/may have happened

(Although if you google about it, there are a few different takes on how the movie ended - very much like concurrent programs)

5 of 36

Twitter: @sudhindraRao

sudhindraRao.com

THANK YOU!

6 of 36

References

  • Memento the movie

Other references

  • Kubernetes
  • Pivotal Concourse

7 of 36

Success!

Build time reduced to from 6 hours -> 2 hours (400% improvement)

Error traceability for each binary

Fast failures means fast feedback

8 of 36

MAIN CHARACTERS

9 of 36

A Kubernetes platform

  • A complete Kubernetes Platform
  • Support for multiple operating systems (also Windows)
  • Support for many clouds
  • Day 2 operations support

10 of 36

  • bosh.io
  • Bosh Director can be the backbone of your cloud deployments
  • Given an OS(stemcell) - BOSH compiles against that stemcell
  • BOSH has cache

BOSH

11 of 36

Continuous Delivery system

  • Automated everything

  • Support for n, n-1, n-2 versions

  • 49 acceptance environments

12 of 36

THE PLOT

13 of 36

We are Running!

  • Build time - 6+ hours

  • Producing 1 tile - NOT one successful tile

  • Testing of this tile takes 70 hours minimum

  • A single failure is very expensive

  • Fix by - Rerun

14 of 36

Look - Find - Package

Precompiled

List of Dependencies

Target OS

Cloud Storage Bucket

BOSH cache

Compile

Combine to form installer tile

15 of 36

Identify concurrency - Who is John G.?

  • Network Latency

  • Cloud Latency

  • Compilation Latency

  • Resource Latency

  • OS specific Latency*

16 of 36

THE TOOLS

17 of 36

Concurrency in Go - A Primer

18 of 36

Goroutines

  • Concurrently executing
  • Similar to threads - But different
  • Much lighter memory footprint - and a growable stack
  • No identity to each routine - no thread-local storage
  • Cheaper to reschedule

19 of 36

Channels

  • Channel is a communication mechanism between goroutines
  • Unbuffered channels
  • Buffered channels

Do not communicate by sharing memory; instead, share memory by communicating.

20 of 36

Pipelines

  • The Go Programming Language - Page 227
  • Very powerful implementation by chaining operations with concurrency
  • No bookkeeping of threads
  • No bookkeeping of state
  • No ‘external’ bookkeeping for the system

21 of 36

What about WaitGroups and Synchronization

Don’t forget to call wg.Done()

Don’t trust, your weakness with threads

Synchronization with scattered booleans is a bad idea

Did you defer cleanup

  • Same issues as waiting for Threads

  • Same issues as maintaining Synchronization constructs

  • Missing a wg.Done() or a defer at an appropriate place leads to deadlock

22 of 36

Don’t forget the basics

  • Test Driven Development(TDD)

  • Malleable Code

  • Separation of Concerns - Identify responsibility

  • Encapsulation

23 of 36

THE NEW CODE

24 of 36

The Algorithm

25 of 36

Building the Pipeline

numberOfResultsExpected := len(tileDependencies.Releases)

releasesToDownload := make(chan CandidateRelease, numberOfResultsExpected)

releasesToCompile := make(chan CandidateRelease, numberOfResultsExpected)

releasesToPublish := make(chan CandidateRelease, numberOfResultsExpected)

results := make(chan CandidateRelease, numberOfResultsExpected)

for i := 0; i < a.maxConcurrentDownloaders; i++ {

go a.downloadWorker.Create(releasesToDownload, releasesToCompile, results)

}

for i := 0; i < a.maxConcurrentCompilers; i++ {

go a.compileWorker.Create(releasesToCompile, releasesToPublish, results)

}

for i := 0; i < a.maxConcurrentPublishers; i++ {

go a.publishWorker.Create(releasesToPublish, results)

}

26 of 36

Encapsulation

Release Candidate

  • Binary Details
  • Success/Failure
  • Embedded Result
    • Path to source
    • Path to artifact
    • Errors

27 of 36

Objects through the pipeline

type Result struct {

PathToDownloadedSourceTarball string

PathsToCompiledTarball map[string]string

PathToCompiledTarballsToPublish []string

Err error

}

type CandidateRelease struct {

Release tile.Release

Result Result

}

type GCS struct {

serviceKeyFilePath string

compiledReleaseBucket string

}

type StorageClient struct {

pathToCLI string

authToken string

}

type BoshClient struct {

pathToBoshCLI string

boshHost string

boshUsername string

boshPassword string

boshCACert string

boshAllProxy string

}

and a few more objects...

28 of 36

Termination Condition

for candidate := range results {

if candidate.Result.Err != nil {

allErrors = append(allErrors, candidate.Result.Err.Error())

}

numberOfResultProcessed++

if numberOfResultProcessed == numberOfResultsExpected {

break

}

}

close(releasesToCompile)

close(releasesToPublish)

close(results)

29 of 36

THE OLD CODE

30 of 36

Concurrency the old way

func (app App) processRelease(...) error {

app.downloader.Download(...)

...

for i := 0; i < workers; i++ {

go func(deps tile.Dependency, args ProgArgs) {

for {

select {

case release := <-releases:

err := app.processRelease(release, deps, args)

result := Result{

release: release,

err: err,

}

results <- result

case <-stop:

return

}

}

}(deps, args)

}

func (app App) Download(...) error {

app.downloader.Compile(...)

...

func (app App) Compile(...) error {

app.downloader.Publish(...)

...

func (app App) Publish(...) error {

app.downloader.CalculateSha(...)

...

31 of 36

Termination Condition

stopWorkers := func() {

for i := 0; i < workers; i++ {

stop <- true

}

close(stop)

}

for result := range results {

if result.err != nil {

errorResults = append(errorResults, result)

}

remaining--

if remaining == 0 {

stopWorkers()

close(results)

close(releases)

}

}

32 of 36

Old code - concurrency issues

  • Had Goroutines
  • Had a couple of channels for signaling state
  • Had sync mechanisms like waitGroups
  • Started breaking as soon as we introduced another OS

We found our ‘John G.’

33 of 36

What we fixed

  • Fixes scattered code which was difficult to read and parse
  • Addressed
    • latency of compilation - different for each binary and based on size and complexity
    • Network latency
    • Cache latency
    • Upload/Download latency from variety of web locations
  • Fixed the lack of single responsibility

34 of 36

MEASURING SUCCESS

1 OS

< 10 binaries

~ 40 environments

4 tiles

per

day

2+ OSes

> 20 binaries

> 100 environments

Predictable build 4 times a day

1 Tile per day

35 of 36

DON’T�BE AFRAID

Refactor

Go back to basics

Prioritize Tech Debt

Optimize at the right time

Ask for Support (Special mention - Kevin Kelani and Adrian Zankich)

36 of 36

Any Questions?

https://jfrog.co/GoLangNYCJune