1 of 142

Coroutines for Microservices

_

Eugene Petrenko

eugene.petrenko@jetbrains.com

@jonnyzzz

JetBrains, @jonnyzzz, Oracle CodeOne 2018

2 of 142

Coroutines

  • Invented in 1960s by� Donald Knuth, Melvin Conway
  • suspend and resume functions

JetBrains, @jonnyzzz, Oracle CodeOne 2018

3 of 142

Blocking Code

JetBrains, @jonnyzzz, Oracle CodeOne 2018

4 of 142

JetBrains, @jonnyzzz, Oracle CodeOne 2018

5 of 142

JetBrains, @jonnyzzz, Oracle CodeOne 2018

6 of 142

Callback Code

JetBrains, @jonnyzzz, Oracle CodeOne 2018

7 of 142

Magic = Coroutines

JetBrains, @jonnyzzz, Oracle CodeOne 2018

8 of 142

Coroutines on JVM

JetBrains, @jonnyzzz, Oracle CodeOne 2018

9 of 142

Project Loom

  • Work in progress
  • Threads are OS-Bound
    • Fibers are lightweight
  • Asynchronous API is hard
    • But the only way to go today

JetBrains, @jonnyzzz, Oracle CodeOne 2018

10 of 142

Coroutines Support

  • JVM Bytecode transformation experiments (3+ projects)
  • Languages
    • Scala, JavaScript, C#, Python, Ruby, Rust, Kotlin,...

JetBrains, @jonnyzzz, Oracle CodeOne 2018

11 of 142

Kotlin 1.3

  • Static typed
  • JVM & Multiplatform
  • Supports coroutines
  • Apache 2.0

JetBrains, @jonnyzzz, Oracle CodeOne 2018

12 of 142

Kotlin Platforms

JetBrains, @jonnyzzz, Oracle CodeOne 2018

13 of 142

Kotlin / Native ❤ LLVM

  • iOS, Android
  • Windows, Linux, Mac
    • Swift, objective-c, Framework
  • WebAssembly

JetBrains, @jonnyzzz, Oracle CodeOne 2018

14 of 142

Coroutines in Kotlin

JetBrains, @jonnyzzz, Oracle CodeOne 2018

15 of 142

Rules

  • suspend fun
    • call suspend fun - OK
    • call fun - OK
  • fun
    • call suspend fun - FAIL
    • call fun - OK

JetBrains, @jonnyzzz, Oracle CodeOne 2018

16 of 142

suspend fun

  • The only keyword in Kotlin for coroutines
  • Few API calls in stdlib
  • The rest done in libraries
    • kotlinx.coroutines
    • <your library>

JetBrains, @jonnyzzz, Oracle CodeOne 2018

17 of 142

Generators

  • Let the kotlinc create an iterator
  • Generator is suspended on yield
  • No need to write state machine

​

JetBrains, @jonnyzzz, Oracle CodeOne 2018

18 of 142

Fibonacci Sequence

JetBrains, @jonnyzzz, Oracle CodeOne 2018

19 of 142

Yield fun?

JetBrains, @jonnyzzz, Oracle CodeOne 2018

20 of 142

sequence = starts coroutine

JetBrains, @jonnyzzz, Oracle CodeOne 2018

21 of 142

suspend

  • suspend lambda fun!

​

  • suspend fun only from suspend fun

​

JetBrains, @jonnyzzz, Oracle CodeOne 2018

22 of 142

How does it work?

JetBrains, @jonnyzzz, Oracle CodeOne 2018

23 of 142

Prints

JetBrains, @jonnyzzz, Oracle CodeOne 2018

24 of 142

Coroutines ❤ Libraries

  • suspend keyword in language
  • Stdlib minimalistic support
  • Libraries for the rest
    • kotlinx.coroutines - on GitHub
    • <make your library>

JetBrains, @jonnyzzz, Oracle CodeOne 2018

25 of 142

kotlinx.coroutines

  • Apache 2.0, on GitHub
  • Multiplatform support for coroutines
  • Let’s use it!

JetBrains, @jonnyzzz, Oracle CodeOne 2018

26 of 142

Remember?

JetBrains, @jonnyzzz, Oracle CodeOne 2018

27 of 142

open fun

JetBrains, @jonnyzzz, Oracle CodeOne 2018

28 of 142

open fun: 1 - start

JetBrains, @jonnyzzz, Oracle CodeOne 2018

29 of 142

open fun: 2 - suspend

JetBrains, @jonnyzzz, Oracle CodeOne 2018

30 of 142

open fun: 3 - subscribe

JetBrains, @jonnyzzz, Oracle CodeOne 2018

31 of 142

open fun: 4 - resume

JetBrains, @jonnyzzz, Oracle CodeOne 2018

32 of 142

open fun: 4’ - fail

JetBrains, @jonnyzzz, Oracle CodeOne 2018

33 of 142

Open fun: 5 - resume

JetBrains, @jonnyzzz, Oracle CodeOne 2018

34 of 142

Microservices?

12.5

JetBrains, @jonnyzzz, Oracle CodeOne 2018

35 of 142

Microservices

  • Not a monolith apps ;)
  • Easy to
    • deploy, scale
    • debug, profile
  • Uses Network actively

JetBrains, @jonnyzzz, Oracle CodeOne 2018

36 of 142

Performance?

JetBrains, @jonnyzzz, Oracle CodeOne 2018

37 of 142

JetBrains, @jonnyzzz, Oracle CodeOne 2018

38 of 142

Perf Math

At 100 instances

20% better => 20 machines less!

$0.096/hour * 20 = $16,819/year

JetBrains, @jonnyzzz, Oracle CodeOne 2018

39 of 142

How we do Microservices?

JetBrains, @jonnyzzz, Oracle CodeOne 2018

40 of 142

Approaches

  • Blocking
    • Threads
  • Non-Blocking
    • Callbacks
    • Promisses

JetBrains, @jonnyzzz, Oracle CodeOne 2018

41 of 142

Example Service

The Service

Service A

Service C

Service B

JetBrains, @jonnyzzz, Oracle CodeOne 2018

42 of 142

Blocking

JetBrains, @jonnyzzz, Oracle CodeOne 2018

43 of 142

Blocking Service

JetBrains, @jonnyzzz, Oracle CodeOne 2018

44 of 142

Thread per Request

  • Easier write code & debug it
    • Motivation of Project Loom
  • Limited scalability
    • Blocking waits
    • Synchronization / Shared state

JetBrains, @jonnyzzz, Oracle CodeOne 2018

45 of 142

How many threads �is possible?

18

JetBrains, @jonnyzzz, Oracle CodeOne 2018

46 of 142

Starting Threads

JetBrains, @jonnyzzz, Oracle CodeOne 2018

47 of 142

macOS / Linux

JetBrains, @jonnyzzz, Oracle CodeOne 2018

48 of 142

macOS / Linux

  • macOS
    • 4k threads
  • Linux
    • 12k threads
  • May require a special OS tuning

JetBrains, @jonnyzzz, Oracle CodeOne 2018

49 of 142

Windows

JetBrains, @jonnyzzz, Oracle CodeOne 2018

50 of 142

Windows 10, JVM 1.8x64

  • 450.000 threads & 37 Gb RAM
    • 2 hours to start
  • 780.000 threads/day
    • 65Gb RAM

JetBrains, @jonnyzzz, Oracle CodeOne 2018

51 of 142

Too many Threads

  • Possible?
    • Linux/Mac - not really
    • Windows - possible
  • Starvation
    • Fair locking - slow

JetBrains, @jonnyzzz, Oracle CodeOne 2018

52 of 142

Too many Threads?

  • Start more machines!
    • Microservices are good to scale
  • Use callback driven programming
    • Let the OS notify / less threads
  • Use Coroutines / project Loom

JetBrains, @jonnyzzz, Oracle CodeOne 2018

53 of 142

Starting Coroutines

JetBrains, @jonnyzzz, Oracle CodeOne 2018

54 of 142

Coroutines

  • Lightweight
    • 1_000_000 is OK to run
  • Not threads
    • uses threads to run
    • suspend and resumes execution

JetBrains, @jonnyzzz, Oracle CodeOne 2018

55 of 142

Threads vs Coroutines

  • Threads
    • Can be interrupted at any moment
      • Synchronization needed
  • Coroutines
    • Interrupted at suspension points only
      • Controlled interrupts, easier

JetBrains, @jonnyzzz, Oracle CodeOne 2018

56 of 142

Non-Blocking

25

JetBrains, @jonnyzzz, Oracle CodeOne 2018

57 of 142

Callbacks

  • Continuation passing style
  • Callbacks / Events / NIO

​

JetBrains, @jonnyzzz, Oracle CodeOne 2018

58 of 142

Callbacks or CPS

  • Continuation passing style
  • Callbacks / Events / NIO

JetBrains, @jonnyzzz, Oracle CodeOne 2018

59 of 142

Callbacks / NIO

  • Non-Blocking
    • thread != request
  • Callbacks are harder to code
    • how to wait for N callbacks?

JetBrains, @jonnyzzz, Oracle CodeOne 2018

60 of 142

Example Service

The Service

Service A

Service C

Service B

JetBrains, @jonnyzzz, Oracle CodeOne 2018

61 of 142

We had Blocking code

JetBrains, @jonnyzzz, Oracle CodeOne 2018

62 of 142

Callbacks: a dream

JetBrains, @jonnyzzz, Oracle CodeOne 2018

63 of 142

Callbacks: a reality

  • Error handling
  • Cancellation
  • Steaming results

=> 2-3 methods or params per fun

JetBrains, @jonnyzzz, Oracle CodeOne 2018

64 of 142

Callbacks: a reality

JetBrains, @jonnyzzz, Oracle CodeOne 2018

65 of 142

Callbacks: a reality

JetBrains, @jonnyzzz, Oracle CodeOne 2018

66 of 142

Callbacks: a reality

JetBrains, @jonnyzzz, Oracle CodeOne 2018

67 of 142

: (

JetBrains, @jonnyzzz, Oracle CodeOne 2018

68 of 142

Non-Blocking�with Libraries

28

JetBrains, @jonnyzzz, Oracle CodeOne 2018

69 of 142

Libraries for Callbacks

  • Promises / CompletableFuture
  • Reactive extensions or Rx
  • <name your lib>

​

JetBrains, @jonnyzzz, Oracle CodeOne 2018

70 of 142

Promises

  • Define the chain of operations
  • Execute it
    • later
    • different threads
    • callback the result

JetBrains, @jonnyzzz, Oracle CodeOne 2018

71 of 142

Observer in Rx

Implements those methods:

  • onNext(T)
  • onError(Throwable)
  • onCompleted()

Use all possible Rx operators

JetBrains, @jonnyzzz, Oracle CodeOne 2018

72 of 142

Rx Observable

JetBrains, @jonnyzzz, Oracle CodeOne 2018

73 of 142

Rx Operators

JetBrains, @jonnyzzz, Oracle CodeOne 2018

74 of 142

Reactive Programming

  • New Language in Language
    • learn operators
  • Define pipeline
  • Execute!

JetBrains, @jonnyzzz, Oracle CodeOne 2018

75 of 142

Non-Blocking�with Coroutines

JetBrains, @jonnyzzz, Oracle CodeOne 2018

76 of 142

Kotlin Coroutines

JetBrains, @jonnyzzz, Oracle CodeOne 2018

77 of 142

Coroutines

  • Linear code
    • No need to learn new things
    • if , for, while, try/catch/, ...
  • Executes asynchronously
  • Easy to read and support

JetBrains, @jonnyzzz, Oracle CodeOne 2018

78 of 142

How does �suspend functions�work?

JetBrains, @jonnyzzz, Oracle CodeOne 2018

79 of 142

suspend function

JetBrains, @jonnyzzz, Oracle CodeOne 2018

80 of 142

Compiled to Callback fun

JetBrains, @jonnyzzz, Oracle CodeOne 2018

81 of 142

The Callback

JetBrains, @jonnyzzz, Oracle CodeOne 2018

82 of 142

Coroutines are callbacks

JetBrains, @jonnyzzz, Oracle CodeOne 2018

83 of 142

Coroutines are callbacks

JetBrains, @jonnyzzz, Oracle CodeOne 2018

84 of 142

Coroutine State Machine

JetBrains, @jonnyzzz, Oracle CodeOne 2018

85 of 142

Contemporary Syntax

JetBrains, @jonnyzzz, Oracle CodeOne 2018

86 of 142

Non-Blocking �with �Language Support

32

JetBrains, @jonnyzzz, Oracle CodeOne 2018

87 of 142

Language Support

  • C#, Python, JavaScript
    • generators or yield return
    • async/await�

Same concept, many impls

​

JetBrains, @jonnyzzz, Oracle CodeOne 2018

88 of 142

Kotlin Coroutines

  • One implementation in kotlinc
    • “suspend fun” support
  • Libraries to provide features
    • kotlinx.coroutines
      • async/await, generators, actors,...

JetBrains, @jonnyzzz, Oracle CodeOne 2018

89 of 142

async / await

  • Offload execution to another coroutine or thread
  • Wait for it without blocking
    • Let current thread do more work

JetBrains, @jonnyzzz, Oracle CodeOne 2018

90 of 142

async/await

JetBrains, @jonnyzzz, Oracle CodeOne 2018

91 of 142

suspend execution

JetBrains, @jonnyzzz, Oracle CodeOne 2018

92 of 142

Using OkHttp & await()

JetBrains, @jonnyzzz, Oracle CodeOne 2018

93 of 142

Suspend & OkHttp

JetBrains, @jonnyzzz, Oracle CodeOne 2018

94 of 142

Coroutines & GRPC

JetBrains, @jonnyzzz, Oracle CodeOne 2018

95 of 142

Use suspend functions

  • easy to integrate with existing
  • kotlinx.coroutines support
    • CompletableFuture, Guava, Rx, ...

JetBrains, @jonnyzzz, Oracle CodeOne 2018

96 of 142

Shared Mutable�State

35

JetBrains, @jonnyzzz, Oracle CodeOne 2018

97 of 142

Shared + Mutable = Danger

  • Good or Hard?
  • Isolate state in coroutines
  • Use Channel<T>
    • from kotlinx.coroutines library

JetBrains, @jonnyzzz, Oracle CodeOne 2018

98 of 142

Channel<T>

SendChannel<T>

ReceiveChannel<T>

JetBrains, @jonnyzzz, Oracle CodeOne 2018

99 of 142

Isolate State

Actor

Supplier A

Supplier C

Supplier B

STATE

CHANNEL

JetBrains, @jonnyzzz, Oracle CodeOne 2018

100 of 142

Actor

JetBrains, @jonnyzzz, Oracle CodeOne 2018

101 of 142

Using Actor

JetBrains, @jonnyzzz, Oracle CodeOne 2018

102 of 142

Workers

Worker A

Worker C

Worker B

CHANNEL

CHANNEL

JetBrains, @jonnyzzz, Oracle CodeOne 2018

103 of 142

JetBrains, @jonnyzzz, Oracle CodeOne 2018

104 of 142

JetBrains, @jonnyzzz, Oracle CodeOne 2018

105 of 142

JetBrains, @jonnyzzz, Oracle CodeOne 2018

106 of 142

Channels

  • Communication between coroutines
  • Avoid shared mutable state

JetBrains, @jonnyzzz, Oracle CodeOne 2018

107 of 142

Select expression

JetBrains, @jonnyzzz, Oracle CodeOne 2018

108 of 142

Kotlin Coroutines

40

JetBrains, @jonnyzzz, Oracle CodeOne 2018

109 of 142

Coroutines

  • suspend keyword for the language
  • libraries to implement features
    • async/await, channels, actors
    • <your library here>

JetBrains, @jonnyzzz, Oracle CodeOne 2018

110 of 142

kotlinx.coroutines �library�on GitHub

JetBrains, @jonnyzzz, Oracle CodeOne 2018

111 of 142

Not only for JVM

JetBrains, @jonnyzzz, Oracle CodeOne 2018

112 of 142

Try Kotlin Coroutines �for your code

JetBrains, @jonnyzzz, Oracle CodeOne 2018

113 of 142

Try Kotlin Coroutines �for microservices

JetBrains, @jonnyzzz, Oracle CodeOne 2018

114 of 142

https://unsplash.com/photos/ukzHlkoz1IE

JetBrains, @jonnyzzz, Oracle CodeOne 2018

115 of 142

One more thing

  • Kotlin 1.3.0 RC is out
  • Stable Coroutines
    • NOT experimental any more
  • kotlinx.coroutines library 1.0.0 RC
  • … and more!

JetBrains, @jonnyzzz, Oracle CodeOne 2018

116 of 142

Learn and Try

Kotlin Coroutines in Practice �by Roman Elizarov

​

https://www.youtube.com/watch?v=a3agLJQ6vt8

JetBrains, @jonnyzzz, Oracle CodeOne 2018

117 of 142

Thank you

Q&A

@jonnyzzz

JetBrains, @jonnyzzz, Oracle CodeOne 2018

118 of 142

Microservices

what we care about™

JetBrains, @jonnyzzz, Oracle CodeOne 2018

119 of 142

JetBrains, @jonnyzzz, Oracle CodeOne 2018

120 of 142

Kotlin Platforms

JetBrains, @jonnyzzz, Oracle CodeOne 2018

121 of 142

JetBrains, @jonnyzzz, Oracle CodeOne 2018

122 of 142

Structured Concurrency

coroutineScope { … }

  • Ensures no coroutine leaks
  • Any error
    • is propagated
    • terminates the execution

JetBrains, @jonnyzzz, Oracle CodeOne 2018

123 of 142

async / await

JetBrains, @jonnyzzz, Oracle CodeOne 2018

124 of 142

async / await

  • Supported in kotlinx.coroutines
    • JDK 8, Guava, and more Futures
    • Integrate into existing code easily
  • Code looks linear
  • Errors & no coroutine leaks

JetBrains, @jonnyzzz, Oracle CodeOne 2018

125 of 142

JetBrains, @jonnyzzz, Oracle CodeOne 2018

126 of 142

State Machine

JetBrains, @jonnyzzz, Oracle CodeOne 2018

127 of 142

kotlinx.coroutines Library

JetBrains, @jonnyzzz, Oracle CodeOne 2018

128 of 142

Structured Concurrency

JetBrains, @jonnyzzz, Oracle CodeOne 2018

129 of 142

Workers

JetBrains, @jonnyzzz, Oracle CodeOne 2018

130 of 142

Channels

JetBrains, @jonnyzzz, Oracle CodeOne 2018

131 of 142

Select

JetBrains, @jonnyzzz, Oracle CodeOne 2018

132 of 142

Suspend

  • Execution i
  • ​

JetBrains, @jonnyzzz, Oracle CodeOne 2018

133 of 142

{ }

JetBrains, @jonnyzzz, Oracle CodeOne 2018

134 of 142

JetBrains, @jonnyzzz, Oracle CodeOne 2018

135 of 142

suspend functions

  • Execution is paused in the middle
    • Thread is free to do other tasks
    • suspend*Coroutine
  • Execution is resumed
    • When something happened

JetBrains, @jonnyzzz, Oracle CodeOne 2018

136 of 142

Using Coroutines

JetBrains, @jonnyzzz, Oracle CodeOne 2018

137 of 142

JetBrains, @jonnyzzz, Oracle CodeOne 2018

138 of 142

JetBrains, @jonnyzzz, Oracle CodeOne 2018

139 of 142

JetBrains, @jonnyzzz, Oracle CodeOne 2018

140 of 142

http://thecodebarbarian.com/2015/03/20/callback-hell-is-a-myth

JetBrains, @jonnyzzz, Oracle CodeOne 2018

141 of 142

http://geekycentral.com/writing-a-recursive-function-using-promises-nodejs/

JetBrains, @jonnyzzz, Oracle CodeOne 2018

142 of 142

JetBrains, @jonnyzzz, Oracle CodeOne 2018