1 of 32

The Many Faces of Concurrency in Python

Sagiv Malihi

Paradigms and tools for building high-performing systems

smalihi@cisco.com

@sagiv

1

2 of 32

Who Am I?

2

3 of 32

Sagiv Malihi
SW Architect – SON Group
@sagiv
smalihi at cisco

3

4 of 32

What are We Building?

4

5 of 32

Self Optimizing (cellular) Networks

Connect to all antennas to constantly make adjustments
Read & analyze tons of statistics
Synchronize several physical locations

5

6 of 32

the system had to grow

Pelephone as 1^st customer
followed by AT&T (!)
now already in tens of operators

6

7 of 32

We Needed to Scale!

7

8 of 32

Concurrency

vs.

Parallelism

vs.

Distributed System

8

9 of 32

Concurrency – running multiple tasks in overlapping time periods

9

10 of 32

Parallelism – when multiple tasks actually take place at the same time (e.g. on separate cores)

10

11 of 32

Distributed Systems – execute tasks in parallel over several machines (in different locations)

11

12 of 32

Concurrency

12

13 of 32

Connecting to thousands of cell towers

Continuously tweaking tilt, coverage, handovers…

13

14 of 32

threading is not a good choice

shared mem + switching = races
the GIL prevents true parallelism
threads are resource-intensive

14

15 of 32

threading does have an upside

using threads is easy
IO is still concurrent
c extensions can release the GIL
IronPython / Jython are GIL-less

15

16 of 32

coroutines to the rescue!

Predictable
Lightweight
Many libraries (incl. asyncio in stdlib)

16

17 of 32

the basic idea is simple

def task1():

s = socket(...)

while True:

yield socket

print socket.read()

def task2():

i = 1

while True:

yield Sleep(1)

print i

i += 1

def eventloop(*tasks):

tasks = {task.next(): task

for task in tasks}

while True:

sockets, sleeps =

filter_tasks(tasks)

ready = select(sockets,

min(sleeps))

tasks = call_task_next(tasks,

ready)

eventloop(task1(), task2())

17

18 of 32

python 3.5 async/await api

import asyncio

async def slow_func():

await asyncio.sleep(1)

return “answer”

async def failed_func():

await asyncio.sleep(1)

raise Exception(...)

async def test():

response = slow_func()

try:

await failed_func()

except Exception as e:

print(e, await response)

loop = asyncio.get_event_loop()

loop.run_until_complete(test())

18

19 of 32

gevent’s magic is a good tradeoff

from gevent import monkey monkey.patch_all()

19

20 of 32

Parallelism

20

21 of 32

Hundreds of GB of binary logs & statistic files

are being parsed & processed every minute

21

22 of 32

multiprocessing is like magic!

from multiprocessing import

Process, Pipe

def f(conn):

conn.send(“hello world”)

conn.close()

parent_conn, child_conn = Pipe()

p = Process(target=f,

args=(child_conn,))

p.start()

print(parent_conn.recv())

22

23 of 32

magic is not always a good thing

multiprocessing fork()’s
does not play well with gevent
or threads
or large datasets in memory
but fixed in python 3.4 !

23

24 of 32

using subprocess is easier

from slaveprocess // uses subprocess + RPyC

import run_in_process

def f():

return (“hello world”)

print (run_in_process(f))

24

25 of 32

Distributed Systems

25

26 of 32

Almost 200 servers

in 10 physical locations

working as a unified cluster

26

27 of 32

distributed DB can really help

keeps a single-point-of-truth
on all servers
can act as a communications channel
we used mongodb

27

28 of 32

try to avoid locking

locks generally lead to deadlocks
optimistic transaction model
nodes change the ‘network image’
verifying consistency before ‘commit’

28

29 of 32

Summary

29

30 of 32

IO bound apps -

avoid threads, consider gevent
or asyncio

CPU bound apps -

subprocess + RPyC

distributed apps -

let a DB do the hard work

30

31 of 32

Thank You!

Questions?

31

32 of 32

32