to GIL or not to GIL:
the Future of Multi-Core (C)Python
Who Am I?
2 / 112
Who Am I?
3
Overview
4
Context
5
An Overview of CPython’s Architecture
6
What Happens When Python Runs?
7
process
env vars
signals
...
What Happens When Python Runs?
8
process
env vars
signals
…
“data”
“heap”
thread (“main”)
TSS (TLS)
“stack”
What Happens When Python Runs?
9
process
CPython runtime
env vars
signals
…
“data”
“heap”
thread (“main”)
TSS (TLS)
“stack”
config
PyMem
...
What Happens When Python Runs?
10
process
env vars
signals
…
“data”
“heap”
thread (“main”)
TSS (TLS)
“stack”
CPython runtime
config
PyMem
...
Interpreter (“main”)
sys
sys.modules
...
What Happens When Python Runs?
11
process
env vars
signals
…
“data”
“heap”
thread (“main”)
TSS (TLS)
“stack”
CPython runtime
config
PyMem
...
interpreter (“main”)
sys
sys.modules
...
Python thread
What Happens When Python Runs?
12
process
env vars
signals
…
“data”
“heap”
thread (“main”)
TSS (TLS)
“stack”
CPython runtime
config
PyMem
...
interpreter (“main”)
sys
sys.modules
...
Python thread
code object A
“bytecode”
10 …
20 …
30 …
40 …
What Happens When Python Runs?
13
process
env vars
signals
…
“data”
“heap”
thread (“main”)
TSS (TLS)
“stack”
CPython runtime
config
PyMem
...
interpreter (“main”)
sys
sys.modules
...
Python thread
code object A
“bytecode”
10 …
20 …
30 …
40 …
What Happens When Python Runs?
14
code object A
“bytecode”
10 …
20 …
30 …
40 …
process
env vars
signals
…
“data”
“heap”
thread (“main”)
TSS (TLS)
“stack”
CPython runtime
config
PyMem
...
interpreter (“main”)
sys
sys.modules
...
Python thread
frame
What Happens When Python Runs?
15
process
env vars
signals
…
“data”
“heap”
thread (“main”)
TSS (TLS)
“stack”
CPython runtime
config
PyMem
...
interpreter (“main”)
sys
sys.modules
...
Python thread
frame
eval loop
code object A
“bytecode”
10 …
20 …
30 …
40 …
The Eval Loop
<set up>
for instruction in code object:
<maybe side-channel stuff>
<occasionally release & re-acquire the GIL>
<execute next instruction>
16
What Happens When Python Runs?
17
process
env vars
signals
…
“data”
“heap”
thread (“main”)
TSS (TLS)
“stack”
CPython runtime
config
PyMem
...
interpreter (“main”)
sys
sys.modules
...
Python thread
frame
eval loop
code object A
“bytecode”
10 …
20 … # call
30 …
40 …
What Happens When Python Runs?
18
process
env vars
signals
…
“data”
“heap”
thread (“main”)
TSS (TLS)
“stack”
CPython runtime
config
PyMem
...
interpreter (“main”)
sys
sys.modules
...
Python thread
frame
eval loop
code object B
“bytecode”
10 …
20 …
frame
What Happens When Python Runs?
19
process
env vars
signals
…
“data”
“heap”
thread (“main”)
TSS (TLS)
“stack”
CPython runtime
config
PyMem
...
interpreter (“main”)
sys
sys.modules
...
Python thread
frame
eval loop
code object B
“bytecode”
10 …
20 …
frame
eval loop
What Happens When Python Runs?
20
process
env vars
signals
…
“data”
“heap”
thread (“main”)
TSS (TLS)
“stack”
CPython runtime
config
PyMem
...
interpreter (“main”)
sys
sys.modules
...
Python thread
frame
eval loop
code object B
“bytecode”
10 …
20 …
frame
eval loop
What Happens When Python Runs?
21
process
env vars
signals
…
“data”
“heap”
thread (“main”)
TSS (TLS)
“stack”
CPython runtime
config
PyMem
...
interpreter (“main”)
sys
sys.modules
...
Python thread
frame
eval loop
code object A
“bytecode”
10 …
20 …
30 …
40 …
What Happens When Python Runs?
22
process
env vars
signals
…
“data”
“heap”
thread (“main”)
TSS (TLS)
“stack”
CPython runtime
config
PyMem
...
interpreter (“main”)
sys
sys.modules
...
Python thread
frame
eval loop
code object A
“bytecode”
10 …
20 …
30 …
40 …
What Happens When Python Runs?
23
process
env vars
signals
…
“data”
“heap”
thread (“main”)
TSS (TLS)
“stack”
CPython runtime
config
PyMem
...
interpreter (“main”)
sys
sys.modules
...
Python thread
frame
code object A
“bytecode”
10 …
20 …
30 …
40 …
What Happens When Python Runs?
24
process
env vars
signals
…
“data”
“heap”
thread (“main”)
TSS (TLS)
“stack”
CPython runtime
config
PyMem
...
interpreter (“main”)
sys
sys.modules
...
Python thread
frame
eval loop
“bytecode”
A10 …
A20 …
B10 …
B20 ...
A30 …
A40 …
Multi-threading!
25
__main__
A10 …
A20 …
B10 …
B20 ...
A30 …
A40 …
def spam():
…
t = threading.Thread(target=spam)
t.start()
…
t.join()
spam()
C10 …
C20 …
Multi-threading!
26
process
env vars
signals
…
“data”
“heap”
thread (“main”)
TSS (TLS)
“stack”
CPython runtime
config
PyMem
...
interpreter (“main”)
sys
sys.modules
...
Python thread
frame
eval loop
__main__
A10 …
A20 …
B10 …
B20 ...
A30 …
A40 …
thread
TSS (TLS)
“stack”
Python thread
frame
eval loop
def spam():
…
t = threading.Thread(target=spam)
t.start()
…
t.join()
spam()
C10 …
C20 …
Multi-threading!
27
process
env vars
signals
…
“data”
“heap”
thread (“main”)
TSS (TLS)
“stack”
CPython runtime
config
PyMem
...
interpreter (“main”)
sys
sys.modules
...
Python thread
frame
eval loop
__main__
A10 …
A20 …
B10 …
B20 ...
A30 …
A40 …
thread
TSS (TLS)
“stack”
Python thread
frame
eval loop
def spam():
…
t = threading.Thread(target=spam)
t.start()
…
t.join()
spam()
C10 …
C20 …
Multi-threading!
28
process
env vars
signals
…
“data”
“heap”
thread (“main”)
TSS (TLS)
“stack”
CPython runtime
config
PyMem
...
interpreter (“main”)
sys
sys.modules
...
Python thread
frame
eval loop
__main__
A10 …
A20 …
B10 …
B20 ...
A30 …
A40 …
def spam():
…
t = threading.Thread(target=spam)
t.start()
…
t.join()
spam()
C10 …
C20 …
Multi-threading!
29
process
env vars
signals
…
“data”
“heap”
thread (“main”)
TSS (TLS)
“stack”
CPython runtime
config
PyMem
...
interpreter (“main”)
sys
sys.modules
...
Python thread
frame
eval loop
__main__
A10 …
A20 …
B10 …
B20 ...
A30 …
A40 …
thread
TSS (TLS)
“stack”
Python thread
frame
eval loop
def spam():
…
t = threading.Thread(target=spam)
t.start()
…
t.join()
spam()
C10 …
C20 …
Multi-threading!
30
process
env vars
signals
…
“data”
“heap”
thread (“main”)
TSS (TLS)
“stack”
CPython runtime
config
PyMem
...
interpreter (“main”)
sys
sys.modules
...
Python thread
frame
eval loop
__main__
A10 …
A20 …
B10 …
B20 ...
A30 …
A40 …
thread
TSS (TLS)
“stack”
Python thread
frame
eval loop
def spam():
…
t = threading.Thread(target=spam)
t.start()
…
t.join()
spam()
C10 …
C20 …
Multi-threading!
31
process
env vars
signals
…
“data”
“heap”
thread (“main”)
TSS (TLS)
“stack”
CPython runtime
config
PyMem
...
interpreter (“main”)
sys
sys.modules
...
Python thread
frame
eval loop
__main__
A10 …
A20 …
B10 …
B20 ...
A30 …
A40 …
thread
TSS (TLS)
“stack”
Python thread
frame
eval loop
def spam():
…
t = threading.Thread(target=spam)
t.start()
…
t.join()
spam()
C10 …
C20 …
Multi-threading!
32
process
env vars
signals
…
“data”
“heap”
thread (“main”)
TSS (TLS)
“stack”
CPython runtime
config
PyMem
...
interpreter (“main”)
sys
sys.modules
...
Python thread
frame
eval loop
__main__
A10 …
A20 …
B10 …
B20 ...
A30 …
A40 …
thread
TSS (TLS)
“stack”
Python thread
frame
eval loop
def spam():
…
t = threading.Thread(target=spam)
t.start()
…
t.join()
spam()
C10 …
C20 …
Multi-threading!
33
process
env vars
signals
…
“data”
“heap”
thread (“main”)
TSS (TLS)
“stack”
CPython runtime
config
PyMem
...
interpreter (“main”)
sys
sys.modules
...
Python thread
frame
eval loop
__main__
A10 …
A20 …
B10 …
B20 ...
A30 …
A40 …
def spam():
…
t = threading.Thread(target=spam)
t.start()
…
t.join()
spam()
C10 …
C20 …
Multi-threading!
34
process
env vars
signals
…
“data”
“heap”
thread (“main”)
TSS (TLS)
“stack”
CPython runtime
config
PyMem
...
interpreter (“main”)
sys
sys.modules
...
Python thread
frame
eval loop
__main__
A10 …
A20 …
B10 …
B20 ...
A30 …
A40 …
def spam():
…
t = threading.Thread(target=spam)
t.start()
…
t.join()
spam()
C10 …
C20 …
Multi-threading!
35
process
env vars
signals
…
“data”
“heap”
thread (“main”)
TSS (TLS)
“stack”
CPython runtime
config
PyMem
...
interpreter (“main”)
sys
sys.modules
...
Python thread
frame
eval loop
thread
TSS (TLS)
“stack”
Python thread
frame
eval loop
def spam():
…
t = threading.Thread(target=spam)
t.start()
…
t.join()
“bytecode”
A10
A20 # t.start()
A40 # t.join()
B10
B20
A30
C10
C20
C10
C20B10
B20
A30
B10
C10B20
C20A30
C10B10
B20
A30
C20
“Race Condition”
A.K.A. “Resource Contention”
36
# thread A
…
spam = read()
spam.a = 42
write(spam)
…
# thread B
…
spam = read()
if spam.a != 42:
…
…
race here
“Race Condition”
A.K.A. “Resource Contention”
37
# thread A
…
spam = read()
spam.a = 42
write(spam)
…
# thread B
…
spam = read()
if spam.a != 42:
…
…
acquire lock A ->
acquire lock A ->
release lock A ->
release lock A ->
“Race Condition”
A.K.A. “Resource Contention”
38
# thread A
…
spam = read()
spam.a = 42
write(spam)
…
# thread B
…
spam = read()
if spam.a != 42:
…
…
acquire lock A ->
acquire lock A ->
release lock A ->
release lock A ->
“Race Condition”
A.K.A. “Resource Contention”
39
# thread A
…
spam = read()
spam.a = 42
write(spam)
…
# thread B
…
spam = read()
if spam.a != 42:
…
…
acquire lock A ->
acquire lock A ->
release lock A ->
release lock A ->
…
spam = read()
spam.a = 42
write(spam)
…
spam = read()
if spam.a != 42:
…
…
The GIL
40
The GIL (“Global Interpreter Lock”)
41
process
env vars
signals
…
“data”
“heap”
thread (“main”)
TSS (TLS)
“stack”
CPython runtime
config
PyMem
...
interpreter (“main”)
sys
sys.modules
...
Python thread
frame
eval loop
__main__
A10 …
A20 …
B10 …
B20 ...
A30 …
A40 …
thread
TSS (TLS)
“stack”
Python thread
frame
eval loop
def spam():
…
t = threading.Thread(target=spam)
t.start()
…
t.join()
spam()
C10 …
C20 …
State At Different Layers
42
process -> | global runtime -> | interpreter -> | thread / stack / ceval |
env vars | GIL | sys module | current frame |
sockets | signal handlers | modules | stack depth |
file handles | Py_AtExit() funcs | atexit handlers | “tracing” |
signals | GC | fork handlers | hook: trace |
(thread-local storage) | allocator (mem) | hook: eval_frame | hook: profile |
... | objects (w/ refcounts) | codecs | current exception |
| pending calls | | context |
| “eval breaker” | | ... |
The GIL (“Global Interpreter Lock”)
43
process
env vars
signals
…
“data”
“heap”
thread (“main”)
TSS (TLS)
“stack”
CPython runtime
config
PyMem
...
interpreter (“main”)
sys
sys.modules
...
Python thread
frame
eval loop
__main__
A10 …
A20 …
B10 …
B20 ...
A30 …
A40 …
thread
TSS (TLS)
“stack”
Python thread
frame
eval loop
def spam():
…
t = threading.Thread(target=spam)
t.start()
…
t.join()
spam()
C10 …
C20 …
The Eval Loop
<set up>
for instruction in code object:
<maybe side-channel stuff>
<occasionally release & re-acquire the GIL>
<execute next instruction>
44
When is the GIL Released?
45
Costs and Benefits
of the GIL
46
Costs and Benefits
of the GIL
47
Effect and Perception
Who does it really affect?
48
Effect and Perception
Who does it really affect?
Why? C implementation releases the GIL around IO and CPU-intensive code.
49
Effect and Perception
Who does it really affect?
So why does the GIL get such a bad wrap?
50
Working Around the GIL
51
Past Efforts to Remove the GIL
52
Other Python Implementations
53
| GIL? | C-API? | latest Py version |
CPython | yes | yes | 3.7 |
no | 2.7 | ||
no | 2.7 | ||
yes (no) | 3.6 | ||
no? | ~3.4+ |
The Future
54
A New C-API
55
The C-API
56
The Problem
57
The Causes
58
The Solutions
59
The Solutions
60
The Solutions
61
Categorizing the C-API
“Do not touch!”
“Use at your own risk!”
“Go for it (but rebuild your extension each Python release)!”
“Worry-free!”
62
The Solutions
63
The Projects
64
<capi-sig@python.org>
The Projects
65
<capi-sig@python.org>
Beyond the C-API...
66
Subinterpreters!!!
67
Interpreters in a Single Process
68
Interpreters in a Single Process
69
CPython runtime
config
PyMem
...
Interpreter (“main”)
sys
sys.modules
...
subinterpreter
sys
sys.modules
...
subinterpreter
sys
sys.modules
...
Py thread
f
el
Py thread
f
el
f
el
Py thread
f
el
Py thread
f
el
f
el
Py thread
f
el
Interpreters in a Single Process
70
CPython runtime
config
PyMem
...
Interpreter (“main”)
sys
sys.modules
...
subinterpreter
sys
sys.modules
...
subinterpreter
sys
sys.modules
...
Py thread
f
el
Py thread
f
el
f
el
Py thread
f
el
Py thread
f
el
f
el
Py thread
f
el
Subinterpreters
71
PEP 554 - “Multiple Interpreters in the Stdlib”
72
PEP 554 - “Multiple Interpreters in the Stdlib”
73
PEP 554: Example 1
import interpreters
interp = interpreters.create()
interp.run(dedent("""
print('spam')
"""))
74
PEP 554: Example 1
import interpreters
interp = interpreters.create()
interp.run(dedent("""
print('spam')
"""))
75
PEP 554: Example 1
import interpreters
interp = interpreters.create()
interp.run(dedent("""
print('spam')
"""))
76
PEP 554: Example 2
interp = interpreters.create()
def func():
interp.run(dedent("""
print('spam')
"""))
t = threading.Thread(target=func)
t.start()
77
PEP 554: Example 3
interp = interpreters.create()
interp.run(dedent("""
x = 'spam'
"""))
interp.run(dedent("""
print(x)
"""))
78
PEP 554: Example 3
interp = interpreters.create()
interp.run(dedent("""
x = 'spam'
"""))
interp.run(dedent("""
print(x)
"""))
79
PEP 554: Example 3
interp = interpreters.create()
interp.run(dedent("""
x = 'spam'
"""))
interp.run(dedent("""
print(x)
"""))
80
PEP 554 - “Multiple Interpreters in the Stdlib”
81
PEP 554 - “Multiple Interpreters in the Stdlib”
82
PEP 554 - “Multiple Interpreters in the Stdlib”
83
For now:
PEP 554: Example 4
(rchan, schan
) = interpreters.create_channel()
interp = interpreters.create()
def func():
interp.run(dedent("""
import spam
data = spam.do_something()
ch.send(data) # blocks
""", channels={ch: schan})
t = threading.Thread(target=func)
t.start()
data = rchan.recv() # blocks
process_data(data)
84
PEP 554: Example 4
(rchan, schan
) = interpreters.create_channel()
interp = interpreters.create()
def func():
interp.run(dedent("""
import spam
data = spam.do_something()
ch.send(data) # blocks
""", channels={ch: schan})
t = threading.Thread(target=func)
t.start()
data = rchan.recv() # blocks
process_data(data)
85
PEP 554: Example 4
(rchan, schan
) = interpreters.create_channel()
interp = interpreters.create()
def func():
interp.run(dedent("""
import spam
data = spam.do_something()
ch.send(data) # blocks
""", channels={ch: schan})
t = threading.Thread(target=func)
t.start()
data = rchan.recv() # blocks
process_data(data)
86
PEP 554: Example 4
(rchan, schan
) = interpreters.create_channel()
interp = interpreters.create()
def func():
interp.run(dedent("""
import spam
data = spam.do_something()
ch.send(data) # blocks
""", channels={ch: schan})
t = threading.Thread(target=func)
t.start()
data = rchan.recv() # blocks
process_data(data)
87
PEP 554: Example 4
(rchan, schan
) = interpreters.create_channel()
interp = interpreters.create()
def func():
interp.run(dedent("""
import spam
data = spam.do_something()
ch.send(data) # blocks
""", channels={ch: schan})
t = threading.Thread(target=func)
t.start()
data = rchan.recv() # blocks
process_data(data)
88
PEP 554: Example 4
(rchan, schan
) = interpreters.create_channel()
interp = interpreters.create()
def func():
interp.run(dedent("""
import spam
data = spam.do_something()
ch.send(data) # blocks
""", channels={ch: schan})
t = threading.Thread(target=func)
t.start()
data = rchan.recv() # blocks
process_data(data)
89
Who Cares?
90
Who Cares?
91
Who Cares?
92
Who Cares?
93
Stop Sharing the GIL!!!
94
State At Different Layers
95
process -> | global runtime -> | interpreter -> | thread / stack / ceval |
env vars | GIL | sys module | current frame |
sockets | signal handlers | modules | stack depth |
file handles | Py_AtExit() funcs | atexit handlers | “tracing” |
signals | GC | fork handlers | hook: trace |
(thread-local storage) | allocator (mem) | hook: eval_frame | hook: profile |
... | objects | codecs | current exception |
| pending calls | | context |
| “eval breaker” | | ... |
Stop Sharing the GIL!
96
Stop Sharing the GIL!
97
Why Hasn’t It Been Done Already?
98
the blockers
(╯°□°)╯︵ ┻━┻
99
the blockers
(╯°□°)╯︵ ┻━┻
100
the blockers
(╯°□°)╯︵ ┻━┻
101
the blockers
(╯°□°)╯︵ ┻━┻
102
C "Globals"
103
the project
104
the project
https://github.com/ericsnowcurrently/multi-core-python
105
the project
https://github.com/ericsnowcurrently/multi-core-python
106
State At Different Layers
107
process -> | global runtime -> | interpreter -> | thread / stack / ceval |
env vars | GIL | sys module | current frame |
sockets | signal handlers | modules | stack depth |
file handles | Py_AtExit() funcs | atexit handlers | “tracing” |
signals | GC | fork handlers | hook: trace |
(thread-local storage) | allocator (mem) | hook: eval_frame | hook: profile |
... | objects | codecs | current exception |
| pending calls | | context |
| “eval breaker” | | ... |
Beneficial Side Effects
108
What’s Next?
109
Thanks!
110
Thanks!
Questions?
111
Resources
112