CPython Bugs
2022-07-13 @ EuroPython 2022
by �Dominik ‘Disconnect3d’ Czarnota
1
[This talk’s video is available on EuroPython’s YT channel]
# whoami
�@disconnect3d_pl�https://disconnect3d.pl/
A quick question first
3
4
5
6
CPython bugtracker stats
7
Let’s talk about bugs
8
“Readline module loading in interactive mode”�https://bugs.python.org/issue12238�Python 2.x/3.x�Reported on 2011-06-02
9
DEMO
“Readline module loading in interactive mode”
10
#include <stdio.h>
__attribute__((constructor)) static void init() {
puts("HACKED!");
}
“Readline module loading in interactive mode”
TL;DR: gcc fakereadline.c -shared -o readline.so && python
===> code execution within Python process
11
#include <stdio.h>
__attribute__((constructor)) static void init() {
puts("HACKED!");
}
Other libs that may be loaded at interpreter runtime
$ strace -e openat python3 2>&1 | egrep '\.so'
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
// (...)
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libm.so.6", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/home/dc/libr/readline.so", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/usr/lib/python3/dist-packages/apt_pkg.cpython-36m-x86_64-linux-gnu.so", O_RDONLY|O_CLOEXEC) = 3
>>> asd # write some code and hit enter
// (...)
openat(AT_FDCWD, "/usr/lib/python3.6/lib-dynload/_bz2.cpython-36m-x86_64-linux-gnu.so", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/usr/lib/python3.6/lib-dynload/_lzma.cpython-36m-x86_64-linux-gnu.so", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/usr/lib/python3.6/lib-dynload/_hashlib.cpython-36m-x86_64-linux-gnu.so", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/usr/lib/x86_64-linux-gnu/libcrypto.so.1.1", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/usr/lib/python3.6/lib-dynload/_ssl.cpython-36m-x86_64-linux-gnu.so", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/usr/lib/x86_64-linux-gnu/libssl.so.1.1", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/usr/lib/python3.6/lib-dynload/_json.cpython-36m-x86_64-linux-gnu.so", O_RDONLY|O_CLOEXEC) = 3
12
Other libs that may be loaded at interpreter runtime
$ strace -e openat python3 2>&1 | egrep '\.so'
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
// (...)
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libm.so.6", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/home/dc/libr/readline.so", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/usr/lib/python3/dist-packages/apt_pkg.cpython-36m-x86_64-linux-gnu.so", O_RDONLY|O_CLOEXEC) = 3
>>> asd # write some code and hit enter
// (...)
openat(AT_FDCWD, "/usr/lib/python3.6/lib-dynload/_bz2.cpython-36m-x86_64-linux-gnu.so", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/usr/lib/python3.6/lib-dynload/_lzma.cpython-36m-x86_64-linux-gnu.so", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/usr/lib/python3.6/lib-dynload/_hashlib.cpython-36m-x86_64-linux-gnu.so", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/usr/lib/x86_64-linux-gnu/libcrypto.so.1.1", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/usr/lib/python3.6/lib-dynload/_ssl.cpython-36m-x86_64-linux-gnu.so", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/usr/lib/x86_64-linux-gnu/libssl.so.1.1", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/usr/lib/python3.6/lib-dynload/_json.cpython-36m-x86_64-linux-gnu.so", O_RDONLY|O_CLOEXEC) = 3
13
Can we mitigate this?
14
Mitigating libreadline loading
static void
pymain_import_readline(const PyConfig *config)
{
if (config->isolated) {
return;
}
if (!config->inspect && config_run_code(config)) {
return;
}
if (!isatty(fileno(stdin))) {
return;
}
PyObject *mod = PyImport_ImportModule("readline");
if (mod == NULL) {
PyErr_Clear();
}
else {
Py_DECREF(mod);
}
}
15
Mitigating libreadline loading
static void
pymain_import_readline(const PyConfig *config)
{
if (config->isolated) {
return;
}
if (!config->inspect && config_run_code(config)) {
return;
}
if (!isatty(fileno(stdin))) {
return;
}
PyObject *mod = PyImport_ImportModule("readline");
if (mod == NULL) {
PyErr_Clear();
}
else {
Py_DECREF(mod);
}
}
16
Mitigating libreadline loading
static void
pymain_import_readline(const PyConfig *config)
{
if (config->isolated) {
return;
}
if (!config->inspect && config_run_code(config)) {
return;
}
if (!isatty(fileno(stdin))) {
return;
}
PyObject *mod = PyImport_ImportModule("readline");
if (mod == NULL) {
PyErr_Clear();
}
else {
Py_DECREF(mod);
}
}
17
Mitigating libreadline loading
static void
pymain_import_readline(const PyConfig *config)
{
if (config->isolated) {
return;
}
if (!config->inspect && config_run_code(config)) {
return;
}
if (!isatty(fileno(stdin))) {
return;
}
PyObject *mod = PyImport_ImportModule("readline");
if (mod == NULL) {
PyErr_Clear();
}
else {
Py_DECREF(mod);
}
}
18
“Isolate mode”
Mitigating libreadline loading
static void
pymain_import_readline(const PyConfig *config)
{
if (config->isolated) {
return;
}
if (!config->inspect && config_run_code(config)) {
return;
}
if (!isatty(fileno(stdin))) {
return;
}
PyObject *mod = PyImport_ImportModule("readline");
if (mod == NULL) {
PyErr_Clear();
}
else {
Py_DECREF(mod);
}
}
19
“Isolate mode”
Use: python -I
(alias python="python -I" on root!)
or actually…
20
21
22
Btw GitHub such wow,
How do i know if the merged commit exists on a tag?
23
Btw GitHub such wow,
How do i know if the merged commit exists on a tag?
24
Btw GitHub such wow,
How do i know if the merged commit exists on a tag?
25
Btw GitHub such wow,
How do i know if the merged commit exists on a tag?
Ok, lets look at another bug(?)
26
“Deprecate and remove code �execution in �pth files”��AKA code execution via installed *but not imported package*
https://bugs.python.org/issue33944�Python 3.x�Reported on 2018-06-22
Thx to Artur Czerpiel for this one :)
27
But what are .pth files at all?�O_o
28
29
30
31
32
33
Want to test this feature bug?
34
* probably just not great design
Just install:�pip install deliverymethod
* note: you should not install packages not audited/reviewed by yourself (...)
35
Just install:�pip install deliverymethod
* note: you should not install packages not audited/reviewed by yourself (...)
36
DEMO
pip install deliverymethod
37
How does it all work?
38
How does it all work?
$ ls -la $HOME/.local/lib/python3.6/site-packages/*.pth
-rw-rw-r-- 1 dc dc 35 Aug 29 22:30 .local/lib/python3.6/site-packages/aaaaaa_deliverymethod.pth
$ cat .local/lib/python3.6/site-packages/aaaaaa_deliverymethod.pth
import sys; import deliverymethod
$ cat .local/lib/python3.6/site-packages/deliverymethod.py
print("Payload delivered")
39
def addpackage(sitedir, name, known_paths):
"""Process a .pth file within the site-packages directory:
For each line in the file, either combine it with sitedir to a path
and add that to known_paths, or execute it if it starts with 'import '.
"""
# (...)
fullname = os.path.join(sitedir, name)
try:
f = io.TextIOWrapper(io.open_code(fullname))
except OSError:
return
with f:
for n, line in enumerate(f):
if line.startswith("#"):
continue
try:
if line.startswith(("import ", "import\t")):
exec(line)
continue
line = line.rstrip()
dir, dircase = makepath(sitedir, line)
if not dircase in known_paths and os.path.exists(dir):
sys.path.append(dir)
known_paths.add(dircase)
except Exception:
# (...)
40
def addpackage(sitedir, name, known_paths):
"""Process a .pth file within the site-packages directory:
For each line in the file, either combine it with sitedir to a path
and add that to known_paths, or execute it if it starts with 'import '.
"""
# (...)
fullname = os.path.join(sitedir, name)
try:
f = io.TextIOWrapper(io.open_code(fullname))
except OSError:
return
with f:
for n, line in enumerate(f):
if line.startswith("#"):
continue
try:
if line.startswith(("import ", "import\t")):
exec(line)
continue
line = line.rstrip()
dir, dircase = makepath(sitedir, line)
if not dircase in known_paths and os.path.exists(dir):
sys.path.append(dir)
known_paths.add(dircase)
except Exception:
# (...)
41
def addpackage(sitedir, name, known_paths):
"""Process a .pth file within the site-packages directory:
For each line in the file, either combine it with sitedir to a path
and add that to known_paths, or execute it if it starts with 'import '.
"""
# (...)
fullname = os.path.join(sitedir, name)
try:
f = io.TextIOWrapper(io.open_code(fullname))
except OSError:
return
with f:
for n, line in enumerate(f):
if line.startswith("#"):
continue
try:
if line.startswith(("import ", "import\t")):
exec(line)
continue
line = line.rstrip()
dir, dircase = makepath(sitedir, line)
if not dircase in known_paths and os.path.exists(dir):
sys.path.append(dir)
known_paths.add(dircase)
except Exception:
# (...)
42
Who uses it?
43
Who uses it?
At least those packages:
(probably others too)
44
Who uses it?
At least those packages:
(probably others too)
45
import os, sys; exec('if \'COV_CORE_SOURCE\' in os.environ:\n try:\n from pytest_cov.embed import init\n init()\n except Exception as exc:\n sys.stderr.write(\n "pytest-cov: Failed to setup subprocess coverage. "\n "Environ: {0!r} "\n "Exception: {1!r}\\n".format(\n dict((k, v) for k, v in os.environ.items() if k.startswith(\'COV_CORE\')),\n exc\n )\n )\n')
Who uses it?
At least those packages:
(probably others too)
46
import os, sys; exec('if \'COV_CORE_SOURCE\' in os.environ:\n try:\n from pytest_cov.embed import init\n init()\n except Exception as exc:\n sys.stderr.write(\n "pytest-cov: Failed to setup subprocess coverage. "\n "Environ: {0!r} "\n "Exception: {1!r}\\n".format(\n dict((k, v) for k, v in os.environ.items() if k.startswith(\'COV_CORE\')),\n exc\n )\n )\n')
Who uses it?
At least those packages:
(probably others too)
47
import os, sys; exec('if \'COV_CORE_SOURCE\' in os.environ:\n try:\n from pytest_cov.embed import init\n init()\n except Exception as exc:\n sys.stderr.write(\n "pytest-cov: Failed to setup subprocess coverage. "\n "Environ: {0!r} "\n "Exception: {1!r}\\n".format(\n dict((k, v) for k, v in os.environ.items() if k.startswith(\'COV_CORE\')),\n exc\n )\n )\n')
Who uses it?
At least those packages:
(probably others too)
* hunter is some code tracing module
48
import hunter; hunter._embed_via_environment()
source:
Can we prevent this behavior?
49
PEP 648 was proposed
50
PEP 648 was proposed
51
Can we prevent this behavior?
52
“socket.inet_aton parsing issue on some libc versions”
https://bugs.python.org/issue37495
Python 2.x/3.x�Reported by me on 2019-07-03
53
“socket.inet_aton parsing issue on some libc versions”
54
“socket.inet_aton parsing issue on some libc versions”
55
“socket.inet_aton parsing issue on some libc versions”
56
“socket.inet_aton parsing issue on some libc versions”
57
DEMO
“socket.inet_aton parsing issue on some libc versions”
In [2]: socket.inet_aton('127.0.0.1')
Out[2]: b'\x7f\x00\x00\x01'
In [3]: socket.inet_aton('8.8.8.8')
Out[3]: b'\x08\x08\x08\x08'
In [4]: socket.inet_aton('134744072')
Out[4]: b'\x08\x08\x08\x08'
In [5]: socket.inet_aton('0x7f.1')
Out[5]: b'\x7f\x00\x00\x01'
In [6]: socket.inet_aton('010.0')
Out[6]: b'\x08\x00\x00\x00'
In [7]: socket.inet_aton('abc')
---------------------------------------------------------------------------
OSError Traceback (most recent call last)
<ipython-input-8-5e88c766a2d5> in <module>
----> 1 socket.inet_aton('abc')
OSError: illegal IP address string passed to inet_aton
58
“socket.inet_aton parsing issue on some libc versions”
In [2]: socket.inet_aton('127.0.0.1')
Out[2]: b'\x7f\x00\x00\x01'
In [3]: socket.inet_aton('8.8.8.8')
Out[3]: b'\x08\x08\x08\x08'
In [4]: socket.inet_aton('134744072')
Out[4]: b'\x08\x08\x08\x08'
In [5]: socket.inet_aton('0x7f.1')
Out[5]: b'\x7f\x00\x00\x01'
In [6]: socket.inet_aton('010.0')
Out[6]: b'\x08\x00\x00\x00'
In [7]: socket.inet_aton('abc')
---------------------------------------------------------------------------
OSError Traceback (most recent call last)
<ipython-input-8-5e88c766a2d5> in <module>
----> 1 socket.inet_aton('abc')
OSError: illegal IP address string passed to inet_aton
59
Standard ipv4
“socket.inet_aton parsing issue on some libc versions”
In [2]: socket.inet_aton('127.0.0.1')
Out[2]: b'\x7f\x00\x00\x01'
In [3]: socket.inet_aton('8.8.8.8')
Out[3]: b'\x08\x08\x08\x08'
In [4]: socket.inet_aton('134744072')
Out[4]: b'\x08\x08\x08\x08'
In [5]: socket.inet_aton('0x7f.1')
Out[5]: b'\x7f\x00\x00\x01'
In [6]: socket.inet_aton('010.0')
Out[6]: b'\x08\x00\x00\x00'
In [7]: socket.inet_aton('abc')
---------------------------------------------------------------------------
OSError Traceback (most recent call last)
<ipython-input-8-5e88c766a2d5> in <module>
----> 1 socket.inet_aton('abc')
OSError: illegal IP address string passed to inet_aton
60
A bit weird but valid ipv4s...
“socket.inet_aton parsing issue on some libc versions”
In [2]: socket.inet_aton('127.0.0.1')
Out[2]: b'\x7f\x00\x00\x01'
In [3]: socket.inet_aton('8.8.8.8')
Out[3]: b'\x08\x08\x08\x08'
In [4]: socket.inet_aton('134744072')
Out[4]: b'\x08\x08\x08\x08'
In [5]: socket.inet_aton('0x7f.1')
Out[5]: b'\x7f\x00\x00\x01'
In [6]: socket.inet_aton('010.0')
Out[6]: b'\x08\x00\x00\x00'
In [7]: socket.inet_aton('abc')
---------------------------------------------------------------------------
OSError Traceback (most recent call last)
<ipython-input-8-5e88c766a2d5> in <module>
----> 1 socket.inet_aton('abc')
OSError: illegal IP address string passed to inet_aton
61
“socket.inet_aton parsing issue on some libc versions”
In [2]: socket.inet_aton('1.1.1.1? Cannot do that')
---------------------------------------------------------------------------
OSError Traceback (most recent call last)
<ipython-input-3-4d1b5f3d76cf> in <module>
----> 1 socket.inet_aton('1.1.1.1? Cannot do that')
OSError: illegal IP address string passed to inet_aton
In [3]: socket.inet_aton('1.1.1.1 to dziala')
Out[3]: b'\x01\x01\x01\x01'
In [4]: socket.inet_aton('0x7f.1 ; oh nie!')
Out[4]: b'\x7f\x00\x00\x01'
62
“socket.inet_aton parsing issue on some libc versions”
In [2]: socket.inet_aton('1.1.1.1? Cannot do that')
---------------------------------------------------------------------------
OSError Traceback (most recent call last)
<ipython-input-3-4d1b5f3d76cf> in <module>
----> 1 socket.inet_aton('1.1.1.1? Cannot do that')
OSError: illegal IP address string passed to inet_aton
In [3]: socket.inet_aton('1.1.1.1 to dziala')
Out[3]: b'\x01\x01\x01\x01'
In [4]: socket.inet_aton('0x7f.1 ; oh nie!')
Out[4]: b'\x7f\x00\x00\x01'
63
Now imagine such code on prod
ip_string = request['ip_string']
if socket.inet_aton(ip_string):
os.system('ping ' + ip_string)
64
Now imagine such code on prod
ip_string = request['ip_string']
if socket.inet_aton(ip_string):
os.system('ping ' + ip_string)
65
That was an actual code!
Now imagine such code on prod
ip_string = request['ip_string']
if socket.inet_aton(ip_string):
os.system('ping ' + ip_string)
66
That was an actual code!
But from a router and written in C & it allowed for RCE �via the router’s ping IP functionality
Found by blasty (thx to whom I checked this in CPython)
And what about socket.inet_aton �usage in Python?
67
And what about socket.inet_aton usage in Python?
68
And what about socket.inet_aton usage in Python?
69
And what about socket.inet_aton usage in Python?
70
& another case in requests
71
socket.inet_aton issues in requests
>>> import requests
>>> print(requests.utils.address_in_network('1.1.1.1 wtf', '1.1.1.1/24'))
True
>>> print(requests.utils.is_ipv4_address('1.1.1.1 disconnect3d was here...'))
True
>>> print(requests.utils.is_valid_cidr('1.1.1.1 obviously not but yes/24'))
True
72
socket.inet_aton issues in requests
>>> import requests
>>> print(requests.utils.address_in_network('1.1.1.1 wtf', '1.1.1.1/24'))
True
>>> print(requests.utils.is_ipv4_address('1.1.1.1 disconnect3d was here...'))
True
>>> print(requests.utils.is_valid_cidr('1.1.1.1 obviously not but yes/24'))
True
73
crypt.crypt @ macOS
TL;DR: crypt.crypt(pwd)�Does not seem to work on macOS but works on Linux
74
crypt.crypt @ macOS
TL;DR: crypt.crypt(pwd)�Does not seem to work on macOS but works on Linux
75
crypt.crypt @ macOS
TL;DR: crypt.crypt(pwd)�Does not seem to work on macOS but works on Linux
76
...and that’s all
Thx! :)
77
CPython bugs - summary
78
by @disconnect3d_pl
Slides link (with demos) at: https://ujeb.se/pybugs