How to speed up your Python with Rust

Collins Muriuki

3RD ANNUAL PYCON SUMMIT KE

Biggest Gathering For People & Organisations Interested In The

Python Programming Language & Its Ecosystem

3RD ANNUAL PYCON SUMMIT KE

Biggest Gathering For People & Organisations Interested In The

Python Programming Language & Its Ecosystem

Talk Outline

• Answering the whys
• Demo 1
• From Rust to Python
• Demo 2

3RD ANNUAL PYCON SUMMIT KE

Biggest Gathering For People & Organisations Interested In The

Python Programming Language & Its Ecosystem

Links to slides and source code

links.collinsmuriuki.xyz

3RD ANNUAL PYCON SUMMIT KE

Biggest Gathering For People & Organisations Interested In The

Python Programming Language & Its Ecosystem

Why Speed up Python anyway?

• Why is Python “low?
• What can make your code slow?
• IO bound tasks
• CPU bound tasks

3RD ANNUAL PYCON SUMMIT KE

Biggest Gathering For People & Organisations Interested In The

Python Programming Language & Its Ecosystem

Understanding Python’s Design

3RD ANNUAL PYCON SUMMIT KE

Biggest Gathering For People & Organisations Interested In The

Python Programming Language & Its Ecosystem

Language Design - Interpreted

• Python is an interpreted language by design
• The source code is first transpiled to platform specific bytecode
• This bytecode is then fed to the interpreter, where it’s translated to machine code
• This all happens at runtime

​

3RD ANNUAL PYCON SUMMIT KE

Biggest Gathering For People & Organisations Interested In The

Python Programming Language & Its Ecosystem

Language Design - Dynamically Typed

• Python is a dynamic language, meaning the type of a variable is determined only at runtime
• For statically typed languages like Rust, this cost is consumed upfront - the compiler must know before hand the type of a variable as well as the amount of stack memory to allocate

3RD ANNUAL PYCON SUMMIT KE

Biggest Gathering For People & Organisations Interested In The

Python Programming Language & Its Ecosystem

Language Design - Garbage Collection

• Python is a garbage collected language
• Every time we create or “overwrite” a “variable” we create a new PyObject, a C struct, for which memory is allocated.
• Allocated memory is automatically freed if the reference count is 0

3RD ANNUAL PYCON SUMMIT KE

Biggest Gathering For People & Organisations Interested In The

Python Programming Language & Its Ecosystem

How python objects are stored in memory - PyObject

3RD ANNUAL PYCON SUMMIT KE

Biggest Gathering For People & Organisations Interested In The

Python Programming Language & Its Ecosystem

Language Design - Single Threaded vs Multithreaded

• Python is single threaded on a single CPU by design
• The mechanism that prevents multiple threads from executing concurrently in Python is the Global Interpreter Lock (GIL)
• The GIL prevents a variable’s reference count from getting increased/ decreased by multiple threads

3RD ANNUAL PYCON SUMMIT KE

Biggest Gathering For People & Organisations Interested In The

Python Programming Language & Its Ecosystem

“Able was I ere I saw Elba.”

​

-Napoleon

3RD ANNUAL PYCON SUMMIT KE

Biggest Gathering For People & Organisations Interested In The

Python Programming Language & Its Ecosystem

Demo 1.0 - Python vs Rust

3RD ANNUAL PYCON SUMMIT KE

Biggest Gathering For People & Organisations Interested In The

Python Programming Language & Its Ecosystem

How to make Python fast using Rust

​

• Rust bindings to the Python interpreter via the following Rust crates:
• pyO3
• cPython
• WebAssembly

3RD ANNUAL PYCON SUMMIT KE

Biggest Gathering For People & Organisations Interested In The

Python Programming Language & Its Ecosystem

Wasm to Python

3RD ANNUAL PYCON SUMMIT KE

Biggest Gathering For People & Organisations Interested In The

Python Programming Language & Its Ecosystem

Compiled Python Module

3RD ANNUAL PYCON SUMMIT KE

Biggest Gathering For People & Organisations Interested In The

Python Programming Language & Its Ecosystem

Demo 2.0 - From Rust to Python

3RD ANNUAL PYCON SUMMIT KE

Biggest Gathering For People & Organisations Interested In The

Python Programming Language & Its Ecosystem

Questions?

THANK YOU

3RD ANNUAL PYCON SUMMIT KE

Biggest Gathering For People & Organisations Interested In The

Python Programming Language & Its Ecosystem