The C-API

Eric Snow

Language Summit

PyCon 2019

The C-API

• historically fundamental to Python’s success
• organic growth
• early efforts to simplify
• core devs: growing concerns
• core devs: increasing efforts

2

The Problem

• some problems (GIL, perf) need low-level changes
• parts of public C-API expose low-level details (e.g. refcounts)
• so...
• fixing those problems requires breaking parts of C-API

3

The Solutions

• someone has to care enough to do the work
• physically separate the categories of C-API
• more opaque structs

4

Categorizing the C-API

• “internal”
• “private”
• “unstable”
• “stable”

​

The "Include" directory is structured along these lines.

“Do not touch!”

“Use at your own risk!”

“Go for it (but rebuild your extension each Python release)!”

“Worry-free!”

5

The Solutions

• someone has to care enough to do the work
• physically separate the categories of C-API
• more opaque structs
• Python (C)FFI
• break compatibility in a few places
• deprecate C-API in favor of something like Cython (official)
• maybe even completely replace the existing C-API
• ...

6

Some of the Projects

• Victor Stinner’s, well, all of it
• https://pythoncapi.readthedocs.io/roadmap.html
• Steve Dower’s proposal
• https://mail.python.org/archives/list/capi-sig@python.org/thread/B2VDVLABM4RQ4ATEJXFZYWEGTBZPUBKW/
• Petr's talk
• bringing sanity to runtime initialization and finalization (e.g. PEP 432)
• enterprise: Facebook/Microsoft/etc.
• me: https://github.com/ericsnowcurrently/multi-core-python

Join us on <capi-sig@python.org>.

7