Upgrade -core to Python 3.13 

(#334)

Work Report�Presented by: Sakshi Kekre

Date: May 6, 2025

The Task and Context

• Taking over upgrade effort from Al (previous contributor)
• Task: Upgrade policyengine-core to Python 3.13
• numpy: 1.x → 2.x (major refactors)
• Goals: Modernize codebase, Ensure compatibility

Why It Is Needed

• Python 3.13: To stay ahead of upcoming ecosystem changes and ensure long-term compatibility as libraries begin to drop support for older versions.
• NumPy 2.1.0: The first version to officially support Python 3.13; Used heavily in -core, we can benefit from improved performance and stricter typing, ensuring compatibility and preventing silent failures or runtime issues.

Current Status

Targets established for success -

​

• Code upgraded to support Python 3.13 & NumPy 2
• All unit tests and smoke test in -core passing
• Locally, downstream country package tests run successfully with upgraded -core in all supported envs (3.11, 3.12)

Migration Process

• Environment Setup: Python 3.13, isolated environment
• Dependency Audit: pipdeptree to find incompatible packages
• Code Refactoring: Replace deprecated APIs, linting with ruff ‘NPY201’ rule
• Testing & Validation: Run tests in -core and downstream packages

How I discovered changes needed

• Ran unit tests during upgrade and hit failures.
• Read the NumPy 2.0 migration guide, which mentions stricter input validation and dtype promotion changes.
• Going further:
• np._set_promotion_state("weak_and_warn") to identify silent behavior changes.
• pytest with warnings.simplefilter("error").

Notable Changes Performed

• Deprecations:
• NumPy aliases, np.infty →np.inf, np.float_, → np.float64, np.round_ →np.round

​

• Replaced enum objects (in the zone array) with their .name strings to ensure proper indexing, avoid object dtype issues, and conform to stricter NumPy 2.0 array behavior.
• To avoid ambiguous behavior under NumPy 2.0's stricter input handling, we explicitly disallow tuple inputs in concat(). This prevents misinterpretation of shapes or dtypes and ensures more predictable behavior. A test was added to enforce this.

Key Differences to Be Aware Of

• Data Types: Removed np.int, np.float
• Behavior Changes: Warnings now raise errors
• NumPy scalar promotion is now consistent — e.g., float32 + 3. stays float32
• Default integer type on 64-bit is now int64, not long (historically linked to C long)
• C extensions need to use new NumPy macros and may break if using legacy struct fields
• You may see subtle differences in numeric results or memory use in edge cases

Future Scope / Deeper Dive needed

• Windows long path in PR.yml
• Sorting logic introduced

​

References

• https://github.com/PolicyEngine/policyengine-core/pull/338
• https://numpy.org/devdocs/numpy_2_0_migration_guide.html
• https://docs.python.org/3/whatsnew/3.13.html
• ​

THANK YOU!