1 of 11

Byte-sized RSE: Season 2, Session 8

Python packaging and pip

​

Session introduction

​

Steve Crouch, University of Southampton

Jeremy Cohen, Imperial College London

​

​

A UNIVERSE-HPC project event

The UNIVERSE-HPC (EP/W035731/1) project is funded through the ExCALIBUR programme and is a collaboration between the universities of Edinburgh, Oxford, Southampton and Imperial College London.

The content of these slides is licensed under CC-BY 4.0 © Jeremy Cohen, Imperial College London and Steve Crouch, University of Southampton..

2 of 11

Session Introduction: Code of Conduct

​

Our Byte-sized RSE sessions are run under the UNIVERSE-HPC Events Code of Conduct:

https://www.universe-hpc.ac.uk/events/code-of-conduct/

​

Please familiarise yourself with the Code of Conduct which applies to all activities in relation to byte-sized RSE sessions.

​

Byte-sized RSE:�Python Packaging

3 of 11

Introduction to packaging

​

  • Packages provide a way of bundling software libraries or applications into an easy to transport binary “archive”

​

  • They are used in almost all languages / operating systems / computing environments to simplify the process of installing / managing software

​

  • If you have worked with Python, JavaScript, … you have probably used packages!

​

Byte-sized RSE:�Python Packaging

4 of 11

Introduction to packaging

​

  • In this session we’ll look in more detail at packaging in Python�
  • We’ll look at:
    • How and why packages are used
    • The pip package management tool
    • What’s inside a package
    • How packages are created
    • How they are shared with others�
  • By the end of this session, you should understand how

package management tooling works and how to create

your own packages!

​

Byte-sized RSE:�Python Packaging

5 of 11

Why use packages?

​

  • Make distribution of your code much simpler
  • Encapsulate applications or libraries in a shareable artefact
  • Support principles of code reuse
  • More efficient: Similar approach to shared libraries - applications/tools/libraries don’t need to include copies of all the code they depend on - simply the code specific to them
  • Significantly simplifies the installation of software and

tools

Byte-sized RSE:�Python Packaging

6 of 11

The pip package installation tool

​

  • pip is a Python tool for installing Python packages
  • pip obtains packages from a package repository and installs them on your system
  • When you ask pip to install a package it:
    • Searches the default package repository (PyPI) or other repositories you have specified for the named package you have requested
    • Looks at the package metadata to check for dependencies
    • It, in turn, looks at dependencies of any dependent packages,

building a dependency tree and installing required packages

    • Once dependencies are installed, pip can install the

requested package

Byte-sized RSE:�Python Packaging

7 of 11

The internals of a Python package

​

  • In addition to Python code a package contains:
    • Metadata - name, licence, developer info, runtime / dev dependencies
    • Licence file
    • Readme file
    • Other assets: data files, images, etc.

Python package (either source package or wheel)

Python code

  • Library/application
  • Tests

Package metadata

(e.g. pyproject.toml)

Licence information

(e.g. LICENCE.md)

General information

(e.g. README.md)

Other files and content…

(Optional) source / compiled code for native extensions

Byte-sized RSE:�Python Packaging

8 of 11

Python package types: Wheels vs source packages

​

  • Packages can be created as “source” packages, containing the library or application’s source code, in addition to other required assets.
    • Includes source code for native extensions in, e.g. C++ or FORTRAN
    • Where native code is provided, necessary compilers / dependencies must be provided to install the package on a user’s system
  • Packages can be “wheels” (.whl extension) - may be specific to a particular operating system / Python version - native extensions are binaries - no compilation required
    • Filename structure is used to specify requirements, e.g.�numpy-2.0.0-cp312-cp312-macosx_14_0_x86_64.whl�numpy-2.0.0-cp39-cp39-win32.whl

Byte-sized RSE:�Python Packaging

9 of 11

Creating packages

​

  • A number of tools are available to simplify the process of building Python packages
    • “Traditional” (more manual) approaches using distutils or setuptools
    • Enhanced approaches using specific packaging tools:
      • Poetry, Hatch, Flit
    • In the tutorial section of this byte-sized RSE session, we’ll look at Poetry
  • Packaging tools help with preparing metadata files, building packages and uploading them to a package repository…

Byte-sized RSE:�Python Packaging

10 of 11

Sharing packages with others

​

  • In order to make your package(s) available to others, you could simply put the package files on a website where people can download them and install them locally, or, if you create releases in GitHub, direct people to the release page where a downloadable source bundle is available
  • A package repository is a far better option!
  • Manage multiple versions of packages
  • Work with package management tools to simplify and automate the process of download and installing a package and its�dependencies
  • The best known Python package repository is PyPI - �the Python Package Index (https://pypi.org)

Byte-sized RSE:�Python Packaging

11 of 11

Interactive tutorial

​

​

Now over to Steve for the interactive element of the session…

The content of these slides is licensed under CC-BY 4.0 © Jeremy Cohen, Imperial College London, Steve Crouch, University of Southampton

Byte-sized RSE:�Python Packaging