Python Packaging and Distribution

This template provides (semi)automated workflows to generate pip|conda installable packages.

Conda

Coda packaging is particularly challenging when performed manually. For this reason, once set all the configuration files (environment.yaml and in conda/*), most of the work will be delegated to a set of GitHub actions (discussed in a separate section).

For a complete guide, refer to Conda Packaging Guide.

Let’s focus on the configuration files.

environment.yaml

This file contains the list of python packages on which the developed package depends. All listed packages must be available on a conda channel (either the official one or additional ones - like conda-forge). Package-specific releases can be specified.

conda/conda_build_config.yaml

Specifies the python version for which the python package has to be generated

conda/meta.YAML

Specifies all the resources needed to build the package and the metadata to embed in the package. Particular care is needed to correctly specify the package dependencies (to be aligned with the one in environment.yaml); the set of packages that have to be present on the host to support the packaging; the set of packages that need to be present on the host to execute the unit tests.

build.sh

This file specifies the command run by the GitHub action in charge of conda packaging.

PyPi

Generating (and distributing) a pip installable package, even by hand, is relatively easy. Let’s focus on both aspects and on the configuration files needed to fulfill such tasks.

For a complete guide, refer to Python Packaging User Guide.

Packaging

The packaging is mainly controlled by two files: setup.py and requirements.txt: once properly configured, the package can be generated with the following command line directive:

python setup.py sdist bdist_wheel

The generated installable (and source) package files will be made available in a new folder: dist/

setup.py

This file describes all the package metadata (version, description, dependencies, and folders to be excluded by the packaging).

requirements.txt

This file is the analog of environment.yaml but for pip (the two should, in principle, be aligned, although this requirement is not enforced). All packages listed (in the specified versions, if any) must be installable through pip.

Distribution

To distribute a pip package, registering an account on the PyPi (or PyPi testing) platform is mandatory. After such a step (and the generation of the installable package), package uploading is obtained using the twine command line tool (PyPi|PyPi testing platforms credentials will be asked).

 pip install twine
 twine upload dist/generated_pip_installable_package_name

Note: All package releases must be identified by different (incremental) version numbers (to be specified in setup.py). It is not possible to overwrite a distributed package imposing the same version number.

Note 2: Even using the dedicated GitHub actions, the first package release must be generated using the twine command line tool.