Table of contents

• Simple packaging
  • pyproject.toml: build-system
  • pyproject.toml: project table
    • Extras
    • Command line
  • Package structure
  • Versioning
  • Including/excluding files in the SDist

Simple packaging

Python packages can now use a modern build system instead of the classic but verbose setuptools and setup.py. The one you select doesn’t really matter that much; they all use a standard configuration language introduced in PEP 621. The PyPA’s Flit is a great option. scikit-build-core and meson-python are being developed to support this sort of configuration, enabling binary extension packages to benefit too. These PEP 621 tools currently include Hatch, PDM, Flit, Whey, and Setuptools. Poetry will eventually gain support in 2.0.

Classic files

These systems do not use or require setup.py, setup.cfg, or MANIFEST.in. Those are for setuptools. Unless you are using setuptools, of course, which still uses MANIFEST.in. You can convert the old files using pipx run hatch new --init or with ini2toml.

Selecting a backend

Backends handle metadata the same way, so the choice comes down to how you specify what files go into an SDist and extra features, like getting a version from VCS. If you don’t have an existing preference, hatchling is an excellent choice, balancing speed, configurability, and extendability.

pyproject.toml: build-system

PY001 Packages must have a pyproject.toml file PP001 that selects the backend:

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[build-system]
requires = ["flit_core>=3.3"]
build-backend = "flit_core.buildapi"

[build-system]
requires = ["pdm-backend"]
build-backend = "pdm.backend"

[build-system]
requires = ["setuptools>=61.0"]
build-backend = "setuptools.build_meta"

pyproject.toml: project table

The metadata is specified in a standards-based format:

[project]
name = "package"
description = "A great package."
readme = "README.md"
license.file = "LICENSE"
authors = [
  { name = "My Name", email = "me@email.com" },
]
maintainers = [
  { name = "My Organization", email = "myemail@email.com" },
]
requires-python = ">=3.8"

dependencies = [
  "typing_extensions",
]

classifiers = [
  "Development Status :: 4 - Beta",
  "License :: OSI Approved :: BSD License",
  "Programming Language :: Python :: 3 :: Only",
  "Programming Language :: Python :: 3.8",
  "Programming Language :: Python :: 3.9",
  "Programming Language :: Python :: 3.10",
  "Programming Language :: Python :: 3.11",
  "Programming Language :: Python :: 3.12",
  "Topic :: Scientific/Engineering :: Physics",
]

[project.urls]
Homepage = "https://github.com/organization/package"
Documentation = "https://package.readthedocs.io/"
"Bug Tracker" = "https://github.com/organization/package/issues"
Discussions = "https://github.com/organization/package/discussions"
Changelog = "https://package.readthedocs.io/en/latest/changelog.html"

You can read more about each field, and all allowed fields, in packaging.python.org, Flit or Whey. Note that “Homepage” is special, and replaces the old url setting.

Extras

It is recommended to use extras instead of or in addition to making requirement files. These extras a) correctly interact with install requires and other built-in tools, b) are available directly when installing via PyPI, and c) are allowed in requirements.txt, install_requires, pyproject.toml, and most other places requirements are passed.

Here is an example of a simple extras:

[project.optional-dependencies]
test = [
  "pytest >=6.0",
]
mpl = [
  "matplotlib >=2.0",
]

Self dependencies can be used by using the name of the package, such as dev = ["package[test,examples]"], but this requires Pip 21.2 or newer. We recommend providing at least test and docs.

Command line

If you want to ship an “app” that a user can run from the command line, you need to add a script entry point. The form is:

[project.scripts]
cliapp = "package.__main__:main"

The format is command line app name as the key, and the value is the path to the function, followed by a colon, then the function to call. If you use __main__.py as the file, then python -m followed by the module will also work to call the app (__name__ will be "__main__" in that case).

Package structure

All packages should have a src folder, with the package code residing inside it, such as src/<package>/. This may seem like extra hassle; after all, you can type “python” in the main directory and avoid installing it if you don’t have a src folder! However, this is a bad practice, and it causes several common bugs, such as running pytest and getting the local version instead of the installed version - this obviously tends to break if you build parts of the library or if you access package metadata.

This sadly is not part of the standard metadata in [project], so it depends on what backend you you use. Hatchling, Flit, PDM, and setuptools use automatic detection, while Whey does not, requiring a tool setting.

If you don’t match your package name and import name (which you should except for very special cases), you will likely need extra configuration here.

You should have a README PY002 and a LICENSE PY003 file. You should have a docs/ folder PY004. You should have a /tests folder PY005 (recommended) and/or a src/<package>/tests folder.

Versioning

You can specify the version manually (as shown in the example), but the backends usually provide some automatic features to help you avoid this. Flit will pull this from a file if you ask it to. Hatchling and PDM can be instructed to look in a file or use git.

You will always need to specify that the version will be supplied dynamically with:

dynamic = ["version"]

Then you’ll configure your backend to compute the version.

Hatchling dynamic versioning

You can tell hatchling to get the version from VCS. Add hatch-vcs to your build-backend.requires, then add the following configuration:

[tool.hatch]
version.source = "vcs"
build.hooks.vcs.version-file = "src/<package>/version.py"

Or you can tell it to look for it in a file (see docs for arbitrary regex’s):

[tool.hatch]
version.path = "src/<package>/__init__.py"

(replace <package> with the package path).

You should also add these two files:

.git_archival.txt:

node: $Format:%H$
node-date: $Format:%cI$
describe-name: $Format:%(describe:tags=true,match=*[0-9]*)$
ref-names: $Format:%D$

And .gitattributes (or add this line if you are already using this file):

.git_archival.txt  export-subst

This will allow git archives (including the ones generated from GitHub) to also support versioning.

Including/excluding files in the SDist

This is tool specific.

• Hatchling info here. Hatchling uses your VCS ignore file by default, so make sure it is accurate (which is a good idea anyway).
• Flit info here. Flit requires manual inclusion/exclusion in many cases, like using a dirty working directory.
• PDM info here.
• Setuptools still uses MANIFEST.in.

Flit will not use VCS (like git) to populate the SDist if you use standard tooling, even if it can do that using its own tooling. So make sure you list explicit include/exclude rules, and test the contents:

# Show SDist contents
tar -tvf dist/*.tar.gz
# Show wheel contents
unzip -l dist/*.whl