Python Packaging and Distribution with build and twine

So you've written a useful Python library. How do you share it with the world so that others can easily install it with a simple pip install your-package? The answer is to package it and upload it to the Python Package Index (PyPI).

In the past, this process involved a setup.py file and could be confusing. Today, the modern, standardized process is much simpler and relies on two key tools: build and twine.

This guide assumes you have a standard project structure with a pyproject.toml file.

Step 1: Configure Your pyproject.toml

Your pyproject.toml file is the heart of your package's configuration. It contains all the metadata that PyPI will display about your project. Ensure the [project] section is filled out completely.

[project]
name = "my-cool-package"
version = "0.1.0"
description = "A short description of my cool package."
readme = "README.md" # Important for PyPI display
requires-python = ">=3.8"
license = { text = "MIT" }
authors = [
    { name = "Your Name", email = "your@email.com" },
]
classifiers = [
    "Programming Language :: Python :: 3",
    "License :: OSI Approved :: MIT License",
    "Operating System :: OS Independent",
]
dependencies = [
    "requests>=2.28",
]

[project.urls]
Homepage = "https://github.com/your-username/my-cool-package"
"Bug Tracker" = "https://github.com/your-username/my-cool-package/issues"

• readme: Specifies the file that will be used as the long description on PyPI.
• classifiers: These are important tags that help users find your package.

Step 2: Install build and twine

These are the tools you'll use to create the distribution packages and upload them.

pip install build twine

Step 3: Build the Distribution Archives

Now, from the root of your project (the same directory as pyproject.toml), run the build command:

python -m build

This command will do two things:

1. It will create a build/ directory for intermediate files.
2. It will create a dist/ directory containing the final distribution packages.

Inside the dist/ directory, you will find two important files:

• my_cool_package-0.1.0-py3-none-any.whl: This is a wheel file. It's a pre-built, binary distribution format that is faster for end-users to install.
• my_cool_package-0.1.0.tar.gz: This is a source archive (sdist). It contains your raw source code and is used as a fallback.

You should always upload both.

Step 4: Upload to PyPI

This is where twine comes in. It's a secure tool for uploading your packages to PyPI.

First, you'll need a PyPI account. If you don't have one, register at pypi.org. It's also highly recommended to enable two-factor authentication (2FA) on your account.

Uploading to TestPyPI (Recommended for your first time):

TestPyPI is a separate instance of the package index where you can test the full distribution process without affecting the real index.

python -m twine upload --repository testpypi dist/*

Twine will prompt you for your username and password. For the password, you should generate an API token from your TestPyPI account settings and use that instead of your actual password.

Uploading to the real PyPI:

Once you've confirmed everything looks good on TestPyPI, you can upload to the official index.

python -m twine upload dist/*

Again, Twine will prompt for your credentials. Use your real PyPI username and an API token generated from your pypi.org account.

Step 5: Verify and Install

Congratulations! Your package is now live. You can navigate to https://pypi.org/project/my-cool-package/ to see your project's page.

Now, you (and anyone else in the world) can install it:

pip install my-cool-package

Automating with a Makefile

To make this process repeatable, you can create a simple Makefile in your project root:

.PHONY: clean build publish

clean:
	rm -rf build dist *.egg-info

build:
	python -m build

publish:
	python -m twine upload dist/*

publish-test:
	python -m twine upload --repository testpypi dist/*

Now, your workflow becomes a series of simple commands:

1. make clean
2. make build
3. make publish-test (to test)
4. make publish (for real)

Conclusion

Packaging and distributing Python code has never been more straightforward. By using a well-configured pyproject.toml and the standard build and twine tools, you can easily share your work with the global Python community. This standardized process ensures that your package is easy to maintain and for others to install and use.