Modern Python Dependency Management: Pip vs. Poetry vs. PDM

Managing dependencies in Python has historically been a source of frustration. The traditional workflow of pip and requirements.txt works, but it lacks features like lock files for deterministic builds and robust dependency resolution. In recent years, modern tools like Poetry and PDM have emerged to solve these problems.

Let's compare the classic approach with these new contenders.

The Classic: venv + pip + requirements.txt

This is the workflow most Python developers first learn.

1. Create a virtual environment: python -m venv .venv
2. Activate it: source .venv/bin/activate
3. Install packages: pip install requests
4. Freeze dependencies: pip freeze > requirements.txt

Strengths:

• Built-in: venv is part of the standard library, and pip is included with most Python distributions. It's available everywhere.
• Simple: For small projects, this workflow is straightforward and easy to understand.

Weaknesses:

• No Lock File: requirements.txt often contains top-level dependencies (e.g., requests==2.31.0). It doesn't pin the versions of sub-dependencies. This means that two developers installing from the same requirements.txt could get different versions of underlying packages, leading to the infamous "it works on my machine" problem.
• Manual Environment Management: You are responsible for creating and activating the virtual environment.
• Poor Dependency Resolution: pip has a basic resolver that can sometimes fail to find a compatible set of packages for complex projects.

Poetry: The All-in-One Solution

Poetry is a tool that aims to manage all aspects of your project: dependency management, packaging, and publishing.

It uses the pyproject.toml file to define project metadata and dependencies.

Workflow:

1. Initialize a project: poetry new my-project or poetry init in an existing one.
2. Add a dependency: poetry add requests. This does three things:
   • Finds a compatible version of requests.
   • Adds it to pyproject.toml.
   • Updates the poetry.lock file with the exact versions of requests and all its sub-dependencies.
3. Install dependencies: poetry install. This installs from the poetry.lock file, guaranteeing a deterministic build.

Strengths:

• True Dependency Locking: The poetry.lock file ensures that every developer and every deployment gets the exact same environment.
• Excellent Dependency Resolver: Poetry has a more advanced dependency resolver than pip, which can handle complex dependency graphs more gracefully.
• Integrated Tooling: It manages virtual environments for you automatically and includes tools for building and publishing your package to PyPI.

Weaknesses:

• Opinionated: Poetry manages virtual environments for you, but it stores them in a central cache directory by default, which can be confusing for developers used to seeing a .venv folder in their project root.

PDM: The Modern PEP-Compliant Tool

PDM (Python Development Master) is a newer tool that embraces the latest Python packaging standards (PEPs) and offers more flexibility than Poetry.

Like Poetry, it uses pyproject.toml and a lock file (pdm.lock).

Workflow:

1. Initialize a project: pdm init
2. Add a dependency: pdm add requests
3. Install dependencies: pdm install

Strengths:

• PEP-Compliant: PDM closely follows the latest packaging PEPs, such as PEP 582 (which proposes a __pypackages__ directory, eliminating the need for virtual environments) and PEP 621 (for standardizing project metadata).
• Flexible: It doesn't manage virtual environments by default (though it can). Its preferred approach with PEP 582 is more modern, though not yet a formal standard.
• Fast: It has a high-performance dependency resolver and can install packages in parallel.

Weaknesses:

• Newer and Less Established: While it's gaining popularity, it doesn't have the same large user base or history as Poetry or Pip.
• PEP 582 is Not Standard: Its default __pypackages__ approach is not yet an official part of Python, which can cause issues with some tools that expect a traditional virtual environment.

Comparison Table

Feature	pip + venv	Poetry	PDM
Locking	No (or via pip-tools)	Yes (poetry.lock)	Yes (pdm.lock)
Configuration	requirements.txt	pyproject.toml	pyproject.toml
Venv Management	Manual	Automatic (in a central cache)	Optional (prefers PEP 582)
Strengths	Built-in, simple for small projects	All-in-one, robust resolver	PEP-compliant, flexible, fast

Recommendation for 2024

• For small scripts or simple projects, the classic pip and venv workflow is still perfectly fine.
• For libraries and applications that will be shared or deployed, Poetry is the recommended choice. It provides a stable, all-in-one solution that solves the core problems of dependency management and packaging with a mature and reliable tool.
• If you are a packaging enthusiast who wants to be on the cutting edge of Python standards and value flexibility, PDM is an excellent and powerful tool to explore.

By adopting a modern tool like Poetry or PDM, you can eliminate a whole class of dependency-related problems, making your development process smoother and your deployments more reliable.