How to lock requirements using pip-tools

Simple instructions for locking requirements in a python project using pip-tools

Locking Dependencies with pip-compile

Opinions vary on how one should make use of lock files, depending on whether the project is the main application, or the project is actually a library that is meant to be consumed by an application or another library.

Lock files are unquestionably useful if you build any application. Python has no concept of lock files, equally it can be argued python has no package dependency files at all and that's why we have many options outside the core python team like setup.py, Pipfile, and the most common requirements.txt as a pattern for Pip. This blog will show you how to lock requirements with command pip-compile (provided by pip-tools).

First, we need a clean virtual environment created with virtualenv.

Install the virtualenv package

The virtualenv package is required to create virtual environments. You can install it with pip:

pip install virtualenv

Create the virtual environment

To create a virtual environment, you must specify a path.

For example to create one in the local directory called ‘python3env’, type the following:

virtualenv python3env

Then You need to activate the python environment by running the following command:

Mac OS / Linux

source python3env/bin/activate

Windows

python3env\Scripts\activate

Any python commands you use will now work with your virtual environment.

Install pip-tools and lock requirements

Now you need to activate the virtual environment to install pip-tools

pip install pip-tools

Once the package has been installed, you need to create a requirements.in file. This file is where you define your project's top-level dependencies (similar to pipenv's Pipfile or pyproject.toml in poetry). A basic example might look something like this:

Django==5.0.*
psycopg2
celery>5.1`;

To "lock" these dependencies, you can run:

pip-compile --output-file=requirements.txt requirements.in

This generates the standard requirements.txt file with all dependencies. Here's the file:

# This file is autogenerated by pip-compile with Python 3.10
# by the following command:
#
#    pip-compile
#

amqp==5.2.0
    # via kombu
asgiref==3.7.2
    # via django
billiard==4.2.0
    # via celery
celery==5.3.6
    # via -r requirements.in
click==8.1.7
    # via
    #   celery
    #   click-didyoumean
    #   click-plugins
    #   click-repl
click-didyoumean==0.3.0
    # via celery
click-plugins==1.1.1
    # via celery
click-repl==0.3.0
    # via celery
django==5.0
    # via -r requirements.in
kombu==5.3.5
    # via celery
prompt-toolkit==3.0.43
    # via click-repl
psycopg2==2.9.9
    # via -r requirements.in
python-dateutil==2.9.0.post0
    # via celery
six==1.16.0
    # via python-dateutil
sqlparse==0.4.4
    # via django
typing-extensions==4.10.0
    # via asgiref
tzdata==2024.1
    # via celery
vine==5.1.0
    # via
    #   amqp
    #   celery
    #   kombu
wcwidth==0.2.13
    # via prompt-toolkit
`;

We didn't have pytz in our requirements.in, but it's included in requirements.txt because it is required by django (which the pip-compile is kind enough to output in the file).

MAKEFILE allows you to run make requirements.txt and it will be updated if and only if the requirements.in file has changed since requirements.txt was last generated.

Example:

requirements.txt: requirements.in
pip-compile --upgrade --output-file=$@ requirements.in`;

Installing the dependencies is as simple as:

pip install -r requirements.txt