|jazzband| |pypi| |pyversions| |buildstatus-travis| |buildstatus-appveyor| |codecov|

==================================
pip-tools = pip-compile + pip-sync

A set of command line tools to help you keep your pip-based packages fresh,
even when you've pinned them. You do pin them, right? (In building your Python application and its dependencies for production, you want to make sure that your builds are predictable and deterministic.)

.. image:: https://github.com/jazzband/pip-tools/raw/master/img/pip-tools-overview.png
:alt: pip-tools overview for phase II

.. |buildstatus-travis| image:: https://img.shields.io/travis/jazzband/pip-tools/master.svg?logo=travis
:alt: Travis CI build status
:target: https://travis-ci.org/jazzband/pip-tools
.. |buildstatus-appveyor| image:: https://img.shields.io/appveyor/ci/jazzband/pip-tools/master.svg?logo=appveyor
:alt: AppVeyor build status
:target: https://ci.appveyor.com/project/jazzband/pip-tools
.. |codecov| image:: https://codecov.io/gh/jazzband/pip-tools/branch/master/graph/badge.svg
:alt: Coverage
:target: https://codecov.io/gh/jazzband/pip-tools
.. |jazzband| image:: https://jazzband.co/static/img/badge.svg
:alt: Jazzband
:target: https://jazzband.co/
.. |pypi| image:: https://img.shields.io/pypi/v/pip-tools.svg
:alt: PyPI version
:target: https://pypi.org/project/pip-tools/
.. |pyversions| image:: https://img.shields.io/pypi/pyversions/pip-tools.svg
:alt: Supported Python versions
:target: https://pypi.org/project/pip-tools/
.. _You do pin them, right?: http://nvie.com/posts/pin-your-packages/

Installation

Similar to pip, pip-tools must be installed in each of your project's
virtual environments_:

.. code-block:: bash

$ source /path/to/venv/bin/activate
(venv)$ pip install pip-tools

Note: all of the remaining example commands assume you've activated your
project's virtual environment.

.. _virtual environments: https://packaging.python.org/tutorials/installing-packages/#creating-virtual-environments

Example usage for pip-compile

The pip-compile command lets you compile a requirements.txt file from
your dependencies, specified in either setup.py or requirements.in.

Run it with pip-compile or python -m piptools compile. If you use
multiple Python versions, you can run pip-compile as py -X.Y -m piptools compile on Windows and pythonX.Y -m piptools compile on other systems.

pip-compile should be run from the same virtual environment as your
project so conditional dependencies that require a specific Python version,
or other environment markers, resolve relative to your project's
environment.

Note: ensure you don't have requirements.txt if you compile
setup.py or requirements.in from scratch, otherwise, it might
interfere.

Requirements from setup.py

Suppose you have a Flask project, and want to pin it for production.
If you have a setup.py with install_requires=['Flask'], then run
pip-compile without any arguments:

.. code-block:: bash

$ pip-compile
#
# This file is autogenerated by pip-compile
# To update, run:
#
#    pip-compile --output-file requirements.txt setup.py
#
click==6.7                # via flask
flask==0.12.2
itsdangerous==0.24        # via flask
jinja2==2.9.6             # via flask
markupsafe==1.0           # via jinja2
werkzeug==0.12.2          # via flask

pip-compile will produce your requirements.txt, with all the Flask
dependencies (and all underlying dependencies) pinned. You should put
requirements.txt under version control.

Without setup.py

If you don't use setup.py (it's easy to write one_), you can create a
requirements.in file to declare the Flask dependency:

.. code-block:: ini

# requirements.in
Flask

Now, run pip-compile requirements.in:

.. code-block:: bash

$ pip-compile requirements.in
#
# This file is autogenerated by pip-compile
# To update, run:
#
#    pip-compile --output-file requirements.txt requirements.in
#
click==6.7                # via flask
flask==0.12.2
itsdangerous==0.24        # via flask
jinja2==2.9.6             # via flask
markupsafe==1.0           # via jinja2
werkzeug==0.12.2          # via flask

And it will produce your requirements.txt, with all the Flask dependencies
(and all underlying dependencies) pinned. You should put both
requirements.in and requirements.txt under version control.

.. _it's easy to write one: https://packaging.python.org/guides/distributing-packages-using-setuptools/#configuring-your-project

Using hashes

If you would like to use Hash-Checking Mode available in pip since
version 8.0, pip-compile offers --generate-hashes flag:

.. code-block:: bash

$ pip-compile --generate-hashes requirements.in
#
# This file is autogenerated by pip-compile
# To update, run:
#
#    pip-compile --generate-hashes --output-file requirements.txt requirements.in
#
click==6.7 \
    --hash=sha256:29f99fc6125fbc931b758dc053b3114e55c77a6e4c6c3a2674a2dc986016381d \
    --hash=sha256:f15516df478d5a56180fbf80e68f206010e6d160fc39fa508b65e035fd75130b \
    # via flask
flask==0.12.2 \
    --hash=sha256:0749df235e3ff61ac108f69ac178c9770caeaccad2509cb762ce1f65570a8856 \
    --hash=sha256:49f44461237b69ecd901cc7ce66feea0319b9158743dd27a2899962ab214dac1
itsdangerous==0.24 \
    --hash=sha256:cbb3fcf8d3e33df861709ecaf89d9e6629cff0a217bc2848f1b41cd30d360519 \
    # via flask
jinja2==2.9.6 \
    --hash=sha256:2231bace0dfd8d2bf1e5d7e41239c06c9e0ded46e70cc1094a0aa64b0afeb054 \
    --hash=sha256:ddaa01a212cd6d641401cb01b605f4a4d9f37bfc93043d7f760ec70fb99ff9ff \
    # via flask
markupsafe==1.0 \
    --hash=sha256:a6be69091dac236ea9c6bc7d012beab42010fa914c459791d627dad4910eb665 \
    # via jinja2
werkzeug==0.12.2 \
    --hash=sha256:903a7b87b74635244548b30d30db4c8947fe64c5198f58899ddcd3a13c23bb26 \
    --hash=sha256:e8549c143af3ce6559699a01e26fa4174f4c591dbee0a499f3cd4c3781cdec3d \
    # via flask

Updating requirements

To update all packages, periodically re-run pip-compile --upgrade.

To update a specific package to the latest or a specific version use the
--upgrade-package or -P flag:

.. code-block:: bash

# only update the flask package
$ pip-compile --upgrade-package flask

# update both the flask and requests packages
$ pip-compile --upgrade-package flask --upgrade-package requests

# update the flask package to the latest, and requests to v2.0.0
$ pip-compile --upgrade-package flask --upgrade-package requests==2.0.0

You can combine --upgrade and --upgrade-package in one command, to
provide constraints on the allowed upgrades. For example to upgrade all
packages whilst constraining requests to the latest version less than 3.0:

.. code-block:: bash

$ pip-compile --upgrade --upgrade-package 'requests<3.0'

Output File

To output the pinned requirements in a filename other than
requirements.txt, use --output-file. This might be useful for compiling
multiple files, for example with different constraints on flask to test a
library with both versions using tox <https://tox.readthedocs.io/en/latest/>__:

.. code-block:: bash

$ pip-compile --upgrade-package 'flask<1.0' --output-file requirements-flask0x.txt
$ pip-compile --upgrade-package 'flask<2.0' --output-file requirements-flask1x.txt

Or to output to standard output, use --output-file=-:

.. code-block:: bash

$ pip-compile --output-file=- > requirements.txt
$ pip-compile - --output-file=- < requirements.in > requirements.txt

Configuration

You might be wrapping the pip-compile command in another script. To avoid
confusing consumers of your custom script you can override the update command
generated at the top of requirements files by setting the
CUSTOM_COMPILE_COMMAND environment variable.

.. code-block:: bash

$ CUSTOM_COMPILE_COMMAND="./pipcompilewrapper" pip-compile requirements.in
#
# This file is autogenerated by pip-compile
# To update, run:
#
#    ./pipcompilewrapper
#
flask==0.10.1
itsdangerous==0.24        # via flask
jinja2==2.7.3             # via flask
markupsafe==0.23          # via jinja2
werkzeug==0.10.4          # via flask

Workflow for layered requirements

If you have different environments that you need to install different but
compatible packages for, then you can create layered requirements files and use
one layer to constrain the other.

For example, if you have a Django project where you want the newest 2.1
release in production and when developing you want to use the Django debug
toolbar, then you can create two *.in files, one for each layer:

.. code-block:: ini

# requirements.in
django<2.2

At the top of the development requirements dev-requirements.in you use -c requirements.txt to constrain the dev requirements to packages already
selected for production in requirements.txt.

.. code-block:: ini

# dev-requirements.in
-c requirements.txt
django-debug-toolbar

First, compile requirements.txt as usual:

.. code-block:: bash

$ pip-compile
#
# This file is autogenerated by pip-compile
# To update, run:
#
#    pip-compile
#
django==2.1.15
pytz==2019.3              # via django

Now compile the dev requirements and the requirements.txt file is used as
a constraint:

.. code-block:: bash

$ pip-compile dev-requirements.in
#
# This file is autogenerated by pip-compile
# To update, run:
#
#    pip-compile dev-requirements.in
#
django-debug-toolbar==2.1
django==2.1.15            # via django-debug-toolbar
pytz==2019.3              # via django
sqlparse==0.3.0           # via django-debug-toolbar

As you can see above, even though a 2.2 release of Django is available, the
dev requirements only include a 2.1 version of Django because they were
constrained. Now both compiled requirements files can be installed safely in
the dev environment.

To install requirements in production stage use:

.. code-block:: bash

$ pip-sync

You can install requirements in development stage by:

.. code-block:: bash

$ pip-sync requirements.txt dev-requirements.txt

Example usage for pip-sync

Now that you have a requirements.txt, you can use pip-sync to update
your virtual environment to reflect exactly what's in there. This will
install/upgrade/uninstall everything necessary to match the
requirements.txt contents.

Run it with pip-sync or python -m piptools sync. If you use multiple
Python versions, you can also run py -X.Y -m piptools sync on Windows and
pythonX.Y -m piptools sync on other systems.

pip-sync must be installed into and run from the same virtual
environment as your project to identify which packages to install
or upgrade.

Be careful: pip-sync is meant to be used only with a
requirements.txt generated by pip-compile.

.. code-block:: bash

$ pip-sync
Uninstalling flake8-2.4.1:
  Successfully uninstalled flake8-2.4.1
Collecting click==4.1
  Downloading click-4.1-py2.py3-none-any.whl (62kB)
    100% |................................| 65kB 1.8MB/s
  Found existing installation: click 4.0
    Uninstalling click-4.0:
      Successfully uninstalled click-4.0
Successfully installed click-4.1

To sync multiple *.txt dependency lists, just pass them in via command
line arguments, e.g.

.. code-block:: bash

$ pip-sync dev-requirements.txt requirements.txt

Passing in empty arguments would cause it to default to requirements.txt.

If you use multiple Python versions, you can run pip-sync as
py -X.Y -m piptools sync ... on Windows and
pythonX.Y -m piptools sync ... on other systems.

Note: pip-sync will not upgrade or uninstall packaging tools like
setuptools, pip, or pip-tools itself. Use pip install --upgrade
to upgrade those packages.

Other useful tools

• pipdeptree_ to print the dependency tree of the installed packages.

• requirements.in/requirements.txt syntax highlighting:

  • requirements.txt.vim_ for Vim.
  • Python extension for VS Code_ for VS Code.

.. _pipdeptree: https://github.com/naiquevin/pipdeptree
.. _requirements.txt.vim: https://github.com/raimon49/requirements.txt.vim
.. _Python extension for VS Code: https://marketplace.visualstudio.com/items?itemName=ms-python.python