Python 3rdparty Pattern
In general, we use the 3rdparty idiom to organize dependencies on code from outside the source tree. This document describes how to make this work for Python code.
Your Python code can pull in code written elsewhere. Pants fetches code via a library that uses pip-style specifications (name and version-range).
To keep all of your code depending on the same versions of third-party
Python artifacts, you might use the idiom of keeping them in a directory
3rdparty/python. If your organization has many such
dependencies, you might arrange them in several directories: this can
ease later "git detective work" when finding out who changed a version.
(Pants itself doesn't have many Python dependencies; thus, we haven't split its
into many directories.)
To define some third-party dependencies, use a
python_requirements in your
file and make a pip
requirements.txt file in the same directory.
3rdparty/python/BUILD file might look like:
ansicolors==1.1.8 beautifulsoup4>=4.6.0,<4.7 coverage>=4.5,<4.6 dataclasses==0.6 docutils>=0.15,<0.17 fasteners==0.15.0 # The MyPy requirement should be maintained in lockstep with the requirement the Pants repo uses # for the mypy task since it configures custom MyPy plugins. That requirement can be found via: # # ./pants \ # --backend-packages=pants.contrib.mypy \ # options \ # --output-format=json \ # --scope=mypy \ # --name=version \ # | jq -r '."mypy.version".value' # mypy==0.770 Markdown==2.1.1 packaging==20.3 parameterized==0.6.1
python_requirements defines a named target for each line in the
requirements.txt line. For example, a line like
requirements.txt defines a target named
ansicolors that pulls in
ansicolors version 1.0.2.
python_requirement_library and python_requirement:
python_requirement_library( name='beautifulsoup', requirements=[ python_requirement(name='beautifulsoup', requirement='BeautifulSoup==3.2.0'), ])
Your Code's BUILD File
In your code's
BUILD file, introduce a dependency on the
python_library( dependencies=[ '3rdparty/python:ansicolors', ], )
Then in your Python code, you can
import from that package:
from colors import green
Managing dependencies for multiple platforms
If you're building a python binary for use on multiple platforms, you might have 3rd-party
dependencies that rely on platform-specific code. In addition to specifying the platforms
with which your binary is intended to be compatible in the
platforms field of your
python_binary target, you will need to make
wheel files for each package
and platform available at build time.
Pants will use the explicitly specified
platforms field of your
target if set for both itself and its dependencies, or will otherwise fall back to the
python-setup.platforms option value.
Pants will look for those files in the location specified in the
in pants.toml. It can understand either a simple local directory of .whl files or a "find links"-friendly
webpage of links formatted like so:
If you opt for the local directory method under version control, you may want to use git-lfs or similar to avoid storing large binaries in your repository. If you opt for a hosted solution, Github pages may be helpful.