The standard Python packaging mechanism, setuptools, expects a build script called
setup.py that describes how to package your project into a distribution: an archive that is uploaded to a package index such as PyPI, and can be installed by pip.
Much of the data you would normally put in a
setup.py file is already known to Pants, so it's convenient to generate these files instead of maintaining them manually for each distributable project.
setup-py goal generates a
setup.py file, based on the information available in your BUILD files. It can also run
setup.py commands to package your code. For example, you can run the
bdist_wheel command to create a wheel.
See here for more information about the
setup.py file and the commands you can run with it.
Benefit of Pants: multiple distributions in the same repository
Typically, repositories without sophisticated tooling and up with a single
setup.py, which includes the entire repo in the distribution.
Pants makes it ergonomic to create multiple distributions from the same repository. Because Pants understands your code's true dependencies, each distribution will only be built with the exact code and metadata that it needs to work.
A Pants repo typically consists of many
python_library targets in BUILD files spread across the codebase. To build distributions, Pants must determine which libraries are bundled into each distribution.
In the extreme case, you could have one distribution per library. But
python_library targets tend to be fine-grained, representing perhaps a single Python package, whereas distributions are coarser grained, representing a larger project or piece of functionality. A codebase might have hundreds or thousands of
python_library targets, but publishing and consuming that many distributions isn't practical.
So in practice, multiple libraries are typically bundled into a single distribution.
This is achieved by marking some libraries as providing a distribution:
python_library( name="mydist_library", dependencies=[ ... ], provides=setup_py( name="mydist", version="2.21.0", description="An example library built with Pants.", author="Pantsbuild", classifiers=[ "Programming Language :: Python :: 3.7", ], namespace_packages=['mydist'], ), )
Obviously, this "exported" library is bundled into the corresponding distribution. But which other libraries are bundled along with it?
Pants applies the following algorithm to determine this:
Given an exported target E, take all the libraries in the transitive dependency closure of E. Some of those libraries may be bundled in E itself, but others may be bundled in some other exported target, E' (in which case Pants will correctly add a requirement on E' in E's metadata).
For each library, L, the exported library in whose artifact L's code is published is chosen to be:
- An exported library that depends, directly or indirectly, on L.
- Is L's closest filesystem ancestor among those satisfying 1.
If there are multiple such exported libraries at the same degree of ancestry, the ownership
is ambiguous and an error is raised. If there is no exported library that depends on T
and is its ancestor, then there is no owner and an error is raised.
This algorithm implies that all libraries bundled in an exported library's distribution must be below it in the filesystem. It also guarantees that a library is only bundled into a single distribution, avoiding the collisions caused by naively allowing the same code to be published in multiple distributions.
Once you've decided which targets are exported, and which libraries each such target exports, you may want to add metadata to the corresponding distributions.
setup.py metadata is inferred by Pants from your code and its dependencies. Other metadata needs to be provided explicitly. In Pants, as shown above, you do so on a
python_library target through the
You can use almost any keyword argument accepted by
setup.py in the
However, you cannot use
packages because Pants will generate these for you, based on the data derived from your code and dependencies.
.with_binaries()to register entry points from
You can use
python_binarytargets to define new entry points on your package. The
python_binarytarget must explicitly set the field
To add a new entry point to your package, call the method
.with_binaries(), like this:
provides=setup_py( ... ).with_binaries( my_command=":binary_target", )
This will register a new
my_command, which has the
You may register multiple binary targets by using a dictionary, such as
Note: you should still include the binary target in the
dependenciesfield. This is done automatically in Pants 2.0+.
To create a directory with all of your package's code and a generated
setup.py file, run
$ ./pants setup-py project:my_library_distribution
setup.py will have its requirements set to include the 3rdparty dependencies of the code bundled in the distribution, and any other distributions from your own repo. For example, if distribution D1 contains code that has a dependency on some library L, and that library is published in distribution D2, then D1's requirements will include a dependency on D2. In other words, Pants does the right thing...
You can also pass arguments through to
--. E.g., to build a wheel you can do:
$ ./pants setup-py project:my_library_distribution -- bdist_wheel --python-tag=py36.py37
How to upload your distributions to a package index
Pants does not have a mechanism to upload your distributions. Instead, run
setup-pyas described above to create the asset, such as the wheel, then use a tool like Twine to upload your package.
Updated about 3 years ago