setup-py

The standard Python packaging mechanism, setuptools, expects a build script called setup.py that describes how to package your project into a distribution: an archive that is uploaded to a package index such as PyPI, and can be installed by pip.

Common distribution formats include sdist, a source distribution format, and wheel, a built distribution format.

Much of the data you would normally put in a setup.py file is already known to Pants, so it's convenient to generate these files instead of maintaining them manually for each distributable project.

The setup-py goal generates a setup.py file, based on the information available in your BUILD files. It can also run setup.py commands to package your code. For example, you can run the bdist_wheel command to create a wheel.

See here for more information about the setup.py file and the commands you can run with it.

👍

Benefit of Pants: multiple distributions in the same repository

Typically, repositories without sophisticated tooling and up with a single setup.py, which includes the entire repo in the distribution.

Pants makes it ergonomic to create multiple distributions from the same repository. Because Pants understands your code's true dependencies, each distribution will only be built with the exact code and metadata that it needs to work.

Mapping libraries to distributions

A Pants repo typically consists of many python_library targets in BUILD files spread across the codebase. To build distributions, Pants must determine which libraries are bundled into each distribution.

In the extreme case, you could have one distribution per library. But python_library targets tend to be fine-grained, representing perhaps a single Python package, whereas distributions are coarser grained, representing a larger project or piece of functionality. A codebase might have hundreds or thousands of python_library targets, but publishing and consuming that many distributions isn't practical.

So in practice, multiple libraries are typically bundled into a single distribution.

This is achieved by marking some libraries as providing a distribution:

python_library(
    name="mydist_library",
    dependencies=[
        ...
    ],
    provides=setup_py(
        name="mydist",
        version="2.21.0",
        description="An example library built with Pants.",
        author="Pantsbuild",
        classifiers=[
            "Programming Language :: Python :: 3.7",
        ],
        namespace_packages=['mydist'],
    ),
)

Obviously, this "exported" library is bundled into the corresponding distribution. But which other libraries are bundled along with it?

Pants applies the following algorithm to determine this:

Given an exported target E, take all the libraries in the transitive dependency closure of E. Some of those libraries may be bundled in E itself, but others may be bundled in some other exported target, E' (in which case Pants will correctly add a requirement on E' in E's metadata).

For each library, L, the exported library in whose artifact L's code is published is chosen to be:

1. An exported library that depends, directly or indirectly, on L.
2. Is L's closest filesystem ancestor among those satisfying 1.

If there are multiple such exported libraries at the same degree of ancestry, the ownership
is ambiguous and an error is raised. If there is no exported library that depends on T
and is its ancestor, then there is no owner and an error is raised.

This algorithm implies that all libraries bundled in an exported library's distribution must be below it in the filesystem. It also guarantees that a library is only bundled into a single distribution, avoiding the collisions caused by naively allowing the same code to be published in multiple distributions.

Adding setup.py metadata to BUILD files

Once you've decided which targets are exported, and which libraries each such target exports, you may want to add metadata to the corresponding distributions.

Some important setup.py metadata is inferred by Pants from your code and its dependencies. Other metadata needs to be provided explicitly. In Pants, as shown above, you do so on a python_library target through the provides field.

You can use almost any keyword argument accepted by setup.py in the setup_py() constructor.

However, you cannot use data_files, package_dir, package_data, or packages because Pants will generate these for you, based on the data derived from your code and dependencies.

📘

Use .with_binaries() to register entry points from python_binary targets

You can use python_binary targets to define new entry points on your package. The python_binary target must explicitly set the field entry_point.

To add a new entry point to your package, call the method .with_binaries(), like this:

Python

provides=setup_py(
    ...
).with_binaries(
    my_command=":binary_target",
)

This will register a new console_script named my_command, which has the entry_point from the python_binary target.

You may register multiple binary targets by using a dictionary, such as .with_binaries({"my_command": ":binary_target"}).

Note: you should still include the binary target in the dependencies field. This is done automatically in Pants 2.0+.

Generating a setup.py file

To create a directory with all of your package's code and a generated setup.py file, run ./pants setup-py:

$ ./pants setup-py project:my_library_distribution

The generated setup.py will have its requirements set to include the 3rdparty dependencies of the code bundled in the distribution, and any other distributions from your own repo. For example, if distribution D1 contains code that has a dependency on some library L, and that library is published in distribution D2, then D1's requirements will include a dependency on D2. In other words, Pants does the right thing...

Running setup.py commands

You can also pass arguments through to setup.py using --. E.g., to build a wheel you can do:

$ ./pants setup-py project:my_library_distribution -- bdist_wheel --python-tag=py36.py37

📘

How to upload your distributions to a package index

Pants does not have a mechanism to upload your distributions. Instead, run setup-py as described above to create the asset, such as the wheel, then use a tool like Twine to upload your package.