Introducing Pants 2.7: Python tool lockfiles, Yapf, Docker, and ./pants peek

Photo by Joakim Honkasalo / Unsplash

We're pleased to announce Pants 2.7.0, the latest release of Pants, the scalable and ergonomic build system.

To update, set pants_version = "2.7.0" in your pants.toml. See upgrade tips.

Python tool lockfiles

Pants now pins every dependency used when installing Python tools like Black, Pytest, and MyPy. This makes your builds more stable and more secure from supply chain attacks (by using pip's checksum validation for downloaded files).

Each tool comes with a default lockfile. If you change the tool's version or add plugins, you can generate a new lockfile by running ./pants generate-lockfiles:

$ ./pants generate-lockfiles
19:00:39.29 [INFO] Wrote lockfile for the resolve `flake8` to 3rdparty/flake8_lockfile.txt
19:00:39.30 [INFO] Wrote lockfile for the resolve `pytest` to 3rdparty/pytest_lockfile.txt

Whenever you change the tool's version or extra requirements, Pants will remind you to regenerate the lockfile.

(This new feature lays the foundation for an upcoming revamp of lockfiles for your user code, including supporting using different lockfiles for different portions of your code, and generating the files for you!)

Experimental Docker support

Pants can now create Docker images for you that include any of Pants' supported artifact types, including archives, PEX binary files, and Python wheel files!

For example, given this Dockerfile and Pants metadata:

project/Dockerfile

FROM python:3.8
ENTRYPOINT ["/bin/my_app"]
COPY project/app.pex /bin/my_app

project/BUILD

pex_binary(name="app", entry_point="app.py")
docker_image(name="docker_app", dependencies=[":app"])

Pants will first build your PEX binary, and then insert it into the Dockerfile's context.

$ ./pants package project/Dockerfile
18:07:29.66 [INFO] Completed: Building project/app.pex
18:07:31.83 [INFO] Completed: Building docker image docker_app:latest
18:07:31.83 [INFO] Built docker image: docker_app:latest
To try out the image interactively:
    docker run -it --rm docker_app:latest [entrypoint args...]
To push your image:
    docker push docker_app:latest
$ docker run -it --rm docker_app:latest
Hello, Docker!

Pants also now supports Hadolint to lint your Dockerfiles. This gives you a consistent interface to run all your project's linters by running ./pants lint: Pants will run all activated linters—selected from Hadolint, Shellcheck, Black, Flake8, isort, Pylint, and more—in parallel.

See https://www.pantsbuild.org/v2.7/docs/docker to get started, and stay tuned for a blog post from our newest maintainer Andreas Stenius about adding this highly requested feature.

Disclaimer: this backend is still experimental and may change based on user feedback. Share what you think in Slack!

Yapf Python formatter

Pants now supports the Yapf formatter, in addition to already supporting Black, isort, Docformatter, and Shfmt!

For example, if we start with this code:

x  =     3 +  3
print( x )

You can run ./pants fmt and Yapf will reformat the file to:

x = 3 + 3
print(x)

Stay tuned for a blog post from Pants Contributor Alexey Tereshenkov about his experience writing his first Pants plugin.

./pants peek

You can now programmatically query targets to see what metadata you're setting for your code, getting back a JSON representation of your metadata.

$ ./pants peek helloworld/app.py
[
  {
    "address": "helloworld/app.py",
    "target_type": "python_library",
    "dependencies": null,
    "interpreter_constraints": ["==3.7.*"],
    "skip_black": false,
    ...
    "skip_mypy": false,
    "sources": [
      "app.py",
    ],
  },
]

For example, Pants supports incremental tool adoption where you can say to skip running the tool on some code by setting fields like skip_flake8=True. Now, you can combine ./pants peek with jq to find all targets where you set skip_flake8=True:

$ ./pants peek :: | jq -r '.[] | select(.skip_flake8 == true) | .["address"]'
helloworld/greet:lib
helloworld/greet:tests
helloworld/util:lib

Likewise, you could set up Pants to run only part of your test suite, and then use ./pants peek and jq to find all tests that set skip_tests=True and pipe those into your old test runner. This allows you to incrementally switch to Pants's test support.

Other changes

• Cache reuse is higher for resolving third-party dependencies.
• python_distribution has a new, more flexible entry_points field, which replaces the now deprecated .with_binaries() mechanism.
• Pants can now build existing setup.py files for you.
• macOS ARM (Apple Silicon) is better supported when installing native tools like protoc.
• ASDF support was added to [python-setup].interpreter_search_paths.
• Pants now uses PEP 561, meaning that plugin authors will benefit from type hints.

See the full changelog for more changes.

Upcoming in Pants 2.8

We've been working on adding support for other languages, starting with Go and Java. We hope to have made enough meaningful progress for alpha/beta testing.

Other upcoming improvements in Pants 2.8:

• TOML dictionaries will work for dict options.
• Google Cloud Functions, similar to the AWS Lambda support.
• pex_binary has a script field to allow setting console scripts.
• [coverage-py].fail_under option to fail if test coverage is too low.
• An experimental_shell_command target to support scripting of build steps.

--

Try out our example repository, and let us know what you think in Slack!