Photo by Hamish Duncan / Unsplash
Make builds more reliable and save time doing so. The upcoming Pants 2.16 introduces a couple of exciting changes to make Pants safer, faster, and more user-friendly. Here we preview a pair of changes which increase hermeticity.
Recently, Pants got a new launcher binary that has a Python interpreter embedded. This was an important step forward for decoupling running Pants from what's installed on the system. However, there was still work to do; internally Pants still scoured the running machine looking for Python binaries for 2 use-cases:
- For Pants' own ability to run Python scripts as part of the build. Python is an intuitive and cross-platform scripting language. As such it is very convenient to use it for complex scripts (like file locking, parsing complicated text, or ironically, determining a compatible Python interpreter given interpreter constraints).
- For running user Python code. If you enable
pants.backend.pythonone or more Python interpreters need to be found/used to run your code, tools, and tests.
Case 1: A Pants-internal Python
For case 1 where Pants just need some Python interpreter to run scripts, we take a two-pronged approach.
For Docker builds we download and sandbox a Python Build Standalone (the same kind of Python interpreter the new Pants binary launcher uses). Pants no longer dictates that a supported (or any) version of Python is present in the docker image running your build. Additionally, Pants is able to choose which version, meaning we could upgrade to faster CPython versions as they are released.
For local builds, we just use sys.executable (the Python interpreter that's running Pants itself). Eventually, the Pants binary launcher will be the only way to run Pants, meaning a hermetic, version-pinned interpreter will be the one-and-only way Pants is run.
Combining these, the result is Pants completely decoupled from the system to run internal scripts, better hermeticity, and faster runs (by not consulting the system and by using the latest Python version available).
Case 2: A Pants-provided Python
A long-standing request of Pants is to have Pants provide a Python interpreter for running your code, tools, and tests, instead of consulting the system. Consulting the system can be time-consuming, error-prone, and results in builds with reduced hermeticity.
Pants 2.16 has added a mechanism for plugins to provide versions of Python compatible with your code, along with an experimental backend to use pyenv as a provider. When the backend is enabled, for each version of Python that needs to be used based on your interpreter constraints, Pants either uses a hermetic, cached version if one exists, or installs it on-the-fly (roughly takes ~2-3 minutes). This results in faster builds (no more scouring the system) as well as ensuring maximal hermeticity when running your code, your tools, and your tests.
How to preview the new hermetic Pythons
Interested in trying out these features? Upgrade to Pants 2.16 to benefit from a hermetic internal Python. Additionally, for a hermetic Python for your code enable pants.backend.python.providers.experimental.pyenv to your [GLOBAL].backend_packages and start enjoying the benefits. As with any experimental backend, we encourage you to share your feedback on Slack!