Pants can create a Google Cloud Function-compatible zip file from your Python code, allowing you to develop your functions in your repository.
FYI: how Pants does this
Under-the-hood, Pants uses the PEX project, to select the appropriate third-party requirements and first-party sources and lay them out in a zip file, in the format recommended by Google Cloud Functions.
Add this to your
[GLOBAL] backend_packages.add = [ "pants.backend.google_cloud_function.python", "pants.backend.python", ]
This adds the new
python_google_cloud_function target, which you can confirm by running
pants help python_google_cloud_function
First, add your Cloud function in a Python file like you would normally do with Google Cloud Functions, such as creating a function
def my_handler_name(event, context) for event-based functions.
Then, in your BUILD file, make sure that you have a
python_sources target with the handler file included in the
sources field. You can use
pants tailor :: to automate this.
python_google_cloud_function target and define the
type fields. The
type should be either
runtime should be one of the values from https://cloud.google.com/functions/docs/concepts/python-runtime. The
handler has the form
handler_file.py:handler_func, which Pants will convert into a well-formed entry point. Alternatively, you can set
handler to the format
# The default `sources` field will include our handler file. python_sources(name="lib") python_google_cloud_function( name="cloud_function", runtime="python38", # Pants will convert this to `project.google_cloud_function_example:example_handler`. handler="google_cloud_function_example.py:example_handler", type="event", )
def example_handler(event, context): print("Hello Google Cloud Function!")
Pants will use dependency inference based on the
handler field, which you can confirm by running
pants dependencies path/to:cloud_function. You can also manually add to the
You can optionally set the
output_path field to change the generated zip file's path.
filestargets will not be included in the built Cloud Function because filesystem APIs like
open()would not load them as expected. Instead, use the
resourcestarget. See Assets and archives for further explanation.
pants package on your
python_google_cloud_function target to create a zipped file.
$ pants package project/:cloud_function Wrote dist/project/cloud_function.zip Handler: handler
Running from macOS and failing to build?
Cloud Functions must run on Linux, so Pants tells PEX and Pip to build for Linux when resolving your third party dependencies. This means that you can only use pre-built wheels (bdists). If your project requires any source distributions (sdists) that must be built locally, PEX and pip will fail to run.
If this happens, you must either change your dependencies to only use dependencies with pre-built wheels or find a Linux environment to run
You can use any of the various Google Cloud methods to upload your zip file, such as the Google Cloud console or the Google Cloud CLI.
You must specify the handler as
handler. This is a re-export of the function referred to by the
handler field of the target.
In the rare case where you need access to PEX features, such as dynamic selection of dependencies, a PEX file created by
pex_binary can be used as a Google Cloud Function package directly. A PEX file is a carefully constructed zip file, and can be understood natively by Google Cloud Functions. Note: using
pex_binary results in larger packages and slower cold starts and is likely to be less convenient than using
The handler of a
pex_binary is not re-exported at the fixed
main.handler path, and the Google Cloud Function handler must be configured as the
__pex__ pseudo-package followed by the handler's normal module path (for instance, if the handler is in
some/module/path.py within a source root, then use
__pex__.some.module.path). This may require being configured via
__pex__ pseudo-package ensures dependencies are initialized before running any of your code.
python_sources() pex_binary( name="gcf", entry_point="gcf_example.py", # specify an appropriate platform(s) for the targeted GCF runtime (complete_platforms works too) platforms=["linux_x86_64-cp39-cp39"], )
def example_handler(event, context): print("Hello GCF!")
pants package project:gcf, and upload the resulting
project/gcf.pex to Google Cloud Functions. You will need to specify the handler as
example_handler and set
project is a source root).
Pants has implemented a new way to package Google Cloud Functions in 2.17, which is now the default in 2.18, resulting in smaller packages and faster cold starts. This involves some changes:
- In Pants 2.16 and earlier, Pants used the Lambdex project. First, Pants would convert your code into a Pex file and then use Lambdex to adapt this to be better understood by GCF by adding a shim handler. This shim handler first triggers the Pex initialization to choose and unzip dependencies, during initialization.
- In Pants 2.17, the use of Lambdex was deprecated, in favour of choosing the appropriate dependencies ahead of time, as described above, without needing to do this on each cold start. This results in a zip file laid out in the format recommended by GCF, and includes a re-export of the handler.
- In Pants 2.18, the new behaviour is now the default behaviour.
- In Pants 2.19, the old Lambdex behaviour will be entirely removed.
When upgrading to Pants 2.18, some changes may be required:
- If you already use Pants 2.17 and set
layout = "zip"in the
pants.toml, you already use the new behaviour: nice one! All you need to do is delete the whole
- If you use Pants 2.16 or earlier, or use Pants 2.17 with
layout = "lambdex", upgrading will change how these targets are built. To migrate, we suggest you first migrate to using
layout = "zip"in Pants 2.17, by following its instructions, and upgrade to Pants 2.18 after that.
Updated 2 months ago