Pants uses the popular Pytest test runner to run Python tests. You may write your tests in Pytest-style, unittest-style, or mix and match both.
Benefit of Pants: runs each test target in parallel
Each test target gets run as a separate process, which gives you fine-grained caching and better parallelism. Given enough cores, Pants will be able to run all your tests at the same time.
This also gives you fine-grained caching. If you run
./pants test ::, and then you only change one target, then only that target will need to rerun and the others will use the cache.
The Pytest community has developed hundreds of plugins to add additional functionality.
To install any plugins, add the Pip requirement string to
pytest_plugins in the scope
[pytest], like this:
[pytest] version = "pytest>=5.4" pytest_plugins.add = [ "pytest-django>=3.9.0,<4", "pytest-rerunfailures==9.0", ]
Testing Python 2 code? Use Pytest 4.x
By default, Pants uses Pytest 5.x, which only supports Python 3 code. If you need to run Python 2 tests, set the option
versionin the scope
We do not recommend using this plugin because its concurrency conflicts with Pants' own parallelism. Using Pants will bring you similar benefits to
pytest-xdistalready: Pants will run each test target in parallel.
Tip: plugins for better Pytest output
To pass arguments to Pytest, put them at the end after
--, like this:
$ <<pantscmd>> test project/app_test.py -- -k test_function1 -vv -s
You can also use the
args option in the
[pytest] scope, like this:
[pytest] args = ["-vv"]
Tip: some useful Pytest arguments
See https://docs.pytest.org/en/latest/usage.html for more information.
-k expression: only run tests matching the expression.
-v: verbose mode.
-s: always print the stdout and stderr of your code, even if a test passes.
How to use Pytest's
You must run
<<pantscmd>> test --debugfor this to work properly. See the next section "Running tests interactively" for more information.
Because Pants runs multiple test targets in parallel, you will not see your test results appear on the screen until the test has completely finished. This means that you cannot use debuggers normally; the breakpoint will never show up on your screen and the test will hang indefinitely (or timeout, if timeouts are enabled).
Instead, if you want to run a test interactively—such as to use a debugger like
pdb—run your tests with
<<pantscmd>> test --debug. For example:
def test_debug(): import pdb; pdb.set_trace() assert 1 + 1 == 2
$ <<pantscmd>> test --debug test_debug_example.py ===================================================== test session starts ===================================================== platform darwin -- Python 3.6.10, pytest-5.3.5, py-1.8.1, pluggy-0.13.1 rootdir: /private/var/folders/sx/pdpbqz4x5cscn9hhfpbsbqvm0000gn/T/.tmpn2li0z plugins: cov-2.8.1, timeout-1.3.4 collected 6 items test_debug_example.py >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PDB set_trace (IO-capturing turned off) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > /private/var/folders/sx/pdpbqz4x5cscn9hhfpbsbqvm0000gn/T/.tmpn2li0z/test_debug_example.py(11)test_debug() -> assert 1 + 1 == 2 (Pdb) 1 + 1 2
You may only run one test target at a time when using
--debugto force re-running a test
Normally, Pants will cache test results. If you run
<<pantscmd>> test app_test.py, then run the same command, Pants will nearly instantly give you the result from its cache.
<<pantscmd>> test --debugwill not use the cache. This can be helpful, for example, to check if a test is flaky.
ipdbintegrates IPython with the normal
pdbdebugger for enhanced features like autocomplete and improved syntax highlighting.
ipdbis very helpful when debugging tests.
To be able to access
ipdbwhen running tests, add this to your
[pytest] pytest_plugins.add = ["ipdb"]
Then, you can use
import ipdb; ipdb.set_trace()in your tests.
Pants can cancel tests which take too long. This is useful to prevent tests from hanging indefinitely.
To add a timeout for a particular
python_tests target, set the
timeout field to an integer value of seconds, like this:
python_tests( name="tests", dependencies=[ ":lib", ], timeout=120, # seconds. )
You can also set a default value and a maximum value in
[pytest] timeout_default = 60 timeout_maximum = 600
If a target sets its
timeout higher than
--pytest-timeout-maximum, Pants will use the value in
Tip: temporarily ignoring timeouts
When debugging locally, such as with
pdb, you might want to temporarily disable timeouts. To do this, set
$ ./pants test project/app_test.py --no-pytest-timeouts
Pytest can generate JUnit XML result files. This allows you to hook up your results, for example, to dashboards.
To save JUnit XML result files, set the option
junit_xml_dir in the
[pytest] scope, like this:
[pytest] junit_xml_dir = "dist/pytest_results"
You may also want to set the option
junit_family in the
[pytest] scope to change the format. Run
<<pantscmd>> help-advanced pytest for more information.
# Run all tests in the repository. <<pantscmd>> test :: # Run all the tests in this target. <<pantscmd>> test helloworld/util:test # Run just the tests in this file. <<pantscmd>> test helloworld/util/lang_test.py # Run just one test. <<pantscmd>> test helloworld/util/lang_test.py --pytest-args='-k test_language_translator'
Updated over 3 years ago