We welcome contributions to Rust! We use Rust to implement the Pants engine in a performant, safe, and ergonomic way.
Still learning Rust? Ask to get added to reviews
We'd be happy to ping you on Rust changes we make for you to see how Rust is used in the wild. Please message us on the #engine channel in [Slack](🔗) to let us know your interest.
Recommendation: share your plan first
Because changes to Rust deeply impact how Pants runs, it is especially helpful to share any plans to work on Rust before making changes. Please message us on [Slack](🔗) in the #engine channel or open a [GitHub issue](🔗).
## Code organization
The code for the top-level Pants Rust crate lives in `
src/rust/engine`. The top-level `
Cargo.toml` file at `
src/rust/engine/Cargo.toml` defines a cargo workspace containing a number of other subcrates, which live in subdirectories of `
src/rust/engine`. Defining multiple subcrates in this way allows changes affecting one subcrate to avoid affecting other subcrates and triggering more recompilation than is necessary.
Several of the particularly important subcrates are:
graph`: the core of Pants's rule graph implementation.
ui`: the dynamic UI.
sharded_lmdb`: custom wrappers around the `
lmdb` crate, which provides bindings to [lmdb](🔗).
fs`: manipulating the filesystem.
process_execution`: running local and remote processes.
## Rust <-> Python interaction
Pants is best conceptualized as a Python program that makes frequent foreign function interface (FFI) calls into Rust code.
The top-level `
engine` Rust crate gets compiled into a library named `
native_engine.so`, which Python code knows how to interact with. We use the Rust [cpython](🔗) crate to manage foreign function interaction.
The C FFI functions that Rust code exposes as a public interface live in `
src/rust/engine/src/externs/interface.rs`. On the Python side, `
src/python/pants/engine/internals/native_engine.pyi` provides type hints for the functions and classes provided by Rust.
Rust can also invoke Python functions and object constructors thanks to the [cpython](🔗) crate. Most of our helper code lives in `
We are planning to port additional functionality from Python to Rust, generally for performance reasons.
## Common commands
Rather than using a global installation of Cargo, use the `
To check that the Rust code is valid, use `
./cargo check`. To check that it integrates correctly with Pants' Python code, use `
MODE=debug ./pants ...` as usual (which will `
compile` first, and is slower than `
MODE=debug` when iterating on Rust
As described in [Setting up Pants](🔗), we default to compiling Rust in release mode, rather than debug mode.
When working on Rust, you typically should set the environment variable `
MODE=debug` for substantially faster compiles.
### Run tests
To run tests for all crates, run:
To run for a specific crate, such as the `
fs` create, run:
To run for a specific test, use Cargo's filtering mechanism, e.g.:
Tip: enabling logging in tests
When debugging, it can be helpful to capture logs with [`
To enable logging:
env_logger = "..."` to `
dev-dependencies` in the crate's `
Cargo.toml`, replacing the `
...` with the relevant version. Search for the version used in other crates.
At the start of your test, add `
let _logger = env_logger::try_init();`.
Add log statements wherever you'd like using `
log::info!()` et al.
Run your test with `
RUST_LOG=trace ./cargo test -p $crate test_name -- --nocapture`, using one of `
debug`, or `
To run in lint mode, add `
### Run Clippy
## The `
fs_util` is a utility that enables you to interact with `
Snapshot`s from the command line. You can use it to help debug issues with snapshotted files.
To build it, run this from the root of the repository:
That will produce `
To inspect a particular snapshot, you'll need to tell `
fs_util` where the storage is and the digest and length of the snapshot to inspect. You can use the `
--local-store-path` flag for that.
For example, this command pretty prints the recursive file list of a directory through the directory subcommand.
Pass the `
--help` flag to see other ways of using `
fs_util`, along with its subcommands. Each subcommand can be passed the `