# Write Integration Tests for a Rust CLI

Yes. An agent writes assert_cmd integration tests and snapshot tests for your Rust CLI, runs cargo test in its sandbox and opens a pull request.

[Run this on Rush](https://cloud.getrush.ai/run/write-cli-tests/rust-cli/), in a Project that has your repository.

## What the agent does

The agent clones the repository into its own cloud sandbox and reads the command surface first. In a CLI like ajeetdsouza/zoxide (version 0.10.0, edition 2024, `rust-version = "1.88.0"`), `src/cmd/cmd.rs` defines the clap subcommands: `add` (one or more paths and an optional `--score`), `query` (keywords plus `--all`, `--interactive`, `--list`, `--score`, `--exclude` and `--base-dir`, with `--interactive` and `--list` declared as conflicting), `remove`, `edit`, `import` (from `atuin`, `autojump`, `fasd`, `z`, `z.lua` or `zsh-z`, with a global `--merge`) and `init` (a shell, `--no-cmd`, `--cmd` defaulting to `z`, and `--hook` of `none`, `prompt` or `pwd`). `assert_cmd`, `tempfile` and `rstest` are already dev-dependencies, so nothing new is needed to start.

Then it finds out what runs under a plain `cargo test`. The repository's tests are real but mostly behind a feature: `tests/completions.rs` starts with `#![cfg(feature = "nix-dev")]`, and the shell tests in `src/shell.rs` are `#[cfg(feature = "nix-dev")]` too. They render every init template across combinations of command name, hook, echo and symlink resolution and check each one with the real shell and with linters such as `shellcheck` and `shfmt`. CI runs them through `just lint test`, which calls `cargo nextest run --all-features` inside `nix-shell`. Without that feature and those shells, the subcommands' behaviour on a real database is untested.

It writes integration tests under `tests/` that run the built binary with `assert_cmd::Command::cargo_bin("zoxide")`. Each test points `_ZO_DATA_DIR` at a `tempfile::TempDir`, which matters twice over: `src/config.rs` requires that path to be absolute, and without it the tests would write into the real data directory. Because `query` filters out paths that no longer exist on disk, each test creates real directories inside the temp dir before adding them. The cases cover `add` then `query` returning the best match, `query --list --score` printing every entry with its score, `--exclude` dropping the current directory, `remove` deleting an entry, `--interactive` and `--list` together failing with clap's conflict error, and each importer reading a fixture file through its own variable (`_Z_DATA` for z, `_ZL_DATA` for z.lua) with and without `--merge`.

For each shell's `init` output it adds snapshot tests: one snapshot per shell and per `--hook` value, plus `--cmd` and `--no-cmd` variants, so a template edit in `templates/*.txt` shows up as a reviewable diff. The repository has no snapshot library, so adding one such as `insta` is a new dev-dependency, and the agent names it as such; the alternative is expected output files under `tests/` compared by hand. It runs `cargo test` until it passes, and `cargo clippy --all-targets -- -D warnings` because the `lint` recipe in the `justfile` does the same.

## Start it with one API call

`POST https://api.prix.dev/o/{org}/sessions`

```json
{
  "agent": "claude",
  "input": "Write integration tests with assert_cmd and tempfile for this Rust CLI's add, query, remove and import commands, and snapshot tests for each shell's init script. Read the clap definitions first and list every subcommand, flag and flag conflict. Run the built binary in each test, point the data directory environment variable at a temporary directory so no test touches real user data, and create real directories for any command that checks paths exist. Cover each importer with a small fixture file in the format it reads, with and without --merge. Snapshot the init output for every shell and every hook value, plus the command-name options. If snapshotting needs a new dev-dependency, say which and why. Note which existing tests only run behind a cargo feature. Run cargo test and cargo clippy --all-targets -- -D warnings until both pass, and open a pull request.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
```

1. **Get an API key.** In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
2. **Send the request.** POST the body in request.json to https://api.prix.dev/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
3. **Read what came back.** Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

## What comes back

What comes back is a pull request on its own branch. For a CLI like zoxide, the diff adds files under `tests/` (one per subcommand or a shared helper module plus test files), fixture files for each import format, a snapshot folder with one file per shell and option combination, and, if the agent chose a snapshot crate, one line in `[dev-dependencies]` in `Cargo.toml` and the matching `Cargo.lock` change. The description maps each test to the subcommand and flag it covers and lists the existing feature-gated tests it left alone.

The session transcript keeps every `cargo test` and clippy run with its output, so the reviewer can see which cases failed first and what was fixed. The pull request's checks run the repository's own workflow, which for zoxide is `just lint test` on its CI matrix, running the new tests beside the shell tests.

Before merging, a reviewer reads the snapshots, since they are now the definition of correct init output, checks that every test sets `_ZO_DATA_DIR` to a temp path, and decides whether the new dev-dependency is acceptable.

## Run it on every release tag

Before each release, one run can check that the tests still cover every subcommand: your release script sends `POST /o/{org}/sessions` with the repository in `resources` and a prompt to add tests for any flag or subcommand added since the last tag. The agent opens a pull request when something is missing.

Set a `webhook_url` on the API key and Rush posts `run.completed` or `run.failed` to it when the turn ends; the release script can wait for that event and read the session's pull request before it tags.

## Why run it on Rush?

Integration tests for a tool that keeps a database of the user's directories are the kind that go wrong on a laptop: one test that forgets `_ZO_DATA_DIR` adds temp paths to the developer's own zoxide history. In a Rush sandbox the agent builds the binary and runs every test on a machine that exists for this run, with no shell history or data directory to damage.

Each `cargo build` and `cargo test` is kept with its output, so the reviewer sees the snapshots being generated and the full passing run without checking out the branch. Several CLIs in one organization can each get a run at once, each in its own sandbox.

## Frequently asked questions

### Will the tests touch my real zoxide database?

No. Every test sets `_ZO_DATA_DIR` to a `tempfile::TempDir`, which zoxide requires to be an absolute path, and the directory is deleted when the test ends. The same applies to the import fixtures, which are read through their own environment variables.

### Does it need every shell installed?

Not for snapshot tests, which compare the text `zoxide init` prints. The existing tests that run the generated scripts in bash, fish, zsh and other shells stay behind the `nix-dev` feature, which the repository's CI turns on inside `nix-shell`.

### Why add a snapshot crate instead of comparing strings?

One snapshot per shell and option combination means dozens of expected outputs, and a snapshot tool writes and updates them for you and shows a readable diff when a template changes. If you would rather not add a dependency, say so in `input` and the agent compares against files under `tests/` instead.

### Does it work for CLIs built with something other than clap?

Yes. `assert_cmd` runs the compiled binary, so it does not matter how arguments are parsed. The agent reads whichever parser the CLI uses to list its subcommands and flags before writing tests.