What the agent does
The agent checks out the pull request's branch and reads how the crate expects to be checked before it runs anything. In a CLI shaped like ajeetdsouza/zoxide, Cargo.toml names version 0.10.0 on edition 2024 with rust-version = "1.88.0", clap 4.3 with derive for the add, edit, import, init, query and remove subcommands in src/cmd/cmd.rs, askama 0.16 for the nine shell templates under templates/, anyhow for every error, and assert_cmd, rstest, rstest_reuse and tempfile as dev-dependencies. The justfile is the real CI definition: just lint runs cargo fmt --all --check, cargo clippy --all-features --all-targets -- -Dwarnings, cargo msrv verify, cargo udeps, mandoc over man/man1/*.1, markdownlint, shellcheck and shfmt over the install script, and just test runs cargo nextest run --all-features --no-fail-fast --workspace, all inside nix-shell; .github/workflows/ci.yml installs Nix and just and calls just lint test on Ubuntu.
The sandbox has no Nix, so the agent runs the same commands directly: cargo fmt --all --check (the rustfmt.toml uses imports_granularity, group_imports and wrap_comments, which need a nightly rustfmt, so it installs one for that step as the justfile does on Windows), cargo clippy --all-features --all-targets -- -Dwarnings, cargo msrv verify against 1.88.0, and cargo test --all-features --no-fail-fast. Which tests can run depends on the diff. The database tests in src/db/ and the rstest keyword table in src/db/stream.rs need nothing but Rust. The template tests in src/shell.rs render each template over the cmd, hook, echo and resolve_symlinks options and pipe the result through bash, dash, elvish, fish, fish_indent, nu, pwsh, tcsh, xonsh, shellcheck, shfmt, black, mypy and pylint; tests/completions.rs is gated behind the nix-dev feature. A pull request such as one touching templates/powershell.txt and src/shell.rs makes the agent install pwsh so the powershell_pwsh case runs, and the review names exactly which shells were installed and which cases ran.
Then it reads the diff against the code that already handles the tricky parts, because most findings in a CLI are about what happens at the edges. Error handling: every fallible call carries an anyhow context such as not a directory: {path} in src/cmd/add.rs or could not render template in src/cmd/init.rs, and writes to stdout go through pipe_exit from src/error.rs so a closed pipe exits quietly; a new unwrap, a bare ? without context, or a println! that bypasses the BrokenPipeHandler is a finding with its line. Shell quoting: the templates wrap "$@" and $(__zoxide_pwd) in quotes and the shellcheck and shfmt cases enforce it, so a template change that drops a quote shows up as a failing test the review quotes. Cross-platform paths: src/cmd/add.rs picks util::canonicalize or util::resolve_path depending on _ZO_RESOLVE_SYMLINKS, Cargo.toml adds dunce and a Windows-only which, and src/config.rs requires _ZO_DATA_DIR to be absolute; a diff that joins paths with a literal slash or compares them as strings is called out.
Two repository conventions get checked on every diff. build.rs regenerates contrib/completions/ from src/cmd/cmd.rs on every build, so a change to a flag or subcommand must arrive with the regenerated completion files committed, and the agent diffs the generated output against the branch to see whether they were. The man pages under man/man1/ and the CHANGELOG.md Unreleased section describe user-visible behaviour; a flag added without its man page line or changelog entry is a suggestion, not a blocker. The review separates blocking findings (a failing check, a lost error context, a path assumption) from suggestions, quotes the exact command output for each check it ran, and pushes nothing to the branch.
Start it with one API call
POST https://api.prix.dev/o/{org}/sessions with this body:
{
"agent": "claude",
"input": "Review the open pull request on this Rust CLI. Check out its branch and read Cargo.toml, the justfile and rustfmt.toml to learn the project's own checks. Run cargo fmt --all --check (install a nightly rustfmt if the config needs it), cargo clippy --all-features --all-targets -- -Dwarnings, cargo msrv verify and cargo test --all-features --no-fail-fast. If the diff touches a template under templates/ or src/shell.rs, install the shell that template targets so its test cases run, and say which shells you installed. Read the diff with attention to error handling (anyhow contexts, the BrokenPipeHandler in src/error.rs), shell quoting in the templates, and cross-platform path handling in src/cmd/add.rs, src/config.rs and src/util.rs. Check whether a change to src/cmd/cmd.rs came with regenerated contrib/completions files, a man page update and a CHANGELOG.md entry. Post one review that cites file and line for every finding, separates blocking findings from suggestions, and quotes the output of each check. Do not push commits.",
"resources": [
{
"url": "https://github.com/your-org/your-app/pull/1"
}
]
}- Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
- Send the request. POST the body in request.json to https://api.prix.dev/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
- Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.
What comes back
What comes back is one review on the pull request. It opens with the checks and their results: the rustfmt, clippy, MSRV and test commands exactly as run, how many tests ran and how many were skipped because a shell was not installed, and the toolchain the preamble reports. Then it lists findings, each as a file and line with a one-sentence reason, blocking ones first. A failed check is quoted as the finding, not paraphrased.
A reviewer reads the shell list before anything else, because a green cargo test with fish and nu missing says less than it seems. Then they check that any flagged contrib/completions/ drift is real by looking at the diff, and that a template finding points at a line a shellcheck or shfmt case would catch. The session read names the pull request, and the transcript holds every command with its output, so the claim that clippy passed with -Dwarnings can be checked line by line.
The review posts no commits. If the author wants the fix made, a follow-up message to the same session, once the turn has finished, asks the agent to push it to the branch; the parked sandbox still has the build cache.
Run it on every pull request opened or pushed
In the console, add an automation to the Project with this prompt and a GitHub trigger on pull_request.opened and pull_request.synchronize. Each push starts a run that checks out that commit and posts a review; a redelivery of the same event starts nothing. Only pull requests opened by an owner, member or collaborator fire it, and pull requests from forks never do, so a stranger cannot start runs against the repository.
An automation can be marked read-only, so a review run can read the repository and comment but never push. For reviews started from your own server instead, send this request with the pull request URL in resources and the delivery id as the Idempotency-Key; a webhook_url on the API key then receives run.completed with the session_url when the review is posted.
The use case this job follows, with its automation and trigger: Review every pull request.
Why run it on Rush?
Reviewing a shell integration honestly means running the templates through the shells they target, and no laptop has elvish, nu, pwsh, tcsh and xonsh installed alongside shellcheck and shfmt. The sandbox installs what the diff needs, runs the rstest cases for that shell, and keeps the output, so the review says what was executed rather than what was read.
Clippy with -Dwarnings and an MSRV check against 1.88.0 take a full build each, and on a laptop they compete with whatever else is being compiled. Each pull request gets its own sandbox and its own target/ directory, several at once, and the pull request receives one review with the command output attached rather than a comment that the author has to trust.
Frequently asked questions
The project runs everything inside nix-shell. Does the review?
No. The sandbox has no Nix, so the agent runs the commands the justfile wraps: cargo fmt --all --check, cargo clippy --all-features --all-targets -- -Dwarnings, cargo msrv verify and cargo test. It names the toolchain it used, from the cargo preamble, so you can compare it with the channel your CI image pins.
Does it run the tests for every shell?
Only the ones the diff touches, plus whatever is already installed. src/shell.rs has a case per shell and per linter, and each needs that interpreter on the machine. The review lists the shells it installed and the cases that were skipped, so a passing run is never mistaken for a full matrix.
Will it flag the completion files?
When src/cmd/cmd.rs changes and contrib/completions/ does not, yes. build.rs regenerates those files from the clap definitions on every build, so a flag added without them leaves the committed completions stale. The agent builds the branch, diffs the generated files against it, and quotes the difference.
Can it fix what it finds?
Not in the review run, which pushes nothing. After the turn finishes, a follow-up message to the same session can ask for the fix, and the agent commits it to the branch from the same sandbox. A review automation marked read-only cannot push even when asked.