All jobs

Add Missing Tests to Every Pull Request in a Rust Service

Yes. On each new pull request an agent reads the diff, writes cargo tests for the untested Rust code, runs cargo test and pushes them to the same branch.

Run this on RushIn a Project that has your repository.

What the agent does

The run starts on the pull request's own branch, so the agent sees the change the way the reviewer does. It diffs the branch against its base and lists every function the change adds or alters. In an Axum and SQLx service like launchbadge/realworld-axum-sqlx, a change to the article handlers lands in src/http/articles/mod.rs, listing.rs or comments.rs, and the agent reads each touched handler (create_article, update_article, favorite_article, list_articles and the rest) together with the router in router() that mounts them under /api/articles and /api/tags.

Next it finds what is already tested. That repository has one unit test, test_slugify at the bottom of src/http/articles/mod.rs, and a long comment above it explaining that the rest of the API is covered by the RealWorld Postman collection that .github/workflows/rust.yml runs after cargo test. So the agent splits the new behavior in two: pure functions such as slugify() get #[test] cases in the same mod tests style the file already uses, and handler behavior that needs a database gets #[tokio::test] integration tests under tests/ that build the router and send requests through tower::ServiceExt::oneshot.

It then works out what those tests need to compile and run. The crate pins sqlx = 0.5 with the postgres feature and uses sqlx::query!, which checks every query against a live database at compile time, and the repository has no offline query data. So the agent starts Postgres in its sandbox, sets DATABASE_URL and HMAC_KEY from .env.sample, and runs sqlx db setup to apply migrations/1_setup.sql through 4_article.sql, exactly as the CI workflow does. #[sqlx::test] arrived in SQLx 0.6, so on 0.5 it writes a small helper that runs each test in a transaction it rolls back, rather than upgrading a dependency in a test-only change.

Finally it runs cargo test until every new test passes, and checks each one fails for the right reason by temporarily breaking the line it covers and confirming the test goes red. It does not edit any file under src/http outside a #[cfg(test)] module. If a test exposes a real bug in the pull request, it leaves the failing case out, explains it in its comment and lets the author decide.

Start it with one API call

POST https://api.prix.dev/o/{org}/sessions with this body:

{
  "agent": "claude",
  "input": "This pull request changes the article handlers in src/http/articles. Read the diff against the base branch, find the new or changed behavior that no test covers, and write cargo test tests for it: #[test] unit tests next to the existing test_slugify for pure functions, and #[tokio::test] integration tests under tests/ that call the router for handler behavior, including the error responses. Start Postgres in the sandbox, set DATABASE_URL and HMAC_KEY as .env.sample describes and run sqlx db setup so sqlx::query! compiles. Run cargo test until the new tests pass. Do not change application code. Push the tests to this pull request's branch and comment on the pull request listing each test and what it covers.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app/pull/123"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.prix.dev/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

What comes back is a new commit on the pull request's own branch, not a second pull request. It holds test code only: new #[test] functions in an existing #[cfg(test)] module and new files under tests/, plus a [dev-dependencies] entry in Cargo.toml if the tests need one the crate does not have yet (for example tower with the util feature for oneshot). The push re-runs the repository's own workflow, so the cargo test step in rust.yml runs the new tests on GitHub as well as in the sandbox.

The agent also posts a comment on the pull request. It lists each test by name with the handler and the case it covers ("update_article_by_non_author_is_forbidden: PUT /api/articles/:slug as another user returns 403"), the behavior it chose not to test and why, and anything it believes is a bug. The session's record keeps every cargo test it ran with the compiler and test output, so a reviewer can see the run where a test was red before it was right.

Before merging, the reviewer reads the tests as they would any code: that each assertion checks the behavior the pull request promises and not an incidental detail, that the database helper cleans up after itself, and that no test depends on another's order.

Run it on every pull request opened

In the console, add an automation with a GitHub trigger on pull_request.opened and this prompt, optionally limited by branch pattern to pull requests against main. Each new pull request then starts its own run on its own branch. Only pull requests from an owner, member or collaborator start a run; pull requests from forks never do. Add pull_request.synchronize if you also want new tests when the author pushes more commits, and set the automation's hourly run limit so a busy day stays within budget.

To drive it from your own server instead, receive GitHub's pull_request webhook and send this request with the pull request's URL in resources, which clones its branch. Put the pull request number in the Idempotency-Key header so a retried webhook does not start a second run.

The use case this job follows, with its automation and trigger: Write the missing tests.

Why run it on Rush?

Writing tests for someone else's Rust change on a laptop means checking out their branch, bringing up Postgres, applying migrations so sqlx::query! compiles, and waiting through a cold cargo build of Axum, SQLx and Tokio, all while your own work sits on another branch. On Rush each pull request gets its own sandbox with its own clone and database, so the compile and the test runs happen there and your working tree is untouched.

The result is checkable. The tests run in the pull request's own CI after the push, and the session keeps the output of every command the agent ran, including the runs where a test was deliberately broken to prove it can fail. Because each run is independent, ten pull requests opened in one afternoon get ten runs at once, each with its own commit and comment for a person to review.

Frequently asked questions

Will it change my application code?

The prompt says not to, and the agent keeps its edits to test modules, files under tests/ and [dev-dependencies]. If the only way to test something is a change to production code, such as making a private function pub(crate), it says so in the comment instead of making the change. Every edit is in the commit it pushes, so you can see exactly what it touched.

My queries use sqlx::query!. How does it compile them?

It brings up Postgres in the sandbox, sets DATABASE_URL and runs your migrations, so the macros check against a real schema the way they do on a developer machine. If your repository has sqlx-data.json or a .sqlx directory for offline mode, it uses that and runs cargo sqlx prepare again if the new tests add queries.

Does it use #[sqlx::test]?

When your SQLx version has it (0.6 and later), yes, because each test then gets its own database. On older versions it writes a helper that wraps each test in a transaction and rolls it back. It does not upgrade SQLx as part of a test change.

What if the pull request has no new behavior to test?

It comments that the change is covered by existing tests or needs none, names the tests that already exercise it, and pushes nothing. A pull request that only renames or reformats code does not get filler tests.