All jobs

Write API Tests for an Axum App

Yes. An agent adds integration tests for your Axum routes with tower's oneshot against Postgres, covering auth, 422 bodies and ownership rules, and opens a PR.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

The agent starts by finding out what already runs. In an app shaped like launchbadge/realworld-axum-sqlx, Cargo.toml pins axum 0.3.4, sqlx 0.5 (0.5.9 in Cargo.lock) with runtime-tokio-native-tls, postgres, uuid and time, tokio 1.14 with macros and rt-multi-thread, tower 0.4.11, tower-http 0.2 for TraceLayer, argon2 0.3.1, jwt 0.15 with HMAC-SHA384 and clap 3.0.0-beta.5. There is no tests/ directory and no #[cfg(test)] module anywhere under src/, so the cargo test --verbose step in .github/workflows/rust.yml runs zero tests. The only check is the last step: cargo run in the background, then the Realworld Postman collection from a pinned fork, run by run-api-tests.sh against http://localhost:8080/api with a postgres service container.

The routes to cover are spread over src/http: users.rs has POST /api/users, POST /api/users/login and GET and PUT /api/user; profiles.rs has GET /api/profiles/:username and POST and DELETE .../follow; articles/mod.rs has /api/articles, /api/articles/feed, /api/articles/:slug, /api/articles/:slug/favorite and /api/tags, and articles/comments.rs the two comment routes. extractor.rs reads Authorization: Token <jwt> into AuthUser, and error.rs maps Unauthorized to 401 with a WWW-Authenticate header, Forbidden to 403, NotFound to 404 and UnprocessableEntity to 422 with a body of {"errors": {field: [message]}}, which is what the on_constraint calls in create_user produce for username taken and email taken.

http::serve builds api_router() with an AddExtensionLayer carrying the private ApiContext and a TraceLayer, then binds 0.0.0.0:8080 inside one function. That is the one seam the agent changes: it extracts pub fn app(config: Config, db: PgPool) -> Router and has serve call it, so production behaviour is unchanged while tests can build the same router. tests/api.rs then constructs Config from the DATABASE_URL and HMAC_KEY the workflow already exports (both fields are public), opens a small PgPool, runs sqlx::migrate!() and drives requests through tower::ServiceExt::oneshot with no socket. hyper 0.14.15 and serde_json 1.0.71, the versions in Cargo.lock, become dev-dependencies for to_bytes and for building and reading JSON bodies.

The cases pin what the code does, not what the spec says: registration answers 200 with a user.token, because create_user returns Json with no status; a second registration with the same username answers 422 with username taken under errors.username; a wrong password answers 401 because verify_password maps argon2::password_hash::Error::Password to Error::Unauthorized; GET /api/user without a header is 401 and with the token is 200; an article created by one user cannot be deleted by another (403); comments can be listed, added and deleted only by their author; GET /api/tags returns the tags of created articles. sqlx 0.5 has no #[sqlx::test], so instead of a database per test each test makes its own usernames and emails with a rand suffix and keeps no shared state, which lets cargo test run them in parallel on one database. migrations/1_setup.sql creates an ICU collation, so the Postgres the agent starts in the sandbox must be ICU-enabled, as the postgres image is. It runs cargo build, cargo test and cargo fmt --check and pastes the output.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "codex",
  "input": "Write integration tests for this Axum API and open a pull request. Read Cargo.toml, Cargo.lock, .github/workflows/rust.yml, src/main.rs, src/config.rs and everything under src/http first. Make the smallest change to production code that lets tests build the router: a public function that returns the Router with its extension and trace layers, called by serve. Add tests/api.rs that builds that router from DATABASE_URL and HMAC_KEY, runs the migrations, and drives requests with tower::ServiceExt::oneshot; add hyper and serde_json as dev-dependencies at the versions already in Cargo.lock. Cover: register (status and token), duplicate username (422 body under errors.username), login with wrong password (401), GET /api/user with and without a token, create and get an article by slug, delete by a different user (403), add, list and delete comments including a non-author delete, and GET /api/tags. Give each test its own random usernames and emails so tests run in parallel on one database; change no migrations. Run cargo build, cargo test and cargo fmt --check and paste the output in the pull request. Do not change the Postman step in the workflow.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

What comes back is a pull request with one new file, tests/api.rs, two dev-dependencies in Cargo.toml with the matching Cargo.lock lines, and a small refactor in src/http/mod.rs that turns the router construction into a public function serve calls. The description lists each test and the status code it pins, and quotes the cargo test summary from the sandbox.

The existing rust.yml workflow already sets DATABASE_URL and HMAC_KEY, runs sqlx db setup and then cargo test --verbose, so the new tests run on the pull request with no workflow change, before the Postman step. A reviewer sees both results on the same check.

A reviewer checks that the refactor in mod.rs changes nothing about layers or the bind address, that every test creates its own users rather than depending on another test's rows, that the asserted statuses match the code's actual behaviour (200 on register, 401 on a wrong password), and that no test reads the HMAC_KEY into a fixture that could leak into logs.

Run it on every nightly schedule

Make this an automation in the Project with a schedule trigger, for example cron 0 2 * * * in your timezone, and the prompt above. Each night one run reads what changed, adds cases for routes that gained behaviour without tests, and opens one pull request on its own branch; a run whose tests fail ends needs_review instead of opening one. An automation has its own max_runs_per_hour cap, separate from any API key.

To start a run from your own tooling instead, send POST /o/{org}/sessions with this body and the repository in resources, and give the key a webhook_url so run.completed arrives with the session_url when the turn ends.

The use case this job follows, with its automation and trigger: Write the missing tests.

Why run it on Rush?

An API with zero Rust tests and a Postman suite as its only check is common, and the hard part of fixing it is not writing assertions but standing up Postgres with the right collation, migrating it, finding a seam into the router and reading bodies through the right crate versions. In a Rush sandbox the agent does all of that with real commands, and cargo test output, the migration run and every file it read stay in the run's record beside the diff.

The cases also document behaviour the spec gets wrong, such as the 200 on registration, which a laptop session tends to skip. Several services can be covered at once by sending one request per repository, each in its own sandbox, and the pull requests arrive with their own CI results.

Frequently asked questions

Does it change production code?

Only the seam: router construction moves into a public function that serve calls, with the same AddExtensionLayer and TraceLayer. Handlers, migrations and the bind address are untouched, and the diff to src/ is a few lines.

Why not a database per test?

#[sqlx::test] with per-test databases arrived in sqlx 0.6, and this app pins 0.5. The agent isolates by data instead: every test registers its own users with random names and creates its own articles, so the tests can run in parallel on the one database the workflow already sets up.

Does the Postman step still run?

Yes. The workflow is unchanged, so run-api-tests.sh still runs against cargo run after cargo test. The new tests are faster feedback on the same pull request, not a replacement.

What if the crate does not compile on the sandbox's Rust toolchain?

Dependencies such as clap 3.0.0-beta.5 and time 0.2 are old. If cargo build fails before any test runs, the agent reports the error in its result and ends needs_review rather than bumping dependencies, since upgrading them is a separate job.

More Axum jobs

The same job on other stacks

Keep reading