# Add Missing Tests to Every Pull Request in a FastAPI App

Yes. An agent reads each pull request to your FastAPI app, writes pytest tests for the users and items routes it changes and pushes them back.

[Run this on Rush](https://cloud.getrush.ai/run/test-every-pull-request/fastapi/), in a Project that has your repository.

## What the agent does

The agent reads the pull request's diff and finds the endpoints it changes. In a FastAPI app built from fastapi/full-stack-fastapi-template those are in `backend/app/api/routes/`: `items.py` defines the list, read, create, update and delete routes on `/items`, and `users.py` defines `/users/me`, `/users/me/password`, `/users/signup` and the superuser routes on `/users/{user_id}`. Shared logic sits in `backend/app/crud.py` and the SQLModel tables in `backend/app/models.py`. For each changed route or CRUD function the agent finds the request that would reach the new code and checks whether any existing test sends it.

It writes the new tests where the template keeps them: `backend/tests/api/routes/test_items.py` and `test_users.py` for endpoints, `backend/tests/crud/test_user.py` for CRUD functions. `backend/tests/conftest.py` already provides a session-scoped `db` fixture that runs `init_db` and cleans up `Item` and `User` rows, a `client` fixture around `TestClient(app)`, and `superuser_token_headers` and `normal_user_token_headers`. The helpers in `tests/utils/item.py` and `tests/utils/user.py` create random items and users. The new tests take those fixtures as arguments and build URLs from `settings.API_V1_STR`, the same way `test_create_item` does, so they need no new setup.

Running them needs the database the template expects. The backend workflow starts the `db` service with `docker compose up -d --wait db mailpit`, runs `uv run bash scripts/prestart.sh` (Alembic `upgrade head` and `app/initial_data.py`, which creates the first superuser from `.env`), then `scripts/tests-start.sh`, which calls `coverage run -m pytest tests/`. The agent starts Postgres with the credentials in `.env` (`POSTGRES_PASSWORD`, `FIRST_SUPERUSER`), applies the migrations with the same prestart script, and runs `uv run pytest tests/` from `backend/`, on the Python version `pyproject.toml` requires (`>=3.14`).

The template's CI also fails the job when coverage drops below 90 percent (`coverage report --fail-under=90`), so a pull request that adds an untested route can turn the check red on its own. The agent's tests target exactly those lines. When a test fails because a route returns the wrong status or leaks a field it should not, the agent keeps that test out of the commit, as the prompt asks, and reports the defect in its comment.

## Start it with one API call

`POST https://api.prix.dev/o/{org}/sessions`

```json
{
  "agent": "claude",
  "input": "This pull request changes the users and items routes. Read the diff against the base branch and list the new or changed behavior in backend/app/api/routes/users.py, items.py and backend/app/crud.py that no existing test covers. Start Postgres with the values in .env, run scripts/prestart.sh, and write pytest tests for each item in backend/tests/api/routes/test_users.py, test_items.py or backend/tests/crud/, using the fixtures in conftest.py (client, db, superuser_token_headers, normal_user_token_headers) and the helpers in tests/utils. Run pytest from backend/ until the new tests pass. Do not change application code: if a test fails because the code is wrong, leave it out and describe the failure. Push the tests to this pull request's branch and post a comment listing each test and the behavior it covers.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app/pull/123"
    }
  ]
}
```

1. **Get an API key.** In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
2. **Send the request.** POST the body in request.json to https://api.prix.dev/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
3. **Read what came back.** Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

## What comes back

What comes back is a commit on the pull request's branch with new test functions in `backend/tests/` and nothing outside it. Each one is a plain pytest function such as `test_update_item_not_owner` that takes `client` and a token-headers fixture, sends the request, and asserts the status code and the fields in `response.json()`. Where the pull request changed `crud.py`, the tests call the CRUD function with the `db` session directly.

The agent posts a comment on the pull request that lists each new test, the route and behavior it covers, and the pytest command it ran. If a route the pull request added returns `200` where it should refuse a normal user, the comment quotes that failing assertion and the route line instead of committing a red test. The template's `test-backend` workflow then reruns the suite with coverage on the pushed commit, so the 90 percent check is evaluated on your runner.

Before merging, a reviewer checks that permission tests use the right fixture (superuser or normal user) for what they claim, that tests creating users clean up through the existing `db` fixture, and that no test depends on another test's rows.

## Run it on every pull request opened

Create an automation on the Project connected to the repository with this prompt and a GitHub trigger on `pull_request.opened`. Each pull request then starts a run in its own sandbox, and its prompt carries the pull request's number, URL, title, author and base branch, so the same automation serves every pull request without editing. Only authors with write access fire it, and forks never do. Use `max_runs_per_hour` to cap how many runs a burst of pull requests can start.

If your own service already listens to GitHub, send this request on each `pull_request` webhook with action `opened`, with the pull request's URL in `resources` so Rush clones its branch. An `Idempotency-Key` built from the delivery id makes a retried webhook return the run it already started.

## Why run it on Rush?

This job needs a running Postgres, migrated, seeded with a superuser, plus a Python 3.14 environment from `uv.lock`. Doing it on a laptop for every pull request means tearing down and rebuilding the Compose stack under your own work. On Rush each run gets its own sandbox with that stack, at the pull request's branch, and several pull requests are tested at the same time without sharing a database.

The proof travels with the result. The trajectory keeps the prestart output, each pytest run and its failures, so a reviewer can see how the agent reached passing tests, and the template's coverage check reruns on the pushed commit. Coverage stays above the template's 90 percent threshold because each change arrives with its tests, not because someone backfills them later.

## Frequently asked questions

### Does it need Docker in the sandbox?

It needs a Postgres the backend can reach at the URL built from `.env`. The template's CI uses `docker compose up db`; the agent can start Postgres in its sandbox directly and point the same settings at it. Either way, the tests run against a real database, as the template's own tests do.

### Will it mock the database or the email sender?

It follows the existing tests, which use a real database session and patch only what they already patch, such as the SMTP settings `test_login.py` patches in its password recovery tests. It does not add a mock layer the suite does not already have.

### What about the React frontend in the template?

The prompt limits the run to the backend routes and `backend/tests/`. The frontend has its own Playwright tests; ask for those in a separate run if a pull request changes the client.

### Can I run it with Codex instead?

Yes. Set `agent` to `codex` in the same request. The prompt, the repository and the result are the same; only the agent doing the work changes.