What the agent does
The agent reads the pull request's diff and finds the endpoints it changes. In a FastAPI app built from fastapi/full-stack-fastapi-template those are in backend/app/api/routes/: items.py defines the list, read, create, update and delete routes on /items, and users.py defines /users/me, /users/me/password, /users/signup and the superuser routes on /users/{user_id}. Shared logic sits in backend/app/crud.py and the SQLModel tables in backend/app/models.py. For each changed route or CRUD function the agent finds the request that would reach the new code and checks whether any existing test sends it.
It writes the new tests where the template keeps them: backend/tests/api/routes/test_items.py and test_users.py for endpoints, backend/tests/crud/test_user.py for CRUD functions. backend/tests/conftest.py already provides a session-scoped db fixture that runs init_db and cleans up Item and User rows, a client fixture around TestClient(app), and superuser_token_headers and normal_user_token_headers. The helpers in tests/utils/item.py and tests/utils/user.py create random items and users. The new tests take those fixtures as arguments and build URLs from settings.API_V1_STR, the same way test_create_item does, so they need no new setup.
Running them needs the database the template expects. The backend workflow starts the db service with docker compose up -d --wait db mailpit, runs uv run bash scripts/prestart.sh (Alembic upgrade head and app/initial_data.py, which creates the first superuser from .env), then scripts/tests-start.sh, which calls coverage run -m pytest tests/. The agent starts Postgres with the credentials in .env (POSTGRES_PASSWORD, FIRST_SUPERUSER), applies the migrations with the same prestart script, and runs uv run pytest tests/ from backend/, on the Python version pyproject.toml requires (>=3.14).
The template's CI also fails the job when coverage drops below 90 percent (coverage report --fail-under=90), so a pull request that adds an untested route can turn the check red on its own. The agent's tests target exactly those lines. When a test fails because a route returns the wrong status or leaks a field it should not, the agent keeps that test out of the commit, as the prompt asks, and reports the defect in its comment.
Start it with one API call
POST https://api.prix.dev/o/{org}/sessions with this body:
{
"agent": "claude",
"input": "This pull request changes the users and items routes. Read the diff against the base branch and list the new or changed behavior in backend/app/api/routes/users.py, items.py and backend/app/crud.py that no existing test covers. Start Postgres with the values in .env, run scripts/prestart.sh, and write pytest tests for each item in backend/tests/api/routes/test_users.py, test_items.py or backend/tests/crud/, using the fixtures in conftest.py (client, db, superuser_token_headers, normal_user_token_headers) and the helpers in tests/utils. Run pytest from backend/ until the new tests pass. Do not change application code: if a test fails because the code is wrong, leave it out and describe the failure. Push the tests to this pull request's branch and post a comment listing each test and the behavior it covers.",
"resources": [
{
"url": "https://github.com/your-org/your-app/pull/123"
}
]
}- Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
- Send the request. POST the body in request.json to https://api.prix.dev/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
- Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.
What comes back
What comes back is a commit on the pull request's branch with new test functions in backend/tests/ and nothing outside it. Each one is a plain pytest function such as test_update_item_not_owner that takes client and a token-headers fixture, sends the request, and asserts the status code and the fields in response.json(). Where the pull request changed crud.py, the tests call the CRUD function with the db session directly.
The agent posts a comment on the pull request that lists each new test, the route and behavior it covers, and the pytest command it ran. If a route the pull request added returns 200 where it should refuse a normal user, the comment quotes that failing assertion and the route line instead of committing a red test. The template's test-backend workflow then reruns the suite with coverage on the pushed commit, so the 90 percent check is evaluated on your runner.
Before merging, a reviewer checks that permission tests use the right fixture (superuser or normal user) for what they claim, that tests creating users clean up through the existing db fixture, and that no test depends on another test's rows.
Run it on every pull request opened
Create an automation on the Project connected to the repository with this prompt and a GitHub trigger on pull_request.opened. Each pull request then starts a run in its own sandbox, and its prompt carries the pull request's number, URL, title, author and base branch, so the same automation serves every pull request without editing. Only authors with write access fire it, and forks never do. Use max_runs_per_hour to cap how many runs a burst of pull requests can start.
If your own service already listens to GitHub, send this request on each pull_request webhook with action opened, with the pull request's URL in resources so Rush clones its branch. An Idempotency-Key built from the delivery id makes a retried webhook return the run it already started.
The use case this job follows, with its automation and trigger: Write the missing tests.
Why run it on Rush?
This job needs a running Postgres, migrated, seeded with a superuser, plus a Python 3.14 environment from uv.lock. Doing it on a laptop for every pull request means tearing down and rebuilding the Compose stack under your own work. On Rush each run gets its own sandbox with that stack, at the pull request's branch, and several pull requests are tested at the same time without sharing a database.
The proof travels with the result. The trajectory keeps the prestart output, each pytest run and its failures, so a reviewer can see how the agent reached passing tests, and the template's coverage check reruns on the pushed commit. Coverage stays above the template's 90 percent threshold because each change arrives with its tests, not because someone backfills them later.
Frequently asked questions
Does it need Docker in the sandbox?
It needs a Postgres the backend can reach at the URL built from .env. The template's CI uses docker compose up db; the agent can start Postgres in its sandbox directly and point the same settings at it. Either way, the tests run against a real database, as the template's own tests do.
Will it mock the database or the email sender?
It follows the existing tests, which use a real database session and patch only what they already patch, such as the SMTP settings test_login.py patches in its password recovery tests. It does not add a mock layer the suite does not already have.
What about the React frontend in the template?
The prompt limits the run to the backend routes and backend/tests/. The frontend has its own Playwright tests; ask for those in a separate run if a pull request changes the client.
Can I run it with Codex instead?
Yes. Set agent to codex in the same request. The prompt, the repository and the result are the same; only the agent doing the work changes.