What the agent does
The agent starts by reading what the suite covers and what the app contains, because the gap between those two lists is the report. In an app like miguelgrinberg/microblog, tests.py is a single unittest module with four tests, all on the User model: password hashing, the Gravatar URL, following and the followed-posts query, run against an in-memory SQLite database with ELASTICSEARCH_URL set to None through a TestConfig subclass of config.Config. The app itself, built by create_app in app/__init__.py, registers four blueprints: main with routes from index and explore through follow, send_message, export_posts and notifications, auth with login, register, reset_password_request and reset_password, api with the user endpoints and the token endpoints behind HTTPBasicAuth and HTTPTokenAuth from app/api/auth.py, and errors with the 404 and 500 handlers that pick JSON or HTML by wants_json_response.
It also reads what the app reaches for, because untested code that talks to a service is where the sandbox has to be careful. requirements.txt pins Flask 3.0.0, Flask-SQLAlchemy 3.1.1, Flask-HTTPAuth, PyJWT, rq, redis and elasticsearch 8.11, and create_app builds a Redis client and an rq queue on every start, though neither connects until used. app/tasks.py runs export_posts as a background job, app/translate.py calls a translation API, app/search.py indexes posts in Elasticsearch, and app/email.py sends mail in a thread. coverage is not in requirements.txt, so the agent installs it in the sandbox only, and nothing in the repository changes.
Then it measures. It runs coverage run --branch --source=app -m unittest tests -v, then coverage report -m and coverage html, so the numbers in the report are measured from this suite rather than estimated from reading. It reads the per-file table and the missing-line ranges, and ranks the uncovered code by what it protects: the login and registration forms in app/auth/routes.py, the password reset token round trip in User.get_reset_password_token and verify_reset_password_token, the API token lifecycle in get_token, check_token and revoke_token and the POST /api/tokens and DELETE /api/tokens routes that call them, create_user and update_user in app/api/users.py which write accounts from JSON, follow and unfollow and send_message in app/main/routes.py, and the error handlers. Rendering-only routes rank lower than ones that write or authenticate.
For each gap it writes one concrete test sketch that fits this codebase: the Flask test client with TestConfig, WTF_CSRF_ENABLED off for form posts, an HTTP Basic header for POST /api/tokens and a Bearer header for the user endpoints, a Message and Notification check for send_message, and the monkeypatches needed to keep send_email, the rq queue and the translator out of the test. It notes the pieces that need a running service, Elasticsearch for app/search.py and Redis for app/tasks.py, and how the existing ELASTICSEARCH_URL = None pattern already isolates one of them. The result is coverage-gaps.md; the agent changes no code and opens no pull request.
Start it with one API call
POST https://api.prix.dev/o/{org}/sessions with this body:
{
"agent": "claude",
"input": "Report the untested code in this Flask app without changing anything. Read tests.py, config.py, app/__init__.py, app/models.py, app/main/routes.py, app/auth/routes.py, app/api/users.py, app/api/tokens.py, app/api/auth.py, app/errors/handlers.py, app/tasks.py, app/translate.py, app/search.py, app/email.py and requirements.txt first. Install the requirements and the coverage package in the sandbox, run the existing unittest suite under coverage with branch measurement and source limited to the app package, and produce the terminal report with missing lines and the HTML report. Write coverage-gaps.md: the exact command and the measured overall line and branch coverage, the per-file table, then the ten riskiest untested functions or routes ranked by what they protect, authentication, tokens, account writes and messaging first, each with its missing lines and a short test sketch using the app's TestConfig, the Flask test client and the auth headers its API expects, and a final section on code that needs Elasticsearch, Redis, mail or the translation API and how to stub each. Change no code and open no pull request.",
"resources": [
{
"url": "https://github.com/your-org/your-app"
}
]
}- Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
- Send the request. POST the body in request.json to https://api.prix.dev/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
- Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.
What comes back
What comes back is coverage-gaps.md as an artifact on the session, which the session read returns with a download link good for an hour. It opens with the command that was run and the overall line and branch percentages from that run, followed by the per-file table from coverage report -m with the missing line ranges, so a reader knows every number is measured against the suite as it is today.
Then come the ten riskiest gaps, each with the file and function, the lines not executed, one or two sentences on what breaks silently if that code regresses, and a test sketch written for this app: which fixture config to use, which route to call, which header or form field to send and what to assert. A closing section lists the modules that depend on Elasticsearch, Redis, mail and the translator, says which the existing TestConfig already neutralizes, and gives the monkeypatch for each of the others.
Before acting on it, a reviewer checks the ranking against what the app actually exposes, picks the sketches worth turning into tests, and decides whether to add coverage to the development requirements so the number is tracked. A follow-up message to the same session asks the agent to write the tests for any sketch and open a pull request, since the sandbox still has the environment installed.
Run it on every pull request to the default branch
A team that wants the gap list refreshed as the code changes starts one run per pull request with a POST /o/{org}/sessions naming the pull request branch in resources, as https://github.com/your-org/your-app/pull/123, and this prompt. A GitHub Actions job can send that request with the key from its secrets and post the artifact link as a comment.
Each run measures in its own sandbox, so two pull requests never share a database file or a coverage data file. A webhook on the key receives run.completed with the session_url, and reading the session returns the artifact's fresh download link.
The use case this job follows, with its automation and trigger: Produce a report or file, no repo.
Why run it on Rush?
A coverage number from a laptop depends on which services happened to be running and which environment variables were set; this app builds a Redis client on start and reads ELASTICSEARCH_URL, MAIL_SERVER and MS_TRANSLATOR_KEY from the environment. In a Rush sandbox the agent starts from a clean environment, installs exactly requirements.txt plus coverage, and runs the suite the way CI would, so the report describes the code rather than one developer's machine.
The report carries its own proof. The coverage run and coverage report -m output is in the transcript beside the table in the report, so a reviewer can check any percentage or missing-line range against the raw run, and the ranking is tied to named functions and lines rather than to impressions.
Frequently asked questions
Does it add tests or change the suite?
No. The result is a report with test sketches, and the repository is left as it was, including requirements.txt. If you want the tests written, a follow-up message to the same session names the sketches and the agent opens a pull request with them.
Does the suite need Redis or Elasticsearch to run under coverage?
Not as it stands. create_app builds a Redis client and an rq queue without connecting, and TestConfig sets ELASTICSEARCH_URL to None, so the four existing tests run with no services. The report says which untested modules would need a service or a stub before a test can reach them.
How does it decide what is riskiest?
By what the code protects, not by line count. Login, registration, password reset tokens, API tokens and the endpoints that create or update users rank first because a regression there is a security or data problem, messaging and following next because they write rows, and pages that only render come last even when they have more uncovered lines.
Are the test sketches specific to this app?
Yes. Each one names the route, the TestConfig to build the app with, the WTF_CSRF_ENABLED setting a form post needs, the Basic or Bearer header the api blueprint expects from app/api/auth.py, and the function to monkeypatch so mail or the task queue stays out of the test.