# Find Untested Code in a Flask App

Yes. An agent runs your Flask test suite under coverage in its own sandbox and reports the riskiest untested routes and model methods, without changing anything.

[Run this on Rush](https://cloud.getrush.ai/run/test-coverage-gaps/flask/), in a Project that has your repository.

## What the agent does

The agent starts by reading what the suite covers and what the app contains, because the gap between those two lists is the report. In an app like `miguelgrinberg/microblog`, `tests.py` is a single `unittest` module with four tests, all on the `User` model: password hashing, the Gravatar URL, following and the followed-posts query, run against an in-memory SQLite database with `ELASTICSEARCH_URL` set to `None` through a `TestConfig` subclass of `config.Config`. The app itself, built by `create_app` in `app/__init__.py`, registers four blueprints: `main` with routes from `index` and `explore` through `follow`, `send_message`, `export_posts` and `notifications`, `auth` with `login`, `register`, `reset_password_request` and `reset_password`, `api` with the user endpoints and the token endpoints behind `HTTPBasicAuth` and `HTTPTokenAuth` from `app/api/auth.py`, and `errors` with the 404 and 500 handlers that pick JSON or HTML by `wants_json_response`.

It also reads what the app reaches for, because untested code that talks to a service is where the sandbox has to be careful. `requirements.txt` pins `Flask 3.0.0`, `Flask-SQLAlchemy 3.1.1`, `Flask-HTTPAuth`, `PyJWT`, `rq`, `redis` and `elasticsearch 8.11`, and `create_app` builds a Redis client and an `rq` queue on every start, though neither connects until used. `app/tasks.py` runs `export_posts` as a background job, `app/translate.py` calls a translation API, `app/search.py` indexes posts in Elasticsearch, and `app/email.py` sends mail in a thread. `coverage` is not in `requirements.txt`, so the agent installs it in the sandbox only, and nothing in the repository changes.

Then it measures. It runs `coverage run --branch --source=app -m unittest tests -v`, then `coverage report -m` and `coverage html`, so the numbers in the report are measured from this suite rather than estimated from reading. It reads the per-file table and the missing-line ranges, and ranks the uncovered code by what it protects: the login and registration forms in `app/auth/routes.py`, the password reset token round trip in `User.get_reset_password_token` and `verify_reset_password_token`, the API token lifecycle in `get_token`, `check_token` and `revoke_token` and the `POST /api/tokens` and `DELETE /api/tokens` routes that call them, `create_user` and `update_user` in `app/api/users.py` which write accounts from JSON, `follow` and `unfollow` and `send_message` in `app/main/routes.py`, and the error handlers. Rendering-only routes rank lower than ones that write or authenticate.

For each gap it writes one concrete test sketch that fits this codebase: the Flask test client with `TestConfig`, `WTF_CSRF_ENABLED` off for form posts, an HTTP Basic header for `POST /api/tokens` and a `Bearer` header for the user endpoints, a `Message` and `Notification` check for `send_message`, and the monkeypatches needed to keep `send_email`, the `rq` queue and the translator out of the test. It notes the pieces that need a running service, Elasticsearch for `app/search.py` and Redis for `app/tasks.py`, and how the existing `ELASTICSEARCH_URL = None` pattern already isolates one of them. The result is `coverage-gaps.md`; the agent changes no code and opens no pull request.

## Start it with one API call

`POST https://api.prix.dev/o/{org}/sessions`

```json
{
  "agent": "claude",
  "input": "Report the untested code in this Flask app without changing anything. Read tests.py, config.py, app/__init__.py, app/models.py, app/main/routes.py, app/auth/routes.py, app/api/users.py, app/api/tokens.py, app/api/auth.py, app/errors/handlers.py, app/tasks.py, app/translate.py, app/search.py, app/email.py and requirements.txt first. Install the requirements and the coverage package in the sandbox, run the existing unittest suite under coverage with branch measurement and source limited to the app package, and produce the terminal report with missing lines and the HTML report. Write coverage-gaps.md: the exact command and the measured overall line and branch coverage, the per-file table, then the ten riskiest untested functions or routes ranked by what they protect, authentication, tokens, account writes and messaging first, each with its missing lines and a short test sketch using the app's TestConfig, the Flask test client and the auth headers its API expects, and a final section on code that needs Elasticsearch, Redis, mail or the translation API and how to stub each. Change no code and open no pull request.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
```

1. **Get an API key.** In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
2. **Send the request.** POST the body in request.json to https://api.prix.dev/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
3. **Read what came back.** Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

## What comes back

What comes back is `coverage-gaps.md` as an artifact on the session, which the session read returns with a download link good for an hour. It opens with the command that was run and the overall line and branch percentages from that run, followed by the per-file table from `coverage report -m` with the missing line ranges, so a reader knows every number is measured against the suite as it is today.

Then come the ten riskiest gaps, each with the file and function, the lines not executed, one or two sentences on what breaks silently if that code regresses, and a test sketch written for this app: which fixture config to use, which route to call, which header or form field to send and what to assert. A closing section lists the modules that depend on Elasticsearch, Redis, mail and the translator, says which the existing `TestConfig` already neutralizes, and gives the monkeypatch for each of the others.

Before acting on it, a reviewer checks the ranking against what the app actually exposes, picks the sketches worth turning into tests, and decides whether to add `coverage` to the development requirements so the number is tracked. A follow-up message to the same session asks the agent to write the tests for any sketch and open a pull request, since the sandbox still has the environment installed.

## Run it on every pull request to the default branch

A team that wants the gap list refreshed as the code changes starts one run per pull request with a `POST /o/{org}/sessions` naming the pull request branch in `resources`, as `https://github.com/your-org/your-app/pull/123`, and this prompt. A GitHub Actions job can send that request with the key from its secrets and post the artifact link as a comment.

Each run measures in its own sandbox, so two pull requests never share a database file or a coverage data file. A webhook on the key receives `run.completed` with the `session_url`, and reading the session returns the artifact's fresh download link.

## Why run it on Rush?

A coverage number from a laptop depends on which services happened to be running and which environment variables were set; this app builds a Redis client on start and reads `ELASTICSEARCH_URL`, `MAIL_SERVER` and `MS_TRANSLATOR_KEY` from the environment. In a Rush sandbox the agent starts from a clean environment, installs exactly `requirements.txt` plus `coverage`, and runs the suite the way CI would, so the report describes the code rather than one developer's machine.

The report carries its own proof. The `coverage run` and `coverage report -m` output is in the transcript beside the table in the report, so a reviewer can check any percentage or missing-line range against the raw run, and the ranking is tied to named functions and lines rather than to impressions.

## Frequently asked questions

### Does it add tests or change the suite?

No. The result is a report with test sketches, and the repository is left as it was, including `requirements.txt`. If you want the tests written, a follow-up message to the same session names the sketches and the agent opens a pull request with them.

### Does the suite need Redis or Elasticsearch to run under coverage?

Not as it stands. `create_app` builds a Redis client and an `rq` queue without connecting, and `TestConfig` sets `ELASTICSEARCH_URL` to `None`, so the four existing tests run with no services. The report says which untested modules would need a service or a stub before a test can reach them.

### How does it decide what is riskiest?

By what the code protects, not by line count. Login, registration, password reset tokens, API tokens and the endpoints that create or update users rank first because a regression there is a security or data problem, messaging and following next because they write rows, and pages that only render come last even when they have more uncovered lines.

### Are the test sketches specific to this app?

Yes. Each one names the route, the `TestConfig` to build the app with, the `WTF_CSRF_ENABLED` setting a form post needs, the Basic or Bearer header the `api` blueprint expects from `app/api/auth.py`, and the function to monkeypatch so mail or the task queue stays out of the test.