What the agent does
The agent clones the repository into its own cloud sandbox and reads how tests already run before it adds anything. In a project shaped like psf/requests, .github/workflows/run-tests.yml has a build job that runs on every push and pull request across a matrix of eight Python versions (3.10 through 3.15-dev and pypy-3.11) on Ubuntu, macOS and Windows, plus two single-runner jobs, no_chardet and urllib3. Every job installs with make, whose first target is init (pip install -r requirements-dev.txt), and tests with make ci, which is python -m pytest tests --junitxml=report.xml. No job measures coverage today.
Then it finds what coverage tooling is already there, so it adds as little as possible. requirements-dev.txt already lists pytest-cov, the test dependency group in pyproject.toml lists it too, and the Makefile has a coverage target that nothing in CI calls: pytest --cov-config .coveragerc --cov-report term --cov-report xml --cov=src/requests tests. The agent also reads .coveragerc, which has one rule, omit = requests/packages/*. Since the package moved under src/, that pattern may no longer match anything, so the agent checks what the report actually includes and says in the pull request whether the omit still does something.
Next it measures. It installs the dev requirements in the sandbox, runs the existing make coverage target, and reads the total from the terminal report. That figure, rounded down to a whole percent, becomes the threshold, passed as --cov-fail-under or written as fail_under under [report] in .coveragerc. It does not pick a round number it hopes for: the floor is what the suite covers today, so the first run passes and any later change that lowers it fails.
Last, it edits the workflow. Coverage runs in one place rather than in all 23 matrix cells (24 combinations minus the excluded PyPy on Windows): one Ubuntu job on a single Python version runs the tests with --cov and --cov-report xml, then uploads coverage.xml with actions/upload-artifact, pinned by commit SHA like the actions/checkout and actions/setup-python steps the workflow already pins. The other matrix jobs keep running make ci unchanged, so the coverage step adds one run instead of slowing every cell. It runs the coverage command again with the threshold set to confirm it passes before it pushes.
Start it with one API call
POST https://api.prix.dev/o/{org}/sessions with this body:
{
"agent": "claude",
"input": "Add test coverage reporting to this repository's GitHub Actions tests. First read the workflow that runs the tests, the Makefile or tox config it calls, and any existing coverage config (.coveragerc, [tool.coverage] in pyproject.toml, pytest-cov in the dev requirements), and reuse what exists. Install the dev requirements in the sandbox, run the test suite with pytest-cov, and use today's total, rounded down to a whole percent, as the fail-under threshold. Collect coverage in one job on one Python version on Ubuntu, not in every matrix cell, upload coverage.xml as a workflow artifact, pin any new action by commit SHA the way the workflow already pins actions, and leave the other jobs unchanged. Run the coverage command again with the threshold to show it passes. Open a pull request that states the measured percentage, which job collects it, and anything stale you found in the existing coverage config.",
"resources": [
{
"url": "https://github.com/your-org/your-app"
}
]
}- Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
- Send the request. POST the body in request.json to https://api.prix.dev/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
- Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.
What comes back
What comes back is a pull request on its own branch. In a repository like requests, the diff touches .github/workflows/run-tests.yml (one job gains the coverage run and an upload step) and the coverage config, either .coveragerc with a fail_under line or the Makefile coverage target with --cov-fail-under. The description states the measured total and the threshold set from it, names the job that collects coverage and why the rest of the matrix does not, and reports what it found in the existing config, such as an omit pattern that points at a path that moved.
The pull request's own checks run the edited workflow, so the coverage job either passes at the new floor or shows the failure in the Actions log, and coverage.xml appears as an artifact on that run. The session transcript keeps every command with its output: the install, the first measuring run with its full per-file report, and the run with the threshold set.
Before merging, a reviewer checks three things: that the threshold matches the number in the transcript rather than a rounded-up guess, that coverage is collected on the Python version the team cares most about, and that the upload step's action is pinned the same way the workflow pins the others.
Run it on every new Python repository
For a team that keeps adding services, the same request runs once per repository: a script on your server sends one POST /o/{org}/sessions per repository URL in resources, and each run gets its own sandbox and its own branch. A key runs at most its max_concurrent sessions at once (4 by default), and one more answers 429, so the script starts the next when one finishes.
Give the key a webhook_url and Rush posts run.completed or run.failed to it when each run's turn finishes, with the session_url to read the pull request link and cost from.
The use case this job follows, with its automation and trigger: Write the missing tests.
Why run it on Rush?
The threshold is only honest if someone measured it, and measuring means installing the full dev requirements, including httpbin and pytest-httpbin for a library like requests, and running the whole suite. In a Rush sandbox that happens on a clean machine with nothing left over from a laptop's virtualenv, so the number in the pull request is the number CI will see on Ubuntu, not one inflated or deflated by a local install.
The record is the proof: the transcript holds the measuring run's full report with every file's percentage, so a reviewer can see where the floor came from and which modules pull it down. The pull request's checks then show the edited workflow passing on GitHub's runners before anyone merges it.
Frequently asked questions
Will it run coverage on every Python version in my matrix?
No. The prompt asks for one coverage job on one Python version on Ubuntu, because collecting the same report in every matrix cell adds run time and only one report gets uploaded. The other jobs keep running the tests the way they did.
How does it choose the fail-under number?
It runs the suite with pytest-cov in the sandbox and takes the total, rounded down to a whole percent. The measuring run is in the transcript with its per-file report, so the reviewer sees exactly where the number came from. Raising it later is a one-line change.
We already have pytest-cov and a coverage target. Will it add a second setup?
The prompt tells it to reuse what exists. In a repository like requests it uses the pytest-cov already in requirements-dev.txt and the flags of the existing make coverage target, and puts the threshold in .coveragerc instead of creating a new config file.
Does it upload to Codecov or Coveralls?
Not unless you ask. The default is a workflow artifact, which needs no account or token. If you want a hosted service, say so in input and add the token as a repository secret yourself; the agent never sees your GitHub secrets.