# Add Test Coverage to CI for an aiohttp Project

Yes. An agent adds pytest-cov to your aiohttp project's test target, combines coverage across every demo, publishes the report from CI, and opens a PR.

[Run this on Rush](https://cloud.getrush.ai/run/add-coverage-to-ci/aiohttp/). Opens the console with this prompt ready to send, in a Project that has your repository.

## What the agent does

The agent reads how the tests already run, because coverage has to be collected the same way. In a project laid out like aio-libs/aiohttp-demos, the root `Makefile` has a `test` target that is nine separate `pytest` invocations, one per demo directory under `demos/`, each with its own `pytest.ini`, its own package (`aiohttpdemo_polls`, `aiohttpdemo_blog`, `aiohttpdemo_chat`, `graph`, `imagetagger`, `moderator`, `moderator_bot`, `motortwit`, `shortify`) and its own `tests/` directory. The root `requirements-dev.txt` pins `pytest` and `pytest-aiohttp` and nothing for coverage, and `make install` pip-installs that file plus each demo's requirements. `.github/workflows/ci.yml` has a `test` job with Postgres and Redis service containers that runs `PYTHONASYNCIODEBUG=1 make test` on Python 3.12, a `lint` job for `make lint` and the docs spelling check, and a `doc` job. Nothing measures coverage, and nothing uploads anything.

Nine invocations mean nine separate measurements unless they share a data file, so the agent adds `pytest-cov` to the test section of `requirements-dev.txt`, writes a root `.coveragerc` whose `[run]` section sets `source` to `demos`, `branch` to true and omits every `tests/` directory, and changes each line of the `test` target to pass `--cov --cov-append --cov-report=` so the invocations accumulate into one `.coverage` file and print nothing until the end. A new `coverage` target then runs `coverage report -m` and `coverage xml`. Before it commits to that, the agent reads each demo's `pytest.ini`: `demos/polls/pytest.ini` turns every warning into an error with `filterwarnings = error`, so a warning the coverage plugin emits, for instance when a package is imported before measurement starts, would fail that demo's run, and the agent checks for exactly that when it runs the target.

In `ci.yml` the `test` job gains three steps after `make test`: `make coverage`, a step that appends the text report to `$GITHUB_STEP_SUMMARY` so the numbers show on the workflow run page, and `actions/upload-artifact` for `coverage.xml` and the `.coverage` data file. The agent does not add a Codecov upload or a minimum threshold, because the first number is the baseline and both are decisions the team makes once they have seen it; the description shows where a `--cov-fail-under` would go.

Then it runs the whole thing in its sandbox the way CI does: `make install`, a Postgres and a Redis started locally for the demos that need them, `PYTHONASYNCIODEBUG=1 make test` and `make coverage`. The per-file report is pasted into the pull request, so a reviewer sees which demos the tests reach and which modules in `demos/` have no test touching them at all.

## Start it with one API call

`POST https://api.phoenix.so/o/{org}/sessions`

```json
{
  "agent": "claude",
  "input": "Add test coverage to CI for this aiohttp project. Read Makefile, requirements-dev.txt, .flake8, .github/workflows/ci.yml and every demos/*/pytest.ini first. Add pytest-cov to requirements-dev.txt, write a root .coveragerc that measures demos with branch coverage and omits the tests directories, and change each pytest line of the Makefile test target to pass --cov --cov-append --cov-report= so the nine demos accumulate into one data file. Add a coverage target that prints coverage report -m and writes coverage xml. In ci.yml, after make test, run make coverage, append the report to the step summary and upload coverage.xml as an artifact. Do not add a threshold or a third-party upload. Install the requirements, start Postgres and Redis in the sandbox, run make test and make coverage exactly as CI does, check that no demo with filterwarnings set to error fails on a coverage warning, and open a pull request with the full report.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
```

1. **Get an API key.** In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
2. **Send the request.** POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
3. **Read what came back.** Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

## What comes back

The pull request changes `Makefile`, `requirements-dev.txt` and `.github/workflows/ci.yml`, adds `.coveragerc`, and adds `.coverage` and `coverage.xml` to `.gitignore`. Its description pastes the `coverage report -m` table from the sandbox run, demo by demo, and notes which demos' `pytest.ini` files have `filterwarnings = error` and that they passed with the plugin on.

The `CI` workflow runs on the pull request, and the `test` job's summary page now carries the same table, with `coverage.xml` attached as an artifact a reviewer can download. The `lint` and `doc` jobs are unchanged.

Before merging, a reviewer decides whether to add `--cov-fail-under` once the baseline is agreed, whether to send `coverage.xml` to a coverage service, and whether the demos with the lowest numbers in the table deserve tests before the threshold goes in.

## Run it on every nightly

Once the report is in CI, the useful question is what changed, and that is a job for a schedule. An automation on the Project with a `schedule` trigger set to a nightly slot in the team's timezone and the prompt "run make test and make coverage, compare the per-file report with the one in the latest run of the CI workflow on the default branch, and if any module in demos/ dropped, comment on the most recent pull request that touched it" posts a comment only when something moved. Automation runs are read-only, so the comment is all it posts.

A team that wants the gaps closed starts one run per demo from its own server, each a `POST /o/{org}/sessions` with the repository in `resources`, a prompt that names the demo directory and asks for tests on its uncovered functions, and a `request_id` that is a UUID v5 of the demo's path, so a retried call returns the run already started rather than opening a second pull request.

## Why run it on Rush?

Coverage across nine separately invoked test suites is a configuration problem before it is a testing problem: without `--cov-append` the last demo overwrites the rest, without `--cov-report=` nine partial tables scroll past, and a `pytest.ini` that turns warnings into errors can fail a demo for a plugin message nobody asked for. The agent reads those files, runs the target in its own sandbox with the same services CI uses, and fixes what breaks before it opens anything.

The install, the nine test runs and the final report stay with the run, and the pull request quotes the table. A team that keeps several aiohttp repositories gets the same change on each at once, with each repository's own baseline rather than a number copied from a sibling.

## Frequently asked questions

### Why one data file instead of a report per demo?

Because the `Makefile` runs the demos as nine `pytest` commands and the team wants one answer. `--cov-append` makes each command add to the same `.coverage` file instead of replacing it, and `--cov-report=` keeps each command quiet, so `make coverage` prints the single table once at the end.

### Will filterwarnings = error break the run?

It can, which is why the agent reads every `demos/*/pytest.ini` and runs the target before committing. A coverage plugin warning in a demo with that setting fails that demo. If one appears, the agent fixes its cause, usually the order in which a package is imported relative to measurement, rather than silencing the warning.

### Why no Codecov and no threshold?

Because the first run produces the baseline, and both decisions depend on it. A threshold set before anyone has seen the number is either too low to matter or fails the next pull request. The pull request shows the table and names where `--cov-fail-under` goes, so adding it later is one line.

### Do the tests need Postgres and Redis?

Some do, which is why `ci.yml` already starts both as service containers for the `test` job. The agent starts the same two in its sandbox so `make test` runs the way CI runs it, and the coverage it reports is the coverage CI will report.

## More aiohttp jobs

- [Add CSRF Protection to an aiohttp App](https://getrush.ai/run/add-csrf-protection/aiohttp/)
- [Add Cursor Pagination to an aiohttp App](https://getrush.ai/run/add-cursor-pagination/aiohttp/)
- [Add Alembic Migrations to an aiohttp App](https://getrush.ai/run/add-database-migrations/aiohttp/)

## The same job on other stacks

- [Add Test Coverage Reporting to Angular CI](https://getrush.ai/run/add-coverage-to-ci/angular/)
- [Add Test Coverage Reporting to ASP.NET Core CI](https://getrush.ai/run/add-coverage-to-ci/aspnet-core/)
- [Add Code Coverage to CI for an Axum API](https://getrush.ai/run/add-coverage-to-ci/axum/)

## Keep reading

- [How a Rush run works on your repository, from request to pull request](https://getrush.ai/cloud/)
- [Compare cloud agents, agent sandboxes and AI code review](https://getrush.ai/compare/)
- [Rush pricing: Free, Pro and Max](https://getrush.ai/pricing/)
