# Add Test Coverage Reporting to Wagtail CI

Yes. An agent runs your Wagtail tests under coverage.py in its sandbox, adds a threshold at the measured number and an HTML report artifact to CI, and opens a PR.

[Run this on Rush](https://cloud.getrush.ai/run/add-coverage-to-ci/wagtail/). Opens the console with this prompt ready to send, in a Project that has your repository.

## What the agent does

The agent reads the workflow and the test settings before it adds anything. In a Wagtail site like wagtail/bakerydemo, `.github/workflows/ci.yml` has three jobs: `lint-server`, which installs `requirements/development.txt` with `uv` and runs `make lint-server` (ruff format and check, `curlylint --parse-only`, `djhtml --check`); `lint-client`, which runs the npm linters on the Node version in `.nvmrc`; and `test-server`, which sets `DJANGO_SETTINGS_MODULE=bakerydemo.settings.test`, installs the same requirements with `uv`, and runs `./manage.py check`, `./manage.py makemigrations --check --noinput` and `./manage.py test`. `requirements/base.txt` pins Django 6.0 and Wagtail 8.0, and `development.txt` adds only `ruff`, `curlylint` and `djhtml`, so `coverage` is not installed anywhere. `bakerydemo/settings/test.py` imports the base settings, switches the cache to local memory, the password hasher to MD5 and the task backend to `ImmediateBackend`, and without a `DATABASE_URL` the base settings fall back to SQLite, which is how the suite runs on GitHub. The tests themselves are `bakerydemo/base/tests/test_home_page.py` and `test_seo.py`, `WagtailPageTestCase` classes that render the home page and check the SEO metadata and `robots.txt`; the `blog`, `breads`, `locations`, `people` and `search` apps have no tests. Nothing reads a coverage number.

The change is three files. `coverage` goes into `requirements/development.txt` beside the linters, pinned. A `.coveragerc` sets `source = bakerydemo`, `branch = true` and an `omit` list for `*/migrations/*`, `*/tests/*`, `bakerydemo/settings/*` and `manage.py`, so the number measures the models, blocks, views and template tags rather than generated code. In `ci.yml`, the `test-server` job's last step becomes `uv run coverage run ./manage.py test`, followed by `uv run coverage report --fail-under=<measured>` and `uv run coverage html`, and an `actions/upload-artifact` step uploads `htmlcov/` so a reviewer can open the report from the workflow run. The `check` and `makemigrations --check` steps stay as they are, and the two lint jobs are untouched.

The measured number comes from the sandbox. The agent creates the `uv` virtual environment the workflow does, installs `requirements/development.txt`, exports the test settings module, runs `coverage run ./manage.py test` and `coverage report`, and takes the `TOTAL` line. With two test modules that render the home page, the figure covers `base/models.py`, `base/blocks.py` and the template tags the home page uses and leaves most of `blog` and `breads` uncovered, so the threshold is set at that total rounded down, not at a target: the pull request passes on the code as it is, and only a drop fails later. It also adds `htmlcov/` and `.coverage` to `.gitignore`.

The agent does not write tests in this run and does not change `settings/test.py` or the `Makefile`; `make lint-server` still passes because `.coveragerc` is not a Python file and `ruff` has no opinion on it. If the team runs tests against PostgreSQL locally by setting `DATABASE_URL`, the coverage command is the same, since `coverage run` wraps `manage.py` whatever the database.

## Start it with one API call

`POST https://api.phoenix.so/o/{org}/sessions`

```json
{
  "agent": "codex",
  "input": "Add test coverage reporting with a threshold to this Wagtail project's CI and open a pull request. Read .github/workflows/ci.yml, requirements/base.txt, requirements/development.txt, bakerydemo/settings/test.py, Makefile and every file under bakerydemo/*/tests first. Add a pinned coverage to requirements/development.txt and a .coveragerc with source = bakerydemo, branch = true, and an omit list for migrations, tests, the settings package and manage.py. In the sandbox create the uv environment the test-server job uses, set DJANGO_SETTINGS_MODULE=bakerydemo.settings.test, run coverage run ./manage.py test and coverage report, and read the TOTAL. In the test-server job replace the test step with uv run coverage run ./manage.py test, then uv run coverage report --fail-under set to that total rounded down, then uv run coverage html, and upload htmlcov as an artifact. Add htmlcov and .coverage to .gitignore. Do not add tests and do not change the lint jobs, settings/test.py or the Makefile. Put the measured total and the coverage report in the pull request description.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
```

1. **Get an API key.** In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
2. **Send the request.** POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
3. **Read what came back.** Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

## What comes back

What comes back is a pull request that changes `.github/workflows/ci.yml`, `requirements/development.txt` and `.gitignore` and adds `.coveragerc`. The description states the total measured in the sandbox, quotes the `coverage report` table with its per-file lines, and names the Django, Wagtail and Python versions it was measured with. The session keeps the full test output, and the workflow run on the pull request attaches `htmlcov/` as an artifact.

A reviewer checks that the `--fail-under` value equals the total in the description, that the `omit` list leaves out only generated and test code, and that `check` and `makemigrations --check` still run before the suite. The pull request's own `test-server` job is the first run of the threshold on GitHub and should pass, because the number came from the same command on the same code; the two lint jobs run unchanged.

## Run it on every new Wagtail site repository

Run it once per site rather than once: a script that reads `GET /o/{org}/github` for the installation's repositories and calls `POST /o/{org}/sessions` with this body and each repository's URL in `resources`, with a `request_id` per repository so a retry returns the run already started. Each site gets its own sandbox, its own measurement and its own threshold.

Give the key a `webhook_url` and Rush POSTs `run.completed`, `run.failed` or `run.cancelled` when each turn ends, signed per Standard Webhooks, with `data.session_url` to read for the pull request. A key runs at most `max_concurrent` sessions at once, 4 by default; a batch over many Wagtail sites can still be sent in full, since each site past that number is queued (`202`, `status: "queued"`) and its run starts, oldest first, when a slot frees.

## Why run it on Rush?

A threshold set by guessing is either ignored or fails on the first pull request. The agent gets the real number by running the real suite under `coverage` in its own sandbox, with the same `uv` environment and the same settings module the workflow uses, and the `coverage report` table is on the session, so the value in the workflow has a recorded source and the per-file lines show where the gaps are.

The job ends at a pull request whose own CI run has executed the new check, not at a suggested YAML block. The same request body works on every Wagtail site a team maintains, each with its own measured figure, and a follow-up on the same parked session can write the first tests for the `blog` or `breads` app using the report to choose where.

## Frequently asked questions

### Where does the threshold come from?

From running `coverage run ./manage.py test` and `coverage report` in the sandbox with `bakerydemo.settings.test`, and reading the `TOTAL` line. It is rounded down and written as `--fail-under` in the workflow, and the pull request description states it with the full table.

### Will it write tests to raise coverage?

Not in this run. It measures what the existing page-rendering tests cover and stops a drop. The `htmlcov/` artifact and the report table show that `blog`, `breads`, `locations`, `people` and `search` have no tests, which is where a follow-up would start.

### Does it need PostgreSQL?

No. Without `DATABASE_URL` the base settings use SQLite, which is what the `test-server` job already relies on, and `coverage run` wraps `manage.py` either way. A team that sets `DATABASE_URL` in the job gets the same measurement against PostgreSQL.

### Does it change the lint jobs or the Makefile?

No. `lint-server` and `lint-client` are untouched, and `.coveragerc` is not something `ruff`, `curlylint` or `djhtml` read. The only workflow change is inside the `test-server` job, after the `check` and `makemigrations --check` steps.

## More Wagtail jobs

- [Add CSV Export to a Wagtail Admin](https://getrush.ai/run/add-csv-export/wagtail/)
- [Add Health Checks to a Wagtail Site](https://getrush.ai/run/add-health-checks/wagtail/)
- [Add a Language Switcher to a Wagtail Site](https://getrush.ai/run/add-language-switcher/wagtail/)

## The same job on other stacks

- [Add Test Coverage to CI for an aiohttp Project](https://getrush.ai/run/add-coverage-to-ci/aiohttp/)
- [Add Test Coverage Reporting to Angular CI](https://getrush.ai/run/add-coverage-to-ci/angular/)
- [Add Test Coverage Reporting to ASP.NET Core CI](https://getrush.ai/run/add-coverage-to-ci/aspnet-core/)

## Keep reading

- [How a Rush run works on your repository, from request to pull request](https://getrush.ai/cloud/)
- [Compare cloud agents, agent sandboxes and AI code review](https://getrush.ai/compare/)
- [Rush pricing: Free, Pro and Max](https://getrush.ai/pricing/)
