All jobs

Find the Test Coverage Gaps in a Django REST Framework API

Yes. An agent runs your pytest suite under coverage, maps every DRF API view and service to the tests that reach it, and writes a report of what has no test.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

The agent clones the project and reads how its tests run before adding anything. In a project laid out like HackSoftware/Django-Styleguide-Example, pytest.ini sets DJANGO_SETTINGS_MODULE = config.django.test and matches tests.py, test_*.py and *_tests.py; config/django/test.py turns DEBUG off, swaps in MD5PasswordHasher, runs Celery eagerly and uses a local-memory cache; requirements/local.txt pins pytest 8.4.2, pytest-django 4.11.1, factory-boy, Faker, mypy 1.18.2 and ruff 0.14.0 but no coverage plugin; requirements/base.txt has Django 5.2.7 and djangorestframework 3.16.1; and .github/workflows/django.yml runs ruff check ., mypy --config mypy.ini styleguide_example/ and pytest against a Postgres service, with a second job that does the same inside docker compose. The agent installs coverage into the sandbox for this run only and leaves the requirement files alone.

Then it maps the API surface. config/urls.py mounts styleguide_example.api.urls under api/, which includes auth/, users/, errors/, files/ and google-oauth2/. Under users/ there are four APIView classes in users/apis.py: UserListApi with a FilterSerializer, LimitOffsetPagination and get_paginated_response, UserDetailApi, UserCreateApi and UserUpdateApi, each delegating to user_list, user_get, user_create and user_update in selectors.py and services.py. Under auth/ are UserSessionLoginApi, UserSessionLogoutApi, UserJwtLoginApi, UserJwtLogoutApi and UserMeApi, the last two behind ApiAuthMixin, which stacks CsrfExemptedSessionAuthentication, SessionAsHeaderAuthentication and JSONWebTokenAuthentication. Under files/ are the four upload views backed by FileStandardUploadService and FileDirectUploadService, and emails/services.py holds email_send, email_failed and email_send_all with a configurable failure rate.

It runs coverage run -m pytest and coverage report -m, then reads the result against that map rather than quoting a percentage. Reading the test tree gives the shape of the gaps before the numbers do: authentication/tests/apis/test_user_login.py covers session and JWT login, logout and UserMeApi with six cases; users/tests/services/test_user_create.py covers user_create but nothing exercises UserListApi, UserDetailApi, UserCreateApi or UserUpdateApi over HTTP; files/tests/flows/test_standard_upload.py and test_direct_upload.py each contain a TestCase class whose body is a docstring describing the cases that should exist and no test methods; and no test imports emails.services or the Celery task behind it. The errors app's TriggerErrorApi walks both exception handlers in api/exception_handlers.py but is itself untested, so a change to drf_default_with_modifications_exception_handler would pass CI.

The report, coverage-gaps.md, lists each untested view, service and selector with its file and line, the branches coverage report -m marks as missed inside partly tested modules such as UserSessionLoginApi when authenticate returns None, and the two upload test files that promise cases without delivering them. For each gap it proposes the test that would close it in the project's own style, TestCase with APIClient and reverse("api:users:list"), and orders the list by how much of the public API the gap exposes. The run writes nothing to the branch.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "claude",
  "input": "Find the test coverage gaps in this Django REST Framework project and write coverage-gaps.md. Read pytest.ini, config/django/test.py, requirements/local.txt, .github/workflows/django.yml, config/urls.py and styleguide_example/api/urls.py, then every apis.py, urls.py, services.py and selectors.py under styleguide_example and every file under a tests directory. Install coverage for this run only, without editing the requirement files, and run coverage run -m pytest followed by coverage report -m. For every APIView, service function and selector, say which test reaches it or that none does, with file and line. Call out test files that contain TestCase classes with no test methods. List the missed branches inside modules that do have tests. For each gap propose one test in the project's own style (TestCase, APIClient, reverse with the api namespace). Order the gaps by how much of the public API they expose. Do not change any file in the repository.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

What comes back is coverage-gaps.md as an artifact on the session, with a download link that works for an hour and is refreshed on each read. It opens with how the suite was run and what settings it ran under, then lists the gaps: views with no HTTP test, services and selectors no test imports, empty test classes, and missed branches in modules that do have tests, each with file:line and the proposed test. The session's result field carries the short version: the number of views, services and selectors with no test, and the two or three gaps that matter most.

The transcript shows the coverage run and coverage report commands and their output as tool_use and tool_result events, so a reader can confirm the suite passed in the sandbox before the gaps were counted, and that the counts come from the tool rather than from reading file names.

A reviewer uses the ordered list as a backlog: the empty upload test classes and the untested users views come first because they sit on public routes, the exception-handler walk next because it guards every error response, and the email services after that. A follow-up message to the same session can ask the agent to write the first of those tests on a branch.

Run it on every Monday morning

Add an automation to the Project with this prompt and a schedule trigger: cron set to Monday at 08:00 in the team's timezone. Each fire starts a run on the default branch, reads the tests as they are that week and writes a fresh report, so a view added on Thursday without a test shows up in the next report rather than in an incident. Set the automation to read only, since this job changes nothing.

For a report tied to a release, send POST /o/{org}/sessions with this body from the release script, with a request_id that is a UUID v5 of the tag so a rerun returns the same run, and give the key a webhook_url so run.completed arrives with data.session_url to fetch the artifact from.

The use case this job follows, with its automation and trigger: Produce a report or file, no repo.

Why run it on Rush?

Coverage numbers are cheap; knowing which public route has no test behind it is not. That needs the suite to actually run, under the project's test settings with a Postgres database, and someone to read the missed lines against the URL map and the service layer. In a Rush sandbox the agent starts the database, installs the one tool the run needs without touching the requirement files, runs the suite, and keeps every command and its output in the run's record, so the report says what was run and what it found.

Because the result is a file and a summary rather than a branch, the run can repeat on a schedule or per release with no risk to the repository, and the same request can run against several Django services at once, each in its own sandbox.

Frequently asked questions

Does it add pytest-cov to the project?

No. The prompt says to install coverage for the run only and leave requirements/local.txt alone. If the team wants coverage in CI, the report notes that django.yml runs pytest directly, and a follow-up can add the plugin and a threshold on a branch.

How does it treat test classes with no test methods?

As gaps, listed separately. files/tests/flows/test_standard_upload.py and test_direct_upload.py each hold a TestCase subclass whose only content is a docstring describing the cases to write. pytest collects nothing from them, so coverage never reaches the upload services through those files; the report quotes the docstrings as the spec for the proposed tests.

Will it count the Django admin and management commands?

Only where the project's own code runs there. The admin_2fa example and the management commands under tasks are listed with whatever test reaches them, or none. Django's and DRF's own modules are left out of the report.

Can the same run write the missing tests?

Not in this request, which changes no file. Once the report is in, send a follow-up to the session naming the gaps to close, and the agent writes those tests on a branch and opens a pull request from the same sandbox.

More Django REST Framework jobs

The same job on other stacks

Keep reading