Audit a Django Repository per Customer Request

Each audit is one session, started by your backend with the customer's repository in resources. The agent begins with how the project pins its environment. In a Django codebase like django/djangoproject.com it finds pyproject.toml with requires-python = "==3.12.*", Django 6.1.1 and every other dependency pinned with ==, a uv.lock, and three files under requirements/ that uv export generated from it. One dependency, django-push, is pinned to a git commit rather than a release, so the agent lists it separately: it cannot be compared to a version on PyPI. It then runs uv tree --outdated --depth 1 against the lock and keeps only the direct dependencies whose latest release is at least one major version ahead of the pin.

Finding the test command takes more than manage.py test. The Makefile has a test target that runs coverage run --source=. --module manage test --verbosity 2 over an APP_LIST of fifteen apps, from accounts to tracdb. The project's own CI in .github/workflows/tests.yml shows what that needs: Postgres with two databases, djangoproject and code.djangoproject, the second loaded from tracdb/trac.sql, and a conf/secrets.json one directory above the checkout, which djangoproject/settings/common.py reads from DATA_DIR. The agent sets up the same databases in its sandbox, writes the same throwaway secrets file outside the repository, and runs make test. When the Playwright tests need browsers, it installs them the way the workflow does, with python -m playwright install --with-deps chromium firefox.

For risky patterns it reads code rather than grepping and stopping. It searches for the usual Django suspects (mark_safe, csrf_exempt, .raw(, .extra(, DEBUG = True, hard-coded secrets, |safe in templates) and opens each hit to judge it. In this repository the @csrf_exempt on receive_webhook in fundraising/views.py sits above a stripe.Webhook.construct_event call that verifies the signature, so it is a deliberate exemption, not a finding. The mark_safe(output) in docs/templatetags/docs.py and the mark_safe( help text in members/models.py are checked for whether any user-supplied value reaches them. Only patterns the agent can explain go in the list, each with its file, line and a one-sentence reason, up to the ten the prompt allows.

The prompt forbids changing files, and the agent works on a fresh clone in its own sandbox, so nothing it does reaches the customer's repository. It ends the turn with the JSON as its final answer: three arrays, outdated_dependencies, tests and risky_patterns, in the field names the prompt gives.

What comes back is the agent's final answer, the JSON, in the session's result. The run.completed webhook carries it in data.result when it fits in 16 KiB; a larger answer is left out of the webhook, and your server reads it from data.session_url with the key. Each tests entry says which command ran and what it needed (for this repository, the two Postgres databases, the Trac dump and the secrets file), so a failure caused by missing setup can be told apart from a failing test. Each risky_patterns entry points at a file and line your customer can open.

Before showing the audit to the customer, your product should validate the JSON against the three arrays it expects and treat anything else as a failed audit. A person reviewing a sample of audits checks that outdated_dependencies lists only direct dependencies, that the git-pinned ones are marked as such, and that each risky pattern has a reason specific to the code rather than a generic warning. The transcript, GET /o/{org}/sessions/{id}/messages, has every command and its output if a customer asks how a result was reached.

When a customer connects a repository in your product, your backend sends POST /o/{org}/sessions with that repository and this prompt, and stores the execution_id from the 202 next to the customer's id: the request has no metadata field, so that mapping is yours. Send an Idempotency-Key built from the customer and the commit, so a retried request returns the audit already running. A scheduled job can repeat the audit weekly for every connected repository, up to the key's max_concurrent runs at once (4 by default; one more is 429).

Set a webhook_url on the key. Rush POSTs run.completed, run.failed or run.cancelled when the turn ends, signed per Standard Webhooks, and retries three more times with the same webhook-id if your endpoint does not answer 2xx within 5 seconds. Your handler verifies the signature, looks up the customer by execution_id, parses data.result and saves the audit.

Running a stranger's Django test suite on your own servers means installing their dependencies, creating their databases and executing their code next to everyone else's. Each Rush session gets its own sandbox, so one customer's make test, Postgres databases and secrets file never share a machine with another's, and a test that does something unexpected stays inside that one sandbox.

An audit is worth what its evidence is worth. The tests entries come from commands that ran, the outdated list from the lockfile and the package index, and the risky patterns from files the agent opened, with every step kept in the transcript. Your product can show the customer the JSON and, when they question a line, the exact command behind it.

How does the agent get access to my customer's repository?

What if the tests need services the repository does not describe?

Can the agent change the customer's code?

Can I enforce the JSON shape?