All jobs

Fix Django Exceptions From PostHog Error Tracking

Yes. An agent reads your top PostHog error-tracking issue, maps its stack to your Django code, reproduces it in pytest and opens a pull request.

Run this on RushIn a Project that has your repository.

What the agent does

The agent starts in PostHog, not in the code. Through the PostHog connection it lists the project's error-tracking issues from the last 7 days, sorts them by occurrences and takes the first. It reads that issue's exception type, message and stack frames, and how many users and sessions hit it. When a run reaches PostHog's MCP server, Rush also loads PostHog's own investigating-error-issue skill into it, so the agent follows PostHog's guidance for reading an issue instead of guessing at the tool calls.

Then it maps the frames to files. In a Django app like paperless-ngx, a frame in the document pipeline lands in src/documents/consumer.py (1,167 lines, with ConsumerPlugin.run, ConsumerError and ConsumeFileDuplicateError), and a frame in the REST API lands in src/documents/views.py (5,713 lines) or serialisers.py. PostHog groups an exception by its stack, so one issue with many occurrences usually points at one line, such as an unhandled None from a parser or a filter that assumes a custom field exists. The agent reads the code around that line and the commits that touched it recently.

It reproduces the exception before fixing it. paperless-ngx tests with pytest 9 through pytest-django, with DJANGO_SETTINGS_MODULE = "paperless.settings", testpaths covering src/documents/tests/ and three other apps, and --numprocesses=auto from pytest-xdist. The agent installs what CI installs, uv sync --group testing --frozen on Python 3.11 or later with Django ~=5.2.13, plus the system packages the backend workflow names (unpaper, tesseract-ocr, imagemagick, ghostscript, poppler-utils). It adds a test beside the existing ones, for example in test_consumer.py or test_api_documents.py, that raises the same exception PostHog recorded, and confirms it fails.

Then it fixes the cause, runs the new test and the module's suite with uv run pytest (leaving out tests marked live, gotenberg or tika, which need services the sandbox does not run unless you ask), and opens a pull request. The upstream repository sends no events to PostHog itself, so in your app the exception reaches PostHog through the capture you added; the agent does not change that wiring unless the bug is in it.

Start it with one API call

POST https://api.prix.dev/o/{org}/sessions with this body:

{
  "agent": "claude",
  "input": "Use the PostHog connection to list this project's error-tracking issues from the last 7 days and take the one with the most occurrences. Read its exception type, message and stack frames. Map the frames to the source under src/, most likely src/documents/consumer.py or src/documents/views.py, and read the recent commits that touched those lines. Install with `uv sync --group testing --frozen`, write a pytest test beside the existing tests that raises the same exception, and confirm it fails. Fix the cause, then run `uv run pytest` on that test and its module until they pass; skip tests marked live, gotenberg or tika. Open a pull request whose description names the PostHog issue, its occurrence count over the 7 days and the frame that pointed to the bug. Do not change anything in PostHog.",
  "connections": [
    "posthog"
  ],
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.prix.dev/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

What comes back is a pull request with two parts: the fix, usually a few lines in the module the top stack frame named, and a new pytest test that raised the PostHog exception before the fix and passes after it. The description names the PostHog issue, its exception type and occurrence count for the 7 days the agent read, the frame that led to the file, and what the fix changes in behavior, such as returning a validation error instead of a 500.

The pull request's GitHub Actions checks run the full backend suite on it, including the parts the agent skipped in its sandbox, and the session's record keeps each PostHog tool call the agent made with what it returned. A reviewer can see exactly which issue it picked and why, rather than trusting a summary.

Before merging, a reviewer checks that the new test reproduces the real exception rather than a nearby one, that the fix handles the bad input instead of swallowing the error with a bare except, and that the occurrence count in the description matches what PostHog shows. After the release ships, the same PostHog issue should stop gaining occurrences.

Run it on every Monday morning

In the console, add an automation to the Project with this prompt and a schedule trigger, such as cron 0 8 * * 1 in your timezone. Each Monday it starts a run that takes the top issue of the past week. The automation runs as the member who set it up, with that member's PostHog connection, so the person whose PostHog access it uses is always the one who configured it.

To act on one issue as soon as it appears, use a webhook trigger instead: whatever watches PostHog for you POSTs the issue to the trigger's URL with Authorization: Bearer <secret>. The body follows the prompt as data, so change the prompt to fix the issue the body names rather than the top one. A retry sent with the same Idempotency-Key starts nothing new.

The use case this job follows, with its automation and trigger: Fix a bug from an alert.

Why run it on Rush?

Turning an error-tracking issue into a fix normally takes three tabs: PostHog for the stack, the editor for the code, and a terminal for a test suite that needs OCR tools and image libraries installed. The agent does all three in one sandbox. It installs tesseract-ocr, ghostscript and the rest there, so your machine does not need them, and it reaches PostHog through the connection without a personal API key pasted into a script.

The pull request is backed by data you can check: the PostHog issue it chose, the tool calls that read it, the failing test before the fix and the passing one after, all kept on the session. Because each run is its own sandbox, you can start one per issue for the top five instead of only the first, and review five independent pull requests.

Frequently asked questions

Can the agent resolve or change issues in PostHog?

The prompt tells it to change nothing in PostHog, and the session record shows every tool call it made, so you can confirm it only read. Mark the issue resolved yourself once the fix is released.

How does Rush reach my PostHog project?

Connect PostHog once with POST /me/connections {"type": "mcp", "name": "posthog", "url": "https://mcp.posthog.com/mcp", "auth": {"mode": "oauth"}}, open the authorize_url it returns while signed in to the console, and approve. Then any run that names posthog in connections reads PostHog as you.

What if the exception comes from the Angular front end, not Django?

paperless-ngx has an Angular app in src-ui/ with Jest and Playwright tests. If the top issue's frames are in front-end code, the agent maps them there and writes a Jest test instead, and the pull request says which side the bug was on.

What if it cannot reproduce the exception?

It says so in its answer and opens no pull request, listing the frames, the inputs it tried and what it would need, such as a sample document or a PostHog property it could not see. A fix with no failing test is not one a reviewer can trust.