# Turn Sentry Errors Into Fix Pull Requests for a Flask App

Yes. A Sentry alert starts a run; the agent reads the issue's stack trace, reproduces it with a pytest test against the Flask app and opens a pull request.

[Run this on Rush](https://cloud.getrush.ai/run/fix-sentry-errors/flask/), in a Project that has your repository.

## What the agent does

The run starts when a Sentry alert rule posts to your handler and the handler sends this request with the alert payload after the prompt. The agent reads the payload for the issue id, then uses the Sentry connection to fetch the issue and its latest event: the exception type and message, the culprit, the release and environment tags, and every in-app frame with its filename, function and line. For a Flask app the frames name files under the package, such as `CTFd/api/v1/challenges.py`, so the agent knows where to look before it opens the repository.

In a Flask app like CTFd/CTFd it then reads the code around those frames in its own clone. `pyproject.toml` pins Flask 2.1.3, Werkzeug 2.2.3, flask-restx 1.3.0 and SQLAlchemy 1.4.54 and requires Python 3.11; `uv.lock` is the source of truth and `requirements.txt` is exported from it, so the agent installs with uv. The challenges API lives in `CTFd/api/v1/challenges.py` as flask-restx resources: `ChallengeList`, `Challenge`, `ChallengeAttempt` at `/api/v1/challenges/attempt`, plus solves, files, hints, flags and ratings. `ChallengeAttempt.post` reads `challenge_id` from JSON or form data, loads the challenge with `first_or_404`, checks the `hidden` and `locked` states, checks prerequisites against the user's solves, and applies the anti-bruteforce window from the `incorrect_submissions_per_min` and `max_attempts_timeout` settings before it hands the request to the class `get_chal_class` returns. A `KeyError` there, when a plugin's challenge type is not installed, is what `Challenge.get` turns into a 500 whose message names the type.

Before changing anything the agent writes a test that reproduces the exception, in `tests/api/v1/test_challenges.py` next to `test_api_challenge_attempt_post_private` and the other attempt tests. The helpers in `tests/helpers.py` build the fixture: `create_ctfd()` for an app with a fresh database, `gen_challenge` and `gen_flag` for the data, `register_user` and `login_as_user` for a client, `destroy_ctfd` at the end. The test sends the request the Sentry event's request data shows and asserts the response the code should have given. It runs that one test with `pytest -k` and keeps the failure output; the suite uses pytest 8.4.2 with pytest-xdist, pytest-randomly and freezegun from the dev dependency group.

Then it fixes the cause, in the API module or in `CTFd/plugins/challenges/__init__.py`, whose `BaseChallenge.attempt`, `solve` and `fail` class methods are what a custom challenge type overrides, and runs the test again until it passes. It runs `make test`, the target the `sqlite.yml` workflow runs with `TESTING_DATABASE_URL` set to `sqlite://`, and the Python half of `make lint`: `ruff check CTFd/ migrations/ tests/` and `ruff format --check`. The MySQL, MariaDB and Postgres workflows run the same suite against a database service; the agent says in the pull request which of those it could run in the sandbox.

## Start it with one API call

`POST https://api.prix.dev/o/{org}/sessions`

```json
{
  "agent": "claude",
  "input": "A Sentry alert fired for this Flask app; its payload follows this prompt. Use the Sentry connection to read the issue, its latest event and the in-app stack frames. Find the failing code, starting from the frames in CTFd/api/v1/challenges.py and the challenge classes in CTFd/plugins/challenges/__init__.py. Add a pytest test in tests/api/v1/test_challenges.py, built with the helpers in tests/helpers.py, that reproduces the exception, then fix the cause until that test passes. Install with uv, run `make test` with TESTING_DATABASE_URL=sqlite:// and `ruff check CTFd/ migrations/ tests/`. Open a pull request that links the Sentry issue, quotes the exception and the frames, and shows the test failing before the fix. Change nothing unrelated to this error. If you cannot reproduce it, open no pull request and say what you tried.\n\nAlert payload:\n{payload}",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ],
  "connections": [
    "sentry"
  ]
}
```

1. **Get an API key.** In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
2. **Send the request.** POST the body in request.json to https://api.prix.dev/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
3. **Read what came back.** Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

## What comes back

What comes back is a pull request on its own branch with two kinds of change: the fix in `CTFd/api/v1/challenges.py` or the plugin module, and a new test function in `tests/api/v1/test_challenges.py` that fails on the old code. The description links the Sentry issue, quotes the exception line and the in-app frames it fixed, shows the test failing before and passing after, and lists the commands it ran. Your repository's own workflows run on the branch as they do on any pull request.

The session keeps the record: each Sentry call the agent made, every `pytest` and `ruff` command with its full output, and every file it read. `GET /o/{org}/sessions/{id}/messages` returns that transcript, and the `run.completed` webhook carries the agent's final summary as `data.result`, so your handler can post the pull request link back to the Sentry issue or to a channel.

Before merging, a reviewer checks three things: that the test fails for the reason Sentry reported and not a nearby one, that the fix does not widen a `try` block or swallow the exception, and that a change to the attempt flow keeps the state, prerequisite and rate-limit checks in the same order. If the right fix is different, a follow-up message to the same session asks for it and the agent continues on the same branch.

## Run it on every Sentry issue alert

In Sentry, give the project an alert rule whose action is a webhook and point it at a small handler on your server. The handler sends this request with the alert's JSON after the prompt and the Sentry issue id as the `Idempotency-Key` header, so a resent alert returns the run it already started instead of a second one. One alert becomes one run and one pull request.

Set a webhook URL on the API key and Rush POSTs `run.completed` to it when the turn finishes, with `session_url` and the result. Or have that handler call a `webhook` trigger on an automation in the Project with this prompt: it forwards the alert's JSON as the body with the trigger's secret as `Authorization: Bearer`, because Sentry signs its own webhooks and cannot send that header. The body becomes the event text after the prompt, and `max_runs_per_hour` caps an issue that keeps firing.

## Why run it on Rush?

Fixing a Sentry issue by hand means reading the stack trace in one tab, finding the commit in another, setting up a database to reproduce it, and then convincing a reviewer that the test you wrote is the same failure. In a Rush sandbox the agent does that in one place: it reads the issue through your connection, installs with uv, runs the suite against SQLite the way CI does, and nothing on anyone's laptop changes.

The pull request carries its proof with it: the failing test, the passing test and the transcript of every command. Several issues can be worked at the same time in separate sandboxes, up to the key's `max_concurrent` runs, so the issues that pile up after a release each get a proposed fix instead of waiting in a queue for one person.

## Frequently asked questions

### How does the agent reach my Sentry project?

Connect Sentry once with `POST /me/connections` as an MCP server and name it in the run's `connections` field, as the request on this page does. A run gets only the connections it names. If you would rather it never resolves or comments on an issue, say so in the prompt; every call it makes is in the transcript.

### Does it need MySQL to run the CTFd tests?

No. The suite runs against SQLite when `TESTING_DATABASE_URL` is `sqlite://`, which is what the `sqlite.yml` workflow does. The sandbox can also run MySQL, MariaDB or Postgres when the fix touches a query that behaves differently there, and the pull request says which it used.

### What if the error is in a challenge plugin?

The agent follows the frames. Challenge types subclass `BaseChallenge` in `CTFd/plugins/challenges/__init__.py` and override `attempt`, `solve` and `fail`; a frame inside a directory under `CTFd/plugins/` is fixed there, with a test that exercises that plugin's type.

### What happens when it cannot reproduce the exception?

It opens no pull request. Its final message says which request bodies and settings it tried, what the event's breadcrumbs showed, and what it would need, such as a request body Sentry scrubbed. You can answer in a follow-up to the same session and it continues from there.