What the agent does
The agent begins with the monitor payload your server passes in and asks Datadog, through its MCP server, for the rest: the monitor's query and thresholds, the error logs for that service and environment in the half hour around the alert, and the APM traces for the failing requests. For a FastAPI service, a trace's resource name carries the route template, such as GET /api/v1/items/{id}, and its error span carries the exception type and the Python traceback. The agent groups those by the innermost frame in your own code, so ten thousand identical 500s become one problem with a sample request.
It then finds the route. In an app built on fastapi/full-stack-fastapi-template the routers live in backend/app/api/routes/ (items.py, users.py, login.py, private.py, utils.py), mounted under API_V1_STR = "/api/v1" from app/core/config.py. A failing GET /api/v1/items/{id} maps to read_item, which calls session.get(Item, id) and raises HTTPException(status_code=404, detail="Item not found") or 403 with "Not enough permissions". An unhandled 500 there means something before those checks broke, so the agent reads the SQLModel models in app/models.py, the helpers in app/crud.py and the latest Alembic revision before it decides where the fault is.
It reproduces the failure as a pytest test before it fixes anything. The template's suite in backend/tests/ uses FastAPI's TestClient, a session-scoped db fixture that calls init_db and deletes every Item and User at the end, and superuser_token_headers and normal_user_token_headers fixtures, so the new test in tests/api/routes/test_items.py or test_users.py sends the same request the trace shows, with the same kind of user. The tests need Postgres (compose.yml runs postgres:18), so the agent brings the database up, runs alembic upgrade head and python app/initial_data.py the way scripts/prestart.sh does, and installs with uv for the Python 3.14 that .python-version names.
With the test failing for the same reason the alert fired, the agent makes the smallest fix that handles the input class, runs the new test, then bash scripts/tests-start.sh, which runs the whole suite under coverage. The template's CI fails under 90 percent coverage (coverage report --fail-under=90), so the test also keeps the new branch covered.
Start it with one API call
POST https://api.prix.dev/o/{org}/sessions with this body:
{
"agent": "claude",
"input": "A Datadog monitor on this FastAPI service fired; its webhook payload is at the end of this prompt. Use the Datadog connection to read the monitor, the error logs and the APM traces for its service and environment from 15 minutes before to 15 minutes after it fired. Group the errors by exception and the innermost frame in this repository, map each trace's route template to its router in backend/app/api/routes, and find the code path that produced them. Start the database the tests need, apply the Alembic migrations, and write a pytest regression test with the existing TestClient and token fixtures that sends the request the traces show and fails for the same reason. Fix the cause, run the full backend test suite, and open a pull request that quotes the log lines and trace ids the fix addresses. If the cause is not in this code, change nothing and answer with what the logs show.\n\nPayload: <Datadog webhook body>",
"resources": [
{
"url": "https://github.com/your-org/your-app"
}
]
}- Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
- Send the request. POST the body in request.json to https://api.prix.dev/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
- Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.
What comes back
When the fault is in the service, the agent opens a pull request with the fix in the router, CRUD helper or model it traced, and a regression test in backend/tests/api/routes/. Its description names the monitor, quotes the Datadog log lines and trace ids it reproduced, shows the request the test sends, and lists the pytest and coverage commands with their result. If the fix needs a schema change, the pull request includes a new Alembic revision and says so at the top.
When the logs show a cause outside the code, such as Postgres refusing connections or a missing SECRET_KEY in one environment, the run makes no change and its answer explains what Datadog showed and where to look. That answer arrives on your webhook with the run, so the alert channel still learns what happened.
Before merging, a reviewer checks that the test fails on the current main for the alert's reason, that the fix does not widen what a normal user can read or change (the 403 checks in items.py are the line to watch), and that any migration runs forward cleanly on a copy of production data.
Run it on every Datadog error-rate alert
Create a webhook in Datadog's webhook integration pointing at your server and mention it in each monitor you want handled, such as an error-rate monitor on service:backend. When it moves to Alert, your server adds the payload to this prompt and sends the request with the repository in resources. Ignore Recovered and No Data transitions so only real failures start a run.
Rush POSTs run.completed or run.failed to the webhook URL set on your API key when the turn ends, with the answer and the session URL, which your server can forward to the incident or the on-call channel. Keep the alert id with the returned execution_id so a monitor that re-notifies during the same incident does not start a second run.
The use case this job follows, with its automation and trigger: Fix a bug from an alert.
Why run it on Rush?
Reproducing a production error locally means matching Python, uv, Postgres and migrations to what the service runs, and that is exactly the work nobody wants to do at night. The agent does it in its own sandbox: its own clone, its own database, its own Datadog session. Your laptop and your local database are not involved, and the run continues whether or not anyone is awake.
Every claim in the pull request can be checked. The trajectory keeps each Datadog query and its response, each migration and each pytest run with its output, and the regression test proves the fix against the request the traces captured. Several alerts in one incident each get their own run and their own sandbox, and a person still decides what merges.
Frequently asked questions
How does the agent read Datadog?
With Datadog's own MCP server, one of the connectors Rush lists. Connect it once through POST /me/connections with "type": "mcp" and OAuth, sign in, and add "connections": ["datadog"] to the request body. The listed server is the US1 site; another Datadog site is added as your own MCP server with its URL.
Will it touch my production database?
No. It reads logs and traces through Datadog, and runs the tests against a database it starts in its own sandbox. It never needs your database credentials for this job.
We use Sentry for errors. Does that matter?
The template initialises sentry_sdk when SENTRY_DSN is set, and that can run beside Datadog. This job reads Datadog; if your errors are only in Sentry, connect Sentry's MCP server instead and change the prompt to read issues from it.
What if the route is async and the error is a timeout?
It reads the trace's span durations to find which call ran long, such as a query or an outbound httpx request, and checks whether the code awaits it without a limit. A timeout caused by load or an upstream outage is reported in the answer rather than patched over with a longer timeout.