What the agent does
The agent reads the alert before it reads the code. Your server passes Datadog's webhook payload into the prompt, and the agent uses the Datadog MCP server to pull the monitor definition, the error logs for the service in the 30 minutes around the alert, and the traces of the failing requests. Phoenix logs crashes with the exception, the controller and action, and an Elixir stacktrace; with JSON logging those land in Datadog as attributes, so the agent can group a burst of errors by exception and the first frame in your own modules, and keep one sample of each with its request id.
Then it maps the errors to controllers. In hexpm/hexpm, the Phoenix app behind hex.pm, API controllers live in lib/hexpm_web/controllers/api/, and package publishing goes through HexpmWeb.API.ReleaseController.publish/2. That action reads the tarball's metadata with release_metadata(body_path), logs it with log_tarball using the x-request-id header, and calls Releases.publish with replace: Map.get(params, "replace", true) and the workload identity context. Before it runs, plug :authorize checks the api:write or package domain, package_owner and organization_billing_active. A failing publish can break at any of those steps, so the agent follows the stacktrace into lib/hexpm/repository/releases.ex and the policy modules instead of stopping at the controller.
It writes the regression test first, in the style the suite already uses. test/hexpm_web/controllers/api/release_controller_test.exs is use HexpmWeb.ConnCase, async: true, builds its world with ExMachina factories (insert(:user), insert(:package, package_owners: [...]), insert(:release, ...)), and groups cases under describe "POST /api/packages/:name/releases". The repository also ships test/support/log_lines.ex, a :logger handler that sends every line a test process logs back to it as {Hexpm.LogLines, level, fields}, so the new test can assert that the log line from the alert is no longer written, not only that the response code changed.
To run it, the agent starts Postgres (docker-compose.yaml uses postgres:14), fetches dependencies for the Elixir version mix.exs requires (~> 1.20), and runs mix test, whose alias creates, loads and migrates the test database before the suite. It runs the new test alone first to see it fail, then the controller's test file and the full suite after the fix.
Start it with one API call
POST https://api.prix.dev/o/{org}/sessions with this body:
{
"agent": "claude",
"input": "A Datadog monitor on this Phoenix service fired; its webhook payload is at the end of this prompt. Use the Datadog connection to read the monitor, then the error logs and traces for its service and environment from 15 minutes before to 15 minutes after it fired. Group the errors by exception and the first frame in this repository, map them to the controller actions under lib/hexpm_web/controllers, and follow the stacktrace into the context modules to find the cause. Write an ExUnit regression test in the matching controller test file, using the existing ConnCase and factories, that sends the request the logs show and fails for the same reason; assert on the log line too where the test helpers allow it. Fix the cause, run mix test, and open a pull request that quotes the log lines and trace ids the fix addresses. If the cause is outside this code, change nothing and answer with what the logs show.\n\nPayload: <Datadog webhook body>",
"resources": [
{
"url": "https://github.com/your-org/your-app"
}
]
}- Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
- Send the request. POST the body in request.json to https://api.prix.dev/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
- Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.
What comes back
A pull request with the fix in the context or controller module the stacktrace led to, and a new test in the matching file under test/hexpm_web/controllers/api/, such as a case in the POST /api/packages/:name/releases block. The description names the monitor, quotes the Datadog log lines with their request ids and trace ids, shows the request the test reproduces, and lists the mix test commands with their result. The session's trajectory keeps each Datadog query and every test run with its full output.
If the logs point outside the app, such as S3 errors from ex_aws_s3 during a tarball upload or a queue backing up behind broadway_sqs, the agent leaves the code alone and its answer explains what it found and which dependency or service to check. That answer comes back on your webhook with the run.
Before merging, a reviewer checks that the test fails on the old code for the alert's reason, that the change does not loosen the :authorize plug or the owner and billing checks on publish, and that a fix to Releases.publish keeps the replace behavior existing clients rely on.
Run it on every Datadog monitor alert
Set up a Datadog webhook that posts to your server and mention it in the monitors that cover the app, such as an error-count monitor on the publish endpoint. When one moves to Alert, your server appends the payload to this prompt and sends the request with the repository in resources; skip Recovered notifications and alerts tagged for infrastructure, which a code change will not fix.
Store the alert id beside the execution_id Rush returns, so a monitor that re-notifies in the same incident is matched to the run already working on it. When the turn finishes, Rush POSTs run.completed or run.failed to the webhook URL on your API key with the answer and the session URL, ready to forward to the incident.
The use case this job follows, with its automation and trigger: Fix a bug from an alert.
Why run it on Rush?
Getting an Elixir app with Postgres, S3 and SQS dependencies to the point where a failing request reproduces takes setup most people only have on one machine. The agent does that setup in its own sandbox with its own clone, its own database and its own Datadog session, and none of it touches your laptop, your local _build or your shell history. It starts when the alert fires, not when someone reads it.
The pull request is evidence a reviewer can check: the quoted log lines and trace ids, a test that fails without the fix, and a trajectory with every query and mix test run and its output. When an incident trips several monitors, each one gets its own run in parallel, and nothing merges until a person approves it.
Frequently asked questions
How does the agent connect to Datadog?
Through Datadog's MCP server, listed among Rush's connectors. Connect it once with POST /me/connections, "type": "mcp" and OAuth, sign in, then add "connections": ["datadog"] to the request so the run can query it. The listed server is the US1 site; add another site's MCP URL as your own server.
Our logs are JSON for another platform. Will Datadog's attributes be there?
The agent reads whatever attributes the log lines carry. hexpm configures logger_json with its Google Cloud formatter in config/prod.exs; the same library has a Datadog formatter, which puts the request id and trace ids where Datadog links logs to traces. Without that link, the agent matches logs and traces by request path and time.
Can it break publishing for everyone?
Only if a reviewer merges a change that does. The fix arrives as a pull request with a test, and the prompt tells it not to change code when the cause is outside the app. Review changes to the authorization plug with extra care.
Does it run the whole ExUnit suite?
Yes, after the targeted test file. The suite runs async: true where the tests allow it, and the pull request lists each mix test command it ran so you can see what was covered.