What the agent does
The agent opens Sentry through the connection the request names and starts from the release, because in a Phoenix app like hexpm/hexpm the release is a commit. mix.exs pulls {:sentry, "~> 13.0"}, config/runtime.exs reads the DSN from HEXPM_SENTRY_DSN, the environment from HEXPM_ENV and the release from GIT_SHA, which the Dockerfile bakes in as a build argument, and the docker job in .github/workflows/main.yml creates the matching Sentry release with a curl to the releases API, with a refs entry linking the repository and commit so issues resolved by a commit close when the deploy that carries it registers. So the latest release in Sentry names a short SHA, the previous one names another, and the diff the agent reads is git log between them in its clone of the repository.
Then it reads what the app filters before it trusts the Sentry graph. Hexpm.Application.sentry_before_send, set as before_send in config/prod.exs, drops websocket protocol crashes, every exception whose Plug.Exception.status is below 500, and a set of metric-only classes: DBConnection.ConnectionError as pool_timeout, db_connect or db_disconnect, Postgrex.Error with too_many_connections or query_canceled, Bandit.TransportError, Plug.Conn.NotSentError and a DBConnection exit. Those are counted instead in the hexpm.sentry.filtered.total PromEx metric by class. Oban job failures reach Sentry only on the attempt that exhausts max_attempts, through report_oban_error?, and drop_frame_vars empties every frame's vars before sending. HexpmWeb.SentryScrubber drops all parameters on the sso, scim, oauth, invites, password and email paths and masks keys such as token, key, code, state and otp. A spike in Sentry is therefore a 5xx or a crash outside those classes; a database saturation shows as a flat Sentry graph and a rising filtered metric, and the report says which of the two it is looking at.
The code side is a route and a module. HexpmWeb.Endpoint uses Sentry.PlugCapture and Sentry.PlugContext, adds Plug.RequestId, and runs HexpmWeb.Plugs.RequestLog, which writes one http.request JSON line per request with method, path, status, duration_us, the controller, action and format once routed, and a credential of key:<id>, token:<jti> or session:<id>, so a Sentry event's request_id pairs with a log line. HexpmWeb.Router has a :browser pipeline with the content security policy, HexpmWeb.Plugs.Attack rate limits and HexpmWeb.Plugs.ReadOnly, an :api pipeline with :authenticate, and an :upload pipeline; controllers live under lib/hexpm_web/controllers and controllers/api. For each new or grown issue the agent keeps the module, function and arity of the top frames, the route and the first-seen time, finds the commits in the release range that touched that module, and checks whether the issue's first event follows the deploy or precedes it, which separates a change-caused spike from a traffic-caused one such as a client flooding a rate-limited route.
It confirms what it can without changing code. mix deps.get and mix compile --warnings-as-errors, as the format CI job does, prove the clone builds; when the suspect module has a test file it runs that one with mix test path, which through the test alias runs ecto.create, ecto.load --skip-if-loaded and ecto.migrate against the Postgres the sandbox starts, as CI does with the postgres:18 service. The report names the most likely commit with the evidence for it, the alternative it ruled out and why, and the fix it would make, and the run ends with the repository untouched.
Start it with one API call
POST https://api.phoenix.so/o/{org}/sessions with this body:
{
"agent": "claude",
"input": "Errors rose after the latest release of this Phoenix app. Through the Sentry connection, find the latest release and the one before it, list the issues that are new in the latest release or whose event count rose after it, and for each keep the exception, the top frames with module, function and arity, the route, the first-seen time and the request id. Read config/runtime.exs, config/prod.exs and lib/hexpm/application.ex to learn what sentry_before_send and the scrubber drop, and say in the report whether the symptom could be a filtered class such as a pool timeout that Sentry never shows. In the repository, diff the two release commits, find the commits that touched the modules the frames name, and check whether each issue's first event follows the deploy. Run mix deps.get and mix compile --warnings-as-errors, and run the test file of the suspect module if one exists. Write sentry-spike.md with: the releases compared, each issue with its Sentry link, source and counts, the most likely cause with its commit and evidence, the alternatives ruled out, and the fix you would make. Change no code and resolve nothing in Sentry.",
"connections": [
"sentry"
],
"resources": [
{
"url": "https://github.com/your-org/your-app"
}
]
}- Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
- Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
- Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.
What comes back
What comes back is sentry-spike.md as an artifact on the session, with a download link good for an hour that reading the session again renews. It opens with the two releases compared as short SHAs and the deploy time, then one section per issue: the Sentry link, the exception and top frames, the controller action or job class, event counts before and after, first seen, and a matching http.request log line when one was available. The finding names a commit in the range, quotes the lines in it that the frames point at, and gives the fix; a section after it lists what was ruled out, including whether the filtered metric rather than Sentry holds the real signal.
The transcript at GET /o/{org}/sessions/{id}/messages keeps every Sentry query the agent made, the git log and git diff it read, the compile and any test run, so each sentence in the report can be traced to a tool call. The session's result is the one-paragraph summary.
Before acting on it, an on-call engineer checks the commit against the frames themselves, confirms the first-seen time against the deploy, and decides whether to open the fix as a pull request, which a follow-up on the same session can do with POST /o/{org}/sessions/{id}/messages.
Run it on every error spike after a deploy
Sentry signs its own alert webhooks and cannot add an Authorization header, so a Rush webhook trigger is not called by Sentry directly. A handler on your server receives Sentry's alert for new issues in the latest release or for an error-rate jump and forwards the payload to the trigger URL, with an Idempotency-Key header from Sentry's event id and Authorization: Bearer <its secret>, so a retried delivery starts no second run; the payload reaches the agent after the prompt as text. The automation's max_runs_per_hour, between 1 and 30, keeps a burst of alerts from starting a run each.
The deploy workflow can start the run instead: after the step that creates the Sentry release, a job waits a few minutes and sends POST /o/{org}/sessions with this body and a request_id that is a UUID v5 of the commit SHA, so the investigation runs once per release and is only read when Sentry shows new issues. Give that key a webhook_url and Rush POSTs run.completed with data.session_url to read for the report.
The use case this job follows, with its automation and trigger: Investigate an incident.
Why run it on Rush?
A Sentry spike in an app that filters this carefully is a question about two things at once: what Sentry shows and what the app chose not to send it. Answering it means reading sentry_before_send and the scrubber, pairing events with request logs by request_id, and diffing two release SHAs, while the pager is going. The agent does that reading in its own sandbox with the repository cloned and Sentry open, and comes back with the commit rather than a hunch.
The report cites its sources: every Sentry query and every git command are in the transcript, the compile proves the clone matches the release, and nothing in Sentry or the repository is changed. Several incidents, or the same spike from two hypotheses, run at once in separate sandboxes, and the sandbox parks when the turn ends, so turning the finding into a pull request is a follow-up rather than a new clone.
Frequently asked questions
Why can the Sentry graph be flat while the site is failing?
Because sentry_before_send drops pool timeouts, connection failures, too_many_connections, cancelled queries, transport errors and unsent responses, and counts them in the hexpm.sentry.filtered.total metric instead. The report checks for that shape first and says when the evidence points at saturation that Sentry was never meant to show.
How does it connect to Sentry?
Through Sentry's MCP server, connected once at /me/connections with a sign-in and named sentry in the request's connections. The run can use only the connections it names, and the connection's token never comes back from the API.
Does it cover Oban job errors?
Yes, with the caveat the code sets: the Oban integration in config/prod.exs reports a job only on the attempt that reaches max_attempts, so a job that fails and retries successfully never appears. The report names the worker module from the frames and reads its commits the same way as a controller's.
Will it resolve the issues or change code?
No. The prompt forbids both. It reads Sentry, reads and compiles the repository, and writes the report. A follow-up on the session can ask for the fix as a pull request once a person agrees with the finding.
What does it need from the repository?
The resources entry clones it through your GitHub App installation, which gives the agent git log between the two release SHAs, the modules the frames name, and the configuration that decides what Sentry receives. Compiling needs Elixir and Erlang in the sandbox at the versions CI uses, and running a test file needs the Postgres the sandbox starts.