What the agent does
The agent clones the repository into its own cloud sandbox and starts from the alert you paste into the prompt: which Deployment restarts, how often, the exit code and when it began. Exit code 137 means the container was sent SIGKILL, which in Kubernetes is usually the kernel's OOM killer at the memory limit or the kubelet killing a container that failed its liveness probe. The agent keeps both explanations open and looks for evidence of each, rather than assuming the first.
In a repository like GoogleCloudPlatform/microservices-demo it reads kubernetes-manifests/cartservice.yaml first. The server container there requests memory: 64Mi and is limited to memory: 128Mi, with cpu: 300m, while helm-chart/values.yaml gives cartService a 256Mi limit. So the same service gets half the memory depending on which deployment path was used, and the agent notes which one the rollout applied. It then reads src/cartservice/src/Dockerfile: a .NET 10 self-contained, trimmed single-file publish running on a chiseled runtime-deps image, with no DOTNET_GCHeapHardLimit or other GC setting in the manifest's env, which only sets REDIS_ADDR to redis-cart:6379.
For the probe explanation, it reads the livenessProbe: gRPC on port 7070, initialDelaySeconds: 15, periodSeconds: 10 and the default failure threshold. It follows the probe into services/HealthCheckService.cs, where Check returns Serving when _cartStore.Ping() is true, and then into cartstore/RedisCartStore.cs, where Ping() returns true without touching Redis. That clears a Redis outage as a probe-failure cause, and the agent says so in the report instead of leaving it as a suspect. It also reads Startup.cs, which picks Redis, Spanner, AlloyDB or an in-memory cache from environment variables, because an overlay from kustomize/components/ (such as memorystore or spanner) that changes REDIS_ADDR changes which store runs.
Last, it reads the 30 most recent commits with git log -p on the manifests, the Helm chart, the Kustomize components and src/cartservice, and lines each change up against the alert's start time: a lowered limit, a base image bump, a package version in cartservice.csproj (Grpc.AspNetCore, StackExchangeRedis, Npgsql), or a new probe setting. Each suspect goes into incident.md with its file and line, why it fits exit code 137 and a 90-second restart cycle, and the one check that confirms or clears it. It changes no code and opens no pull request.
Start it with one API call
POST https://api.prix.dev/o/{org}/sessions with this body:
{
"agent": "claude",
"input": "Incident: Grafana alerts that the cartservice pods restart every 90 seconds in CrashLoopBackOff with exit code 137 since this morning's rollout. Read the code, the configuration and the last 30 commits, find the most likely causes, and write incident.md with a timeline, each suspect with file and line evidence, and the check that would confirm or clear it. Change no code.",
"resources": [
{
"url": "https://github.com/your-org/your-app"
}
]
}- Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
- Send the request. POST the body in request.json to https://api.prix.dev/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
- Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.
What comes back
What comes back is incident.md, attached to the session as an artifact with a download link that works for an hour. It opens with a timeline built from the commit history: each change to the cartservice manifests, chart values, overlays and source in the window, with its commit and time, set against the alert you described. Then come the suspects, ranked. Each one names the file and line (the limits.memory line in cartservice.yaml, the cartService.resources block in values.yaml, a probe setting, a dependency bump), explains how it produces a SIGKILL every cycle, and gives the check that settles it, such as kubectl describe pod showing Last State: Terminated, Reason: OOMKilled, kubectl get events showing Liveness probe failed, or container_memory_working_set_bytes for the pod climbing to its limit before each restart.
The report also lists what the agent ruled out and why, such as the Redis-backed health check that cannot fail because Ping() always returns true. That keeps an on-call engineer from spending the first twenty minutes on a dead end. The session's transcript keeps every command the agent ran, git log, grep and each file it read, so any line of the report can be traced back to its source.
Before acting on it, the engineer runs the confirming checks against the live cluster, which the agent did not have, and decides on the fix: raise the limit, set a GC heap limit, roll back one commit or change a probe. A follow-up message to the same session can ask the agent to prepare that change as a pull request once the cause is confirmed.
Run it on every CrashLoopBackOff alert
Point a Grafana alert's webhook contact point at a small handler on your server that turns the alert into this request: the Deployment name, the restart interval and the exit code go into the prompt, and the repository that holds the manifests goes in resources. Each alert then starts its own run in its own sandbox, and the first pass is under way before anyone is paged out of bed.
Set a webhook URL on the API key, and Rush POSTs run.completed with the session_url when the turn finishes. Your handler reads the session, takes the incident.md link from its artifacts and posts it in the incident channel or on the alert itself.
The use case this job follows, with its automation and trigger: Investigate an incident.
Why run it on Rush?
A CrashLoopBackOff investigation usually starts with someone cloning three repositories, grepping manifests and reading Helm values while also watching the pods. In a Rush sandbox the agent does the reading side in parallel with the people on call: it has the whole repository, the full commit history and a shell, and it cannot touch the cluster or change the deployment while it works.
The report is only useful if each claim can be checked, so the agent's work is kept: every file it opened and every command it ran with its output is in the session's transcript. When several services crash at once, each alert can start its own run, up to the key's concurrent run limit, and each comes back with a report of its own.
Frequently asked questions
Does the agent connect to my cluster or run kubectl?
Not in this request. It reads the repository and the commit history, and the report names the kubectl and metrics checks a person runs against the cluster. If your Grafana Cloud stack is connected, the agent can query Prometheus and Loki through that connection as well.
We deploy with Helm and Kustomize overlays. Does it understand which values apply?
It reads the base manifests, the chart's values.yaml and the overlays in the repository, and reports where they disagree, such as a memory limit that differs between the raw manifest and the chart. It cannot see which one your pipeline applied unless that is in the repository or the prompt, so say so in the prompt when you know.
Will it change my manifests or open a pull request?
No. The prompt says to change no code, and the result is a Markdown report attached to the session. If you want the fix as a pull request, send a follow-up to the same session once you have confirmed the cause.
How does it tell an OOM kill from a liveness probe kill?
Both end in exit code 137, so the agent looks for evidence of each in the code: memory limits and runtime settings for one, probe timing and what the health check actually tests for the other. The report gives the check that separates them, the Reason in the pod's last state or the probe failures in its events.