All jobs

Add Liveness and Readiness Probes to Kubernetes Deployments

Yes. An agent reads each container's health signals, adds liveness and readiness probes to your Deployments, verifies them on a kind cluster and opens a PR.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

The agent reads the manifests and the application code they run. In a repository like dockersamples/example-voting-app, k8s-specifications/ holds five Deployments and four Services with no probes and no resource requests: vote and result each expose containerPort: 80 and are published through NodePort Services on ports 8080 and 8081, worker has no port at all, db runs postgres:15-alpine on 5432 with an emptyDir volume, and redis runs redis:alpine on 6379. The repository's docker-compose.yml already encodes health for three of them: vote is checked with curl -f http://localhost at a 15 second interval, a 5 second timeout, 3 retries and a 10 second start period, redis with healthchecks/redis.sh, which expects PONG from redis-cli ping, and db with healthchecks/postgres.sh, which runs SELECT 1 through psql over TCP. The code confirms what each check means: vote/app.py is a Flask app whose single route / renders a template on GET and only touches Redis on POST, result/server.js is an Express app that serves / from views/ and retries its Postgres connection up to 1000 times, and worker/Program.cs loops on a Redis list and writes to Postgres with no HTTP listener.

From that the agent adds probes that match each container. vote and result get httpGet probes on / against the named port 80, with the compose values as the starting point: a readiness probe with a 10 second initial delay and a 5 second timeout, and a liveness probe with a higher failureThreshold so a slow page does not restart a healthy pod. Because vote's GET handler never calls Redis, its probe answers the question the kubelet asks, is this process serving, and does not turn a Redis outage into a restart loop; the pull request says this explicitly. db gets an exec readiness probe running pg_isready -U postgres -h 127.0.0.1, which the Postgres image ships, mirroring the TCP check the compose script makes, and redis gets exec probes running redis-cli ping. worker has neither a port nor a health endpoint, so the agent leaves it without probes and explains that a probe there would need code, a small HTTP listener or a heartbeat file, rather than inventing one.

Each probe uses the named containerPort rather than a number, so a port change in one place does not silently break the check, and periodSeconds, timeoutSeconds and failureThreshold are set on every probe rather than left to defaults, with the reasoning in a comment. The agent validates the files with kubectl apply --dry-run=client -f k8s-specifications/ and kubeconform before any cluster exists.

The sandbox has no Docker, so the agent installs it first (sudo apt-get install -y docker.io, then starts dockerd), creates a kind cluster, applies the directory and waits with kubectl rollout status on each Deployment. The images are public on Docker Hub, so kind pulls them. kubectl get pods showing every pod READY 1/1 and kubectl describe pod showing no probe failure events is the check; the agent also scales redis to zero and confirms vote stays Running and Ready, which demonstrates the probe does not depend on the backing service.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "claude",
  "input": "Add liveness and readiness probes to the Kubernetes Deployments in this repository. Read every file under k8s-specifications, docker-compose.yml, the healthchecks directory and the entry point of each service first, so each probe matches what the container actually serves. Use httpGet probes on the named port for the HTTP services, exec probes with the database and cache images' own client tools for the stores, and leave a container with no port and no health endpoint without probes, saying why. Set initialDelaySeconds, periodSeconds, timeoutSeconds and failureThreshold explicitly on every probe, starting from the compose healthcheck values where they exist. Validate with kubectl apply --dry-run=client and kubeconform, then install Docker in the sandbox, create a kind cluster, apply the manifests, wait for every rollout, show every pod Ready, and confirm the web pods stay Ready when the cache is scaled to zero. Open a pull request that lists each probe and the evidence for it.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

What comes back is a pull request on its own branch that edits vote-deployment.yaml, result-deployment.yaml, db-deployment.yaml and redis-deployment.yaml under k8s-specifications/, each gaining a readinessProbe and a livenessProbe with explicit timings, and leaves worker-deployment.yaml and the four Service files untouched. docker-compose.yml is not changed; it was the source for the timings, not the target.

The description is a table of each container, its probe type, path or command, timings and the file in the repository that justified the choice, followed by the kubectl get pods output from the kind cluster, the rollout status lines, and the result of the Redis scale-down check. The commands are in the transcript at GET /o/{org}/sessions/{id}/messages.

Before merging, a reviewer compares the timings with what the real cluster's node resources allow, decides whether the worker should gain a health endpoint in a follow-up, and checks whether the same probes belong in the Compose file's Kubernetes stack equivalent, docker-stack.yml, which the pull request leaves alone.

Run it on every Linear issue labelled reliability

A platform team that tracks hardening work in Linear starts one run per issue from an automation with a linear trigger on a reliability label. The automation adds the issue after this prompt, so a Deployment the issue names, or a restart loop it describes, is what the agent reads first, and the run starts in the repository the automation names.

Each run opens its own pull request from its own sandbox with its own kind cluster, and an automation run has its own max_runs_per_hour cap. The pull request links the issue and carries the pod status output, so review is a read of the probe table against the cluster output.

The use case this job follows, with its automation and trigger: Turn a Linear label into a PR.

Why run it on Rush?

Probes written from a template are the common cause of restart loops: a liveness check on an endpoint that calls the database, an initial delay shorter than the app's startup, a timeout the default leaves at one second. The agent reads the handler behind each path before choosing it, takes timings from the health checks the repository already runs in Compose, and then watches real pods come up on a kind cluster in its own sandbox, so the version you review has been scheduled and passed its own probes.

The result says what it did not do as plainly as what it did: the worker gets no probe and the reason is written down. Several repositories can be handled at once, each in its own sandbox and cluster, with one pull request each.

Frequently asked questions

Why does the worker get no probes?

It has no port and no health endpoint; it is a loop that pops a Redis list and writes to Postgres. A meaningful probe would need application code, such as a heartbeat file the loop touches and an exec probe that checks its age. The pull request describes that option rather than adding a probe that checks nothing.

Will a Redis outage restart the vote pods?

No. The liveness probe hits / with GET, and the Flask handler only connects to Redis on POST, so the probe measures whether the process serves requests, not whether its dependency is up. The agent demonstrates this on the kind cluster by scaling redis to zero and showing vote stays Ready.

Where do the timings come from?

From docker-compose.yml where a healthcheck exists: the vote service's interval, timeout, retries and start period map onto periodSeconds, timeoutSeconds, failureThreshold and initialDelaySeconds. Where Compose has no check, the agent sets values explicitly and explains them in a comment.

Does this need a real cluster?

No. The agent validates the YAML with a client-side dry run and kubeconform, then runs the manifests on a kind cluster inside its sandbox after installing Docker. The public images pull from Docker Hub, so the whole check needs no credentials and no access to your cluster.

More Kubernetes jobs

The same job on other stacks

More jobs like this: Turn a Linear label into a PR

Keep reading