All jobs

Add Health Checks to a chi API

Yes. An agent splits a chi API's health route into a liveness probe and a readiness probe that pings Postgres, tests both and opens a pull request.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

The agent starts from what the service already answers. In a chi API like dhax/go-base, api/api.go registers r.Get("/healthz", ...) with a handler that writes ok and nothing else, so the route says the process is up even when the database is gone. It is registered on the main router after logging.NewStructuredLogger, which means every probe writes a request started and a request complete line through logrus, and the debug level that cmd/serve.go sets by default lets both through. The database handle comes from database.DBConn() in database/postgres.go, which runs an unexported checkConn (SELECT 1 through bun) once at startup and never again. api/server.go wraps an http.Server and its Start() waits on os.Interrupt before calling Shutdown.

It then separates the two questions a probe asks. Liveness moves to the front of the r.Use list, so it answers before the request logger, the fifteen-second middleware.Timeout and the JSON content-type middleware. chi ships middleware.Heartbeat for exactly this, but it answers with a single ., and this API answers ok; the agent writes a middleware of the same shape that keeps the ok body, so anything already polling the route sees no change. Readiness is a new /readyz handler in a small api/health.go. It takes a one-method interface with PingContext, which the *bun.DB that New already holds satisfies, pings with a short context deadline that the pull request states, and answers 200 with {"status":"ok"} or 503 with the name of the failed check and no connection string. It is registered outside the group that puts the JWT verifier in front of /admin and /api, because a probe has no token.

The container files get the same treatment. docker-compose.yml lists postgres under depends_on for server with no condition, so the API can start before Postgres accepts connections and DBConn then fails; the agent adds a pg_isready healthcheck to the postgres:17-alpine service, changes the dependency to condition: service_healthy, and gives server a healthcheck that calls /readyz with the BusyBox wget already present in the alpine stage of the Dockerfile, since that stage installs only ca-certificates and has no curl.

Then it runs both paths. A table test in the style of auth/pwdless/api_test.go drives the readiness handler through an httptest server with a pinger that succeeds, one that returns an error and one that blocks past the deadline, and asserts the status and body of each. For the real thing the agent installs Postgres in the sandbox, sets DB_DSN and AUTH_JWT_SECRET, runs go run main.go migrate and go run main.go serve, reads /healthz and /readyz with curl, stops Postgres, reads them again and starts it back. It finishes with go vet ./... and go test -v ./..., the command in .github/workflows/test.yml, and adds a short health section to the README next to the route tables.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "claude",
  "input": "Add proper health checks to this chi API. Read api/api.go, api/server.go, database/postgres.go, logging/logger.go, cmd/serve.go, Dockerfile, docker-compose.yml and auth/pwdless/api_test.go first. Keep GET /healthz as a liveness probe with its current plain-text ok body, but serve it from a small middleware placed first in the middleware list, shaped like chi's middleware.Heartbeat (which would change the body to a dot), so probes skip the request logger and the timeout. Add GET /readyz outside the JWT group: it pings the database through a small interface with PingContext and a short deadline, answers 200 with a JSON status or 503 naming the failed check, and never includes the DSN. Add a pg_isready healthcheck to the postgres service in docker-compose.yml, make the server wait for it with condition: service_healthy, and give the server a healthcheck that calls /readyz with the wget in the alpine image. Write table tests for the handler with a passing, a failing and a slow pinger. Then install Postgres, run the migrate and serve commands, curl both routes, stop Postgres, curl them again, and run go vet and go test -v ./.... Document the two routes in the README and open a pull request.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

A pull request with api/health.go and its test, the liveness middleware and the /readyz route in api/api.go, the two healthchecks and the dependency condition in docker-compose.yml, and the README section. Its description shows four responses: /healthz and /readyz with Postgres up, and the same two with Postgres stopped, where liveness still answers 200 and readiness answers 503. It also quotes the go test output and says which deadline the ping uses.

A reviewer checks that readiness tests only what the API cannot serve without. Here that is the database; the SMTP host is left out on purpose, because email.NewMailer() already stops the process at start when a configured host cannot be reached, and a mail outage should not pull every instance out of rotation. They also check the probe paths against whatever load balancer or orchestrator is in front. The pull request notes one thing it does not change: Start() listens for os.Interrupt only, so readiness does not yet drop on SIGTERM.

Run it on every terminal hand-off

An engineer who has just watched a deploy route traffic to an instance with a dead database connection sends this body from the terminal with the repository in resources, and goes back to the incident notes. The run reads the real router and the real compose file, so the probes it writes fit this service's middleware order and its image, which has wget and no curl.

Send a request_id, a UUID, with the body so a retried start returns the run it already began. When the turn ends, GET /o/{org}/sessions/{id} names the pull request. The sandbox parks with its disk and memory, so a follow-up message to the same session can ask for a Redis check on a service that has one, or for Kubernetes probe YAML that points at the two paths, without setting Postgres up again.

The use case this job follows, with its automation and trigger: Hand off a long task.

Why run it on Rush?

A readiness probe is only worth merging once someone has seen it fail. Doing that by hand means stopping a database on a laptop that other work depends on. The agent has its own sandbox, installs Postgres there, stops it on purpose and records what both routes answer before and after, and every one of those commands and responses stays in the run's record for the reviewer.

Handing it off also keeps the change small and specific. The run reads this router's r.Use order, sees that the request logger would log every probe, and puts the liveness answer in front of it; on another chi service with middleware.RealIP enabled or a second datastore the same prompt produces a different, equally fitted diff.

Frequently asked questions

Why two routes when /healthz already exists?

They answer different questions. Liveness says the process can answer at all, and a failure means restart it. Readiness says it can do useful work right now, and a failure means stop sending it traffic. One route that pings the database would restart healthy processes during a Postgres failover.

Will existing monitors of /healthz break?

No. The path, the 200 and the ok body are the same; only where the answer comes from moves, to a middleware in front of the stack. What changes is that those requests no longer pass through the logrus request logger, so probe lines disappear from the logs.

Does /readyz leak anything about the database?

It names the check that failed and nothing more. The handler does not echo the error from the driver, which can contain the host and user from DB_DSN; that detail goes to the log entry instead.

Can the agent test the compose healthchecks?

The sandbox has no Docker, so the agent installs it first with sudo apt-get install -y docker.io and starts dockerd. That package carries no docker compose v2 plugin, so it uses docker-compose v1 for the file, and the pull request says which it ran.

More chi jobs

The same job on other stacks

Keep reading