What the agent does
The agent reads the router before it measures anything. In a chi service like dhax/go-base, go.mod declares go 1.25.0 with github.com/go-chi/chi/v5 v5.4.0, go-chi/render, logrus, viper, cobra and uptrace/bun on Postgres. api/api.go builds one chi.NewRouter() in New(enableCORS bool) and stacks middleware.Recoverer, middleware.RequestID, middleware.Timeout(15 * time.Second), the logrus request logger from logging/logger.go and render.SetContentType, then mounts /auth, puts /admin and /api in a group behind the JWT verifier, answers /healthz with ok and hands everything else to SPAHandler("public") on /*. The admin router nests a parameter route, /accounts/{accountID}, which is the case that decides how the labels have to be built. Configuration comes from viper with AutomaticEnv(), so ENABLE_CORS in .env.example is read as enable_cors in api/server.go.
It adds github.com/prometheus/client_golang and a small metrics package with one middleware. The middleware wraps the writer with chi's own middleware.NewWrapResponseWriter, calls the next handler, and only then reads chi.RouteContext(r.Context()).RoutePattern(). Read before the handler runs, the pattern is still empty or partial, because chi fills it in as each mounted router matches; read afterwards it is the full template, /admin/accounts/{accountID}, so a thousand account ids make one series rather than a thousand. It records a request counter labelled with method, route pattern and status code, and a duration histogram labelled with method and route pattern. Requests that fall through to the single-page app are recorded under the /* pattern, and the agent leaves the raw path out of every label.
The endpoint is promhttp.Handler() on /metrics, registered beside /healthz and outside the JWT group, behind a new ENABLE_METRICS setting that defaults to off in cmd/serve.go the way the other defaults are set there. The handler exposes the Go runtime and process collectors that come with the default registry, and the agent registers collectors.NewDBStatsCollector on the *sql.DB inside the bun handle that database.DBConn() returns, so pool saturation shows up next to request latency. .env.example, the docker-compose.yml environment block and the README feature list each get the new setting.
Then it proves the labels. The existing tests in auth/pwdless/api_test.go build a chi router on an httptest server with a mocked store, and the new test follows that shape: a router with the middleware and a /items/{id} route, three requests with different ids, and an assertion on the scraped text that one series carries route="/items/{id}" with a count of three. The agent runs go vet ./... and go test -v ./..., the command .github/workflows/test.yml uses. Since api.New refuses to start without a database, it also installs Postgres in the sandbox, sets DB_DSN and AUTH_JWT_SECRET, runs go run main.go migrate and go run main.go serve, calls a few routes with curl and reads /metrics back.
Start it with one API call
POST https://api.phoenix.so/o/{org}/sessions with this body:
{
"agent": "claude",
"input": "Add Prometheus metrics to this chi API. Read go.mod, api/api.go, api/server.go, api/admin/accounts.go, logging/logger.go, cmd/serve.go, .env.example and auth/pwdless/api_test.go first. Add github.com/prometheus/client_golang and a metrics package with one chi middleware: a request counter labelled by method, route pattern and status, and a duration histogram labelled by method and route pattern. Take the route label from chi.RouteContext(r.Context()).RoutePattern() after the next handler has run, never from the URL path, and take the status from chi's WrapResponseWriter. Serve promhttp.Handler() on /metrics next to /healthz, outside the JWT group, behind an ENABLE_METRICS setting that defaults to off, and register a DB stats collector for the bun connection. Write a test that sends several ids to a parameter route and asserts they land in one series. Run go vet and go test -v ./..., then start Postgres, run the migrate and serve commands and read /metrics with curl. Document the setting in .env.example, docker-compose.yml and the README, and open a pull request.",
"resources": [
{
"url": "https://github.com/your-org/your-app"
}
]
}- Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
- Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
- Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.
What comes back
A pull request with the new metrics package and its test, the middleware and the /metrics route in api/api.go, the default in cmd/serve.go, the client_golang lines in go.mod and go.sum, and the setting in .env.example, docker-compose.yml and the README. Its description lists every metric name with its labels, quotes the go test output, and pastes the part of a real scrape that shows the account route as one templated series. The existing workflow in .github/workflows/main.yml runs lint, test and build on the pull request, so those checks are on it before anyone opens the diff.
A reviewer checks three things. First, that no label can take an unbounded value: method, pattern and status only. Second, where /metrics is reachable, since it sits on the same port as the API and the pull request says whether the deployment should restrict it at the proxy or move it to a second listener. Third, that the histogram buckets suit the service, because the client's default buckets top out below the fifteen-second timeout the router already enforces.
Run it on every Linear issue labelled observability
In the console, add an automation on the Project that holds the service, with a linear trigger on linear.issue.labeled, the label observability and the prompt above. When someone on the team labels an issue that names a service or a signal they are missing, the run starts with the issue text after the prompt, works in the Project's repository and opens a pull request on its own branch. The automation's runner needs a Linear connection, made in the console under Settings → Connections, and a label applied by an app integration does not fire the trigger.
A platform team with many chi services can start the same work from its own tooling instead: one POST /o/{org}/sessions per repository with this body, each with a request_id that is a UUID v5 of the repository name and the issue id, so a retried script returns the runs it already started and adds none.
The use case this job follows, with its automation and trigger: Turn a Linear label into a PR.
Why run it on Rush?
The part of this job that goes wrong is cardinality, and it only shows when real requests hit a parameter route. The agent has its own sandbox with Go, a Postgres it installed and the service running in it, so it can send requests with different ids and read the scrape, and the run keeps each command with its output. The pull request therefore shows the series the code actually produces, where a change written without running the service would show only the code.
Each service gets a run of its own, so the middleware is fitted to that router: one that mounts sub-routers, one that already has middleware.RealIP switched on, one that serves a single-page app from the same port like this one. Starting ten of them from the API does not need ten checkouts on someone's laptop; runs past the key's concurrency limit are created as queued and start as slots free.
Frequently asked questions
Why read the route pattern after the handler runs?
chi builds the pattern as routing descends through mounted routers, so before the handler it is incomplete. After next.ServeHTTP returns, RoutePattern() is the full template such as /admin/accounts/{accountID}. Using the URL path instead would create one series per account id.
Does it replace the logrus request logger?
No. logging/logger.go keeps writing its request started and request complete lines with resp_status and resp_elapsed_ms. The metrics middleware sits beside it in the r.Use list and shares nothing with it, so log fields and metric labels can change separately.
Is /metrics behind the JWT check?
No, because a Prometheus scraper does not carry a user token. It is registered outside the group that mounts /admin and /api, and it only exists when ENABLE_METRICS is on. The pull request names the options for limiting who can reach it.
What does the test cover without a database?
The middleware test builds its own small router, as the existing auth tests do, so it needs no Postgres. The end-to-end read of /metrics against the real api.New router is done in the sandbox with a database and kept in the run's record.
More chi jobs
The same job on other stacks
- Add Prometheus Metrics to a Django REST Framework API (Django REST Framework)
- Add Prometheus Metrics to an Echo API (Echo)
- Add Prometheus Metrics to a Fastify App (Fastify)