What the agent does
The agent reads the module and the service before it adds a dependency. In a service laid out like cookbook/crud in labstack/echox, go.mod declares go 1.26.0 with toolchain go1.27.1 and takes github.com/labstack/echo/v5 v5.4.4 and github.com/labstack/echo-contrib/v5 v5.0.1, whose only use in the tree is echoprometheus in cookbook/prometheus/server.go. cookbook/crud/server.go keeps users in a map under a sync.Mutex, builds the app in newServer() with middleware.RequestLogger() and middleware.Recover(), registers GET and POST /users and GET, PUT and DELETE /users/:id, and starts with echo.StartConfig{Address: ":1323"}. cookbook/crud/server_test.go already has newTestServer, request and expectStatus helpers and drives every route through e.ServeHTTP with httptest, including a -race test with twenty goroutines. The Makefile runs go test -race ./..., and there is no workflow directory, so that command is the whole check.
Setup goes in one new file, cookbook/crud/telemetry.go. initTracer(ctx) builds a resource with service.name from OTEL_SERVICE_NAME, defaulting to crud, an otlptracehttp exporter that reads OTEL_EXPORTER_OTLP_ENDPOINT and the other standard variables itself, a sdktrace.NewTracerProvider with a batch span processor, and sets the W3C TraceContext and Baggage propagators as the global propagator. It returns a shutdown function that main() calls with a five-second timeout after sc.Start returns, so the last batch is flushed when the process exits. When the endpoint variable is unset the exporter still constructs with its default of localhost:4318, so a developer with no collector sees nothing break; the agent notes that in the file rather than adding a branch that silently disables tracing.
The middleware is written against Echo's own *echo.Context, because the hand-written form works on the v5 API regardless of which contrib packages have caught up with it; the agent checks the current otelecho release first and uses it instead when it builds against v5. Tracing(next) extracts the parent from the request headers with the global propagator, starts a server span named after the matched route template, so GET /users/:id rather than GET /users/42, puts the span's context back on the request with c.SetRequest, calls next, and then records http.request.method, http.route, url.path, http.response.status_code and, for a returned error, the status code the error carries and span.RecordError. A 5xx sets the span status to error; a 404 for a missing user does not, which matches the semantic conventions. The agent registers it in newServer() before RequestLogger and adds the trace id to the logger's fields, so a log line and its trace can be joined.
The tests reuse the existing helpers. A tracetest.SpanRecorder on a SimpleSpanProcessor replaces the exporter in newTestServer, and telemetry_test.go asserts one span per request named GET /users/:id with status code 200 for a stored user and 404 for a missing one, an unset span status on the 404, an error status when a handler returns a plain error, and that a request carrying a traceparent header produces a span whose trace id equals the header's. The twenty-goroutine race test runs unchanged with the middleware in place, which is the check that the recorder and the mutex do not deadlock. go mod tidy records the new go.opentelemetry.io/otel modules, and make test is the proof the pull request carries.
Start it with one API call
POST https://api.phoenix.so/o/{org}/sessions with this body:
{
"agent": "claude",
"input": "Add OpenTelemetry tracing to this Echo v5 API. Read go.mod, Makefile, cookbook/crud/server.go and cookbook/crud/server_test.go first, and check whether the current otelecho release builds against the Echo version in go.mod; if it does not, write the middleware by hand. Add cookbook/crud/telemetry.go: a tracer provider with an OTLP HTTP exporter configured from the standard OTEL_* environment variables, a resource with service.name defaulting to crud, W3C trace context and baggage propagation, and a shutdown function main calls with a timeout after the server stops. Add a Tracing middleware registered in newServer before RequestLogger that extracts the parent context, names the span after the matched route template, records method, route, path and status code, records a returned error with its status, and sets error status only for 5xx. Put the trace id on the request logger's fields. In telemetry_test.go use a SpanRecorder: assert the span name and status code for a found and a missing user, the unset status on 404, the error status on a handler error, and that an incoming traceparent header is honoured. Run go mod tidy and make test, and open a pull request with the test output and the environment variables an operator sets.",
"resources": [
{
"url": "https://github.com/your-org/your-app"
}
]
}- Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
- Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
- Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.
What comes back
The pull request adds telemetry.go and telemetry_test.go under cookbook/crud/, changes newServer() and main() in server.go, and updates go.mod and go.sum with the otel, sdk, exporters/otlp/otlptrace/otlptracehttp and sdk/trace/tracetest modules. Its description pastes the make test output with the race detector on and lists the variables an operator sets: OTEL_EXPORTER_OTLP_ENDPOINT, OTEL_SERVICE_NAME and OTEL_RESOURCE_ATTRIBUTES.
Each span is named by route template, so a collector shows five operations for this service rather than one per user id, and a request that arrives with a traceparent from an upstream service joins that trace instead of starting its own. The 404 and 400 answers stay unflagged; the error status is reserved for the Recover middleware's 500s and for handlers that return an error of their own.
Before merging, a reviewer points the service at a real collector once to see the spans arrive, checks that the shutdown timeout fits the deployment's grace period, and decides whether the request logger should keep its own line now that the span carries the same fields.
Run it on every service without tracing
A team instrumenting several Go services sends this body once per repository to POST /o/{org}/sessions from the terminal or a short script, each with its own repository in resources and a request_id that is a UUID v5 of the repository's full name, so a retried call returns the run already started. Each run reads its own entry point: a service on Echo v4 gets the released otelecho middleware, one that already has a tracer provider gets only the middleware and the tests, and one with a Gin or chi router gets that router's instrumentation under the same span names.
Each run opens its own pull request on its own branch, and GET /o/{org}/sessions lists them newest first so the person who started the batch can read every make test result from one place.
The use case this job follows, with its automation and trigger: Hand off a long task.
Why run it on Rush?
Tracing is easy to add and hard to prove: a middleware that compiles can still name every span by raw path, drop the incoming trace context, or mark every 404 as an error, and none of that shows up until the spans reach a dashboard. The agent proves it in its sandbox with an in-memory recorder, reading the spans the way a collector would, and the assertions it writes stay in the suite.
The run keeps every command and its output, so the reviewer sees go mod tidy and the race-detector run without checking out the branch. Several services can be instrumented at once, each in its own sandbox, and the one that already has tracing is left with a note instead of a second tracer provider.
Frequently asked questions
Why write the middleware instead of using otelecho?
The service is on Echo v5, whose context type differs from v4, and the agent checks whether the current otelecho release builds against it before deciding. When it does, the released middleware is used and the hand-written one is not added. When it does not, the hand-written version is about forty lines over the otel API and gives the same span names and attributes.
Which exporter and protocol does it configure?
OTLP over HTTP with the otlptracehttp exporter, because it needs no gRPC dependency and every collector accepts it. The endpoint, headers and TLS settings come from the standard OTEL_EXPORTER_OTLP_* variables the exporter reads itself, so switching to a hosted backend is a change in the deployment, not in the code.
Does it slow the request path?
The middleware starts one span per request and the batch processor exports in the background, so the handler waits on nothing. The existing race test runs with the middleware installed and twenty concurrent goroutines, which is the check that the recorder adds no lock contention around the user map.
What about the request logger?
It stays, with the trace id added as a field, so a log line can be looked up from its trace and the other way round. Whether to keep both once the spans carry the same method, route and status is raised in the pull request as a question for the team.
Will a developer without a collector see errors?
The exporter retries and then logs a failure through the otel error handler; the API keeps serving. The agent leaves the default endpoint in place and says so in telemetry.go rather than adding a flag that turns tracing off when the variable is missing, since that branch is the kind that hides a misconfigured deployment.