What the agent does
In a FastAPI service like testdrivenio/fastapi-tdd-docker, the job starts with a dependency audit. project/requirements.txt pins fastapi==0.115.12, uvicorn==0.34.1, tortoise-orm==0.25.0, asyncpg==0.30.0 and httpx==0.28.1, with two plain pin files instead of a lockfile. The agent reads project/app/main.py, where create_application() mounts the ping and summaries routers, and project/app/db.py, where register_tortoise wires the TORTOISE_ORM dict to Postgres. It installs from requirements-dev.txt, which adds pytest 8.3.5, and records the baseline: python -m pytest ., plus the flake8, black --check and isort --check-only steps that .github/workflows/main.yml runs inside the built container.
The agent adds the OpenTelemetry SDK packages and the three instrumentations this service needs. opentelemetry-instrumentation-fastapi wraps the application factory to emit HTTP server spans with route and status attributes. Tortoise ORM 0.25 carries its own telemetry integration, so database spans on asyncpg queries come from configuring it rather than patching. The summarizer in project/app/summarizer.py fetches articles with newspaper3k, and the agent wraps the httpx calls that path depends on with opentelemetry-instrumentation-httpx so outbound HTTP shows up as its own spans. Everything is configured through the standard OTEL_* environment variables: OTEL_SERVICE_NAME names the service and OTEL_EXPORTER_OTLP_ENDPOINT points at a collector, so deployment needs no code change to switch destinations.
To prove the wiring, the agent runs a local OpenTelemetry collector in the sandbox, starts uvicorn against a throwaway Postgres container, and exercises the real endpoints: GET /ping, POST /summaries with a payload, GET /summaries and GET /summaries/{id}. The background task that calls generate_summary produces an outbound span the synchronous checks would miss, so the agent waits for it before exporting. Spans are exported to a file in OTLP JSON and attached to the run, alongside the collector config it used. The pytest suite is run again because register_tortoise has its own instrumentation hooks and the tests mock app.dependency_overrides, a combination that has to keep working with telemetry enabled.
The pull request adds the new requirements, a small instrumentation module imported by create_application(), the collector config, and the exported trace file as evidence. It leaves the existing Dockerfiles, the three lint steps and the Heroku deploy job in .github/workflows/main.yml untouched. If an instrumentation cannot be enabled without changing behavior, the agent records that in the description instead of shipping a partial span set silently.
Start it with one API call
POST https://api.prix.dev/o/{org}/sessions with this body:
{
"agent": "claude",
"input": "Add OpenTelemetry tracing to this service: HTTP server spans, database spans and outbound HTTP spans, configured through the standard OTEL_* environment variables. Run a local collector in the sandbox, exercise the main endpoints, and open a pull request with an export of the traces.",
"resources": [
{
"url": "https://github.com/your-org/your-app"
}
]
}- Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
- Send the request. POST the body in request.json to https://api.prix.dev/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
- Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.
What comes back
The pull request contains the instrumentation module, the added OpenTelemetry packages in requirements.txt and requirements-dev.txt, a collector config, and a trace export file showing spans from the exercised endpoints. Its description says which endpoints were hit and which of the three span kinds each request produced. The CI commands still pass: python -m pytest . plus flake8, black and isort, so the service builds and lints exactly as before.
A reviewer opens the trace export and checks that a POST /summaries request carries a server span, a database span for the insert, and an outbound span from the background summarizer. The OTEL_* variables in the collector config should be the only configuration surface; a hardcoded endpoint in code is a review finding. The session read returns the repository and pull request information, and the transcript holds every command the agent ran, including the collector logs and the export step.
Run it on every new service deployed
When a service repo gains its first production deployment, your automation sends POST /o/{org}/sessions with the repository URL and this tracing prompt, so every service gets the same baseline instrumentation instead of whichever spans each team remembered. Install the Rush GitHub App on the repository first and authenticate with an API key; an Idempotency-Key header keeps a retried submission from starting two runs.
Keep the returned execution_id and read the session when the run settles, or receive the key's completion webhook. If a service needs an extra span attribute or a different sampler, send a follow-up on the finished turn; the messages endpoint wakes the parked sandbox where it left off. A follow-up sent while the turn is still running is refused with a conflict, so check status first.
The use case this job follows, with its automation and trigger: Hand off from your terminal.
Why run it on Rush?
Tracing work means running a collector, a database and the service together and clicking through real requests to see spans, which turns a laptop into a port-and-container juggling exercise. On Rush the agent gets its own sandbox, so the collector listens where it wants and the sandbox is parked, disk and memory intact, when the turn ends. The transcript keeps the collector logs and the export command beside the code changes, so the proof that spans exist is inspectable rather than asserted.
For a whole set of FastAPI services, submit one repository per run within your key's concurrency allowance and each gets instrumented the same way. You review the pull requests independently and use a follow-up where a service needs its sampler or resource attributes adjusted. The useful proof is a trace export tied to real endpoints, with the test suite still green.
Frequently asked questions
Which spans will the service emit?
Three kinds: HTTP server spans from the FastAPI instrumentation, database spans from Tortoise ORM's telemetry integration over asyncpg, and outbound HTTP spans from the httpx instrumentation. The trace export in the pull request shows all three on a real request, which is the acceptance evidence.
Does adding tracing change how the service behaves?
It should not. Instrumentation is middleware and client wrappers, not route changes, and the prompt asks the agent to prove it by rerunning the pytest suite and the three lint steps the repository's CI already runs. Any behavior change belongs in the review notes, not slipped into the diff.
How do I point it at my own collector?
Through the environment. OTEL_SERVICE_NAME and OTEL_EXPORTER_OTLP_ENDPOINT are the configuration surface, so your deployment sets the endpoint for its collector and no code change is needed. A pull request that hardcodes an endpoint should be sent back.
What if a dependency has no instrumentation?
The agent says so in the pull request description and covers the gap with manual spans where the gap matters, as with the background summarizer task. Missing coverage is listed for review rather than hidden.
The tests mock settings and the database. Will they still pass?
They must. The tests override get_settings through app.dependency_overrides and register Tortoise against a test database, and the run reruns python -m pytest . after instrumentation is enabled. A failure there means the wiring broke the test setup, and the run ends with that failure named.