What the agent does
The agent reads how the app is built and tested before it adds a tracer, because an instrumentation that starts before the application is created, or that emits a deprecation warning under a test suite that treats warnings as errors, breaks the build instead of adding spans. In an app laid out like the polls demo in aio-libs/aiohttp-demos, requirements-dev.txt pins aiohttp 3.13.5, aiohttp-jinja2 1.6, asyncpg 0.32.0, trafaret_config 2.0.2, SQLAlchemy 2.0.54, pytest 9.1.1, pytest-aiohttp 1.1.1 and flake8. aiohttpdemo_polls/main.py has init_app, which creates a web.Application, stores the config under a web.AppKey, sets up Jinja2, appends pg_context to cleanup_ctx, and calls setup_routes, setup_static_routes and setup_middlewares; main then calls web.run_app with the host and port from config/polls.yaml. settings.py parses that YAML through trafaret_config against the TRAFARET schema in utils.py, which allows exactly postgres, host and port and refuses any other key. db.py builds a SQLAlchemy 2 async engine with create_async_engine and an async_sessionmaker inside pg_context, and views.py opens a session per request for index, poll, results and vote. middlewares.py renders the 404.html and 500.html templates and logs unhandled exceptions. pytest.ini sets asyncio_mode = auto, filterwarnings = error and xfail_strict, and tests/conftest.py builds the app with init_app against config/polls_test.yaml and a real PostgreSQL. The repository's ci.yml runs flake8 demos, then every demo's tests with PYTHONASYNCIODEBUG=1 against PostgreSQL and Redis services, and builds the Sphinx docs with spelling checks.
Tracing becomes a module and a config block. The agent adds opentelemetry-api, opentelemetry-sdk, opentelemetry-instrumentation-aiohttp-server, opentelemetry-instrumentation-sqlalchemy and opentelemetry-exporter-otlp-proto-http to requirements-dev.txt, pinned like their neighbours, and extends TRAFARET with an optional tracing key holding enabled, service_name and endpoint, because the schema rejects keys it does not know and a new key in polls.yaml alone would fail config parsing. aiohttpdemo_polls/tracing.py has one function, setup_tracing(config), that builds a Resource with service.name, a TracerProvider with a BatchSpanProcessor around an OTLPSpanExporter pointed at the configured endpoint, and calls AioHttpServerInstrumentor().instrument(tracer_provider=provider), which the instrumentation's documentation names as the entry point that traces aiohttp.web.Application requests. It returns the provider so a caller can shut it down.
Where each call sits follows the documentation and the app's own lifecycle. main calls setup_tracing before init_app, since the instrumentor patches application creation. In pg_context, right after create_async_engine, the agent calls SQLAlchemyInstrumentor().instrument(engine=engine.sync_engine), the form that works for an async engine, so every select in views.py becomes a child span of its request span, and the context's teardown calls uninstrument and flushes the provider so the last spans leave before engine.dispose(). Static files under /static/ are excluded with the OTEL_PYTHON_AIOHTTP_SERVER_EXCLUDED_URLS variable the documentation lists, set in the config rather than the environment. Request spans are named GET /poll/{question_id} from the matched route, which is what the instrumentation's get_default_span_name produces when a resource is resolved, so one route is one name regardless of the id.
Tests use the SDK's in-memory exporter, so no collector is needed. A new tests/test_tracing.py builds the app with tracing on and an InMemorySpanExporter in place of OTLP, requests /poll/1/results through the aiohttp_client fixture, and asserts one server span with that route name, a child span from SQLAlchemy, and the HTTP status attribute set; a second test asserts that /static/style.css produces no span. Because filterwarnings = error is on, any deprecation warning the OpenTelemetry packages raise on the pinned Python fails the test, so the agent pins versions that are silent and says which in the pull request. It runs flake8 demos and pytest demos/polls against a PostgreSQL it starts in its sandbox, and pastes the tails into the pull request.
Start it with one API call
POST https://api.phoenix.so/o/{org}/sessions with this body:
{
"agent": "claude",
"input": "Add OpenTelemetry tracing to this aiohttp app. Read requirements-dev.txt, aiohttpdemo_polls/main.py, settings.py, utils.py, db.py, routes.py, views.py, middlewares.py, config/polls.yaml, pytest.ini, tests/conftest.py and the repository's CI workflow first. Add pinned opentelemetry-api, opentelemetry-sdk, opentelemetry-instrumentation-aiohttp-server, opentelemetry-instrumentation-sqlalchemy and opentelemetry-exporter-otlp-proto-http to requirements-dev.txt. Extend the trafaret schema with an optional tracing block holding enabled, service_name and endpoint, and add it to polls.yaml. Write aiohttpdemo_polls/tracing.py with setup_tracing(config) that builds a Resource and TracerProvider with a BatchSpanProcessor and OTLP HTTP exporter and calls AioHttpServerInstrumentor().instrument, called from main before init_app. Instrument the SQLAlchemy engine's sync_engine inside pg_context after it is created and uninstrument and flush on teardown. Exclude the static route. Write tests with InMemorySpanExporter that assert a server span named for the matched route with a SQLAlchemy child span and the status attribute, and that static files produce no span; keep pytest's filterwarnings=error passing. Run flake8 and pytest for this demo against PostgreSQL in the sandbox and open a pull request with the output. Do not change other demos or the CI workflow.",
"resources": [
{
"url": "https://github.com/your-org/your-app"
}
]
}- Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
- Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
- Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.
What comes back
The pull request adds aiohttpdemo_polls/tracing.py and tests/test_tracing.py, changes main.py, db.py and utils.py, adds the tracing block to config/polls.yaml and config/polls_test.yaml, and pins five packages in requirements-dev.txt. Its description lists each span the test saw for one request to the results page, with its name, its parent and its attributes, and pastes the tails of flake8 demos and pytest demos/polls from the sandbox.
The repository's CI workflow runs on the pull request as before, since make install reads the demo's requirements file and make test runs pytest demos/polls, so the reviewer sees the new tests pass against the workflow's PostgreSQL service with PYTHONASYNCIODEBUG=1 on. No span leaves the test process; the OTLP exporter is only built when tracing.enabled is true and no in-memory exporter is supplied.
Before merging, a reviewer sets the endpoint for the collector the team runs, decides whether vote should carry the question and choice ids as span attributes, and decides whether the 500 handler in middlewares.py should record the exception on the active span so a traced error shows the traceback rather than only the status.
Run it on every aiohttp service
A team with several aiohttp services sends this body once per repository from its own server, each a POST /o/{org}/sessions with that repository in resources and a request_id that is a UUID v5 of the repository's full name, so a retried call returns the run already started. Each run reads its own app: a service on asyncpg without SQLAlchemy gets the asyncpg instrumentation instead, one that already configures a TracerProvider for its outgoing ClientSession gets the server instrumentor added to that provider, and one whose config is environment variables rather than YAML gets OTEL_EXPORTER_OTLP_ENDPOINT read directly.
Once spans flow, a slow trace is the next event. A Grafana contact point can send a webhook with a custom Authorization: Bearer header, so an alert on request span duration can call a Rush webhook trigger directly with the trigger's secret, with an Idempotency-Key header on the trigger's URL made from the alert's fingerprint so a repeated notification does not start a second run. The automation's prompt asks for the slow route's handler and its queries to be read and a comment naming the likely cause.
The use case this job follows, with its automation and trigger: Hand off a long task.
Why run it on Rush?
OpenTelemetry on aiohttp is a few calls in the right order and a few traps specific to the app: an instrumentor that must run before the application exists, a config schema that refuses unknown keys, an async engine that has to be instrumented through its sync engine, and a test suite that fails on any warning. The agent reads the lifecycle and the test configuration first and runs the suite in its own sandbox against a real database, so what reaches the pull request has passed the repository's own rules.
The span list and the test output stay with the run and are pasted into the pull request, so the review is of traces the app produced. Several services get the same instrumentation at once, each with its own engine, config format and route list.
Frequently asked questions
Why extend the config schema instead of using environment variables?
Because this app validates polls.yaml with a trafaret schema that refuses unknown keys, and its tests build the app from a second YAML file. A tracing block in the schema keeps one source of configuration and lets the test config turn the exporter off. The endpoint can still be overridden by OTEL_EXPORTER_OTLP_ENDPOINT, which the SDK reads.
Will the tests need a collector?
No. The tests pass an InMemorySpanExporter into setup_tracing, read the finished spans from it and assert on their names and attributes. The OTLP exporter is only built when tracing is enabled and no exporter is supplied.
Why instrument the sync engine?
Because SQLAlchemy's async engine wraps a sync engine, and the OpenTelemetry SQLAlchemy instrumentation attaches to that one. Instrumenting engine.sync_engine inside pg_context is what makes each select in the views a child of the request span.
What about the warnings-as-errors setting?
It stays. pytest.ini turns every warning into a failure, so the agent pins OpenTelemetry versions that raise none on the pinned Python and says which in the pull request. A future upgrade that starts warning will fail the suite, which is the point of the setting.
More aiohttp jobs
The same job on other stacks
- Add OpenTelemetry Tracing to an ASP.NET Core API (ASP.NET Core)
- Add OpenTelemetry Tracing to a chi API (chi)
- Add OpenTelemetry Tracing to a Django REST Framework API (Django REST Framework)