What the agent does
The agent reads how the application is assembled before adding a middleware, because the order of setup_* calls in an aiohttp app decides what a metrics middleware sees. In an app like the polls demo in aio-libs/aiohttp-demos, aiohttpdemo_polls/main.py builds web.Application(), stores the parsed config under an AppKey from typedefs.py, sets up Jinja2 templates, appends pg_context from db.py to cleanup_ctx for the SQLAlchemy async engine over asyncpg, and calls setup_routes, setup_static_routes and setup_middlewares. routes.py registers /, /poll/{question_id}, /poll/{question_id}/results and /poll/{question_id}/vote plus a static route; middlewares.py holds one error middleware that renders 404.html and 500.html. requirements-dev.txt pins aiohttp, aiohttp-jinja2, asyncpg, SQLAlchemy, trafaret_config, pytest and pytest-aiohttp; setup.py lists the install requirements; pytest.ini turns on asyncio_mode = auto and filterwarnings = error, so any deprecation fails the suite. tests/conftest.py builds the app with init_app for the cli fixture over aiohttp_client, creates a test database against a local Postgres as postgres, and tests/test_integration.py covers the index, results, a 404 and a vote. The repository's Makefile runs pytest demos/polls and flake8 demos, and .github/workflows/ci.yml runs them with Postgres and Redis services.
The metrics live in a new aiohttpdemo_polls/metrics.py on prometheus_client. It creates a CollectorRegistry per application, stored under its own AppKey, rather than using the library's default registry, because the test suite calls init_app once per test and registering http_requests_total twice on one process-wide registry raises a duplicated timeseries error. A @web.middleware times each request, resolves the route label from request.match_info.route.resource.canonical, which is /poll/{question_id} whatever the id, and records a Counter labelled by method, route and status and a Histogram labelled by method and route; a request that matched no route, which the existing error middleware turns into the 404 page, gets a fixed label. The middleware is appended in setup_middlewares before the error middleware so it observes the status the client gets, and a /metrics handler that returns generate_latest(registry) with CONTENT_TYPE_LATEST is registered in setup_routes, excluded from its own counter.
prometheus_client goes into requirements-dev.txt beside the other pinned lines and into setup.py. tests/test_metrics.py uses the existing cli and tables_and_data fixtures: two requests to / then a scrape asserting http_requests_total{method="GET",route="/",status="200"} 2.0, a request to /no-such-route landing under the 404 label, a check that the scrape itself is not counted, and a check that the response is served with the Prometheus content type.
The agent installs Postgres in its sandbox, creates the postgres superuser password that init_db.py expects, runs pytest tests -vv from the demo directory under the filterwarnings = error setting, and runs flake8 from the repository root, whose .flake8 sets a 90-column limit and the bugbear checks the new file must satisfy. It starts the app once with python -m aiohttpdemo_polls, votes through the browser flow with curl and scrapes /metrics, keeping the output for the pull request.
Start it with one API call
POST https://api.phoenix.so/o/{org}/sessions with this body:
{
"agent": "claude",
"input": "Add Prometheus metrics to the aiohttp app in demos/polls. Read requirements-dev.txt, setup.py, pytest.ini, aiohttpdemo_polls/main.py, routes.py, middlewares.py, typedefs.py, db.py, tests/conftest.py, tests/test_integration.py, config/polls_test.yaml, init_db.py, the root Makefile, .flake8 and .github/workflows/ci.yml first. Add prometheus_client to requirements-dev.txt and setup.py. Write aiohttpdemo_polls/metrics.py with a CollectorRegistry per application under its own AppKey, a middleware recording http_requests_total labelled by method, route pattern from request.match_info.route.resource.canonical and status, and http_request_duration_seconds labelled by method and route, with a fixed label for unmatched routes, and a /metrics handler using generate_latest and CONTENT_TYPE_LATEST that is not counted itself. Register the middleware in setup_middlewares before the error middleware and the route in setup_routes. Write tests/test_metrics.py on the existing cli and tables_and_data fixtures covering the counter after two requests, the 404 label, the scrape not counting itself and the content type. Start Postgres in the sandbox with the superuser password init_db.py expects, run pytest tests -vv under the filterwarnings=error setting, run flake8 from the repository root, start the app and scrape /metrics once with curl, and open a pull request that lists the metric names and labels.",
"resources": [
{
"url": "https://github.com/your-org/your-app"
}
]
}- Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
- Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
- Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.
What comes back
What comes back is a pull request that adds aiohttpdemo_polls/metrics.py and tests/test_metrics.py, changes main.py, routes.py, middlewares.py, typedefs.py, requirements-dev.txt and setup.py. The description lists the two metric names with their labels, explains the per-application registry in one paragraph for the next person who wonders why the default one is not used, and pastes the first lines of a scrape from the sandbox.
The run's record keeps the Postgres setup, the full pytest tests -vv output with the existing integration tests and the new ones, the flake8 run and the curl scrape, so a reviewer sees the suite pass under filterwarnings = error rather than hoping no deprecation was introduced.
Before merging, a reviewer decides whether /metrics should be reachable on the application port, which config/polls.yaml binds to 127.0.0.1:8080, or served by a second web.Application on another port, and whether the histogram's default buckets fit the latencies they expect.
Run it on every Linear issue labelled observability
In the console, add an automation to the Project with this prompt and a linear trigger on linear.issue.labeled with the label observability. Labelling an issue starts one run with the issue after the prompt; the run reads which service or which extra metric the issue asks for, such as a gauge for the database pool size from the engine in db.py, adds it beside the request metrics and opens a pull request that links the issue. Changes made by an app integration never fire the trigger.
The same body sent to POST /o/{org}/sessions with the repository in resources starts one run by hand. A request_id that is a UUID v5 of the issue identifier makes a retried call return that run instead of a second pull request.
The use case this job follows, with its automation and trigger: Turn a Linear label into a PR.
Why run it on Rush?
The parts of this change that go wrong are invisible in a diff: a registry shared across the test suite's repeated app factories, a route label that is the raw path and grows a series per poll id, a middleware placed after the one that converts exceptions so it never sees a 500. The agent runs the real suite against a real Postgres in a sandbox of its own, under the project's strict warning setting, and scrapes the real endpoint, so each of those is observed before the pull request exists.
Every command's output stays with the run, so the reviewer reads the test output and the scrape rather than reproducing them, and the same request can go to every aiohttp service in the org at once, each in its own sandbox with its own pull request.
Frequently asked questions
Why a registry per application instead of the default one?
The test suite builds a fresh app with init_app for every test. On the process-wide default registry the second Counter with the same name raises a duplicated timeseries error; a CollectorRegistry created in init_app and stored under an AppKey is dropped with the app. The /metrics handler reads that registry.
Will the route label explode with poll ids?
No. The label is request.match_info.route.resource.canonical, the registered pattern such as /poll/{question_id}, so every id shares one series. Requests that match no route get one fixed label.
Does it work with the project's warning settings?
The suite runs with filterwarnings = error, so a deprecated call in the new code fails the tests rather than printing. The agent runs the suite under that setting and fixes any warning the metrics code raises before opening the pull request.
Is the endpoint authenticated?
Not by this change; it is served on the application port like the pages. The pull request names the two usual alternatives, a second web.Application on a port only the scraper reaches, or a check on the scraper's address in the handler, as a reviewer's decision.