All jobs

Add Rate Limiting to an aiohttp App

Yes. An agent adds a per-client token bucket middleware to an aiohttp app, answers 429 with Retry-After, tests it with pytest-aiohttp and opens a PR.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

aiohttp ships no rate limiter, so the agent starts by reading how the app is wired. In a project laid out like the polls demo in aio-libs/aiohttp-demos, aiohttpdemo_polls/main.py builds the application in init_app: it loads the YAML config through get_config in settings.py, sets up Jinja2, appends pg_context to app.cleanup_ctx, then calls setup_routes, setup_static_routes and setup_middlewares. routes.py registers four handlers, GET /, GET /poll/{question_id}, GET /poll/{question_id}/results and POST /poll/{question_id}/vote, and middlewares.py appends one middleware that maps 404 and 500 to templates and re-raises every other web.HTTPException. requirements-dev.txt pins aiohttp==3.13.5, aiohttp-jinja2==1.6, asyncpg==0.32.0 and SQLalchemy==2.0.54, with pytest==9.1.1 and pytest-aiohttp==1.1.1 for tests, and typedefs.py already defines typed web.AppKey entries for the config and the session factory.

The handler worth protecting is vote in views.py: it reads the form with request.post(), casts choice to an integer, calls db.vote and redirects with HTTPFound to the results page. Nothing stops a client from posting it in a loop. The agent writes aiohttpdemo_polls/ratelimit.py with a token bucket keyed by client address: a RateLimiter class holding a dict of buckets, a capacity and a refill_per_second, and an acquire(key) method that refills from the loop clock and returns either None or the number of seconds until the next token. The clock is injected (time.monotonic by default) so tests can control it. A @web.middleware function reads request.remote, and only X-Forwarded-For when the config says the app sits behind a trusted proxy, calls acquire, and raises web.HTTPTooManyRequests with a Retry-After header when the bucket is empty. The limiter instance lives on the app under a new web.AppKey in typedefs.py, so handlers and tests reach it the same way they reach the database.

Configuration follows the pattern the demo already uses. The agent adds a rate_limit block to config/polls.yaml and config/polls_test.yaml with requests, period and trusted_proxy, extends the TRAFARET schema in utils.py so a missing or malformed block fails at startup the way a bad postgres block does, and in middlewares.py appends the limiter after the existing error middleware. Order matters here: middlewares run outermost first, so the error middleware wraps the limiter, sees a 429 as an HTTPException with no override and re-raises it unchanged. The limit applies to POST routes only, which in this app means the vote handler; page views and static files are not counted.

Tests go next to tests/test_integration.py, which already has a cli fixture from aiohttp_client and tables_and_data for a seeded database. The new tests/test_ratelimit.py posts a vote past the configured requests count, asserts the last answer is 429 with a Retry-After header, advances the injected clock and asserts the next vote goes through. One unit test drives RateLimiter.acquire directly with a fake clock. pytest.ini sets asyncio_mode = auto and turns every warning into an error, so the agent runs pytest tests from demos/polls as make test does, with a PostgreSQL started in its sandbox for the integration fixtures, and runs flake8 as the repository's make lint does. Along the way it reads db.vote and notes in the pull request that result.votes += 1 runs before the if not result check, so an unknown choice raises AttributeError instead of RecordNotFound; it leaves that fix out of scope unless asked.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "claude",
  "input": "Add per-client rate limiting to this aiohttp app. Read aiohttpdemo_polls/main.py, routes.py, views.py, middlewares.py, settings.py, utils.py, typedefs.py, config/polls.yaml, config/polls_test.yaml, requirements-dev.txt, pytest.ini and tests/conftest.py first. Write a token bucket limiter keyed by client address with an injectable clock, expose it on the app through a web.AppKey, and add a middleware that answers 429 with a Retry-After header for POST routes once a client is over the limit. Put requests, period and trusted_proxy in the YAML config and validate them in the trafaret schema. Append the middleware after the existing error middleware and explain why in a comment. Add tests that exhaust the limit, assert the 429 and the header, advance the clock and assert recovery. Run pytest and flake8 the way the Makefile does and open a pull request that shows the command output.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

The pull request adds aiohttpdemo_polls/ratelimit.py and tests/test_ratelimit.py, and changes middlewares.py, typedefs.py, utils.py, config/polls.yaml and config/polls_test.yaml. Its description quotes the pytest output for the existing integration tests and the new ones, the flake8 result, and a curl loop against python -m aiohttpdemo_polls that shows the first answers as 302 redirects and the one past the limit as 429 with its Retry-After value.

The repository's CI workflow runs make lint and make test with PostgreSQL and Redis services on the pull request, so the reviewer sees the same tests pass on GitHub. The run's record keeps every command the agent ran, including the curl loop, so a reviewer who doubts the middleware ordering can read the request that was limited and the one that was not.

Before merging, a reviewer settles the real requests and period for production, whether trusted_proxy should be on behind their load balancer, and whether the limit should also cover the results page. The agent changes no route, no template and no database code.

Run it on every aiohttp service without a rate limit

A team that runs several aiohttp services starts one run per repository from its own server: a POST /o/{org}/sessions each, with the repository in resources and a request_id that is a UUID v5 of the repository's full name, so a retried call returns the run already started instead of a second one. Each run reads its own routes.py and middlewares.py, so a service that already has a limiter gets a report and no pull request, and one that keeps its config in environment variables instead of YAML gets the limit read from there.

To keep the limit honest, an automation in the Project with a github trigger on pull_request.opened and the prompt "if this pull request adds a POST route, check it is covered by the rate limit middleware and that the tests exercise it, and say so in a comment" reviews each new handler. The trigger adds the pull request after the prompt and fires only for work by owners, members and collaborators.

The use case this job follows, with its automation and trigger: Hand off a long task.

Why run it on Rush?

A rate limiter is easy to write and easy to get subtly wrong: a middleware appended in the wrong position gets its 429 turned into a rendered error page, a bucket keyed on X-Forwarded-For lets any client pick its own key, and a test that sleeps for real seconds makes the suite slow. The agent reads the app's own middleware order, runs the real server in its sandbox and posts to it, and only then opens the pull request with the output attached.

Each run has its own machine with its own PostgreSQL, so running this against a dozen aiohttp services at once costs no more attention than one, and each pull request carries the pytest, flake8 and curl output that justifies it.

Frequently asked questions

Why not use an existing aiohttp rate limiting package?

aiohttp has no limiter of its own, and the third-party packages on PyPI differ in maintenance and in how they identify a client. A token bucket is about forty lines, has no dependency to pin in requirements-dev.txt, and the agent tests it directly. If the reader prefers a package, the prompt can name it and the agent will wire that instead.

What key does the limiter use?

request.remote, the peer address aiohttp reports. When trusted_proxy is on in the config, it takes the first address in X-Forwarded-For instead, because behind a proxy every request shares the proxy's address. The agent does not read that header by default, since a client can send it.

Will this slow down the tests?

No. The limiter takes a clock function, and the tests pass a fake one they advance by hand, so exhausting and refilling a bucket takes no wall-clock time. The existing integration tests still need the PostgreSQL that tests/conftest.py sets up.

Does the limit apply to every route?

Only to POST routes, which in the polls app means the vote handler. GET pages and the static route are not counted. Extending it to a path prefix or a per-route marker is a small change the reviewer can ask for in a follow-up.

More aiohttp jobs

The same job on other stacks

Keep reading