All jobs

Add Rate Limits to Celery Tasks

Yes. An agent adds per-task rate limits to a Celery app, checks them against a real worker and Redis broker in its sandbox, and opens a PR with the worker log.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

The agent reads how tasks are defined and configured before touching a limit. In an app laid out like miguelgrinberg/flask-celery-example, everything is in app.py: a Flask app, a Celery instance built with Celery(app.name, broker=...) followed by celery.conf.update(app.config), so configuration arrives as uppercase CELERY_BROKER_URL and CELERY_RESULT_BACKEND keys pointing at redis://localhost:6379/0. Two tasks: send_async_email, which sends through Flask-Mail with Gmail's SMTP host and credentials from MAIL_USERNAME and MAIL_PASSWORD, and long_task, bound, which reports progress through update_state for a web page that polls /status/<task_id>. The index route calls send_async_email.delay(...) or apply_async(args=[...], countdown=60). requirements.txt pins Celery 4.4.1, Flask 1.1.1 and redis 3.4.1; run-redis.sh builds Redis from source; there are no tests and no README note about the mail provider's sending limits.

A rate limit belongs on the task that talks to a third party, which is the mail task. Celery takes it as the rate_limit argument of the task decorator, in its n/s, n/m or n/h form, so the agent changes the decorator to @celery.task(rate_limit=os.environ.get('MAIL_RATE_LIMIT')): unset means no limit, as before, and the value is whatever the provider allows, which the reader knows and the agent does not. It puts it in the decorator rather than in task_annotations on purpose: this app configures Celery with old-style uppercase keys, and Celery 4 refuses a configuration that mixes old and new names, so adding one new-style key would break startup. long_task gets no limit; it is bound by the page that polls it, not by a provider.

Then it explains what the limit does and does not do, in the README and the pull request, because this is where rate limits surprise people. A Celery rate limit is per worker process, not per cluster: two workers each send at the configured rate. A limited task that cannot run yet sits in the worker's prefetch buffer, so when emails pile up they can delay long_task on the same worker; the agent routes mail to its own queue with CELERY_ROUTES in the same uppercase style, and documents starting one worker with -Q mail and one for the rest, so a backlog of emails never holds a progress bar. A countdown still applies: the task is scheduled first and limited when it becomes due. And the limit can be changed on a running worker with celery -A app.celery control rate_limit app.send_async_email <rate> without a restart, which the README now says.

In its sandbox the agent installs redis-server from apt instead of running run-redis.sh, creates a virtualenv from requirements.txt, sets MAIL_RATE_LIMIT, starts a worker with celery -A app.celery worker --loglevel=info -Q mail, and enqueues a burst of send_async_email calls from a small script that sets app.config['MAIL_SUPPRESS_SEND'] = True, which makes Flask-Mail skip the SMTP connection, since no mail credentials exist in the sandbox. The worker log then shows each task received at once and succeeded at the spacing the limit dictates, and that log is the proof. It adds a tests/test_rate_limit.py that imports app with the variable set and asserts send_async_email.rate_limit equals it, so the wiring cannot silently disappear.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "claude",
  "input": "Add a rate limit to the task in this Celery app that sends email. Read app.py, requirements.txt, run-redis.sh, templates/index.html and the README first, and note that Celery is configured through uppercase CELERY_* keys passed with conf.update. Set the limit with the task decorator's rate_limit argument, read from a MAIL_RATE_LIMIT environment variable in Celery's n/m form, unset meaning unlimited; do not add new-style configuration keys beside the old-style ones. Route the mail task to its own queue in the same configuration style and document running a worker per queue. Leave the progress task unlimited. In the sandbox, install Redis, start a worker on the mail queue with the variable set, enqueue a burst of mail tasks with Flask-Mail's MAIL_SUPPRESS_SEND on so nothing is sent, and keep the worker log. Add a test that asserts the task's rate_limit equals the variable. Explain in the README that the limit is per worker process and how to change it at run time with celery control. Open a pull request whose description quotes the worker log lines that show the spacing.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

What comes back is a pull request that changes app.py and the README and adds tests/test_rate_limit.py. The description quotes the worker log from the sandbox run, with the received lines arriving together and the succeeded lines spaced as the limit dictates, names the queue the mail task now uses and the worker commands for each queue, and states that the limit is per worker process.

The run's record keeps the complete worker session and the script that enqueued the burst, so a reviewer can read every timestamp rather than trust a description of the spacing.

Before merging, a reviewer sets MAIL_RATE_LIMIT to what the mail provider actually permits, decides how many mail workers will run since each has its own limit, and confirms the deployment starts the second worker for the new queue, because a queue nobody consumes is a silent backlog.

Run it on every Linear issue labelled celery

In the console, add an automation to the Project with a linear trigger on linear.issue.labeled and the label celery, with this prompt. When someone files an issue about a provider refusing bursts of mail and labels it, the run starts with the issue as its context, makes the change in the Project's repository and opens a pull request on its own branch linked to the issue. A label added by an app integration never fires the trigger, and the automation starts at most max_runs_per_hour runs in an hour.

To start it from a support tool instead, send this body to POST /o/{org}/sessions with the issue text appended to the prompt and a request_id that is a UUID v5 of the issue id, so a retried delivery returns the run already started.

The use case this job follows, with its automation and trigger: Turn a Linear label into a PR.

Why run it on Rush?

A rate limit is a line of code and a paragraph of understanding, and the paragraph is what goes wrong: the limit is set in a configuration style the app does not use and the worker refuses to start, or it is set per worker and three workers send three times the rate, or limited emails sit in front of other tasks on the same queue. In a Rush sandbox the agent runs a real broker and a real worker, enqueues a burst and reads the timestamps, so the behaviour in the pull request is observed rather than described.

The worker log stays with the run, so the reviewer sees the spacing between tasks and not a claim about it. A team with several Celery services runs the same job on each in parallel, each in its own sandbox with its own Redis and its own pull request.

Frequently asked questions

Why read the limit from an environment variable?

Because the right number belongs to the mail provider's terms and the account's tier, which differ between the sandbox, staging and production. The decorator reads MAIL_RATE_LIMIT at import time, unset means unlimited as before, and celery control rate_limit changes it on a live worker when the provider changes the terms.

Is the limit shared across workers?

No. Celery enforces a task's rate limit in each worker process separately, so the total sending rate is the limit multiplied by the number of worker processes consuming the mail queue. The README the agent writes says this and the reviewer sizes the value with it in mind.

Why a separate queue for mail?

A limited task that cannot run yet waits inside the worker that already reserved it, ahead of whatever that worker reserved after it. With mail on its own queue and its own worker, a backlog of emails cannot delay the progress task the web page is polling.

Does it work with the countdown the app already uses?

Yes. apply_async(countdown=60) schedules the task for a minute later; when it becomes due the worker applies the rate limit like any other execution. The two compose, and the test the agent adds only checks the limit is wired, since timing with a countdown is what the sandbox log shows.

How is this checked without sending mail?

Flask-Mail skips the SMTP connection when MAIL_SUPPRESS_SEND is set, so the task runs to completion with no credentials and nothing leaves the sandbox. The worker log still records when each task was received and when it succeeded, which is all the limit affects.

More Celery jobs

More jobs like this: Turn a Linear label into a PR

Keep reading