# Add Task Routing to a Celery App

Yes. An agent reads your tasks, routes the slow ones to their own queue with task_routes, proves it with workers in the sandbox, and opens a PR.

[Run this on Rush](https://cloud.getrush.ai/run/add-task-routing/celery/). Opens the console with this prompt ready to send, in a Project that has your repository.

## What the agent does

The agent starts with what the workers are doing. In an app laid out like miguelgrinberg/flask-celery-example, everything lives in `app.py`: a `Celery(app.name, broker=...)` built from `CELERY_BROKER_URL` and `CELERY_RESULT_BACKEND` on `redis://localhost:6379/0`, two tasks, and the Flask routes that call them. `send_async_email` sends one message through Flask-Mail, either `.delay()` right away or `.apply_async(countdown=60)` from the form, and `long_task` is `bind=True`, loops for ten to fifty seconds calling `self.update_state(state='PROGRESS', ...)` with `time.sleep(1)` between steps, and `/status/<task_id>` polls its `AsyncResult`. Both go through the one default `celery` queue, so a handful of `long_task` runs started from the page hold every worker process and the email a visitor asked for waits behind them. The README's worker command is `celery -A app.celery worker --loglevel=info`, and `requirements.txt` pins `celery==4.4.1` with `redis==3.4.1`.

It splits the queues by what each task costs. `task_routes` in the Celery config maps `app.long_task` to a `slow` queue and `app.send_async_email` to a `mail` queue, written with the module-qualified names Celery registers (the agent reads them from `celery -A app.celery inspect registered` rather than guessing), with `task_default_queue` left at `celery` so a task added later still has somewhere to go. The config moves from `app.config` uppercase keys into `celery.conf.update(...)` with the lowercase names Celery 4 reads, which is also where `task_acks_late=True` and `worker_prefetch_multiplier=1` go for the `slow` queue's worker, so a ten-second task is not reserved four at a time by a process that then dies. The `countdown=60` path is untouched: ETA tasks sit in the worker that reserved them, and `prefetch_multiplier=1` on the mail worker keeps that from starving the queue.

The README's one worker line becomes two, `celery -A app.celery worker -Q mail --concurrency 2` and `celery -A app.celery worker -Q slow --concurrency 4`, plus a note that a worker started with no `-Q` serves only the default queue and will no longer pick up either task. A `Procfile`-style `run-workers.sh` beside `run-redis.sh` starts both for local use.

In its sandbox the agent installs the pinned requirements, starts Redis with `run-redis.sh`, starts both workers, and sends one `long_task.apply_async()` and one `send_async_email.delay(...)` with `MAIL_SUPPRESS_SEND` set so Flask-Mail builds the message without an SMTP connection. It reads `celery -A app.celery inspect active_queues` to show each worker consuming only its queue, and `redis-cli LLEN slow` before the worker takes the job, and pastes both into the pull request.

## Start it with one API call

`POST https://api.phoenix.so/o/{org}/sessions`

```json
{
  "agent": "claude",
  "input": "Add task routing to this Celery app. Read app.py, requirements.txt, run-redis.sh and the README first. Route long_task to a slow queue and send_async_email to a mail queue using task_routes with the names Celery registers; check them with celery -A app.celery inspect registered rather than guessing. Move the Celery settings into celery.conf.update with the lowercase Celery 4 names, keep task_default_queue at celery, and set task_acks_late and worker_prefetch_multiplier 1. Do not change the countdown=60 path or the progress reporting in long_task. Update the README's worker command to one worker per queue and add run-workers.sh beside run-redis.sh. In the sandbox start Redis and both workers, send one task of each kind with MAIL_SUPPRESS_SEND set, and quote inspect active_queues and the redis-cli queue length in the pull request. Open a pull request.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
```

1. **Get an API key.** In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
2. **Send the request.** POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
3. **Read what came back.** Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

## What comes back

What comes back is a pull request touching `app.py`, `README.md` and a new `run-workers.sh`. The description quotes the `inspect registered` output that the route names were taken from, the `inspect active_queues` block showing the `mail` worker on `mail` and the `slow` worker on `slow`, and the worker log lines where each task was received and succeeded.

The run's record keeps both workers' full logs and the Redis server output, so a reviewer can see the `long_task` progress updates arriving on the slow worker while the mail worker stays idle. Nothing was sent: `MAIL_SUPPRESS_SEND` kept Flask-Mail from opening an SMTP connection, and the pull request says so.

Before merging, a reviewer sets the concurrency per worker for the real host, and decides whether `task_acks_late` is right for `send_async_email`: with it on, a worker killed mid-send delivers the message twice, and the pull request names that trade-off next to the setting.

## Run it on every Linear issue labelled queues

A team with several Celery services makes this a rule. In the Project for each repository, an automation with the prompt above and a `linear` trigger on `linear.issue.labeled` for the label `queues` starts a run when someone files "move the report export off the default queue". The trigger appends the issue after the prompt, so the run reads which task the issue names and routes that one, keeping everything else where it was.

Each run opens its own pull request on its own branch and links the issue. For a repository that already has `task_routes`, the run extends the existing map rather than adding a second one, and the README's worker list gains one line.

## Why run it on Rush?

Routing is a dict, and the mistakes are in the names and the worker flags: a route keyed on the function name instead of the registered `app.long_task`, a worker started without `-Q` that silently serves nothing you routed, a prefetch multiplier left at 4 so one worker reserves forty seconds of work. Each one is invisible in a diff and obvious with a worker running. In a Rush sandbox the agent starts Redis and the workers, reads the registered names from Celery itself, and the pull request quotes what the broker saw.

The worker logs stay with the run, so the next person who adds a queue can read exactly what `active_queues` looked like when the split was made. Across a dozen services on the same pattern the job runs in parallel, one sandbox each, and each pull request has its own proof.

## Frequently asked questions

### Will the progress bar on the page still work?

Yes. `long_task` still calls `self.update_state` and `/status/<task_id>` still reads the result backend; only the queue the task waits in changes. The result backend stays on the same Redis database.

### What happens to the one worker command I run today?

A worker started with no `-Q` consumes only the default `celery` queue, and after this change neither task goes there, so it would sit idle. The README and `run-workers.sh` start one worker per queue; a single worker with `-Q celery,mail,slow` also works for a laptop.

### Why keep task_default_queue?

So a task added next month without a route still has a queue and a worker to run it. The pull request leaves it at `celery` and the README says a worker for it is still needed if new tasks are added without a route.

### Does this need a newer Celery?

No. `task_routes`, `task_acks_late` and `worker_prefetch_multiplier` are the lowercase names Celery 4 reads, and the project pins 4.4.1. An upgrade is a different pull request.

### Can the slow queue have a rate limit too?

Yes. Say so in the prompt and the agent adds `task_annotations` with a `rate_limit` for `app.long_task`, which is per worker process, and notes that in the README next to the concurrency flag.

## More Celery jobs

- [Add Task Time Limits to a Flask Celery App](https://getrush.ai/run/add-task-timeouts/celery/)
- [Fix the Celery broker_connection_retry Deprecation Warning](https://getrush.ai/run/fix-broker-connection-retry-deprecation-warning/celery/)
- [Fix "Error: No such option '-A'" when a Celery worker starts after moving to Celery 5](https://getrush.ai/run/fix-celery-no-such-option-a/celery/)

## More jobs like this: Turn a Linear label into a PR

- [Add Team Invitations to a Filament Panel](https://getrush.ai/run/add-team-invitations/filament/)
- [Shard a Jest Suite Across Parallel CI Jobs for an Express API](https://getrush.ai/run/add-test-sharding/jest/)
- [Shard Playwright Tests Across GitHub Actions Runners](https://getrush.ai/run/add-test-sharding/playwright/)

## Keep reading

- [How a Rush run works on your repository, from request to pull request](https://getrush.ai/cloud/)
- [Compare cloud agents, agent sandboxes and AI code review](https://getrush.ai/compare/)
- [Rush pricing: Free, Pro and Max](https://getrush.ai/pricing/)
