All jobs

Add a Scheduled Job to a Flask App That Uses RQ

Yes. An agent adds a periodic job to your RQ queue, schedules it with RQ's own scheduler, wires the worker flag into your process managers, tests it, and opens a PR.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

The agent reads how jobs run today. In a Flask app laid out like miguelgrinberg/microblog, app/__init__.py opens Redis.from_url(app.config['REDIS_URL']) and an rq.Queue named microblog-tasks on it; requirements.txt pins rq==1.15.1 and redis==5.0.1. The only job is export_posts in app/tasks.py, which builds its own app with create_app() and pushes a context at import, reports progress through job.meta and a Task row from app/models.py, and is enqueued by User.launch_task as the dotted name app.tasks.export_posts. The worker is started by hand: Procfile has worker: rq worker microblog-tasks, and deployment/supervisor/microblog-tasks.conf runs the same command under supervisor. Nothing runs on a clock, so a job that should happen nightly, such as clearing Notification rows older than a month or re-running Post.reindex() from the SearchableMixin, has no place to live.

RQ 1.15 ships a scheduler of its own, started with rq worker --with-scheduler, that honours enqueue_at and enqueue_in on a queue; it has no cron syntax, which is what the separate rq-scheduler package adds. The agent picks the built-in one, since it adds no dependency and the worker command already exists, and writes the periodic job as a function in app/tasks.py that does its work and then calls enqueue_in on app.task_queue for its next run, with the interval read from a new config.py setting beside REDIS_URL. A flask tasks schedule command in app/cli.py, next to the translate group, enqueues the first run and is idempotent: it checks the queue's ScheduledJobRegistry for a job with the same id before adding one, so running it on every deploy from boot.sh is safe.

The job itself follows the shape of export_posts: it runs inside the module's app context, logs through app.logger, and wraps its body so an exception is logged and the next run is still scheduled. For the notification cleanup the body is one delete on Notification where timestamp is older than the configured age, committed through db.session; the agent writes that one because add_notification in app/models.py already deletes same-named rows per user and leaves the rest to grow. The worker command gets --with-scheduler in Procfile and in the supervisor file, since a worker without the flag runs the job once and never again.

Tests go in tests.py, which already defines a TestConfig with an in-memory SQLite URL and a UserModelCase on unittest. The agent adds a case that points REDIS_URL at a Redis it installs in the sandbox, creates notifications with old and new timestamps, runs the job function directly and asserts only the old rows are gone; a second case calls the schedule command twice and asserts the registry holds one job. It runs python -m unittest tests.py and starts rq worker --with-scheduler microblog-tasks against the same Redis long enough to see the first run and the re-enqueue in the worker log, and opens the pull request with both.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "claude",
  "input": "Add a periodic job to this Flask app that uses RQ. Read requirements.txt, config.py, app/__init__.py, app/tasks.py, app/models.py, app/cli.py, app/main/routes.py, boot.sh, Procfile, deployment/supervisor/microblog-tasks.conf, Dockerfile and tests.py first. Write a job in app/tasks.py that deletes Notification rows older than an age read from a new config setting, following export_posts for app context, logging and error handling, and that re-enqueues itself with enqueue_in on app.task_queue after each run using an interval from config. Add a flask tasks schedule command in app/cli.py that enqueues the first run only if the ScheduledJobRegistry has no job with that id, and call it from boot.sh after flask db upgrade. Add --with-scheduler to the rq worker command in Procfile and the supervisor config. Add unittest cases in tests.py for the cleanup and for the idempotent schedule command. Install Redis in the sandbox, run python -m unittest tests.py, run the worker with the scheduler long enough to show one run and the re-enqueue in its log, and open a pull request with the output.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

What comes back is a pull request that changes app/tasks.py, app/cli.py, config.py, boot.sh, Procfile, deployment/supervisor/microblog-tasks.conf and tests.py, with no new entry in requirements.txt. The description shows the job, the schedule command and the worker log lines in which the job runs and its successor is enqueued.

The run's record keeps the unittest output and the worker's log, so a reviewer sees the re-enqueue happen rather than reads that it would.

Before merging, a reviewer sets two values the pull request names with their defaults: how old a notification must be before it is deleted, and how often the job runs. Both are config settings, so changing them is a deploy and not a code change.

Run it on every Linear issue labelled agent

Requests for scheduled jobs arrive one at a time: a nightly cleanup this week, a weekly reindex next month. With Linear connected under Settings → Connections in the console, add an automation to the Project with a linear trigger on linear.issue.labeled and the label agent. Its saved prompt asks for what this one asks for, without the specifics: read the issue, add the job in the style of the tasks module, schedule it with the worker's own scheduler, wire the flag into every process manager, test it, and open a pull request with the worker log. The issue text arrives after the prompt, so the job's name and timing come from the issue itself.

The trigger fires only when someone in your Rush org adds the label, and a change made by an app integration never fires it. A burst of labels is held to the automation's max_runs_per_hour. The run's pull request links the issue, so the person who asked sees the worker log before anyone merges.

The use case this job follows, with its automation and trigger: Turn a Linear label into a PR.

Why run it on Rush?

A scheduled job has three failure modes that a diff cannot show: the worker runs without the scheduler flag, the first run is never enqueued, or the job runs once and forgets to schedule the next. The agent starts a Redis and a worker in its own sandbox and watches the log until the second run is enqueued, and the record keeps that log beside the diff.

The same body runs against every Flask service that uses RQ, each in its own sandbox with its own Redis and its own pull request, and the result is the same shape in each: a job, a command, a flag in the process managers, and the log that shows the clock working.

Frequently asked questions

Why not rq-scheduler and a cron string?

Because rq==1.15.1 already has a scheduler in the worker, and a job that re-enqueues itself with enqueue_in needs no second process and no new package. If the issue asks for a calendar expression such as the first of the month, the pull request says so and the agent adds rq-scheduler with its cron call instead.

What if two workers run with --with-scheduler?

RQ's scheduler takes a lock in Redis so only one of them moves scheduled jobs onto the queue at a time, and the schedule command checks the registry before enqueueing, so starting several workers or redeploying does not duplicate the job.

Does the job interfere with export_posts?

No. It runs on the same microblog-tasks queue and the same worker, one job at a time, and it writes no Task row and no progress notification because it belongs to no user. A long export delays it by the length of the export, which the pull request notes as the reason to add a second queue if the schedule ever needs to be exact.

How do I see that it is running in production?

The worker log prints each run, and the queue's ScheduledJobRegistry lists the next one with its time, readable in a flask shell from app.task_queue.scheduled_job_registry. The agent adds both checks to the pull request description so an operator can confirm the schedule without reading the code.

More RQ jobs

The same job on other stacks

More jobs like this: Turn a Linear label into a PR

Keep reading