What the agent does
The agent clones the repository into its own cloud sandbox and starts from the pattern you describe: queue depth past 20,000 every night at 02:00, draining only by 09:00, with worker memory climbing. A backlog that starts at the same time every night is almost always the scheduler, so the agent reads what is scheduled first, then what each scheduled task dispatches, then how the workers that drain it are configured.
In an app like koel/koel it reads routes/console.php, where six Schedule::job(new RunCommandJob(...))->daily() entries run koel:scan, koel:prune, koel:podcasts:sync, the two clean-up commands and model:prune. config/app.php sets the timezone to UTC, so daily() fires at midnight UTC, and the agent works out whether that matches a 02:00 alert in your Grafana timezone. It reads app/Jobs/RunCommandJob.php: the job sets $timeout = 0, wraps itself in WithoutOverlapping with dontRelease() and a 24-hour lock, and runs the command with Artisan::call inside the queue worker. A full library scan therefore runs inside one long-lived worker process, which explains memory rising in that worker.
Then it follows what the scan dispatches per song. app/Services/Transcoding/TranscodeOnScan.php dispatches a TranscodeSongJob at line 45 for each song that needs transcoding when TRANSCODE_ON_SCAN is on, and app/Services/AudioAnalysis/AnalyzeAudioOnScan.php dispatches an AnalyzeSongAudioJob at line 36 when ANALYZE_AUDIO_ON_SCAN is on. app/Observers/AlbumObserver.php dispatches thumbnail and small-cover jobs at lines 27 and 31 whenever an album with a cover is saved. Each transcode job may run up to TRANSCODE_TIMEOUT (300 seconds by default) plus a 30-second margin, and each analysis job up to 330 seconds. The agent counts how many jobs one scan can enqueue and how long they take with the worker count you run.
It reads config/queue.php, where the database connection is the default and retry_after is 360 seconds, and checks it against each job's $timeout, because a job that runs past retry_after is handed to a second worker and done twice. Last, it reads the 30 most recent commits with git log -p for a new scheduled task, a new per-song job, a changed default or a removed WithoutOverlapping, and writes incident.md. It changes no code.
Start it with one API call
POST https://api.prix.dev/o/{org}/sessions with this body:
{
"agent": "claude",
"input": "Incident: Grafana shows the Laravel queue depth climbing past 20,000 jobs every night at 02:00 and not draining until 09:00, with worker memory rising steadily. Read the code, the configuration and the last 30 commits, find the most likely causes, and write incident.md with a timeline, each suspect with file and line evidence, and the check that would confirm or clear it. Change no code.",
"resources": [
{
"url": "https://github.com/your-org/your-app"
}
]
}- Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
- Send the request. POST the body in request.json to https://api.prix.dev/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
- Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.
What comes back
What comes back is incident.md, attached to the session as an artifact. It opens with the nightly timeline as the code defines it: what the scheduler starts and when, in your timezone, and what each of those dispatches. The suspects follow in order, each with the file and line, the estimated number of jobs or seconds it contributes, and the check that confirms it: SELECT queue, count(*) FROM jobs GROUP BY queue or a count by payload->displayName at 03:00, php artisan queue:monitor, the worker's memory per job from your process metrics, or the failed_jobs table for jobs that timed out and were retried.
The report separates what the code proves from what depends on your settings, since a per-song job only exists when its environment flag is on. Every file the agent read and every command it ran are in the session's transcript.
A reviewer checks the counts against the real jobs table before changing anything. Typical fixes are moving the scan's follow-up jobs to their own queue with their own workers, running workers with --max-jobs or --memory so they restart cleanly, or aligning retry_after with the longest job timeout. A follow-up message to the same session can ask the agent to make one of them as a pull request.
Run it on every morning queue depth check
Run it on a schedule: a cron job on your server at 03:00 that reads the queue depth and, when it is above your threshold, sends this request with the number in the prompt. Or point a Grafana alert on queue depth at a handler that does the same. Each run reads the repository as it is that night.
With a webhook URL on the API key, Rush POSTs run.completed when the turn finishes, so the report is waiting for whoever starts work in the morning.
The use case this job follows, with its automation and trigger: Investigate an incident.
Why run it on Rush?
A nightly backlog spans the scheduler, the jobs, observers, queue config and environment flags, and the person who wrote each piece may not be around at 02:00. In a Rush sandbox the agent reads all of it at once, with PHP and Composer available to run php artisan schedule:list or the test suite, without touching your workers or your database.
The result is a report a team can act on: each suspect with its file and line, each count with how it was derived, and the full record of the work in the session's transcript. If several apps share the problem, each gets its own run, up to the key's concurrent run limit.
Frequently asked questions
Does it work with Horizon and Redis queues?
Yes. With Horizon it also reads config/horizon.php for supervisors, queues, maxProcesses and balancing, which often explain why one queue drains slowly while others sit idle. The checks it suggests use Horizon's dashboard and metrics instead of the jobs table.
Why would worker memory keep rising?
A queue worker is one long-lived PHP process, so anything a job leaves in memory stays until the worker restarts. A job that runs a whole Artisan command, as RunCommandJob does, is the usual culprit. The report points at the job and suggests --max-jobs or --memory on queue:work as the check and the mitigation.
Can it look at our failed_jobs table or logs?
Not on its own, since it works from the repository. Attach an export of failed_jobs or a log file to the request's input and the agent ties each failure to the job class and line.
Will it change the schedule or the jobs?
No. The prompt says to change no code, and the result is a Markdown report. Ask for a fix in a follow-up once you have confirmed the cause.