What the agent does
The agent reads the failure before the code. With the workflow_run event in its prompt, it runs gh run view <run id> --log-failed and pulls out the failing step. For a Flask app like miguelgrinberg/microblog, the test command is python tests.py: a single unittest module at the repository root that ends with unittest.main(verbosity=2), so the log names each test method with ok, FAIL or ERROR, followed by a traceback. The agent quotes that traceback, and tells an assertion failure (FAIL, the code returns the wrong thing) apart from an error (ERROR, the code raised), because they lead to different places.
Then it builds the environment the workflow builds. microblog pins its dependencies exactly in requirements.txt: Flask==3.0.0, Flask-SQLAlchemy==3.1.1, SQLAlchemy==2.0.23, alembic==1.12.1, Flask-Migrate==4.0.5, elasticsearch==8.11.0, redis==5.0.1 and rq==1.15.1. The agent installs those into a fresh virtualenv with the Python version the workflow's setup-python step names, so a failure caused by a dependency the branch bumped shows up the same way in the sandbox as on the runner.
Next it reproduces the failing test alone. tests.py defines a TestConfig that sets TESTING = True, SQLALCHEMY_DATABASE_URI = 'sqlite://' and ELASTICSEARCH_URL = None, and UserModelCase builds the app with create_app(TestConfig), pushes an app context and calls db.create_all() in setUp, then drops everything in tearDown. So the agent can run python -m unittest tests.UserModelCase.test_follow_posts with no Postgres, Redis or Elasticsearch: create_app creates the Redis.from_url client and the microblog-tasks queue but never connects during these tests. From the traceback it reads into app/models.py (avatar, follow, is_following, following_posts) or the blueprints under app/auth, app/main and app/api, and fixes the code the failing commit changed.
When a commit changes a model, the agent also checks migrations/versions/ for a matching Alembic revision, since the test's db.create_all() passes without one while the deployed database would not. After the fix it reruns python tests.py in full, so every test method passes, not only the one that failed.
Start it with one API call
POST https://api.prix.dev/o/{org}/sessions with this body:
{
"agent": "claude",
"input": "The GitHub Actions workflow on this branch failed; the workflow_run event is at the end of this prompt. Read the failed job's log with gh run view <run id> --log-failed and quote the failing test and its traceback. Install the pinned requirements into a fresh virtualenv with the Python version the workflow uses. Reproduce the failure with python tests.py, running the failing test method alone first with python -m unittest. Fix the cause in the application code (change a test only if the test is wrong, and say so), check that any model change has a matching migration, rerun python tests.py until every test passes, and open a pull request against the failing branch that quotes the error it fixed.\n\nEvent: <workflow_run payload>",
"resources": [
{
"url": "https://github.com/your-org/your-app"
}
]
}- Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
- Send the request. POST the body in request.json to https://api.prix.dev/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
- Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.
What comes back
A pull request against the branch that failed, with the fix in the model method or blueprint view the traceback named and, when a model changed, a new Alembic revision under migrations/versions/. The description quotes the FAIL or ERROR block from the GitHub log, lists the commands the agent ran in its sandbox (the single test method, then the full python tests.py), and states plainly if a test was changed and why. The workflow reruns on the pull request, and the trajectory keeps every command with its output.
When the failure comes from outside the application, such as a package that no longer installs on the runner's Python or a step that timed out, the agent reports what it found and opens no pull request. That answer is in the webhook Rush sends when the run ends.
A reviewer checks that the fix restores the behavior the test describes, such as the order of posts following_posts returns, that a changed assertion is genuinely wrong rather than inconvenient, and that any new migration applies to the existing database without data loss.
Run it on every failed test workflow
Register a webhook for GitHub's workflow_run event and send this request when a run completes with conclusion: failure. Set resources to https://github.com/<owner>/<repo>/tree/<head_branch> so the agent clones the failing branch, put the event in input, and exclude branches the agent created so its own failed fixes do not start new runs.
Track each workflow_run.id with the execution_id Rush returns so a redelivered event is ignored. When the run ends, Rush POSTs run.completed or run.failed to the webhook URL on your API key, with the answer and session URL, which your server can turn into a comment on the failing pull request.
The use case this job follows, with its automation and trigger: Fix a bug from an alert.
Why run it on Rush?
A small Flask app's tests run in seconds, but fixing a red build still means switching branches, rebuilding a virtualenv with the exact pins the branch has, and losing your place in other work. The agent does it in its own sandbox with its own clone and virtualenv, so your environment and your checkout stay where you left them, and the fix starts when the build fails.
The result is a pull request with evidence: the quoted traceback, the trajectory of every install and test command with its output, and the workflow rerunning on the pull request. If several branches fail at once, each gets its own run in parallel, and each fix is reviewed by a person before it merges.
Frequently asked questions
My tests use pytest, not unittest. Does that change anything?
Only the command. Put your test command in the prompt, such as pytest -x, and the agent reruns the failing test by node id instead of by TestCase method. pytest also runs unittest classes like microblog's, so either works there.
Does it need Redis or Elasticsearch running?
Not for microblog's tests, which use in-memory SQLite and set ELASTICSEARCH_URL to None. If your workflow starts services for its tests, the agent starts the same ones in its sandbox.
Will it upgrade my pinned dependencies?
No. It installs exactly what requirements.txt pins. If the failure is caused by a pin that no longer installs, it says so and proposes the smallest version change in the pull request, for you to accept or reject.
What does it do with a test that fails only sometimes?
It runs the test several times in the sandbox. If it cannot make the failure happen, it opens no pull request and its answer names the test as possibly flaky, with the log it read.