What the agent does
A Python CLI's test workflow can fail with no commit behind the failure, and the first thing the agent works out is whether this is one of those. In a Typer project like fastapi/fastapi-cli, .github/workflows/test.yml runs on pushes to main, on pull requests and on a cron of 0 0 * * 1. Its matrix is Python 3.14 on Ubuntu, Windows and macOS with uv-resolution: highest, plus five more cells, two of which, 3.10 on Ubuntu and 3.12 on macOS, use lowest-direct. Each cell exports UV_PYTHON and UV_RESOLUTION and installs with uv sync --no-dev --group tests --extra standard, with no --locked. A highest cell still installs what uv.lock holds, because the lock was made in that mode. A lowest-direct cell does not: uv finds the lock out of date for that mode and resolves again, taking the floors in pyproject.toml for direct dependencies (typer >= 0.26.1, rich-toolkit >= 0.14.8) and the newest release of everything they pull in. That cell can pair the oldest allowed Typer with a Rich newer than the one in the lock, so it can turn red on the Monday cron with no commit behind it. A later job combines coverage from all cells and runs coverage report --fail-under=100.
The agent reads the failed job's log with gh run view <run id> --log-failed, names the cell, and rebuilds it. For an Ubuntu cell that is three commands: set UV_PYTHON and UV_RESOLUTION to the cell's values, run the same uv sync, then uv run bash scripts/test.sh, which is coverage run -m pytest tests. It compares uv pip list in that environment with uv.lock, where Typer is 0.27.3, Rich 14.2.0 and rich-toolkit 0.20.6, and the packages that differ are the suspects. If the same commit passed last week, a package that moved since then is the cause.
In a Typer CLI the suspects fall into a few families. Help output: tests/test_cli.py drives the app through typer.testing.CliRunner and asserts whole sentences of dev --help and run --help, which only works because tests/conftest.py sets typer.rich_utils.MAX_WIDTH to 3000 and FORCE_TERMINAL to false so Rich never wraps a line. A Typer or Rich release that renders help differently, or stops reading those attributes, fails test_dev_help, test_run_help and test_callback_help together. Floors: code that calls something newer than a declared minimum passes on highest and fails on lowest-direct. Private imports: src/fastapi_cli/utils/cli.py imports loop_first from rich._loop and builds on BaseStyle and Element from rich-toolkit, so a release that moves any of them fails every cell at import. Coverage: the 100 percent floor is met only by the cells together, since config.py imports tomllib and falls back to tomli on Python 3.10.
The fix is the smallest change that makes the failed cell pass without loosening a check. For a help-text failure the agent first runs python -m fastapi_cli dev --help at a normal terminal width to see whether users would notice; if the output is fine, the change belongs in the test setup. For a floor failure it raises the minimum in pyproject.toml to the first version that has the API, as the project's release notes show was done for Typer before, and refreshes uv.lock. For a moved private symbol it switches to the public import if one exists and says so if none does. It never lowers --fail-under. Then it reruns the failed cell and the other resolution mode on Linux, plus bash scripts/lint.sh, which is mypy, ty and ruff.
Start it with one API call
POST https://api.phoenix.so/o/{org}/sessions with this body:
{
"agent": "claude",
"input": "The Test workflow for this Typer command-line tool failed; the failed workflow run follows this prompt. Read .github/workflows/test.yml, pyproject.toml, uv.lock, scripts/test.sh, scripts/lint.sh, tests/conftest.py and the source under src/ first. Read the failed job's log with gh run view <run id> --log-failed and name the job, the matrix cell (OS, Python, uv resolution) and the first failing test or command. If the cell ran on Ubuntu, rebuild it in the sandbox: set UV_PYTHON and UV_RESOLUTION to the cell's values, run the same uv sync command the workflow runs, then uv run bash scripts/test.sh. Compare uv pip list with uv.lock and name the packages that differ. If the cell ran on Windows or macOS, reason from the log and say so. Decide the cause: changed help rendering, an API newer than a declared minimum, a moved import, a coverage gap, or a real regression in src/. Make the smallest fix: adjust the test setup when user-visible output is unchanged, raise a minimum version in pyproject.toml when the code needs it and refresh uv.lock, use a public import when one exists. Do not lower the coverage threshold, do not skip or delete a test, and do not cap a dependency unless nothing else works, with the reason. Rerun the failed cell, the other resolution mode on Ubuntu, and bash scripts/lint.sh. Open a pull request that names the failed run and cell, quotes the failing lines, explains the cause in one paragraph, keeps any dependency change in its own commit, and pastes each command with its output.",
"resources": [
{
"url": "https://github.com/your-org/your-app"
}
]
}- Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
- Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
- Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.
What comes back
A pull request that names the failed run and the matrix cell, quotes the lines of its log that decided the diagnosis, and states the cause in a paragraph: which package moved between the lock and the failing environment, or which line in src/ regressed. The diff is small and in one of four places: tests/conftest.py or a test's expectation, a minimum version in pyproject.toml with uv.lock, an import in src/, or a fix to behaviour.
The body pastes the commands the agent ran with their output: the uv sync for the cell, the failing pytest run before the fix, the passing one after, the second resolution mode and the lint script. When the cell was Windows or macOS, it says plainly that the fix was reasoned from the log and that the matrix on the pull request is the rerun.
A reviewer reads any change under tests/ as a claim that users see no difference, and checks it against the --help output pasted in the description. Some repositories guard dependency files: this one has a guard-dependencies.yml workflow that closes a pull request touching pyproject.toml or uv.lock unless an organisation member or Dependabot wrote it, which is why the dependency change sits in its own commit and is named at the top.
Run it on every failed test workflow
An automation in the Project that holds the repository, with this prompt and a github trigger on workflow_run.failure, starts one run per failed workflow run. The trigger puts the run's event after the prompt, which is how the agent gets the run id without listing runs. A scheduled Monday failure and a failure on a push both arrive this way; CI runs from forks never fire the trigger. The automation's max runs per hour caps how many a bad morning can start.
From your own server, a handler for GitHub's workflow_run webhook does the same with more control: on conclusion: failure it sends POST /o/{org}/sessions with this prompt and the run id in input, the repository in resources, and a request_id that is a UUID v5 of the run id, so a redelivered webhook returns the run already started.
The use case this job follows, with its automation and trigger: Fix a bug from an alert.
Why run it on Rush?
Reproducing one matrix cell means building an environment nobody keeps around: a specific Python, a resolution mode that takes the oldest allowed version of every direct dependency and the newest of the rest, and the project's extras. On a laptop that fights with whatever environment is already there. A sandbox starts empty, so the cell is rebuilt exactly, and when three projects fail on the same Monday because one upstream package shipped, three runs work in three sandboxes at once.
The diagnosis is checkable. The package list of the failing environment, the diff against the lock and both test runs are in the run's transcript at GET /o/{org}/sessions/{id}/messages, so the sentence in the pull request that blames a release can be verified against output.
Frequently asked questions
CI failed on a schedule with no new commits. Can it still find the cause?
Yes, and that case is the clearest. The lowest-direct cells resolve their environment fresh on every run, so a scheduled run can pick up a transitive release the last green run did not have. The agent rebuilds the cell, lists what differs from uv.lock, and tests the suspects one at a time until the failure follows a single package.
Will it pin the dependency that broke us?
Only as a last resort and with the reason written down. A cap hides the incompatibility until the next lock update brings it back, and it constrains everyone who installs the tool. The agent prefers to adapt the code or the test setup to the new release, or to raise a floor when the code already needs a newer version.
Can it change the expected help text in a test?
When the rendered help changed because Typer or Rich changed and the content is the same, yes, and it shows the before and after output. When an option's name, default or description changed in src/, that is a behaviour change, and the agent treats the test as correct and the code as the thing to fix or explain.