# Review Pull Requests on a Python Library

Yes. An agent checks out the pull request, runs the library's own check, test and coverage scripts in its sandbox, and posts one review that cites file and line.

[Run this on Rush](https://cloud.getrush.ai/run/review-pull-requests/python/), in a Project that has your repository.

## What the agent does

The agent clones the repository into its own cloud sandbox at the pull request's branch and reads how the project checks itself before it reads the diff. In a Python library like encode/httpx, `pyproject.toml` builds with `hatchling`, requires Python 3.9 or later, depends on `certifi`, `httpcore==1.*`, `anyio` and `idna`, and declares the extras `brotli`, `cli`, `http2`, `socks` and `zstd`. There is no tox file and no Makefile: `scripts/install` creates a `venv` and installs `requirements.txt`, which pins the tooling (`ruff==0.12.11`, `mypy==1.17.1`, `pytest==8.4.1`, `coverage[toml]==7.10.6`, `trio==0.31.0`, `uvicorn==0.35.0`) and installs the package itself with every extra. The agent runs that script rather than guessing at a `pip install`.

Then it runs the same three scripts `.github/workflows/test-suite.yml` runs on every pull request to `master`: `scripts/check` (`ruff format --diff`, `mypy httpx tests`, `ruff check`), `scripts/test` (`coverage run -m pytest`) and `scripts/coverage`, which fails under 100 percent with `--show-missing`. `[tool.mypy]` is `strict = true` with `disallow_untyped_defs = false` only for `tests.*`, and `[tool.pytest.ini_options]` turns every warning into an error with `filterwarnings = ["error", ...]`, so a new `DeprecationWarning` from a dependency fails the suite even when no assertion does. The tests start servers on ports 8000 and 8001, as `.github/CONTRIBUTING.md` warns, which is one more reason to run them in a sandbox that nothing else is using.

With the commands' output in hand the agent reads the diff against the files it touches. A change under `httpx/_client.py` is read with both `tests/client/test_client.py` and `tests/client/test_async_client.py`, because `Client` and `AsyncClient` mirror each other; a change to `httpx/_models.py` is read with `tests/models/`. It checks what the repository's own `PULL_REQUEST_TEMPLATE.md` asks for: a test for each change, a documentation update under `docs/` where the public interface moved, and a line under `## [UNRELEASED]` in `CHANGELOG.md`, which follows Keep a Changelog. It flags a new public name missing from `tests/test_exported_members.py`, a parameter added to the sync client but not the async one, a `# type: ignore` that silences a real strict-mode error, and a test that passes only with the `network` marker.

The agent posts one review on the pull request. Each finding names the file and line, says whether it blocks or is a suggestion, and quotes the command that showed it: the `mypy` line, the `coverage report` line listing the uncovered statements, or the failing test's name. It does not push commits to the author's branch. Where a finding is a judgement rather than a failure, such as a transport change that may alter `Timeout` behaviour, it says which test it ran in the sandbox to check and what came back.

## Start it with one API call

`POST https://api.prix.dev/o/{org}/sessions`

```json
{
  "agent": "claude",
  "input": "Review this pull request on a Python library. Check out the branch, run scripts/install, then scripts/check, scripts/test and scripts/coverage exactly as .github/workflows/test-suite.yml does, and read pyproject.toml for the ruff, mypy and pytest settings (mypy is strict; pytest turns warnings into errors). Read the diff together with the tests for each module it touches: httpx/_client.py with tests/client/, httpx/_models.py with tests/models/. Look for a change made to Client but not AsyncClient or the reverse, a new public name missing from tests/test_exported_members.py, a type: ignore that hides a real error, statements the coverage report lists as missing, a warning the suite now raises, and a change to the public interface with no docs/ update or no line under [UNRELEASED] in CHANGELOG.md, as .github/PULL_REQUEST_TEMPLATE.md asks. Post one GitHub review that quotes the check, test and coverage output, separates blocking issues from suggestions, and cites file and line for every finding. Do not push commits.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app/pull/42"
    }
  ]
}
```

1. **Get an API key.** In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
2. **Send the request.** POST the body in request.json to https://api.prix.dev/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
3. **Read what came back.** Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

## What comes back

What comes back is one review on the pull request, approved or requesting changes, whose body opens with the three script results from the sandbox: whether `scripts/check` passed, how many tests `scripts/test` ran and what failed, and the `coverage report` summary with any statements listed as missing. Below that, each finding is a file, a line, a sentence on what is wrong and a sentence on what to do, with blocking items before suggestions.

The author's branch is untouched: the agent pushes nothing, so the review reads like a colleague's rather than a bot's rewrite. The session at `GET /o/{org}/sessions/{id}` names the repository and the pull request, and its transcript keeps every command with its output, so a maintainer who doubts a finding can read the exact `mypy` or `pytest` line behind it rather than rerun the suite.

Before acting on the review, a maintainer checks that a blocking item really is one: a 100 percent coverage gate means an uncovered branch blocks here where it would be a nit elsewhere, and a warning-as-error failure may belong to a dependency rather than the diff. The review says which it thinks it is; the maintainer decides.

## Run it on every pull request opened

In the console, add an automation to the Project with this prompt and a `github` trigger on `pull_request.opened` and `pull_request.synchronize`, with `branch_patterns` set to the default branch (`master` in httpx). Each push to a pull request starts a run with that pull request's branch checked out and the event after the prompt, in its own sandbox with its own `venv`, so two reviews never share ports 8000 and 8001. Mark the automation read only so its runs comment and never push.

Only pull requests opened by an owner, member or collaborator fire the trigger; pull requests from forks never do, so a first-time contributor's branch is reviewed by a person first. `max_runs_per_hour` bounds a busy afternoon: a fire past the cap is skipped as `rate_limited` and the pull request is reviewed on its next push.

## Why run it on Rush?

A review of a library like this one stands on three commands, and running them locally means a fresh virtual environment, two free ports and several minutes of `pytest` under `coverage`. The agent does that in its own sandbox, from the project's own `scripts/`, and keeps the output, so a comment about a missing branch cites the `coverage report` line rather than a hunch.

One sandbox per pull request means a dependabot morning with a dozen grouped bumps is reviewed in parallel, and a finding about a strict `mypy` error on Python 3.9 is the same one the matrix in `test-suite.yml` would report later. Nothing is pushed to the author's branch, and every command the review rests on is in the transcript.

## Frequently asked questions

### Does the agent run the full matrix of Python versions?

No. The sandbox runs one interpreter, the one `scripts/install` finds as `python3`, and the review says which. `test-suite.yml` runs 3.9 through 3.13 on GitHub when the pull request is pushed, so a version-specific failure shows up there; the agent's job is the reading and the three scripts, not replacing CI.

### Will it post a review on every push?

With `pull_request.synchronize` on the trigger, yes, one per push, each in a fresh sandbox. If that is too much, keep only `pull_request.opened` and start a rerun by hand from the Project when the author asks, or send a follow-up to the first run's session so it re-reads the new commits where it left off.

### What if the coverage gate fails because of code the pull request did not touch?

`scripts/coverage` fails under 100 percent whatever the cause, so the agent reads the `--show-missing` lines and says whether the uncovered statements are in the diff. If they are not, it reports the failure as pre-existing rather than blocking, and the maintainer decides whether the pull request should fix it.

### Can it change the branch to fix what it finds?

Not from this prompt, and not from an automation marked read only. A maintainer who wants the fix sends a follow-up on the same session asking for a commit, or starts a separate run with the pull request as the resource and a prompt that allows pushing.