Review Pull Requests on a Python Library

The agent clones the repository into its own cloud sandbox at the pull request's branch and reads how the project checks itself before it reads the diff. In a Python library like encode/httpx, pyproject.toml builds with hatchling, requires Python 3.9 or later, depends on certifi, httpcore==1.*, anyio and idna, and declares the extras brotli, cli, http2, socks and zstd. There is no tox file and no Makefile: scripts/install creates a venv and installs requirements.txt, which pins the tooling (ruff==0.12.11, mypy==1.17.1, pytest==8.4.1, coverage[toml]==7.10.6, trio==0.31.0, uvicorn==0.35.0) and installs the package itself with every extra. The agent runs that script rather than guessing at a pip install.

Then it runs the same three scripts .github/workflows/test-suite.yml runs on every pull request to master: scripts/check (ruff format --diff, mypy httpx tests, ruff check), scripts/test (coverage run -m pytest) and scripts/coverage, which fails under 100 percent with --show-missing. [tool.mypy] is strict = true with disallow_untyped_defs = false only for tests.*, and [tool.pytest.ini_options] turns every warning into an error with filterwarnings = ["error", ...], so a new DeprecationWarning from a dependency fails the suite even when no assertion does. The tests start servers on ports 8000 and 8001, as .github/CONTRIBUTING.md warns, which is one more reason to run them in a sandbox that nothing else is using.

With the commands' output in hand the agent reads the diff against the files it touches. A change under httpx/_client.py is read with both tests/client/test_client.py and tests/client/test_async_client.py, because Client and AsyncClient mirror each other; a change to httpx/_models.py is read with tests/models/. It checks what the repository's own PULL_REQUEST_TEMPLATE.md asks for: a test for each change, a documentation update under docs/ where the public interface moved, and a line under ## [UNRELEASED] in CHANGELOG.md, which follows Keep a Changelog. It flags a new public name missing from tests/test_exported_members.py, a parameter added to the sync client but not the async one, a # type: ignore that silences a real strict-mode error, and a test that passes only with the network marker.

The agent posts one review on the pull request. Each finding names the file and line, says whether it blocks or is a suggestion, and quotes the command that showed it: the mypy line, the coverage report line listing the uncovered statements, or the failing test's name. It does not push commits to the author's branch. Where a finding is a judgement rather than a failure, such as a transport change that may alter Timeout behaviour, it says which test it ran in the sandbox to check and what came back.

What comes back is one review on the pull request, approved or requesting changes, whose body opens with the three script results from the sandbox: whether scripts/check passed, how many tests scripts/test ran and what failed, and the coverage report summary with any statements listed as missing. Below that, each finding is a file, a line, a sentence on what is wrong and a sentence on what to do, with blocking items before suggestions.

The author's branch is untouched: the agent pushes nothing, so the review reads like a colleague's rather than a bot's rewrite. The session at GET /o/{org}/sessions/{id} names the repository and the pull request, and its transcript keeps every command with its output, so a maintainer who doubts a finding can read the exact mypy or pytest line behind it rather than rerun the suite.

Before acting on the review, a maintainer checks that a blocking item really is one: a 100 percent coverage gate means an uncovered branch blocks here where it would be a nit elsewhere, and a warning-as-error failure may belong to a dependency rather than the diff. The review says which it thinks it is; the maintainer decides.

In the console, add an automation to the Project with this prompt and a github trigger on pull_request.opened and pull_request.synchronize, with branch_patterns set to the default branch (master in httpx). Each push to a pull request starts a run with that pull request's branch checked out and the event after the prompt, in its own sandbox with its own venv, so two reviews never share ports 8000 and 8001. Mark the automation read only so its runs comment and never push.

Only pull requests opened by an owner, member or collaborator fire the trigger; pull requests from forks never do, so a first-time contributor's branch is reviewed by a person first. max_runs_per_hour bounds a busy afternoon: a fire past the cap is skipped as rate_limited and the pull request is reviewed on its next push.

A review of a library like this one stands on three commands, and running them locally means a fresh virtual environment, two free ports and several minutes of pytest under coverage. The agent does that in its own sandbox, from the project's own scripts/, and keeps the output, so a comment about a missing branch cites the coverage report line rather than a hunch.

One sandbox per pull request means a dependabot morning with a dozen grouped bumps is reviewed in parallel, and a finding about a strict mypy error on Python 3.9 is the same one the matrix in test-suite.yml would report later. Nothing is pushed to the author's branch, and every command the review rests on is in the transcript.

Does the agent run the full matrix of Python versions?

Will it post a review on every push?

What if the coverage gate fails because of code the pull request did not touch?

Can it change the branch to fix what it finds?