All jobs

Review Alembic Migrations in Every Pull Request

An agent applied an Alembic revision while a reader held user_account: its DROP COLUMN queued and blocked new reads, and the downgrade restored no avatars.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

The pull request, litestar-org/litestar-fullstack #249, adds profile avatar upload. Its revision 2026-04-08_replace_avatar_url_with_avatar_fi_75cc19aa927c.py drops user_account.avatar_url and adds an avatar JSONB column. The agent reads alembic.ini, env.py, src/py/app/lib/settings.py and conftest.py first, and finds no lock_timeout in the revision or in env.py.

With no Docker in the sandbox, it ran Postgres 16.2 from the pgserver pip package, applied the base head dc54b25d8b6c, seeded 1000 users (500 with an avatar_url), and applied the revision while a second session held a SELECT, then an UPDATE, open on user_account. Then it ran make-migrations, downgrade -1 and upgrade head.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "claude",
  "input": "Review the Alembic revision added by the open pull request in resources. Read src/py/app/db/migrations/alembic.ini, env.py and every file under versions, src/py/app/lib/settings.py, the models under src/py/app/db/models, src/py/tests/conftest.py, .github/workflows/ci.yaml and tools/deploy/docker/docker-compose.yml first. Start Postgres in your sandbox with a fresh database and synthetic data only, never a production dump: run uv sync --all-groups, apply the base branch with uv run app database upgrade, insert rows into every table the new revision touches, then apply the revision while another session holds an open transaction on each of those tables and record which statements wait. Confirm the revision's down_revision is the current head with alembic heads and history. Run uv run app database make-migrations into a scratch revision, report whether autogenerate found anything the pull request missed, and delete the scratch file. Run alembic downgrade -1 and upgrade head and report what the downgrade drops. Comment on the pull request at the exact lines of the revision: the operation, what happened with data and under a concurrent transaction, the safe alternative, and what the downgrade loses. Change no file and push nothing.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app/pull/42"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

Tried on litestar-org/litestar-fullstack at 81b3af9: pull request #249, a drafted review with comments on four line ranges of the revision (34-36, 38-41, 43-48, 50-51), not posted, and no code change. The migration waited about 10.3 s behind the held transaction, pg_locks showed its AccessExclusiveLock | f while it waited, and a new SELECT count(*) on user_account timed out after 2 s. Autogenerate found nothing missing, and after the downgrade every row's avatar_url was empty (1000 | 0). The repository's pytest suite already fails on the unchanged PR checkout without Docker (11 failed, 223 passed, 360 errors in 52.03s).

For the reviewer: whether losing every existing avatar_url, OAuth-provided ones included, is acceptable, and whether to split the change into expand and contract releases run with SET LOCAL lock_timeout = '3s', as the review suggests. The timings come from a 1000-row table.

Run it on every pull request labelled migration

An automation with this prompt and a github trigger on pull_request.labeled, labels set to migration and branch_patterns set to main, starts a review when someone with write access labels a pull request that adds a revision; triggers cannot filter on paths. A pull request from a fork, like #249, never fires a trigger.

The use case this job follows, with its automation and trigger: Review every pull request.

Why run it on Rush?

The revision reads as two one-line column changes. On a live table, the DROP COLUMN queued behind an ordinary reader and, while queued, stopped every new read of user_account, logins included. And the data_downgrades docstring, "No data migration needed", is wrong: rolling back brings no avatar back in either direction.

Frequently asked questions

Why didn't the run use the repository's tests?

pytest_databases needs the docker binary, which the sandbox lacks: the untouched checkout ended 11 failed, 223 passed, 360 errors. The lock experiment used the postgresql+asyncpg URL conftest.py uses, because the psycopg URL crashed in SQLAlchemy's version parse.

How did it run alembic downgrade -1?

Through the app's CLI as uv run app database downgrade --no-prompt -- -1, because the bare -1 is rejected as an option.

More Alembic jobs

The same job on other stacks

Keep reading