What the agent does
The agent reads the repository's own definition of green before it reads the diff. In an app like cypress-io/cypress-realworld-app, package.json pins react 18.2.0 and react-dom 18.2.0, vite ^8.2.2, typescript 5.8.3, xstate 4.38.3 with @xstate/react, formik 2.4.6 with yup, and Cypress ^16.1.1 with vitest ^3.2.4; .node-version says 22.20.0 and packageManager is yarn@1.22.22, so yarn.lock is the lockfile. The install job in .github/workflows/main.yml runs yarn types, yarn lint, yarn test:unit:ci and yarn build:ci inside the cypress/browsers:22.20.0 image, and lint is eslint plus prettier --check with the 100 column width from .prettierrc. Those four commands are what the agent runs on the head commit first.
Then it works out which test layer the diff touches, because the layers run differently. Unit tests in src/__tests__/ and src/utils/__tests__/ run under Vitest with fileParallelism: false in vite.config.ts, since they share data/database.json through lowdb and reseed it with seedDatabase() in beforeEach. Component specs sit next to components as src/**/*.cy.tsx, such as SignInForm.cy.tsx and TransactionDateRangeFilter.cy.tsx, and run with yarn cypress:run:component. End-to-end specs live in cypress/tests/ui/ and cypress/tests/api/ and need the full stack: yarn start:ci serves the built app through scripts/testServer.ts on port 3000 and the Express API from backend/app.ts on 3001, with NODE_ENV=test so the /testData routes that cy.task('db:seed') posts to are mounted. The agent builds, starts both, and runs only the specs whose feature the diff changes, for example cypress/tests/ui/new-transaction.spec.ts for a change under src/components/TransactionCreate*.tsx or backend/transaction-routes.ts.
The review reads the diff against the patterns the app relies on. Screens are driven by XState machines in src/machines/, so a component that calls axios directly instead of sending an event to createTransactionMachine or transactionFiltersMachine is a finding. Backend routes in backend/ chain ensureAuthenticated and validateMiddleware([...]) from backend/helpers.ts with validators from backend/validators.ts, and validateMiddleware answers 422 with the error list; a new route or a new body field with no validator is blocking. Money is integer cents: createTransaction in backend/database.ts stores amount * 100 and the UI formats with dinero.js, so a change that compares a dollar string to a stored amount is wrong even when the spec still passes. Selectors in specs use data-test attributes through cy.getBySel, so a renamed attribute without a spec change breaks the suite on GitHub later.
The review posts once, with file and line for every finding, blocking issues separated from suggestions, and the exact results of yarn types, yarn lint, yarn test:unit:ci and each Cypress spec it ran, with the browser and viewport. It does not push a commit, does not raise retries.runMode in cypress.config.ts, and when a spec is one of the ones gated on auth0_configured or okta_configured in cypress/tests/ui-auth-providers/, it says that those cannot run without credentials rather than reporting them as passed.
Start it with one API call
POST https://api.prix.dev/o/{org}/sessions with this body:
{
"agent": "claude",
"input": "Review this pull request on a React 18 and Vite app with an Express API in backend/, XState machines in src/machines/, Vitest unit tests in src/__tests__/ and Cypress specs under cypress/tests/. Check it out with Node 22.20.0 from .node-version and yarn install --frozen-lockfile. Run yarn types, yarn lint and yarn test:unit:ci. Build with yarn build:ci, start the stack with yarn start:ci, wait for ports 3000 and 3001, and run only the specs in cypress/tests/ui or cypress/tests/api whose feature the diff touches, with yarn cypress:run --spec <file>. Read the diff with the machine in src/machines/ and the route file in backend/ it concerns. Look for components that call the API outside their machine, backend routes or body fields without a validator in backend/validators.ts, amounts that are not integer cents, and changed data-test attributes with no matching spec change. Post one GitHub review that quotes every result including the browser each spec ran in, separates blocking issues from suggestions, and cites file and line for every finding. Do not push commits and do not change retries in cypress.config.ts.",
"resources": [
{
"url": "https://github.com/your-org/your-app/pull/42"
}
]
}- Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
- Send the request. POST the body in request.json to https://api.prix.dev/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
- Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.
What comes back
What comes back is one review on the pull request and the full command record on the session. The review opens with the four CI commands and their results, then each Cypress spec it ran with the browser, the number of tests in that spec and any failure quoted with its assertion. Specs it did not run are listed as not run, including the auth-provider specs that need credentials. Findings follow, each under a file and line.
A reviewer checks that the specs the agent chose cover the diff: a change to backend/transaction-routes.ts should have run cypress/tests/api/api-transactions.spec.ts as well as the UI spec. They check that a Vitest failure was not caused by the shared data/database.json, which fileParallelism: false exists to protect, and that no finding asks the author to add a fixed cy.wait in milliseconds. The pull request's own push to GitHub then runs the full matrix in main.yml, five parallel containers each for Chrome, Firefox and the two mobile viewports.
The transcript holds every tool_use and tool_result, so the yarn start:ci log and each cypress run summary can be read later. GET /o/{org}/sessions/{id} returns the review text as result, and a follow-up can ask the parked sandbox to run one more spec against the same build.
Run it on every pull request opened or updated
Add an automation to the Project for this repository with this prompt and a github trigger on pull_request.opened and pull_request.synchronize, with the branches you merge to in branch_patterns; this repository's default branch is develop. The trigger appends the pull request after the prompt, so each pull request and each push to it gets its own sandbox, its own build and its own browser. Pull requests from forks never fire it, and neither do ones from outside the owner, member and collaborator roles.
Set read_only on the automation so its runs comment and never push. The repository's renovate.json opens dependency pull requests on their own schedule, and main.yml already skips renovate/** branches on push; max_runs_per_hour on the automation keeps a batch of those from starting more reviews than you want.
The use case this job follows, with its automation and trigger: Review every pull request.
Why run it on Rush?
A real review of a React pull request in this app means a build, two servers, a seeded JSON database and a browser running the Cypress spec for the feature that changed, with a console open to catch the XState transition that did not fire. That is a laptop's worth of ports and state for every pull request. A Rush run does it in its own sandbox with a computer and a browser, keeps every command and the spec output, and parks when the turn ends.
The review is grounded because it quotes what ran: the type check, the lint, Vitest and the named spec with its browser. Findings point at the machine, the validator or the cents arithmetic, with the line. Several pull requests are reviewed at once, each with its own build, and the author gets one review to answer before the full Chrome and Firefox matrix on GitHub finishes.
Frequently asked questions
Does it run the whole Cypress suite?
No. It runs the specs whose feature the diff touches, named in the review, and the full matrix in main.yml runs on GitHub when the pull request's checks start. Running everything in one sandbox would take longer than the review is worth and the result would duplicate CI.
Why does it start the built app and not `yarn dev`?
Because CI does. yarn start:ci serves the build/ folder through scripts/testServer.ts with NODE_ENV=test, which mounts the /testData routes that cy.task('db:seed') needs. yarn dev runs Vite with NODE_ENV=development, a different configuration from the one the pull request is judged on.
What about the Auth0, Okta, Cognito and Google specs?
They live in cypress/tests/ui-auth-providers/ and are gated on credentials in cypress.config.ts. The sandbox has none, so the review lists them as not run. The default yarn start:ci stack signs in through the local Express session, which is what the ui/ specs use.
Will it flag a flaky spec as a bug in the pull request?
It reruns a failing spec once with the same build and says whether the failure repeated. cypress.config.ts sets retries.runMode: 2, so a spec that passes on retry is reported as such rather than hidden. A finding about flakiness points at the intercept or seed timing, not at the author's change, unless the diff introduced it.
Can the review cover the backend only?
Yes. A pull request that changes only files under backend/ gets yarn types, yarn lint, the Vitest files in src/__tests__/ that call backend/database.ts directly, and the matching cypress/tests/api/ spec, which needs the API on port 3001 but exercises no React code.