# Review Pull Requests on a Shopify App

Yes. An agent checks a Shopify app pull request for route authentication, webhook and scope config, API version and Prisma drift, and posts one review.

[Run this on Rush](https://cloud.getrush.ai/run/review-pull-requests/shopify-app/). Opens the console with this prompt ready to send, in a Project that has your repository.

## What the agent does

The agent checks out the pull request in its own sandbox and reads the files that define the app's contract with Shopify, since most review findings on an embedded app are a mismatch between two of them. In an app built from Shopify/shopify-app-template-react-router, `app/shopify.server.ts` configures `shopifyApp` with `ApiVersion.October26`, Prisma session storage and `AppDistribution.AppStore`; `.graphqlrc.ts` names the same API version for code generation; and `shopify.app.toml` declares `scopes = "write_products"`, `api_version = "2026-10"` under both `[webhooks]` and `[events]`, and one subscription per topic whose `uri` matches a flat route file, such as `/webhooks/app/uninstalled` and `app/routes/webhooks.app.uninstalled.tsx`. `package.json` requires Node 22.12 or newer and pins the five React Router packages at exactly `7.18.2`. `.github/workflows/ci.yml` installs with npm, yarn and pnpm on Node 22 and 24 and runs `npx tsc --noEmit`, `npm run lint`, `npx prisma generate && npx prisma validate` and `npm run build`.

It runs those commands on the head, plus two the workflow does not. `npm run typecheck` runs `react-router typegen` before `tsc`, which catches a route whose generated types changed. For the database it applies `prisma/migrations/` to a fresh SQLite file with `prisma migrate deploy` and then asks `prisma migrate diff` whether `prisma/schema.prisma` matches what the migrations built. `prisma validate` passes on a schema edit that has no migration; the diff does not, and `npm run setup`, which production runs as `prisma generate && prisma migrate deploy`, would deploy the app without the new column.

Then it reads the diff against the template's own rules. Every `loader` and `action` under `app/routes/app.*` must call `authenticate.admin(request)`, as `app.tsx`, `app._index.tsx` and `app.additional.tsx` do, and every webhook route must call `authenticate.webhook(request)` and return a `Response`. A new webhook topic belongs in `shopify.app.toml` and not in an `afterAuth` hook, which the README explains runs only after install and when a token expires. A new Admin GraphQL operation is checked against the declared scopes, and a changed API version must move in the server config, the GraphQL config and both TOML sections together. The README's embedded-app rules are applied line by line: a plain `<a>` where `Link` should be used, `redirect` imported from `react-router` where the one returned by `authenticate.admin` is required. Webhook handlers are read for repeat delivery, following the uninstall handler's own comment that a webhook can fire more than once and after the session is gone.

Two repository-specific checks close the review. `.github/dependabot.yml` ignores React Router releases above 7.18.2 and explains at length that a later patch tightened action-origin validation and broke form posts under `shopify app dev`, so a pull request that bumps those packages gets that comment quoted back with a request for the fix it describes. And the pull request template's checklist asks for a `CHANGELOG.md` entry and README updates, which the agent verifies. It posts one review, and states what it could not do: `shopify app dev` needs a signed-in Shopify account and a development store, so the app was built and type-checked but not opened in an admin.

## Start it with one API call

`POST https://api.phoenix.so/o/{org}/sessions`

```json
{
  "agent": "claude",
  "input": "Review the pull request this run was started for, on a Shopify app built with React Router. Read the diff and every file it touches, plus package.json, shopify.app.toml, shopify.web.toml.liquid, app/shopify.server.ts, .graphqlrc.ts, app/routes.ts, every file under app/routes, prisma/schema.prisma and prisma/migrations, .eslintrc.cjs, .github/workflows/ci.yml, .github/dependabot.yml, .github/PULL_REQUEST_TEMPLATE.md and the Gotchas section of the README. In your sandbox install dependencies with npm and run npm run typecheck, npm run lint, npx prisma generate, npx prisma validate and npm run build on the head, then apply the migrations to a fresh SQLite file and use prisma migrate diff to report any difference from the schema. Check that every loader and action under the app routes calls authenticate.admin, every webhook route calls authenticate.webhook and tolerates repeat delivery, new topics are declared in shopify.app.toml with a matching route, new GraphQL operations fit the declared scopes, the API version agrees across the server config, the GraphQL config and the TOML, embedded navigation follows the README rules, and the React Router pin explained in dependabot.yml is respected. Check the template's checklist. Post one review with a verdict, findings citing file and line, and the command output. Push nothing, and say that the app was not run in a Shopify admin.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
```

1. **Get an API key.** In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
2. **Send the request.** POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
3. **Read what came back.** Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

## What comes back

One review on the pull request. It gives a verdict, then findings grouped by kind: authentication, webhooks and scopes, API version, database, navigation, dependencies. Each cites a file and line and says whether it blocks. Under the findings are the results of the type check, lint, Prisma commands and build, the migration diff when there is one, and a line for each checklist item in the pull request template.

Nothing is pushed, and the complete logs remain in the run's record. A maintainer acts on the blocking findings and then does the part a sandbox cannot: runs `shopify app dev` against a development store and clicks through the changed screen inside the admin, where session and iframe problems actually appear.

## Run it on every pull request opened

Create an automation in the app's Project with this prompt and a `github` trigger on `pull_request.opened` and `pull_request.synchronize`, with `read_only` on so its runs can comment and cannot push. The pull request event is added after the prompt, and every push to the branch starts a review of the new head. The trigger's `authors` setting can limit reviews to named GitHub logins. Bot senders never fire it, so Dependabot's weekly pull requests are left for a person to look at.

From your own server, the same review starts with `POST /o/{org}/sessions`: put the pull request URL in `resources` so the clone is its branch, and send a `request_id` that is a UUID v5 of GitHub's delivery id so a redelivery starts nothing new. A `webhook_url` set on the key receives `run.completed` with the `session_url` when the review has been posted.

## Why run it on Rush?

The costly mistakes in a Shopify app are consistent ones: a route that skips authentication, a topic with no handler, a schema change with no migration. They pass a build. The agent runs the repository's commands and the migration replay in its own sandbox, reads the diff against the TOML and the server config, and keeps all of the output, so each finding points at a line and, where a command found it, at that command's message.

Because each review has its own sandbox with its own `node_modules` and SQLite file, reviews of several pull requests run at the same time without sharing state, and a contributor who pushes a fix gets the new head reviewed while the maintainer is elsewhere.

## Frequently asked questions

### Can the agent run the app inside the Shopify admin?

The review does not attempt it. `shopify app dev` signs in to a Shopify account and installs the app on a development store, which the sandbox has no credentials for. The review builds and type-checks the app and says plainly that the embedded behaviour was not exercised.

### How does it know a GraphQL operation needs a new scope?

It reads the operation and the `scopes` line in `shopify.app.toml` and checks the objects and mutations against Shopify's documentation for the pinned API version. Where the documentation is clear it cites it; where it is not, the finding is marked as a question for the author.

### Why replay migrations when CI already runs prisma validate?

`prisma validate` checks that the schema file is well formed. It does not compare the schema with the migration history. Applying the migrations to an empty database and diffing against the schema is what shows a model change that nobody generated a migration for.

### Will it review Dependabot pull requests?

A `github` trigger does not fire for bot senders, so those are not reviewed automatically. A maintainer can start a run for one by hand. The React Router pin and its explanation in `dependabot.yml` are checked on every human pull request that touches `package.json`.

## More Shopify app jobs

- [Write Vitest Tests for a Shopify App's Webhooks and Actions](https://getrush.ai/run/write-vitest-tests/shopify-app/)
- [Add Subscription Billing to a Shopify App](https://getrush.ai/run/add-app-billing/shopify-app/)
- [Add an App Proxy to a Shopify App](https://getrush.ai/run/add-app-proxy/shopify-app/)

## The same job on other stacks

- [Review Pull Requests That Change Sidekiq Jobs in Rails](https://getrush.ai/run/review-pull-requests/sidekiq/)
- [Review Pull Requests Automatically in a Spring Boot App](https://getrush.ai/run/review-pull-requests/spring-boot/)
- [Review Pull Requests in a SQLAlchemy App](https://getrush.ai/run/review-pull-requests/sqlalchemy/)

## Keep reading

- [How a Rush run works on your repository, from request to pull request](https://getrush.ai/cloud/)
- [Compare cloud agents, agent sandboxes and AI code review](https://getrush.ai/compare/)
- [Rush pricing: Free, Pro and Max](https://getrush.ai/pricing/)
