# Add Visual Regression Tests With Playwright

Yes. An agent adds toHaveScreenshot tests for your app's main screens, masks the parts that change per run, records Linux baselines for CI, and opens a PR.

[Run this on Rush](https://cloud.getrush.ai/run/add-visual-regression-tests/playwright/). Opens the console with this prompt ready to send, in a Project that has your repository.

## What the agent does

The agent reads the existing end-to-end setup before adding screenshots to it. In a React app laid out like alan2207/bulletproof-react, `apps/react-vite/playwright.config.ts` has `testDir: './e2e'`, a `setup` project matching `*.setup.ts`, and a `chromium` project with `storageState: 'e2e/.auth/user.json'` that depends on it; `retries` is 2 and `workers` is 1 on CI, the reporter is `html`, and `webServer` runs `yarn dev --port 3000` with `reuseExistingServer` off on CI. `e2e/tests/auth.setup.ts` registers a user generated by `@ngneat/falso` through `src/testing/data-generators.ts`, logs out, logs in and saves the storage state. `smoke.spec.ts` and `profile.spec.ts` drive discussions and the profile page. The `test-e2e` script starts the MSW mock server with `pm2` from `mock-server.ts`, `.env.example-e2e` points the app at it on port 8080, and `.github/workflows/react-vite-ci.yml` installs browsers with `yarn playwright install --with-deps` and uploads `playwright-report/` and `mocked-db.json`.

Two facts shape the tests. The user is random on every run, so any screen that shows the name or initials would differ each time, and the mock database starts empty, so the discussions list is an empty state unless the test creates one. The agent writes `e2e/tests/visual.spec.ts` with one `test` per screen: the landing page at `/`, the login and register pages signed out, and the dashboard, the empty discussions list, the create-discussion drawer and the profile page signed in. Each calls `expect(page).toHaveScreenshot()` with `fullPage: true` and a `mask` for the user menu button that `profile.spec.ts` already locates by role, and the signed-out tests use a fresh context so the stored state does not apply.

In `playwright.config.ts` the agent adds `expect.toHaveScreenshot` defaults of `animations: 'disabled'` and a small `maxDiffPixelRatio`, so a sub-pixel antialiasing difference is not a failure and a moved button is. Baselines are platform-specific by design, since the default snapshot path carries the platform suffix, so the agent records them in its Linux sandbox with `yarn playwright test visual --update-snapshots`, which is what `ubuntu-latest` in the workflow will compare against. The `.png` files are committed under `e2e/tests/visual.spec.ts-snapshots/`.

The CI job's `upload-artifact` step gains `test-results/` beside `playwright-report/`, because that is where Playwright writes the expected, actual and diff images when a screenshot test fails. The agent runs the whole suite twice in its sandbox, once to record and once to confirm the recorded baselines match, and runs `yarn lint`, since `e2e/.eslintrc.cjs` applies the Playwright plugin's rules to the new spec.

## Start it with one API call

`POST https://api.phoenix.so/o/{org}/sessions`

```json
{
  "agent": "claude",
  "input": "Add visual regression tests to the Playwright suite in apps/react-vite. Read playwright.config.ts, e2e/tests/auth.setup.ts, e2e/tests/smoke.spec.ts, e2e/tests/profile.spec.ts, e2e/.eslintrc.cjs, .env.example-e2e, mock-server.ts, package.json, .gitignore and .github/workflows/react-vite-ci.yml first. Write e2e/tests/visual.spec.ts with full-page toHaveScreenshot tests for the landing page, the login and register pages in a fresh signed-out context, and the dashboard, empty discussions list, create-discussion drawer and profile page signed in, masking the user menu button. Set animations disabled and a small maxDiffPixelRatio under expect.toHaveScreenshot in the config. Start the mock server as the test-e2e script does, record the baselines with --update-snapshots on Linux, run the suite again to confirm they match, commit the snapshot images, add test-results to the CI artifact upload, run yarn lint, and open a pull request quoting the two runs.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
```

1. **Get an API key.** In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
2. **Send the request.** POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
3. **Read what came back.** Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

## What comes back

What comes back is a pull request that adds `visual.spec.ts` and its snapshot directory, changes `playwright.config.ts` and the workflow's artifact path, and nothing in `src/`. The description lists each screen and its baseline image, quotes the recording run and the confirming run, and says which element is masked and why.

The run's record keeps both Playwright runs, so a reviewer can see that the second run compared seven baselines and passed rather than assume the recording was stable.

Before merging, a reviewer looks at the committed images once, since they are the contract, and decides whether `maxDiffPixelRatio` should be tighter for the small screens and whether the screenshots should also be taken at a mobile viewport, which is a second project in the config.

## Run it on every pull request

Once the baselines are in, CI compares every push against them in the `e2e` job as it stands. For the review itself, an automation in the Project with a `github` trigger on `pull_request.opened` and `pull_request.synchronize` runs a prompt that checks out the pull request's branch, runs `yarn playwright test visual`, and when a screenshot differs posts a review comment that names the screen and attaches the diff image from `test-results/` as the run's artifact, so the reviewer sees what moved without opening the job. The trigger adds the event after the prompt.

A team with several Vite apps starts one run per repository with one `POST /o/{org}/sessions` each, this prompt as `input` and the repository in `resources`, with a `request_id` that is a UUID v5 of the repository's full name so a retry returns the run it already started. Each run reads its own setup project and data generators, so an app whose dashboard shows a date gets that date masked.

## Why run it on Rush?

Screenshot baselines recorded on a laptop do not match a Linux runner, and tests written without masking fail on the second run because the generated user changed. Both mistakes are found only by running the suite twice in the environment CI uses. The agent records in its own Linux sandbox, confirms, and keeps both outputs.

The pull request is one spec, a config block and a folder of images, so review is looking at seven screens and one ratio rather than checking out the branch and running a browser. Across several apps the job runs in parallel, one sandbox and one pull request each, with baselines recorded for that app's screens.

## Frequently asked questions

### Why are the baselines recorded on Linux?

Because the workflow runs on `ubuntu-latest`, and Playwright names each snapshot with the platform it was taken on. A baseline recorded on macOS would never be compared in CI; one recorded in the agent's Linux sandbox is the one the job uses.

### What happens when a screen changes on purpose?

The test fails with the expected, actual and diff images in `test-results/`, which the workflow now uploads. Running `yarn playwright test visual --update-snapshots` on Linux rewrites the baseline, and the new image in the pull request is the reviewer's evidence of the change.

### Why mask the user menu rather than fix the data?

Because `auth.setup.ts` registers a random user from `@ngneat/falso` on every run and the rest of the suite depends on that. A mask over the one element that shows the name keeps the screenshot stable without changing how the suite signs in.

### Does this slow the e2e job down?

By the time of seven page loads and screenshots on one worker. The suite already runs with `workers: 1` on CI, so the screenshot tests queue behind the smoke and profile tests; they share the same dev server and mock API.

## More Playwright jobs

- [Fix Flaky Playwright Tests](https://getrush.ai/run/fix-flaky-tests/playwright/)
- [Review Pull Requests in a Playwright Test Suite](https://getrush.ai/run/review-pull-requests/playwright/)
- [Upgrade Playwright and Its Browsers in a Test Suite](https://getrush.ai/run/upgrade-playwright/playwright/)

## The same job on other stacks

- [Add Visual Regression Tests to Storybook Stories](https://getrush.ai/run/add-visual-regression-tests/storybook/)

## More jobs like this: Write the missing tests

- [Migrate a Cypress Test Suite to Playwright](https://getrush.ai/run/migrate-cypress-to-playwright/cypress/)
- [Migrate Cypress Tests to Playwright in a Node.js App](https://getrush.ai/run/migrate-cypress-to-playwright/nodejs/)

## Keep reading

- [How a Rush run works on your repository, from request to pull request](https://getrush.ai/cloud/)
- [Compare cloud agents, agent sandboxes and AI code review](https://getrush.ai/compare/)
- [Rush pricing: Free, Pro and Max](https://getrush.ai/pricing/)
