# Upgrade Playwright and Its Browsers in a Test Suite

Yes. The agent moves @playwright/test, the lockfile and every CI image tag to one version, reruns the suite in each browser and opens a pull request.

[Run this on Rush](https://cloud.getrush.ai/run/upgrade-playwright/playwright/). Opens the console with this prompt ready to send, in a Project that has your repository.

## What the agent does

A Playwright version is written in more places than `package.json`, and the upgrade fails quietly when one of them is missed. In a suite like MarcusFelling/demo.playwright, `devDependencies` has `@playwright/test` at `^1.46.0` and `package-lock.json` resolves both `@playwright/test` and `playwright` to 1.46.0. The container image `mcr.microsoft.com/playwright:v1.46.0-jammy` is written once in `.github/workflows/template-test.yml` and three times in `.github/workflows/test.yml`. Every job then runs `npx playwright install` inside that container. That last step is why a mismatch would go unnoticed: if the package moved and the image tag did not, the job would download a second set of browsers on every run and still pass. The agent searches the repository for the version string and for `mcr.microsoft.com/playwright` and lists every hit before it changes one.

It reads the release notes for each minor between the pinned version and the target, which it takes from `npm view @playwright/test version`, and turns them into a checklist against this suite: the Node versions the target supports against the `node-version: 18` the workflows set, which Ubuntu tags the target's image is published under, and any change to browsers, headless mode or APIs the specs use. It reads the specs for the parts of Playwright that move most between releases. Here that is `_android` in `android/tests/example.spec.ts`, which Playwright exports as experimental, `launchPersistentContext` with `--load-extension` and `backgroundPages()` in `chrome-extension/tests/example.spec.ts`, where the extension's `manifest.json` declares `manifest_version` 2, the CDP session in `performance/tests/example.spec.ts`, and `toHaveScreenshot` in `visual-comparison`.

Then it upgrades and runs. `npm install --save-dev @playwright/test@<target>` updates the manifest and the lock together, and `npx playwright install --with-deps` brings the matching browsers into the sandbox. The agent runs each folder's suite with its own config, in every project that config defines, with `--retries=0`. It runs the same commands on the base commit first, so a test that already failed before the upgrade is not blamed on it. Packages that ride on Playwright are checked against their peer ranges: `@axe-core/playwright` and `playwright-lighthouse`, which here sits beside `lighthouse` 9.

What breaks gets the documented replacement, not a workaround. A renamed option is renamed. A screenshot that differs after a browser bump is examined: the agent attaches the expected, actual and diff images, and regenerates the Linux baseline `landing-chromium-linux.png` only when the difference is rendering and not content. It cannot produce `landing-chromium-win32.png` and says so. If the target's Chromium no longer loads a Manifest V2 extension, that is reported as a finding with the error, since porting the extension is a separate job. Finally it moves all four image tags to the target version, on an Ubuntu tag the target publishes, and leaves unrelated workflow problems on a list for the team.

## Start it with one API call

`POST https://api.phoenix.so/o/{org}/sessions`

```json
{
  "agent": "claude",
  "input": "Upgrade Playwright in this repository to the latest stable release. Read package.json, package-lock.json, .gitignore, every playwright.config.ts, every workflow under .github/workflows, the devcontainer config and every spec first. List every place a Playwright version or a mcr.microsoft.com/playwright image tag is written, and every package that depends on Playwright with its peer range. Get the target from npm view @playwright/test version. Read the release notes for each minor between the pinned version and the target and write a checklist for this suite: supported Node versions against the workflows' node-version, published image tags, changes to bundled browsers and headless mode, and changed or removed APIs that the specs use. Before upgrading, run npm ci, npx playwright install --with-deps and each folder's suite with --retries=0 in every project its config defines, and record what already fails; do not run suites that need credentials or an Android device, and list them as not run. Then run npm install --save-dev @playwright/test@<target>, install the matching browsers and rerun the same suites. Fix each new failure with the replacement the release notes document. For a screenshot difference, attach expected, actual and diff, and regenerate only the Linux baseline and only when the content is unchanged. Update every image tag in the workflows to the target version on a tag that exists, and raise node-version only if the target requires it. Do not raise retries or timeouts, do not skip tests, and do not fix unrelated workflow problems; list them. Open a pull request with the checklist, each item marked applied or not applicable with the file it touched, and a before and after table of results per folder and browser.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
```

1. **Get an API key.** In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
2. **Send the request.** POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
3. **Read what came back.** Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

## What comes back

A pull request that changes `package.json` and `package-lock.json`, the image tag in each workflow that names one, and the specs or configs the new release required. Its description opens with the list of every place the old version was written and what each became, so a reviewer can confirm none is left with one search.

Below that is the checklist built from the release notes, each item marked `applied` with the file it touched or `not applicable` with the search that found no use, and a table with one row per folder and browser: the result on the base commit and the result on the upgraded branch. A test that fails in both columns is marked as failing before the upgrade. Any regenerated screenshot is shown with its diff image. Suites the agent did not run are named with the reason: secrets it was not given, an emulator, a runner operating system the sandbox does not have.

A reviewer checks three things. The image tags and the lockfile name the same version. No spec lost an assertion while being adapted. And every new baseline image shows a rendering change only. The repository's own CI then runs on the pull request inside the new container image, which is the first time that image is exercised; the sandbox runs used browsers installed by `npx playwright install`, not the image.

## Run it on every Playwright release

Playwright ships a minor about once a month, so a schedule fits better than a release hook. An automation in the Project that holds the repository, with this prompt and a `schedule` trigger whose `cron` picks one morning a month in your `timezone`, starts one run. When `npm view` reports the version already pinned, the agent ends the run with nothing to do and opens no pull request.

For many repositories with their own suites, a script on your server lists them with `GET /o/{org}/github` and sends this body to `POST /o/{org}/sessions` once per repository, each with a `request_id` that is a UUID v5 of the repository's full name and the target version, so running the script twice starts nothing new. A key runs a limited number at once and queues the rest.

## Why run it on Rush?

The upgrade itself is one command. The work is the two full passes of every suite in every browser, before and after, and the reading in between. Downloading three browsers and running a dozen configs twice ties up a laptop and gives a result that depends on its screen scaling and fonts. A sandbox starts clean each time, and the commands and their output, the failed attempts included, stay in the run's transcript at `GET /o/{org}/sessions/{id}/messages`.

The before and after table is what makes the pull request mergeable without rerunning it locally. A reviewer sees which failures the upgrade caused, which it inherited and which tests were never run, and the session read at `GET /o/{org}/sessions/{id}` names the pull request and what the run cost.

## Frequently asked questions

### Dependabot already bumps @playwright/test. What does this add?

An npm update from Dependabot changes `package.json` and the lockfile. The image tag in a workflow's `container.image` is a separate update, when one is configured at all, so the two can merge days apart, and neither adapts a spec to a changed API or looks at a screenshot diff. This run moves every place together and shows the suite's results on both sides.

### Will it update our screenshot baselines?

Only the ones for the platform the sandbox runs, Linux, and only when the diff shows a rendering change with the same content. Baselines for other platforms, such as a `-win32.png` file, are left as they are and listed, because they can only be produced on that platform.

### Can it run suites that need secrets, like an API token?

Not unless you provide them. A secret saved to the Project reaches every session started in that Project as an environment variable, so those suites can run there. Without that, the agent lists them as not run and leaves them to your CI on the pull request.

### Does it also upgrade Node in the workflows?

Only when the target Playwright release does not support the Node version the workflows set. In that case it raises `node-version` to the lowest version the release notes name and says so at the top of the description. Otherwise it leaves Node alone.

## More Playwright jobs

- [Add Accessibility Tests to a Playwright Suite](https://getrush.ai/run/add-accessibility-tests/playwright/)
- [Add Merged Test Reports to a Playwright Workflow](https://getrush.ai/run/add-report-merging/playwright/)
- [Add a Playwright Smoke Suite to a Next.js App](https://getrush.ai/run/add-smoke-tests/playwright/)

## More jobs like this: Upgrade dependencies weekly

- [Upgrade Flutter Packages in pubspec.yaml](https://getrush.ai/run/upgrade-pub-dependencies/flutter/)
- [Upgrade Puppeteer 1.x Scripts to the Current Release](https://getrush.ai/run/upgrade-puppeteer/puppeteer/)
- [Upgrade PyJWT From 1.x to 2.x in a Flask API](https://getrush.ai/run/upgrade-pyjwt/pyjwt/)

## Keep reading

- [How a Rush run works on your repository, from request to pull request](https://getrush.ai/cloud/)
- [Compare cloud agents, agent sandboxes and AI code review](https://getrush.ai/compare/)
- [Rush pricing: Free, Pro and Max](https://getrush.ai/pricing/)
