All jobs

Add Playwright End-to-End Tests to a Chrome Extension

Yes. An agent loads your built extension into Chromium with Playwright, tests the popup, options page and content script, wires it into CI and opens a PR.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

The agent starts by learning how the extension is built and what it does, because an end-to-end test loads the built dist/ directory, not the TypeScript. In an extension laid out like chibat/chrome-extension-typescript-starter, package.json builds with webpack --config webpack/webpack.prod.js, and webpack/webpack.common.js has four entries, popup, options, background and content_script, writes them to dist/js/, splits shared code into vendor.js for everything but the service worker, and copies public/ into dist/ with copy-webpack-plugin. public/manifest.json is Manifest V3: action.default_popup is popup.html, options_ui.page is options.html, the content script runs on <all_urls>, the service worker is js/background.js, and the only permission is storage. The existing tests are Jest with ts-jest over src/__tests__/sum.ts, which checks a sum function and nothing the extension does; .github/workflows/build.yml runs npm ci, npm run build and npm test on Node 20 and 22.

Then it reads the behaviour to pin down. src/popup.tsx shows the active tab's URL, counts up on a button and writes the count to the badge with chrome.action.setBadgeText, and its second button sends {color: "#555555"} to the active tab; src/content_script.tsx listens for that message, sets document.body.style.backgroundColor and responds with a string naming the colour; src/options.tsx restores favoriteColor and likesColor from chrome.storage.sync, saves them, and shows Options saved. for one second. src/background.ts only schedules an empty poll every thirty seconds, so there is nothing to assert on it beyond the worker starting.

The suite it writes uses @playwright/test with a fixture that calls chromium.launchPersistentContext on an empty profile with --disable-extensions-except and --load-extension pointing at dist, and channel: 'chromium' so the tests run headless. It waits for the service worker to register and reads the extension id from the worker's URL. The popup test opens chrome-extension://<id>/popup.html as a page, clicks count up, and reads the badge back with chrome.action.getBadgeText from inside the page. The options test picks green, ticks the checkbox, clicks Save, expects Options saved., reloads and expects the select to still read green, which proves storage.sync round-trips in the profile. The content-script test routes a fake https:// page with page.route so the <all_urls> match applies, then sends the colour message from the service worker with chrome.tabs.sendMessage, because a popup opened as a tab would query itself as the active tab, and asserts both the body colour and the response string.

The tests live under e2e/, outside Jest's roots: ["src"], so npm test keeps running only the unit tests, and Playwright compiles its own TypeScript, so tsconfig.json with its rootDir: src is not touched. The agent adds playwright.config.ts, a test:e2e script, and in build.yml a step that runs npx playwright install --with-deps chromium and then the suite after the existing build step, on both Node versions. In its sandbox it runs npm ci, npm run build, installs Chromium the same way, runs the suite and keeps the output with the pull request.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "claude",
  "input": "Add Playwright end-to-end tests for this Chrome extension. Read package.json, public/manifest.json, every file under src and webpack, jest.config.js, tsconfig.json and .github/workflows/build.yml first. Build the extension with npm run build, then write tests under e2e that load dist into Chromium with a persistent context, read the extension id from the service worker, and cover the popup's count-up button and badge text, the options page saving and restoring its values through chrome.storage.sync, and the content script changing the page background when it receives the colour message, sending that message from the service worker to a page you serve with page.route. Keep the tests out of Jest's roots and leave tsconfig.json alone. Add a playwright.config.ts, a test:e2e script, and a CI step in build.yml that installs Chromium with its dependencies and runs the suite after the build. Run the suite in the sandbox and open a pull request that quotes its output.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

What comes back is a pull request that adds e2e/ with the fixture and three spec files, playwright.config.ts, a test:e2e script and @playwright/test as a dev dependency in package.json and package-lock.json, and one new step in .github/workflows/build.yml. Nothing under src/ or public/ changes. The description quotes the suite's output from the sandbox run and says which Chromium Playwright installed.

The run's record keeps the webpack build log, the playwright install output and every test's pass line, plus the trace Playwright writes on a failure, so a reviewer can see each assertion held against the real built extension before CI runs it once.

Before merging, a reviewer checks that the CI step runs after npm run build and not before, decides whether the suite should run on both Node versions or one, and reads the fake page the content-script test serves to confirm it is a plain https:// address and not a real site.

Run it on every pull request

Once the suite exists, a github trigger on pull_request.opened and pull_request.synchronize starts a run for each pull request a person opens, with the automation's prompt asking the agent to build the extension, run npm run test:e2e, and post one review that quotes any failing assertion with its spec file and line. The trigger adds the pull request after the prompt, so the prompt names no branch.

Each run checks out that pull request's branch in its own sandbox, installs the same Chromium, and runs the suite against the build of that branch. A pull request that changes the popup gets a review that says which popup assertion moved; a pull request a bot opens never fires the trigger, so a handler on the reader's server starts those runs with the …/pull/<n> URL in resources.

The use case this job follows, with its automation and trigger: Write the missing tests.

Why run it on Rush?

An extension test written without a browser proves nothing about the extension: the unit test in this starter checks sum, and a popup that throws on chrome.tabs.query or a content script that never matches still ships green. In a Rush sandbox the agent builds the real dist/, loads it into a real Chromium with the real Manifest V3 service worker, and watches the badge, the storage and the page change, with the output and traces kept in the record.

The same job across a team's extensions runs in parallel, each in its own machine with its own Chromium and its own pull request, and the suite each run leaves behind keeps running on every later pull request without the agent.

Frequently asked questions

Why launchPersistentContext and not chromium.launch?

Chrome only loads unpacked extensions into a persistent context, and chrome.storage.sync needs a profile directory to persist across the reload the options test does. The fixture gives each test file a fresh empty profile so one test's saved colour never leaks into another.

Can it run headless in CI?

Yes, with channel: 'chromium', which is Chromium's newer headless mode and supports extensions; the older headless mode does not. The CI step installs that Chromium with --with-deps so the runner has the shared libraries it needs.

Why send the colour message from the service worker instead of clicking the popup button?

Because in a test the popup is opened as a tab, and chrome.tabs.query({active: true, currentWindow: true}) in popup.tsx then returns the popup itself, which has no content script. Sending chrome.tabs.sendMessage from the worker to the routed page exercises the same listener in content_script.tsx and asserts the same response string.

Will Jest start picking up the Playwright tests?

No. jest.config.js sets roots to src, and the new tests live under e2e/, so npm test is unchanged. Playwright runs them from its own config with npm run test:e2e.

What about the service worker's polling?

background.ts schedules an empty function every thirty seconds and does nothing else, so the suite only waits for the worker to register, which it needs anyway to learn the extension id. A worker that later does real work gets its own spec in the same fixture.

More Chrome extension jobs

The same job on other stacks

More jobs like this: Write the missing tests

Keep reading