# Shard Playwright Tests Across GitHub Actions Runners

Yes. An agent splits a Playwright suite across parallel GitHub Actions jobs with --shard, merges the blob reports into one HTML report, and opens a PR.

[Run this on Rush](https://cloud.getrush.ai/run/add-test-sharding/playwright/). Opens the console with this prompt ready to send, in a Project that has your repository.

## What the agent does

The agent starts from how the suite runs in CI today. In a repository laid out like MarcusFelling/demo.playwright, each folder is its own project with its own `playwright.config.ts`, and `.github/workflows/test.yml` runs one job per folder: most of them call the reusable `.github/workflows/template-test.yml` with `TESTGROUP` and `BASEURL` inputs, while `chrome-extension`, `github-api` and the disabled `android` and `oauth` jobs are written out in full. The template runs on `ubuntu-24.04` inside the `mcr.microsoft.com/playwright:v1.46.0-jammy` container, sets up Node 18 with the npm cache keyed on the root `package-lock.json`, runs `npm ci` and `npx playwright install`, then `HOME=/root npx playwright test --retries 0` in the group's directory with `continue-on-error: true`, and uploads `playwright-report` as an artifact named after the group, which a GitHub environment links to as `testresults.marcusfelling.com/<group>`.

Then it reads the slowest group. `todomvc/playwright.config.ts` sets `workers: process.env.CI ? 1 : undefined`, two retries on CI, `trace: 'on'`, the `list` and `html` reporters, `baseURL` from the folder's `.env`, and three projects for Desktop Chrome, Firefox and Safari. `todomvc/tests/example.spec.ts` holds 25 tests, so CI runs 75 browser tests one at a time on one runner, each recording a trace. `svgomg` has the same config shape with six tests. The root `package.json` pins `@playwright/test ^1.46.0`, which has `--shard` and the `blob` reporter with `merge-reports`. `.gitignore` already ignores `**/playwright-report/` and `**/test-results/` but not `blob-report`.

The change is in the template, so every group can opt in without its own workflow. `template-test.yml` gains a `shards` input that defaults to `'[1]'`, a `strategy.matrix.shard` built with `fromJSON(inputs.shards)` and `fail-fast: false`, and the test step runs `npx playwright test --retries 0 --shard=${{ matrix.shard }}/${{ strategy.job-total }}`. Each shard uploads `blob-report` as `blob-report-<group>-<shard>`, and a second job, `merge`, with `needs: test` and `if: always()` so it runs after a shard fails, downloads them with `pattern: blob-report-<group>-*` and `merge-multiple: true` into one directory, runs `npx playwright merge-reports --reporter html` on it, and uploads the result under the group's old artifact name so the environment link keeps pointing at one report. In `todomvc/playwright.config.ts` and `svgomg/playwright.config.ts` the reporter becomes `blob` on CI and stays `list` plus `html` locally; `workers: 1` stays, since the split is across machines. `test.yml` passes `shards: '[1, 2, 3]'` for `todomvc` and leaves every other call site alone, and `.gitignore` gains `**/blob-report/`.

To prove it without a runner, the agent installs Chromium with `npx playwright install --with-deps chromium` in its sandbox, runs `npx playwright test --shard=1/3 --list` and `--shard=2/3 --list` in `todomvc` to show the tests split with none shared, runs each shard with `--reporter=blob --project=chromium` against the `.env` base URL, merges the three blob reports with `merge-reports --reporter html` and attaches the resulting `index.html` to the run, and checks the two workflow files with `actionlint`. The pull request links the first workflow run on the branch so a reviewer can see three `todomvc` shards and one merged report.

## Start it with one API call

`POST https://api.phoenix.so/o/{org}/sessions`

```json
{
  "agent": "claude",
  "input": "Shard the todomvc Playwright group across parallel GitHub Actions jobs. Read .github/workflows/test.yml, .github/workflows/template-test.yml, package.json, .gitignore, todomvc/playwright.config.ts, todomvc/tests/example.spec.ts, todomvc/.env and svgomg/playwright.config.ts first. In template-test.yml add a shards input defaulting to '[1]', a matrix from fromJSON(inputs.shards) with fail-fast off, pass --shard=<matrix.shard>/<strategy.job-total> to playwright test, upload each shard's blob-report under a name with the group and shard, and add a merge job that runs even when a shard failed, downloads the shard artifacts with merge-multiple, runs npx playwright merge-reports --reporter html, and uploads playwright-report under the group's existing artifact name. Switch the todomvc and svgomg configs to the blob reporter on CI only. In test.yml pass shards '[1, 2, 3]' for todomvc and change no other job. Add blob-report to .gitignore. Install Chromium in your sandbox, show --shard=1/3 --list and --shard=2/3 --list for todomvc, run the three shards with the blob reporter on Chromium, merge them, attach the merged index.html, and run actionlint on both workflow files. Change no test.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
```

1. **Get an API key.** In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
2. **Send the request.** POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
3. **Read what came back.** Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

## What comes back

The pull request changes `.github/workflows/template-test.yml`, `.github/workflows/test.yml`, `todomvc/playwright.config.ts`, `svgomg/playwright.config.ts` and `.gitignore`, and no test file. Its description shows the two `--list` outputs side by side so a reviewer can see each test lands in exactly one shard, the three local shard runs, the `merge-reports` command, the `actionlint` result, and a link to the branch's first workflow run with its three `todomvc` jobs and the `merge` job.

Every group that does not pass `shards` runs as before: one job, one `--shard=1/1`, and its report merged from a single blob, so the artifact a reviewer downloads has the same name and layout it always had. The `chrome-extension`, `github-api`, `android` and `oauth` jobs, which do not use the template, are untouched. Traces are still recorded per test and travel inside each blob into the merged report.

Before merging, a reviewer picks the shard count for `todomvc` against the runner minutes they want to spend, since three shards is a starting point rather than a measurement, decides whether `svgomg` should also shard or only switch reporters, and checks the environment URL still resolves to the merged artifact.

## Run it on every Linear issue labelled ci

Add an automation to the Project with this prompt and a `linear` trigger on `linear.issue.labeled` with the label your CI work uses in its `labels`. Each labelled issue starts one run with the issue as its context, on its own branch; the trigger adds the issue after the prompt and fills in no placeholder, and a change made by an app integration never fires it, so the agent's own comment on the issue starts nothing.

The automation's `max_runs_per_hour` holds a burst of labels to that many runs an hour: a fire past it is created `queued` and starts when the hour has room, and one past a second hour's worth is skipped as `rate_limited` until the issue is labelled again. From your own server, the same run starts with this request and the issue URL in `resources`, with a `request_id` that is a UUID v5 of the issue id so a retried start returns the run already begun.

## Why run it on Rush?

Sharding is a workflow change that is only right when the matrix, the artifact names, the merge step and the reporter switch all agree, and a mistake shows up as a missing report rather than a failing build. The agent runs the shards and the merge in its own sandbox before touching CI, and checks the YAML with `actionlint`, so the first workflow run on the branch is a confirmation rather than the first test.

The transcript keeps the `--list` outputs, the shard runs and the merge with their output, so a reviewer reads the split and the merged report from one page. Several repositories that share this template pattern can be sharded at once, each run in its own sandbox with its own browsers, and nothing merges until a person reads the diff.

## Frequently asked questions

### Why not just raise workers instead of sharding?

`workers` parallelises on one machine, and the config sets it to one on CI on purpose: the demo runs three browsers with traces on, and a GitHub runner has limited cores. `--shard` splits the same list across machines, so each job keeps one worker and the wall clock drops with the number of jobs. The two can be combined later if a runner has room.

### What does merge-reports need?

Every shard must write the `blob` reporter, and the merge must run on the same Playwright version, which the template guarantees through the pinned container image and `npm ci`. The merge job downloads all `blob-report-<group>-*` artifacts into one directory and produces an HTML report identical in shape to a single run's, traces included.

### Will a failing shard still produce a report?

Yes. The test step keeps `continue-on-error: true`, the matrix has `fail-fast: false` so one red shard does not cancel the others, and the merge job runs with `if: always()`. A reviewer gets one report with the failures marked rather than two partial ones.

### Does this change how tests run locally?

No. The reporter switch keys on `process.env.CI`, so `npx playwright test` in `todomvc` still prints the list and opens the HTML report. `--shard` is only passed in the workflow; locally the whole suite runs as before.

## More Playwright jobs

- [Set Test, Expect and Global Timeouts in a Playwright Suite](https://getrush.ai/run/add-test-timeouts/playwright/)
- [Add Visual Regression Tests With Playwright](https://getrush.ai/run/add-visual-regression-tests/playwright/)
- [Fix Flaky Playwright Tests](https://getrush.ai/run/fix-flaky-tests/playwright/)

## The same job on other stacks

- [Shard Cypress Specs Across CI Containers Without Cypress Cloud](https://getrush.ai/run/add-test-sharding/cypress/)
- [Shard a Jest Suite Across Parallel CI Jobs for an Express API](https://getrush.ai/run/add-test-sharding/jest/)

## More jobs like this: Turn a Linear label into a PR

- [Add a Theme App Extension to a Shopify App](https://getrush.ai/run/add-theme-app-extension/shopify-app/)

## Keep reading

- [How a Rush run works on your repository, from request to pull request](https://getrush.ai/cloud/)
- [Compare cloud agents, agent sandboxes and AI code review](https://getrush.ai/compare/)
- [Rush pricing: Free, Pro and Max](https://getrush.ai/pricing/)
