# Add Merged Test Reports to a Playwright Workflow

Yes. An agent switches your parallel Playwright CI jobs to the blob reporter, adds a job that merges them into one HTML report, proves the merge, and opens a PR.

[Run this on Rush](https://cloud.getrush.ai/run/add-report-merging/playwright/). Opens the console with this prompt ready to send, in a Project that has your repository.

## What the agent does

The agent starts with how the reports are produced now. In a repository laid out like MarcusFelling/demo.playwright, each folder, `basic`, `todomvc`, `svgomg`, `visual-comparison`, `accessibility` and the rest, has its own `playwright.config.ts` with `reporter: [['list'], ['html']]`, `trace: 'on'` and `workers: process.env.CI ? 1 : undefined`. `.github/workflows/test.yml` runs one job per folder, most of them through the reusable `template-test.yml`, which runs `npx playwright test --retries 0` in the folder with `continue-on-error: true` and uploads `playwright-report` as an artifact named after the group. So a pull request ends with eight separate `playwright-report` artifacts, each a zip to download and open, and a failure in `todomvc` on webkit is found by guessing which zip to open. The container image is `mcr.microsoft.com/playwright:v1.46.0-jammy` and `package.json` pins `@playwright/test ^1.46.0`, which has the `blob` reporter and `merge-reports`.

It changes what each job writes, not how it runs. Every folder's config keeps `list` and gains `['blob']` on CI in place of `html`, with `outputDir` left at the default `blob-report`; the HTML reporter stays for local runs through a `process.env.CI` check so a developer's `npx playwright test` still opens a report. In `template-test.yml` and the hand-written jobs the upload step changes to `blob-report` with a name of `blob-${{ inputs.TESTGROUP }}`, and because the `basic` folder runs with `testIgnore` and a different `baseURL`, the agent checks that its blob still merges with the others, which `merge-reports` allows as long as the Playwright version matches.

A new `merge-reports` job in `test.yml` runs after every test job with `if: always()`, since the test steps are `continue-on-error`, downloads every `blob-*` artifact into one directory with `actions/download-artifact` and `pattern`, runs `npx playwright merge-reports --reporter html ./all-blob-reports`, and uploads the one `playwright-report` that comes out, named `html-report--attempt-${{ github.run_attempt }}` so a re-run does not overwrite the first. The job runs in the same container image so the merge uses the same Playwright. The README's line about extracting the zip is updated to name the one artifact.

In its sandbox the agent installs with `npm ci`, runs `npx playwright test` in `todomvc` and `svgomg` with `CI=1` so each writes a `blob-report`, copies both into one directory, runs the merge, and opens the resulting `index.html` to confirm both groups appear with their traces attached. It quotes the merge command's output and the list of files in `playwright-report` in the pull request, and runs `npx playwright test --list` in each changed folder to show no config error.

## Start it with one API call

`POST https://api.phoenix.so/o/{org}/sessions`

```json
{
  "agent": "claude",
  "input": "Add merged test reports to this Playwright repository's CI. Read .github/workflows/test.yml, .github/workflows/template-test.yml, package.json, the README and every playwright.config.ts first. In each config use the blob reporter on CI and keep html for local runs. In template-test.yml and the inline jobs upload blob-report as blob-<group>. Add a merge-reports job that runs after all test jobs with if: always(), downloads every blob-* artifact, runs npx playwright merge-reports --reporter html, and uploads one playwright-report named with the run attempt; run it in the same Playwright container image. Update the README's artifact instructions. In the sandbox run todomvc and svgomg with CI set, merge their blobs, and quote the merge output and the report's file list in the pull request. Run npx playwright test --list in every changed folder. Open a pull request.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
```

1. **Get an API key.** In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
2. **Send the request.** POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
3. **Read what came back.** Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

## What comes back

What comes back is a pull request touching every `playwright.config.ts`, `.github/workflows/test.yml`, `.github/workflows/template-test.yml` and `README.md`. The description quotes the `merge-reports` output from the sandbox, the file listing of the merged `playwright-report` with both groups' trace zips in `data/`, and the `--list` output per folder showing the same test counts as before.

The run's record keeps the two sandbox test runs in full, so a reviewer can see each group writing its blob and the merge reading both. The agent ran against the public demo sites the configs name, which is what the workflow does too.

Before merging, a reviewer decides whether the per-group artifacts should also stay. The pull request drops them in favour of the one merged report; keeping both is one more upload step per job and doubles the artifact storage.

## Run it on every pull request

Once the merge job is in, every pull request's checks end with one report, and the review of that report is a job an agent can do. In the Project for the repository, an automation with a `github` trigger on `pull_request.opened` and `pull_request.synchronize` and a prompt that reads the merged report starts a run per pull request; the run downloads the `html-report` artifact, reads the failures and their traces, and posts a review comment naming the test, the step and the screenshot.

A re-delivered event is answered `deduped` by the trigger, so it does not open a second review. The same automation on several repositories runs in parallel, one sandbox each.

## Why run it on Rush?

Eight artifacts become one through two reporter lines and a workflow job, and the thing that goes wrong is the plumbing: a blob written with one Playwright version and merged with another refuses to load, a `download-artifact` pattern that misses one group silently drops its tests from the report, a merge job without `if: always()` never runs on the pull request that needed it. In a Rush sandbox the agent writes two real blobs with the pinned version, merges them, and opens the result before touching CI.

The merged report and the commands stay with the run, so the next person adding a test group knows the artifact name pattern and why the merge job runs in the container. Across many repositories with per-folder Playwright projects the job runs in parallel, one sandbox each, with each pull request carrying its own merge output.

## Frequently asked questions

### Does this change how tests run or how many run?

No. Each job still runs its folder with the same workers, retries and projects; only the reporter changes from `html` to `blob` on CI. The `--list` output the agent quotes shows the same test counts per folder as before.

### Will I still get an HTML report locally?

Yes. The configs choose `blob` only when `CI` is set, so `npx playwright test` on a laptop still writes `playwright-report` and `npx playwright show-report` still opens it.

### What if one test job fails to start?

The merge job has `if: always()` and downloads whatever `blob-*` artifacts exist, so the report covers the groups that ran. The job's log lists the artifacts it found, and a missing group is visible there rather than silently absent.

### Why run the merge in the Playwright container?

Because `merge-reports` reads blobs with the Playwright that wrote them, and a version mismatch is refused. The container image and the `package.json` pin are the same release, and the merge job uses the same image as the test jobs.

### Can the merged report be published to GitHub Pages instead of an artifact?

Yes. The merge job's output directory is a static site; say so in the prompt and the agent adds a deploy step with `actions/deploy-pages` on the `main` branch and keeps the artifact for pull requests.

## More Playwright jobs

- [Add a Playwright Smoke Suite to a Next.js App](https://getrush.ai/run/add-smoke-tests/playwright/)
- [Add Test Retries to a Playwright Suite](https://getrush.ai/run/add-test-retries/playwright/)
- [Shard Playwright Tests Across GitHub Actions Runners](https://getrush.ai/run/add-test-sharding/playwright/)

## More jobs like this: Review every pull request

- [Add SpotBugs to a Gradle Build](https://getrush.ai/run/add-spotbugs/gradle/)
- [Add Spotless Formatting to a Maven Build](https://getrush.ai/run/add-spotless-formatting/maven/)
- [Address Review Comments on a Laravel Pull Request](https://getrush.ai/run/answer-review-comments/laravel/)

## Keep reading

- [How a Rush run works on your repository, from request to pull request](https://getrush.ai/cloud/)
- [Compare cloud agents, agent sandboxes and AI code review](https://getrush.ai/compare/)
- [Rush pricing: Free, Pro and Max](https://getrush.ai/pricing/)
