# Add Accessibility Tests to a Playwright Suite

Yes. An agent adds axe-core scans to a Playwright suite, one per signed-in screen, reads the first findings to fix waits rather than hide rules, and opens a PR.

[Run this on Rush](https://cloud.getrush.ai/run/add-accessibility-tests/playwright/). Opens the console with this prompt ready to send, in a Project that has your repository.

## What the agent does

The agent reads the suite before adding a scan, because an accessibility test is only useful on the pages a person actually reaches. In an app laid out like the `react-vite` app in alan2207/bulletproof-react, `playwright.config.ts` sets `testDir: './e2e'`, two projects, a `setup` project matched on `.setup.ts` and a `chromium` project that depends on it and loads `storageState` from `e2e/.auth/user.json`, a `webServer` that runs `yarn dev --port 3000`, `fullyParallel`, two retries on CI with `trace: 'on-first-retry'`, and the `html` reporter. `e2e/tests/` holds `auth.setup.ts`, a `smoke.spec.ts` that walks discussions and comments end to end, and a `profile.spec.ts`. `package.json` runs `test-e2e` by starting the mock API with `pm2` and then `playwright test`, and `.github/workflows/react-vite-ci.yml` has an `e2e` job that moves `.env.example-e2e` to `.env`, installs browsers with `--with-deps`, runs it and uploads `playwright-report`. `@axe-core/playwright` is not installed.

The scans go in one new file, `e2e/tests/accessibility.spec.ts`, under the `chromium` project so they run signed in: one test per screen the smoke test visits, the landing page, `/app`, `/app/discussions`, one discussion with a comment, and the profile, each navigating, waiting for the heading the smoke test already waits for, and running `new AxeBuilder({ page }).withTags(['wcag2a', 'wcag2aa', 'wcag21a', 'wcag21aa']).analyze()`. The assertion is `expect(violations).toEqual([])`, and on failure the test attaches the violations as JSON with `testInfo.attach`, so the HTML report that CI already uploads carries the node, the rule id and the help URL for each.

The first run decides the shape of the pull request, which is why the agent reads it rather than committing a green file. Rules that fire on the Radix dialog and drawer used by `FormDrawer` and `ConfirmationDialog` are checked against the component before being excluded: an `aria-dialog-name` finding on a drawer that does have a title is a false positive from a scan run before the animation finishes, which a `waitFor` on the dialog's heading fixes in the test, not with a rule exclusion. A `color-contrast` finding on the gray `text-gray-500` empty state in `comments-list.tsx` is a real finding and is listed in the description with the hex values, not fixed, because colour is a design decision.

In its sandbox the agent runs `yarn install`, `yarn playwright install --with-deps`, starts the mock server the way `test-e2e` does, and runs `yarn playwright test` to see the existing three specs green. It adds `@axe-core/playwright` as a dev dependency, writes the spec, runs it, reads the violations, adjusts the waits, and runs the whole suite three times to see the scans hold under `fullyParallel`. The pull request quotes the final run and the violations list from the first run, so a reviewer sees what was found before the waits were fixed.

## Start it with one API call

`POST https://api.phoenix.so/o/{org}/sessions`

```json
{
  "agent": "claude",
  "input": "Add accessibility tests to this app's Playwright suite. Read playwright.config.ts, e2e/tests/auth.setup.ts, e2e/tests/smoke.spec.ts, e2e/tests/profile.spec.ts, package.json and .github/workflows/react-vite-ci.yml first. Add @axe-core/playwright as a dev dependency and write e2e/tests/accessibility.spec.ts under the chromium project: one test per screen the smoke test visits, each waiting for that screen's heading, scanning with the wcag2a, wcag2aa, wcag21a and wcag21aa tags, asserting no violations, and attaching the violations JSON to the test on failure. Run yarn install, install browsers, start the mock server as test-e2e does, and run the existing suite first. Run the new spec, read every violation, fix timing with waits in the test and never with a rule exclusion unless the component shows the finding is wrong, and list real findings in the description with the element and rule. Run the whole suite three times. Open a pull request quoting the first run's violations and the final green run.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
```

1. **Get an API key.** In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
2. **Send the request.** POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
3. **Read what came back.** Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

## What comes back

What comes back is a pull request with one new spec, one new dev dependency in `package.json` and the lockfile, and no change to `playwright.config.ts` or the workflow, because the new file matches the `chromium` project's `.spec.ts` pattern and the `e2e` job already uploads the report the attachments land in. The description quotes the final run, five scans green beside the three existing specs, and the first run's violations with rule ids.

Any finding the agent judged real is in the description with the element, the rule and axe's help link, left for the team rather than patched in a test file: a scan that passes because a rule was excluded is the outcome the request forbids.

Before merging, a reviewer decides whether the `color-contrast` finding on the empty state is fixed in the stylesheet now or tracked, and whether the scans should also run in the `webkit` and `firefox` projects the config does not yet define, which would be two more project entries and a longer CI job.

## Run it on every pull request

Once the scans are in, CI runs them on every push to a pull request through the `e2e` job that exists already. The review on top comes from an automation in the Project with a `github` trigger on `pull_request.opened` and `pull_request.synchronize` and a prompt that reads the uploaded report: a run per push that, when a scan fails, posts a review quoting the rule, the element's selector and the file in `src/` that renders it, and when the diff adds a screen under `src/app/routes`, asks for a scan of it in the same pull request.

Only work by an owner, member or collaborator fires the trigger; a pull request from a fork never does. Each run installs that pull request's branch in its own sandbox with its own mock server, so two reviews never share a `mocked-db.json`.

## Why run it on Rush?

An axe scan is a few lines, and the work is the first run: separating findings that are timing, findings that are the component, and findings that are the design, then deciding which the test owns. Done quickly, the result is a spec full of rule exclusions that passes and checks nothing. In a Rush sandbox the agent runs the scans against the real app and mock server, reads each finding against the component that produced it, and fixes waits rather than hiding rules.

The record of the run keeps the first run's violations and the three final runs, and the pull request quotes them, so the reviewer sees what the scans caught before they went green. Across every frontend a team keeps on Playwright, the same request runs in parallel, one sandbox and one pull request each.

## Frequently asked questions

### Why run the scans signed in?

Because the screens that matter are behind the login: discussions, comments and the profile. The new spec sits under the chromium project, which depends on the setup project and loads e2e/.auth/user.json, so each scan starts with the same session the smoke test uses.

### What happens when a scan finds something?

The test fails and attaches the violations as JSON, so the HTML report CI already uploads shows the rule, the element and axe's help page. The agent reads each finding from the first run: a timing issue gets a wait in the test, a real issue is listed in the description for the team, and nothing is excluded by rule unless the component proves the finding wrong.

### Does this slow the e2e job down?

By the five scans, each a page load and an axe pass of about a second. The suite already runs with one worker on CI and two retries, so the added time is seconds, and the pull request quotes the run time of the final suite.

### Which WCAG level is checked?

A and AA for WCAG 2.0 and 2.1, through the four tags passed to withTags. Best-practice rules that are not WCAG criteria are left out so a failure always maps to a criterion the team can cite; adding the best-practice tag is one more string if they want it.

## More Playwright jobs

- [Add Merged Test Reports to a Playwright Workflow](https://getrush.ai/run/add-report-merging/playwright/)
- [Add a Playwright Smoke Suite to a Next.js App](https://getrush.ai/run/add-smoke-tests/playwright/)
- [Add Test Retries to a Playwright Suite](https://getrush.ai/run/add-test-retries/playwright/)

## The same job on other stacks

- [Add Accessibility Tests to Storybook](https://getrush.ai/run/add-accessibility-tests/storybook/)
- [Add Accessibility Tests to a Vitest Component Suite](https://getrush.ai/run/add-accessibility-tests/vitest/)
- [Add Accessibility Tests to a Cypress Suite](https://getrush.ai/run/add-accessibility-tests/cypress/)

## Keep reading

- [How a Rush run works on your repository, from request to pull request](https://getrush.ai/cloud/)
- [Compare cloud agents, agent sandboxes and AI code review](https://getrush.ai/compare/)
- [Rush pricing: Free, Pro and Max](https://getrush.ai/pricing/)
