All jobs

Shard Cypress Specs Across CI Containers Without Cypress Cloud

Yes. An agent replaces record and parallel with a spec list per matrix row, balanced by test count from your specs, so a run with no record key still shards.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

The agent reads how the matrix is sharded today. In a React and Express app laid out like cypress-io/cypress-realworld-app, .github/workflows/main.yml runs four UI jobs, Chrome, Chrome mobile, Firefox and Firefox mobile, each with matrix: containers: [1, 2, 3, 4, 5] and cypress-io/github-action@v6 set to record: true, parallel: true, a group name and spec: cypress/tests/ui/*. The split between the five containers is decided by Cypress Cloud, which reads CYPRESS_RECORD_KEY from secrets and projectId: "7s5okt" from cypress.config.ts. A run that has no key, a pull request from a fork, cannot record and so cannot be parallelised; the five containers would each stop before running a spec.

It then reads the specs to balance by hand. cypress/tests/ui/ has seven: transaction-feeds with 11 it blocks, auth with 8, notifications with 7, new-transaction and transaction-view with 6 each, bankaccounts with 4 and user-settings with 3. Five groups by that count come out close: transaction-feeds; auth; notifications with user-settings; new-transaction with bankaccounts; and transaction-view. The mobile jobs run the same specs at a 375 by 667 viewport through config, with the throttled responses cypress/support/e2e.ts adds when isMobile() is true, so the same groups serve all four jobs.

The matrix changes from numbers to spec lists: matrix: specs: with five comma-separated entries, passed as spec: ${{ matrix.specs }}, and record, parallel and group removed along with CYPRESS_PROJECT_ID and CYPRESS_RECORD_KEY from each job's env. fail-fast: false stays, since one group's failure should not cancel the others. Nothing else moves: the install job still builds once, each row still downloads the build artifact, starts with yarn start:ci and waits for ports 3000 and 3001.

In its sandbox the agent runs yarn install --frozen-lockfile, yarn build:ci and yarn start:ci, then runs yarn cypress:run --spec with each group in turn, reading the time Cypress prints per spec, and reorders a group when the counts and the clock disagree. It checks the matrix syntax with actionlint on the workflow. The per-group times and the actionlint output go in the description, with a note that a new spec file needs a line in the matrix, which is the cost of not using Cloud.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "claude",
  "input": "Shard the Cypress UI specs across the CI matrix without Cypress Cloud. Read .github/workflows/main.yml, cypress.config.ts, cypress/support/e2e.ts and the specs under cypress/tests/ui first. Count the it blocks per spec and split the seven specs into five groups of close size. In each of the four UI jobs replace the containers matrix with a specs matrix holding those groups, pass each as spec, and remove record, parallel, group and the Cypress Cloud secrets from the job. Keep fail-fast false and everything else as it is. Run yarn build:ci, yarn start:ci and each group with yarn cypress:run --spec in the sandbox, quote the time per spec, adjust a group if the times are far apart, run actionlint on the workflow, and open a pull request.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

What comes back is a pull request with one file changed, .github/workflows/main.yml: four matrix blocks now list spec groups instead of container numbers, and the record, parallel, group lines and the two Cypress Cloud variables are gone from each UI job. cypress.config.ts keeps its projectId; nothing in the repository other than the workflow changes.

The description lists the five groups with their it counts and the time each took in the agent's run, the actionlint result, and the reason the split is by hand: a run without CYPRESS_RECORD_KEY now runs every spec. The run's record keeps each cypress:run with its per-spec table.

Before merging, a reviewer decides whether to keep Cloud recording on main pushes, where the key exists, as a separate job; the agent leaves that out, so the matrix is the same for every event.

Run it on every new spec file

The split goes stale when a spec is added. An automation with a github trigger on pull_request.opened and pull_request.synchronize starts a run that reads the diff for a new file under cypress/tests/ui/, checks whether main.yml names it, and comments on the pull request with the group it should join by it count; when every spec is in the matrix, it says nothing.

The trigger adds the pull request to the prompt after it. The comment is the whole result; the prompt says not to push to the author's branch.

The use case this job follows, with its automation and trigger: Hand off a long task.

Why run it on Rush?

Balancing by hand needs the counts and, better, the times, and the times need the app built and running. The agent has both in its sandbox and does the arithmetic once with the real numbers; a reviewer reading the pull request sees why transaction-feeds runs alone.

Every cypress:run and its per-spec table stay with the run, so the groups are a measurement, not a guess. Across a team's Cypress projects the same request runs at once, one sandbox each, and each pull request carries its own timings.

Frequently asked questions

Why not the cypress-split plugin?

It balances automatically from a split index and a count, and is a reasonable next step. The agent starts with a spec list because it adds no dependency and the balance is visible in the workflow; the description names cypress-split for a team that would rather not edit the matrix for each new spec.

Do the mobile jobs need different groups?

No. They run the same seven specs with a 375 by 667 viewport passed through config, and the throttle in cypress/support/e2e.ts slows every API response the same way, so the relative sizes hold.

What happens to runs on main, where the key exists?

They shard the same way. The agent removes recording from all four jobs so the matrix behaves identically on every event; a reviewer who wants Cloud on main adds one job with record: true and an if on the branch.

Does retries.runMode still apply?

Yes. cypress.config.ts sets two retries in run mode regardless of how specs are split, so a flaky spec gets the same chances in each group.

More Cypress jobs

The same job on other stacks

More jobs like this: Hand off a long task

Keep reading