All jobs

Add Mutation Testing to a TypeScript Library

Yes. An agent sets up Stryker on a TypeScript library's Vitest suite, sorts the surviving mutants, and opens a pull request with the config and a CI job.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

Coverage says a line ran; mutation testing says a test would notice if the line were wrong. The agent begins by reading how the suite is wired, because Stryker has to run the same tests the project does. In a library like remeda/remeda the code lives in packages/remeda inside an npm workspace, and packages/remeda/vitest.config.ts defines three Vitest projects: runtime over src/**/*.test.ts with isolate: false, types over src/**/*.test-d.ts with typecheck only, and a property-based project. src/ holds one file per function with its tests beside it, add.ts next to add.test.ts, and an internal/ directory with the shared helpers such as heap.ts and fusedSequence.ts. CONTRIBUTING.md requires 100% coverage, which is exactly the situation where mutation testing adds information: every line runs, so the question is which assertions are loose.

The agent adds @stryker-mutator/core and @stryker-mutator/vitest-runner as devDependencies of packages/remeda and writes stryker.config.json there. mutate is src/**/*.ts minus *.test.ts, *.test-d.ts, *.test-prop.ts, src/index.ts and src/internal/types/**, the same exclusions the coverage block in vitest.config.ts already makes. The runner is vitest pointed at the runtime project only, since type-level tests cannot catch a runtime mutant and the property suite is too slow per mutant. incremental: true with a committed .stryker-tmp ignore and an incrementalFile under reports/ keeps later runs to the files that changed. Thresholds start at what the first run measures, not at an aspiration.

Then it runs Stryker, first on a few files to check the wiring, src/add.ts and src/internal/heap.ts, then over the whole mutate set with the --concurrency the sandbox allows. The report that matters is the list of surviving mutants with file and line: a boundary changed from < to <= that no test distinguishes, a return value replaced by a constant that an assertion never checks. The agent reads each survivor in context and sorts them into test gaps, equivalent mutants that no test could kill, and mutations in defensive code, and writes that sorting into the pull request rather than a bare score.

Finally it adds a job to .github/workflows/ci-library.yml, beside the existing typecheck, build, type-test and runtime-test jobs, that runs npm --workspace=remeda run test:mutation in incremental mode on pull requests and uploads the HTML report as an artifact. The job does not fail the build on the first merge; the threshold that does is a number the team sets once they have seen a few reports.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "claude",
  "input": "Add mutation testing with Stryker to this TypeScript library. Install @stryker-mutator/core and the Vitest runner in the library package, write a stryker config that mutates src but not test files, index.ts or internal/types, runs only the runtime Vitest project, and uses incremental mode. Run it on two small files first to prove the wiring, then on the whole source. Read every surviving mutant and sort them into real test gaps, equivalent mutants and defensive code, with file and line. Add a CI job that runs it incrementally on pull requests and uploads the HTML report without failing the build yet. Open a pull request with the config, the job and the sorted survivor list.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

The pull request adds stryker.config.json and a test:mutation script to the library package, the two devDependencies, the ignore entry for Stryker's temp directory, and one job in the CI workflow. Its description carries the mutation score from the full run, the per-file table for the files with survivors, and the sorted list: each surviving mutant with its file, line, the mutation Stryker applied, and the agent's reading of why no test caught it.

A reviewer goes straight to the test-gap rows, because each one is a concrete assertion to add: a boundary case in a function whose tests only cover the interior, a data-last call path whose result is never compared. The equivalent-mutant rows are the ones to mark as ignored in the config so they stop appearing. The full Stryker output, including the wiring run on two files, is in the session transcript at GET /o/{org}/sessions/{id}/messages.

A follow-up to the same session, POST /o/{org}/sessions/{id}/messages, can ask the agent to write the tests for the gap rows it found; the sandbox still has the install and the incremental file, so that run starts where this one stopped.

Run it on every pull request that changes src

After the setup lands, the CI job runs Stryker incrementally on each pull request. For a deeper pass, send POST /o/{org}/sessions with a prompt that names the pull request branch in resources (https://github.com/your-org/your-app/pull/<n>) and asks for the survivors in the changed files only; the agent runs the mutants for those files and comments on what the new tests leave uncaught. A request_id that is a UUID v5 of the pull request URL makes a retried send return the same run.

A webhook URL on the API key delivers run.completed with the session URL, so a bot can post the survivor table on the pull request once the run settles.

The use case this job follows, with its automation and trigger: Write the missing tests.

Why run it on Rush?

A full mutation run over a library with hundreds of source files is hours of CPU, and the output is a long list that someone has to read with the code open. Doing that on a laptop blocks the laptop; doing it in a sandbox leaves the machine free and keeps the full report in the transcript. The reading is where an agent earns its keep: sorting survivors into gaps, equivalents and noise is the work, and the pull request delivers the sorting, not just the score.

Because the run installs the project exactly as CI does and runs the same Vitest project, the result is one a reviewer can reproduce from the config in the pull request. Several packages in a monorepo can be measured at once, each in its own run.

Frequently asked questions

Will this slow down CI?

The job runs in incremental mode, so a pull request only re-tests mutants in files it changed, and the first full run happens in the sandbox rather than on a runner. The job uploads a report and does not fail the build until the team sets a threshold.

Why only the runtime Vitest project?

A mutant is a change to runtime behaviour, so only runtime assertions can kill it. The type-level tests in *.test-d.ts check signatures and would pass or fail identically for every mutant, and the property project is too slow to run once per mutant.

What is an equivalent mutant?

A mutation that produces a program with the same behaviour, such as changing a condition that can never be false. No test can kill it, so the agent lists those separately with a reason, and they become ignore entries in the config rather than test work.

Does it work with Jest or Mocha?

Yes. Stryker has runners for Jest, Mocha, Karma and a command runner for anything else; the agent reads your test script and picks the matching runner. The config shape and the survivor sorting are the same.

More TypeScript library jobs

The same job on other stacks

Keep reading