All jobs

Add Mutation Testing to a .NET Solution

Yes. An agent installs Stryker.NET as a local tool, configures it per unit test project with the right project under test, runs it once, and opens a PR.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

The agent reads the solution before it mutates anything. In a solution laid out like jasontaylordev/CleanArchitecture, global.json pins the .NET SDK to 10.0.403 with rollForward: latestFeature, Directory.Build.props sets net10.0, TreatWarningsAsErrors, Nullable and an ArtifactsPath under artifacts/, and Directory.Packages.props manages every version centrally, including nunit 5.0.0, NUnit3TestAdapter 6.3.1, Microsoft.NET.Test.Sdk 18.11.0, coverlet.collector 10.1.0, Moq 4.21.0 and Shouldly 4.3.0. CleanArchitecture.slnx lists seven source projects and six test projects. Two of the test projects are plain unit tests: Domain.UnitTests, which references only Domain, and Application.UnitTests, which references Application and Infrastructure and covers the Common/Behaviours pipeline. The rest need a database or a browser: Application.FunctionalTests uses Respawn, Infrastructure.IntegrationTests and TestAppHost run under Aspire, and Web.AcceptanceTests drives Playwright. .github/workflows/build.yml restores, builds Release and runs dotnet test --no-build; renovate.json updates packages on Mondays. There is no .config/dotnet-tools.json.

Stryker.NET is installed the way its documentation describes for a repository: dotnet new tool-manifest creates .config/dotnet-tools.json, dotnet tool install dotnet-stryker records the current release in it, and anyone else runs dotnet tool restore. The tool needs the .NET 10 runtime, which global.json already requires. Stryker runs from a unit test project's directory and finds the project under test itself when the test project references one; Domain.UnitTests is that case. Application.UnitTests references two projects, so its stryker-config.json names "project": "Application.csproj", the option the documentation requires exactly when a test project references more than one project. The agent writes one stryker-config.json per unit test project with the stryker-config section, reporters set to html, json and progress, and the default Standard mutation level, and leaves thresholds at the documented defaults of 80, 60 and a break of 0, so the first runs report rather than fail.

The agent then runs dotnet tool restore and dotnet stryker in each of the two directories in the sandbox, which builds the project under test, generates mutants for its sources, and runs the NUnit suites against each. Because Directory.Build.props moves build output to artifacts/, the agent reads Stryker's log to confirm its build step found the test assembly under that layout before trusting the score, and says in the pull request what it saw. The HTML and JSON reports land in a StrykerOutput folder inside each test project, which the agent adds to .gitignore beside the existing artifacts/ entry.

Mutation runs take far longer than dotnet test, so they do not join build.yml. The agent adds .github/workflows/mutation.yml with workflow_dispatch and a weekly schedule, which restores tools, runs dotnet stryker in both directories with --reporter json --reporter html, and uploads the two StrykerOutput folders as workflow artifacts. The README gains a section on reading a report and on raising break once the team has a baseline. The Domain run is where the first surviving mutants usually sit, in value objects such as Colour, whose tests in Domain.UnitTests/ValueObjects check conversions and equality; the pull request lists the survivors it found by file and line without claiming a number it did not read.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "claude",
  "input": "Add mutation testing with Stryker.NET to this solution. Read global.json, Directory.Build.props, Directory.Packages.props, CleanArchitecture.slnx, every csproj under src and tests, .github/workflows/build.yml, renovate.json, .gitignore and README.md first. Create a tool manifest with dotnet new tool-manifest and install dotnet-stryker into it. Add a stryker-config.json to tests/Domain.UnitTests and tests/Application.UnitTests with reporters html, json and progress and default thresholds; the Application test project references two projects, so set project to Application.csproj there. Run dotnet tool restore and dotnet stryker in each directory, check in Stryker's log that its build found the test assembly under the ArtifactsPath layout, and add StrykerOutput to .gitignore. Add .github/workflows/mutation.yml with workflow_dispatch and a weekly schedule that restores tools, runs both configurations and uploads the StrykerOutput folders as artifacts. Document how to read a report in the README. Open a pull request that quotes the scores and lists surviving mutants by file and line.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

The pull request adds .config/dotnet-tools.json, two stryker-config.json files, .github/workflows/mutation.yml and a README section, and changes .gitignore. Its description quotes the mutation score each run printed, lists surviving mutants by file and line, and states whether Stryker's build step found the assemblies under artifacts/.

A reviewer reads the survivors list first, because that is the value of the change: each line is a branch or operator no test notices. They then decide which of them deserve a test now, and whether break should be set to the observed score so the weekly run fails on a regression rather than only reporting.

build.yml is untouched, so pull request checks stay as fast as before, and the tool manifest means every developer gets the same Stryker release.

Run it on every Monday morning

An automation on the Project with a schedule trigger set to a weekly cron in the team's timezone starts one read-only run that restores the tool, runs both configurations and puts the scores and the survivors in its result, which GET /o/{org}/sessions/{id} returns; an automation run reads and comments but does not push, so it reports and the fixes are separate runs.

A team that wants the first setup on several solutions sends this body once per repository from its own server, each POST /o/{org}/sessions naming the repository in resources with a request_id that is a UUID v5 of the repository's full name, so a repeated call returns the run already started.

The use case this job follows, with its automation and trigger: Write the missing tests.

Why run it on Rush?

A mutation run is slow, noisy and needs the whole solution built, which is why teams postpone it. The agent builds the solution, runs Stryker against both test projects in its own sandbox, and keeps the logs and reports with the pull request, so the reviewer reads scores and survivors rather than a plan to run it later.

The configuration follows the documented rules for this solution's shape: one config per unit test project, the explicit project where a test project references two. A team with several .NET solutions starts the same request on each at once and gets one pull request per solution, each with its own survivors list.

Frequently asked questions

Why only the two unit test projects?

Application.FunctionalTests, Infrastructure.IntegrationTests, TestAppHost and Web.AcceptanceTests need a database, Aspire or a browser for every test, and a mutation run executes the suite once per mutant. The two unit test projects run in memory, which is what makes mutation testing practical.

Why does Application.UnitTests need the project option?

It references both Application and Infrastructure. Stryker's documentation says project is required when a test project references more than one project, so the config names Application.csproj. Domain.UnitTests references only Domain and needs no such line.

Does this slow down pull request builds?

No. build.yml keeps running dotnet test --no-build. Mutation runs live in mutation.yml, started by hand or by the weekly schedule, and publish their reports as workflow artifacts.

What about the artifacts output layout?

Directory.Build.props sets ArtifactsPath, so assemblies are not under bin/. The run reads Stryker's log to confirm where it found the test assembly and the pull request reports it, rather than assuming either way.

Should break be set from the start?

The config keeps the documented default of 0, so the first runs only report. Once the team has seen a score, setting break to it makes the scheduled run exit non-zero on a regression, which the README explains.

More .NET jobs

The same job on other stacks

Keep reading