What the agent does
The agent clones the solution into its own cloud sandbox and reads how tests already run. In a solution like jasontaylordev/CleanArchitecture, global.json pins the SDK at 10.0.403 with rollForward: latestFeature, Directory.Build.props sets net10.0, TreatWarningsAsErrors and an ArtifactsPath of artifacts, and Directory.Packages.props manages every package version centrally, including coverlet.collector at 10.1.0, nunit at 5.0.0, NUnit3TestAdapter at 6.3.1 and Microsoft.NET.Test.Sdk at 18.11.0. Four of the six projects under tests reference coverlet.collector already: Application.UnitTests, Application.FunctionalTests, Domain.UnitTests and Infrastructure.IntegrationTests. Web.AcceptanceTests drives Playwright through Reqnroll and TestAppHost boots the Aspire host, so neither measures code coverage and neither should. .github/workflows/build.yml restores, builds in Release, installs the Playwright browser and runs dotnet test --no-build --configuration Release, with no collector argument, no results directory and nothing uploaded.
So the collector is installed but never asked to run. The agent changes the test step to dotnet test --no-build --configuration Release --collect:"XPlat Code Coverage" --settings coverlet.runsettings --results-directory artifacts/test-results, and adds coverlet.runsettings at the repository root. The runsettings pick the Cobertura format, exclude the test assemblies and the Migrations namespace by filter, and exclude members marked GeneratedCodeAttribute and CompilerGeneratedAttribute, so the number reflects code a person wrote. Because the solution is CleanArchitecture.slnx and one dotnet test runs every project, the collector writes one coverage.cobertura.xml per test project under the results directory, which is why the next step merges them.
It adds a tool manifest at .config/dotnet-tools.json with dotnet-reportgenerator-globaltool, so the merge uses a pinned version that dotnet tool restore installs in CI and on a laptop alike. A new workflow step runs reportgenerator over artifacts/test-results/**/coverage.cobertura.xml with -reporttypes:HtmlInline;MarkdownSummaryGithub;Cobertura, appends the Markdown summary to $GITHUB_STEP_SUMMARY, and uploads the HTML report with actions/upload-artifact@v7, the same major the repository's release.yml already uses. The step runs with if: always() after the tests, so a failing test still produces the report that shows what it covered. The #if (!UseApiOnly) markers the workflow carries, because the repository is a dotnet new template, are kept exactly where they are.
It proves the pipeline in the sandbox before opening the pull request. It runs dotnet restore, dotnet build --configuration Release and the new test command on the projects that do not need a browser or the Aspire host, confirms that a Cobertura file appears per project, runs dotnet tool restore and reportgenerator, and reads the resulting summary to check that the Migrations folder and the test assemblies are absent from it. The README gets a short section on reading the coverage report from a workflow run and on producing it locally with the same two commands. No threshold is enforced in this change; the pull request states the line and branch figures the sandbox run produced for the projects it could run and leaves the gate for the reviewer to set.
Start it with one API call
POST https://api.phoenix.so/o/{org}/sessions with this body:
{
"agent": "claude",
"input": "Add code coverage reporting to CI for this .NET solution. Read global.json, Directory.Build.props, Directory.Packages.props, every test project and the existing workflows first, and say which test projects already reference a coverage collector and which should be excluded. Change the dotnet test step to collect XPlat Code Coverage in Cobertura format with a runsettings file that excludes test assemblies, migrations and generated code. Add a dotnet tool manifest with ReportGenerator, a step that merges the per-project results into a Markdown summary for the job summary and an HTML artifact, running even when tests fail. Keep any template markers in the workflow as they are. Run restore, build, the new test command and the merge in your sandbox for the projects that need no browser or app host, confirm the exclusions worked, add a README section, and open a pull request with the output. Do not add a coverage threshold; report the figures and leave the gate to the reviewer.",
"resources": [
{
"url": "https://github.com/your-org/your-app"
}
]
}- Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
- Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
- Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.
What comes back
What comes back is a pull request changing .github/workflows/build.yml and adding coverlet.runsettings, .config/dotnet-tools.json and a README section. The description lists each test project with whether it is measured and why, the exclusion filters and attributes in the runsettings, and the Markdown summary the sandbox run produced, with line and branch figures per assembly for the projects it could run.
The run's record keeps the dotnet test run with the collector attached and the reportgenerator output, so a reviewer can see one Cobertura file per project, the merge finding all of them, and the summary without the migrations or the test assemblies in it.
Before merging, a reviewer runs the workflow once on the branch to see the job summary and the uploaded HTML, decides whether the acceptance tests should ever be included, and picks a threshold for a follow-up change now that the first real number exists.
Run it on every .NET solution without coverage in CI
A team with several .NET repositories hands this off from the terminal once per repository: one POST /o/{org}/sessions with this body and the repository in resources, a request_id that is a UUID v5 of the repository name so a retry returns the run it already started, and the key's webhook_url reporting each finished run with its pull request.
Each run reads that repository's own test projects and workflow, so a solution that already uses coverlet.msbuild instead of the collector, or that runs tests from a build/test.ps1 script, gets the change in the place its tests actually run rather than a copy of this one.
The use case this job follows, with its automation and trigger: Hand off a long task.
Why run it on Rush?
Coverage in CI is one command flag plus a dozen details, and every detail is a chance to measure the wrong thing: the test assemblies counting themselves, generated migrations dragging the number down, a report step that is skipped when a test fails. The agent reads the real projects, writes the runsettings against them, and runs the collector and the merge in its own sandbox before the pull request exists, so the first figure the team sees is one that was actually produced.
The same body runs across every .NET repository an organisation keeps, each pull request carrying its own project table and its own summary, and each run's record holding the commands that produced them. That is how a team gets coverage reported the same way everywhere without a person repeating the setup in each repository.
Frequently asked questions
Why not include the acceptance tests in the coverage number?
Because Web.AcceptanceTests drives a browser against the running app through Playwright and the Aspire host, and code executed in another process is invisible to the in-process collector. It does not reference coverlet.collector today and the agent keeps it that way; its value is in the behaviour it checks, not in lines covered.
Does this change what dotnet test does locally?
Only if you pass the same arguments. Running dotnet test with no flags behaves as before. The README section shows the two commands, the test run with --collect and --settings, and dotnet tool restore followed by reportgenerator, that reproduce the CI report on a laptop.
Will a failing test hide the report?
No. The merge and upload steps use if: always(), so a red test run still writes the summary and the artifact. That is deliberate: the report of a failing run is often what tells you which path the failing test never reached.
Can I add a minimum threshold?
Yes, and the pull request shows where: the runsettings or a /p:Threshold argument fail the step under a chosen line or branch percentage. The agent reports the current figures and leaves the gate to the reviewer, because a threshold chosen before the first measurement is usually either ignored or immediately lowered.
Does it work with xUnit or MSTest instead of NUnit?
Yes. The collector attaches to dotnet test regardless of framework, and the runsettings exclude by assembly filter and attribute, not by framework. In a solution that uses coverlet.msbuild rather than the collector package, the agent writes the equivalent MSBuild properties instead.