What the agent does
The agent reads the build before it touches it. In a library laid out like jhy/jsoup, pom.xml compiles for Java 8 with a Java 11 overlay in src/test/java11 for the multi-release jar, runs unit tests with maven-surefire-plugin 3.6.0 under -Xss640k, and runs *IT classes with maven-failsafe-plugin 3.6.0, which excludes the long-running tag and sets junit.jupiter.execution.parallel properties. Tests are org.junit.jupiter:junit-jupiter 5.14.4. There are 105 production files under src/main/java/org/jsoup and 111 test files, and .github/workflows/build.yml runs mvn verify on JDK 8, 17 and 25 across Linux, Windows and macOS. Nothing in the pom mentions PIT, JaCoCo or any coverage tool.
Mutation testing on every class of a parser would take far longer than one run should, so the agent picks a scope and says why. It starts with org.jsoup.safety, where Cleaner.java (247 lines) and Safelist.java (695 lines) are the XSS-relevant code and CleanerTest already has 59 test methods, and org.jsoup.helper.Validate and DataUtil, which have 13 and 34. It adds pitest-maven 1.30.1 with pitest-junit5-plugin 1.2.3 as the plugin's own dependency (the README says 1.2.3 needs pitest 1.19.4 or above, and the plugin goes on PIT's classpath, not the project's), inside a pitest profile so mvn verify and the three-platform matrix are unchanged. targetClasses names those packages, targetTests the matching test packages, and excludedTestClasses leaves out org.jsoup.integration.*, whose TestServer starts Netty and whose ConnectIT, SessionIT and ParserSoakIT belong to failsafe. jvmArgs carries the same -Xss640k surefire uses, because a stack-overflow test that passes under surefire must not fail inside PIT for a different stack size.
The agent runs ./mvnw -Ppitest test-compile org.pitest:pitest-maven:mutationCoverage on JDK 17, the version the repository's codeql.yml already builds with, and reads target/pit-reports/index.html and the XML report. For each surviving mutant it reads the mutated line and decides: a boundary mutation in a length check that no test exercises gets a test; a mutant that changes a log message or an equivalent branch is listed in the pull request as equivalent, with the reason, and not chased. New tests go next to the existing ones, SafelistTest and ValidateTest, in the same style (plain @Test methods, assertEquals and assertThrows), and production code is left as it is: the use case is tests, not refactoring.
A .github/workflows/mutation.yml runs the same goal on pull requests, on ubuntu-latest with JDK 17 and cache: maven like build.yml, with mutationThreshold set to the score the agent measured, rounded down, so the check fails only when a change makes the tests weaker. It uploads target/pit-reports as a workflow artifact. timestampedReports is set to false so the report path is stable for the upload step.
Start it with one API call
POST https://api.phoenix.so/o/{org}/sessions with this body:
{
"agent": "claude",
"input": "Add PIT mutation testing to this Maven project. Read pom.xml (the surefire and failsafe configuration, the JUnit version and the compiler setup), .github/workflows/build.yml and codeql.yml, and the test packages under src/test/java. Add pitest-maven with pitest-junit5-plugin as the plugin's dependency inside a profile named pitest, scoped with targetClasses and targetTests to the packages where mutation testing pays off, excluding integration tests that start servers, and carrying surefire's argLine in jvmArgs. Run the mutationCoverage goal on JDK 17, read the report, and write unit tests next to the existing ones that kill surviving mutants where a real test exists to write; list the mutants you judge equivalent with a reason. Add a workflow that runs the goal on pull requests with a mutationThreshold at the measured score and uploads the report. Do not change production code. Open a pull request that quotes the mutation score before and after your tests and the surefire result.",
"resources": [
{
"url": "https://github.com/your-org/your-app"
}
]
}- Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
- Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
- Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.
What comes back
The pull request changes pom.xml (the pitest profile with the plugin, its JUnit 5 dependency and the scope) and adds .github/workflows/mutation.yml plus new or extended test classes in the scoped packages. Its description quotes the line coverage and mutation score PIT printed for the scope before the new tests and after them, the count of mutants generated, killed, survived and timed out, the surefire summary showing the existing suite still passes, and the list of mutants left as equivalent with one line each on why.
A reviewer checks the scope first: the packages named in targetClasses are the ones where a weak test matters, and the excluded test classes are the ones that need a network or a server. Then the threshold: a number just under the measured score catches regressions without failing the first pull request that touches an untested branch. Then the tests themselves, which should read like the file's existing tests and assert behaviour, not the mutant.
Nothing changes for mvn verify, the release process or the Java 8 target. The profile runs only when asked for, and the workflow is the only place that asks.
Run it on every pull request that changes a scoped package
An automation on the Project with a github trigger on pull_request.opened and pull_request.synchronize runs read-only with a prompt that runs the pitest profile for the classes the pull request changed and comments the surviving mutants on the pull request, each with its file, line and mutator, so the author sees which new branch no test reaches. The event the trigger adds after the prompt names the pull request; nothing is substituted into the prompt text.
When a comment names survivors the team wants killed, a handler on your own server sends this body to POST /o/{org}/sessions with the pull request branch in resources as https://github.com/your-org/your-app/tree/<branch> and the survivors appended to the prompt, with a request_id that is a UUID v5 of the head commit SHA so a redelivered event returns the run already started. That run pushes the tests to the branch, and the key's webhook_url receives run.completed.
The use case this job follows, with its automation and trigger: Write the missing tests.
Why run it on Rush?
A mutation run is the slowest check in a Java build, and on a laptop it competes with everything else for cores while the developer waits to read a report. In its own sandbox the agent runs PIT with the thread count the machine has, reads the report, writes a test, runs the goal again, and keeps every command and every score in the run's record, so the pull request's numbers can be checked against the record rather than taken on trust.
The judgement calls are the work: which packages to scope, which survivors are equivalent, what threshold to set. Those come out of reading the report next to the code, and the pull request carries the reasoning so the reviewer argues with a written case instead of rerunning the tool.
Frequently asked questions
Will this slow down the normal build?
No. The plugin lives in a profile that only -Ppitest activates, so mvn verify, the three-platform matrix and a release build do not run it. Only the new workflow and anyone who asks for the profile pay for the mutation run.
Why not mutate the whole project?
PIT runs the tests that cover each mutant once per mutant, so the time grows with the product of mutants and test runtime. On a parser with over a hundred production files that is hours. The agent scopes to packages where a weak test has consequences, such as the HTML safelist and the input helpers, and the profile's targetClasses is one line to widen later.
Does it work with JUnit 5?
Yes, through pitest-junit5-plugin, declared as a dependency of the pitest-maven plugin rather than of the project. With pitest-maven the plugin resolves the matching junit-platform-launcher at runtime. A project on JUnit 6, where the platform artifacts moved to a 6.0.x version, is a different case: the plugin's own issue tracker has an open report on it, and the agent says so instead of pretending the run worked.
What is an equivalent mutant?
A mutation that changes the bytecode but not the behaviour, such as replacing a >= with > on a branch the earlier check already excluded. No test can kill it. The agent lists each one it finds with the line and the reason, so the threshold is set against the killable mutants and the list is there for the next person who reads the report.
Will the threshold block unrelated pull requests?
The threshold applies to the scoped packages only, and the agent sets it just under the score it measured. A pull request that does not touch those packages produces the same score. One that adds an untested branch there lowers it, which is the point.