What the agent does
The agent clones the repository into its own cloud sandbox and reads the build before touching it. In the jwt-auth-tests sample of ktorio/ktor-samples, build.gradle.kts applies kotlin("jvm") 2.4.20, the serialization plugin and io.ktor.plugin 3.6.0, depends on ktor-server-core, ktor-server-netty, ktor-server-auth, ktor-server-auth-jwt, content negotiation and kotlinx-json, uses ktor-server-test-host and kotlin("test") for tests, runs them on JUnit Platform and targets jvmToolchain(17) through Gradle 9.2.1. There is no coverage tool anywhere in the repository: no Kover, no JaCoCo. CI is build-all.yml, which on every pull request sets up JDK 24 and runs build-all.sh, a script that walks every sample directory and runs ./gradlew clean check --console=plain, so anything attached to check runs in CI without a workflow change.
It then reads what the tests cover. ApplicationTest.userIsGreetedProperly builds an RSA key pair, feeds jwt.privateKey, issuer, audience and realm through MapApplicationConfig, installs the module with a fake JwkProvider, posts to /login, and calls /hello with the bearer token. That is the only test. Main.kt also has a challenge that answers 401 with the text Token is not valid or has expired, a validate block that returns no principal when the username claim is empty, and acceptLeeway(3) on the verifier, none of which any test reaches.
The agent applies the Kover Gradle plugin in build.gradle.kts, points its HTML and XML reports at the default build/reports/kover location, and wires koverXmlReport and koverVerify into check. Then it writes the missing tests next to the existing one: /hello with no header answers 401 with the challenge text, a token signed by a different key answers 401, a token whose username claim is empty is rejected, and /login with a malformed body is refused. It runs ./gradlew check once, reads the line and branch percentages from report.xml, and sets koverVerify rules just under those numbers, so the floor reflects the suite the pull request adds rather than a guess.
For CI it adds one step to build-all.yml after the script: upload every **/build/reports/kover/** directory as a workflow artifact with if: always(), so the HTML report is downloadable even when a later sample fails. It runs ./gradlew clean check --console=plain the way the script does, confirms the verify task passes and the report files exist, and opens a pull request with the tests, the plugin lines, the rules and the workflow step, pasting the coverage table into the description.
Start it with one API call
POST https://api.phoenix.so/o/{org}/sessions with this body:
{
"agent": "claude",
"input": "Add test coverage to CI for this Ktor app and write the tests it is missing. Read build.gradle.kts, settings.gradle.kts, the Gradle wrapper version, every source and test file, application.conf and every workflow under .github first. Apply the Kover plugin, produce XML and HTML reports, and attach report generation and verification to the check task so the existing CI command runs them. Write tests for the paths the current suite misses, including the unauthorised challenge, a token from the wrong key and an empty username claim. Run the suite once, set the verification thresholds just under the measured line and branch coverage, add a CI step that uploads the report directory as an artifact even on failure, run the same command CI runs, and open a pull request with the coverage table in its description. Do not change the application code.",
"resources": [
{
"url": "https://github.com/your-org/your-app"
}
]
}- Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
- Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
- Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.
What comes back
What comes back is a pull request on its own branch. The diff adds the Kover plugin and rules to build.gradle.kts, new test functions in ApplicationTest.kt beside userIsGreetedProperly, and an upload step in .github/workflows/build-all.yml. Main.kt is unchanged. The description holds the coverage table from report.xml and says the thresholds are a floor to raise.
The run's record keeps the full ./gradlew clean check --console=plain output, with each test named and the verify task's result, and the listing of build/reports/kover, so a reviewer sees the measured numbers and that the floor sits under them.
Before merging, a reviewer checks that the thresholds are not so tight that the next honest refactor fails CI, that the artifact step's glob matches the sample layout, and that the new 401 tests assert on the challenge text the module actually sends.
Run it on every nightly
In the console, a scheduled automation starts one run per Ktor repository each night, in the Project that has it, with this prompt. Each run reads that service's current coverage from the last report, writes tests for the least covered module and raises the floor only when the new tests pass, so the threshold climbs with the suite instead of blocking it.
From your own server, the same work is one POST /o/{org}/sessions per repository with it in resources and a request_id. The API key's webhook posts run.completed with the session_url when a turn ends, so a script can collect the pull request links and the new percentages for the morning.
The use case this job follows, with its automation and trigger: Write the missing tests.
Why run it on Rush?
A coverage floor is only honest when it comes from a measured run of the suite that the pull request itself contains, and a workflow step is only proven when the report lands where the glob expects. The sandbox runs the Gradle build with the repository's own wrapper and JDK, reads report.xml and keeps every line, so the numbers in the pull request are the numbers CI will see.
Writing the missing tests needs the module's own MapApplicationConfig and fake JwkProvider pattern, which the agent reads before adding a case. One run returns the plugin, the tests, the floor and the artifact step together, and the same prompt runs across every Ktor service at once, each measuring its own suite.
Frequently asked questions
Why Kover rather than JaCoCo?
Kover is JetBrains' coverage tool for Kotlin, understands Kotlin constructs such as inline functions and coroutines, and configures in a few lines of build.gradle.kts. JaCoCo works too, and the prompt can name it if the team already reports to a JaCoCo-based service.
Will it change the CI command?
No. build-all.sh already runs ./gradlew clean check for each sample, and the agent attaches the report and verify tasks to check. The one workflow change is an artifact upload step so the HTML report can be downloaded.
Who picks the threshold?
The first measured run does. The agent sets line and branch minimums just under what the new suite reaches, states them in the pull request as a floor, and leaves raising them to the team.
Does it modify the application code?
No. Main.kt and application.conf stay as they are. Only the build file, the test file and the workflow change, so the pull request can be reviewed as a testing change.