All jobs

Add Test Coverage Reporting to Quarkus CI

Yes. An agent adds JaCoCo to a Quarkus Maven build through quarkus-jacoco, measures coverage in its sandbox, sets a CI check at that number and opens a PR.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

The agent reads the build before touching it. In a Quarkus service shaped like the getting-started module of quarkusio/quarkus-quickstarts, pom.xml imports quarkus-bom 3.40.1.1, depends on quarkus-rest, and for tests on quarkus-junit and rest-assured; the maven-surefire-plugin is pinned at 3.5.4 with java.util.logging.manager set to the JBoss log manager, the quarkus-maven-plugin has extensions on, and a native profile wires maven-failsafe-plugin for the *IT classes. GreetingResourceTest is a @QuarkusTest that calls /hello and /hello/greeting/{name} with REST Assured, GreetingResourceIT extends it for the native image, and mvnw wraps Maven 3.9.11. There is no JaCoCo configuration and nothing in the build reads a coverage number, so a drop in coverage is invisible until someone looks.

Quarkus has its own way of doing this, and the same repository shows it in tests-with-coverage-quickstart. The agent follows that module rather than the plain JaCoCo setup a Spring project would use: it adds quarkus-jacoco as a test-scoped dependency, which instruments the classes Quarkus transforms at test time and writes target/jacoco-quarkus.exec plus a report in target/jacoco-report, and adds jacoco-maven-plugin 0.8.14 with the prepare-agent execution configured as that module does, exclClassLoaders set to *QuarkusClassLoader, destFile pointing at the same .exec file and append on, so plain JUnit tests and @QuarkusTest classes land in one data file. A report execution in post-integration-test reads that file, and a check execution with a BUNDLE rule on LINE and BRANCH covered ratios fails verify under a minimum. The agent runs ./mvnw verify in the sandbox, reads the totals from target/jacoco-report/jacoco.csv, and writes the minimums at the measured figures rounded down, so the pull request passes on the code as it is and only a drop fails later.

Then it adds the workflow the module does not have. .github/workflows/ci.yml runs on pull requests and pushes to main with actions/setup-java on Temurin 21, the version the repository's own build-pr-development.yml uses, caches ~/.m2/repository, runs ./mvnw -B verify, and uploads target/jacoco-report and target/surefire-reports as artifacts, named with the run so legs do not collide if a matrix is added later. The JaCoCo threshold lives in two properties at the top of pom.xml, jacoco.line.minimum and jacoco.branch.minimum, so raising it later is a one-line change that the pull request's checks will verify.

The agent does not write new tests in this run and does not change the native profile: the *IT classes run against a native executable where JaCoCo cannot instrument, which the coverage quickstart's README states, so the check is bound to the JVM test run only. It also leaves the surefire configuration alone, since quarkus-jacoco picks up the agent through the plugin's argLine property without further wiring.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "codex",
  "input": "Add JaCoCo coverage reporting and a coverage check to this Quarkus project and open a pull request. Read pom.xml, the test classes under src/test/java and any workflow under .github/workflows first. Add quarkus-jacoco as a test-scoped dependency and jacoco-maven-plugin with a prepare-agent execution that excludes *QuarkusClassLoader, writes target/jacoco-quarkus.exec with append on, a report execution in post-integration-test writing target/jacoco-report, and a check execution with BUNDLE LINE and BRANCH COVEREDRATIO limits read from two pom properties. Run ./mvnw verify in the sandbox, read the line and branch totals from target/jacoco-report/jacoco.csv, and set the two properties to those figures rounded down. Add .github/workflows/ci.yml that runs ./mvnw -B verify on Temurin 21 with the Maven repository cached and uploads target/jacoco-report and target/surefire-reports as artifacts. Do not add tests and do not change the native profile. Put the measured totals and the verify summary in the pull request description.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

What comes back is a pull request that changes pom.xml and adds .github/workflows/ci.yml. The description states the line and branch totals measured in the sandbox, the JDK they were measured on, and the ./mvnw verify summary with the jacoco-check execution passing. The session keeps the full Maven log, and the workflow run on the pull request attaches the HTML report as an artifact a reviewer can download and open.

A reviewer checks that the two thresholds match the numbers in the description, that quarkus-jacoco is test-scoped so nothing reaches the production jar, and that the native profile is unchanged. The pull request's own ci.yml run is the first time the check executes on GitHub, and it should pass because the threshold was set from the same command on the same code.

Run it on every new Quarkus service repository

Run it once per service rather than once: a script that reads GET /o/{org}/github for the installation's repositories and calls POST /o/{org}/sessions with this body and each repository's URL in resources, with a request_id per repository so a retry returns the run already started. Each service gets its own sandbox, its own measurement and its own threshold.

Give the key a webhook_url and Rush POSTs run.completed, run.failed or run.cancelled when each turn ends, signed per Standard Webhooks, with data.session_url to read for the pull request. A key runs at most max_concurrent sessions at once, 4 by default, but a batch over many Quarkus services does not have to pace itself: runs past that number answer 202 with status: "queued" and start as the key's slots free.

The use case this job follows, with its automation and trigger: Write the missing tests.

Why run it on Rush?

A coverage threshold is only useful when it comes from the real number, and in Quarkus the real number depends on getting quarkus-jacoco and the class-loader exclusion right, because the bytecode Quarkus transforms at test time is otherwise reported as uncovered. The agent runs ./mvnw verify in its own sandbox with the configuration the framework's own quickstart uses, reads the CSV the report writes, and the Maven output stays on the session, so the threshold in the pull request has a recorded source.

The job ends at a pull request with a workflow that has already run once on GitHub, not at a snippet of XML. The same request body works on every Quarkus service a team owns, each with its own measured figure, and a request to raise the threshold or add a matrix of JDKs is a follow-up on the same parked session with its ~/.m2 already filled.

Frequently asked questions

Why quarkus-jacoco rather than the JaCoCo plugin alone?

Because Quarkus transforms classes when it starts a @QuarkusTest, and the plain agent attributes that bytecode wrongly or not at all. The quarkus-jacoco extension instruments the transformed classes and writes to the same .exec file, which is why the extension's own quickstart pairs it with the plugin and excludes *QuarkusClassLoader in prepare-agent.

Where does the threshold number come from?

From ./mvnw verify in the sandbox and the LINE_COVERED and BRANCH_COVERED columns of target/jacoco-report/jacoco.csv, totalled over the bundle. The figures are rounded down and written into two pom.xml properties, and the pull request description states them.

Will it write tests to raise coverage?

Not in this run. It measures what the existing @QuarkusTest classes cover and stops a drop. Raising coverage is a separate request, and the HTML report attached to the workflow run shows which classes to start with.

Does it cover the native integration tests?

No. JaCoCo cannot instrument a native executable, so the *IT classes under the native profile are outside the measurement and the check is bound to the JVM run of verify. The profile is left exactly as it was.

More Quarkus jobs

The same job on other stacks

Keep reading