What the agent does
The agent starts from the build files. In a header-only library like jarro2783/cxxopts, CMakeLists.txt declares cxxopts as an INTERFACE library, reads the version from include/cxxopts.hpp (3.3.1, from the CXXOPTS__VERSION_* defines) and exposes five options, CXXOPTS_BUILD_EXAMPLES, CXXOPTS_BUILD_TESTS, CXXOPTS_ENABLE_INSTALL, CXXOPTS_ENABLE_WARNINGS and CXXOPTS_USE_UNICODE_HELP, the first four on by default when the project is built standalone. cmake/cxxopts.cmake holds the helpers behind them; cxxopts_enable_warnings appends -Wall -Werror -Wextra and friends to CMAKE_CXX_FLAGS. test/CMakeLists.txt builds options_test and options_test_noregex from main.cpp and options.cpp against the vendored catch.hpp, two no-exceptions smoke binaries on GCC and Clang, a libFuzzer target on Clang, and --build-and-test subprojects for find_package, add_subdirectory and pkg-config. On GCC and Clang the two parser test binaries already carry -fsanitize=undefined.
Nothing measures coverage. .github/workflows/build.yml configures with cmake .., runs make and ctest on Ubuntu; cmake.yml runs a Release build on Ubuntu 22.04 and 24.04 with g++-9, g++-10 and clang++, on macOS and on Windows with ClangCl, each ending in ctest -C Release --output-on-failure; cifuzz.yml runs the OSS-Fuzz harness and meson.yml covers the Meson build. The agent adds a sixth option, CXXOPTS_ENABLE_COVERAGE, off by default, and a cxxopts_enable_coverage function next to the warnings helper that adds --coverage with -O0 -g for GCC and Clang and prints a warning and does nothing on MSVC, the same compiler guard test/CMakeLists.txt uses for its no-exceptions targets. Because cxxopts is an INTERFACE target, the flags land on the test executables, which is where the header is compiled.
Then it builds that configuration in the sandbox: cmake -S . -B build-coverage -DCMAKE_BUILD_TYPE=Debug -DCXXOPTS_ENABLE_COVERAGE=ON, cmake --build build-coverage, ctest --test-dir build-coverage/test --output-on-failure, then gcovr --root . --filter include/ --print-summary --xml --html-details from the build directory. The filter matters: the test translation units include test/catch.hpp, which is far larger than the library, and without it the report is mostly Catch. It checks that the sanitizer flags and the coverage flags coexist in one build, and that the --build-and-test subprojects, which get only the compiler and standard passed through, still pass unchanged. The 52 TEST_CASEs in test/options.cpp are what the report measures against the 3,192 lines of include/cxxopts.hpp.
The CI change is one new job, coverage, in cmake.yml, on ubuntu-24.04 with the default g++: configure with the option on, build, test, pip install gcovr, run gcovr, upload the HTML report as a workflow artifact and the Cobertura XML to Codecov with codecov/codecov-action guarded by if: env.CODECOV_TOKEN != '', so the job passes on forks and before anyone creates a token. The existing matrix is untouched: the Release builds keep their warnings-as-errors flags, and coverage runs once, in Debug, on one compiler. .gitignore gains *.gcda, *.gcno and build-coverage/, and the README gets the configure line so a contributor can reproduce the numbers locally.
Start it with one API call
POST https://api.phoenix.so/o/{org}/sessions with this body:
{
"agent": "claude",
"input": "Add test coverage reporting to this CMake project. Read CMakeLists.txt, cmake/*.cmake, test/CMakeLists.txt and every workflow under .github/workflows first. Add an off-by-default CMake option that compiles the test executables with gcov instrumentation (--coverage -O0 -g) on GCC and Clang and is a no-op with a warning on MSVC, following the project's existing option and helper-function style. Build that configuration in Debug, run ctest, and produce a gcovr report filtered to the library's own headers and sources, with a text summary, Cobertura XML and HTML details. Add one CI job on Ubuntu that does the same, uploads the HTML report as an artifact and sends the XML to Codecov only when a CODECOV_TOKEN secret is set. Leave the existing build matrix and its flags unchanged, add the coverage artifacts to .gitignore, document the configure command in the README, and open a pull request whose description includes the gcovr summary from your run.",
"resources": [
{
"url": "https://github.com/your-org/your-app"
}
]
}- Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
- Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
- Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.
What comes back
What comes back is a pull request on its own branch with five kinds of change: the new option and helper in CMakeLists.txt and cmake/cxxopts.cmake, the coverage job in .github/workflows/cmake.yml, the .gitignore entries, a README paragraph, and nothing in test/. The library header and the tests are not edited; coverage is a build configuration, not a code change.
The description leads with the --print-summary block from the sandbox build, lines and branches for include/cxxopts.hpp, then the ctest summary showing every existing test still passes with the instrumented binaries, and a list of the configure, build and gcovr commands so a reviewer can rerun them. The full command output is in the run's transcript at GET /o/{org}/sessions/{id}/messages, and the HTML report from the sandbox is attached to the session as an artifact.
Before merging, a reviewer checks that the new job appears on the pull request's own checks and ran green, opens the uploaded HTML artifact to see which parser paths are uncovered, and decides whether to add the Codecov token now or leave the guarded upload for later. A follow-up message to the same session can raise the bar: a --fail-under-line threshold on gcovr once the team agrees on a number.
Run it on every new C++ repository
A team with several CMake libraries and tools starts one run per repository with POST /o/{org}/sessions, this prompt and the repository in resources. The agent reads each project's own build: a project whose tests are plain executables registered with add_test gets the same treatment as one on Catch or GoogleTest, and a project that already builds with Clang gets llvm-cov gcov passed to gcovr through --gcov-executable. Each run opens its own pull request, and a request_id that is a UUID v5 of the repository name makes a retried start return the run already begun.
The key's webhook_url receives run.completed with a session_url for each run, so a script can read the pull request link and the first coverage summary from the session and post them where the team tracks test debt.
The use case this job follows, with its automation and trigger: Write the missing tests.
Why run it on Rush?
A coverage build is the kind of change that is easy to get subtly wrong on a laptop: flags that work with one compiler and not another, a report that is mostly Catch, a job that breaks on forks because a secret is missing. The agent builds the instrumented configuration in a clean sandbox with the project's own toolchain list in front of it, runs every existing test against it, and only then writes the workflow, so the first time the job runs on GitHub is not the first time the commands have run.
The pull request carries the commands and their output, and the run keeps the whole transcript, so a reviewer can see the exact gcovr invocation that produced the summary rather than trusting a number. Several C++ repositories can be done at once, each in its own sandbox.
Frequently asked questions
Will the coverage flags slow down or change the normal builds?
No. The option is off by default, so cmake .. with no arguments builds exactly as before, and the Release matrix in cmake.yml is not edited. Coverage is one extra job that configures its own build directory in Debug with the option on.
What about MSVC?
--coverage is a GCC and Clang flag. The helper checks CMAKE_CXX_COMPILER_ID, the same way test/CMakeLists.txt guards its -fno-exceptions targets, and on MSVC it prints a warning and leaves the flags alone. The Windows job in the matrix never turns the option on.
Why gcovr rather than lcov?
Either works. gcovr is one pip install, reads both GCC's gcov and llvm-cov gcov output, and writes the text summary, Cobertura XML and HTML details from a single command, which keeps the workflow job short. If your project already uses lcov, say so in the prompt and the agent uses that instead.
Does the report count the test framework or the tests themselves?
Not with the filter the agent sets. test/catch.hpp is compiled into every test binary and would dominate an unfiltered report, so gcovr is run with --filter include/, and the summary describes cxxopts.hpp only. If you want test files counted too, the filter is one line in the workflow.
Can it fail the build when coverage drops?
Yes, once you choose a number. gcovr takes --fail-under-line and --fail-under-branch, and the pull request leaves those out on purpose so the first run reports rather than blocks. A follow-up on the same session adds the threshold.
More CMake jobs
The same job on other stacks
- Add Test Coverage Reporting to a Cobra CLI's CI (Cobra)
- Add Test Coverage to CI for a Django REST Framework API (Django REST Framework)
- Add Code Coverage to CI for a .NET Solution (.NET)