All jobs

Review Pull Requests on a CMake C++ Library

Yes. An agent builds the pull request with the library's own CMake options and warning flags, runs ctest and clang-format, and posts one review citing file and line.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

The agent checks out the pull request's branch into its own cloud sandbox and reads how the library expects to be built before it judges a diff. In a library like gabime/spdlog, CMakeLists.txt declares cmake_minimum_required(VERSION 3.10...3.21), defaults the C++ standard to 11 unless SPDLOG_USE_STD_FORMAT raises it to 20, and exposes a long list of options: SPDLOG_BUILD_TESTS, SPDLOG_BUILD_TESTS_HO, SPDLOG_BUILD_WARNINGS, SPDLOG_FMT_EXTERNAL, SPDLOG_NO_EXCEPTIONS, SPDLOG_SANITIZE_ADDRESS, SPDLOG_SANITIZE_THREAD, SPDLOG_TIDY and the WCHAR family. cmake/utils.cmake defines spdlog_enable_warnings, which under SPDLOG_BUILD_WARNINGS compiles with -Wall -Wextra -Wconversion -pedantic -Werror on GCC and Clang and /W3 /WX on MSVC. .github/workflows/linux.yml builds a matrix of GCC 9, 11 and 12 and Clang 12 and 15 across C++11, 17 and 20 with one ASAN and one TSAN row, installs CMake 3.24.2, configures with -DSPDLOG_BUILD_WARNINGS=ON -DSPDLOG_BUILD_TESTS=ON and runs make -j 4 then ctest -j 4 --output-on-failure; macos.yml adds a std::format row and windows.yml the Visual Studio generators. tests/CMakeLists.txt lists every test_*.cpp, fetches Catch2 v3.5.0 when no package is found, runs the executable with --order decl and RUN_SERIAL ON, and adds test_errors.cpp only when exceptions are on.

The library ships two ways, and that shapes the review. include/spdlog/common.h defines SPDLOG_HEADER_ONLY unless SPDLOG_COMPILED_LIB is set; each public header has an -inl.h twin, and src/spdlog.cpp, src/async.cpp, src/color_sinks.cpp, src/file_sinks.cpp, src/stdout_sinks.cpp and src/cfg.cpp include those twins into the compiled spdlog::spdlog target, while spdlog::spdlog_header_only is an interface target. So a change that adds a function to a header without its -inl.h definition, or a definition the compiled sources never include, builds in one mode and fails to link in the other, and the agent checks both rather than trusting the one the author tried.

For each pull request the agent configures two builds in the sandbox: GCC at C++11 with the compiled library, warnings and tests on, and Clang at C++20 with SPDLOG_USE_STD_FORMAT=ON and SPDLOG_BUILD_TESTS_HO=ON, which is the header-only path. It runs ctest --output-on-failure on each and quotes the result. It runs clang-format --dry-run -Werror on the changed files with the repository's .clang-format, Google style with a four-space access modifier offset, since scripts/format.sh is what the maintainers run and a reformatting diff hides the real change. Then it reads the diff with the files it depends on: a new sink against the existing sink's sink_it_ and flush_ contract and its -inl.h, a change in pattern_formatter against the flags test_pattern_formatter.cpp covers, anything that throws against SPDLOG_NO_EXCEPTIONS, any size_t to int narrowing that -Wconversion will turn into an error on the warnings rows, and any new header for whether tests/CMakeLists.txt needs a test file added.

The review is one GitHub review, not a stream of comments. It opens with the two build and test results and the format check, then lists blocking findings and suggestions separately, each with a file and line: a definition missing from the compiled path, a behaviour change without a test, a wchar_t path that only Windows compiles, an option added to CMakeLists.txt without a line in the README's option list. A compile failure is quoted as the finding rather than worked around, and the agent pushes nothing to the branch.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "claude",
  "input": "Review the open pull request on this CMake C++ library. Check it out and read CMakeLists.txt, cmake/utils.cmake, tests/CMakeLists.txt, .clang-format and .github/workflows/linux.yml first; in a library shaped like gabime/spdlog, build twice in your sandbox with warnings and tests on, once with GCC at C++11 against the compiled library and once with Clang at C++20 with SPDLOG_USE_STD_FORMAT and the header-only tests, run ctest --output-on-failure on both, run clang-format --dry-run -Werror on the changed files, and quote each result. Read the diff with the files it depends on: whether every new declaration has its definition in the matching -inl.h and reaches the compiled sources, whether a thrown exception respects SPDLOG_NO_EXCEPTIONS, whether a narrowing conversion will fail the -Wconversion -Werror rows, and whether a changed behaviour has a test in tests/. Post one GitHub review that cites file and line for every finding, separates blocking issues from suggestions, and includes the command results. Do not push commits.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app/pull/1"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

What comes back is a review on the pull request and the full command log on the session. The review opens with the two configure, build and ctest results and the clang-format check, then lists blocking findings and suggestions, each with a file and line: a function declared in a header with no -inl.h body, a throw on a path the no-exceptions build compiles, a -Wconversion error the author's compiler did not raise, a sink change with no test. A failure in either build is quoted as the result rather than worked around.

Before acting on it, a maintainer reads the compiled-versus-header-only findings first, since those break downstream builds in whichever mode the author did not try, then the warning findings, and treats the style items as optional. The session's result holds the review text, and the transcript shows every command with its output, so a finding can be traced to the compiler line that produced it.

Run it on every pull request opened

Create an automation in the Project with this prompt and a github trigger on pull_request.opened and pull_request.synchronize, with read_only on so its runs can comment but never push. The trigger adds the pull request event after the prompt, and each push to the branch starts a fresh run against the new head. Set max_runs_per_hour to match how often contributors push, since two full builds of a library with this many options take a while per run.

To drive it from your own server instead, receive the GitHub webhook, put the pull request URL in resources so the clone is the head commit, and send this body with a request_id that is a UUID v5 of the delivery id. A webhook URL on the key then receives run.completed with session_url when the review is posted; a review past the key's max_concurrent answers 202 with status: "queued" and starts, oldest first, when a slot frees.

The use case this job follows, with its automation and trigger: Review every pull request.

Why run it on Rush?

A C++ library review is only as good as the compilers it was tried on, and the CI matrix here is the maintainers' statement of what counts: several GCC and Clang releases, three standards, sanitizers, two ways of linking. The agent builds the two configurations that catch the most in its own sandbox and keeps every compiler line and test result in the run's record, so the review quotes an error rather than guessing a change looks fine.

The same run fits any CMake project: an application with presets in CMakePresets.json, a library on Conan or vcpkg where the review builds against the pinned dependency versions, or a header-only library where the check is that every header compiles alone. Each gets the project's own options and warning flags, and a review that cites what its own build said.

Frequently asked questions

Does it build the whole CI matrix?

No. It builds the two configurations that disagree most, the compiled library at the oldest standard and the header-only path at the newest with std::format, and names the rows it did not run. A follow-up can ask for the sanitizer rows on a change that touches threading, such as the async logger or the MPMC queue.

What does it check on a new sink?

That the class implements sink_it_ and flush_ like the existing sinks, that its definitions live in a matching -inl.h the compiled sources include, that it compiles with exceptions off, and that a test_*.cpp exercises it and is listed in tests/CMakeLists.txt. Each miss is a finding with the file and line it belongs at.

Will it reformat the pull request?

No. It runs clang-format in dry-run mode with the repository's style and reports the lines that differ, so the author runs scripts/format.sh themselves. The agent never pushes to the branch.

What if a dependency fetch fails in the sandbox?

The tests fetch Catch2 from GitHub at a pinned commit when no package is installed. If that fetch fails, the review says the tests could not be built and still reports the two library builds, the format check and the reading of the diff, rather than posting nothing.

More CMake jobs

The same job on other stacks

Keep reading