All jobs

Write Unit Tests for a C++ Project Built with Bazel

Yes. An agent adds GoogleTest through Bzlmod, writes cc_test targets for the C++ libraries that have none, runs bazel test and opens a pull request.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

In a Bazel workspace the tests are the smaller half of the job; the other half is the build graph that lets them compile. The agent reads that graph first. In a module like cpp-tutorial/stage3 in bazelbuild/examples, MODULE.bazel is one line, bazel_dep(name = "rules_cc", version = "0.0.17"). main/BUILD defines a cc_library named hello-greet and a cc_binary named hello-world that depends on it and on //lib:hello-time. lib/BUILD defines hello-time with visibility = ["//main:__pkg__"]. Each rule is loaded from its own file, @rules_cc//cc:cc_library.bzl and @rules_cc//cc:cc_binary.bzl. There is no cc_test anywhere in the module, no .bazelversion in it, and the Buildkite config in .bazelci/tutorial-cpp.yml lists only build_targets for the three C++ stages on Linux, macOS and Windows.

It adds the test framework the way the module already declares dependencies: a bazel_dep on googletest from the Bazel Central Registry, marked dev_dependency = True so nothing that depends on this module inherits it. Each test target lives in the package of the library it tests. That is a convenience for hello-greet and a requirement for hello-time: its visibility admits only //main, and a target is always visible inside its own package, so a cc_test in lib/BUILD can depend on it without widening what the library exposes. The tests link @googletest//:gtest_main, load cc_test from @rules_cc//cc:cc_test.bzl to match the existing load style, and are named after the library with a _test suffix.

The two libraries need different kinds of test. get_greet in main/hello-greet.cc returns "Hello " plus its argument, so its test is plain value assertions: an ordinary name, an empty string, a name with spaces and one with non-ASCII bytes. print_localtime in lib/hello-time.cc takes nothing, returns nothing and writes std::asctime of the current time to std::cout. The agent tests it without changing it, by swapping the stream buffer of std::cout for a string stream during the call and asserting on the shape of what was written: one line, ending in a newline, with the fixed width asctime produces. It restores the buffer in a fixture's teardown so a failed assertion cannot leave the stream redirected for the next test.

Then it runs everything. With Bazelisk installed in the sandbox it runs bazel test //... --test_output=errors from the module directory, and bazel run //main:hello-world -- Bazel once to confirm the binary still builds and greets. Current GoogleTest needs C++17, and a compiler's default standard differs by platform, so the agent sets the standard for the build in a .bazelrc instead of relying on the sandbox's compiler. It records the Bazel version Bazelisk resolved, since the module pins none, and adds test_targets beside build_targets in the three stage 3 tasks of .bazelci/tutorial-cpp.yml so CI runs what the pull request adds.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "codex",
  "input": "Write unit tests for the C++ libraries in this Bazel module and open a pull request. Read MODULE.bazel, every BUILD file, every .cc and .h file, any .bazelrc and .bazelversion, and the CI config that builds this module first. Use bazel query to list the cc_library targets and which of them no cc_test depends on. Add googletest as a dev dependency in MODULE.bazel from the Bazel Central Registry. For each untested library add a cc_test in the same package as the library, loaded from rules_cc the way the existing rules are loaded, linked against gtest_main, and do not widen any target's visibility. Cover every public function declared in the headers: value assertions for functions that return something, and for functions that only write to standard output, capture the stream during the call, assert on the format of what was written and restore the stream in teardown. Do not change the library or binary sources. Set the C++ standard GoogleTest needs in a .bazelrc so the tests build the same on every platform the CI config lists. Install Bazelisk, run bazel test //... with --test_output=errors until it passes, run the binary once, and add the test targets to the CI config for this module. In the pull request description list each new target with the functions it covers, the Bazel version used, and the bazel test summary.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

A pull request with a small, readable diff: one added line group in MODULE.bazel, a cc_test block in each of main/BUILD and lib/BUILD, two test source files, a .bazelrc with the language standard, and test_targets in the CI config. No library or binary source changes and no visibility attribute is touched. The description lists each new target with the functions it covers, the Bazel version that ran, and the bazel test summary line for each target.

A reviewer checks three things the summary cannot show. That the stream capture is restored on every path, including a failing assertion. That the assertions on print_localtime are about format only, since a test that compares against the clock fails at midnight or in another time zone. And that googletest is a dev dependency, so a consumer of the module does not resolve it. They also decide whether to pin a Bazel version with a .bazelversion file; the agent reports which version it used and leaves that choice to the team.

The description also names the limit of testing this code as written: a function that reads the clock and prints cannot have its value asserted. It proposes, without doing it, a seam that would allow that, a function that formats a given std::time_t, which a follow-up message to the same session can ask for.

Run it on every pull request that adds a C++ library

In the Project that holds the repository, add an automation with a github trigger on pull_request.opened and pull_request.synchronize. Its prompt tells the run to query the pull request's branch for cc_library targets that no cc_test depends on, write tests for the ones the pull request added, push them to that branch and comment with the bazel test summary, or comment that every library is covered. The trigger appends the pull request to the prompt and fills in no placeholder, so the rule lives in the prompt.

Each run starts in a fresh sandbox, so it pays for one cold build of the test framework; scoping the query to the packages the pull request changed keeps the rest short. The automation's max runs per hour bounds how many start during a busy afternoon; events past it wait as queued runs and start when the hour has room.

The use case this job follows, with its automation and trigger: Write the missing tests.

Why run it on Rush?

Adding the first test to a Bazel C++ module is mostly toolchain work: resolving a registry dependency, getting a compiler flag right on more than one platform, and waiting for the first cold build of the test framework. It is the kind of task that sits undone because it interrupts whatever else is on the developer's machine. The agent does it in its own sandbox, where a cold build costs nobody's attention, and the full bazel output, including the first failed attempts, is kept in the run's record.

The pull request is small enough to review in minutes, and the evidence that it works is not a claim: the session's transcript, read with GET /o/{org}/sessions/{id}/messages, shows each bazel test invocation and its result. A reviewer who wants to see why a flag was added can find the compiler error that prompted it.

Frequently asked questions

Does it work with a WORKSPACE file instead of MODULE.bazel?

Yes, with a different dependency step. In a repository that has not moved to Bzlmod the agent declares GoogleTest with http_archive and a checksum in the existing WORKSPACE, and leaves the migration to Bzlmod as a separate change. The test targets themselves are the same.

Why are the tests in the library's package and not a tests directory?

Because of visibility. A library visible only to one package cannot be depended on from a separate tests package without changing its visibility, and widening that for a test changes what production code may depend on. Same-package tests need no change.

Can it use Catch2 or another framework?

Yes. Name the framework in the prompt and the agent adds that module from the registry and links its main target. If the repository already has tests in some framework, it follows what is there without being told.

Will the tests run on Windows and macOS in CI?

They are written to. The agent can only execute them on Linux in its sandbox, so the description says which platforms were run and which were not, and the language standard is set per platform in .bazelrc because MSVC spells the flag differently from GCC and Clang.

More Bazel jobs

The same job on other stacks

Keep reading