What the agent does
The agent starts with what is tested and what is not. In a Cobra project like spf13/cobra-cli, the cmd package has three real tests: TestGoldenAddCmd and TestGoldenInitCmd generate a project and a command and compare the files with the golden files under cmd/testdata/, using compareFiles from golden_test.go, which shells out to diff -u when the contents differ, and TestValidateCmdName is a table of nine inputs for the dash and underscore handling in add.go. helpers_test.go holds one helper, assertNoErr, and project_test.go is a single comment, todo: write tests. go.mod declares Go 1.15 with cobra v1.6.1 and viper v1.12.0, the Makefile's test target runs richgo test -v ./..., and .github/workflows/test.yml runs it on Ubuntu and macOS for Go 1.14 to 1.17 and on Windows under MSYS2, so a new test must pass on all three and must not use anything newer than Go 1.14, which rules out t.TempDir and leaves ioutil.TempDir, the function init_test.go already uses.
What lacks tests is readable from the source. licenses.go has matchLicense, which compares a name case-insensitively against every license's PossibleMatches, so Apache-2.0, apache20 and apache 2.0 all map to apache, GPL-3.0, gplv3 and gpl to gpl3, BSD-2-Clause and simple bsd to freebsd, and MIT to mit; getLicense, which takes the --license flag first, then a custom license.header and license.text from viper, then the license key, and falls back to none; and copyrightLine, which reads author and a year key viper holds for tests, as its own comment says. project.go has Project.Create, which makes the directory, writes main.go and cmd/root.go from the templates in tpl/main.go and the LICENSE from the license text, and Command.Create, which writes cmd/<name>.go. init.go has fileToURL, which turns a path split on the OS separator into a URL path, and initializeProject, whose . argument keeps the working directory. findLicense and parseModInfo end in cobra.CheckErr, which exits the process on failure, so only their success paths are testable without a subprocess.
The agent writes cmd/licenses_test.go with a table over every PossibleMatches entry, in its own case and upper-cased, asserting the key matchLicense returns, plus "" and an unknown name returning the empty string; a test for getLicense that sets userLicense, then viper keys one at a time with viper.Reset() between cases, checking the flag wins over the config and a custom header and text come back verbatim; and a test for copyrightLine with author and year set through viper. It fills cmd/project_test.go: Project.Create into an ioutil.TempDir with Viper true, the Apache license and a fixed year, comparing main.go, cmd/root.go and LICENSE with the three golden files through compareFiles, a case with the none license checking that the LICENSE file is empty, and Command.Create with a parent other than rootCmd checking the generated file names that parent. cmd/init_test.go gains a table for fileToURL with nested paths and a case for initializeProject with . returning the working directory. Every test uses t.Run with a name, resets viper in a defer, and removes what it wrote.
Then the agent runs the suite the way CI does: go vet ./..., go test -v ./... and, with richgo installed, make test, which also runs go get -v ./... through install_deps. It runs go test -race ./... once to show the viper resets do not race, and go test -cover ./cmd before and after to put the coverage change in the pull request. One deliberate break, swapping the key matchLicense returns for gpl, shows the new table failing, and it is restored before the commit.
Start it with one API call
POST https://api.phoenix.so/o/{org}/sessions with this body:
{
"agent": "claude",
"input": "Write unit tests for the untested code in this Cobra CLI. Read go.mod, the Makefile, .github/workflows/test.yml, every file under cmd/ including the existing tests and testdata, and tpl/main.go first. Add cmd/licenses_test.go with a table over every PossibleMatches entry for matchLicense in its own case and upper-cased, plus empty and unknown names, a test of getLicense covering the flag, a custom header and text, the license key and the none default with viper reset between cases, and a test of copyrightLine with author and year set through viper. Fill cmd/project_test.go with Project.Create into a temporary directory compared against the golden files, a none-license case with an empty LICENSE, and Command.Create with a non-root parent. Add a fileToURL table and an initializeProject dot case to cmd/init_test.go. Use ioutil.TempDir, not t.TempDir, so Go 1.14 still builds. Run go vet, go test -v ./..., go test -race, make test with richgo installed, and go test -cover on the cmd package before and after, then break matchLicense on purpose to show a failure and restore it. Open a pull request with the commands and their output.",
"resources": [
{
"url": "https://github.com/your-org/your-app"
}
]
}- Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
- Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
- Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.
What comes back
The pull request adds cmd/licenses_test.go and changes cmd/project_test.go and cmd/init_test.go; no non-test file changes. Its description quotes the go test -v ./... output listing every subtest, the -race run, the coverage figure for cmd before and after, the failing run with matchLicense broken and the passing run after it is restored, and the make test output through richgo.
A reviewer checks that each table names the license key and not only the alias, that viper is reset so the golden tests, which set useViper and license globally, still pass in any order, and that nothing in the new tests needs Go 1.15 or later. After merging, the Test workflow runs the new tests on every push on all three operating systems.
Run it on every nightly schedule
An automation in the Project that holds the repository takes a schedule trigger with cron set to a night-time hour. Its runs are read-only, so the saved prompt asks the run to compute go test -cover per package, pick the one with the least coverage, write tests for it in a scratch copy, run them and answer with the diff and the coverage change, which the key's session read or the console shows in the morning. A run that should open the pull request is started from the team's own server with this body and a request_id that is a UUID v5 of the package path, so a retried start returns the run already started.
A module with several packages gets one such run per package, each with its own input naming the path.
The use case this job follows, with its automation and trigger: Write the missing tests.
Why run it on Rush?
Tests for a code generator are only worth merging once they have run against the generator, with the golden files, the viper state the other tests share and the diff the comparison helper calls. The agent installs the toolchain in its own sandbox, runs vet, the tests, the race detector and the Makefile target, breaks the code on purpose to show a red run and keeps every command and its output in the run's record, so the pull request shows what the tests catch.
The tests follow the package's own habits, compareFiles and the golden files, assertNoErr, ioutil.TempDir and viper's year key, and respect the Go 1.14 floor the CI matrix sets. Ten CLIs get ten pull requests, each testing that project's own untested functions.
Frequently asked questions
Why not test findLicense and parseModInfo too?
Both call cobra.CheckErr on failure, which prints the error and exits the process, so a failing case cannot be asserted inside the test binary. Their success paths are covered through getLicense and initializeProject; a subprocess test for the exits is a separate change the pull request names.
Will the new tests clash with the golden tests?
The golden tests set useViper and license through viper and never reset them, so the new tests reset viper before and after each case and restore the userLicense variable. The pull request's go test run executes the whole package together and with -race to show the order does not matter.
Why ioutil.TempDir instead of t.TempDir?
The CI matrix in test.yml builds with Go 1.14, and t.TempDir arrived in 1.15, so a test using it would fail to compile on one row of the matrix. init_test.go already uses ioutil.TempDir, and the new tests do the same with a deferred os.RemoveAll.
Does this change any generator output?
No. Only test files change, and the golden files under cmd/testdata/ are read, not rewritten. The generated main.go, cmd/root.go and LICENSE are compared against them exactly as the existing tests do.