All jobs

Add Snapshot Tests to a Go CLI's Rendered Output

Yes. An agent adds golden-file tests for what your Go CLI prints, one per input and style, with an update flag and a check that a changed render fails, in a PR.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

The agent reads how the command produces output and what is already tested, because a snapshot is only useful on a path that is deterministic. In a CLI laid out like charmbracelet/glow, main.go ends in executeCLI(cmd, src, w io.Writer): it reads the source, strips frontmatter with utils.RemoveFrontmatter, wraps a non-Markdown file in a code block with utils.WrapCodeBlock, builds a glamour renderer from utils.GlamourStyle(style, isCode), the width flag and a base URL, runs utils.ApplyTextSizing over the result and writes it to w unless the pager or the TUI was asked for. The existing tests are narrow: glow_test.go parses flags on rootCmd, url_test.go skips itself because it needs the network, and ui/ tests the TUI model and the pager's render. Nothing pins what glow README.md prints.

Three things decide the output besides the input. style and width are package-level variables set by flags and by validateOptions, which picks the notty style when stdout is not a terminal and a width of 80 when none is given. And utils.ApplyTextSizing reads GLOW_TEXT_SIZING, falling back to terminal detection, so a test that does not set it would render differently on a developer's terminal and in CI; ui/pager_test.go already sets that variable with t.Setenv. The new test does the same, sets the two variables for each case and restores them with t.Cleanup, and calls executeCLI with a source whose reader is the file and whose URL is its path, writing into a buffer.

Inputs go under testdata/: a Markdown file with headings, a list, a table, a fenced block and links, one with YAML frontmatter to cover the strip, and a small .go file to cover the code-block wrap. Each renders in the notty style, which carries no escape codes and diffs cleanly, and in dark at width 80, whose output includes ANSI sequences and is still a byte-exact file. The expected output lives beside each input as <name>.<style>.golden, and a -update flag on the test rewrites the files so a deliberate change to rendering is one command, go test ./... -run TestRenderGolden -update, followed by a diff a reviewer reads.

In its sandbox the agent runs go test ./... and go test -race ./..., the way .github/workflows/coverage.yml runs the suite, and golangci-lint run as the Taskfile.yaml lint task does, since build.yml and lint.yml reuse the shared workflows that build, lint and scan every push. Then it proves the test bites: it changes one heading style in a scratch copy, runs the test to capture the failure with its diff, and reverts, quoting that failure in the pull request. The committed goldens are regenerated once from clean code, never edited by hand.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "claude",
  "input": "Add golden-file snapshot tests for this CLI's rendered output. Read main.go, glow_test.go, utils/utils.go, utils/textsize.go, ui/pager_test.go, Taskfile.yaml and .github/workflows/coverage.yml first. Write a test in package main that calls executeCLI with a source built from each file under a new testdata directory, in the notty and dark styles at width 80, with GLOW_TEXT_SIZING set off and the style and width variables restored after each case, and compares the output to a .golden file beside the input. Add an -update flag that rewrites the goldens. Include a Markdown file with headings, a list, a table, a code block and links, one with frontmatter, and a .go file. Run go test ./..., go test -race ./... and golangci-lint run. Then change one style in a scratch copy, run the test to capture the failing diff, revert, and quote it. Open a pull request.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

What comes back is a pull request adding one test file in package main and a testdata/ directory with the inputs and their .golden files, and nothing else: no change to main.go, the flags or the renderer. The description quotes go test ./... with the new test listed, the race run, the lint run, and the failing diff from the scratch change so a reviewer sees what a rendering regression looks like.

The shared build workflow and coverage.yml run on the pull request, so the goldens are checked on the CI runner's Go as well as the sandbox's, which is the first place a non-deterministic render would show. The run's record keeps every command and the diff.

Before merging, a reviewer looks through the dark goldens once to confirm they are the intended rendering, since from then on they are the definition, and decides whether a style the team ships, such as light, should get its own set.

Run it on every pull request that touches rendering

Once merged, the test runs on every push on its own. Where an agent helps is the pull request that changes rendering on purpose, a glamour bump or a style tweak, where the goldens fail as designed. Add an automation in the Project with a github trigger on pull_request.opened and pull_request.synchronize and a prompt that runs the test, reads each failing diff, and posts one review comment explaining what changed in the output and whether it looks intended. The trigger adds the pull request event after the prompt, so no placeholder is needed.

For a change the author confirms, a follow-up to the same run asks it to regenerate the goldens with the update flag and push them to the pull request's branch, so the diff of expected output is reviewed in the same place as the code.

The use case this job follows, with its automation and trigger: Write the missing tests.

Why run it on Rush?

Snapshot tests are quick to write and easy to write wrong: a golden captured with the wrong width, a style that depends on the terminal, or an environment variable left set from the last case. The agent reads how validateOptions and ApplyTextSizing decide those things and pins each one in the test, then runs the suite in its own sandbox under the race detector and the linter the repository uses.

The failing diff from the scratch change stays with the run and the pull request quotes it, so a reviewer sees the test catch a regression rather than trusting that it would. The same request runs on every Go CLI a team keeps, in parallel, each reading its own render path.

Frequently asked questions

Why both notty and dark?

notty is what the command prints when piped, with no escape codes, so its goldens are readable in a diff. dark is what a terminal user sees, and its ANSI sequences are where a style regression shows. Both are byte-exact files; only the second is hard to read by eye.

Why set GLOW_TEXT_SIZING?

Because utils.ApplyTextSizing otherwise detects the terminal, and a golden captured on a terminal that supports text sizing would fail everywhere else. Setting it off per test, as ui/pager_test.go sets it on, makes the output the same in a sandbox, on a laptop and in CI.

How is a golden updated when rendering changes on purpose?

Run the test with the -update flag; it rewrites every .golden file from the current output. The regenerated files go into the same pull request as the change, so the reviewer reads the output diff next to the code diff.

Does this test the TUI?

No. It covers the non-interactive path, executeCLI writing to a writer, which is what glow file.md and a pipe use. The TUI under ui/ keeps its own model tests; a terminal-driven snapshot of it is a separate change.

More Go CLI jobs

The same job on other stacks

Keep reading