All jobs

Add Test Coverage Reporting to Go CI

A Rush agent measures your Go coverage in its sandbox, adds a threshold at that number and a coverage artifact to CI, and opens a pull request with passing checks.

Run this on RushIn a Project that has your repository.

What the agent does

In a Go service like gothinkster/golang-gin-realworld-example-app, coverage is already half there, and the agent finds that before writing anything. .github/workflows/ci.yml has a test job on a Go matrix of 1.21, 1.22 and 1.23 that runs bash ./scripts/gofmt.sh and then bash ./scripts/coverage.sh, and uploads ./coverage.out to Coveralls with coverallsapp/github-action@v2 on the 1.23 leg only. scripts/coverage.sh writes mode: atomic, loops over every directory with .go files, runs go test -coverprofile=profile.out -covermode=atomic in each and appends the profiles. Nothing reads the total, nothing fails when it drops, and the report is not kept as an artifact.

The agent runs the script in the sandbox to get today's number. The module is Go 1.21 with gin v1.10.0 and gorm with gorm.io/driver/sqlite, which pulls in mattn/go-sqlite3 and so needs cgo and a C compiler; common/database.go writes its test database to ./data/gorm_test.db. The unit tests live in articles/unit_test.go, common/unit_test.go and users/unit_test.go. It runs go tool cover -func=coverage.out and reads the total line, which is the figure the threshold will use.

Then it changes the workflow. After the coverage step it adds a step that runs go tool cover -func=coverage.out, parses the total percentage and exits non-zero when it is below the measured figure, written into the workflow as an environment variable so raising it later is a one-line change. It adds actions/upload-artifact for coverage.out and an HTML report from go tool cover -html, named per Go version so the three matrix legs do not overwrite each other. The Coveralls step and its permissions stay as they are.

It keeps scripts/coverage.sh rather than replacing it with go test ./... -coverprofile, since the existing job and Coveralls already consume its output; a rewrite would be a separate change. The threshold is set to the number measured now, rounded down, so the pull request passes on the code as it is and only a drop fails later.

Start it with one API call

POST https://api.prix.dev/o/{org}/sessions with this body:

{
  "agent": "codex",
  "input": "Run scripts/coverage.sh in the sandbox and read the total from go tool cover -func=coverage.out. In .github/workflows/ci.yml, after the existing coverage step, add a step that fails the job if total coverage falls below that figure (rounded down, kept in one env variable), and upload coverage.out plus an HTML report from go tool cover -html as an artifact named per Go version. Keep scripts/coverage.sh and the Coveralls step. Open a pull request with the measured number in its description and make sure its checks pass.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.prix.dev/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

The pull request changes .github/workflows/ci.yml only: one threshold step, one artifact upload and an environment variable holding the threshold. Its description states the total measured in the sandbox, the Go version it was measured on, and how the check parses go tool cover output. The pull request's own test job runs on all three Go versions, and each leg shows the threshold step passing and a coverage artifact attached to the workflow run.

A reviewer checks two things: that the threshold matches the number in the description, and that it is the same across matrix legs (coverage can differ slightly between Go versions; if it does, the agent sets the threshold at the lowest leg and says so). The lint, security and api-test jobs are untouched and should pass as before.

Run it on every new Go service repository

When a team creates a Go repository from a template, have the automation that creates it also send POST /o/{org}/sessions with this body and the new repository in resources. A webhook on the key reports run.completed when the pull request is open.

For existing services, send one run per repository; each measures its own number, so no repository gets a threshold borrowed from another.

The use case this job follows, with its automation and trigger: Write the missing tests.

Why run it on Rush?

Setting a coverage threshold well needs the real number from the real test command, measured on the same Go version CI uses, which for a cgo dependency like go-sqlite3 also means a working C toolchain. The agent measures it in its own sandbox and keeps the go test and go tool cover output in the transcript, so the threshold in the workflow has a recorded source.

The job ends at a pull request whose checks have run, not a suggested YAML snippet. Many services can be done at once within the key's concurrency limit, each with its own measured threshold.

Frequently asked questions

My workflow already uploads to Coveralls. Will it remove that?

No. This repository already sends coverage.out to Coveralls on the Go 1.23 leg, and the agent leaves that step and its token as they are. It adds a threshold and a downloadable artifact beside it.

Where does the threshold number come from?

From running scripts/coverage.sh in the sandbox and reading the total line of go tool cover -func. It is rounded down and written into the workflow, and the pull request description states the measured value.

Will it write tests to raise coverage?

Not in this run. It measures what exists and stops a drop. Raising coverage is a separate request, and the HTML artifact shows which files to start with.

Does it need a third-party coverage service?

No. The threshold check uses go tool cover, which ships with Go, and the report is a GitHub Actions artifact.