# Add Test Coverage Reporting to CI for a Rust CLI

Yes. An agent measures line coverage of a Rust CLI with cargo llvm-cov, including the binary its tests spawn, adds a CI job with a threshold and opens a PR.

[Run this on Rush](https://cloud.getrush.ai/run/add-coverage-to-ci/rust-cli/). Opens the console with this prompt ready to send, in a Project that has your repository.

## What the agent does

The agent reads how the crate is tested before it adds a job. In a command-line tool like sharkdp/fd, `Cargo.toml` names the `fd-find` crate at 10.5.0 on edition 2024 with `rust-version` 1.90.0, one `[[bin]]` called `fd` at `src/main.rs`, the features `use-jemalloc` and `completions` on by default, and `diff`, `tempfile`, `filetime` and `test-case` as dev-dependencies. `[profile.dev]` sets `debug = "line-tables-only"`. `tests/tests.rs` holds 109 `#[test]` functions, and `src` has unit tests of its own in `exec/mod.rs`, `filter/size.rs`, `filter/time.rs`, `walk.rs`, `fmt/mod.rs` and `fmt/input.rs`. Nothing in the repository mentions coverage: no workflow step, no badge, no `codecov.yml`.

One detail in `tests/testenv/mod.rs` decides the tooling. The integration tests do not call library code; `TestEnv::new` builds a temporary directory with a fake `.git`, an `.fdignore`, a `.gitignore` and a symlink, finds the executable through `CARGO_BIN_EXE_fd`, and `assert_output` spawns `fd` as a child process for every assertion. A coverage tool that only instruments the test executable would report `src` as nearly untested. `cargo llvm-cov` builds the binary with the same instrumentation flags, and its profile file pattern includes the process id, so each spawned `fd` writes its own profile and the report merges them. The agent proves this in the sandbox by running `cargo llvm-cov --all-features --locked --summary-only` and reading the per-file table: `src/walk.rs` and `src/filter` should show lines covered, not zero.

It also reads how `.github/workflows/CICD.yml` is written, because the new job must look like its neighbours. `permissions` is `contents: read` at the top, every `uses:` carries a version or a full commit SHA with a comment (`actions/checkout@v7.0.1`, `dtolnay/rust-toolchain@stable`, `actions/upload-artifact` pinned to the SHA of v7), checkouts pass `persist-credentials: false`, and the `build` job already runs the suite across fourteen targets while `min_version` runs clippy and the tests on the MSRV. The agent therefore adds one `coverage` job on `ubuntu-latest` with the same checkout step, `dtolnay/rust-toolchain@stable` with the `llvm-tools-preview` component, `cargo install cargo-llvm-cov --locked` so no new third-party action enters the file, then `cargo llvm-cov --all-features --locked --lcov --output-path lcov.info` and a second invocation that writes the summary table into `$GITHUB_STEP_SUMMARY`. The threshold is `--fail-under-lines` set a few points under the number it measured, so the job fails on a real drop and not on noise. The `lcov.info` goes up with the same pinned `upload-artifact` the build job uses.

The agent leaves `CHANGELOG.md` alone, because `CONTRIBUTING.md` asks for an entry when a change alters what `fd` does, and a CI job does not. It does read the rest of that file: a pull request must disclose AI involvement and the extent of it, so the body says the job was written by an agent and lists what it ran. It does not open the ticket `CONTRIBUTING.md` asks contributors to open first, because that is a maintainer's decision on their own repository and this run is on yours.

## Start it with one API call

`POST https://api.phoenix.so/o/{org}/sessions`

```json
{
  "agent": "claude",
  "input": "Add test coverage reporting to this Rust CLI's GitHub Actions workflow. Read Cargo.toml, .github/workflows/CICD.yml, tests/tests.rs and tests/testenv/mod.rs first. Install the stable toolchain with the llvm-tools-preview component and cargo-llvm-cov with cargo install --locked. Measure line coverage with cargo llvm-cov --all-features --locked --summary-only and confirm that code exercised only through the binary the integration tests spawn via CARGO_BIN_EXE_fd is counted; if it is not, find out why before adding anything. Add one coverage job to CICD.yml on ubuntu-latest that reuses the workflow's existing checkout and toolchain actions at the versions already pinned there, installs cargo-llvm-cov without adding a new third-party action, writes an lcov file, prints the summary into the step summary, fails under a line threshold a few points below what you measured, and uploads the lcov file with the upload-artifact action already used in the file. Keep permissions at contents: read and keep persist-credentials: false on the checkout. Do not change CHANGELOG.md, Cargo.toml, Cargo.lock or any source file. Open a pull request that shows the measured per-file table, states the threshold and why, and discloses that an agent wrote the change, as CONTRIBUTING.md asks.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
```

1. **Get an API key.** In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
2. **Send the request.** POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
3. **Read what came back.** Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

## What comes back

What comes back is a pull request that changes one file, `.github/workflows/CICD.yml`, adding a `coverage` job beside `ensure_cargo_fmt`, `lint_check`, `min_version` and `build`. The body carries the per-file table the agent measured in the sandbox, the line threshold it chose with the measured number it sits under, and a sentence that an agent wrote the job and what it ran, which `CONTRIBUTING.md` requires of any pull request where an AI tool was used.

When the workflow runs on the pull request, the job's step summary shows the same table, the run has an `lcov.info` artifact, and the threshold step passes or fails on its own line. The other jobs run as before: the formatting and clippy checks, the MSRV test and the fourteen-target build matrix are untouched, since the agent added a job rather than editing one.

A reviewer checks three things: that the coverage numbers for `src/walk.rs`, `src/filter` and `src/exec` are plausible given 109 integration tests that run the binary, that the threshold leaves room for ordinary churn, and that every action in the new job is pinned the way the rest of the file pins them.

## Run it on every Rust CLI repository without coverage

A team with several command-line crates starts one run per repository: a `POST /o/{org}/sessions` with this prompt and that repository in `resources`, with a `request_id` per repository so a retried request returns the run it already started. Each run measures its own crate in its own sandbox and sets its own threshold, so a tool whose tests spawn its binary gets a job that captures that, and one with ordinary unit tests gets a simpler one.

Up to the key's `max_concurrent` runs proceed at once. Give the key a `webhook_url` and Rush POSTs `run.completed` or `run.failed` when each turn ends, with `data.session_url` to read the pull request from.

## Why run it on Rush?

Adding coverage to a crate whose tests spawn its own binary is mostly a measurement question: does the tool see the child process or not. Answering it means installing `llvm-tools-preview` and `cargo-llvm-cov`, building the crate twice with instrumentation and reading the per-file table, which on a laptop displaces whatever you were building. In a Rush sandbox the agent does that with its own toolchain and keeps the table in the transcript, and the threshold in the pull request is a number it measured, not a guess.

A workflow that pins every action and runs on fourteen targets took care to write. The agent reads those conventions from the file and follows them, and a reviewer sees one job in one diff with the evidence for its threshold attached.

## Frequently asked questions

### Why cargo llvm-cov rather than tarpaulin?

Because the integration tests in `tests/tests.rs` run `fd` as a child process through `CARGO_BIN_EXE_fd`, and `cargo llvm-cov` instruments the binary Cargo builds for the tests and merges the profile each process writes. The agent confirms this by checking that files only reached through the binary show covered lines in the summary.

### Will the job slow down the release builds?

No. It is a separate job on `ubuntu-latest`, so it runs in parallel with the `build` matrix and is not a dependency of the tarball, Debian package or release steps. The `build` job keeps running the suite on every target; coverage is measured once, on Linux.

### Why not add a codecov upload?

It would mean a new third-party action and a token, and nothing in the repository asks for a hosted dashboard. The job uploads `lcov.info` as a workflow artifact and prints the summary into the step summary, which a reviewer can read on the run page. Adding a hosted service is a one-line follow-up if you want it.

### What about the Windows-only code paths?

Code under `#[cfg(windows)]` is not compiled on Linux, so it neither counts for nor against the line total. The threshold is set against the Linux measurement, which is the only one the job takes.

## More Rust CLI jobs

- [Add Criterion Benchmarks to a Rust CLI](https://getrush.ai/run/add-criterion-benchmarks/rust-cli/)
- [Add Dependabot to a Rust CLI](https://getrush.ai/run/add-dependabot/rust-cli/)
- [Add Fuzz Tests to a Rust CLI's Parser](https://getrush.ai/run/add-fuzz-tests/rust-cli/)

## The same job on other stacks

- [Add Test Coverage to CI for a Slack Bolt App](https://getrush.ai/run/add-coverage-to-ci/slack-bolt/)
- [Add Test Coverage to CI for a Streamlit App](https://getrush.ai/run/add-coverage-to-ci/streamlit/)
- [Add Test Coverage Reporting to Symfony CI](https://getrush.ai/run/add-coverage-to-ci/symfony/)

## Keep reading

- [How a Rush run works on your repository, from request to pull request](https://getrush.ai/cloud/)
- [Compare cloud agents, agent sandboxes and AI code review](https://getrush.ai/compare/)
- [Rush pricing: Free, Pro and Max](https://getrush.ai/pricing/)
