What the agent does
The agent reads how the crate is tested before it adds a job. In a command-line tool like sharkdp/fd, Cargo.toml names the fd-find crate at 10.5.0 on edition 2024 with rust-version 1.90.0, one [[bin]] called fd at src/main.rs, the features use-jemalloc and completions on by default, and diff, tempfile, filetime and test-case as dev-dependencies. [profile.dev] sets debug = "line-tables-only". tests/tests.rs holds 109 #[test] functions, and src has unit tests of its own in exec/mod.rs, filter/size.rs, filter/time.rs, walk.rs, fmt/mod.rs and fmt/input.rs. Nothing in the repository mentions coverage: no workflow step, no badge, no codecov.yml.
One detail in tests/testenv/mod.rs decides the tooling. The integration tests do not call library code; TestEnv::new builds a temporary directory with a fake .git, an .fdignore, a .gitignore and a symlink, finds the executable through CARGO_BIN_EXE_fd, and assert_output spawns fd as a child process for every assertion. A coverage tool that only instruments the test executable would report src as nearly untested. cargo llvm-cov builds the binary with the same instrumentation flags, and its profile file pattern includes the process id, so each spawned fd writes its own profile and the report merges them. The agent proves this in the sandbox by running cargo llvm-cov --all-features --locked --summary-only and reading the per-file table: src/walk.rs and src/filter should show lines covered, not zero.
It also reads how .github/workflows/CICD.yml is written, because the new job must look like its neighbours. permissions is contents: read at the top, every uses: carries a version or a full commit SHA with a comment (actions/checkout@v7.0.1, dtolnay/rust-toolchain@stable, actions/upload-artifact pinned to the SHA of v7), checkouts pass persist-credentials: false, and the build job already runs the suite across fourteen targets while min_version runs clippy and the tests on the MSRV. The agent therefore adds one coverage job on ubuntu-latest with the same checkout step, dtolnay/rust-toolchain@stable with the llvm-tools-preview component, cargo install cargo-llvm-cov --locked so no new third-party action enters the file, then cargo llvm-cov --all-features --locked --lcov --output-path lcov.info and a second invocation that writes the summary table into $GITHUB_STEP_SUMMARY. The threshold is --fail-under-lines set a few points under the number it measured, so the job fails on a real drop and not on noise. The lcov.info goes up with the same pinned upload-artifact the build job uses.
The agent leaves CHANGELOG.md alone, because CONTRIBUTING.md asks for an entry when a change alters what fd does, and a CI job does not. It does read the rest of that file: a pull request must disclose AI involvement and the extent of it, so the body says the job was written by an agent and lists what it ran. It does not open the ticket CONTRIBUTING.md asks contributors to open first, because that is a maintainer's decision on their own repository and this run is on yours.
Start it with one API call
POST https://api.phoenix.so/o/{org}/sessions with this body:
{
"agent": "claude",
"input": "Add test coverage reporting to this Rust CLI's GitHub Actions workflow. Read Cargo.toml, .github/workflows/CICD.yml, tests/tests.rs and tests/testenv/mod.rs first. Install the stable toolchain with the llvm-tools-preview component and cargo-llvm-cov with cargo install --locked. Measure line coverage with cargo llvm-cov --all-features --locked --summary-only and confirm that code exercised only through the binary the integration tests spawn via CARGO_BIN_EXE_fd is counted; if it is not, find out why before adding anything. Add one coverage job to CICD.yml on ubuntu-latest that reuses the workflow's existing checkout and toolchain actions at the versions already pinned there, installs cargo-llvm-cov without adding a new third-party action, writes an lcov file, prints the summary into the step summary, fails under a line threshold a few points below what you measured, and uploads the lcov file with the upload-artifact action already used in the file. Keep permissions at contents: read and keep persist-credentials: false on the checkout. Do not change CHANGELOG.md, Cargo.toml, Cargo.lock or any source file. Open a pull request that shows the measured per-file table, states the threshold and why, and discloses that an agent wrote the change, as CONTRIBUTING.md asks.",
"resources": [
{
"url": "https://github.com/your-org/your-app"
}
]
}- Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
- Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
- Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.
What comes back
What comes back is a pull request that changes one file, .github/workflows/CICD.yml, adding a coverage job beside ensure_cargo_fmt, lint_check, min_version and build. The body carries the per-file table the agent measured in the sandbox, the line threshold it chose with the measured number it sits under, and a sentence that an agent wrote the job and what it ran, which CONTRIBUTING.md requires of any pull request where an AI tool was used.
When the workflow runs on the pull request, the job's step summary shows the same table, the run has an lcov.info artifact, and the threshold step passes or fails on its own line. The other jobs run as before: the formatting and clippy checks, the MSRV test and the fourteen-target build matrix are untouched, since the agent added a job rather than editing one.
A reviewer checks three things: that the coverage numbers for src/walk.rs, src/filter and src/exec are plausible given 109 integration tests that run the binary, that the threshold leaves room for ordinary churn, and that every action in the new job is pinned the way the rest of the file pins them.
Run it on every Rust CLI repository without coverage
A team with several command-line crates starts one run per repository: a POST /o/{org}/sessions with this prompt and that repository in resources, with a request_id per repository so a retried request returns the run it already started. Each run measures its own crate in its own sandbox and sets its own threshold, so a tool whose tests spawn its binary gets a job that captures that, and one with ordinary unit tests gets a simpler one.
Up to the key's max_concurrent runs proceed at once. Give the key a webhook_url and Rush POSTs run.completed or run.failed when each turn ends, with data.session_url to read the pull request from.
The use case this job follows, with its automation and trigger: Write the missing tests.
Why run it on Rush?
Adding coverage to a crate whose tests spawn its own binary is mostly a measurement question: does the tool see the child process or not. Answering it means installing llvm-tools-preview and cargo-llvm-cov, building the crate twice with instrumentation and reading the per-file table, which on a laptop displaces whatever you were building. In a Rush sandbox the agent does that with its own toolchain and keeps the table in the transcript, and the threshold in the pull request is a number it measured, not a guess.
A workflow that pins every action and runs on fourteen targets took care to write. The agent reads those conventions from the file and follows them, and a reviewer sees one job in one diff with the evidence for its threshold attached.
Frequently asked questions
Why cargo llvm-cov rather than tarpaulin?
Because the integration tests in tests/tests.rs run fd as a child process through CARGO_BIN_EXE_fd, and cargo llvm-cov instruments the binary Cargo builds for the tests and merges the profile each process writes. The agent confirms this by checking that files only reached through the binary show covered lines in the summary.
Will the job slow down the release builds?
No. It is a separate job on ubuntu-latest, so it runs in parallel with the build matrix and is not a dependency of the tarball, Debian package or release steps. The build job keeps running the suite on every target; coverage is measured once, on Linux.
Why not add a codecov upload?
It would mean a new third-party action and a token, and nothing in the repository asks for a hosted dashboard. The job uploads lcov.info as a workflow artifact and prints the summary into the step summary, which a reviewer can read on the run page. Adding a hosted service is a one-line follow-up if you want it.
What about the Windows-only code paths?
Code under #[cfg(windows)] is not compiled on Linux, so it neither counts for nor against the line total. The threshold is set against the Linux measurement, which is the only one the job takes.
More Rust CLI jobs
The same job on other stacks
- Add Test Coverage to CI for a Slack Bolt App (Slack Bolt)
- Add Test Coverage to CI for a Streamlit App (Streamlit)
- Add Test Coverage Reporting to Symfony CI (Symfony)