# Add Criterion Benchmarks to a Rust Workspace

Yes. An agent adds Criterion benchmarks to the crates in your Rust workspace, keeps the locked build and MSRV job green, and opens a PR with a first baseline.

[Run this on Rush](https://cloud.getrush.ai/run/add-criterion-benchmarks/rust/). Opens the console with this prompt ready to send, in a Project that has your repository.

## What the agent does

The agent reads the workspace before it picks what to measure. In a workspace shaped like rust-lang/mdBook, the root `Cargo.toml` lists the `mdbook` binary crate, `crates/*` (`mdbook-core`, `mdbook-markdown`, `mdbook-html`, `mdbook-driver`, `mdbook-renderer`, `mdbook-preprocessor`, `mdbook-summary`, `xtask`) and two example members, sets `edition` 2024 and `rust-version` 1.88.0 for every member, pins every dependency under `[workspace.dependencies]`, and marks `pulldown-cmark` 0.13.4 as part of the public API. The root crate already has a `[[test]]` target named `gui` with `harness = false`, so a custom harness is an accepted pattern here. `.github/workflows/main.yml` runs `cargo test --workspace --locked` on stable, beta, nightly, three targets and an `msrv` job on 1.88.0, then `cargo clippy --workspace --all-targets --no-deps -- -D warnings` and `cargo fmt --check`. There is no `benches` directory in any crate.

Two things are worth timing and both have a public entry point. `mdbook_markdown::new_cmark_parser(text, &MarkdownOptions)` turns a chapter into pulldown events, and `mdbook::MDBook::load(root)` followed by `build()` renders a whole book; the `render_markdown` function in `mdbook-html` is `pub(crate)`, so the agent does not reach for it. The agent adds `criterion` to `[workspace.dependencies]` at the newest release whose minimum Rust fits 1.88.0, as a dev dependency of `mdbook-markdown` with `benches/parse.rs` declared under `[[bench]]` with `harness = false`, which parses every chapter under `guide/src` and drains the events, and of the root crate with `benches/build_guide.rs`, which loads `guide/` and builds it into a `tempfile` directory, `tempfile` already being a dev dependency there. The corpus is the repository's own 35 guide chapters, so the numbers mean something to the maintainers.

Because CI passes `--locked`, the agent commits the `Cargo.lock` update that adds Criterion and its tree, and because clippy runs with `--all-targets -D warnings`, the bench files are written to lint clean under the workspace's `[workspace.lints]`, including `missing_docs` on anything public. It runs `cargo test --workspace --locked` and `cargo test --workspace --no-default-features` to confirm bench targets do not disturb the test builds, `cargo +1.88.0 check --workspace --all-targets` for the MSRV job, `cargo clippy --workspace --all-targets --no-deps -- -D warnings` and `cargo fmt --check`.

Then it runs `cargo bench --workspace` once in the sandbox to produce a baseline and pastes Criterion's summary lines for both benchmarks into the pull request, and adds a short `Benchmarks` section to `CONTRIBUTING.md` beside the existing `Tests` section with the two commands and a note that `cargo xtask test-all` leaves benches out.

## Start it with one API call

`POST https://api.phoenix.so/o/{org}/sessions`

```json
{
  "agent": "codex",
  "input": "Add Criterion benchmarks to this Rust workspace and open a pull request. Read Cargo.toml, Cargo.lock, crates/mdbook-markdown/Cargo.toml, crates/mdbook-markdown/src/lib.rs, crates/mdbook-driver/src/mdbook.rs, guide/book.toml, CONTRIBUTING.md and .github/workflows/main.yml first. Add criterion to workspace.dependencies at the newest release whose minimum Rust version is at or below the workspace rust-version. In crates/mdbook-markdown add benches/parse.rs as a harness-less bench target that parses every Markdown file under guide/src with new_cmark_parser and drains the events. In the root crate add benches/build_guide.rs that loads guide/ with MDBook::load and builds it into a tempfile directory. Use only public APIs. Commit the Cargo.lock change. Run cargo test --workspace --locked, cargo test --workspace --no-default-features, cargo check --workspace --all-targets on the rust-version toolchain, cargo clippy --workspace --all-targets --no-deps -- -D warnings and cargo fmt --check, then cargo bench --workspace once, and paste the check results and Criterion's summary lines in the pull request description. Add a Benchmarks section to CONTRIBUTING.md next to Tests. Do not change any source file under src or crates/*/src and do not update pulldown-cmark.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
```

1. **Get an API key.** In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
2. **Send the request.** POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
3. **Read what came back.** Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

## What comes back

What comes back is a pull request with `criterion` in the root `Cargo.toml` and in the two crates' dev dependencies, two `[[bench]]` entries with `harness = false`, `crates/mdbook-markdown/benches/parse.rs`, `benches/build_guide.rs` at the root, the `Cargo.lock` update, and a `Benchmarks` section in `CONTRIBUTING.md`. The description holds the output of each check the agent ran and Criterion's summary lines for the parse and build benchmarks from the sandbox, labelled as a single baseline on that machine.

The repository's `CI` workflow runs on the pull request: the test matrix including the `msrv` job, the clippy job with `--all-targets`, rustfmt and the typos check, so the bench targets are compiled and linted on every toolchain the project supports even though no job runs them.

A reviewer checks that both benchmarks call only public functions, that `pulldown-cmark` did not move in `Cargo.lock`, that the bench files pass the workspace lints without `allow` attributes, and that the corpus is the guide rather than a synthetic string. Then they decide whether to run `cargo bench` on a dedicated machine before trusting numbers across commits.

## Run it on every new workspace crate

A team that maintains several Rust workspaces starts one run per repository with `POST /o/{org}/sessions`, this prompt and the repository in `resources`. Each run reads that workspace's own public API and CI, so a crate with a `[[test]] harness = false` precedent gets the same shape and one with an MSRV job gets its Criterion version chosen to match, and each run ends in its own pull request.

The key's `webhook_url` receives `run.completed` for every run it starts, with a `session_url` for the result, so one script can collect the baseline summaries from each description into a single page.

## Why run it on Rush?

Adding benchmarks to a locked workspace touches the things CI is strict about: the lockfile, `--all-targets` clippy, the MSRV job and the public API boundary. In a Rush sandbox the agent runs every one of those checks the way `main.yml` does, on the toolchains the workflow names, and then runs the benchmarks once; every command and its output is kept in the run's transcript, so the pull request carries the evidence rather than a promise.

The result is a pull request that CI compiles on every supported target before a reviewer opens it, with a baseline attached and the limits of that baseline stated. The same request works on every Rust workspace the team owns, each in its own sandbox at the same time.

## Frequently asked questions

### Why not benchmark the HTML renderer directly?

Because `render_markdown` in `mdbook-html` is `pub(crate)`, and a bench target is a separate crate that only sees public items. The agent benchmarks the public parser in `mdbook-markdown` and the public `MDBook::load` and `build` path, which covers the renderer end to end without widening the API.

### Will the benches break the locked build or the MSRV job?

The agent commits the `Cargo.lock` change so `--locked` resolves, and picks the Criterion release whose minimum Rust version fits the workspace's `rust-version`, checking it with `cargo check --workspace --all-targets` on that toolchain before pushing.

### Are the numbers in the pull request reliable?

They are one run on one sandbox, labelled as such. Criterion stores its baseline under `target/criterion`, which is not committed, so comparisons across commits need the same machine. The pull request gives the maintainers the harness and corpus; the stable numbers come from running it where they choose.

### Does cargo test now run the benchmarks?

No. `cargo test` does not build bench targets unless asked with `--benches` or `--all-targets`, and `harness = false` benches are not test binaries. The agent runs both `cargo test --workspace --locked` and the `--no-default-features` variant to show the test builds are unchanged.

## More Rust jobs

- [Add Fuzz Tests to a Rust Project](https://getrush.ai/run/add-fuzz-tests/rust/)
- [Add Graceful Shutdown to a Rust Axum Service](https://getrush.ai/run/add-graceful-shutdown/rust/)
- [Dockerize a Rust API](https://getrush.ai/run/dockerize-app/rust/)

## The same job on other stacks

- [Add Criterion Benchmarks to a Rust CLI](https://getrush.ai/run/add-criterion-benchmarks/rust-cli/)
- [Add Criterion Benchmarks to a Tokio Server](https://getrush.ai/run/add-criterion-benchmarks/tokio/)

## More jobs like this: Hand off a long task

- [Schedule Database Jobs With pg_cron on Supabase](https://getrush.ai/run/add-cron-jobs/supabase/)

## Keep reading

- [How a Rush run works on your repository, from request to pull request](https://getrush.ai/cloud/)
- [Compare cloud agents, agent sandboxes and AI code review](https://getrush.ai/compare/)
- [Rush pricing: Free, Pro and Max](https://getrush.ai/pricing/)
