All jobs

Add Fuzz Tests to a Rust Project

Yes. An agent adds a cargo-fuzz crate with targets for your input parsers, seeds each corpus from your existing unit tests, runs them, and opens a PR.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

The agent reads where the project parses untrusted text before it picks targets, because fuzzing a function that only takes well-formed structs finds nothing. In a crate shaped like sharkdp/fd, Cargo.toml declares one [[bin]] named fd at src/main.rs with edition = "2024" and rust-version = "1.90.0", no library target, and dependencies on regex, regex-syntax, jiff, clap, aho-corasick and anyhow. src/main.rs declares the modules privately, and the parsers that take user strings are SizeFilter::from_string in src/filter/size.rs, which runs a regex over inputs like +10k and -1GiB and multiplies the quantity by a unit, TimeFilter::before and after in src/filter/time.rs, which try a jiff::Span, a Timestamp and a civil DateTime in turn, and pattern_has_uppercase_char and pattern_matches_strings_with_leading_dot in src/regex_helper.rs, which parse a pattern into a regex_syntax HIR and walk it. Each already has a #[cfg(test)] module with example inputs. .github/workflows/CICD.yml runs cargo fmt -- --check, cargo clippy --all-targets --all-features -- -Dwarnings, the tests on the minimum supported Rust version and a build matrix.

It runs cargo fuzz init to create fuzz/Cargo.toml and fuzz/fuzz_targets/, then writes one target per parser. Because the crate has no library, a fuzz target cannot use fd_find::...; the agent includes each parser's source into the fuzz crate with a #[path = "../../src/filter/size.rs"] module attribute, which works because size.rs, time.rs and regex_helper.rs depend only on their external crates and not on crate::. size_filter calls SizeFilter::from_string on the bytes as a string and, when it parses, asserts is_within is consistent for the boundary value. time_filter calls before and after and asserts a filter that parses applies to a time on the right side. regex_helper calls both pattern functions and asserts neither panics on an invalid pattern, since regex_syntax errors are expected and must return false, not unwind. A fourth target, size_filter_arith, uses arbitrary to build a quantity and a unit and checks the multiplication does not overflow silently, which is the kind of bug a parser that uses * rather than checked_mul hides.

Each target gets a seed corpus under fuzz/corpus/<target>/ taken from the inputs in that file's unit tests, so the fuzzer starts from +1KiB, -100k, 2018-01-01, 10min and the pattern strings, and a fuzz/.gitignore for the artifacts/ directory. The fuzz crate is added to exclude in Cargo.toml next to /benchmarks/* so it is not packaged for crates.io, and fuzz/Cargo.toml names the same regex, regex-syntax, jiff and anyhow versions the main crate uses so the included modules compile unchanged. Fuzzing needs libFuzzer and a nightly toolchain, so the agent documents cargo +nightly fuzz run size_filter -- -max_total_time=60 in CONTRIBUTING.md and keeps the targets out of the stable cargo test run.

Before pushing it runs each target for a bounded time with -max_total_time and reports what each found. A crash produces a file under fuzz/artifacts/<target>/, and for a reproducible one the agent adds the input as a regular #[test] in the parser's own test module and leaves the fix for a separate change, saying so in the pull request, unless the fix is a one-line checked_mul. It also adds a fuzz-build job to CICD.yml that installs nightly and runs cargo fuzz build inside fuzz/, so the targets keep compiling when a parser's signature changes, and runs cargo fmt -- --check and cargo clippy --all-targets --all-features -- -Dwarnings on both crates.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "claude",
  "input": "Add fuzz tests to this Rust project and open a pull request. Read Cargo.toml, src/main.rs, src/filter/size.rs, src/filter/time.rs, src/regex_helper.rs, Makefile, CONTRIBUTING.md and .github/workflows/CICD.yml first. Run cargo fuzz init and write fuzz targets for SizeFilter::from_string, TimeFilter::before and after, and the two functions in regex_helper.rs; the crate has no library target, so include each parser's source into the fuzz crate with #[path] module attributes rather than restructuring the crate. Assert the parsed values are consistent, not just that nothing panics, and add a target that checks the size multiplication cannot overflow. Seed each corpus from the inputs in the existing unit tests. Exclude fuzz/ from the package in Cargo.toml, document how to run the targets on nightly in CONTRIBUTING.md, and add a CI job that runs cargo fuzz build. Run each target with -max_total_time for a short bounded time; for any reproducible crash, add the input as a regular test and describe it in the pull request. Run cargo fmt -- --check, cargo clippy --all-targets --all-features -- -Dwarnings and cargo test and paste the results in the pull request. Do not change the parsers' behaviour except for a one-line overflow fix.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

What comes back is a pull request that adds the fuzz/ crate with Cargo.toml, .gitignore, four targets under fuzz_targets/ and seed corpora, one line in the root Cargo.toml's exclude, a section in CONTRIBUTING.md, a job in CICD.yml, and any regression tests the bounded runs produced. The parsers' source files are otherwise untouched. The description lists each target, the property it asserts, the seeds it starts from, and the result of each run with its duration and executions, pasted from the fuzzer's own output.

On a repository with the demo's CICD workflow, the pull request runs formatting, clippy with warnings as errors, the tests on the minimum supported Rust version and the build matrix, and the new fuzz-build job compiles the targets on nightly, so a target that drifts from a parser's signature fails before merge.

A reviewer checks that each target asserts something about the parsed value rather than only surviving, that the #[path] includes point at files with no crate:: imports so they keep compiling, and reads any crash the runs found. Then they decide how long to fuzz on a schedule, since the pull request's runs are bounded and a longer run on a nightly job is where deeper inputs are reached.

Run it on every Rust crate with hand-written parsers

A team with several Rust tools sends one POST /o/{org}/sessions per repository with this prompt and that repository in resources. Each run finds its own parsers and picks targets for them, so a crate with a library target gets targets that import it directly and a binary-only crate gets #[path] includes like this one, and each run opens its own pull request.

Give the API key a webhook_url and Rush POSTs run.completed when each turn ends, with the session_url to read the pull request and the fuzzer output from.

The use case this job follows, with its automation and trigger: Write the missing tests.

Why run it on Rush?

A fuzz target is only worth adding if it has been run, because the first minutes of fuzzing a parser that has never been fuzzed are the ones that find the overflow and the unexpected panic. That needs a nightly toolchain, libFuzzer, and time on a machine that is not someone's laptop. In a Rush sandbox the agent installs the toolchain, runs every target for a bounded time, and keeps the fuzzer's own output and any crash artifacts in the run's record, so the pull request arrives with the first findings rather than with empty harnesses.

The targets include the parser sources as they are, so no module becomes public and the crate's layout does not change for the sake of testing. The corpora start from inputs the unit tests already name, the CI job keeps the targets compiling, and the same request works on any Rust crate that parses text from a user, a file or the network.

Frequently asked questions

Why #[path] includes instead of a library target?

The crate is binary-only, with every module declared in src/main.rs. Adding a lib.rs to expose the parsers would change the crate's public surface and its build. The three parser files depend only on external crates, so including them into the fuzz crate by path compiles them unchanged; the pull request says which files that holds for.

Do the fuzz targets run in cargo test?

No. They need libFuzzer and a nightly toolchain, so they are a separate crate under fuzz/, excluded from the package and run with cargo +nightly fuzz run <target>. The CI job only builds them. A crash a run finds is turned into a regular #[test] in the parser's module so it does run on stable.

What does a target assert beyond not panicking?

That the parsed value is consistent: a size filter that parses must report its own boundary as within range, a time filter that parses must apply to a time on the expected side, and the regex helpers must return false for a pattern regex_syntax rejects rather than unwinding. A separate target checks the unit multiplication cannot overflow.

How long does the agent fuzz?

For the bounded time set by -max_total_time on each target, and the pull request pastes the fuzzer's own summary for each run. That is enough to shake out shallow panics; the reviewer sets a longer run on a schedule for deeper inputs.

More Rust jobs

The same job on other stacks

Keep reading