Write Property-Based Tests for a Rust Crate

The agent reads the crate and its CI before writing a property, because a property test that breaks the MSRV job or the no_std build is a net loss. In a crate shaped like tokio-rs/bytes, Cargo.toml declares version 1.12.1 on edition 2021 with rust-version = "1.57", features std on by default, an optional serde dependency and an optional extra-platforms dependency on portable-atomic; the only dev-dependency is serde_test, and loom is a dev-dependency under cfg(loom) only. clippy.toml repeats msrv = "1.57". .github/workflows/ci.yml runs cargo fmt --all --check, a minrust job with cargo hack check --feature-powerset --optional-deps --rust-version, ci/test-stable.sh, which runs cargo hack test --each-feature --optional-deps and cargo test --all-features, cross-target tests, no_std builds for thumbv6m and thumbv7m with --no-dev-deps, and nightly jobs for ci/miri.sh, ci/tsan.sh, ci/panic-abort.sh and loom with RUSTFLAGS="--cfg loom" cargo test --lib.

Then it reads what the existing tests already pin down, so the properties cover the gaps rather than restate examples. tests/test_bytes.rs has example-based tests for slice, split_off, split_to, truncate, the freeze_after_* family, reserve_* growth and reuse, extend_from_within, advance_* and the bytes_mut_unsplit_* cases; tests/test_buf.rs, test_buf_mut.rs, test_chain.rs, test_take.rs, test_limit.rs, test_reader.rs and test_iter.rs cover the Buf and BufMut adaptors; test_bytes_odd_alloc.rs and test_bytes_vec_alloc.rs swap the global allocator. Every one of them fixes specific lengths and offsets. The public surface in src/bytes.rs and src/bytes_mut.rs is where the invariants live: Bytes::slice, slice_ref, split_off, split_to, truncate, try_into_mut, and BytesMut::split, split_to, split_off, unsplit, try_unsplit, reserve, try_reclaim, extend_from_slice, extend_from_within, resize, zeroed and freeze.

The properties are written against a Vec<u8> model in a new tests/test_properties.rs, with proptest added as a dev-dependency. For random bytes and an index up to the length: split_off(at) followed by concatenation equals the original and the lengths add up; split_to the same from the other side; slice(a..b) equals the model's &data[a..b] for every valid range form, and slice_ref of that subslice equals slice(a..b); truncate(n) leaves the first min(n, len) bytes; BytesMut::split_to then unsplit restores the original bytes; a random sequence of extend_from_slice, extend_from_within, resize, truncate, split_to and advance applied to both a BytesMut and the model leaves them equal at every step, and freeze at the end equals the model too. For the adaptors, a.chain(b) read to the end equals the two inputs concatenated, take(n) yields exactly min(n, remaining) bytes, and copy_to_bytes of any prefix matches the model. Panicking cases, such as split_off past the length, get should_panic properties on indices above len.

The agent then runs the properties the way CI will see them. It runs cargo test --all-features and cargo hack test --each-feature --optional-deps, so the new file compiles with and without std and serde, and gates it with #![cfg(feature = "std")] if a property needs the Vec model under std only. It confirms the minrust job still resolves by running cargo hack check --feature-powerset --optional-deps --rust-version with the new dev-dependency, and pins proptest to an older release if the current one will not resolve on 1.57. It marks the file #[cfg(not(miri))] so ci/miri.sh does not spend its budget on random inputs, notes that loom runs --lib and never sees integration tests, and runs cargo fmt --all --check. It sets PROPTEST_CASES low in CI through a proptest! config with a comment on how to raise it locally, and opens the pull request with the test output and any shrunken counterexample it found.

What comes back is a pull request on its own branch that adds tests/test_properties.rs and a proptest line under [dev-dependencies] in Cargo.toml, with nothing changed under src/. The description lists each property in one line with the API it exercises, states which feature combinations it ran through cargo hack, says whether proptest had to be pinned for the minrust job and to what, and quotes the test summary. If a property found a counterexample, the shrunken input and the assertion are in the description, and the property stays in the file marked #[ignore] with a comment, because a failing property on a released crate is a finding for the maintainers rather than something to silence.

A reviewer reads the model-based sequence test first, since it carries most of the coverage, and checks that the model's semantics match the documented behaviour of advance and truncate rather than the implementation's. They confirm the cfg gates: std for the model, not(miri) for the nightly job, and nothing that hides the tests from cargo test --all-features. The transcript holds every cargo run, so the feature matrix in the description can be checked against the commands that produced it.

An automation in the Project with a schedule trigger, a cron expression for Sunday night and your timezone, starts one run per week with a prompt that asks for properties on whichever public functions still have only example tests. Each run opens its own pull request on its own branch, and a week with nothing left to cover opens none.

For several crates, a script on your server lists the organization's repositories with GET /o/{org}/github, checks each Cargo.toml for a proptest or quickcheck dev-dependency, and sends one POST /o/{org}/sessions with this prompt for each crate without one, with a request_id per repository so a re-run is safe. A webhook_url on the API key delivers run.completed with the session_url for each pull request.

Property tests are cheap to write and expensive to land: the first draft breaks the MSRV job, the second does not compile without std, the third makes miri take an hour. The agent runs the crate's own CI matrix in its sandbox before opening the pull request, with cargo hack and the rust-version check, so what arrives is already green against the jobs that would have rejected it.

The pull request explains each property in a sentence and shows the runs behind it, and a counterexample arrives as a shrunken input with the assertion that failed, which is the most useful bug report a maintainer can get. Several crates can be covered at once, each in its own sandbox and branch.

Why proptest and not quickcheck?

Will the MSRV job still pass?

Does it change any code under src?

What about miri, loom and the no_std targets?