Write Property-Based Tests for a TypeScript Library

The agent first learns how the library builds and tests itself, because a property suite that runs under a different command than the existing tests gets ignored. In a library like colinhacks/zod it reads the root package.json and finds packageManager: nub@0.8.3, a test script of nub exec --node vitest run, Vitest 4 and TypeScript ~5.5.4, with .nvmrc set to Node 24. AGENTS.md says Nub reads and writes the existing pnpm-lock.yaml, so it installs with nub install --frozen-lockfile and adds a dependency with nub add, never with npm. In vitest.config.ts it notes that typecheck is on for every *.test.ts and that scripts/fail-on-console.ts fails any test that logs, so the new tests carry no console.log.

Then it picks the functions whose behaviour can be stated as a law rather than an example. In Zod those sit in packages/zod/src/v4: z.codec with its decode and encode pair, where z.encode(c, z.decode(c, x)) should give back x for every valid input; toJSONSchema in core/to-json-schema.ts and fromJSONSchema in classic/from-json-schema.ts, where a schema converted out and back in should accept and reject the same values; and parse, where parsing a value that already parsed should return an equal value. It reads the existing files in classic/tests, such as codec.test.ts and from-json-schema.test.ts, so the properties cover inputs those example-based tests do not, rather than restating them.

fast-check is not a direct dependency. The lockfile has fast-check@3.23.2 only because effect pulls it in, so the agent adds it as a root dev dependency with the package manager the repository uses and keeps the lockfile change in the pull request. It writes the properties beside the existing tests, for example packages/zod/src/v4/classic/tests/codec.property.test.ts, builds arbitraries that generate both schemas and values (strings with unicode and surrogate pairs, numbers including -0, NaN and the safe-integer edges, nested objects and arrays), and sets numRuns well above the default of 100 so a run explores more than a handful of shapes.

It runs the new files with nub exec --node vitest run <path> and then the whole suite with nub run test, which also re-runs the zod tests in the compile-mode project. When a property fails, fast-check shrinks the input to a small counterexample and prints the seed and path that replay it. The agent does not change the library to make the property pass: it keeps the failing case as a test marked with test.fails or test.skip so the suite stays green, and writes the counterexample, the seed and the law it breaks into the pull request description for a maintainer to judge.

What comes back is a pull request that adds test files and a dev dependency and touches nothing under the library's source. In a repository shaped like Zod it adds files such as codec.property.test.ts and json-schema.property.test.ts under packages/zod/src/v4/classic/tests, and fast-check in the root package.json with the matching pnpm-lock.yaml change. Each test name states the law it checks, and the arbitraries live at the top of the file so a reviewer can see exactly which inputs were generated.

The description lists every property, its numRuns, and whether it held. A property that failed appears with its shrunk counterexample, the seed and path to replay it, and the one-line reason it matters, for example a value that encodes to a string the decoder rejects. The repository's own CI runs on the pull request, and the session's record keeps every command the agent ran in its sandbox, with the full fast-check output for each failure.

Before merging, a reviewer checks that each property is a real promise of the library rather than an accident of the current code, that the arbitraries do not generate inputs the documentation says are unsupported, and that the higher run count keeps the suite within the time the team accepts in CI. A reported counterexample is a question for a maintainer: fix the library in a separate change, document the limit, or narrow the arbitrary.

Property tests find the most when they run against new code. When your release workflow tags a candidate, have it send this request with the repository in resources, adjusted to name the modules that changed since the last release. Each request is its own session in its own sandbox, and the answer comes back to the webhook URL on your API key as run.completed or run.failed with the session URL, so the release job can wait for it and hold the tag when a property breaks.

For a monorepo with several published packages, send one request per package in parallel; a key runs up to its max_concurrent sessions at once (4 by default) and answers 429 past that, so queue the rest.

Property tests are only as good as the number of cases they run, and a high numRuns across round-trip properties can keep a laptop busy while it shrinks one failure after another. On Rush the agent installs the dependencies with the repository's own package manager and runs the full suite, compile-mode project included, in its own sandbox, so your machine and working tree are not involved and a long shrink does not block you.

The proof is the output, not a summary of it. The session keeps each vitest run with its full output, so a counterexample in the pull request can be traced to the exact run, seed and generated value that produced it. And because the prompt forbids source changes, the pull request is safe to review as tests only: anything it found is reported, never quietly fixed.

Will it change my library code to make a property pass?

Does it work with Jest instead of Vitest?

How do I reproduce a counterexample it reports?

Will a high example count slow down CI?