All jobs

Add Property-Based Tests With fast-check to a Jest Suite

Yes. An agent adds fast-check to a Jest project and writes properties for its pure modules: pick, the password and ObjectId validators and pagination maths.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

In an Express API laid out like hagopj13/node-express-boilerplate, the unit tests under tests/unit cover the error middleware, the user model and the two Mongoose plugins, and the integration tests under tests/integration drive the auth and user routes with supertest. Nothing tests src/utils/pick.js, which copies own properties for a list of keys, or src/validations/custom.validation.js, whose objectId and password validators are the custom rules every Joi schema in src/validations leans on. Those tests, where they exist, are example-based: one good password, one bad one. Jest is 26.6.3 in yarn.lock, run with jest -i, and the suite's model tests need the MongoDB that tests/utils/setupTestDB.js connects to.

The agent adds fast-check as a dev dependency (its current release names Node 12.17 or later in engines, inside the repository's own node >=12 range) and writes properties where the code has an invariant rather than an example. For pick: for any object and any list of keys, every key of the result is in both the list and the object's own keys, each value is the same reference as the input's, a key that only exists on the prototype (an object made with Object.create) is never copied, and pick(undefined, keys) is {}. For password: any string shorter than eight characters gets the length message, any string of eight or more with at least one digit and one letter comes back unchanged, and any string of eight or more with no digit or no letter gets the second message. For objectId: any twenty-four character hex string passes and any string of another length or with a non-hex character fails, generated with fc.hexaString and fc.string and the helpers object the validator calls on failure stubbed to return its message.

Two properties reach into Mongoose without a database. For toJSON.plugin.js, the agent builds a schema with a top-level private: true path and a nested one, makes documents from generated field values, and asserts that toJSON() has an id equal to the string of _id, no _id, __v, createdAt or updatedAt, and neither private path, for every generated document. For paginate.plugin.js, it calls the static with a stand-in this whose countDocuments resolves a generated total and whose find records the skip and limit it receives, and asserts for any limit and page strings that a non-positive or non-numeric limit falls back to ten, skip is (page - 1) * limit, and totalPages is Math.ceil(total / limit). A last property round-trips token.service.js: for any user id string and token type, jwt.verify(generateToken(userId, expires, type), config.jwt.secret).sub is the id, with the environment from .env.example loaded so src/config/config.js accepts it.

The new files sit beside the existing ones: tests/unit/utils/pick.test.js, tests/unit/validations/custom.validation.test.js, tests/unit/services/token.service.test.js, and new describe blocks in the two plugin tests. Each property runs under fc.assert with fast-check's default number of runs, and the agent notes in the pull request that a failure prints the seed and the shrunk counterexample, and that passing { seed } to fc.assert replays it. In its sandbox it installs Docker, starts a mongo container, and runs the whole suite with yarn test and then yarn lint, since the Airbnb config and eslint-plugin-jest apply to the new files.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "claude",
  "input": "Add property-based tests with fast-check to this Express and Jest project. Read package.json, jest.config.js, .env.example, every file under tests/, src/utils/pick.js, src/validations/custom.validation.js, src/middlewares/validate.js, src/models/plugins/paginate.plugin.js, src/models/plugins/toJSON.plugin.js and src/services/token.service.js first. Install fast-check as a dev dependency. Write properties, not examples: for pick (own keys only, same references, prototype keys never copied, a missing object gives an empty one), for the password and objectId validators (length and character-class invariants over generated strings), for the toJSON plugin (id replaces _id, private and timestamp paths are gone, for any generated document), for the paginate plugin's limit, page, skip and totalPages maths using a stand-in model, and a sign-then-verify round trip for generateToken. Put them under tests/unit beside the existing files. Start MongoDB in the sandbox with the variables from .env.example, run yarn test and yarn lint, and quote both. Open a pull request.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

What comes back is a pull request that adds fast-check to package.json and yarn.lock, three new test files under tests/unit and new blocks in the two plugin tests, and changes nothing under src/. The description lists each property in one line as the invariant it states, quotes the yarn test summary with the new test count and the yarn lint result, and says which generators each property uses.

If a property finds a counterexample in the existing code, the pull request says so with the shrunk input quoted and leaves the code as it is, since a validator that rejects an input the property expected to pass is a product decision, not a test bug; the test is marked with the case it found so the reviewer can decide.

Before merging, a reviewer reads each invariant and asks whether it is the rule the team means: a password rule that is really "at least eight characters, one letter, one digit" is now written down as one, and a future change to it fails a test that says why.

Run it on every pull request that touches a validator or utility

An automation in the Project with a github trigger on pull requests fires on pull_request.opened and pull_request.synchronize. The trigger has no path filter, so its prompt asks the agent to answer in one line when the diff touches nothing under src/utils, src/validations or src/models/plugins, and otherwise to find any invariant the change adds or weakens, and open a pull request with a property for it, with the counterexample quoted when the property fails. The trigger adds the pull request's title and description after the prompt as event text.

A run of this kind is read-only when the automation says so, in which case it answers with what it found instead of pushing, and the answer is on the session. Either way each run is its own session with its own record, listed under the automation.

The use case this job follows, with its automation and trigger: Write the missing tests.

Why run it on Rush?

Writing a good property means reading the code until the invariant is clear, then letting the generator find the input nobody thought of. In a Rush sandbox the agent reads every module it is asked about, runs the suite against a MongoDB it starts itself, and sees the first counterexample before the pull request opens, rather than leaving a failing property for the reader to debug.

The run's record keeps every test run and the pull request quotes the summaries, so review is reading invariants rather than re-running anything. A team with several Jest services runs the job on each at once, one sandbox and one pull request each, each with properties for that service's own helpers.

Frequently asked questions

Does fast-check need a different test runner?

No. fc.assert(fc.property(...)) is an ordinary expression inside a Jest test, so the new files run under the existing yarn test with Jest 26 and the existing jest.config.js. The only addition is the dev dependency.

Will a property that fails intermittently flake the suite?

A property that fails has found a real input the code mishandles; fast-check prints the seed and the shrunk counterexample, and fc.assert takes a seed option so the same run is replayed exactly. The pull request records any counterexample it found rather than loosening the property to make it pass.

Why stand-in models for the paginate property instead of MongoDB?

The existing paginate tests already use MongoDB for the query path. The property is about the arithmetic on limit, page, skip and totalPages, which the plugin computes before any query runs, so it calls the static with an object that resolves a generated total and records the skip and limit it was given, and runs in milliseconds for every generated case.

Do the properties change what the validators accept?

No. They state what the validators do today, read from custom.validation.js. If a generated input shows that statement is wrong, the pull request quotes the input and leaves the validator alone, so the reviewer decides whether the code or the rule should change.

More Jest jobs

More jobs like this: Write the missing tests

Keep reading