All jobs

Run a Jest Suite in Parallel Against One MongoDB

Yes. An agent finds why your Jest suite runs in band, gives each worker its own MongoDB database, drops the flags that force serial runs, times both, and opens a PR.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

The agent reads why the suite is serial before it removes the flag that makes it so. In an API laid out like hagopj13/node-express-boilerplate, package.json runs jest -i --colors --verbose --detectOpenHandles as test and jest -i --coverage as coverage, with jest locked at 26.6.3 in yarn.lock. The -i is there for a reason: tests/utils/setupTestDB.js connects Mongoose's default connection in beforeAll, and in beforeEach calls deleteMany() on every collection, and src/config/config.js builds that URL as MONGODB_URL plus -test when NODE_ENV is test. The three files under tests/integration/, 88 tests for auth, users and docs, all share that one database, so two workers would wipe each other's documents between tests. The 17 unit tests under tests/unit/ never touch the database.

It also reads the second flag, because --detectOpenHandles implies --runInBand in Jest: with it on, dropping -i changes nothing. The fix is per-worker isolation, and Jest already provides the handle. Every worker runs with JEST_WORKER_ID set, starting at 1, so setupTestDB.js connects to ${config.mongoose.url}-${process.env.JEST_WORKER_ID} instead of the bare URL. Because src/app.js and the models use Mongoose's default connection, which is the one setupTestDB opens, every request the integration tests send through supertest lands in that worker's database and nothing in src/ changes. afterAll gains a dropDatabase() before disconnect() so a MongoDB that serves many CI runs does not fill with -test-1, -test-2 databases.

The scripts then change shape. test becomes jest --colors --verbose, coverage becomes jest --coverage, and a new test:debug keeps -i --detectOpenHandles for the day a handle leaks and someone needs to find it. --maxWorkers is not pinned: Jest's default of cores minus one fits the agent's sandbox and the CI machine alike. .travis.yml, which starts a mongodb service and runs yarn lint then yarn test on Node 12, needs no change, and neither does docker-compose.test.yml, which runs yarn test in the app container.

The proof needs a MongoDB. The sandbox has no Docker, so the agent installs it with sudo apt-get install -y docker.io, starts dockerd, and runs a MongoDB 4.4 container on the default port with MONGODB_URL pointing at it, the same shape as .env.example. It runs yarn test with the old scripts and the new ones, twice each, reading the wall-clock time and the pass count from Jest's summary, which must be the same 105 both ways, and then lists the databases on the instance after a run to show the per-worker ones were dropped. A test that fails only in parallel is reported with its file and the shared state it points to, not rewritten, because it is a finding about the suite.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "claude",
  "input": "Make the Jest suite run in parallel. Read package.json, jest.config.js, src/config/config.js, tests/utils/setupTestDB.js, every file under tests/, .travis.yml, docker-compose.test.yml and .env.example first, and explain why the suite runs in band today, including what --detectOpenHandles implies. Give each worker its own database by adding JEST_WORKER_ID to the test database name in setupTestDB.js and dropping it in afterAll. Remove -i and --detectOpenHandles from test and coverage, and keep them in a new test:debug script. Install Docker in your sandbox, run a MongoDB container, run the suite with the old and new scripts twice each and record the times and pass counts, then list the databases to show the per-worker ones are gone. Report any test that fails only in parallel with its file instead of changing it. Run yarn lint and open a pull request quoting the timings and the summaries.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

What comes back is a pull request touching package.json and tests/utils/setupTestDB.js, with the reasoning in its description: why -i was there, why --detectOpenHandles would have kept the suite serial anyway, the four timings with their pass counts, and the database listing after a parallel run.

The run's record keeps the Docker install, the container start, every Jest summary and the yarn lint output, so a reviewer can see 105 tests passing in both modes and not just a faster number.

Before merging, a reviewer checks that the CI machine's MongoDB can take several connections at once, which the Travis service and the compose file both can, and decides whether test:debug should also set --maxWorkers=1 explicitly for readers who do not know what --detectOpenHandles implies. The pull request names both.

Run it on every new service generated from the template

This repository is also a generator, create-nodejs-express-app, so every service made from it starts with the serial scripts. A handler on your server that watches for new repositories whose package.json names jest -i sends this body once per repository to POST /o/{org}/sessions, with the repository in resources and a request_id that is a UUID v5 of the repository's full name, so a retried call returns the run already started. Each run measures that service's own suite, so one with a different test layout gets a pull request that fits it.

A Project automation with a schedule trigger every day runs a prompt that reads the suite's last CI duration and starts this job only when the test step has grown past a limit the prompt names, and it stays within its max_runs_per_hour cap however many repositories it watches.

The use case this job follows, with its automation and trigger: Write the missing tests.

Why run it on Rush?

The reason a suite is serial is usually one shared resource, and finding it means reading the test setup, not the flags. The agent reads both, makes the one change that lets workers coexist, and proves it against a real MongoDB in a sandbox it set up itself, with the timings and the database listing kept in the run's record.

Services generated from one template all carry the same scripts, so the same run applies to each of them in parallel, one sandbox and one small pull request each, with each pull request's numbers measured on that service's own tests.

Frequently asked questions

Why does --detectOpenHandles matter here?

Because Jest documents that the flag implies --runInBand, so a suite that keeps it stays serial no matter what else changes. Keeping it in a separate test:debug script preserves the tool for the case it is for, a handle that stops Jest exiting, without paying for it on every run.

Does anything in src/ change?

No. The app and its models use Mongoose's default connection, and setupTestDB.js is what opens it in tests, so suffixing the database name there is enough. src/config/config.js still builds the -test URL and the suffix is added on top of it.

What does Jest set JEST_WORKER_ID to with one worker?

1, in band or not, so the database name is always suffixed and the afterAll drop always has something to drop. A developer running one test file locally gets -test-1 and never sees a difference.

Will the Travis job need more MongoDB memory?

Several small databases on one instance cost about what one does, and the suite drops them after each file. The agent reports the instance's database count after a run so a reviewer can see nothing accumulates.

More Jest jobs

The same job on other stacks

More jobs like this: Write the missing tests

Keep reading