All jobs

Migrate Tests From tape to the Node.js Test Runner (node:test)

Yes. An agent ports a tape suite to node:test and node:assert, proves the new tests can still fail, removes tape and its reporter, and opens a pull request.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

Node now has a test runner in the standard library, node:test, and a suite written with tape maps onto it closely: a named test with a callback, plain assertions, TAP-style output. The traps are in the places where the two differ, mostly around when a test is considered finished. SlexAxton/css-colorguard, a PostCSS plugin, has a suite that shows the main one. test/test.js is a table of 15 cases, and one loop turns each into tape(test.message, function (t) { t.plan(3); check(...) }). The check helper runs PostCSS, which returns a promise, and calls back from .then with the CSS and the warnings. The three assertions, two t.equal and one t.deepEqual, happen inside that callback, and tape ends each test when the planned three have arrived.

The agent runs the suite first to fix a baseline. npm test here is npm run lint && tape test/*.js | tap-spec, and on Node 22 it reports 45 passing. That number is assertions, three for each of the 15 cases. node:test counts tests, so the same suite will report 15, and the agent states the change in the pull request so that nobody reads the smaller number as lost coverage.

Then it ports, and the finishing rule is what it changes with care. node:test treats a test as done when its function returns or the promise it returned settles. A conversion that keeps the callback style returns nothing, so the assertions run after the runner has moved on, and a failing one surfaces as asynchronous activity after the test ended instead of as a clean failure of that test. The agent therefore makes check return the PostCSS promise and has each test return it, with the assertions in the .then. t.equal becomes assert.strictEqual and t.deepEqual becomes assert.deepStrictEqual, both strict like their tape counterparts, and t.plan(3) goes away because a returned promise already tells the runner when the test is over.

A port that only shows green proves little, so the agent breaks one expectation on purpose, confirms that all affected tests fail by name with a readable diff, and reverts the break. It then updates the tooling around the suite. The test script becomes npm run lint && node --test, without the pipe to tap-spec, since the runner prints a readable report by itself. It does not write node --test test/: on Node 22 a bare directory argument is taken as a module path and fails with Cannot find module. tape and tap-spec leave devDependencies. The project's .travis.yml still lists stable, iojs and 0.12, so the agent records the new minimum Node for running the tests and leaves the published package's runtime support alone.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "claude",
  "input": "Migrate this project's tests from tape to the built-in Node.js test runner (node:test with node:assert). First run the existing test script and record the totals and anything it prints besides results. Port every test file: tape's test function to node:test, t.equal to assert.strictEqual, t.deepEqual to assert.deepStrictEqual, t.ok to assert.ok, t.throws to assert.throws, t.end() to returning from the function, and callback or t.plan based asynchronous tests to functions that return or await a promise so the runner knows when each test is finished. Do not leave any assertion running after its test function has returned. Keep test names and the order of cases. Then prove the new suite can fail: change one expected value, run it, confirm the right tests fail with a clear message and a non-zero exit code, and revert. Update the test script in package.json to use node --test with a form that discovers the test files on the oldest Node version you set, remove tape and any TAP reporter that was only used with it from devDependencies, and regenerate the lockfile if there is one. Run the linter the project uses on the new test files. State the minimum Node version the tests now need without changing the engines field of the published package unless asked. Explain any change in reported totals, such as assertions counted before and tests counted now. Open a pull request.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

A pull request that rewrites the test files, edits two or three lines of package.json, and removes two development dependencies. For css-colorguard that takes a fresh install's node_modules from 173 top-level entries to 53, because the reporter and the framework brought most of the tree with them. The description gives the totals before and after and explains the unit: 45 assertions under tape, 15 tests under node:test, the same 15 cases either way.

The description includes the deliberate failure. It shows the output from the run where one expectation was broken, with the failing test names and the assertion diff, and then the green run after the revert. That pair is the evidence that the port did not turn the suite into one that passes whatever the code does, which is the usual way a test migration goes wrong.

A reviewer checks the Node floor. The suite now needs a Node release that has a stable node:test, while the library itself may still run on older ones, and CI has to test on a version that satisfies the former. In a repository whose CI file names interpreters as old as io.js, the description says plainly that the CI configuration needs its own update and whether the agent made it.

Run it on every Linear issue moved to Ready

Test framework migrations are planned work, usually one tracker issue per package. Create an automation in the Project with a linear trigger on linear.issue.state_changed, and map the state your team uses for agreed work to exec in its states setting. When a member of your Rush org moves an issue into that state, a run starts with the issue after the saved prompt, so notes in the issue such as which reporter the team wants reach the agent.

Map an earlier state to plan if you want to see the approach first: a plan run is read-only and posts its plan as a comment on the issue, and moving the issue on to the exec state starts the run that opens the pull request. The trigger needs a Linear connection for the member the automation runs as. It fills in nothing in the prompt, so keep the prompt general and put package-specific wishes in the issue.

The use case this job follows, with its automation and trigger: Upgrade dependencies weekly.

Why run it on Rush?

The question a test migration has to answer is whether the new suite still catches bugs, and that takes running it several ways: as it was, after the port, with a planted failure, and after the cleanup with the old packages gone. The agent does all of that in its own sandbox from a fresh install, so a leftover node_modules from before the change cannot make the result look better than it is. Each of those runs is in the session's transcript with its full output.

It is also work that repeats across an organisation's small packages, each with its own mix of callback tests, plans and reporters. A run per package, each on its own machine and branch, turns a chore that would occupy someone for days into a set of pull requests that arrive with the same evidence in the same layout. The reviewer's job becomes comparing a before and after table and one planted failure per package.

Frequently asked questions

Does node:test have an equivalent of t.plan?

Recent Node versions have a t.plan() on the test context, but it is not needed for most ports. Returning the promise, or using an async function, tells the runner exactly when the test ends. The agent removes plans that only existed to end a callback-style test and keeps the assertion count visible in the description.

Why did the number of tests drop after the migration?

tape's summary counts assertions and node:test counts test functions. A file with 15 tests of three assertions each reports 45 under tape and 15 under the built-in runner. Nothing was removed, and the pull request states both numbers.

Do I still need tap-spec or another reporter?

No. The built-in runner prints a readable summary and can emit TAP or other formats through its --test-reporter option. Piping through an external formatter is no longer necessary, and removing it also removes that formatter's dependencies from your install.

Which Node version does the test suite need afterwards?

One where node:test is stable, which is Node 20 or later. That applies to running the tests, not to using your package. The agent leaves the published engines range alone unless you ask it to change it.

Will the agent rewrite my tests in a different style?

No. It keeps names, order and structure, including table-driven loops, and changes only what the new runner requires. Stylistic rewrites would make the diff hard to verify line by line, which defeats the purpose of a mechanical migration.

More jobs like this: Upgrade dependencies weekly

Keep reading