All jobs

Replace node-fetch With Native fetch in a Node.js Library

Yes. An agent removes node-fetch for Node's built-in fetch, repairs the HTTP mocks and the agent option that stop working, and opens a pull request with tests.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

The agent reads the one module that makes requests and everything that leans on it. In a library like pusher/pusher-http-node that module is lib/requests.js. It requires node-fetch and abort-controller, builds a signed URL, and calls fetch(url, { method, body, headers, signal, agent: config.agent }), wrapping any failure in the library's own RequestError. Around it: index.d.ts imports the Response type from node-fetch and declares agent?: Agent; the README tells users to pass new http.Agent({ keepAlive: true }); @types/node-fetch sits in dependencies; engines.node says >= 10.0.0 while .github/workflows/test.yml tests 18.x, 20.x and 22.x. The tests are Mocha with nock 12.0.3, and npm run local-test reported 129 passing when this page was written.

The code change is five deleted lines, since fetch and AbortController are globals in every Node version the workflow tests. The agent makes it and runs the suite, and the result is the first real finding: 103 passing and 26 failing, in about 20 seconds where the suite used to finish in under one. nock 12 patches Node's http module, and the built-in fetch does not go through it, so the mocks never match and the calls leave the machine. Some came back from the real host as Unexpected status code 400. Upgrading nock to the newest release does not help directly: nock 15.0.5 is an ES module, and the tests' require("nock") then fails in every hook with nock.disableNetConnect is not a function. nock 14.0.17 still loads with require and does intercept fetch, and with it the suite is at 128 passing and 1 failing. The last one matches request headers host and content-length as well as content-type, and the agent narrows that expectation and explains which header no longer reaches the mock.

Then it tests what no existing test covers. It starts a local server and makes one request through each client with an http.Agent whose createConnection is counted. node-fetch 2.7.0 opens its connection through the agent. Node 22's fetch completes the request and never touches it. So the library's documented agent option, which is how users enable keep-alive or a proxy, would be accepted and ignored. Errors change shape too: a refused connection is a FetchError with code: 'ECONNREFUSED' from node-fetch and a TypeError: fetch failed with no code from the built-in, and an abort is an AbortError class in one and a DOMException named AbortError in the other. Both end up in RequestError.error, which callers can read.

Those two findings make this a breaking change, and the agent treats it as one. It adds a dispatcher option that is passed to fetch, makes agent throw a clear error instead of being dropped, updates index.d.ts to the global Response type and the README examples, raises engines.node to 18, and removes node-fetch, abort-controller and @types/node-fetch, which takes 12 packages out of the tree. It adds a test for the new option and one for the rejected agent, and proposes a major version in the changelog without publishing anything.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "claude",
  "input": "Replace node-fetch with Node's built-in fetch in this library. Read package.json, lib/requests.js, lib/config.js, lib/errors.js, index.d.ts, the README section on configuration, the tests under tests/integration and .github/workflows/test.yml first. Run npm ci and npm run local-test and record the count and the duration. Remove the node-fetch and abort-controller requires, run the suite again, and explain every failure: check whether the installed HTTP mocking library intercepts built-in fetch, and whether requests are reaching the network. Find a version of the mocking library that intercepts fetch and still loads the way the tests load it, and get the suite passing without weakening what each test asserts beyond what is needed; say which assertions changed and why. Then test what the suite does not: with a local HTTP server, compare node-fetch and built-in fetch on whether the agent option is used, on the error for a refused connection, and on the error for an aborted request. If a documented option stops working, do not leave it silently ignored: replace it with the built-in equivalent, reject the old one with a clear message, and update the types and the README. Raise engines.node, remove the packages that are no longer needed, and add tests for the new behaviour. Do not run the remote tests, publish, or tag. Open a pull request with the output of every command and a list of breaking changes.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

A pull request that changes lib/requests.js, lib/config.js, index.d.ts, the README, package.json, package-lock.json and several test files. Its description opens with a list of breaking changes: the minimum Node version, the agent option replaced by dispatcher, and the class and fields of the error found in RequestError.error. Under that are the suite results at each stage, 129, then 103 with 26 failing, then 128, then all passing, and the output of the comparison script for the agent option and the two error cases.

A reviewer starts with the breaking list, because the code diff is small and the surface change is not. They decide whether rejecting agent is right for their users or whether a deprecation period with a warning is kinder. They check the mock upgrade: nock 14 declares node >=18.20.0 <20 || >=20.12.1, which fits the 18.x, 20.x and 22.x matrix. And they note what was not run: npm test in CI also runs tests/remote, including a proxy test, against a real Pusher app with a secret the sandbox does not have.

Run it on every package that still ships node-fetch

Dropping the polyfill is done once per package. For an org with several SDKs or services, send this body to POST /o/{org}/sessions once for each repository whose dependencies include node-fetch, with the repository in resources and a request_id that is a UUID v5 of its name, so repeating the batch starts nothing twice.

Libraries and applications come back differently, and the prompt handles both: an application has no public option to break, so its pull request is mostly the mock upgrade, while a library's leads with the breaking list. A key runs up to max_concurrent at once and the rest wait as queued; if the key has a webhook_url, each run posts its result there when the turn ends.

The use case this job follows, with its automation and trigger: Hand off a long task.

Why run it on Rush?

Two of the three problems in this migration produce no error. Mocks that stop intercepting let tests call the real network, and an option that is ignored returns a normal response. Seeing either takes a controlled environment: a local server, a counted connection, a suite run where the duration is noticed. The agent builds that in its sandbox, runs both clients side by side at the locked versions, and keeps the output, so the claim that agent is ignored comes with the script and the line it printed.

It also matters that the work happens off anyone's laptop. A test suite that suddenly sends signed requests to a production API host is not something to discover with real credentials in the environment. The sandbox has only what the run was given, the failing calls carry the made-up keys from the tests, and the run's record shows each of those calls with the URL in its error.

Frequently asked questions

Why did my nock tests start failing after switching to fetch?

Older nock versions intercept Node's http and https modules, and the built-in fetch does not use them. In this repository nock 12.0.3 intercepted nothing after the switch and 26 tests failed, some with responses from the real host. nock 14 intercepts fetch and still loads with require; nock 15 is an ES module, so tests that use require fail in their hooks.

Does native fetch support the agent option?

No. Passing an http.Agent is accepted without error and has no effect: in a direct test on Node 22 the request completed without opening a connection through the agent. Keep-alive and proxy settings go through a dispatcher instead, so a library that exposed agent has to change its public options.

Which Node version can drop node-fetch?

fetch has been available without a flag since Node 18 and stopped being experimental in Node 21. A package that declares support for older versions, as this one does with >= 10.0.0, has to raise engines.node in the same release.

Do error handlers need to change?

If they inspect the underlying error, yes. node-fetch throws a FetchError carrying a code such as ECONNREFUSED; the built-in throws a TypeError with the message fetch failed and no code on it. Abort handling that checks err.name === 'AbortError' keeps working with both.

More jobs like this: Hand off a long task

Keep reading