What the agent does
The agent reads what the library promises and what CI proves. In a package like sindresorhus/ky, the readme's first lines say it targets modern browsers, Node.js, Bun and Deno, and the .github/workflows/main.yml tests on macos-latest across Node latest, 24 and 22 with npx ava 'test/**' '!test/browser.ts', plus one job that installs Playwright and runs test/browser.ts. Nothing runs on Bun or Deno. package.json is "type": "module" with exports pointing at ./distribution/index.js, engines.node >=22, a build script of del-cli distribution && tsc --project tsconfig.dist.json, and a test script that chains xo, the build, tsc --project tsconfig.test.json and ava; AVA loads tests through --import=tsx/esm with workerThreads: false. .npmrc has package-lock=false. The test directory holds about forty files, and one of them, test/request-stream-support-runtimes.ts, already simulates Bun and Deno by replacing globalThis.Request and setting globalThis.Deno, which is a sign the real runtimes were never in the loop. source/utils/is-network-error.ts is where behaviour differs by runtime, and the readme says network errors are detected with runtime-specific heuristics.
AVA runs on Node only, so the agent does not try to port the suite. It writes test/runtimes/smoke.js, plain JavaScript that imports the built distribution/index.js and uses nothing but node:http, node:assert/strict and fetch, all of which Node 22, Bun and Deno 2 provide. The file starts a local server on an ephemeral port and runs the cases that matter across runtimes: a GET that parses JSON, a POST with a JSON body and the content-type it sets, searchParams appended to the URL, a 500 that is retried then succeeds with retry: 1, a 404 that throws HTTPError with the response attached, a timeout of 50 ms against a slow route that throws TimeoutError, a beforeRequest hook that adds a header, and a request to a closed port that throws NetworkError, which is the case is-network-error.ts has to get right on each runtime. It prints one line per case and exits non-zero on the first failure, so it reads the same in three CI logs.
The agent installs Bun and Deno in the sandbox, builds the package once, and runs the file three ways: node test/runtimes/smoke.js, bun test/runtimes/smoke.js and deno run --allow-net --allow-read test/runtimes/smoke.js. Where a case behaves differently, it reads the source before deciding who is right: Bun giving a HEAD response an empty body is documented in the readme, so the smoke test does not assert on it, while a NetworkError that one runtime surfaces as a bare TypeError would be a library bug, reported in the pull request with the runtime and version rather than papered over in the test.
In main.yml it adds two jobs, bun and deno, on ubuntu-latest, each checking out, installing Node to run npm install and npm run build, then oven-sh/setup-bun@v2 or denoland/setup-deno@v2 with the latest stable, and the one command. A runtimes script in package.json runs the Node variant so a contributor can run it locally, and the file passes xo with the project's rules, since npm test lints everything. The readme's support section gains two lines naming the Bun and Deno versions CI tests on.
Start it with one API call
POST https://api.phoenix.so/o/{org}/sessions with this body:
{
"agent": "claude",
"input": "Make CI test this TypeScript library on Bun and Deno as well as Node. Read package.json, .npmrc, every tsconfig, the readme's runtime support sections, the CI workflow, the test directory and the source files that branch on runtime first. Write a runtime-neutral smoke test in plain JavaScript that imports the built package and uses only node:http, node:assert and fetch, starting a local server and covering a JSON GET, a JSON POST, search params, a retried 500, an HTTPError on 404, a TimeoutError, a hook, and a NetworkError on a closed port. Install Bun and Deno in the sandbox, build the package, and run the file on Node, Bun and Deno; where a runtime differs, decide from the source and the readme whether it is a documented difference or a bug, and report any bug in the pull request instead of hiding it in the test. Add bun and deno jobs to the workflow that build and run the file, add a package script for the Node variant, make the file pass the project's linter, and note the tested versions in the readme. Open a pull request with all three runs' output.",
"resources": [
{
"url": "https://github.com/your-org/your-app"
}
]
}- Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
- Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
- Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.
What comes back
What comes back is a pull request that adds test/runtimes/smoke.js, edits .github/workflows/main.yml, package.json and readme.md, and changes nothing under source/. The description shows the smoke test's output from each of the three runtimes with the runtime version on the first line, and a short list of any case where a runtime differed and what the agent concluded from the source.
The run's record keeps the Bun and Deno installs, the build, the three runs and the xo pass, so a reviewer sees that the file ran where it claims to and lints under the project's own rules. If a runtime surfaced a real difference in how a network error is wrapped, that is in the description as a finding with the file and line in source/utils/is-network-error.ts, not as a special case in the test.
Before merging, a reviewer opens the workflow run on the pull request and sees bun and deno green beside the Node matrix, then reads the smoke test once to confirm the cases are the ones a Bun or Deno user would hit first. From then on a change that breaks on one runtime fails the job named for it.
Run it on every library that claims a runtime
A maintainer with several packages sends one POST /o/{org}/sessions per repository with this body and the repository in resources. Each run reads that package's own readme and workflow, so a library that promises Deno but not Bun gets one job, and one whose tests already run on Bun gets only the Deno job and the smoke test. A request_id that is a UUID v5 of the repository name makes a retried request return the run it already started.
The key's webhook_url reports each run the key starts, with the pull request URL in result, so a script that sends one request per package collects the pull requests without polling.
The use case this job follows, with its automation and trigger: Write the missing tests.
Why run it on Rush?
A runtime claim in a readme is only as good as the CI job behind it, and writing that job means installing two more runtimes, building the package and watching the same cases behave on each. The agent does that in its own sandbox and keeps every run's output, so the pull request carries the proof instead of a promise.
Where runtimes disagree, the agent reads the source and the readme and reports a finding rather than bending the test, which is the difference between a job that catches regressions and one that documents them.
Frequently asked questions
Why a separate smoke test instead of running AVA on Bun and Deno?
AVA runs on Node, and the suite relies on tsx, Express and Playwright. A plain JavaScript file against the built package tests what a Bun or Deno user actually imports, and it runs the same way on all three runtimes with no test framework in between.
Does the smoke test replace the existing suite?
No. The Node matrix and the browser job keep running the full AVA suite. The smoke test covers the behaviour that differs by runtime, such as network error detection and timeouts, and is deliberately short so it stays easy to run on a new runtime.
What if Bun or Deno fails a case?
The agent reads the source to decide whether the runtime's behaviour is documented, as Bun's empty HEAD body is in the readme, or a bug in the library's runtime detection. A documented difference is left out of the assertions; a bug is reported in the pull request with the file and line, and the job stays red until it is fixed.
Which versions does CI test?
The latest stable Bun and Deno at the time the job runs, through the two setup actions, with the version printed on the smoke test's first line in the log. The readme names the versions the pull request's run used, and pinning is one line in each job if the team prefers it.
More TypeScript library jobs
More jobs like this: Write the missing tests
- Run Docstring Examples as Tests in CI With pytest (pytest)
- Add Playwright End-to-End Tests to a Chrome Extension (Chrome extension)
- Add End-to-End Tests to a Tauri App (Tauri)