What the agent does
The agent reads the test convention before it writes a test. In a repository like koajs/examples, package.json runs NODE_ENV=test mocha --harmony --reporter spec --require should */test.js, with mocha ^5.0.0, should ^3.3.2 and supertest ^3.0.0 as dev dependencies, koa ^2.2.0, koa-body ^4.0.8, koa-static ^3.0.0, koa-logger ^3.0.0 and @koa/router ^8.0.5 as dependencies, and engines of node >= 7.6; .eslintrc.yml extends koa with mocha: true. Each example folder is an app.js and, in seventeen of twenty folders, a test.js that does const app = require('./app'); const server = app.listen(); const request = require('supertest').agent(server);, closes the server in after, and asserts with expect or should. Three folders have no test: upload, conditional-middleware and stream-server-side-events.
Two of the three cannot be tested as written. upload/app.js and conditional-middleware/app.js create new Koa() without module.exports and end with an unconditional app.listen(3000), so requiring either from a test binds port 3000 and exports nothing, where every other example writes const app = module.exports = new Koa(); and if (!module.parent) app.listen(3000);. The agent makes that same two-line change in both, which does not alter node app.js. upload/app.js also logs uploading %s -> %s on every POST; the agent guards it with the 'test' != process.env.NODE_ENV check compose/app.js and vhost/app.js already use, so the reporter output stays clean.
The tests follow the folder's own behaviour. For upload, koa-body runs with multipart: true, koa-static serves public/ with index.html and 404.html, a custom middleware redirects idempotent requests with no body to /404.html, and the POST handler reads ctx.request.files.file, streams it to a random name under os.tmpdir() and redirects to /. upload/test.js asserts GET / is 200 with the HTML from public/index.html, GET /missing is a 302 to /404.html, and a POST / built with supertest's attach('file', …) on a small buffer is a 302 to / and leaves a file in the temp directory, in the way multipart/test.js checks with fs.stat. For conditional-middleware, the agent exports ignoreAssets alongside the app and tests it directly with a recording middleware: wrapped, the middleware runs for / and /some.html and not for /style.css, /app.js or /favicon.ico, while the app still answers Hello World to all of them.
stream-server-side-events/app.js sets the type to text/event-stream; charset=utf-8, Cache-Control: no-cache and Connection: keep-alive, calls ctx.req.setTimeout(Number.MAX_VALUE), pipes db.subscribe() through the SSE transform in sse.js into the body, and unpipes on socket close or error. The subscription in db.js never ends, so a supertest expect would wait forever. The test instead uses http.get against server.address().port, asserts the three headers on response, reads until the body contains data: 0\n\n and data: 1\n\n, then destroys the request and asserts the server's connection count returns to zero, which is what the close() handler promises. The agent runs npm install, npm test and npm run lint in its sandbox and keeps the output, including the three new describe blocks in the spec reporter.
Start it with one API call
POST https://api.phoenix.so/o/{org}/sessions with this body:
{
"agent": "claude",
"input": "Write tests for the Koa examples in this repository that have none. Read package.json, .eslintrc.yml, .travis.yml and Readme.md, then list every folder with an app.js and no test.js and read those apps and any helper files beside them, plus hello-world/test.js, multipart/test.js, errors/test.js and compose/app.js for the conventions. Where an app calls app.listen unconditionally without module.exports, change it to the const app = module.exports = new Koa() and if (!module.parent) app.listen(3000) form the other examples use, and guard any console.log with the NODE_ENV test check compose/app.js uses. Write a test.js per folder with mocha, should and supertest: for upload, GET / serving public/index.html, GET of a missing path redirecting to /404.html, and a multipart POST with a file field redirecting to / and leaving a file in the OS temp directory; for conditional-middleware, export ignoreAssets and prove with a recording middleware that it skips .js, .css and .ico paths and runs otherwise; for the server-sent events example, use http.get to assert the event-stream content type and cache headers, read the first two data events, destroy the request and check the connection closes. Run npm install, npm test and npm run lint, include the output, and open a pull request.",
"resources": [
{
"url": "https://github.com/your-org/your-app"
}
]
}- Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
- Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
- Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.
What comes back
What comes back is a pull request that adds upload/test.js, conditional-middleware/test.js and stream-server-side-events/test.js, and changes upload/app.js and conditional-middleware/app.js only in their export and listen lines and the one guarded log. The other seventeen tests and package.json are unchanged, since the */test.js glob picks the new files up on its own.
The description explains why the two apps had to export themselves, shows the spec reporter output with the three new blocks, and quotes the lint result. The commands and their full output are in the transcript at GET /o/{org}/sessions/{id}/messages.
Before merging, a reviewer runs npm test once, checks that the SSE test cannot hang by confirming its timeout and destroy path, and decides whether the uploaded temp file should be removed in after.
Run it on every pull request that adds an example folder
Run it from an automation on the Project with a github trigger on pull_request.opened and pull_request.synchronize and the prompt "if this pull request adds a folder with an app.js and no test.js, write that test in the repository's mocha and supertest style and push it to this branch". The trigger puts the pull request after the prompt, and the agent checks out that branch; only pushes by an owner, member or collaborator fire it.
For the repository as it is now, send this body to POST /o/{org}/sessions with a request_id that is a UUID v5 of the repository's full name, so a retry returns the run already started. The key's webhook_url reports run.completed with session_url when the turn finishes.
The use case this job follows, with its automation and trigger: Write the missing tests.
Why run it on Rush?
The untested examples are untested for mechanical reasons, a port bound at require time, a stream that never ends, a multipart body to construct, and each needs a real server started and stopped to get right. The agent starts those servers in its own sandbox, sends the real requests, and watches the connection count fall, with every attempt kept in the run.
The pull request carries the spec output, so a reviewer sees the three new blocks pass under the same npm test command CI runs, and a repository of many small apps can be covered in one run rather than folder by folder on a laptop.
Frequently asked questions
Why change upload/app.js and conditional-middleware/app.js at all?
Both bind port 3000 the moment they are required and export nothing, so no test can reach them. The module.exports and module.parent lines are the convention the other eighteen apps already follow, and node app.js behaves exactly as before.
How does the SSE test avoid hanging?
The subscription stream never ends by design, so the test does not wait for the response to finish. It reads until the first two data: events arrive, destroys the request, and asserts the server has no open connections, inside mocha's default timeout.
Does the upload test leave files behind?
The handler writes to a random name under the OS temp directory, and the test checks that file exists with fs.stat the way multipart/test.js does. Cleaning it up in after is a one-line addition the pull request offers as an option.
Will the tests run on the Node version the repository targets?
They use async functions and http.get, which node >= 7.6 from engines supports, and the same mocha --harmony command. Nothing new is added to devDependencies.
More Koa jobs
The same job on other stacks
- Write Supertest Tests for an Express API (Express)
More jobs like this: Write the missing tests
- Write Terraform Tests for a Terraform Module (Terraform)
- Write Testcontainers Tests for a Spring Boot App (Spring Boot)