All jobs

Write Integration Tests for a Knex and Express API

Yes. An agent adds tests that call the real routes against a real Postgres, gives them their own database, and reports the bugs they expose in a PR.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

The agent starts from what exists, which here is nothing. In an API like robmclarty/knex-express-project-sample the test script in package.json is the string ha ha, no., there are no devDependencies and there is no test directory. What the repository does offer is a clean seam. server/index.js builds the Express app and exports it without listening, and server/start.js is the only file that calls listen, so a test can import the app and bind it to a free port. The models in server/models are thin wrappers over server/helpers/model-guts.js, which means a mocked database would test almost nothing: the queries are the logic.

So the tests talk to PostgreSQL. config/database.js picks knexfile[process.env.NODE_ENV || 'development'], and knexfile.js has development, staging and production but no test entry, so starting with NODE_ENV=test throws Cannot read properties of undefined (reading 'client'). The agent adds a test block that points at a database of its own, because the files in db/seeds begin by deleting every row of their table and a test run must never do that to development data. It installs PostgreSQL with sudo apt-get install -y postgresql, creates that database, and has a before hook run knex.migrate.latest() and knex.seed.run(). An after hook closes the server and calls knex.destroy(); without it the connection pool keeps the process alive and the test command never exits.

It adds no test framework when the runtime already has one. On a current Node the suite uses node:test, node:assert/strict and the global fetch, and the test script becomes node --test. The cases follow the route files: list, read, create, update and delete for users and projects, plus login and register from server/routes/auth_routes.js. Each asserts the status, the JSON shape against the selectableProps list of its model, and that a user body never contains password.

Written against the real routes, several of those tests fail, and that is the point. User.verify in server/models/user.js builds its query but never returns it, so POST /login answers 500 with Cannot read properties of undefined (reading 'then'), and the password check left running rejects with nothing to catch it, which ends the process. auth_controller.js references CONFLICT without importing it, so registering a taken username answers 500 with CONFLICT is not defined. getProjects ignores the user id in /users/:id/projects and returns every project, and GET /projects/:id for an id that does not exist answers 200 with an empty array and the message Project found. One obstacle comes before any of this: the locked bcrypt 3.0.8 and pg 7.18.1 do not install and connect on Node 22, so the agent starts from the dependency upgrade if it has merged, or makes that move as a separate first commit.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "claude",
  "input": "Write integration tests for this Express and Knex API and open a pull request. Read package.json, knexfile.js, config/database.js, server/index.js, server/start.js, every file under server/routes, server/controllers and server/models, server/helpers/model-guts.js, server/middleware/error_middleware.js and the migrations and seeds first. If npm ci or a database connection fails on the sandbox's Node, make the smallest dependency change that fixes it in its own commit and explain it. Install PostgreSQL with apt. Add a test environment to knexfile.js with its own database so tests never touch development data. Use node:test, node:assert/strict and fetch; add no test framework. Import the app from server/index.js, listen on port 0, run migrations and seeds in a before hook, and close the server and destroy the Knex pool in an after hook. Cover every route in user_routes.js, project_routes.js and auth_routes.js: status code, response shape, that password is never returned, a missing id, a duplicate username, a project created for a user that does not exist, and that one user's project list excludes another's. Do not change application code to make a test pass. Mark each test that fails because of a bug as a todo test whose reason names the file and line, and list those bugs in the pull request with the response you observed. Set the test script to node --test and paste its full output.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

The pull request adds test files, a test block in knexfile.js and a working test script, and no new dependency. Its description has a table of routes with what each test asserts, followed by the full output of node --test. In node:test a test marked todo still runs and prints its failure, but it does not fail the command, so npm test exits zero while every known bug stays visible in the output with the reason string beside it.

The second section is the bug list, and it is written from responses the agent received, not from reading. For each: the request, the status and body that came back, the file and line responsible, and the todo test that pins it. For the login route it includes the server's exit and stack trace, because a route that answers 500 and then stops the process is a different severity from one that only answers 500. It also notes that a project posted for a user id that does not exist answers 500 with the SQL text and the constraint name in the body, which says something about error_middleware.js as well as the route.

A reviewer checks that the test database name cannot be confused with the development one, then decides the order of the fixes. Removing a todo marker after a fix turns that case into a regression test, so each bug can be its own small pull request. The cases share one seeded database, so the reviewer also looks for tests that depend on rows another test created.

Run it on every weeknight

An automation in the Project that holds the repository, with a schedule trigger whose cron is 0 2 * * 1-5 and a timezone, starts one run each weeknight. Its prompt is narrower than this page's: list the routes and model functions with no test, write tests for the ones added or changed since the last run, run the suite against a fresh local PostgreSQL, and open one pull request, or end with one line when everything is covered.

To catch the gap earlier, add a second automation with a github trigger on pull_request.opened and pull_request.synchronize, marked read only. Its runs can read the branch and comment but cannot push, so the prompt asks only which changed routes or queries lack a test, and the author decides what to do with the answer. Each automation has its own max runs per hour.

The use case this job follows, with its automation and trigger: Write the missing tests.

Why run it on Rush?

A database-backed suite is only believable if it ran against a database. In a Rush sandbox the agent installs PostgreSQL, applies the migrations, seeds the tables and runs node --test, and the whole output is kept in the run's record. The bug list in the pull request comes from status codes and bodies the agent saw, with the commands that produced them in the transcript, so a reviewer does not have to rebuild the environment to believe a route really answers 500.

Some of these failures only exist at run time. The missing return in User.verify can be spotted by reading, but the fact that a login attempt stops the whole server only shows when the route is called and the process is watched. The sandbox is the agent's own machine, so a crashed server or a wiped table costs nothing, and the same request can run on several services at once, one repository per run.

Frequently asked questions

Why not mock Knex and skip the database?

Because the models here are almost entirely queries. A mock would assert that the code calls the functions the test told it to expect, and it would not have caught a list that ignores its filter or a lookup that answers 200 for a missing row. A local PostgreSQL in the sandbox is cheap, and the same tests run in CI against a service database.

Do the tests need Docker?

No. The sandbox has no Docker and the tests do not need it: the agent installs PostgreSQL with apt and connects over localhost. In CI, the test block in knexfile.js is an ordinary connection setting that you point at whatever database your pipeline provides.

Will it fix the bugs it finds?

Not in this request, which tells it to leave application code alone so the tests describe the app as it is. To have them fixed, send a follow-up with POST …/sessions/{id}/messages. That wakes the parked sandbox where it left off, with the database and dependencies still installed; parked sandboxes are kept for 7 idle days.

Why node:test and not Jest or Mocha?

The repository has no test runner and no devDependencies, and a current Node ships a runner, an assertion module and fetch. Adding nothing keeps the pull request about tests. If your repository already uses a runner, the agent writes in that one and follows its existing setup files.

More Knex.js jobs

The same job on other stacks

Keep reading