What the agent does
The agent reads how tests are meant to run before writing one. In an app shaped like OWASP/NodeGoat, package.json has "test": "node node_modules/grunt-cli/bin/grunt test", and Gruntfile.js defines that task as env:test followed by mochaTest:unit, whose src is test/unit/*.js with the spec reporter. That directory does not exist: test/ holds only e2e/ (Cypress specs run by .github/workflows/e2e-test.yml on Node 10, 12 and 14 against mongo:4.0) and security/profile-test.js, a Selenium and ZAP script behind the separate testsecurity task. mocha resolves to 2.5.3 and should to 8.4.0 in package-lock.json, mongodb to the 2.2.36 callback driver, and .jshintrc enforces double quotes, four-space indent, esversion 9 and "use strict" in every file, with lint.yml running grunt jshint on pull requests. The Dockerfile pins node:12-alpine, so the tests avoid syntax newer than that.
The code worth testing is the data layer in app/data/, seven constructor functions that each take a connected db and wrap a collection: UserDAO (addUser, validateLogin, getUserById, getNextSequence on the counters collection), AllocationsDAO (update with upsert, getByUserIdAndThreshold), ContributionsDAO (update, getByUserId with default values of 2), MemosDAO (insert, getAllMemos sorted by timestamp), BenefitsDAO, ProfileDAO and ResearchDAO. config/env/all.js reads MONGODB_URI with a mongodb://localhost:27017/nodegoat default, and artifacts/db-reset.js, run by the db:seed script, seeds users and counters. This app is a deliberately vulnerable teaching project, and several DAO methods carry comments marking the fix, such as the plaintext password in addUser and the $where string in getByUserIdAndThreshold.
So the tests pin current behaviour and name the vulnerabilities rather than fixing them, because changing app/ is out of scope for a test job. test/unit/user-dao.js connects with MongoClient.connect to MONGODB_URI with a nodegoat_test database name, drops it in before, and checks that addUser assigns the next _id from counters, that validateLogin accepts the right password and returns an error with invalidPassword set for the wrong one, that getUserByUserName finds a user case-sensitively, and that the stored password field equals the input, with a comment pointing at the A2 fix the file describes. allocations-dao.js checks update upserts and attaches userName, and that getByUserIdAndThreshold filters by stocks and that a non-numeric threshold is accepted as the $where string, documented as A1. contributions-dao.js checks the 2/2/2 defaults and the upsert; memos-dao.js checks ordering by timestamp descending. Every test uses the callback style the 2.2 driver and Mocha 2 expect, should assertions, and a done per test.
The agent runs the suite the project's way: npm ci, a MongoDB started in the sandbox the way e2e-test.yml does, then MONGODB_URI=mongodb://localhost:27017/nodegoat_test npm test, and npx grunt jshint so the new files pass the same lint as the app. It adds a unit-test job to lint.yml mirroring the e2e workflow's MongoDB step so grunt test runs on pull requests on Node 12 and 14, and pastes the Mocha spec output and the jshint result.
Start it with one API call
POST https://api.phoenix.so/o/{org}/sessions with this body:
{
"agent": "claude",
"input": "Write unit tests for this Express app's MongoDB data layer and open a pull request. Read package.json, package-lock.json, Gruntfile.js, .jshintrc, Dockerfile, the workflows under .github, config/env/*.js, artifacts/db-reset.js and every file in app/data first. Put the tests where the Gruntfile's mochaTest:unit task already looks, test/unit/*.js, one file per DAO, using the Mocha 2 and should versions in the lockfile, the callback-style mongodb 2.2 driver, double quotes, four-space indent and \"use strict\", and no syntax newer than Node 12. Connect to MONGODB_URI with a nodegoat_test database and drop it before the suite. Pin current behaviour, including the documented vulnerabilities in the DAO comments, and reference those comments instead of fixing the code; change nothing under app/. Cover UserDAO, AllocationsDAO, ContributionsDAO and MemosDAO at least. Add a unit-test job to the lint workflow that starts MongoDB like the e2e workflow does and runs npm test on Node 12 and 14. Run npm test against a MongoDB you start, and npx grunt jshint, and paste both outputs.",
"resources": [
{
"url": "https://github.com/your-org/your-app"
}
]
}- Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
- Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
- Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.
What comes back
What comes back is a pull request with four to seven new files under test/unit/, a shared test/unit/helpers/db.js for the connection and drop, a new job in .github/workflows/lint.yml, and no change under app/. The description lists each test with the DAO method it pins, marks the tests that document a known vulnerability by its OWASP label, and quotes the Mocha spec output and the jshint result from the sandbox.
lint.yml runs on the pull request, so the new unit-test job runs the suite on Node 12 and 14 against MongoDB as a check beside jshint, and e2e-test.yml still runs the Cypress suite unchanged.
A reviewer checks that the tests use a separate database name and never the seeded nodegoat one, that the vulnerability-pinning tests say plainly that they describe wrong behaviour so a later fix knows to flip them, and that no test depends on another's inserts. Then they decide whether grunt test should also become a required check.
Run it on every nightly schedule
Make this an automation in the Project with a schedule trigger, cron 0 2 * * * in your timezone, and a prompt that asks for tests on the modules changed in the last week. Each night one run reads git log, adds cases for a DAO or route handler that changed without tests, and opens one pull request on its own branch; a run whose suite fails ends needs_review instead. An automation has its own max_runs_per_hour cap, separate from any API key.
To start a run from your own tooling instead, send POST /o/{org}/sessions with this body and the repository in resources, and give the key a webhook_url so run.completed arrives with the session_url when the turn ends.
The use case this job follows, with its automation and trigger: Write the missing tests.
Why run it on Rush?
A configured test task pointing at an empty directory is the kind of debt that lasts years, because writing the first tests means standing up the database, matching an old driver's callback API and an old Mocha, and keeping jshint happy. In a Rush sandbox the agent starts MongoDB, runs grunt test for real and grunt jshint after it, and both outputs stay in the run's record beside the diff, so the reviewer reads a passing spec report rather than a description of one.
On a teaching codebase like this one the tests also serve as a map of the known vulnerabilities, each pinned and labelled, which is more useful to the next contributor than a fix nobody asked for. The same request works on any Express app with a data layer and a test task that runs nothing.
Frequently asked questions
Why does it not fix the plaintext passwords and the $where query?
Because this job writes tests and the prompt says to change nothing under app/, and because in NodeGoat those are intentional lessons with the fix in a comment beside them. The tests pin the current behaviour and cite the comment, so when someone applies the fix the test tells them exactly which assertion to flip.
Does it mock MongoDB?
No. The DAOs are thin wrappers over driver calls such as update with upsert and findAndModify, so a mock would test the mock. The tests run against a real MongoDB on a separate nodegoat_test database, started in the sandbox and in the new workflow job the same way e2e-test.yml already starts one.
Why Mocha 2 and should 8 instead of a current framework?
They are what package-lock.json resolves and what the mochaTest:unit task already runs, and the Dockerfile and CI run Node 10 to 14. Keeping them means npm test works as it always was meant to, with no new dependency and no syntax those Node versions reject.
Will the Cypress end-to-end suite still run?
Yes. e2e-test.yml is untouched and test:ci still runs Cypress against the started app. The new unit-test job in lint.yml is faster feedback on the data layer, not a replacement for the browser tests.
More Node.js jobs
The same job on other stacks
- Write Unit Tests for an Obsidian Plugin (Obsidian plugin)
- Write Unit Tests for a Payload CMS App (Payload CMS)
- Write Unit Tests for a Rust Project (Rust)