All jobs

Add Test Coverage to CI for a Koa API

Yes. An agent turns your Koa API's jest run into a coverage run with a threshold, uploads the report from CI, moves the job off a retired runner, and opens a PR.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

The agent reads the test setup and the workflow together, because coverage in CI is only useful if the job runs and the numbers are collected from the right files. In a Koa API laid out like javieraviles/node-typescript-koa-rest, package.json has test as jest, test:coverage as jest --collect-coverage, a jest block with roots, a testMatch of **/tests/**/*.+(ts|tsx|js) and ts-jest as the transform, on jest and ts-jest 26 with TypeScript 4.2, and no collectCoverageFrom, coverageThreshold or reporters. The tests are tests/controller/user.test.ts, which mocks typeorm and class-validator and exercises UserController, and tests/entity/user.test.ts, nine lines on the User entity; src/ also has config.ts, cron.ts, logger.ts, protectedRoutes.ts, unprotectedRoutes.ts, server.ts and controller/general.ts, which no test imports. tsconfig.json excludes tests, and .gitignore lists dist/**/* and *.log but not coverage.

.github/workflows/ci.yml is the second read. Its build job runs on ubuntu-16.04, a label GitHub retired, with actions/checkout@master and actions/setup-node@v1 on Node 12.x, then npm ci, npm run build and npm run test; three more jobs deploy to Heroku and run Newman and Locust against it, all on the same label. Nothing runs today, so the first change is the runner: the agent moves build to ubuntu-latest, actions/checkout to a release tag and actions/setup-node to a current major with the Node version from engines, which says 14, and notes that the deploy, integration and load jobs need the same move and the Heroku secrets, which it leaves to the reviewer.

Then the coverage configuration. The jest block gets collectCoverageFrom: ["src/**/*.ts", "!src/server.ts"], so untested files count against the number rather than disappearing from it, with server.ts excluded because it connects to Postgres and starts the process on import; coverageDirectory: "coverage"; and coverageReporters: ["text", "lcov", "json-summary"]. It runs npm run test:coverage in the sandbox, reads the table, and sets coverageThreshold.global at the measured lines and statements figures rounded down, so a pull request that lowers them fails and one that raises them does not. coverage goes into .gitignore and .dockerignore.

In the workflow, npm run test becomes npm run test:coverage, a step writes the totals from coverage/coverage-summary.json to $GITHUB_STEP_SUMMARY so the number is on the job page, and actions/upload-artifact keeps coverage/lcov-report for a reviewer to open. The agent runs npm ci, npm run build and npm run test:coverage the way the job will, and pastes the coverage table and the threshold it chose into the pull request.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "claude",
  "input": "Add test coverage to this Koa API's CI. Read package.json, .github/workflows/ci.yml, tsconfig.json, .gitignore, .dockerignore, Dockerfile, the two test files under tests and the files under src first. Add collectCoverageFrom for src excluding server.ts, a coverage directory, text, lcov and json-summary reporters, and a global coverageThreshold set at the lines and statements the current suite measures, rounded down. Add coverage to .gitignore and .dockerignore. In ci.yml, move the build job off ubuntu-16.04 to ubuntu-latest, pin actions/checkout to a release tag, update actions/setup-node and take the Node version from engines, run npm run test:coverage instead of npm run test, write the totals from coverage-summary.json to the job summary, and upload the lcov report as an artifact. Leave the Heroku jobs as they are and say what they need. Run npm ci, npm run build and npm run test:coverage, and open a pull request with the coverage table and the output. Do not write new tests.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

The pull request changes package.json, .github/workflows/ci.yml, .gitignore and .dockerignore. Its description pastes the coverage table from the sandbox, names the threshold it set and why, lists the files under src/ that have no test and now count, and says which actions it pinned and which jobs still need the runner and secrets fixed.

CI's build job runs on the pull request for the first time since the runner label was retired, with the coverage totals in its summary and the lcov-report artifact attached; the three Heroku jobs still wait on their label and are listed in the description.

Before merging, a reviewer decides whether the threshold should sit at the measured figure or a little under it to leave room for refactoring, whether cron.ts and logger.ts belong in the measured set or in an exclusion, and whether to fix the Heroku jobs in this pull request or retire them.

Run it on every nightly, before the test-writing run

A team that lets an agent write missing tests each night gives it a number to move. An automation on the Project with a schedule trigger (cron 0 2 * * * in the team's timezone) and the prompt "run npm run test:coverage, pick the file under src with the lowest line coverage, write tests for its untested paths without changing production code, and open a pull request" runs on a repository that has this change in place; each night's pull request raises the figure the threshold then protects.

For the first change itself, a team with many services starts one run per repository from its own server: one POST /o/{org}/sessions each, with that repository in resources and a request_id that is a UUID v5 of the repository's full name, so a retried call returns the run already started. Each run reads its own jest block and workflow, so a service that already collects coverage gets only the threshold and the summary step.

The use case this job follows, with its automation and trigger: Write the missing tests.

Why run it on Rush?

The number a threshold protects has to be measured before it is set, on the same files CI will measure, and the only honest way is to run the suite with the final configuration. The agent does that in its own sandbox on the Node version engines names, reads the table, and sets the threshold from it, so the pull request opens green with a threshold that means something rather than a round number that fails next week.

The run keeps the coverage table, the build output and the workflow diff, and the pull request quotes them, so a reviewer sees the measured figure without pulling the branch. For a team with many services, each gets its own run and its own pull request, each with its own measured threshold.

Frequently asked questions

Why exclude server.ts from coverage?

Because importing it calls createConnection against DATABASE_URL and starts the Koa listener, so a coverage run that pulled it in would try to reach Postgres. Excluding it keeps the measured set to the code the tests can load; a later integration test can bring it back.

Why does the job need to move runners at all?

ubuntu-16.04 is a label GitHub no longer provides, so every job in ci.yml has been queued and failed since it was retired. Coverage in a job that never runs is no coverage; the agent moves build and leaves the Heroku jobs for the reviewer, since they also need secrets the sandbox does not have.

Will the threshold make the first Dependabot or feature pull request fail?

Only if it lowers coverage. The threshold is the measured figure rounded down, so a change that keeps the suite as it is passes, and a change that deletes a test or adds untested code under src/ fails with the file and the figure in the log, which is the point.

Does the agent write tests for the untested files?

Not in this run; the prompt says not to. The run measures and reports, and the files with no test are listed in the pull request. A second run with a test-writing prompt, or the nightly schedule above, is where those numbers move.

More Koa jobs

The same job on other stacks

Keep reading