# Add Test Coverage to CI for a Slack Bolt App

Yes. An agent adds c8 coverage around your node:test suite, covers the listener registration files, sets a floor in CI and opens a PR with the report.

[Run this on Rush](https://cloud.getrush.ai/run/add-coverage-to-ci/slack-bolt/). Opens the console with this prompt ready to send, in a Project that has your repository.

## What the agent does

The agent reads how the app is tested today. In a Bolt app laid out like slack-samples/bolt-js-starter-template, `package.json` is an ES module with `@slack/bolt` 5 and `dotenv` as its only dependencies and Biome as its only dev dependency; `npm test` runs `node --test tests/**/*.spec.js` with no test framework beyond `node:test`, and `npm run lint` runs `biome check`. The code is six listener folders under `listeners/`, `actions`, `commands`, `events`, `messages`, `shortcuts` and `views`, each with a callback file and an `index.js` that registers it on the app, plus `listeners/index.js` that calls the six `register` functions, and two entry points, `app.js` for Socket Mode and `app-oauth.js` for an HTTP install flow. `tests/` mirrors the callbacks one spec file each, built on `mock.fn()` fakes for `ack`, `client`, `respond` and `logger`, with a happy path and an error-logging case per callback.

`.github/workflows/test.yml` runs `npm install`, `npm run lint` and `npm test` on Node 20, 22 and 24 for every push to `main` and every pull request, with a five-minute timeout; `dependencies.yml` approves and auto-merges Dependabot's patch and minor updates, and `dependabot.yml` schedules npm and `github-actions` monthly. No step measures coverage, and the six `index.js` registration files and `listeners/index.js` have no tests at all, so a listener that is written and tested but never registered passes CI.

The agent adds `c8` as a dev dependency and a `test:coverage` script, `c8 npm test`, which wraps the existing command through V8's built-in coverage rather than instrumenting files. A `.c8rc.json` at the root sets `all: true` so untested files count, `include` to `listeners/**`, `exclude` to the two entry points, since `app.js` and `app-oauth.js` start a real app and belong to a smoke test rather than a unit suite, reporters `text` for the job log and `lcov` for an artifact, and `check-coverage` with a `lines` floor chosen from the first report so the job passes on the current tree and fails on a drop. The `coverage/` directory joins `.gitignore` beside `node_modules`.

To make the registration files count, the agent writes `tests/listeners/register.spec.js` in the same style as the existing specs: a fake app whose `action`, `command`, `event`, `message`, `shortcut` and `view` methods are `mock.fn()`, a call to `registerListeners`, and assertions that each method was called once with the identifier the `manifest.json` declares, `/sample-command`, `sample_shortcut_id`, `sample_action_id`, `sample_view_id` and `app_home_opened`. CI gets one more step in `test.yml` after `npm test`, `npm run test:coverage`, and an `actions/upload-artifact` step for `coverage/lcov.info` from the Node 24 job only. The agent runs `npm run lint`, because Biome checks the new spec and config too, and `npm run test:coverage` on the sandbox's Node before opening the pull request, as `AGENTS.md` asks.

## Start it with one API call

`POST https://api.phoenix.so/o/{org}/sessions`

```json
{
  "agent": "claude",
  "input": "Add test coverage to CI for this Slack Bolt app. Read package.json, biome.json, manifest.json, AGENTS.md, app.js, every file under listeners/ and tests/, and .github/workflows/test.yml first. Add c8 as a dev dependency and a test:coverage script that runs c8 npm test. Write .c8rc.json with all: true, include listeners/**, exclude app.js and app-oauth.js, text and lcov reporters, and check-coverage with a lines floor set just under what the current suite reaches. Add coverage/ to .gitignore. Write tests/listeners/register.spec.js in the style of the existing specs: a fake app with mock.fn() methods, a call to registerListeners, and one assertion per listener that it was registered under the identifier manifest.json declares. Add an npm run test:coverage step to test.yml after npm test and upload coverage/lcov.info as an artifact from the newest Node in the matrix. Run npm run lint and npm run test:coverage until both pass, then open a pull request that includes the coverage table and explains the floor.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
```

1. **Get an API key.** In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
2. **Send the request.** POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
3. **Read what came back.** Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

## What comes back

The pull request adds `c8` to `devDependencies` and the `test:coverage` script in `package.json`, updates `package-lock.json`, adds `.c8rc.json`, `tests/listeners/register.spec.js` and the `coverage/` line in `.gitignore`, and extends `.github/workflows/test.yml` with the coverage and artifact steps. Its description carries the `text` table c8 prints, file by file, the floor it chose and why, and the Biome result.

The table is the first time the app has had a per-file number: the callbacks the existing specs cover, the registration files the new spec covers, and any branch in a callback that no test reaches, such as a `catch` block in a listener whose error path has no case yet. Those lines are listed in the description as the next tests worth writing rather than hidden behind a total.

Before merging, a reviewer checks that the floor is low enough to pass on the three Node versions in the matrix and high enough to catch an untested listener, decides whether the artifact should be uploaded from every matrix job or one, and whether to publish the `lcov` file to a coverage service later, which would be one more step and a token.

## Run it on every pull request

A team with several Slack apps on the same template sends this body once per repository from its own server, each a `POST /o/{org}/sessions` with that repository in `resources` and a `request_id` that is a UUID v5 of the repository's full name, so a retried call returns the run already started. Each run reads its own layout: an app that moved its listeners into TypeScript gets the same c8 wrapper around its compiled test command, and one that already uses Jest gets Jest's own `collectCoverage` instead.

Once the floor is in place, an automation on the Project with a `github` trigger on `pull_request.opened` and `pull_request.synchronize` and the prompt "if the coverage step failed, read the table, write the missing cases for the listener that dropped below the floor in the style of tests/, and push them to the pull request branch" starts one run per push. The trigger adds the pull request after the prompt, only work by an owner, member or collaborator fires it, and the run pushes to the contributor's branch only when the check failed.

## Why run it on Rush?

Coverage on a small app is less about the number than about what it reveals: here, that every callback had a test and no registration did, which is the one bug a Bolt app can ship without a failing test. The agent reads the suite, finds that gap, writes the test in the project's own `mock.fn()` style, and sets the floor from a real run rather than a round number.

The sandbox runs the lint and the test commands for real on a Node version from the matrix, and their output stays with the run, so the pull request shows the table the reviewer will see in CI. A team with many Slack apps runs the same request across all of them at once, each in its own sandbox, and each gets a floor fitted to its own suite.

## Frequently asked questions

### Why c8 instead of Node's built-in --experimental-test-coverage?

The matrix runs Node 20, 22 and 24, and the built-in reporter and its threshold flags differ across those versions, while c8 reads the same V8 coverage data and behaves the same on all three. The description notes that a repository dropping Node 20 can switch to the built-in flags later with the same floor.

### Why exclude app.js and app-oauth.js?

Both files construct an `App` and call `app.start()` at import, so loading them in a unit test opens a Socket Mode connection or an HTTP server. They are the place for a smoke test with a running Slack app, not for line coverage, and leaving them in would only lower the total without saying anything useful.

### Does the new spec test Bolt itself?

No. It passes a fake app with `mock.fn()` methods to `registerListeners` and asserts which identifier each listener was registered under, so it catches a listener that is written but never wired, or wired under a name `manifest.json` does not declare. Bolt's own routing stays Bolt's responsibility.

### Will the floor make every small pull request fail?

The floor is set just under the current total, so a change that adds a listener with a test keeps passing, and one that adds a listener without a test fails with the file named in the table. The `text` reporter prints that table in the job log, so the failing line is visible without downloading the artifact.

## More Slack Bolt jobs

- [Add a Custom Workflow Builder Step to a Slack Bolt App](https://getrush.ai/run/add-custom-workflow-step/slack-bolt/)
- [Write a Dockerfile for a Bolt for JavaScript Slack App](https://getrush.ai/run/add-dockerfile/slack-bolt/)
- [Deduplicate Retried Slack Events in a Bolt App](https://getrush.ai/run/add-event-deduplication/slack-bolt/)

## The same job on other stacks

- [Add Test Coverage to CI for a Streamlit App](https://getrush.ai/run/add-coverage-to-ci/streamlit/)
- [Add Test Coverage Reporting to Symfony CI](https://getrush.ai/run/add-coverage-to-ci/symfony/)
- [Add Code Coverage to CI for a Tokio Service](https://getrush.ai/run/add-coverage-to-ci/tokio/)

## Keep reading

- [How a Rush run works on your repository, from request to pull request](https://getrush.ai/cloud/)
- [Compare cloud agents, agent sandboxes and AI code review](https://getrush.ai/compare/)
- [Rush pricing: Free, Pro and Max](https://getrush.ai/pricing/)
