All jobs

Write Unit Tests for a Slack Bolt App

Yes. An agent splits your Bolt app's listeners from startup, tests them with node:test and mocked say and ack, adds a test script and CI workflow, and opens a PR.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

The agent reads the app before it tests it, because a Bolt listener is only testable once it is a function you can call without a Slack connection. In an app like slack-samples/bolt-js-getting-started-app, everything is in app.js: an App built with token: process.env.SLACK_BOT_TOKEN, socketMode: true and appToken: process.env.SLACK_APP_TOKEN; app.message('hello', …) which calls say with a section block containing Hey there <@${message.user}>!, a button accessory with action_id: 'button_click' and a plain text fallback; app.action('button_click', …) which awaits ack() and then say with <@${body.user.id}> clicked the button; and an async IIFE that calls app.start(process.env.PORT || 3000). package.json is type: module with @slack/bolt at ^5.1.0 pinned to 5.1.0 in package-lock.json, a start script and nothing else: no dev dependencies, no test script. manifest.json grants channels:history, chat:write and im:history and subscribes to message.channels and message.im. The only workflow, dependencies.yml, approves and auto-merges Dependabot patch and minor updates with no test gating them.

The agent moves the two handlers into listeners/messages.js and listeners/actions.js as named exports, helloMessage({ message, say }) and buttonClick({ body, ack, say }), and adds listeners/index.js with registerListeners(app) that calls app.message('hello', helloMessage) and app.action('button_click', buttonClick). app.js keeps the App construction and the start call and imports registerListeners. The behaviour is byte-for-byte the same; the handlers are now importable without tokens, a socket connection or a running App.

Tests use the runner Node ships, so no test framework is added. test/messages.test.js imports helloMessage, calls it with { message: { user: 'U123' }, say: mock.fn() } from node:test, and asserts with node:assert/strict that say was called once, that the payload's first block is a section whose text contains <@U123>, that its accessory is a button with action_id button_click, and that the text fallback is present so notifications read correctly. test/actions.test.js calls buttonClick with mocked ack and say, asserts both were called once, that ack was called before say by comparing the mocks' call order, and that say received <@U456> clicked the button for body.user.id of U456. A third test asserts registerListeners registers exactly the hello pattern and the button_click action id against a fake app that records calls.

package.json gains "test": "node --test", which discovers the test/ directory on the Node versions the agent targets, and .github/workflows/test.yml runs npm ci and npm test on push and pull request across a Node matrix. The agent runs npm ci, npm test and node --check app.js in its sandbox and includes the output. It does not run npm start, because that needs real Slack tokens, and it says so. manifest.json and dependencies.yml are untouched; the description notes that the Dependabot auto-merge now has a check to wait on if branch protection requires it.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "claude",
  "input": "Add unit tests to this Slack Bolt app. Read app.js, package.json, manifest.json and the GitHub workflows first. Move the message and action listeners into a listeners directory as named exported functions with a registerListeners(app) entry point, keeping app.js for App construction and start and keeping the behaviour identical. Write tests with node:test and node:assert/strict, no extra framework: the hello listener's say payload (section text with the user mention, button accessory with the button_click action id, text fallback), the button listener calling ack before say with the right mention, and registerListeners registering exactly the expected pattern and action id against a fake app. Add a test script that runs node --test and a workflow that runs npm ci and npm test on a Node matrix. Do not change manifest.json or the dependency auto-merge workflow. Do not start the app, since that needs Slack tokens; say so. Run the tests and include the output. Open a pull request.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

What comes back is a pull request that adds listeners/index.js, listeners/messages.js, listeners/actions.js, test/messages.test.js, test/actions.test.js, test/register.test.js and .github/workflows/test.yml, and changes app.js (construction and start only, listeners imported) and package.json (the test script). manifest.json, package-lock.json and dependencies.yml are unchanged.

The description lists each test and what it asserts, quotes the node --test summary and the node --check app.js result, and states that the app was not started because it needs SLACK_BOT_TOKEN and SLACK_APP_TOKEN. The commands and full output are in the transcript at GET /o/{org}/sessions/{id}/messages; the new workflow runs on the pull request.

Before merging, a reviewer diffs the moved handler bodies against the originals, decides whether the new check should be required before Dependabot's auto-merge, and installs the branch to a workspace once to confirm Socket Mode still connects. A follow-up message to the same session can add a test for the next listener.

Run it on every Slack app repository with no test script

A team that owns several Bolt apps starts one run per repository with one POST /o/{org}/sessions each, the repository in resources and this prompt as input. Each run reads its own listeners, so an app with shortcuts, views and client.chat.postMessage calls gets a mocked client and tests for each handler rather than the two in this example.

Send a request_id, a UUID v5 of the repository's full name, so a retried call returns the run it already started. The key's webhook_url reports when each run the key starts finishes, with the run's final answer in result.

The use case this job follows, with its automation and trigger: Write the missing tests.

Why run it on Rush?

Bolt apps stay untested because the handlers are closures inside app.message and app.action calls that need a connected App; the refactor that frees them is small but touches the whole file, and nobody wants to do it without a check. The agent does the split and the tests together in a sandbox of its own, runs them, and keeps the output.

The pull request carries the moved code, the test list and the passing summary, so a reviewer checks the diff of the handler bodies rather than setting up a runner. Several apps can be done at once, each in its own sandbox with its own pull request.

Frequently asked questions

Does testing need Slack tokens?

No. The tests call the listener functions directly with mocked say and ack, so no App is built and no connection is opened. Only npm start needs SLACK_BOT_TOKEN and SLACK_APP_TOKEN, and the agent does not run it.

Why node:test instead of Jest or Vitest?

The repository has no dev dependencies, and Node's built-in runner covers mocking, assertions and discovery of the test/ directory. Adding nothing keeps the Dependabot surface as it is. If your other apps standardise on Jest, say so in the prompt and the agent uses that instead.

Will the refactor change behaviour?

The handler bodies move unchanged into named functions and are registered with the same 'hello' pattern and 'button_click' action id; one test asserts exactly that registration. app.js still builds the App with the same options and starts it the same way.

What about Block Kit validation?

The tests assert the structure the handler builds: block type, text, accessory type and action id, and the fallback text. They do not call Slack's API to validate the payload; the manifest's scopes and the live workspace remain the end-to-end check.

More Slack Bolt jobs

The same job on other stacks

Keep reading