All jobs

Set Up GitHub Actions CI for a Slack Bolt App

Yes. An agent adds a CI workflow to your Bolt for JavaScript app that installs, syntax-checks and tests it without any Slack token, and opens a PR.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

The agent starts from what the repository already automates, because a first CI workflow has to fit beside it. In an app laid out like slack-samples/bolt-js-getting-started-app, package.json is an ES module with one script, start: node app.js, one dependency, @slack/bolt at ^5.1.0, a committed package-lock.json, no devDependencies, no test script and no engines field. app.js builds new App({ token: process.env.SLACK_BOT_TOKEN, socketMode: true, appToken: process.env.SLACK_APP_TOKEN }), registers app.message('hello', ...), which replies with a section block carrying a button whose action_id is button_click, registers app.action('button_click', ...), and calls app.start(process.env.PORT || 3000) at the bottom of the file, so importing the module starts the app. manifest.json grants the bot channels:history, chat:write and im:history, subscribes it to message.channels and message.im, and turns Socket Mode and interactivity on. .github/dependabot.yml checks npm weekly, and .github/workflows/dependencies.yml approves and enables gh pr merge --auto --squash for every patch and minor Dependabot pull request, with dependabot/fetch-metadata pinned to a commit SHA and a version comment. Nothing builds, lints or runs the app before that auto-merge fires.

Then it works out what can be proven without Slack. Socket Mode opens a WebSocket with the app-level token the moment app.start runs, so npm start is not a CI step: with no token it exits with an error, and with one it connects to a real workspace from a runner. The agent moves the one piece of logic that can be checked in isolation out of app.js: the greeting blocks become greetingBlocks(userId) in blocks.js, which returns the section block, the button and the plain-text fallback, and app.js imports it. A test directory gets blocks.test.js on node:test and node:assert, which asserts the button's action_id matches the string app.action listens for and that the fallback text is present, and manifest.test.js, which parses manifest.json and checks that chat:write is in the bot scopes and that both message.channels and message.im are subscribed, since app.message never fires without them. package.json gets "test": "node --test" and nothing new under devDependencies.

The workflow itself is .github/workflows/ci.yml: on push to main and on pull_request, permissions: contents: read, one job that checks out, sets up Node on the current LTS line with cache: npm, runs npm ci, node --check app.js and node --check blocks.js, npm test, and npm audit --audit-level=high. Each action is pinned to a commit SHA with a version comment, the same convention dependencies.yml already uses, so the two workflows read alike and Dependabot can bump both. The job is named ci, and the pull request description says to add it as a required status check on main, because that setting is the one thing that makes the existing gh pr merge --auto wait for a green run instead of merging the moment the approval lands, and it is a repository setting the agent cannot change from a pull request.

Before it opens anything, the agent runs the whole sequence in its own sandbox: npm ci against the lockfile, both node --check calls, node --test with the two new files, npm audit at the chosen level, and actionlint over ci.yml. No Slack token is set and no connection is made; the point of the exercise is that none is needed.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "claude",
  "input": "Add GitHub Actions CI to this Slack Bolt app. Read package.json, package-lock.json, app.js, manifest.json, .github/dependabot.yml and .github/workflows/dependencies.yml first. Do not start the app in CI: it uses Socket Mode and needs real tokens. Move the greeting blocks out of app.js into blocks.js as an exported function, and write node:test tests in test/ that check the button action_id matches the app.action listener and that manifest.json has the chat:write scope and the message.channels and message.im events. Add a test script that runs node --test, with no new dependencies. Write .github/workflows/ci.yml that runs on push to main and on pull requests with contents: read, sets up Node on the current LTS with npm caching, and runs npm ci, node --check on app.js and blocks.js, npm test and npm audit --audit-level=high. Pin each action to a commit SHA with a version comment, as dependencies.yml does. Run every step in your sandbox and actionlint on the workflow, and open a pull request that says to make the ci job a required check so the Dependabot auto-merge waits for it.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

What comes back is a pull request that adds .github/workflows/ci.yml, blocks.js, test/blocks.test.js and test/manifest.test.js, adds a test script to package.json, and changes app.js only where it now imports greetingBlocks. The description lists the five steps the job runs and why npm start is not one of them, pastes the node --test summary and the npm audit result from the sandbox, and ends with the one setting to change by hand: require the ci check on main.

The run's record keeps every command's output, so a reviewer reads the test output and the audit report before the workflow has run once on GitHub.

Before merging, a reviewer confirms the Node line in setup-node is the one the app is deployed on, decides whether an ESLint config should follow in a second pull request now that there is somewhere to run it, and turns on the required check so the auto-merge in dependencies.yml starts waiting for it.

Run it on every new Slack app repository

A team that keeps several Bolt apps sends this body once per repository from its own server, each a POST /o/{org}/sessions with that repository in resources and a request_id that is a UUID v5 of the repository's full name, so a retried call returns the run already started. Each run reads its own app: one on @slack/bolt with an HTTP receiver gets a step that boots the receiver with a dummy signing secret and checks the health route, which Socket Mode cannot offer, and one written in TypeScript gets tsc --noEmit before the tests.

Once the workflow exists, an automation on the Project with a schedule trigger (cron 0 9 * * 1 in your timezone, after Dependabot's weekly check) and the prompt "for each open pull request by dependabot that bumps @slack/bolt by a major version, read the package's release notes and comment on which listeners in app.js the new version changes" runs once a week; a pull request a bot opened fires no github trigger, so the schedule is what reaches it.

The use case this job follows, with its automation and trigger: Hand off a long task.

Why run it on Rush?

A Slack app is awkward to put under CI because the obvious smoke test, starting it, needs tokens and a workspace. Copied workflows either skip the app entirely or leak a token into a runner. The agent reads how the app starts, separates what can be checked from what needs Slack, and writes a job that proves the former without touching the latter.

Every step runs once in the sandbox and its output stays with the run, so the pull request shows what green looks like before the first workflow run. A team with many Bolt apps runs the same request across all of them at once, each in its own sandbox.

Frequently asked questions

Does CI need a Slack token?

No. The workflow never starts the app. It checks that the files parse, that the tests on the block payload and the manifest pass, and that the audit is clean, none of which talks to Slack. A SLACK_BOT_TOKEN or SLACK_APP_TOKEN in repository secrets stays out of the workflow.

Why move the blocks out of app.js?

Because app.js calls app.start at import time, any test that imports it starts the app. Putting the greeting blocks in blocks.js gives the tests a module with no side effects, and app.js changes by one import line. The behaviour in Slack is identical.

Will the Dependabot auto-merge still work?

Yes, and it gets safer. gh pr merge --auto merges as soon as the pull request is mergeable; with ci set as a required check, mergeable now means the tests passed on the bumped @slack/bolt. Without that setting the workflow still runs, but a red run does not stop the merge.

Does it add ESLint or Prettier?

No. The prompt forbids new dependencies, so the checks use Node's own --check flag and node:test. The pull request description says where an ESLint step would go in the workflow if the team wants one later.

More Slack Bolt jobs

The same job on other stacks

Keep reading