What the agent does
The agent reads how the app connects to Slack today, because that is the only thing changing. In a Bolt app like slack-samples/bolt-js-getting-started-app, package.json is an ES module with one dependency, @slack/bolt ^5.1.0, and one script, start running node app.js; there are no development dependencies and no tests. app.js builds new App({ token: process.env.SLACK_BOT_TOKEN, socketMode: true, appToken: process.env.SLACK_APP_TOKEN }), listens for messages containing hello with a Block Kit button whose action_id is button_click, acknowledges that action with ack() and replies, and calls app.start(process.env.PORT || 3000). manifest.json has socket_mode_enabled: true, interactivity enabled with no request URL, message.channels and message.im as bot events, and the scopes channels:history, chat:write and im:history. README.md tells a developer to create the app from the manifest and export the bot token and an app-level token.
Then it reads what Bolt does when Socket Mode is off. With socketMode absent and a signingSecret given, App builds an HTTPReceiver, which listens on the port passed to app.start, accepts POST on /slack/events by default (the endpoints option changes it), verifies Slack's request signature with the signing secret, answers Slack's url_verification challenge itself, and serves any customRoutes the app adds. processBeforeResponse and signatureVerification are the two options a reader is likely to meet, and the agent leaves both at their defaults. The button_click action and the two message events both arrive at that one endpoint, so one request URL covers the manifest's interactivity and event subscriptions.
The change is three files. app.js drops socketMode and appToken, adds signingSecret: process.env.SLACK_SIGNING_SECRET, and adds a customRoutes entry answering GET /health with a plain 200 so a load balancer has something to probe; the listeners are untouched. manifest.json sets socket_mode_enabled: false and adds a request_url under both event_subscriptions and interactivity, pointing at /slack/events on a placeholder host the README tells the reader to replace. README.md replaces the app-level token step with the signing secret from the app's Basic Information page, says the app now needs a public HTTPS URL that Slack can reach (a tunnel during development), and notes that Slack verifies the URL when it is saved, which the running app must answer. The agent follows AGENTS.md, which asks for no code comments, so the reasoning goes in the pull request instead.
A test proves the receiver without a Slack workspace. The agent adds a test script running node --test and test/receiver.test.js, which builds the App with the signing secret and tokenVerificationEnabled: false so the constructor makes no auth.test call, starts it on a free port the test picks, signs a url_verification body the way Slack does with the x-slack-signature and x-slack-request-timestamp headers, posts it to /slack/events and asserts the challenge comes back; a second case sends a wrong signature and asserts the request is refused; a third requests /health. It runs npm install, npm test and node --check app.js, and starts the app once with placeholder environment variables to confirm it binds the port and logs that it is running.
Start it with one API call
POST https://api.phoenix.so/o/{org}/sessions with this body:
{
"agent": "claude",
"input": "Move this Bolt for JavaScript app from Socket Mode to HTTP. Read package.json, app.js, manifest.json, README.md and AGENTS.md first. In app.js remove socketMode and appToken, add signingSecret from SLACK_SIGNING_SECRET so Bolt uses its HTTPReceiver on the existing app.start port, keep the message and action listeners as they are, and add a customRoutes entry that answers GET /health with 200. In manifest.json set socket_mode_enabled to false and add request_url under event_subscriptions and under interactivity pointing at /slack/events on a placeholder host. Update README.md: replace the app-level token step with the signing secret, say a public HTTPS URL is needed and that Slack verifies it on save. Add a test script using node --test and test/receiver.test.js that constructs the App with tokenVerificationEnabled false, starts it on a free port the test picks, posts a correctly signed url_verification body to /slack/events and expects the challenge back, posts a badly signed one and expects a refusal, and requests /health. Follow AGENTS.md: no code comments. Run npm install, npm test and node --check app.js, start the app once with placeholder variables to confirm it listens, and open a pull request that lists what a developer must change in their Slack app settings.",
"resources": [
{
"url": "https://github.com/your-org/your-app"
}
]
}- Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
- Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
- Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.
What comes back
What comes back is a pull request that changes app.js, manifest.json, README.md and package.json and adds test/receiver.test.js. The description is a checklist for the Slack side: turn Socket Mode off, set the request URL for events and interactivity, copy the signing secret, and reinstall if the manifest change asks for it; it also says the SLACK_APP_TOKEN variable is no longer read, so it can be deleted from wherever it is stored.
The run's record holds the npm test output with the challenge round trip and the refused signature, and the app's own startup log line, so a reviewer can see the receiver answered a signed request rather than trust the diff.
Before merging, a reviewer decides where the HTTPS endpoint will live and whether /slack/events should sit behind a path prefix, which the endpoints option changes, and whether the health route should check anything beyond the process being up. The agent keeps the default path and a plain 200 and says why.
Run it on every Bolt app moving to a hosted deployment
A team with several Socket Mode bots to host runs this once per repository from its own server: one POST /o/{org}/sessions per repository, each with that repository in resources and a request_id that is a UUID v5 of the repository's full name, so a retried call returns the run already started instead of opening a second pull request. Each run reads its own app.js and manifest and the pull requests arrive independently.
A key's webhook_url reports each run's end with the pull request in the payload, so the person doing the Slack settings changes gets one notification per bot with its checklist.
The use case this job follows, with its automation and trigger: Hand off a long task.
Why run it on Rush?
The part that fails in this migration is the signature check, and the only way to know the endpoint accepts Slack's request is to sign one the way Slack does and send it. The agent does that in its own sandbox against the real receiver, with no workspace involved, and the test stays in the repository so the next change to app.js is checked the same way.
Every command stays with the run, so the pull request can cite the test output and the startup line. Doing the same for every bot a team runs costs no more attention than one, because each run has its own machine and its own record.
Frequently asked questions
Why switch away from Socket Mode at all?
Socket Mode holds an outbound WebSocket from one process, which suits a laptop or a single container but not a deployment behind a load balancer with several replicas, and it needs an app-level token. The HTTP receiver is a normal web endpoint that scales like any other, verified per request with the signing secret.
Do the listeners change?
No. app.message('hello', ...) and app.action('button_click', ...) are the same code with either receiver. Only the App options and the manifest change, plus the customRoutes health route the agent adds.
How can the test run without a Slack workspace?
The receiver answers Slack's url_verification challenge itself after checking the signature, so the test signs a body with the same secret the app is given and posts it. tokenVerificationEnabled: false stops the App constructor from calling Slack's auth.test with the placeholder token.
What has to happen in the Slack app settings?
Socket Mode off, a request URL for event subscriptions and for interactivity, both pointing at the app's /slack/events, and the signing secret copied into the environment. Slack sends a challenge when the URL is saved, so the new version must be running and reachable first; the pull request lists these steps in order.