What the agent does
The agent reads how the process is shaped before it adds a signal handler, because in a clustered server the primary and the workers have to stop in different ways. In a server like socketio/chat-example, index.js is one ES module: when cluster.isPrimary, it forks one worker per availableParallelism() with PORT set to 3000 + i and calls setupPrimary() from @socket.io/cluster-adapter; otherwise it opens chat.db through sqlite and sqlite3, runs CREATE TABLE IF NOT EXISTS messages with an id, a unique client_offset and content, builds an Express app, createServer(app) and new Server(server, { connectionStateRecovery: {}, adapter: createAdapter() }), serves index.html on /, and ends with server.listen(port). package.json has type: module, depends on socket.io ^4.7.2, @socket.io/cluster-adapter ^0.2.2, express ^4.18.2, sqlite ^5.0.1 and sqlite3 ^5.1.6, and has a single start script. There is no test, no lint and no .github directory.
The message flow is what makes a clean stop safe. The chat message handler inserts the row and then broadcasts it with result.lastID as the server offset; a client that reconnects sends auth.serverOffset and, when socket.recovered is false, gets every row with a greater id replayed from the database. The browser side in index.html connects with ackTimeout: 10000 and retries: 3, so an emit that gets no acknowledgement because the worker was closing is retried with the same clientOffset, and the UNIQUE constraint on client_offset turns a duplicate into a no-op. Nothing in memory is needed to resume, which means a worker can close its sockets at any time as long as it finishes the insert it is in the middle of before it closes the database.
The worker gets a shutdown function that runs once on SIGTERM and SIGINT. It calls server.close() so no new connection is accepted, then io.close(), which disconnects every socket on that worker and lets the clients begin reconnecting, then awaits db.close() from the sqlite wrapper, and exits with code 0. A timer read from SHUTDOWN_TIMEOUT_MS, defaulting to ten seconds and unref'd so it never keeps the process alive, forces an exit with code 1 if a handler hangs. Because io.close() tears down client sockets before the database closes, the agent moves the db.run insert to finish before it looks at the closing flag, so a message that arrived just before the signal is still stored and acknowledged. The primary gets its own handler: on SIGTERM it sends SIGTERM to each entry in cluster.workers, waits for their exit events, and then exits itself, which is what a container runtime or systemd needs since only the primary receives the signal there.
The repository has no test runner, so the agent adds test/shutdown.test.js on node:test, a test script running node --test, and socket.io-client as a development dependency. The test spawns node index.js as a child process, waits for the server running at line a worker logs, connects a client to that port, sends SIGTERM to the child, and asserts that the client receives a disconnect event, that the child exits with code 0 before the timeout, and that chat.db opens cleanly afterwards, with no journal file left beside it. It runs npm test in its sandbox and also runs the server under node index.js, connects two browser tabs, sends a message, stops the server with kill -TERM, restarts it and shows the tabs reconnecting with the message history intact.
Start it with one API call
POST https://api.phoenix.so/o/{org}/sessions with this body:
{
"agent": "claude",
"input": "Add graceful shutdown to this Socket.IO server. Read package.json, index.js and index.html first. In each worker, handle SIGTERM and SIGINT once: stop accepting connections, close the Socket.IO server so clients reconnect, close the SQLite database, and exit 0, with a forced exit after a timeout read from SHUTDOWN_TIMEOUT_MS. Make sure an insert in progress finishes before the database closes. In the cluster primary, forward SIGTERM to every worker, wait for them to exit, then exit. Add a node:test test that spawns the server, connects a socket.io-client, sends SIGTERM and asserts the client is disconnected, the process exits 0 and the database reopens cleanly, plus a test script. Run the test, exercise the shutdown by hand with two browser tabs and show them reconnecting with history, and open a pull request with the output.",
"resources": [
{
"url": "https://github.com/your-org/your-app"
}
]
}- Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
- Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
- Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.
What comes back
The pull request changes index.js, package.json and package-lock.json, and adds test/shutdown.test.js. Its description shows the order of the four steps in the worker, the primary's forwarding loop, the node --test output, and the log lines from a manual stop and restart with the reconnecting clients.
A reviewer sees that the handler is a few dozen lines at the bottom of the file the server already lives in, that the forced-exit timer is the only new number and is configurable, and that the test drives a real child process rather than a mock. The run's record keeps the browser session with the two tabs and the restart.
Before merging, a reviewer picks the timeout for their deployment, decides whether the primary should re-fork a worker that dies outside a shutdown, which the file does not do today, and whether to log the drain steps through something other than console.log. The message handler and the recovery logic are unchanged.
Run it on every Socket.IO service that drops clients on deploy
A team with several Socket.IO services starts one run per repository from its own server: a POST /o/{org}/sessions each, with the repository in resources and a request_id that is a UUID v5 of the repository's full name, so a retried call returns the run already started. Each run reads its own entry point, so a service without cluster gets the worker handler alone, one on the Redis adapter gets its adapter connection closed in the same sequence, and one that already traps SIGTERM gets the missing steps added to the existing handler rather than a second one.
Afterwards, an automation on the Project with a github trigger on pull_request.opened and the prompt "if this pull request opens a new connection, timer or file handle in index.js, check that the shutdown function closes it and comment with the step to add" keeps the drain complete as the server grows. The trigger adds the pull request after the prompt, and only work by an owner, member or collaborator fires it.
The use case this job follows, with its automation and trigger: Hand off a long task.
Why run it on Rush?
A shutdown bug only shows up when a process is stopped, which is exactly when nobody is watching. The agent sends the real signal to a real child process in its sandbox, watches the client reconnect in a browser, checks the database file afterwards, and turns the whole sequence into a test, so the pull request proves the server drains rather than asserting it.
The test output, the server logs and the browser session stay with the run and are quoted in the pull request. A team with several real-time services runs the same change on each at once, each in its own sandbox, and reviews one pull request per repository with the same evidence attached.
Frequently asked questions
Do clients lose messages during a restart?
Not in this design. Offsets live in SQLite: the server replays rows with an id above the client's serverOffset when socket.recovered is false, and the client retries an unacknowledged emit with the same clientOffset, which the unique constraint on client_offset deduplicates. The handler finishes an in-flight insert before closing the database so the last message is stored.
Why does the primary need its own handler?
Under a container runtime or systemd, the signal goes to PID 1, which here is the cluster primary; workers are not in the same signal path. The primary forwards SIGTERM to each entry in cluster.workers, waits for their exit events and then exits, so the whole group stops together. From a terminal the shell signals the group anyway and the forwarding is harmless.
What does io.close() actually do?
It disconnects every connected socket on that server and closes the underlying HTTP server. Clients see a disconnect event and start reconnecting with the options set in index.html. The worker calls server.close() first so no new connection lands in between, then closes the database after the sockets are gone.
Why a forced exit timer?
A handler that waits on something that never completes, such as a database call that hangs, would otherwise keep the process alive until the orchestrator kills it. The timer, read from SHUTDOWN_TIMEOUT_MS with a ten-second default, exits with code 1 in that case so the failure is visible, and it is unref'd so it never delays a normal exit.