# Add Graceful Shutdown and Zero-Downtime Reloads with PM2

Yes. An agent makes your Express app finish in-flight requests and close MongoDB on SIGTERM, signal readiness to PM2, set reload timeouts, and opens a PR.

[Run this on Rush](https://cloud.getrush.ai/run/add-graceful-shutdown/pm2/). Opens the console with this prompt ready to send, in a Project that has your repository.

## What the agent does

The agent reads what happens when the process is told to stop. In a Node service laid out like hagopj13/node-express-boilerplate, `src/index.js` connects Mongoose, then listens, and on `SIGTERM` logs a line and calls `server.close()` with no callback: the process stops accepting connections but never disconnects Mongoose and never exits, so PM2 waits out its kill timeout and sends `SIGKILL`. `exitHandler`, used for `uncaughtException` and `unhandledRejection`, closes the server and then exits with code 1 even after a clean close. `ecosystem.config.json` runs one instance of `src/index.js` with `autorestart` and `time` set and nothing about readiness or timeouts, and `package.json` starts it with `pm2 start ecosystem.config.json --no-daemon`, which `docker-compose.prod.yml` runs as `yarn start` on a `node:alpine` image. Mongoose is 5.7 and Express 4.17.

It rewrites the shutdown as one function with a deadline. On `SIGTERM` or `SIGINT` the process logs the signal, calls `server.close` so new connections are refused while open ones finish, waits for the close callback, then closes the Mongoose connection with `mongoose.connection.close(false)` so queued operations drain first, and exits 0. A timer created with `unref` fires if that takes longer than the app's own deadline, logs which step was still running, and exits 1, so a stuck connection cannot hold the old process open for ever. The existing `exitHandler` is folded in: an uncaught exception runs the same sequence and keeps exit code 1, and a clean signal no longer reports failure. A second signal during shutdown is ignored rather than starting the sequence twice.

Then the PM2 side. The agent adds `process.send('ready')` after `listen` succeeds, guarded by whether `process.send` exists so `yarn dev` under nodemon still works, and sets `wait_ready: true` with a `listen_timeout` in the ecosystem file so PM2 counts the app as up only once MongoDB is connected and the port is open. It sets `kill_timeout` a little above the app's own deadline, since PM2's documented default is 1600 milliseconds and the Mongoose close alone can take longer, and switches `exec_mode` to `cluster` with `instances` still at 1, because `pm2 reload` starts the new process and waits for its ready message before stopping the old one only in cluster mode; in fork mode a reload is a restart with a gap.

The proof is a test and a reload. A new `tests/integration/shutdown.test.js` spawns `src/index.js` with `child_process`, waits for the listening line, opens a request to a route the agent adds for the test that answers after a delay, sends `SIGTERM`, and asserts the response arrives and the exit code is 0; the suite needs MongoDB, so the agent installs a server in its sandbox and sets `MONGODB_URL`. Then it runs `pm2 start ecosystem.config.json`, loops `curl` against `/v1/docs` while running `pm2 reload app`, and counts refused connections, which should be none. It runs `yarn test`, `yarn lint` and `yarn prettier` and keeps all of it.

## Start it with one API call

`POST https://api.phoenix.so/o/{org}/sessions`

```json
{
  "agent": "claude",
  "input": "Add graceful shutdown and zero-downtime reloads. Read package.json, ecosystem.config.json, src/index.js, src/app.js, src/config/config.js, docker-compose.prod.yml, Dockerfile and tests/utils/setupTestDB.js first. On SIGTERM and SIGINT stop accepting connections with server.close, wait for in-flight requests, close the Mongoose connection without forcing, and exit 0; add an unref'd deadline timer that logs the step still running and exits 1; keep exit code 1 for uncaughtException and unhandledRejection; ignore a second signal during shutdown. Send process.send('ready') after listen when process.send exists, and in ecosystem.config.json set wait_ready, listen_timeout, a kill_timeout above the app's deadline, and exec_mode cluster with instances 1. Write an integration test that spawns the process, holds a slow request open, sends SIGTERM and asserts the response completes and the exit code is 0. Run yarn test against a local MongoDB, then start under pm2 and run pm2 reload while a curl loop hits /v1/docs and count refused connections. Run yarn lint and yarn prettier, and open a pull request with the output.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
```

1. **Get an API key.** In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
2. **Send the request.** POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
3. **Read what came back.** Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

## What comes back

What comes back is a pull request that changes `src/index.js` and `ecosystem.config.json` and adds one integration test. The description shows the shutdown sequence with its deadline, the three ecosystem keys with the values chosen and why they are ordered the way they are, the integration test output, and the curl loop's count of refused connections across a `pm2 reload`.

The run's record keeps the pm2 logs from the reload, so a reviewer can see the ready message, the new process listening and the old one exiting 0 in order. A reviewer checks that the deadline is longer than the slowest request the app serves and shorter than the orchestrator's own grace period, and that the Docker image's `yarn start` still runs with `--no-daemon`, which a cluster-mode app honours.

## Run it on every service that restarts with dropped requests

A team with several Node services on this boilerplate starts one run per repository with one `POST /o/{org}/sessions` each, this prompt as `input` and the repository in `resources`. Each run reads its own `index.js` and ecosystem file, so a service that already drains connections gets only the readiness and timeout settings, and one that runs without PM2 gets the shutdown sequence and a description of what its process manager needs instead.

Send a `request_id` that is a UUID v5 of the repository's full name so a retried call returns the run it already started. The key's `webhook_url` reports when each run finishes, with `session_url` to read the pull request from.

## Why run it on Rush?

Graceful shutdown is a sequence with an order and a deadline, and it is almost never tested, because testing it means a real process, a real database and a signal sent at the right moment. Most services find out at the first deploy under load that `server.close` without an exit leaves PM2 to kill them. The agent writes the sequence, runs it as a spawned process with a request held open, and reloads under PM2 while counting refused connections, so the pull request carries the two results that matter.

The pull request is one entry file and one config file, so review is reading the order of the steps and three numbers. A company with many services on this boilerplate runs them all at once, each in its own sandbox with its own pull request.

## Frequently asked questions

### Why cluster mode with one instance?

Because `pm2 reload` overlaps the old and new process only in cluster mode; in fork mode it is a restart with a gap. One instance keeps memory and behaviour as they are. Raising `instances` later is a one-line change, and the description says what to check first, such as anything the app keeps in process memory.

### What does wait_ready change?

Without it PM2 treats the app as up the moment the process starts, so a reload can stop the old process before the new one has connected to MongoDB. With `wait_ready` and the `ready` message sent after `listen`, the old process is kept until the new one can serve, and `listen_timeout` bounds how long PM2 waits for that message.

### Does the test need PM2?

No. The integration test spawns `src/index.js` directly and sends the signal itself, so it runs under `yarn test` like the rest of the suite. The PM2 reload check is a separate step the agent runs in its sandbox and reports, since it depends on the ecosystem file rather than the code.

### What about Kubernetes or Docker stop?

Both send `SIGTERM` and wait a grace period before `SIGKILL`, so the same sequence applies; the deadline in the app and `kill_timeout` in PM2 both need to sit under that period. The agent names the three values in the description so an operator can line them up with the platform's.

## The same job on other stacks

- [Add Graceful Shutdown to a Quarkus Service](https://getrush.ai/run/add-graceful-shutdown/quarkus/)
- [Add Graceful Shutdown to a React Router App](https://getrush.ai/run/add-graceful-shutdown/react-router/)
- [Add Graceful Shutdown to a Rust Axum Service](https://getrush.ai/run/add-graceful-shutdown/rust/)
- [Add Graceful Shutdown to an Express and Sequelize App](https://getrush.ai/run/add-graceful-shutdown/sequelize/)
- [Add Graceful Shutdown to a Socket.IO Server](https://getrush.ai/run/add-graceful-shutdown/socket-io/)
- [Add Graceful Shutdown on SIGTERM to a Tokio Server](https://getrush.ai/run/add-graceful-shutdown/tokio/)

## Keep reading

- [How a Rush run works on your repository, from request to pull request](https://getrush.ai/cloud/)
- [Compare cloud agents, agent sandboxes and AI code review](https://getrush.ai/compare/)
- [Rush pricing: Free, Pro and Max](https://getrush.ai/pricing/)
