All jobs

Add Prometheus Metrics to a Socket.IO Server

Yes. An agent adds prom-client to your Socket.IO server: gauges for connected sockets, counters per event, pool and HTTP metrics on /metrics, with tests, as a PR.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

The agent reads how the server is assembled before it instruments it. In a project shaped like socketio/socket.io-chat-platform, server/src/index.js exports createApp(httpServer, config), which opens a pg.Pool, applies the SQL files in sql/ with postgres-migrations, mounts an Express app with etag and x-powered-by off, and creates new Server(httpServer, { cors, adapter: createAdapter(pgPool) }) with the PostgreSQL adapter so several nodes share rooms. A connection middleware joins each socket to its channel rooms, its user room and its session room; the connection handler registers channel:create, channel:join, channel:list, channel:search, user:get, user:reach, user:search, message:send, message:list, message:ack and message:typing, each a handler that validates its payload with Ajv and answers through an acknowledgement callback, and a disconnect handler that waits a grace delay before marking the user offline. entrypoint.js is the production entry with JSON logs from winston and a SIGTERM handler that awaits close(); the root compose.yaml runs three backend containers behind nginx, whose upstream hashes on $remote_addr and proxies only /api/.

There is no metrics library in server/package.json and no /metrics route, so the agent adds prom-client and a src/metrics.js that owns one Registry, calls collectDefaultMetrics on it for event loop lag, heap and GC, and defines the application metrics: a socket_io_connected_sockets gauge set from io.engine.clientsCount on each connection and disconnection, a socket_io_events_total counter with an event label incremented from socket.onAny, a socket_io_event_duration_seconds histogram measured by wrapping each registered handler from the moment the event arrives until its acknowledgement callback fires, with a status label taken from the OK or ERROR the handler answers, and pg_pool_connections gauges for the pool's totalCount, idleCount and waitingCount read at scrape time. The Express side gets an http_request_duration_seconds histogram over /login, /signup, /logout and /self, the four routes src/auth/ mounts.

The route is GET /metrics on the Express app, answering with register.contentType and register.metrics(). Because nginx proxies only /api/, that route is reachable inside the compose network and not from the browser, which is the right default; the agent adds a prometheus service to a compose override with a scrape config that lists backend01, backend02 and backend03 on port 3000, since each process has its own registry and a hashed upstream would otherwise show one node's numbers. It leaves createApp's return shape alone except for exposing the registry, so index.js, entrypoint.js and the tests keep working.

The repository tests with mocha against a real PostgreSQL, so the agent installs PostgreSQL in its sandbox, sets the postgres password to the changeit that test/util.js expects, and runs npm test. It adds test/metrics.js, which reuses the existing setup() helper, sends a message from Alice's first socket, fetches /metrics and asserts that socket_io_connected_sockets is at least three, that socket_io_events_total{event="message:send"} is present and that the pool gauge appears. It runs npm run format so Prettier accepts the new files, and keeps the full test output for the pull request.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "claude",
  "input": "Add Prometheus metrics to this Socket.IO server and open a pull request. Read server/package.json, server/index.js, server/entrypoint.js, server/src/index.js, server/src/util.js, every handler under server/src/, server/src/auth/index.js, server/test/util.js, one existing test, server/Dockerfile, compose.yaml and nginx/nginx.conf first. Add prom-client with one registry and default metrics; add a gauge for connected sockets from the engine's client count, a counter per event name from socket.onAny, a histogram of handler duration from event arrival to acknowledgement with a status label, gauges for the pg pool's total, idle and waiting counts, and a histogram for the Express auth routes. Serve GET /metrics on the Express app. Keep the metrics route off the public nginx path and add a compose override with a Prometheus service that scrapes each backend. Install PostgreSQL in your sandbox with the password the tests expect, add a mocha test that sends a message and asserts on the scraped metrics, run npm test and npm run format, and open a pull request with the test output and a sample of the /metrics text.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

The pull request adds server/src/metrics.js and server/test/metrics.js, changes server/src/index.js to register the metrics and the route, adds prom-client to server/package.json and package-lock.json, and adds a compose override with a Prometheus scrape config. Its description shows the mocha output for the whole suite and a trimmed sample of the /metrics text with the new families and their help strings.

In production each backend answers /metrics with its own counts, Prometheus scrapes the three by service name, and the dashboards that follow are sums and rates over them: sockets per node, events per second by name, acknowledgement latency by event and status, and a pool that is waiting more than it is idle. None of this changes what a client sees; the handlers still answer the same acknowledgements.

Before merging, a reviewer decides whether the per-event histogram buckets fit their latencies, whether /metrics should also require a network policy or a token where the compose network is not the boundary, and whether the collectDefaultMetrics prefix should carry the service name.

Run it on every real-time service without metrics

A team with several Socket.IO services sends this body once per repository to POST /o/{org}/sessions, each with its own repository in resources and a request_id that is a UUID v5 of the repository's full name, so a retried call returns the run already started. Each run reads its own handlers and adapter and names its own metrics, and the key's webhook_url receives one run.completed per repository with the pull request in result.

After the first pull request, the useful recurring run is the one that keeps the metrics honest. An automation with a github trigger on pull_request.opened and pull_request.synchronize starts a run that reads the diff for new socket.on handlers or routes and comments when one is missing from the instrumentation, so coverage of events does not drift. An automation run is not tied to a key and has its own max_runs_per_hour cap.

The use case this job follows, with its automation and trigger: Hand off a long task.

Why run it on Rush?

Metrics for a Socket.IO server are not a middleware you drop in: the interesting numbers live in acknowledgement callbacks, in the engine's client count, in the adapter and the pool, and in the fact that three processes sit behind one hashed proxy. The agent reads those specifics in its own sandbox, places each metric where the event actually happens, and proves it with a test that drives a real socket through a real PostgreSQL.

The whole run is kept, from installing PostgreSQL to the mocha output and the scraped text, so the review is about bucket choices and exposure rather than whether it works. Several services can be instrumented at once, each in its own sandbox.

Frequently asked questions

Will the metrics be correct with three backends behind nginx?

Each process keeps its own registry, so Prometheus must scrape each backend, which the compose override does by service name. Queries then sum or rate across the instance label; scraping through the hashed nginx upstream would show one node at random, which is why /metrics stays off the proxied path.

How is handler duration measured when handlers answer through a callback?

The agent wraps each handler at registration: the timer starts when the event arrives and stops when the handler calls its acknowledgement, with a label for the OK or ERROR status it answered. A handler that returns without calling back is not observed, which matches how the handlers already treat a missing callback.

Does it add metrics for the PostgreSQL adapter?

It exposes the pool the adapter and the app share, through totalCount, idleCount and waitingCount. Adapter-level counts of broadcast packets are not exposed by the adapter's public API, so the pull request says that a per-room gauge would need the admin UI or a custom hook.

Do the existing tests still pass?

Yes. createApp returns the same logger, pgPool and close it did, with the registry added, and the new test uses the same setup() helper the others use, so the suite runs as one npm test against the same database.

More Socket.IO jobs

The same job on other stacks

Keep reading