All jobs

Add Slow Query Logging to a Mongoose App

Yes. An agent adds a Mongoose plugin that times every query and aggregate, logs the slow ones with collection, operation and filter keys, tests it and opens a PR.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

The agent reads how queries are made and how the app logs before it adds a timer, because Mongoose's own debug option prints every operation and never says how long one took. In an API built from hagopj13/node-express-boilerplate, mongoose resolves to 5.13.20 in yarn.lock; src/index.js connects with the URL and options from src/config/config.js, where a Joi schema validates every environment variable and useCreateIndex, useNewUrlParser and useUnifiedTopology are set. The models in src/models/ build their schemas with mongoose.Schema and share two plugins from src/models/plugins/, toJSON and paginate, applied per schema. Logging is winston in src/config/logger.js, with a printf format of level: message and a console transport, and src/config/morgan.js logs each request's :response-time through it, so the app already knows how long a request took but not which query inside it was slow.

It then writes a third plugin beside the two that exist, src/models/plugins/slowQuery.plugin.js, and registers it once with mongoose.plugin() in src/index.js before any model is required, so every schema gets it without editing each model file. The plugin adds a pre hook that records process.hrtime.bigint() on this and a post hook that computes the elapsed milliseconds, both on a regular expression matching the query middleware names (find, findOne, countDocuments, updateOne, updateMany, deleteOne, deleteMany, findOneAndUpdate and the rest) and separately on aggregate, whose this is an Aggregate with a pipeline rather than a Query with a filter. Past a threshold it logs one line through the winston logger at warn: the collection name, the operation, the elapsed time, and for a query the keys of this.getFilter(), never the values, since filters in this app hold emails and tokens.

The threshold is configuration, not a constant. A SLOW_QUERY_MS key joins the Joi schema in config.js with a default and a description, is read as config.mongoose.slowQueryMs, and gets a commented line in .env.example. A value of 0 logs every operation, which is what the test uses. The agent leaves mongoose.set('debug') alone, because that prints the full filter with its values to stdout and the repository's ESLint config, with the airbnb base and no-console as an error, is a sign the project does not want that in production logs.

In its sandbox the agent starts MongoDB with docker compose up mongodb from docker-compose.yml, runs yarn lint and yarn test, then adds tests/unit/models/plugins/slowQuery.plugin.test.js next to the two plugin tests that exist. It uses setupTestDB from tests/utils/, sets the threshold to 0, spies on logger.warn (the suite's restoreMocks: true in jest.config.js undoes the spy after each test), runs User.find({ email: 'x' }), User.countDocuments() and a one-stage User.aggregate(), and asserts one warning each naming users, the operation, and for the find the key email and not the value. A last test raises the threshold high and asserts no warning.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "claude",
  "input": "Add slow query logging to this Mongoose app. Read src/index.js, src/config/config.js, src/config/logger.js, src/models/plugins/ and src/models/user.model.js first. Write a plugin in src/models/plugins/ with pre and post hooks on the query middleware and on aggregate that time each operation with process.hrtime.bigint() and log through the winston logger at warn when it exceeds a threshold: collection, operation, elapsed ms and the filter's keys only, never its values. Register it globally with mongoose.plugin() in src/index.js before models load. Add SLOW_QUERY_MS to the Joi schema in config.js with a default and an .env.example line; 0 logs everything. Do not enable mongoose debug. Add a unit test beside the existing plugin tests using setupTestDB, a spy on logger.warn and the threshold at 0, covering find, countDocuments and aggregate, plus one with a high threshold that logs nothing. Run yarn lint and yarn test against a MongoDB from docker-compose.yml and quote the results. Open a pull request.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

What comes back is a pull request with one new plugin file and its test, a registration line in src/index.js, a key in the Joi schema in src/config/config.js, and a line in .env.example; no model file changes. The description shows one sample warning line as the logger formats it and quotes yarn lint and yarn test passing with the new test's cases listed.

The run's record keeps the test output, so a reviewer can see the three warnings the threshold of 0 produced, with users and the operation names, and the absence of a warning at the high threshold, without running the suite.

Before merging, a reviewer picks the production threshold and decides whether warn is the right level for their alerting, since the morgan handler already logs slow requests at info and the new line is the one that says which query inside the request was slow.

Run it on every Mongoose service

A team with several services built from the same boilerplate starts one run per repository from a script on its own server: GET /o/{org}/github lists every repository the installation reaches, and for each one whose package.json depends on mongoose the script calls POST /o/{org}/sessions with this body and that repository in resources, with a request_id that is a UUID v5 of the repository's full name, so running the script twice returns the runs already started.

Each run reads its own repository. One that already registers a global plugin gets the hooks added to it; one on Mongoose 6 or later, where remove middleware is gone and count is deprecated, gets a pattern that matches the operations that release has; one with a different logger gets the warning sent through that logger. The pull requests arrive one per repository, each with its own test run in the record.

The use case this job follows, with its automation and trigger: Hand off a long task.

Why run it on Rush?

The slow request is visible in the morgan line; the slow query behind it is not, and finding it by hand means turning on debug, reading every operation with its values, and timing them in your head. The plugin is a page of code whose mistakes are subtle: a hook that does not cover aggregate, a timer that leaks filter values into logs, a regular expression that misses findOneAndUpdate. In a Rush sandbox the agent writes the plugin and the test that pins all three, and runs them against a real MongoDB.

The record keeps the lint and test runs, and the pull request shows the exact line format a reviewer will later search for. Across every Mongoose service a team maintains, the job runs in parallel, one sandbox and one pull request each.

Frequently asked questions

Why not just set mongoose.set('debug', true)?

Because it prints every operation with its full filter and documents to stdout, with no timing, and no way to keep only the slow ones. The plugin logs a single line per slow operation, through the app's logger, with keys instead of values.

Does it slow queries down?

Two hrtime reads and a comparison per operation, which is far below the cost of the round trip to MongoDB. Nothing is logged below the threshold, so a fast query costs only the timer.

Will it cover aggregations and the paginate plugin?

Yes. aggregate has its own middleware with a pipeline instead of a filter, and the plugin registers a second pair of hooks for it that logs the stage names. The paginate plugin runs countDocuments and find under the hood, so both show up as ordinary queries.

Can the threshold differ per environment?

Yes. SLOW_QUERY_MS is read from the environment through the same Joi schema as every other setting, so staging can log at a lower value than production. The test sets it to 0 to log everything.

More Mongoose jobs

The same job on other stacks

Keep reading