# Add Cursor Pagination to a Mongoose and Express API

Yes. An agent adds keyset pagination to your Mongoose paginate plugin with a nextCursor, keeps page and limit working for existing clients, tests it, and opens a PR.

[Run this on Rush](https://cloud.getrush.ai/run/add-cursor-pagination/mongoose/). Opens the console with this prompt ready to send, in a Project that has your repository.

## What the agent does

The agent starts from how lists are paginated today, because in this kind of app pagination is a schema plugin, not a line in a controller. In a project like hagopj13/node-express-boilerplate, `package.json` is on `mongoose` ^5.7.7, `express` ^4.17.1 and `joi` ^17, tests run with `jest` ^26 and `supertest`, and `yarn.lock` is the lockfile. `src/models/plugins/paginate.plugin.js` adds a static `paginate(filter, options)` to any schema: it parses `sortBy` as a comma list of `field:asc|desc` and defaults to `createdAt`, reads `limit` (default 10) and `page` (default 1), computes `skip = (page - 1) * limit`, runs `countDocuments(filter)` and `find(filter).sort().skip().limit()` in parallel, applies `populate`, and resolves `{ results, page, limit, totalPages, totalResults }`. `user.model.js` applies it with `timestamps: true`, `user.controller.js` picks `name` and `role` as the filter and `sortBy`, `limit` and `page` as options, `user.validation.js` allows those query keys through Joi, and the swagger comments in `user.route.js` document `limit` and `page`. `tests/unit/models/plugins/paginate.plugin.test.js` covers only `populate`, and `tests/integration/user.test.js` expects `GET /v1/users` to answer `page: 1, limit: 10, totalPages: 1, totalResults: 3` for three seeded users.

Skip-based paging has two costs that grow with the collection: `skip` reads and discards every earlier document, and a document inserted while a client walks the pages shifts every later page by one, so rows repeat or vanish. The agent adds a cursor path to the same plugin rather than a second one. A cursor is the base64url encoding of a JSON pair, the sort field's value and the `_id` of the last document returned. When `options.cursor` is present, the query becomes `{ $and: [filter, { $or: [{ [field]: { $gt: value } }, { [field]: value, _id: { $gt: id } }] }] }` with `$lt` for a descending sort, the sort gets `_id` appended as a tiebreaker, and `skip` is not used. It fetches `limit + 1` documents to learn whether another page exists, drops the extra one, and returns `nextCursor` built from the last document, or `null` at the end. A cursor is accepted only with a single sort field, because a tuple comparison over two user-chosen fields with mixed directions is not a `$or` of two clauses; a `sortBy` with two fields plus a cursor answers 400 through the project's `ApiError`.

Without `cursor`, the plugin behaves exactly as before, so the existing integration test and every current client keep working, and every response now also carries `nextCursor` for the page it returned, so a client can start with the ordinary call and then follow the cursor. In `user.validation.js` the agent adds `cursor: Joi.string()` to `getUsers.query`, in `user.controller.js` it picks `cursor` into the options, and in `user.route.js` it documents the parameter and the new field in the swagger block that `src/docs/swaggerDef.js` serves. Because the tiebreaker makes `{ createdAt: 1, _id: 1 }` the access path, it adds that compound index to `userSchema` and notes in the pull request that the index builds on deploy.

The tests follow the project's layout. In `paginate.plugin.test.js` the agent reuses the `Project` and `Task` models the file already defines: it inserts 25 tasks, walks pages of 10 by `nextCursor` and asserts 25 distinct ids and a `null` cursor at the end; inserts a task between page one and page two and asserts no repeat; walks in `name:desc` order; and asserts that a malformed cursor and a two-field `sortBy` with a cursor are rejected. In `tests/integration/user.test.js` a new case requests `/v1/users?limit=2`, follows `nextCursor` as the admin from `tests/fixtures/user.fixture.js`, and asserts the third user arrives on the second page. The suite needs MongoDB: `tests/utils/setupTestDB.js` connects to `MONGODB_URL` with `-test` appended, and `.travis.yml` used a `mongodb` service. The sandbox has no Docker, so the agent installs it (`sudo apt-get install -y docker.io`, then starts `dockerd`) and runs the `mongo:4.2.1-bionic` image that `docker-compose.yml` names on port 27017, sets the variables from `.env.example`, and runs `yarn lint` and `yarn test`.

## Start it with one API call

`POST https://api.phoenix.so/o/{org}/sessions`

```json
{
  "agent": "claude",
  "input": "Add cursor-based pagination to this Express and Mongoose API without breaking page-based pagination. Read package.json, src/models/plugins/paginate.plugin.js, src/models/user.model.js, src/services/user.service.js, src/controllers/user.controller.js, src/validations/user.validation.js, src/routes/v1/user.route.js, tests/unit/models/plugins/paginate.plugin.test.js, tests/integration/user.test.js and tests/utils/setupTestDB.js first. Extend the paginate plugin so that when options.cursor is present it uses a keyset query on the single sort field plus _id as a tiebreaker, fetches limit plus one to detect the next page, and returns nextCursor as a base64url-encoded value, null at the end; without a cursor keep the existing behaviour and add nextCursor to the response. Reject a cursor combined with more than one sort field with a 400. Add the cursor query parameter to the Joi schema, the controller and the swagger comments, and add a compound index on createdAt and _id to the user schema. Write unit tests that walk pages by cursor with and without a concurrent insert and in descending order, and an integration test on GET /v1/users. Start MongoDB in the sandbox, run yarn lint and yarn test until they pass, and open a pull request.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
```

1. **Get an API key.** In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
2. **Send the request.** POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
3. **Read what came back.** Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

## What comes back

What comes back is a pull request that changes `src/models/plugins/paginate.plugin.js`, `src/models/user.model.js`, `src/controllers/user.controller.js`, `src/validations/user.validation.js` and `src/routes/v1/user.route.js`, and adds cases to `tests/unit/models/plugins/paginate.plugin.test.js` and `tests/integration/user.test.js`. The description shows one response with `nextCursor`, the decoded shape of a cursor, and the `yarn test` output with the new cases listed by name.

The run's record holds the MongoDB start, the lint run and the full Jest output, so a reviewer can check that the existing `GET /v1/users` assertions on `page`, `limit`, `totalPages` and `totalResults` still pass unchanged.

Before merging, a reviewer decides whether the cursor should also carry the sort field name so a client cannot pair a cursor from one ordering with another, whether `totalResults` should still be computed on cursor requests or dropped to save the count, and whether other models that use the plugin need the same compound index. The response shape for page-based calls is unchanged.

## Run it on every list endpoint still on skip and limit

A team with several Express services on Mongoose sends one `POST /o/{org}/sessions` per repository with this prompt and that repository in `resources`, with a `request_id` that is a UUID v5 of the repository's full name so a retried call returns the run already started. Each run reads its own plugin and validation layer, so a service that paginates in the controller rather than a plugin gets the cursor logic where its paging lives, and each run opens its own pull request.

Give the API key a `webhook_url` and Rush POSTs `run.completed` when each turn ends, with the `session_url` to read the pull request link and the test output from.

## Why run it on Rush?

Cursor pagination is a correctness change disguised as a feature: the hard part is not the encoding but the boundary condition, the tiebreaker on `_id`, the direction flip for descending sorts and the client that walks pages while rows are inserted. Those are exactly the cases a developer skips when the change is squeezed in between other work. The agent writes them as tests first and runs them against a real MongoDB, so the pull request shows the walk with a concurrent insert and no repeated id.

On Rush the database, the lint run and the test suite run in the run's own sandbox and their output stays with the run. Every service that shares the plugin can be handled as its own run and reviewed as its own pull request.

## Frequently asked questions

### Why not switch to mongoose-paginate-v2 or another package?

The project already owns a plugin with its own response shape, tests and swagger documentation, and every list route depends on that shape. Extending it keeps `page`, `limit`, `totalPages` and `totalResults` as they are and adds `nextCursor` beside them; replacing the plugin would change every response and every client for the same result.

### What is inside the cursor?

A base64url string of a JSON pair: the value of the sort field for the last document on the page and that document's `_id`. It is opaque to clients but not secret, so it carries no data a client did not already receive. A cursor that fails to decode, or that is sent with a two-field `sortBy`, is answered with a 400 through the project's `ApiError`.

### Do page and limit still work?

Yes. When no `cursor` is sent the plugin runs the same `skip` and `limit` query and `countDocuments` as before, and the existing integration tests assert that. The only addition on those responses is `nextCursor`, so a client can switch to cursors from any page.

### Why does it need a new index?

The keyset query filters on the sort field and `_id` together and sorts on both, so a compound index `{ createdAt: 1, _id: 1 }` lets MongoDB answer it without scanning. The agent adds it to `userSchema`; other schemas that use the plugin and are listed by a different field need their own, and the pull request names them.

## More Mongoose jobs

- [Add Database Indexes to a Mongoose App](https://getrush.ai/run/add-database-indexes/mongoose/)
- [Add Database Migrations to a Mongoose App](https://getrush.ai/run/add-database-migrations/mongoose/)
- [Add Dependabot to an Express and Mongoose API](https://getrush.ai/run/add-dependabot/mongoose/)

## The same job on other stacks

- [Add Cursor Pagination to a Prisma API](https://getrush.ai/run/add-cursor-pagination/prisma/)
- [Add Cursor Pagination to a Quarkus Panache REST Endpoint](https://getrush.ai/run/add-cursor-pagination/quarkus/)
- [Add Cursor Pagination to a Rocket App](https://getrush.ai/run/add-cursor-pagination/rocket/)

## Keep reading

- [How a Rush run works on your repository, from request to pull request](https://getrush.ai/cloud/)
- [Compare cloud agents, agent sandboxes and AI code review](https://getrush.ai/compare/)
- [Rush pricing: Free, Pro and Max](https://getrush.ai/pricing/)
