# Trim Message History in a LangGraph Agent

Yes. An agent trims what call_model sends to a token budget, keeps tool calls with their results, caps one search result's size, tests it and opens a PR.

[Run this on Rush](https://cloud.getrush.ai/run/trim-message-history/langgraph/). Opens the console with this prompt ready to send, in a Project that has your repository.

## What the agent does

The agent measures the growth before it limits it. In a LangGraph project like langchain-ai/react-agent, `State` in `src/react_agent/state.py` keeps `messages` under the `add_messages` reducer, and `call_model` in `graph.py` sends the system prompt followed by `*state.messages`, all of them, on every step. One question with one search adds four messages: the question, the model's tool call, the tool's result and the answer. The result is the heavy one, since `search` in `tools.py` returns whatever Tavily sends for `max_search_results`, which `context.py` defaults to 10. On the LangGraph server threads are kept between requests, and the README says follow-ups are appended to the same thread, so nothing ever shrinks the list. With a stub in place of Tavily that returns ten results of about 4,800 characters each, `count_tokens_approximately` puts the first model request at 31 tokens and the eighth, four questions in, at about 48,000.

It trims what is sent, not what is stored. In `call_model` the agent passes `state.messages` through `trim_messages` from `langchain_core.messages` with `strategy="last"`, `token_counter=count_tokens_approximately`, a `max_tokens` budget and `start_on="human"`, and gives the model the result. The state is left whole, so the thread in Studio and any audit of the conversation still show every turn. `start_on` is the setting that keeps the request valid: without it the kept window can open on an assistant message or a tool result whose call was cut away, and providers refuse a tool result with no matching call. The approximate counter is deliberate. `load_chat_model` builds whatever `provider/model` string the `Context` names, Anthropic, OpenAI or Fireworks from the dependencies in `pyproject.toml`, so the agent uses a counter that needs no provider and leaves headroom in the budget.

Running it exposes two edges the agent handles in the same change. When the newest turn alone is over the budget, `trim_messages` returns an empty list, and the model would be asked to answer a system prompt with no question. Trimming cannot fix a single oversized message, so the agent bounds it at the source: `search` shortens each result's content to a fixed length before returning it, and a test proves one full turn fits the default budget. The second edge is in configuration. The budget becomes a field on `Context`, `max_context_tokens`, beside `max_search_results`, and `Context.__post_init__` fills fields from the environment with `os.environ.get`, which returns text. `MAX_SEARCH_RESULTS=5` already arrives as the string `'5'`, and a string budget makes `trim_messages` fail with a `TypeError` comparing `int` and `str`, so the agent converts integer fields there and says so in the pull request.

The tests sit in `tests/unit_tests/` and call no provider. A fake model records the messages it receives, and a stub search returns long results; both replace names fetched with `importlib.import_module`, since `import react_agent.graph as g` hands back the compiled graph that `__init__.py` re-exports. The cases: a long thread stays under the budget on every model call while `state.messages` keeps its full length; the first kept message is always a human one; a tool call is never sent without its result; one turn with ten maximum-length results still fits; and the budget read from the environment is an integer. Then `uv sync --extra dev` for `ruff` and `mypy`, which are in the `dev` extra, and the three commands from `.github/workflows/unit-tests.yml`: `ruff check .`, `mypy --strict src/` and `pytest tests/unit_tests`.

## Start it with one API call

`POST https://api.phoenix.so/o/{org}/sessions`

```json
{
  "agent": "claude",
  "input": "Stop this LangGraph agent's prompt from growing without limit. Read pyproject.toml, uv.lock, the README, every file under src/react_agent, tests/conftest.py, both test directories, the Makefile and .github/workflows first. Measure the growth: with a fake model that records what it is sent and a stub in place of TavilySearch that returns long results, run several questions on one thread and report the approximate token count of each model request. Then, in call_model, pass state.messages through trim_messages (strategy last, token_counter count_tokens_approximately, start_on human) with a budget from a new Context field max_context_tokens, and send the trimmed list to the model. Do not remove anything from state. Check what trim_messages returns when the newest turn alone exceeds the budget, and prevent that case by shortening each search result's content to a fixed length in src/react_agent/tools.py. Context.__post_init__ reads fields from the environment as strings: make integer fields integers, and say in the pull request that this also changes max_search_results. Add unit tests under tests/unit_tests, with no API key: every model request on a long thread is under the budget while state keeps every message, the kept window starts on a human message, no tool call is sent without its result, one turn with maximum-length results fits, and MAX_CONTEXT_TOKENS from the environment is an int. Run uv sync --extra dev, ruff check ., mypy --strict src/ and pytest tests/unit_tests, do not run tests/integration_tests, paste the output and the before and after measurements in the pull request and open it.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
```

1. **Get an API key.** In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
2. **Send the request.** POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
3. **Read what came back.** Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

## What comes back

A pull request across three files under `src/react_agent/` and one new test file. `graph.py` gains the trimming call inside `call_model`; `context.py` gains the `max_context_tokens` field and the integer conversion; `tools.py` shortens each result; `state.py` is untouched, because the stored history does not change. The description leads with two columns of numbers from the agent's measurement, the approximate size of each model request on a long thread before and after, so the effect is visible without running anything.

A reviewer checks the budget against the smallest context window among the models the team runs, remembering the count is an estimate. They ask what the agent loses when turn one falls out of the window: if users state a name or a constraint early and refer to it later, trimming alone will forget it, and the follow-up is a summary or a store. And they read the truncation in `search` for whether cutting a result mid-sentence is acceptable for their answers, or whether fewer results in full would serve better.

## Run it on every context length error

The symptom arrives as a provider error on a long thread, usually through error tracking. Sentry signs its own webhook requests and cannot add the `Authorization: Bearer <secret>` header a Rush `webhook` trigger requires, so the event goes to a handler on your server first. The handler checks Sentry's signature, keeps issues whose message mentions context length, and sends `POST /o/{org}/sessions` with this page's body, the issue's title and stack trace appended to `input`, the agent's repository in `resources`, and a `request_id` that is a UUID v5 of the Sentry issue id. The same issue firing again then returns the run that is already working on it.

## Why run it on Rush?

The interesting cases here only appear when the graph is run for many turns with large tool results: the window that opens on an orphaned tool message, the empty list when one turn is too big, the string that should have been a number. A developer checking the change by hand asks two questions and sees it work. The agent drives the graph through a long scripted thread in its sandbox, records the size of every request, and keeps the commands and their output in the session's transcript, where the numbers in the pull request can be checked.

No model is called to do it. The fake records what it would have been sent, which is the thing being limited, so the measurement costs no provider tokens and needs no key in the sandbox. That also makes it safe to start from an alert at any hour: the run cannot spend against your model account or touch a live thread, and what it leaves is a branch, a diff and proof that the diff does what it says.

## Frequently asked questions

### Does trimming delete messages from the thread?

No. `trim_messages` returns a shorter list for one model call and leaves `state.messages` alone, so the checkpoint keeps the whole conversation. Deleting from state is a different operation, done by returning `RemoveMessage` entries from a node, and this change does not use it.

### Why did trim_messages return an empty list?

With `strategy="last"` and `start_on="human"`, it keeps the newest messages that fit and then drops anything before the first human message in that window. When the newest turn is larger than `max_tokens`, nothing qualifies. Raise the budget or make the largest message smaller, which is why this change also shortens search results.

### How accurate is count_tokens_approximately?

It is an estimate from the text, with no tokenizer and no network call, so it is fast and works for any provider. It will differ from what a given model counts. Set the budget with room to spare, well under the smallest context window you run.

## More LangGraph jobs

- [Upgrade Python Dependencies Weekly for a LangGraph Agent](https://getrush.ai/run/upgrade-python-dependencies/langgraph/)
- [Write pytest Tests for a LangGraph Agent](https://getrush.ai/run/write-pytest-tests/langgraph/)
- [Add Conversation Memory to a LangGraph Agent](https://getrush.ai/run/add-conversation-memory/langgraph/)

## More jobs like this: Fix a bug from an alert

- [Add Connection Pooling to a GORM App](https://getrush.ai/run/add-connection-pooling/gorm/)
- [Add Connection Pooling Limits to a Prisma App on Serverless](https://getrush.ai/run/add-connection-pooling/prisma/)
- [Add Database Indexes to a SQLModel App With Alembic](https://getrush.ai/run/add-database-indexes/sqlmodel/)

## Keep reading

- [How a Rush run works on your repository, from request to pull request](https://getrush.ai/cloud/)
- [Compare cloud agents, agent sandboxes and AI code review](https://getrush.ai/compare/)
- [Rush pricing: Free, Pro and Max](https://getrush.ai/pricing/)
