# Add Structured Output to a LangGraph Agent

Yes. An agent adds a final node that turns your graph's last answer into a typed schema on the state, proves it with fake models, and opens a PR.

[Run this on Rush](https://cloud.getrush.ai/run/add-structured-output/langgraph/). Opens the console with this prompt ready to send, in a Project that has your repository.

## What the agent does

The agent starts with how the graph ends, because structured output is a change to the last step and to what the caller reads. In an agent laid out like langchain-ai/react-agent, `src/react_agent/graph.py` builds a `StateGraph` over `State` with `InputState` as the input schema and `Context` as the context schema, cycles between `call_model` and a `ToolNode` over `TOOLS`, and `route_model_output` returns `__end__` when the last `AIMessage` has no tool calls. `state.py` holds only `messages` and the managed `is_last_step`, with a comment listing other fields one could add. `langgraph.json` serves the compiled `graph` as `agent`, so Studio and the API read whatever the state exposes. The unit tests under `tests/unit_tests/` cover the context only; the one integration test asserts a substring of the final message's content, which is exactly the fragility a schema removes.

It then adds a `respond` node after the loop. A `pydantic` model in a new `schemas.py` names the fields the caller wants, say an `answer`, a list of `sources` and a `confidence`; `State` gains an optional `structured_response` of that type; and `respond` calls the context's model with `with_structured_output` over the last few messages and writes the result into that field. `route_model_output` returns `respond` instead of `__end__` when there are no tool calls, and `respond` has an edge to `__end__`. The `is_last_step` branch in `call_model`, which answers with an apology when the step budget runs out, now also flows into `respond`, so a caller always gets the schema, with the apology in `answer`.

Two things come from reading rather than from the API reference. The model is loaded through `load_chat_model` from a `provider/model` string in `Context`, so `respond` reuses that loader rather than naming a provider, and the structured call is made on the unbound model, since tools are not wanted at that step. And `InputState` is deliberately narrower than `State`; the new field goes on `State` only, so the input contract to the graph does not change and Studio's input form stays the same.

In its sandbox the agent runs `uv sync`, `ruff check .`, `mypy --strict src/` and `pytest tests/unit_tests` as `.github/workflows/unit-tests.yml` does. It writes unit tests that need no API key: one invokes the compiled graph with the loader patched to a `GenericFakeChatModel` whose scripted answer has no tool calls, and a `respond` whose structured call is patched to return a known schema instance, asserting the field on the result; one asserts the `is_last_step` path still ends with a populated schema; one checks `mypy` accepts the new field's type by running on the test module too. The daily integration test is left as it is.

## Start it with one API call

`POST https://api.phoenix.so/o/{org}/sessions`

```json
{
  "agent": "claude",
  "input": "Add structured output to this LangGraph agent. Read src/react_agent/graph.py, state.py, context.py, utils.py, langgraph.json, tests/unit_tests/ and .github/workflows/unit-tests.yml first. Add a pydantic schema in src/react_agent/schemas.py with answer, sources and confidence, an optional structured_response field of that type on State but not InputState, and a respond node that loads the context's model through load_chat_model, calls with_structured_output on it over the recent messages, and writes the result to that field. Route call_model to respond instead of __end__ when there are no tool calls, including the is_last_step path, and respond to __end__. Do not change the tools or the prompts. Run uv sync, ruff check, mypy --strict src/ and pytest tests/unit_tests, add unit tests that patch the loader with GenericFakeChatModel and the structured call with a fixed instance to assert the field on the graph's result on both paths, and open a pull request quoting the output.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
```

1. **Get an API key.** In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
2. **Send the request.** POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
3. **Read what came back.** Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

## What comes back

What comes back is a pull request that adds `schemas.py` and the unit tests, and changes `state.py` (one field), `graph.py` (the `respond` node, the routing change and one edge) and the README's section on the graph's output. The description quotes the `ruff`, `mypy --strict` and `pytest` output, shows the schema as the caller will read it from `result["structured_response"]`, and says that `messages` is still there for anyone reading the free-text answer.

The run's record keeps every command and its output, so a reviewer can read the test that drove the graph end to end with fake models and the assertion on the populated schema, with no provider call made.

Before merging, a reviewer settles the schema's fields, since the agent chose three to prove the path, and decides whether `confidence` should be a bounded float with a validator or dropped, and whether the integration test should now assert on the schema instead of a substring of the text.

## Run it on every customer request

This is the change that lets a product call the agent and read the answer as data. Once the schema is in, your server starts one run per customer request with `POST /o/{org}/sessions`, the request in `input`, and reads `result` from the session or the `run.completed` webhook; the structured answer is what your code parses, with `messages` kept for the audit trail.

Send a `request_id` that is a UUID v5 of your own request id so a retried call returns the run it started rather than a second one, and a key at its `max_concurrent` limit queues the rest in order, so a burst needs no retry loop.

## Why run it on Rush?

Structured output in a hand-built graph is a routing change, a state change and a schema at once, and the wrong version runs: a `with_structured_output` call on the tool-bound model, a field put on `InputState` so every caller must send it, a step-budget path that still ends in free text. In a Rush sandbox the agent makes the change under the repository's `mypy --strict` and drives the compiled graph with fake models on both ending paths before anyone merges.

The test output and the type check stay with the run and the pull request quotes them, so the reviewer reads the assertion on `structured_response` rather than a description of the node. Across several agents the job runs in parallel, one sandbox each, each with its own proof.

## Frequently asked questions

### Does the free-text answer go away?

No. `messages` keeps the full conversation including the final `AIMessage`; `structured_response` is added beside it. A caller that reads text today keeps working, and one that wants data reads the new field.

### Why a separate respond node rather than binding the schema to call_model?

Because `call_model` is bound to the tools, and a model asked for both tool calls and a schema on every step does neither well. The loop stays as it is and the schema is produced once, at the end, from the model with no tools attached.

### What happens when the step budget runs out?

The existing `is_last_step` branch writes an apology message, and the pull request routes that path through `respond` too, so the caller still gets the schema with the apology in `answer` instead of a bare message.

### Does this need a particular model?

`with_structured_output` needs a model that supports tool calling or JSON mode, which the Anthropic and OpenAI models the context already names both do. The node loads whatever `Context.model` says, so switching models is still a config change.

## More LangGraph jobs

- [Handle Tool Errors in a LangGraph Agent](https://getrush.ai/run/add-tool-error-handling/langgraph/)
- [Trim Message History in a LangGraph Agent](https://getrush.ai/run/trim-message-history/langgraph/)
- [Upgrade Python Dependencies Weekly for a LangGraph Agent](https://getrush.ai/run/upgrade-python-dependencies/langgraph/)

## More jobs like this: Run an agent per customer task

- [Answer Support Tickets from a Django Codebase](https://getrush.ai/run/answer-support-tickets-from-code/django/)
- [Answer Support Tickets from a Go Codebase](https://getrush.ai/run/answer-support-tickets-from-code/go/)
- [Answer Support Tickets from a PocketBase Codebase](https://getrush.ai/run/answer-support-tickets-from-code/pocketbase/)

## Keep reading

- [How a Rush run works on your repository, from request to pull request](https://getrush.ai/cloud/)
- [Compare cloud agents, agent sandboxes and AI code review](https://getrush.ai/compare/)
- [Rush pricing: Free, Pro and Max](https://getrush.ai/pricing/)
