# Add llms.txt to an mdBook Site

Yes. An agent generates llms.txt and llms-full.txt from your mdBook's SUMMARY.md, builds them in CI beside the HTML, and opens a PR with the output.

[Run this on Rush](https://cloud.getrush.ai/run/add-llms-txt/mdbook/). Opens the console with this prompt ready to send, in a Project that has your repository.

## What the agent does

The agent reads how the book is built before it adds anything to the output, because mdBook copies files from `src/` into `book/` as they are and a wrong path silently publishes nothing. In a book laid out like rust-lang/rust-by-example, `book.toml` names the title and a one-sentence description, points `[output.html]` at the GitHub repository with an `edit-url-template`, turns on `hash-files`, adds one stylesheet from `theme/css/`, and declares an optional `gettext` preprocessor that runs after `links` for the translations in `po/`. `src/SUMMARY.md` is the table of contents, nested three levels deep, over 198 Markdown files. `.github/workflows/rbe.yml` installs mdBook 0.5.1 from its release tarball, runs `mdbook test`, `mdbook build`, the `linkcheck.sh` script from the rust-lang/rust repository, then builds each of `ja zh es ko` with `MDBOOK_BOOK__LANGUAGE` into `book/<lang>` and uploads `book` as an artifact. There is no script directory and no package manager; `theme/head.hbs` is plain JavaScript that injects a language picker.

`llms.txt` is the short file: the book's title as a heading, its description from `book.toml` as a blockquote, then one link per chapter in `SUMMARY.md` order with the absolute URL under the site root, each with the chapter's first sentence after the dash. `llms-full.txt` is the whole book as one Markdown document, chapters in the same order, each under a heading at its depth in the summary. The agent writes `tools/llms.py`, which needs only the Python 3 that the runner already has, parses `SUMMARY.md` for the link list and the nesting, and for the full file strips mdBook's own directives (`{{#include}}` blocks and `{{#playground}}`) that appear in a few chapters while keeping the fenced `rust,editable` examples, since those are the point of this book. The site root is an argument so the same script serves a translation built under `book/<lang>`.

The workflow gains one step after `mdbook build`, `python3 tools/llms.py --site-url https://doc.rust-lang.org/rust-by-example/ book`, which writes `book/llms.txt` and `book/llms-full.txt` beside `index.html`, and the translation loop runs it once more per language into `book/<lang>` with that language's root. Because `src/` is untouched, `mdbook test` and the link checker behave exactly as before, and `hash-files` does not apply to the two files, so their URLs are stable. The files leave the workflow inside the `book` artifact; whatever publishes that directory publishes them.

To check it, the agent installs the same mdBook 0.5.1 in its sandbox, runs `mdbook build` and the script, opens `book/llms.txt` to confirm every chapter in `SUMMARY.md` is listed once in order with a URL that exists under `book/`, counts the headings in `book/llms-full.txt` against the summary, greps the full file for any leftover `{{#` directive, and runs `mdbook test` to show the book's examples still compile.

## Start it with one API call

`POST https://api.phoenix.so/o/{org}/sessions`

```json
{
  "agent": "claude",
  "input": "Add llms.txt and llms-full.txt to this mdBook site. Read book.toml, src/SUMMARY.md, theme/head.hbs, .github/workflows/rbe.yml, README.md and CONTRIBUTING.md first. Write tools/llms.py, dependency-free Python 3, that parses SUMMARY.md and the built book to produce llms.txt (title, description from book.toml, one absolute link per chapter in summary order with a one-line summary) and llms-full.txt (every chapter as Markdown in summary order under headings at its summary depth, with mdBook include and playground directives stripped and fenced code kept). Take the site URL and the book directory as arguments so translations under book/<lang> get their own files. Add a workflow step after mdbook build that writes both files into book/, and run it in the translation loop too. Install the same mdbook version the workflow uses, build, run the script, verify every chapter is listed once with a path that exists, grep the full file for leftover directives, run mdbook test, and open a pull request with the output.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
```

1. **Get an API key.** In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
2. **Send the request.** POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
3. **Read what came back.** Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

## What comes back

The pull request adds `tools/llms.py` and changes `.github/workflows/rbe.yml`. Its description shows the first lines of the generated `llms.txt`, the chapter count in the summary against the link count in the file and the heading count in `llms-full.txt`, the size of each file, the empty result of the directive grep, and the `mdbook test` output.

The `CI` workflow runs on the pull request with its existing steps unchanged and the new step after `mdbook build`, and the `rust-by-example` artifact it uploads now holds `llms.txt` and `llms-full.txt` at its root and under each language directory. No chapter, theme file or `book.toml` setting changes.

Before merging, a reviewer checks the site URL the step passes, decides whether the full file should include the chapters marked as tests or drafts if the book has any, and confirms with whoever publishes `book` that the two new files are served, since this repository's workflow uploads an artifact rather than deploying.

## Run it on every documentation site without llms.txt

A team that keeps several books sends this body once per repository to `POST /o/{org}/sessions`, each with its own repository in `resources` and a `request_id` that is a UUID v5 of the repository's full name, so a retried call returns the run already started instead of opening a second pull request. Each run reads its own `book.toml`: a book that deploys with `peaceiris/actions-gh-pages` gets the step before the deploy, one with a `[output.markdown]` renderer configured gets the full file built from that output instead of the sources, and one without translations skips the loop.

Once merged, every push rebuilds the two files with the book, so no further run is needed. A `webhook_url` on the key reports each `run.completed` with the pull request in `result`, and a `run.failed` names a book whose build did not pass in the sandbox, which is worth a look on its own.

## Why run it on Rush?

The format is simple, but a correct file depends on the book's own conventions: where `SUMMARY.md` nests, which chapters use include directives, what the public URL root is, and how translations are laid out. The agent reads the summary and the workflow, writes a script that fits them, and proves it on a real build with the same mdBook version CI uses, in its own sandbox.

The build output, the chapter counts and the `mdbook test` run stay with the run and are pasted into the pull request, so a reviewer checks numbers rather than reruns a build. Several books can get the same files at once, each from its own summary.

## Frequently asked questions

### Why a script instead of an mdBook plugin?

An alternative renderer in `book.toml` would do it, but configuring a second renderer moves the HTML output from `book/` to `book/html/`, which breaks the link checker path and the artifact in this workflow. A script that runs after the build leaves every existing path alone. The pull request notes the renderer option for a book that already has several.

### Does the full file include the runnable examples?

Yes, as fenced code blocks with their `rust,editable` info string kept. Only mdBook's own directives are stripped, because they are expanded by the HTML renderer and would appear as literal braces in a text file.

### What about the translations?

The workflow already builds each language into `book/<lang>` with `MDBOOK_BOOK__LANGUAGE`, and the new step runs once per language with that directory and root, so each translation gets its own `llms.txt` and `llms-full.txt` in its own language. A book without a `po/` directory skips the loop.

### Will the file go stale?

No. It is generated from `SUMMARY.md` on every build, so a new chapter appears in it as soon as it is in the summary. Nothing is committed to `src/`, so there is no second copy to forget.

## More mdBook jobs

- [Add Mermaid Diagrams to an mdBook](https://getrush.ai/run/add-mermaid-diagrams/mdbook/)
- [Add Open Graph Tags to an mdBook Site](https://getrush.ai/run/add-open-graph-tags/mdbook/)
- [Add PDF Export to an mdBook Book](https://getrush.ai/run/add-pdf-export/mdbook/)

## The same job on other stacks

- [Add llms.txt to a MkDocs Site](https://getrush.ai/run/add-llms-txt/mkdocs/)
- [Add llms.txt to a Pelican Site](https://getrush.ai/run/add-llms-txt/pelican/)
- [Add llms.txt to a Sphinx Docs Site](https://getrush.ai/run/add-llms-txt/sphinx/)

## Keep reading

- [How a Rush run works on your repository, from request to pull request](https://getrush.ai/cloud/)
- [Compare cloud agents, agent sandboxes and AI code review](https://getrush.ai/compare/)
- [Rush pricing: Free, Pro and Max](https://getrush.ai/pricing/)
