What the agent does
The agent reads how the site is built before it adds any route. In a site like the docs/ workspace of withastro/starlight, astro.config.mjs configures @astrojs/starlight with a title, a sidebar of five groups (Start Here with explicit slugs, then Guides, Components, Reference and Resources autogenerated from directories), seventeen locales with root as English, lastUpdated: true, an editLink.baseUrl, and a starlight-links-validator plugin that only loads when CHECK_LINKS is set. src/content.config.ts defines the docs collection with docsLoader() and docsSchema(). package.json builds with astro build on astro ^7.2.10, the Starlight package comes from the workspace at 0.42.6, the package manager is pnpm 11, and public/_headers sets long cache headers for /_astro/* only. Nothing serves /llms.txt: no page under src/pages, no endpoint, and no plugin.
The agent then picks the mechanism. The community plugin starlight-llms-txt generates llms.txt, llms-full.txt and llms-small.txt at build time from the content collection and the sidebar, and it is the smallest change with the least to maintain, so the agent adds it to devDependencies with pnpm add -D in docs/ and to the plugins array in astro.config.mjs, alongside the conditional links validator so both plugins keep working. It configures projectName from the Starlight title, a one-paragraph description, and exclude patterns for the sixteen non-root locales so the file carries each page once, in English, rather than seventeen times; the translated copies remain in the HTML site and are not needed for a model reading the docs.
Where the site has pages the sidebar does not list, the agent reads src/content/docs/ directly: index.mdx, getting-started.mdx, manual-setup.mdx, environmental-impact.mdx and 404.md at the top level, and the guides, components, reference and resources directories. It confirms that 404.md is excluded, that the Start Here entries appear first in the same order the sidebar uses, and that the reference/configuration.mdx page, the one a model needs most, is present in full in llms-full.txt. It then adds /llms.txt to the footer or the head as a <link rel="alternate" type="text/plain"> only if the team asks; by default it leaves the HTML untouched and documents the URL in the pull request.
In the sandbox it runs pnpm install --frozen-lockfile from the monorepo root, which the workspace's minimumReleaseAge setting allows for an already-published package, then pnpm --filter starlight-docs build, and inspects docs/dist/llms.txt and docs/dist/llms-full.txt: it counts the entries, checks that every link resolves to a file in dist/, and checks that no non-root locale slipped in. It also runs pnpm --filter starlight-docs linkcheck, which is the build with CHECK_LINKS=true, to show the links validator and the new plugin coexist. The pull request shows the head of both generated files.
Start it with one API call
POST https://api.phoenix.so/o/{org}/sessions with this body:
{
"agent": "claude",
"input": "Add llms.txt and llms-full.txt to this Starlight documentation site and open a pull request. Read astro.config.mjs, src/content.config.ts, package.json and the src/content/docs tree first. Use the starlight-llms-txt plugin: add it as a dev dependency with the repository's package manager, register it in the Starlight plugins array without breaking any existing conditional plugin, set projectName and description from the site config, and exclude every non-root locale so each page appears once. Do not change the HTML pages or the sidebar. Build the site in the sandbox, verify that dist/llms.txt lists every English page the sidebar lists and that every link in it resolves to a built file, and quote the head of both generated files and the build output in the pull request.",
"resources": [
{
"url": "https://github.com/your-org/your-app"
}
]
}- Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
- Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
- Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.
What comes back
What comes back is a pull request that changes docs/astro.config.mjs, docs/package.json and pnpm-lock.yaml. The config gains one import and one entry in the plugins array with projectName, description and exclude set; the existing starlight-links-validator condition is preserved. No content file, component or sidebar entry changes.
The description quotes the astro build output from the sandbox, the first lines of the generated llms.txt with the Start Here entries in sidebar order, the size of llms-full.txt, and the result of the link check over the generated file. It names the two URLs the site will serve after deploy, /llms.txt and /llms-full.txt, and the locales excluded.
Before merging, a reviewer reads the generated llms.txt on the pull request's preview deploy if the host provides one, checks the description wording, and decides whether llms-small.txt should also be published or excluded in the plugin options. If the team wants translated variants later, removing a locale from exclude is a one-line change.
Run it on every docs site without an llms.txt
A company with several documentation sites sends one POST /o/{org}/sessions per repository with this prompt and the repository in resources. GET /o/{org}/github lists the repositories the Rush GitHub App installation reaches, so a script can start a run for each Astro or Starlight site that has no llms.txt route. Each run reads that site's own astro.config and content tree, so a single-locale site gets no exclude list and a site with custom src/pages gets them inspected too.
Set a webhook URL on the API key and Rush POSTs run.completed or run.failed when the turn finishes, with session_url to read the pull request link. Send a fresh UUID as request_id with each request so a retried request returns the run already started.
The use case this job follows, with its automation and trigger: Hand off a long task.
Why run it on Rush?
The difficult part of an llms.txt is not the file format but making sure it matches the site: the same pages, in the sidebar's order, once each, with links that resolve. In its own sandbox the agent builds the real site with the real pnpm workspace, reads the generated files out of dist/, and verifies every link against the built output, with each command and its result kept in the run's record. The pull request shows the file a model will read, not a description of it.
Because the change is a plugin registration and a lockfile update, the diff is small enough to review in a minute, and the proof is in the quoted build output. The same request runs over every docs site a company maintains at once, one sandbox and one pull request each.
Frequently asked questions
Why a plugin instead of a hand-written llms.txt in public/?
Because a static file goes stale the first time a page is added or renamed. The plugin reads the docs content collection and the sidebar at build time, so every astro build regenerates llms.txt and llms-full.txt to match the site, and a reviewer never has to remember to update them.
Will the translated pages be included?
Not by default. The agent excludes every non-root locale so each page appears once, in the site's default language, which keeps llms-full.txt to one copy of the docs. If you want a translated variant, remove that locale from the plugin's exclude option.
Does this change the HTML site or the sidebar?
No. The plugin only adds routes to the build output. Pages, components, the sidebar and the _headers file are untouched, and the existing starlight-links-validator keeps running under CHECK_LINKS exactly as before.
Can the agent verify the files without deploying?
Yes. It builds the site in its sandbox and reads dist/llms.txt and dist/llms-full.txt directly, checking entry counts, locale exclusions and that each linked path exists under dist/. What it cannot check is your host's caching or redirects, which is why the pull request names the final URLs for a post-deploy look.