# Add llms.txt to a Jekyll Site

Yes. An agent adds a Liquid-templated llms.txt to your Jekyll site listing every page and collection document with its description, builds and tests it, opens a PR.

[Run this on Rush](https://cloud.getrush.ai/run/add-llms-txt/jekyll/). Opens the console with this prompt ready to send, in a Project that has your repository.

## What the agent does

The agent reads how the site is put together before writing a line of Liquid. In a site like github/choosealicense.com, `_config.yml` sets `title`, `description` and `url: https://choosealicense.com`, declares one collection, `licenses`, with `output: true` and `permalink: /licenses/:path/`, and gives every document in it `hidden: true` and `layout: license` through `defaults`. `_licenses/` holds 47 `.txt` documents whose front matter carries `title`, `spdx-id`, `description`, `how`, `using`, `permissions`, `conditions` and `limitations`, with `nickname`, `featured` and `redirect_from` on some; `mit.txt` and `gpl-3.0.txt` are `featured: true`. The pages are `index.html` at `/`, `licenses.html`, which shows eight overviews through `_includes/license-overview.html`, and `community.md`, `non-software.md`, `no-permission.md`, `about.md`, `appendix.md` and `terms-of-service.md`. `robots.txt` is the model to follow: empty front matter, then Liquid writing `{{ site.url }}/sitemap.xml`. The `Gemfile` pins `jekyll ~> 4.3` with `jekyll-sitemap`, `jekyll-redirect-from`, `jekyll-seo-tag`, `jekyll-github-metadata` and `jekyll-polyglot`, and `_config.yml` lists seven `languages` with `exclude_from_localization` for assets, `favicon.ico`, `robots.txt` and `CNAME`.

The agent writes `llms.txt` at the repository root with front matter of `permalink: /llms.txt` and `sitemap: false` and no layout, so Jekyll renders the Liquid and writes the text as it is. The body opens with `# {{ site.title }}`, a blockquote of `{{ site.description }}`, and one paragraph saying what the site is for. A `## Choosing a license` section links the guide pages with `absolute_url`, so every link is `https://choosealicense.com/...` whatever `--baseurl` the deploy passes. A `## Licenses` section loops over `site.licenses` sorted by `title` and writes one line per document, `- [MIT License](https://choosealicense.com/licenses/mit/): ` followed by the `description`, with the `nickname` in parentheses when there is one and a second line naming its permissions, conditions and limitations by the labels in `_data/rules.yml` rather than by tag. The two `featured` licenses come first under their own heading, the rest follow, and `## Optional` holds `appendix` and `terms-of-service`.

Polyglot is the part that needs care. It renders every page once per language and writes the copies under `/fr/`, `/es/` and the rest unless the path is in `exclude_from_localization`, so without a change the build would produce seven `llms.txt` files and the root one would be whichever language ran last. The agent adds `llms.txt` to that list, which leaves one English file at the root, and notes in the pull request how a per-language file would work: the layout already looks translations up in `site.data.i18n[site.active_lang].licenses`, and the same lookup in the template would give `/fr/llms.txt` French descriptions. `.txt` is served as `text/plain` by GitHub Pages, and `sitemap: false` keeps `jekyll-sitemap` from listing it.

To check it, the agent installs the Ruby that `.ruby-version` names, runs `bundle install` and `bundle exec jekyll build`, and reads `_site/llms.txt`: 47 license lines, one per file in `_licenses/`, the MIT entry with its description word for word, no `_site/fr/llms.txt`. It then runs `script/cibuild`, which is `bundle exec rake test` (a build, the RSpec suite under `spec/`, and html-proofer over `_site` with the Links, Images and Scripts checks) and `rubocop`. `spec/i18n_spec.rb` scans the HTML and Markdown sources for `t.html` keys, so a text file using none passes it, and html-proofer reads HTML only. The agent adds `spec/llms_txt_spec.rb`, which reads `_site/llms.txt` after the build the `test` task performs and asserts that every license's `url` appears in it, so a license added later without a description or with a broken permalink fails CI rather than silently dropping out of the file.

## Start it with one API call

`POST https://api.phoenix.so/o/{org}/sessions`

```json
{
  "agent": "claude",
  "input": "Add llms.txt to this Jekyll site. Read _config.yml, the Gemfile, robots.txt, every page at the root, the collection documents' front matter, _data/rules.yml, the Rakefile, script/cibuild and the workflows under .github/workflows first. Write llms.txt at the root with front matter and Liquid, modelled on robots.txt: the site title and description, a section linking the guide pages, and one line per collection document with its title, absolute URL and description, grouped so featured documents come first. Use absolute URLs. If the site uses jekyll-polyglot, add the file to exclude_from_localization so one copy is written at the root. Keep it out of the sitemap. Build the site, show the generated file, add a spec that checks every collection document appears in it, run the full test script, and open a pull request with the output.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
```

1. **Get an API key.** In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
2. **Send the request.** POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
3. **Read what came back.** Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

## What comes back

The pull request adds `llms.txt` and `spec/llms_txt_spec.rb` and changes one list in `_config.yml`. Its description pastes the generated `_site/llms.txt` in full, states how many license lines it holds against the count of files in `_licenses/`, and shows the `rake test` and `rubocop` output.

The `Build and Test` workflow runs on the pull request with `script/cibuild`, so a reviewer sees the new spec pass beside the existing license and i18n specs and the html-proofer run. The run's record keeps the build log, the `ls _site/*/llms.txt` check showing no localized copies, and the diff of `_site` against a build without the change, which is one new file.

Before merging, a reviewer edits the opening paragraph, decides whether the guide pages belong in the main section or under Optional, and whether to ask for translated copies under each language prefix. The pages, the license documents, the layouts and the sitemap are unchanged.

## Run it on every Jekyll site you maintain

A team with several Jekyll sites sends this body to `POST /o/{org}/sessions` once per repository, each with its own repository in `resources` and a `request_id` that is a UUID v5 of the repository's full name, so a retried call returns the run already started instead of opening a second pull request. Each run reads its own `_config.yml`: a site with several collections gets a section per collection, a blog gets its `site.posts` with dates, a site without polyglot gets no `exclude_from_localization` change, and a site whose documents have no `description` gets the first paragraph of each instead.

Once the file is merged no further run is needed when content changes: Jekyll regenerates `llms.txt` on every build, and the spec fails the build if a document is missing from it. The key's `webhook_url` reports each run the key starts with the pull request in `result`, and a `run.failed` names a site whose build broke, for instance one whose Ruby version the sandbox could not install.

## Why run it on Rush?

On a Jekyll site, `llms.txt` is not a plugin but a template, and the questions are all local: which collections exist, where descriptions live, what the permalink of each document is, how an i18n plugin multiplies pages, and which CI checks would read the new file. The agent answers each from the repository, builds the site in its own sandbox, and shows the generated file and the full test script passing.

The build log, the generated file and the test output stay with the run and are quoted in the pull request. Several sites can get the same change at once, each in its own sandbox with its own evidence, rather than one person copying a template between repositories and hoping each site's front matter matches.

## Frequently asked questions

### Why not a Jekyll plugin?

Because a text file with front matter is all Jekyll needs: it renders the Liquid and writes the result, as this site already does for `robots.txt`. A plugin would be one more gem to pin, and on a site built by the native GitHub Pages build, an unlisted plugin is silently ignored. The template approach works on either build.

### What happens with jekyll-polyglot?

Polyglot renders every page per language and writes copies under each prefix unless the path is in `exclude_from_localization`. The agent adds `llms.txt` there so one file is written at the root. If you want `/fr/llms.txt` with French descriptions, the same template can read `site.data.i18n[site.active_lang]` the way `_layouts/license.html` does; the pull request shows where.

### Does the file stay current?

Yes. It is generated on every build from `site.licenses` and the pages, so a new license document appears the next time the site builds. The added spec reads the built file and checks that every document's URL is in it, so a document that fails to render fails `rake test` on the pull request.

### Will html-proofer or the sitemap pick it up?

html-proofer checks HTML files under `_site`, so it never opens the text file, and the absolute URLs in it are the same permalinks the HTML pages already link to. `sitemap: false` in the front matter tells `jekyll-sitemap` to leave it out, as a machine-readable index does not belong in the sitemap.

## More Jekyll jobs

- [Add Mermaid Diagrams to a Jekyll Blog on GitHub Pages](https://getrush.ai/run/add-mermaid-diagrams/jekyll/)
- [Add Open Graph and Twitter Card Tags to a Jekyll Blog on GitHub Pages](https://getrush.ai/run/add-open-graph-tags/jekyll/)
- [Add Pagination to a Jekyll Blog](https://getrush.ai/run/add-pagination/jekyll/)

## The same job on other stacks

- [Add llms.txt to an mdBook Site](https://getrush.ai/run/add-llms-txt/mdbook/)
- [Add llms.txt to a MkDocs Site](https://getrush.ai/run/add-llms-txt/mkdocs/)
- [Add llms.txt to a Pelican Site](https://getrush.ai/run/add-llms-txt/pelican/)

## Keep reading

- [How a Rush run works on your repository, from request to pull request](https://getrush.ai/cloud/)
- [Compare cloud agents, agent sandboxes and AI code review](https://getrush.ai/compare/)
- [Rush pricing: Free, Pro and Max](https://getrush.ai/pricing/)
