# Add a Sitemap to a Jekyll Site

Yes. An agent adds jekyll-sitemap to your Gemfile and plugin list, points robots.txt at the generated sitemap.xml, builds the site to check it and opens a PR.

[Run this on Rush](https://cloud.getrush.ai/run/add-sitemap/jekyll/). Opens the console with this prompt ready to send, in a Project that has your repository.

## What the agent does

The agent reads the site's configuration first, because a sitemap plugin only does the right thing when `url` and `baseurl` are right and the pages it should skip are marked. In a site built on Poole, as in poole/poole, `_config.yml` sets `url: https://getpoole.com`, `baseurl: ""`, `permalink: pretty` and `paginate: 1`, lists `jekyll-gist`, `jekyll-paginate` and `jekyll-seo-tag` under `plugins`, and excludes the `Gemfile`, `Gemfile.lock`, `README.md` and `LICENSE.md` from the output. The `Gemfile` names the same three plugins and `jekyll` with no versions; `Gemfile.lock` resolves them to Jekyll 4.0.0 and jekyll-seo-tag 2.6.1. `_includes/head.html` already calls `{% seo title=false %}` and links `atom.xml`, which is a Liquid template with `layout: null`. The pages are `index.html`, `about.md`, `archive.md` and a `404.html` whose front matter sets `permalink: 404.html`; the four posts under `_posts/` carry dates in their filenames and no `last_modified_at`. There is no `robots.txt` and no sitemap anywhere.

The plugin is jekyll-sitemap, which the agent adds in three places: `gem "jekyll-sitemap"` in the `Gemfile`, `jekyll-sitemap` in the `plugins` list of `_config.yml`, and the lockfile, which it refreshes with `bundle install` so the pull request carries the resolved version, 1.4.0, which depends on `jekyll >= 3.7, < 5.0` and so fits the Jekyll 4 this site pins. The plugin's own template, `lib/sitemap.xml` in jekyll/jekyll-sitemap, decides what goes in: every document of every collection with output, every HTML page, and static files, each as an absolute URL built from `url` and `baseurl`, skipping anything whose front matter says `sitemap: false` and the page whose URL is `/404.html`. That last rule means Poole's 404 page is left out without a change, and `atom.xml` and `styles.scss` are left out because neither renders to HTML. Posts get a `lastmod` from their date; pages get one only when they set `last_modified_at`, so the agent adds nothing invented there.

`robots.txt` is the second file. The agent adds one at the root with a `layout: null` front matter block, `User-agent: *`, `Allow: /` and a `Sitemap:` line written as `{{ site.url }}{{ site.baseurl }}/sitemap.xml`, so the address follows the configuration rather than being typed twice. Because `jekyll-sitemap` is one of the plugins the GitHub Pages build allows, with version 1.4.0 listed in `pages.github.com/versions.json` under the `github-pages` gem, the same two files work whether the site is built by Pages itself or by a workflow that runs the repository's own Jekyll 4.

Then the agent builds. It installs the gems with `bundle install`, runs `JEKYLL_ENV=production bundle exec jekyll build` and reads `_site/sitemap.xml`: it checks that the four posts, the home page, `about/` and `archive/` are present as `https://getpoole.com/...` URLs, that `404.html` is not, and that each post's `lastmod` matches the date in its filename. It fetches `_site/robots.txt` and confirms the `Sitemap:` line resolves to the same host. Because Poole's lockfile dates from Jekyll 4.0.0 with an `ffi` and `sassc` of the same age, the agent notes in the pull request whether `bundle install` needed a `bundle update` to install on the Ruby in its sandbox, and includes that lockfile change only if it did.

## Start it with one API call

`POST https://api.phoenix.so/o/{org}/sessions`

```json
{
  "agent": "claude",
  "input": "Add an XML sitemap to this Jekyll site with the jekyll-sitemap plugin. Read _config.yml, the Gemfile and Gemfile.lock, _includes/head.html, the pages at the root and the _posts directory first. Add gem \"jekyll-sitemap\" to the Gemfile and jekyll-sitemap to the plugins list in _config.yml, run bundle install so the lockfile records the version, and add a robots.txt at the root with layout: null, User-agent: *, Allow: / and a Sitemap line built from site.url and site.baseurl. Do not add sitemap: false to pages the plugin already skips, such as the 404 page, and do not invent last_modified_at dates. Build the site with JEKYLL_ENV=production bundle exec jekyll build, check that _site/sitemap.xml lists every post and page with absolute URLs and leaves out the 404 page, and that _site/robots.txt points at it. If bundle install needs a bundle update to work on the current Ruby, say so and include the lockfile change. Open a pull request with the commands and their output.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
```

1. **Get an API key.** In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
2. **Send the request.** POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
3. **Read what came back.** Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

## What comes back

The pull request changes `Gemfile`, `Gemfile.lock` and `_config.yml`, adds `robots.txt`, and touches no layout, include or post. Its description quotes the `jekyll build` output and the generated `sitemap.xml`, with the URL list the reviewer can compare against the site, and names the plugin version the lockfile now carries. If the old lockfile would not install, the description says which gems moved and why, separately from the sitemap change, so the reviewer can judge the two on their own.

A reviewer checks that `url` in `_config.yml` is the address the site is really served from, because every `loc` in the sitemap is built from it, and that no page they consider private is listed; a page they want out gets `sitemap: false` in its front matter, which the pull request shows how to do. After merge, the next build publishes `/sitemap.xml` and `/robots.txt`, and the sitemap can be submitted in Google Search Console or Bing Webmaster Tools, which the pull request leaves to the reader because it needs their account.

## Run it on every Jekyll site with no sitemap

A team that runs several Jekyll sites sends this body once per repository from its own server, each `POST /o/{org}/sessions` naming one repository in `resources` and carrying a `request_id` that is a UUID v5 of the repository's full name, so a retried call returns the run already started. Each run reads its own configuration: a site with `baseurl` set gets a `Sitemap:` line that includes it, a site that already has a `robots.txt` gets one appended line, and a site built by the GitHub Pages pipeline rather than its own Jekyll gets a note that the plugin is already in the `github-pages` gem and needs only the `plugins` entry.

The key's `webhook_url` reports each run the key starts with the pull request URL in `result`, so a script that sent five requests collects five pull requests without polling.

## Why run it on Rush?

A sitemap is small, but whether it is right depends on the whole site: the host in `url`, the permalink style, which pages render to HTML and which do not, and whether the plugin's own exclusions already cover the 404 page. The agent reads those files, runs the real `jekyll build` in its own sandbox, reads the XML it produced and keeps the output in the run's record, so the pull request shows the list of URLs that will be submitted rather than a promise that a plugin was added.

It also meets the repository where it is. A lockfile from Jekyll 4.0.0 may not install on a current Ruby, and the agent finds that out by running `bundle install` rather than by assuming, then reports the lockfile change as its own decision for the reviewer. Several sites, each with its own `baseurl` and pages, get several pull requests, each checked against its own build.

## Frequently asked questions

### Does this work on GitHub Pages' own build?

Yes. `jekyll-sitemap` 1.4.0 is in the `github-pages` gem that the Pages pipeline uses, so a site built that way needs only the `plugins` entry in `_config.yml`. A site that pins Jekyll 4 in its `Gemfile`, as this one does, is built by its own workflow instead, and there the `Gemfile` line is what installs the plugin.

### How do I keep a page out of the sitemap?

Add `sitemap: false` to its front matter. The plugin skips those pages, and it already skips the page at `/404.html`, which is why the pull request does not touch the 404 page.

### Will the dates be right?

Posts get `lastmod` from their date, which Jekyll takes from the filename unless the front matter sets `date`. Pages have no date, so they get no `lastmod` unless they set `last_modified_at`; the agent adds none, because an invented date is worse than no date to a crawler.

### Why not write sitemap.xml as a Liquid template like atom.xml?

The plugin's template is the same idea, maintained by the Jekyll team, and it handles collections, static files and the `sitemap: false` exclusion. A hand-written template is a fine alternative for a site that cannot add gems, and the pull request links the plugin's `lib/sitemap.xml` as the starting point for that.

### Does the sitemap include the paginated index pages?

Pages that `jekyll-paginate` generates, such as `/page2/`, are HTML pages and so appear in the sitemap. For a blog that is harmless, since they are real pages a reader can open; a reader who wants them out can exclude them with `sitemap: false` once their site's pagination plugin lets generated pages carry front matter of their own, and the pull request notes this as a separate question.

## More Jekyll jobs

- [Add JSON-LD Structured Data to a Jekyll Site](https://getrush.ai/run/add-structured-data/jekyll/)
- [Add a Table of Contents to Jekyll Posts](https://getrush.ai/run/add-table-of-contents/jekyll/)
- [Add Tag Pages to a Jekyll Blog](https://getrush.ai/run/add-tag-pages/jekyll/)

## The same job on other stacks

- [Add a Sitemap to an mdBook Site](https://getrush.ai/run/add-sitemap/mdbook/)
- [Add a Sitemap to a Nuxt App](https://getrush.ai/run/add-sitemap/nuxt/)
- [Add a Sitemap to a Pelican Site](https://getrush.ai/run/add-sitemap/pelican/)

## Keep reading

- [How a Rush run works on your repository, from request to pull request](https://getrush.ai/cloud/)
- [Compare cloud agents, agent sandboxes and AI code review](https://getrush.ai/compare/)
- [Rush pricing: Free, Pro and Max](https://getrush.ai/pricing/)
