# Add a Sitemap to a Pelican Site

Yes. An agent installs the pelican-sitemap plugin, configures it in pelicanconf.py, points robots.txt at the result, builds the site to check it and opens a PR.

[Run this on Rush](https://cloud.getrush.ai/run/add-sitemap/pelican/). Opens the console with this prompt ready to send, in a Project that has your repository.

## What the agent does

The agent reads the settings before it touches them, because a Pelican plugin is enabled by what `pelicanconf.py` leaves out as much as by what it says. In a site laid out like getpelican/pelican-blog, `pelicanconf.py` sets `SITENAME`, `PATH = "content"`, `THEME = "themes/notmyidea"`, `TIMEZONE = "Europe/Rome"`, `DEFAULT_PAGINATION = 5`, `STATIC_PATHS = ["images", "extra"]` and an `EXTRA_PATH_METADATA` table that moves `extra/robots.txt` and the favicon files to the site root; it turns content caching off and configures Markdown extensions, and it has no `PLUGINS` setting at all. `publishconf.py` imports it, sets `SITEURL = "https://blog.getpelican.com"` and `DELETE_OUTPUT_DIRECTORY = True`. The `Makefile` builds with `pelican content -o output -s pelicanconf.py` for `make html` and with `publishconf.py` for `make publish`, and `make upload` rsyncs `output/` to the project's server. `content/news/` holds twenty-one posts, all reStructuredText but one Markdown file, and `content/pages/` holds `index.rst`, `gratitude.rst` and `not-found.rst`. `content/extra/robots.txt` disallows `/drafts/` and `/feeds/` and names no sitemap. There is no requirements file and no workflow, and `.gitignore` keeps `output/` out of the repository.

The plugin is `pelican-sitemap`, published by the Pelican project on PyPI, 1.2.2 at the time of reading, which requires `pelican>=4.5`. Pelican's plugin documentation explains the part that matters here: since 4.5 a plugin installed under the `pelican.plugins` namespace is discovered and registered automatically as long as `PLUGINS` is left unset, and the `pelican-plugins` command lists the ones that are active. Because this site never set `PLUGINS`, installing the package is enough to enable it, and the agent says so in the pull request rather than adding a `PLUGINS` list that would turn auto-discovery off for every plugin added later. The plugin writes `sitemap.xml` into the output directory and reads one setting, a `SITEMAP` dictionary with `format`, `priorities` and `changefreqs` for articles, pages and indexes, and an `exclude` list of regular expressions; a page can also leave itself out with `private: True` in its metadata, and an article can set its own `ChangeFreq` and `Priority`.

The agent adds that dictionary to `pelicanconf.py` under the existing settings: `format` set to `xml`, articles at a higher priority than index pages, `changefreqs` of `monthly` for articles and pages and `daily` for indexes, and `exclude` entries for `/drafts/` and `/feeds/` so the sitemap and `robots.txt` agree about what should not be crawled. It reads `content/pages/not-found.rst`, which exists to be served as the site's 404 page, and marks it `private` so it is not offered to search engines as a page. One line goes at the end of `content/extra/robots.txt`, `Sitemap: https://blog.getpelican.com/sitemap.xml`, with the host taken from `SITEURL` in `publishconf.py`, because `EXTRA_PATH_METADATA` already publishes that file at the root. A `requirements.txt` with `pelican`, `markdown` for the one Markdown post, and `pelican-sitemap` is added so the plugin is installed wherever the site is built, since nothing in the repository recorded its dependencies before.

Then the agent builds. It installs those packages in its sandbox, where Pelican 4.12.0 needs Python 3.11 or newer, runs `pelican-plugins` to confirm `sitemap` is listed, and runs `make publish`, which uses `publishconf.py` so the URLs in the sitemap carry the real host instead of the empty `SITEURL` of the development settings. It opens `output/sitemap.xml` and checks that every post under `content/news/` appears with a `lastmod`, that the gratitude page appears and the not-found page does not, and that nothing under `/feeds/` is listed. The README mentions only `make upload`, so the pull request explains that the plugin must be installed on the machine that runs it, which is what the new requirements file is for.

## Start it with one API call

`POST https://api.phoenix.so/o/{org}/sessions`

```json
{
  "agent": "claude",
  "input": "Add an XML sitemap to this Pelican site with the pelican-sitemap plugin. Read pelicanconf.py, publishconf.py, the Makefile, the README, content/extra/robots.txt and the content directory first. Do not add a PLUGINS setting if there is none; rely on namespace plugin discovery and say so. Add a SITEMAP dictionary to pelicanconf.py with format xml, priorities and changefreqs for articles, pages and indexes, and exclude patterns matching what robots.txt disallows. Mark the 404 page private. Append a Sitemap line to robots.txt using the SITEURL from publishconf.py. Add a requirements.txt with pelican, markdown and pelican-sitemap. Install them, run pelican-plugins to confirm the plugin is active, build with the publish settings and check output/sitemap.xml lists every article and page except the private one. Open a pull request with the build output and a sample of the sitemap.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
```

1. **Get an API key.** In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
2. **Send the request.** POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
3. **Read what came back.** Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

## What comes back

The pull request changes `pelicanconf.py`, `content/extra/robots.txt` and `content/pages/not-found.rst`, and adds `requirements.txt`. Its description quotes the `pelican-plugins` output showing `sitemap` as active, the `make publish` summary, and the first entries of `output/sitemap.xml` with their `loc`, `lastmod`, `changefreq` and `priority` values, so a reviewer can see the host and the dates without building. The theme is untouched: the sitemap is a file the plugin writes, not a template.

A reviewer checks the priorities and frequencies against how often the site actually changes, whether any other page should be `private`, and whether `requirements.txt` should pin versions, which the agent leaves as ranges and says so. Pulling the branch, installing the requirements and running `make publish` reproduces the file locally; `make html` produces one too, but with `SITEURL` empty its URLs are relative, which the pull request points out so nobody reads that as a bug.

## Run it on every Pelican site without a sitemap

A team that keeps several Pelican sites sends this body once per repository from its own server, each `POST /o/{org}/sessions` naming one repository in `resources` and carrying a `request_id` that is a UUID v5 of the repository's full name, so a retried call returns the run already started. Each run reads its own settings: a site that already sets `PLUGINS` gets `sitemap` appended to that list instead, a site with a `requirements.txt` gets one line added to it, and a site with a different `robots.txt` location gets the `Sitemap` line where its file actually lives.

The key's `webhook_url` reports each run the key starts with the pull request URL in `result`, so a script that sent the requests collects the pull requests without polling, and a run on a site whose Pelican is older than 4.5 ends with a pull request that explains the upgrade needed before the plugin can load.

## Why run it on Rush?

The settings file says whether the plugin loads, but only a build says what the sitemap contains: which URLs the plugin chose, whether `SITEURL` was set when it ran, and whether the pages that should be absent are. The agent installs Pelican and the plugin in its own sandbox, runs the publish build, reads the generated file and keeps every command in the run's record, so the pull request shows the sitemap the site will serve rather than a configuration that ought to produce one.

The change respects how the site is already run. No `PLUGINS` list appears where none existed, because that would silently disable discovery of every other namespace plugin; the `robots.txt` line uses the host the publish settings use; and the new requirements file records a dependency the deploy step needs. Ten Pelican sites get ten pull requests, each fitted to its own settings and content.

## Frequently asked questions

### Do I have to add a PLUGINS setting?

Not if you do not have one. Pelican 4.5 and later discover plugins installed under the `pelican.plugins` namespace on their own, and `pelican-sitemap` is one of them. If your settings already list `PLUGINS`, discovery is off and the agent appends `sitemap` to that list instead.

### Will it work with my theme?

Yes. The plugin writes `sitemap.xml` into the output directory during the build and does not use any template, so the theme is not changed. The only file the site serves that mentions it is `robots.txt`, which gets one `Sitemap` line.

### How do I keep a page out of the sitemap?

Add `private: True` to its metadata, or add a pattern to the `exclude` list in the `SITEMAP` setting. The pull request does the first for the 404 page and the second for the `/drafts/` and `/feeds/` paths that `robots.txt` already disallows.

### Why does the sitemap have relative URLs when I run make html?

Because `pelicanconf.py` sets `SITEURL` to an empty string for local development and the plugin uses it as the prefix. `make publish` builds with `publishconf.py`, which sets the real host, and that is the file to upload. The pull request says this so the local build is not mistaken for a bug.

### Can I set a different priority for one article?

Yes. The plugin reads `ChangeFreq` and `Priority` from an article's own metadata, with the same allowed values as the global setting, so a post you want crawled more often can say so without changing the defaults for the rest of the site.

## More Pelican jobs

- [Add Structured Data to a Pelican Blog](https://getrush.ai/run/add-structured-data/pelican/)
- [Add a Typo Check to a Pelican Site](https://getrush.ai/run/add-typo-check/pelican/)
- [Add Dark Mode to a Pelican Blog](https://getrush.ai/run/add-dark-mode/pelican/)

## The same job on other stacks

- [Add a Sitemap to a React Router App](https://getrush.ai/run/add-sitemap/react-router/)
- [Add a Sitemap to a Sphinx Docs Site](https://getrush.ai/run/add-sitemap/sphinx/)
- [Add a Sitemap to a SvelteKit App](https://getrush.ai/run/add-sitemap/sveltekit/)

## Keep reading

- [How a Rush run works on your repository, from request to pull request](https://getrush.ai/cloud/)
- [Compare cloud agents, agent sandboxes and AI code review](https://getrush.ai/compare/)
- [Rush pricing: Free, Pro and Max](https://getrush.ai/pricing/)
