What the agent does
The agent reads the settings before it touches them, because a Pelican plugin is enabled by what pelicanconf.py leaves out as much as by what it says. In a site laid out like getpelican/pelican-blog, pelicanconf.py sets SITENAME, PATH = "content", THEME = "themes/notmyidea", TIMEZONE = "Europe/Rome", DEFAULT_PAGINATION = 5, STATIC_PATHS = ["images", "extra"] and an EXTRA_PATH_METADATA table that moves extra/robots.txt and the favicon files to the site root; it turns content caching off and configures Markdown extensions, and it has no PLUGINS setting at all. publishconf.py imports it, sets SITEURL = "https://blog.getpelican.com" and DELETE_OUTPUT_DIRECTORY = True. The Makefile builds with pelican content -o output -s pelicanconf.py for make html and with publishconf.py for make publish, and make upload rsyncs output/ to the project's server. content/news/ holds twenty-one posts, all reStructuredText but one Markdown file, and content/pages/ holds index.rst, gratitude.rst and not-found.rst. content/extra/robots.txt disallows /drafts/ and /feeds/ and names no sitemap. There is no requirements file and no workflow, and .gitignore keeps output/ out of the repository.
The plugin is pelican-sitemap, published by the Pelican project on PyPI, 1.2.2 at the time of reading, which requires pelican>=4.5. Pelican's plugin documentation explains the part that matters here: since 4.5 a plugin installed under the pelican.plugins namespace is discovered and registered automatically as long as PLUGINS is left unset, and the pelican-plugins command lists the ones that are active. Because this site never set PLUGINS, installing the package is enough to enable it, and the agent says so in the pull request rather than adding a PLUGINS list that would turn auto-discovery off for every plugin added later. The plugin writes sitemap.xml into the output directory and reads one setting, a SITEMAP dictionary with format, priorities and changefreqs for articles, pages and indexes, and an exclude list of regular expressions; a page can also leave itself out with private: True in its metadata, and an article can set its own ChangeFreq and Priority.
The agent adds that dictionary to pelicanconf.py under the existing settings: format set to xml, articles at a higher priority than index pages, changefreqs of monthly for articles and pages and daily for indexes, and exclude entries for /drafts/ and /feeds/ so the sitemap and robots.txt agree about what should not be crawled. It reads content/pages/not-found.rst, which exists to be served as the site's 404 page, and marks it private so it is not offered to search engines as a page. One line goes at the end of content/extra/robots.txt, Sitemap: https://blog.getpelican.com/sitemap.xml, with the host taken from SITEURL in publishconf.py, because EXTRA_PATH_METADATA already publishes that file at the root. A requirements.txt with pelican, markdown for the one Markdown post, and pelican-sitemap is added so the plugin is installed wherever the site is built, since nothing in the repository recorded its dependencies before.
Then the agent builds. It installs those packages in its sandbox, where Pelican 4.12.0 needs Python 3.11 or newer, runs pelican-plugins to confirm sitemap is listed, and runs make publish, which uses publishconf.py so the URLs in the sitemap carry the real host instead of the empty SITEURL of the development settings. It opens output/sitemap.xml and checks that every post under content/news/ appears with a lastmod, that the gratitude page appears and the not-found page does not, and that nothing under /feeds/ is listed. The README mentions only make upload, so the pull request explains that the plugin must be installed on the machine that runs it, which is what the new requirements file is for.
Start it with one API call
POST https://api.phoenix.so/o/{org}/sessions with this body:
{
"agent": "claude",
"input": "Add an XML sitemap to this Pelican site with the pelican-sitemap plugin. Read pelicanconf.py, publishconf.py, the Makefile, the README, content/extra/robots.txt and the content directory first. Do not add a PLUGINS setting if there is none; rely on namespace plugin discovery and say so. Add a SITEMAP dictionary to pelicanconf.py with format xml, priorities and changefreqs for articles, pages and indexes, and exclude patterns matching what robots.txt disallows. Mark the 404 page private. Append a Sitemap line to robots.txt using the SITEURL from publishconf.py. Add a requirements.txt with pelican, markdown and pelican-sitemap. Install them, run pelican-plugins to confirm the plugin is active, build with the publish settings and check output/sitemap.xml lists every article and page except the private one. Open a pull request with the build output and a sample of the sitemap.",
"resources": [
{
"url": "https://github.com/your-org/your-app"
}
]
}- Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
- Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
- Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.
What comes back
The pull request changes pelicanconf.py, content/extra/robots.txt and content/pages/not-found.rst, and adds requirements.txt. Its description quotes the pelican-plugins output showing sitemap as active, the make publish summary, and the first entries of output/sitemap.xml with their loc, lastmod, changefreq and priority values, so a reviewer can see the host and the dates without building. The theme is untouched: the sitemap is a file the plugin writes, not a template.
A reviewer checks the priorities and frequencies against how often the site actually changes, whether any other page should be private, and whether requirements.txt should pin versions, which the agent leaves as ranges and says so. Pulling the branch, installing the requirements and running make publish reproduces the file locally; make html produces one too, but with SITEURL empty its URLs are relative, which the pull request points out so nobody reads that as a bug.
Run it on every Pelican site without a sitemap
A team that keeps several Pelican sites sends this body once per repository from its own server, each POST /o/{org}/sessions naming one repository in resources and carrying a request_id that is a UUID v5 of the repository's full name, so a retried call returns the run already started. Each run reads its own settings: a site that already sets PLUGINS gets sitemap appended to that list instead, a site with a requirements.txt gets one line added to it, and a site with a different robots.txt location gets the Sitemap line where its file actually lives.
The key's webhook_url reports each run the key starts with the pull request URL in result, so a script that sent the requests collects the pull requests without polling, and a run on a site whose Pelican is older than 4.5 ends with a pull request that explains the upgrade needed before the plugin can load.
The use case this job follows, with its automation and trigger: Hand off a long task.
Why run it on Rush?
The settings file says whether the plugin loads, but only a build says what the sitemap contains: which URLs the plugin chose, whether SITEURL was set when it ran, and whether the pages that should be absent are. The agent installs Pelican and the plugin in its own sandbox, runs the publish build, reads the generated file and keeps every command in the run's record, so the pull request shows the sitemap the site will serve rather than a configuration that ought to produce one.
The change respects how the site is already run. No PLUGINS list appears where none existed, because that would silently disable discovery of every other namespace plugin; the robots.txt line uses the host the publish settings use; and the new requirements file records a dependency the deploy step needs. Ten Pelican sites get ten pull requests, each fitted to its own settings and content.
Frequently asked questions
Do I have to add a PLUGINS setting?
Not if you do not have one. Pelican 4.5 and later discover plugins installed under the pelican.plugins namespace on their own, and pelican-sitemap is one of them. If your settings already list PLUGINS, discovery is off and the agent appends sitemap to that list instead.
Will it work with my theme?
Yes. The plugin writes sitemap.xml into the output directory during the build and does not use any template, so the theme is not changed. The only file the site serves that mentions it is robots.txt, which gets one Sitemap line.
How do I keep a page out of the sitemap?
Add private: True to its metadata, or add a pattern to the exclude list in the SITEMAP setting. The pull request does the first for the 404 page and the second for the /drafts/ and /feeds/ paths that robots.txt already disallows.
Why does the sitemap have relative URLs when I run make html?
Because pelicanconf.py sets SITEURL to an empty string for local development and the plugin uses it as the prefix. make publish builds with publishconf.py, which sets the real host, and that is the file to upload. The pull request says this so the local build is not mistaken for a bug.
Can I set a different priority for one article?
Yes. The plugin reads ChangeFreq and Priority from an article's own metadata, with the same allowed values as the global setting, so a post you want crawled more often can say so without changing the defaults for the rest of the site.
More Pelican jobs
The same job on other stacks
- Add a Sitemap to a React Router App (React Router)
- Add a Sitemap to a Sphinx Docs Site (Sphinx)
- Add a Sitemap to a SvelteKit App (SvelteKit)