All jobs

Add a Sitemap to an Astro Blog

Yes. An agent adds @astrojs/sitemap with the site value you already set, links it from the layout and robots.txt, builds, and opens a pull request.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

The agent reads what the site already knows about itself. In a blog like withastro/blog-tutorial-demo on its complete branch, package.json depends on astro at 7.0.3 in the lockfile, @astrojs/preact, @astrojs/rss and preact, with dev, build and preview scripts and nothing else. astro.config.mjs already sets site to https://example.com and registers the preact() integration, which is the one value a sitemap needs and the reason the RSS endpoint in src/pages/rss.xml.js can call context.site. public/ holds only favicon.svg, so there is no robots.txt, and BaseLayout.astro writes the <head> with charset, favicon, viewport, Astro.generator and the title.

Then it lists what the build will produce. src/pages has index.astro, about.astro, blog.astro, four Markdown posts under posts/ that use MarkdownPostLayout.astro with pubDate and tags in their frontmatter, tags/index.astro, and tags/[tag].astro, whose getStaticPaths reads every post with import.meta.glob and makes one page per unique tag. rss.xml.js is an endpoint, not a page, so it must not appear in the sitemap. The agent runs npm ci and npm run build once before changing anything, to see the current dist/ and the tag pages the glob produces.

The change is small and in three places. It installs @astrojs/sitemap, adds sitemap() to integrations in astro.config.mjs beside preact(), and keeps site as it is, since the integration refuses to build without one. In BaseLayout.astro it adds <link rel="sitemap" href="/sitemap-index.xml" /> to the head so every page points crawlers at the index. It writes public/robots.txt with User-agent: *, Allow: / and a Sitemap: line built from the configured site, and notes in the pull request that example.com is the tutorial's placeholder and must be the real domain before deploy.

It runs npm run build again and reads dist/sitemap-index.xml and dist/sitemap-0.xml, checking that the entries are the home, about, blog and tag index pages, the four posts and one page per tag, each as an absolute URL under site with the trailing slash Astro's default directory output gives, and that rss.xml is absent. It starts npm run preview, fetches /sitemap-index.xml and /robots.txt with curl, and keeps the output. There are no tests or workflows in the repository to run, so that build output is the evidence.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "codex",
  "input": "Add a sitemap to this Astro site and open a pull request. Read package.json, astro.config.mjs, src/layouts/BaseLayout.astro, src/pages/rss.xml.js, src/pages/tags/[tag].astro and the contents of public/ first. Run npm ci and npm run build once to see what dist/ contains today. Install @astrojs/sitemap and add sitemap() to integrations in astro.config.mjs, keeping the existing site value. Add a link rel=sitemap to /sitemap-index.xml in the head of BaseLayout.astro. Create public/robots.txt that allows all crawlers and names the sitemap index under the configured site. Run npm run build again, read dist/sitemap-index.xml and dist/sitemap-0.xml, and confirm the entries are exactly the pages under src/pages plus one page per tag, with rss.xml left out; if rss.xml appears, exclude it with the integration's filter option. Start npm run preview and fetch /sitemap-index.xml and /robots.txt with curl. Paste the sitemap contents and the curl output in the pull request, and say in the description that site must be the real domain before deploying. Do not change any page, post or the RSS endpoint.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

What comes back is a pull request with @astrojs/sitemap in package.json and the lockfile, sitemap() added to the integrations in astro.config.mjs, one <link rel="sitemap"> line in src/layouts/BaseLayout.astro, and a new public/robots.txt. The description pastes dist/sitemap-0.xml from the build, so a reviewer sees every URL the crawler will be offered, and the curl answers from npm run preview for the index and the robots file.

No page, post or the RSS endpoint changes, so the diff is four files. A reviewer checks that the URL list matches src/pages and the tag pages getStaticPaths produces, that rss.xml is not listed, that the Sitemap: line in robots.txt uses the same host as site, and that the site value is the real domain, which the pull request calls out because the tutorial ships example.com.

After merging, the next deploy serves /sitemap-index.xml, and submitting that URL to a search console is the one manual step left.

Run it on every new Astro site

A team that runs several Astro sites starts one run per repository with POST /o/{org}/sessions, this prompt and the repository in resources. The agent reads each site's own astro.config.mjs, so a site with no site value gets a pull request that adds one from its deployment URL rather than a failing build, and a site with pages behind authentication gets a filter that leaves them out.

The key's webhook_url receives run.completed for each run it starts, with data.session_url for the pull request link.

The use case this job follows, with its automation and trigger: Hand off a long task.

Why run it on Rush?

The integration is one line, and the mistakes are in what it emits: an endpoint listed as a page, a placeholder host in every URL, or a draft page offered to crawlers. In a Rush sandbox the agent builds the site before and after the change, reads the generated XML, and serves it with npm run preview, so the pull request shows the exact URL list rather than a promise that the integration was installed.

The same request works for every Astro site a team maintains, each run in its own sandbox with the build output kept in the transcript, and the reviewer's job is to read one XML file against the pages they know they publish.

Frequently asked questions

Why keep site as example.com?

The tutorial repository sets site to https://example.com, and the sitemap integration needs some value to build absolute URLs. The agent changes no page content and leaves the placeholder in place, and the pull request says plainly that it has to be the deployed domain before the sitemap is useful.

Does the RSS feed end up in the sitemap?

It should not: src/pages/rss.xml.js is an endpoint, and the integration lists HTML pages. The agent checks dist/sitemap-0.xml after the build rather than assuming, and uses the integration's filter option to drop it if it appears.

Will the tag pages be included?

Yes. src/pages/tags/[tag].astro builds one static page per tag through getStaticPaths, and those pages are in dist/ when the sitemap is written, so each tag URL is listed alongside the posts under src/pages/posts.

Can it set lastmod from each post's pubDate?

The integration takes a serialize function that can set lastmod per URL, but the frontmatter pubDate is not available to it at that point without extra work. This pull request leaves lastmod out; a follow-up on the same run can add it if the content collections branch is adopted.

More Astro jobs

The same job on other stacks

Keep reading