All jobs

Write Playwright Tests for an Eleventy Blog

Yes. An agent builds your Eleventy site, serves it, writes Playwright tests for the pages, feed and sitemap it finds, adds a CI workflow and opens a PR.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

The agent clones the site into its own cloud sandbox and reads how it is built before writing a test. In a site like 11ty/eleventy-base-blog, package.json says "type": "module", pins @11ty/eleventy at ^3.1.7 with the image, navigation, RSS and syntax-highlight plugins, declares engines.node as >=18 with a .nvmrc of 22, ships a package-lock.json so npm is the package manager, and has build and start scripts but no test script at all. eleventy.config.js sets the input directory to content, the output to _site, drops pages with draft: true from a build run but keeps them with a (draft) title suffix during --serve, registers an Atom feed at /feed/feed.xml limited to ten posts, and bundles inline CSS and JavaScript into dist. The content folder holds a home page that shows the three latest posts, an Archive page, a tags index with one page per tag, an about page, a 404.md with permalink: 404.html, a sitemap.xml.njk that lists every page with a permalink, and five posts under content/blog.

It then decides what is worth asserting, which for a static site is the output rather than the templates. From _includes/layouts/base.njk it takes the skip link to #main, the <nav> whose current entry carries aria-current="page", the <link rel="alternate"> pointing at the feed, and the <heading-anchors> web component wrapped around the content. From content/index.njk it takes the Latest 3 Posts heading and the sentence saying how many more posts the archive holds, which with five posts is two. From content/blog.njk it takes the archive list, and from content/blog/blog.11tydata.js the fact that every post gets the posts tag and the layouts/post.njk layout, so each post page has the post title as its <h1> and a readable date produced by the readableDate filter in _config/filters.js.

It adds @playwright/test as a dev dependency, a playwright.config.js whose webServer runs npx @11ty/eleventy --serve --quiet and waits for the dev server's port, and a tests folder with one spec per area: navigation (the four eleventyNavigation entries, the active one marked, the skip link focusable), home and archive (post counts and the order newest first), a post page (heading, date, the tags links that resolve to tag pages), the 404 page, the feed (fetched as text, parsed as XML, one <entry> per post, each <link> an absolute URL under the metadata.url of https://example.com/), and the sitemap (every <loc> returns a 200 from the running server). Because the dev server includes drafts and a build does not, the spec that counts posts reads the five Markdown files under content/blog first and derives the expected number instead of hard-coding it, and a comment says why.

It runs the suite in the sandbox with npx playwright install --with-deps chromium first, fixes any selector that was wrong against the real markup, adds a test script to package.json, and writes .github/workflows/test.yml that runs npm ci, the Playwright browser install and npm test on pushes and pull requests. The sample gh-pages.yml.sample deployment workflow is left as it is. The pull request shows the test output from the sandbox, and nothing under content, _includes or eleventy.config.js changes.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "claude",
  "input": "Write Playwright end-to-end tests for this Eleventy site. Read package.json, eleventy.config.js, the _includes layouts and everything under content first, and build the site once so you test real output. Add @playwright/test, a playwright.config.js whose webServer starts the Eleventy dev server, and specs for navigation and the skip link, the home and archive post lists, one post page, the 404 page, the Atom feed and the sitemap; derive expected post counts from the content files rather than hard-coding them, and say in a comment why drafts make the dev server and a build differ. Install the Chromium browser, run the suite until it passes, add a test script and a GitHub Actions workflow that runs it, and open a pull request with the test output. Do not change templates, content or the Eleventy config.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

What comes back is a pull request adding @playwright/test to devDependencies, a test script, playwright.config.js, a tests folder with one spec file per area of the site, and .github/workflows/test.yml. The description lists each spec and what it asserts, in the words of the markup it checked, and ends with the Playwright run from the sandbox: which tests ran, against which browser, and how long the dev server took to come up.

The run's record keeps the build and the test run, so a reviewer can see which selectors the agent tried before settling on the ones in the specs and that the feed and sitemap assertions were checked against the served files rather than written from memory.

Before merging, a reviewer runs npm ci && npm test once locally, decides whether the workflow should also run a build to catch a draft that leaks into production, and checks that no spec depends on the placeholder text in the sample posts, which the first real post will replace.

Run it on every nightly, for a site without tests

Create an automation in the Project that has the site, with a schedule trigger set to a cron such as 0 2 * * * in your timezone and this prompt. Once the first pull request is merged, the same prompt makes later runs extend the suite: each one reads the specs that exist, looks for a page or template added since the last run, and opens a pull request only when it found something untested. An automation run has its own max_runs_per_hour cap.

A team with several Eleventy sites starts one run per repository from a script instead: one POST /o/{org}/sessions per site with this body, a request_id that is a UUID v5 of the repository name so a retry returns the run already started, and the key's webhook_url reporting each finished run with its pull request.

The use case this job follows, with its automation and trigger: Write the missing tests.

Why run it on Rush?

Playwright tests for a static site are cheap to write and expensive to get wrong, because every assertion is about rendered output that only exists after a build. The agent builds and serves the site in its own sandbox, installs the browser there, and iterates on selectors against the real HTML until the suite passes, so the pull request arrives with the evidence rather than a promise.

The same body works across every Eleventy site a team owns, each run reading that site's own config and content, and each pull request carrying its own run output. Nothing is installed on anyone's laptop, and a reviewer judges the tests by reading them and the output beside them.

Frequently asked questions

Will the tests run against the dev server or a built site?

Against the dev server by default, because playwright.config.js starts npx @11ty/eleventy --serve --quiet as its webServer. The agent's comment in the post-count spec explains the one difference that matters: the dev server includes drafts, a build drops them. If you deploy from _site, switch the webServer command to a build followed by a static file server and the specs still hold.

Does it work with pnpm or Yarn?

Yes. The agent reads the lockfile to pick the package manager, and in eleventy-base-blog that is package-lock.json, so it uses npm ci in the workflow. With pnpm-lock.yaml or yarn.lock it writes the matching install step and script instead.

What happens when I rename a page or add a nav entry?

The navigation spec reads the eleventyNavigation entries from the rendered <nav> and checks each link resolves, so adding an entry passes without edits. Renaming the Archive page or changing numberOfLatestPostsToShow in content/index.njk fails the home spec, which is the point: the test names the text it expects, so the fix is one line.

Will it check the images the image plugin generates?

Only when a post uses one. eleventy.config.js turns img tags into picture elements with AVIF and WebP sources through eleventyImageTransformPlugin, and the sample fourth post has an image, so the post spec asserts a <picture> with those sources is present. A site with no images gets no image assertion.

Can it run the suite in CI without a browser install step?

No. Playwright downloads browsers on demand, so .github/workflows/test.yml runs npx playwright install --with-deps chromium after npm ci. The workflow installs Chromium only, which keeps the job short; add firefox or webkit to the config's projects and the install step when you want them.

More Eleventy jobs

The same job on other stacks

Keep reading