What the agent does
In an Astro site like satnaing/astro-paper the agent reads package.json first: astro ^7.0.3 with @astrojs/mdx, @astrojs/rss and @astrojs/sitemap, Node 22.12 or newer in engines, a pnpm-lock.yaml, and a build script that runs astro check, astro build, then pagefind --site dist and copies dist/pagefind into public/. There is no test script and no test runner; ci.yml installs with pnpm 11.3.0 on Node 24 and runs lint, format:check and build. The search page depends on that Pagefind index, which only exists after a build, so the tests have to run against the built output rather than astro dev.
Then it maps the routes under src/pages: index.astro, about.astro, posts/[...page].astro paginated at perPage 4 from astro-paper.config.ts, posts/[...slug]/index.astro with a sibling index.png.ts that renders an Open Graph image with satori and sharp for every post without its own ogImage, tags/index.astro and tags/[tag]/[...page].astro, search.astro, rss.xml.ts, og.png.ts, robots.txt.ts and 404.astro. src/content.config.ts loads posts from src/content/posts with a glob that skips files starting with an underscore, and postFilter in src/utils drops drafts and, in production, posts whose pubDatetime is still in the future. The agent picks its fixtures from what the build will actually emit: a published post such as adding-new-post, a tag that post carries, and the draft in src/content/posts/examples that must not appear.
It adds @playwright/test as a dev dependency with pnpm, writes playwright.config.ts with a webServer that runs pnpm preview against the finished dist on the port compose.yaml exposes, 4321, and a test script in package.json. The specs live in a tests directory: the home page shows the site title and the featured and recent post lists; a post page renders its heading, its Datetime and its Tag links; the tags index lists each tag and a tag page lists only posts with that tag; the search page mounts the Pagefind UI, and typing a word from a known post yields a result that links to it; GET /rss.xml answers with an XML content type and an item per published post with the link getPostUrl builds; GET /og.png and a post's index.png answer image/png.
The run executes pnpm install, pnpm build and pnpm exec playwright install chromium, then runs the suite headless until it passes, and also runs pnpm lint and pnpm format:check so the new files follow eslint.config.js and the Prettier plugins for Astro and Tailwind. Because getStaticPaths for the OG images returns nothing when dynamicOgImage is off and search.astro rewrites to 404 when the search feature is not pagefind, the tests read astro-paper.config.ts and skip the cases those flags disable instead of failing on a site that turned them off.
Start it with one API call
POST https://api.prix.dev/o/{org}/sessions with this body:
{
"agent": "claude",
"input": "Write Playwright tests for this Astro site that build it, serve the output, and check the home page, a post page, tag pages, search and the RSS feed respond and render. Read package.json and astro-paper.config.ts first: the site builds with pnpm, the build step runs Pagefind after astro build, and search only works against the built output. Add @playwright/test as a dev dependency, a playwright.config.ts whose webServer runs pnpm preview, and a test script. Cover the home page, one published post page including its tags and date, the tags index and one tag page, the Pagefind search page with a query that matches a known post, GET /rss.xml with one item per published post, and the Open Graph PNG routes. Skip a case cleanly when its feature flag is off in astro-paper.config.ts. Run pnpm build, then the suite headless until it passes, then pnpm lint and pnpm format:check, and open a pull request.",
"resources": [
{
"url": "https://github.com/your-org/your-app"
}
]
}- Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
- Send the request. POST the body in request.json to https://api.prix.dev/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
- Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.
What comes back
The pull request adds playwright.config.ts, a tests directory with one spec per area, @playwright/test under devDependencies, a test script in package.json and the updated pnpm-lock.yaml. It changes no page, component or content file. Its description lists each test with the route it hits and the assertion it makes, and quotes the Playwright summary from the run.
A reviewer opens the spec for search first, since it is the one that would silently pass on an empty index: it should wait for the Pagefind UI to mount and assert on a result link, not just on the input. Then they check that the post slug and tag used as fixtures exist in src/content/posts and are not drafts, and that the config reads the flags from astro-paper.config.ts rather than hard-coding them.
The session record keeps the build output, the Playwright run and every file the agent read, so the claim that the suite passed against the built site can be checked line by line. A request to add the about page or the archives route is a follow-up message to the same session, and the agent extends the same branch.
Run it on every pull request
Once the suite exists, run it on every pull request from a GitHub Actions job: pnpm install, pnpm build, pnpm exec playwright install --with-deps chromium and pnpm test, beside the lint, format and build steps already in ci.yml. The agent's pull request can add that job when you ask for it in the prompt.
To have an agent write or extend tests for each new theme or site built from this template, a handler on your server sends this request with the new repository in resources whenever one is created, with the repository name as the Idempotency-Key, and reads the pull request link from the run.completed webhook.
The use case this job follows, with its automation and trigger: Write the missing tests.
Why run it on Rush?
Writing end-to-end tests for a static site is mostly waiting: build, index, serve, run, read the failure, change a selector, build again. In a Rush sandbox the agent does that loop on its own machine with its own browser install, while your working copy and your dev server stay untouched.
The pull request comes with proof rather than a promise: the build log, the Playwright output and the transcript of what the agent read before choosing each fixture post and tag. Several sites built from the same theme can get the same request at once, each in its own sandbox.
Frequently asked questions
Why run against pnpm preview instead of astro dev?
Search. The build script runs Pagefind over dist and copies the index into public/pagefind, and search.astro loads that bundle. astro dev never produces it, so a search test under the dev server would fail or be skipped. Preview serves the same files a deploy would.
Will it change my pages or content?
No. The diff is the Playwright config, the tests, the dev dependency and the lockfile. If a test reveals a real bug in a page, the agent reports it in the pull request and leaves the fix for a separate request.
Which post does it use as a fixture?
One the build will publish: not a draft, not under a folder starting with an underscore, and with a pubDatetime in the past, because postFilter hides the others in production. The pull request names the post and tag it chose so you can swap them for your own content.
Does it work with npm or yarn instead of pnpm?
Yes. This repository has a pnpm-lock.yaml and its CI uses pnpm, so the agent follows that. With a package-lock.json or yarn.lock it uses that tool for the install, the build and the Playwright browser install.