All jobs

Write Playwright Tests for a SolidJS App

Yes. An agent adds Playwright to your SolidJS app, writes specs for its hash-routed pages against a mocked API, runs them headless in its sandbox and opens a PR.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

The agent reads how the app is built and served before it writes a test, because a SolidJS app bundled by Rollup has no dev server of its own. In an app like solidjs/solid-realworld, package.json has "test": "test" as a placeholder, start runs serve public -l 5000 -s next to rollup -w -c, and build is rollup -c --environment production. rollup.config.js compiles src/index.js with babel-preset-solid into an ES module bundle in public/, and rollup-plugin-delete wipes everything in that folder except public/index.html on each build, so the agent keeps test output in test-results/ and playwright-report/ at the root and lists both in the ignore file. public/index.html loads the RealWorld theme and icons from CDNs, which the tests do not need to intercept.

Routing is the next thing it maps. src/store/createRouteHandler.js reads window.location.hash.slice(2) and listens for hashchange, and src/App.js matches that string against regular expressions for editor, settings, login, register, article/<slug>, @<username> with an optional /favorites, and the empty home route. src/components/NavLink.js writes every link as #/<route>, so a spec navigates with page.goto('/#/login') and asserts on the hash, never on a server path. The store in src/store/index.js keeps the JWT in localStorage under the key jwt, which gives the specs a cheap way to start a test signed in without walking through the form each time.

Every request goes through src/store/createAgent.js to https://api.realworld.io/api, a public demo backend the tests must not depend on. The agent writes a page.route fixture for **/api/** that answers the shapes the store expects: GET /articles?limit=10&offset=0 with articles and articlesCount, GET /tags with tags, POST /users/login and GET /user with a user carrying a token, GET /articles/<slug> with one article, and GET /articles/<slug>/comments. With articlesCount set above ten, src/components/ArticleList.js renders the .pagination list, since src/store/createArticles.js divides the count by its LIMIT of 10, and a spec can click page two and check the offset query of the request that follows.

The specs then cover what FRONTEND_INSTRUCTIONS.md lists as the app's pages: the home page shows the conduit brand and the Global Feed tab, with Your Feed only once a token is present; registering through src/pages/Auth.js sends username, email and password and lands on #/, after which src/components/NavBar.js shows New Post, Settings and the username; the article page renders the title and the Markdown body that src/pages/Article/Article.js passes through marked; the editor creates an article and redirects to its slug; the profile page lists the author's articles. playwright.config.js builds the app and serves public/ on port 5000 with the serve package already in devDependencies, and the agent installs @playwright/test with Chromium, runs the suite headless in its sandbox, and replaces the placeholder test script. Because .github/workflows/gh-pages.yml builds on Node 14 with actions/setup-node@v2, it adds a separate test job on a current Node line rather than changing the deploy job.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "claude",
  "input": "Add Playwright end-to-end tests to this SolidJS app. Read package.json, rollup.config.js, src/App.js, src/store/createRouteHandler.js, src/store/createAgent.js and the pages under src/pages first. The app uses hash routing and calls a public RealWorld API, so mock every request under /api with page.route and never hit the network. Write specs for the home feed and its pagination, registering and signing in, reading an article, creating one in the editor, and the profile page. Configure a webServer that builds the app and serves public/ with the serve package, run the suite headless with Chromium, add a CI job for it on a current Node version without changing the GitHub Pages deploy job, and open a pull request with the passing output.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

What comes back is a pull request on its own branch adding playwright.config.js, an e2e/ directory with a fixtures module for the mocked API and one spec per page, @playwright/test in devDependencies with the lockfile updated, a real test script, ignore entries for the report folders, and a test job in the workflow that installs Chromium with --with-deps and runs npx playwright test. No file under src/ changes.

The description lists each spec and the flow it covers, quotes the suite's summary line from the sandbox run, and names the one route the mocks had to shape carefully: the offset and limit query that the pagination spec asserts on. It also says which routes are not covered yet, such as settings and comments, so the next run has a starting point.

Before merging, a reviewer reads the mocked response shapes against the RealWorld API spec the app follows, checks that no spec can reach api.realworld.io when a mock is missing, and runs the job once on the pull request to see it pass on GitHub's runner as it did in the sandbox.

Run it on every Sunday night

An automation in the Project with a schedule trigger, a cron for Sunday night and your timezone, starts one run a week with this prompt: find the page or flow with no spec, write one against the mocked API, and open a pull request. The trigger puts the slot that fired it after the prompt and fills in no placeholder; the saved prompt is the whole instruction, and the run reads the repository to decide what is still untested. An automation's runs are capped by its own max_runs_per_hour.

Each run opens its own pull request, so the team reviews one spec on Monday morning with the transcript beside it: the routes it mocked, the commands it ran, and the suite's output. A branch that cannot pass is left as a branch with the failing output in the result rather than merged.

The use case this job follows, with its automation and trigger: Write the missing tests.

Why run it on Rush?

A SolidJS app with hash routing and a third-party demo API is easy to test badly: specs that depend on the public backend go red when its data changes, and specs that navigate by server path never find the page. The agent reads the router, the store and the agent module first, and the mocks it writes come from the request shapes in the code rather than from guesses.

On Rush the build, the static server and Chromium all run in the agent's own sandbox, and the transcript keeps the exact commands and the suite's output, so the pull request's claim that the suite passes comes with the evidence. Many apps can get the same treatment at once, each in its own sandbox with its own pull request.

Frequently asked questions

Will the tests hit the real RealWorld API?

No. A shared fixture intercepts every request to /api/ with page.route and answers from fixtures in the repository. A request with no mock fails the spec rather than falling through to the network, so a missing mock shows up as a test failure, not as a dependency on the demo backend.

How do the specs navigate when the app has no server routes?

Through the hash. The router reads window.location.hash, so the specs call page.goto('/#/login') and the like and assert on page.url() ending in the expected hash. The static server's -s flag only matters for a direct load; after that, navigation is client-side.

Does this change the GitHub Pages deploy?

No. The deploy job keeps building on its pinned Node version and publishing public/. The agent adds a separate job for the tests on a current Node line, because @playwright/test needs a newer runtime than the deploy job uses.

Can it also test signed-in flows without the form?

Yes. The store reads the token from localStorage under jwt on load, so a spec can set that key before the first navigation and mock GET /user to return the signed-in user. The registration spec still exercises the form once so the real flow is covered.

More SolidJS jobs

The same job on other stacks

Keep reading