# Write Pytest Tests for a Reflex App

Yes. An agent writes browser tests for your Reflex app with AppHarness and Playwright, matching the layout your CI already discovers, runs them and opens a PR.

[Run this on Rush](https://cloud.getrush.ai/run/write-pytest-tests/reflex/). Opens the console with this prompt ready to send, in a Project that has your repository.

## What the agent does

The agent reads how the app and its neighbours are tested before writing anything, because Reflex apps are tested by running them. In a repository like reflex-dev/reflex-examples, the `todo` app has `rxconfig.py` with `app_name="todo"` and the Radix themes plugin, `requirements.txt` pinning `reflex>=0.9.2`, and `todo/todo.py` with a `State` whose `items` list starts with three entries, an `add_item` handler that reads `new_item` from the submitted form, and a `finish_item` handler that pops an item; the page is a heading, a form with an input named `new_item` and an `Add` button with `reset_on_submit=True`, and an ordered list where each item has an icon button that finishes it. There is no `tests` directory and no `requirements-dev.txt`. The repository's `.github/workflows/app_harness.yml` discovers every example that has a `tests` directory, fails if it lacks `requirements-dev.txt`, installs `reflex` with both requirement files, runs `playwright install chromium --with-deps --only-shell`, `reflex init` and `pytest tests -vv` with `APP_HARNESS_HEADLESS=1`. Two neighbours show the house style: `counter/tests/test_counter.py` drives the app through `AppHarness.create(root=...)` and a Selenium driver from `harness.frontend()` with `poll_for_content`, while `form-designer/tests/conftest.py` keeps a session-scoped `AppHarness` fixture and the tests use Playwright's `page` fixture against `harness.frontend_url`.

The agent follows the newer pattern. `todo/tests/conftest.py` defines a session-scoped `todo_app` fixture that enters `AppHarness.create(root=Path(__file__).parent.parent)`, which compiles the app, starts the backend under uvicorn in a thread and the frontend as a dev server subprocess, and exposes `frontend_url` once the frontend is listening. `todo/requirements-dev.txt` lists `pytest`, `playwright`, `pytest-playwright`, `uvicorn` and `psutil`, the same lines `form-designer` uses, so the existing workflow picks the app up with no change to CI.

`todo/tests/test_todo.py` then pins down the behaviour in `todo.py`. One test opens the page and asserts the three default items are listed in order. One fills the input with a new item, clicks `Add`, waits for a fourth list item and asserts the input is empty again because of `reset_on_submit`. One submits an empty form and asserts the list is unchanged, since `add_item` ignores a blank `new_item`. One clicks the check button inside the first list item and asserts that item is gone and the other two remain. Each test uses Playwright's `expect` with its own timeout rather than sleeps, and locates elements by role and text so a styling change does not break them.

Running them needs the frontend toolchain `reflex init` downloads and a Chromium that Playwright installs, both of which the agent sets up in its sandbox with the workflow's own commands. It runs `pytest tests -vv` under `APP_HARNESS_HEADLESS=1`, reads each failure itself, and also runs `reflex export` once, because the repository's `check_export.yml` builds every example that way and the test files must not break it. `ruff format` and `ruff` from `.pre-commit-config.yaml` run over the new files, and the app's own code is left unchanged.

## Start it with one API call

`POST https://api.phoenix.so/o/{org}/sessions`

```json
{
  "agent": "claude",
  "input": "Write pytest tests for the Reflex app in the todo directory and keep its code unchanged. Read todo/rxconfig.py, todo/requirements.txt, todo/todo/todo.py, .github/workflows/app_harness.yml, .github/workflows/check_export.yml, .pre-commit-config.yaml, counter/tests/test_counter.py, form-designer/tests/conftest.py and form-designer/requirements-dev.txt first. Add todo/requirements-dev.txt with the same lines as form-designer. Write todo/tests/conftest.py with a session-scoped AppHarness fixture rooted at the app, and todo/tests/test_todo.py using Playwright's page fixture against frontend_url: the three default items in order, adding an item and the input resetting, an empty submit changing nothing, and finishing the first item. Use expect with locators by role and text, no sleeps. In the sandbox run playwright install chromium --with-deps --only-shell, reflex init, pytest tests -vv with APP_HARNESS_HEADLESS=1, reflex export, and ruff over the new files, fix what fails, and open a pull request that lists each test and what it asserts.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
```

1. **Get an API key.** In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
2. **Send the request.** POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
3. **Read what came back.** Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

## What comes back

What comes back is a pull request that adds `todo/requirements-dev.txt`, `todo/tests/conftest.py` and `todo/tests/test_todo.py` and changes nothing else. The description lists the four tests and the line in `todo.py` each one pins down, notes that `app_harness.yml` discovers the new `tests` directory on its own, and reports what `reflex export` produced.

The run's record keeps the `reflex init`, `pytest tests -vv`, `reflex export` and `ruff` output, so a reviewer sees the tests pass in a headless browser rather than trusting a sentence.

Before merging, a reviewer decides whether the shared fixture should move up to a repository-level `conftest.py` so the next example gets it for free, and whether to pin `pytest-playwright` in `requirements-dev.txt` as the other apps leave it unpinned.

## Run it on every night

In the console, add an automation to the Project with this prompt and a `schedule` trigger, a `cron` of `0 2 * * *` and your `timezone`. Each night one run finds the apps that changed since the last run and still have no `tests` directory, writes tests for them the same way, and opens one pull request on its own branch. A run whose tests cannot be made to pass ends `needs_review` rather than merging anything.

The same body sent to `POST /o/{org}/sessions` with the repository in `resources` starts one run by hand, with the app directory written into the prompt. A `request_id` that is a UUID v5 of the repository and directory together makes a retried call return that run instead of a second pull request.

## Why run it on Rush?

A Reflex test is a running app: a backend, a compiled frontend, a browser and the toolchain `reflex init` fetches on first use. None of that belongs on a laptop that is also editing the code, and the agent's sandbox has room for all of it. It runs the suite after every file it writes and reads the failures itself, so a locator that matches two buttons or an `expect` that races the websocket is fixed before anyone reviews it.

Because every `pytest` run and the `reflex export` stay with the record, the pull request can say which handlers in `todo.py` are now covered and show the headless run that proved it, instead of asking a reviewer to install Chromium and find out.

## Frequently asked questions

### Can I test the State class without a browser?

Handlers like `add_item` are plain methods on a `State` subclass, but the list the page shows comes from the compiled frontend through a websocket, so this repository's own tests drive the app in a browser with `AppHarness`. The agent follows that pattern; pure handler tests are a separate, smaller layer it mentions in the pull request.

### Why Playwright rather than the Selenium driver that AppHarness.frontend() returns?

Both work, and `counter` uses Selenium. The `form-designer` tests use Playwright's `page` fixture with `frontend_url`, its `expect` waits on its own, and `app_harness.yml` already installs Chromium for Playwright, so new tests follow the newer of the two patterns.

### Does CI need any change?

No. The workflow lists every directory with a `tests` folder and requires `requirements-dev.txt` beside it; adding both to `todo` puts it in the matrix on the next push.

### What does the agent need in its sandbox?

Python with the two requirement files, the frontend toolchain `reflex init` installs, and a Chromium from `playwright install`. It runs the same commands as the workflow under `APP_HARNESS_HEADLESS=1`, so what passes there is what CI runs.

## More Reflex jobs

- [Run a Long Task in the Background in a Reflex App](https://getrush.ai/run/add-background-tasks/reflex/)
- [Add Chat Export to a Reflex Chat App](https://getrush.ai/run/add-chat-export/reflex/)
- [Add Dark Mode to a Reflex App](https://getrush.ai/run/add-dark-mode/reflex/)

## The same job on other stacks

- [Write Pytest Tests for a Textual Terminal App](https://getrush.ai/run/write-pytest-tests/textual/)
- [Write pytest Tests for Celery Tasks](https://getrush.ai/run/write-pytest-tests/celery/)
- [Write pytest Tests for a discord.py Bot](https://getrush.ai/run/write-pytest-tests/discord-py/)

## Keep reading

- [How a Rush run works on your repository, from request to pull request](https://getrush.ai/cloud/)
- [Compare cloud agents, agent sandboxes and AI code review](https://getrush.ai/compare/)
- [Rush pricing: Free, Pro and Max](https://getrush.ai/pricing/)
