# Write Integration Tests for a Reflex App

Yes. An agent writes browser tests for a Reflex app with reflex.testing.AppHarness and Playwright, runs them headless in its sandbox and opens a PR that CI picks up.

[Run this on Rush](https://cloud.getrush.ai/run/write-integration-tests/reflex/). Opens the console with this prompt ready to send, in a Project that has your repository.

## What the agent does

The agent reads the app and the tests the repository already has before it writes new ones. In a repository like reflex-dev/reflex-examples, the `todo` app is one file, `todo/todo/todo.py`: a `State` with an `items` list that starts as `Write Code`, `Sleep` and `Have Fun`, an `add_item` handler that reads `new_item` from the submitted form data and appends it only when it is not empty, and a `finish_item` handler that removes an item by value. The page renders a `Todos` heading, an `rx.form` with an input named `new_item`, the placeholder `Add a todo...` and an `Add` button, with `on_submit` set to `add_item` and `reset_on_submit` true, then an ordered list built with `rx.foreach` where each item has an icon button with a check icon that calls `finish_item`. `todo/requirements.txt` pins `reflex>=0.9.2`, `rxconfig.py` names the app `todo`, and there is no `tests` directory and no `requirements-dev.txt`.

Two sibling apps show the house style. `counter/tests/test_counter.py` opens the app with `AppHarness.create(root=Path(__file__).parent.parent)` in a fixture, drives it with the Selenium driver from `harness.frontend()`, and waits with `poll_for_content` and `AppHarness._poll_for`. `form-designer/tests/conftest.py` makes the harness session-scoped and its tests use the `page` fixture from pytest-playwright, `page.goto(harness.frontend_url)`, locators by role and placeholder, and Playwright's `expect`. `AppHarness` in `reflex.testing` compiles the app, starts the backend on a free port and the frontend dev server with `PORT=0`, and exposes `frontend_url`; its `DEFAULT_TIMEOUT` is 15 seconds and `POLL_INTERVAL` a quarter second. `.github/workflows/app_harness.yml` builds its matrix from every `tests` directory in the repository, requires a `requirements-dev.txt` beside it, installs `reflex` with both requirement files, runs `playwright install chromium --with-deps --only-shell`, then `reflex init` and `pytest tests -vv` with `APP_HARNESS_HEADLESS` set to 1 and `REFLEX_TELEMETRY_ENABLED` false.

The agent adds `todo/tests/conftest.py` with a session-scoped `todo_app` fixture that yields a started `AppHarness` for the app's root, and `todo/tests/test_todo.py` with four tests in the Playwright style: the page loads with the heading and the three starting items in order; filling the input and clicking `Add` appends a fourth item and clears the input, which checks `reset_on_submit`; clicking the check button inside the `Sleep` item removes that item and leaves the other two, which checks that `finish_item` removes by value and not by position; and submitting the form with an empty input adds nothing, which checks the guard in `add_item`. Each test waits with `expect` on the list rather than sleeping, and the assertions read the list items by role so a style change does not break them. `todo/requirements-dev.txt` lists `pytest`, `playwright`, `pytest-playwright`, `uvicorn` and `psutil`, the same five lines as the form designer's, which is what the workflow installs.

Then the agent runs it the way the workflow does. In its own sandbox it makes a virtual environment in `todo/`, installs `reflex`, the current 0.10.1, with both requirement files, installs the Chromium shell with Playwright, runs `reflex init`, which fetches the frontend toolchain the first time, and runs `pytest tests -vv` with `APP_HARNESS_HEADLESS=1`. It then breaks the app on purpose, dropping the `if new_item:` guard, reruns to see the empty-submit test fail, and restores it, so the pull request can show that the tests catch a regression. Because the workflow's matrix is built with `find . -name tests -type d`, the new directory joins it with no change to the workflow file.

## Start it with one API call

`POST https://api.phoenix.so/o/{org}/sessions`

```json
{
  "agent": "claude",
  "input": "Write integration tests for the todo app in this Reflex repository. Read todo/todo/todo.py, todo/rxconfig.py, todo/requirements.txt, counter/tests/test_counter.py, form-designer/tests/conftest.py, form-designer/tests/test_login.py, form-designer/requirements-dev.txt and .github/workflows/app_harness.yml first. Add todo/tests/conftest.py with a session-scoped AppHarness fixture for the app, and todo/tests/test_todo.py using the pytest-playwright page fixture: the starting items render in order, adding an item through the form appends it and clears the input, finishing an item removes exactly that item, and submitting an empty input adds nothing. Wait with Playwright's expect, not sleeps. Add todo/requirements-dev.txt matching the form designer's. Create a virtual environment, install reflex and both requirement files, install the Chromium shell with Playwright, run reflex init and pytest tests -vv headless, then remove the empty-input guard in the app to show a failing test and restore it. Open a pull request with the commands and their output.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
```

1. **Get an API key.** In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
2. **Send the request.** POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
3. **Read what came back.** Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

## What comes back

The pull request adds `todo/tests/conftest.py`, `todo/tests/test_todo.py` and `todo/requirements-dev.txt` and changes nothing under `todo/todo/`. Its description quotes the `pytest tests -vv` output with the four tests, the failing run with the guard removed and the passing run after it is restored, and the `reflex init` and `playwright install` output, so a reviewer sees the environment the tests ran in. The workflow file has no diff, and the description shows the matrix line the next CI run will add for `todo`.

A reviewer checks that the locators are by role and placeholder rather than by generated class names, that the harness fixture is session-scoped so the app compiles once, and that the empty-submit test asserts on the list and not only on the input. After merging, the app-harness workflow runs the new directory on every push and pull request.

## Run it on every nightly schedule

An automation in the Project that holds the repository takes a `schedule` trigger with `cron` set to a night-time hour. Its runs are read-only, so the saved prompt asks the run to find the example apps with no `tests` directory, write the tests in a scratch copy, run them and answer with the ones that pass and the diff that adds them, which the key's session read or the console shows in the morning. A run that should open the pull request is started from the team's own server with this body and a `request_id` that is a UUID v5 of the app's directory name, so a retried start returns the run already started.

A repository with several untested apps gets one such run per app, each with its own `input` naming the directory.

## Why run it on Rush?

A browser test is only evidence once it has run against the compiled app in a real browser, and that takes a toolchain: Python, the frontend bundler `reflex init` fetches, Chromium and the harness that starts both servers. The agent builds all of it in its own sandbox, runs the tests headless as CI will, makes them fail on purpose and keeps every command and its output in the run's record, so the pull request shows green and red rather than claiming them.

The tests follow the repository's own patterns, the session-scoped harness from the form designer, the requirement file the workflow insists on and the headless flag it sets, so they run in the existing matrix with no workflow change. Ten apps in the repository get ten pull requests, each with tests written against that app's state and handlers.

## Frequently asked questions

### Why Playwright rather than Selenium?

Both are in use in the repository: the counter test uses the Selenium driver from `AppHarness.frontend()`, and the form designer uses pytest-playwright. The workflow installs the Chromium shell through Playwright, and its `expect` waits for the list to change without hand-written polling, so the new tests follow the form designer.

### How long do the tests take?

The first run pays for `reflex init` and the frontend build, which `AppHarness` does once per session because the fixture is session-scoped; the four tests then share one app and one browser. The pull request quotes the `pytest` timing from the sandbox run.

### Will CI pick the new tests up?

Yes, with no workflow change. `app_harness.yml` lists every directory named `tests` in the repository to build its matrix and fails any app without a `requirements-dev.txt`, which is why the pull request adds that file alongside the tests.

### What if the app's state changes later?

The tests read the page by role, text and placeholder, so a new colour or layout does not touch them, while a renamed button, a changed starting list or a dropped guard does, which is what they are for. A deliberate change updates one assertion in `test_todo.py`.

## More Reflex jobs

- [Write Pytest Tests for a Reflex App](https://getrush.ai/run/write-pytest-tests/reflex/)
- [Run a Long Task in the Background in a Reflex App](https://getrush.ai/run/add-background-tasks/reflex/)
- [Add Chat Export to a Reflex Chat App](https://getrush.ai/run/add-chat-export/reflex/)

## The same job on other stacks

- [Write Integration Tests for a Sequelize and Express API](https://getrush.ai/run/write-integration-tests/sequelize/)
- [Write Integration Tests for a Socket.IO Chat Server](https://getrush.ai/run/write-integration-tests/socket-io/)
- [Write Integration Tests for an ASP.NET Core API](https://getrush.ai/run/write-integration-tests/aspnet-core/)

## Keep reading

- [How a Rush run works on your repository, from request to pull request](https://getrush.ai/cloud/)
- [Compare cloud agents, agent sandboxes and AI code review](https://getrush.ai/compare/)
- [Rush pricing: Free, Pro and Max](https://getrush.ai/pricing/)
