# Fix Flaky Tests in a React App An agent runs your flaky React or Cypress test with retries off until it fails, fixes the race or shared data behind it, and proves it with 20 clean runs. [Run this on Rush](https://cloud.getrush.ai/run/fix-flaky-tests/react/), in a Project that has your repository. ## What the agent does In a React app like cypress-io/cypress-realworld-app, the first thing the agent notices is that retries are already hiding flakes. cypress.config.ts sets retries: { runMode: 2 }, so a test that fails twice and passes on its third try still reports green in yarn test:headless. To see the real failure rate, the agent runs the named spec with retries turned off, for example yarn cypress run --spec cypress/tests/ui/notifications.spec.ts --config retries=0, and counts failures over repeated runs. The app has three test layers, and the agent works out which one the alert is about. Vitest runs the unit tests in src/__tests__ and src/utils/__tests__ with yarn test:unit:ci. Cypress component tests live beside the components as src/**/*.cy.tsx. End-to-end specs live in cypress/tests/ui and cypress/tests/api and need the full stack: yarn start:ci serves the Vite front end and the Express API from backend/app.ts, on Node 22.20.0 from .nvmrc, with yarn 1.22.22 as the package manager. The agent starts that stack in the sandbox before running any end-to-end spec. Most end-to-end flakes here come from two places, and the agent checks both. Each spec resets data through the db:seed task, which posts to /testData/seed on the API, so a spec that assumes a record a previous spec changed, or that runs before seeding finishes, depends on timing. And the UI specs wait on network aliases: the files under cypress/tests/ui use cy.wait with aliases dozens of times, and a missing or misplaced intercept means the test races the request. The only numeric wait in the repository is a cy.wait(2000) in cypress/support/auth-provider-commands/cognito.ts. Once it has the cause, the agent fixes it in the test or the app: an intercept registered before the action that triggers the request, an assertion that waits on visible state instead of a fixed time, a seed that completes before the visit, or a component that renders before its data arrives. It does not raise retries, add cy.wait with a number or skip the test. Then it runs the spec 20 times with retries=0 and records the after count beside the before count. ## Start it with one API call `POST https://api.prix.dev/o/{org}/sessions` ```json { "agent": "claude", "input": "The test named in the alert below fails intermittently. Install with yarn, start the app with yarn start:ci when it is an end-to-end spec, and run that one spec with retries turned off (--config retries=0) in a loop until it fails. Find why (a missing or late intercept, seed data, ordering, time, randomness or a race), fix the test or the code, run it 20 times with retries=0 to show it now passes every time, and open a pull request with the before and after counts. Never raise retries, add a numeric cy.wait or skip the test.\n\nAlert: ", "resources": [ { "url": "https://github.com/your-org/your-app" } ] } ``` 1. **Get an API key.** In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token. 2. **Send the request.** POST the body in request.json to https://api.prix.dev/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox. 3. **Read what came back.** Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory. ## What comes back The pull request changes the spec and, where the test caught a real rendering or data bug, the React component or API route behind it. The description names the cause, shows the loop that reproduced it with retries turned off and how many runs failed, and the 20 passing runs afterwards. When Cypress saves screenshots or videos of a failure in the sandbox, the agent can return them as files, which the session lists as its artifacts, so a reviewer sees the moment the test broke. A reviewer checks that the test now waits on something real, an intercepted request or a visible element, and that retries in cypress.config.ts stayed where they were. If the flake was in a Vitest unit test instead, the same rule applies: the fix removes the timing or shared state, not the assertion. The commands and their output are in the session transcript. ## Run it on every test that passed on retry Cypress reports which tests needed a retry to pass. Have your CI send each of those to a handler that calls POST /o/{org}/sessions with the repository in resources and the spec and test title in the prompt, with the spec path as the Idempotency-Key so a flake seen in many builds starts one run. When the run finishes, your key's webhook posts the result. Over time this turns the retry setting from a place flakes hide into a list of fixes, one pull request per flaky test. ## Why run it on Rush? Reproducing an end-to-end flake needs the whole app running, a browser, and the same spec run twenty or fifty times, which ties up a laptop and is easy to stop too early. A Rush sandbox has its own browser and runs the Vite front end and the Express API side by side, so the agent can loop the spec as long as needed, and every Cypress run is kept with its output. Several flaky specs can be investigated at once, each in its own sandbox with its own seeded data, so one investigation never resets the database under another. Each returns a pull request with counts you can check against the transcript. ## Frequently asked questions ### Why turn retries off when CI uses them? With runMode set to 2, a test that fails two times out of three still passes, so the real failure rate is invisible. Turning retries off for the investigation is the only way to count failures before and after the fix. The pull request leaves your retry setting as it was. ### Can it run Cypress end-to-end tests in the cloud? Yes. The sandbox is a full machine with a browser, so the agent starts the front end and API with the repository's own scripts and runs cypress run against them. Specs that need outside sign-in providers, such as the Auth0 or Okta specs, need those credentials and are reported as not run without them. ### Does it handle flaky Vitest or Jest tests too? Yes. The same loop works with vitest --run on one file. Unit-test flakes in React are usually timers, unresolved promises or state left in a module, and the fix goes in the test setup or the component. ### Why 20 runs instead of 50? End-to-end specs take much longer per run than unit tests, so 20 is a practical number that still makes a lucky pass unlikely. Change the number in the prompt if your flake is rarer.