# Run a pytest Suite in Parallel With pytest-xdist

Yes. An agent adds pytest-xdist, checks the suite's fixtures and ports are safe per worker, measures the gain in CI's own command, and opens a PR.

[Run this on Rush](https://cloud.getrush.ai/run/add-parallel-tests/pytest/). Opens the console with this prompt ready to send, in a Project that has your repository.

## What the agent does

The agent reads how the suite is run before it adds a worker flag, because parallel tests fail in the places a serial suite never noticed. In a project laid out like pallets/werkzeug, `pyproject.toml` keeps the test tools in a `tests` dependency group (`pytest`, `pytest-timeout`, `ephemeral-port-reserve`, `watchdog`, `cffi`, `cryptography`), runs pytest through tox with `uv-venv-lock-runner` and the flags `-v --tb=short --basetemp={env_tmp_dir}`, and sets `filterwarnings = ["error"]`. `.github/workflows/tests.yaml` runs `uv run --locked --no-default-groups --group dev tox run` across nine matrix rows, including Windows, Mac, a free-threaded `3.15t` and PyPy. The suite has 588 tests in `tests/`, and the slow ones are the dev-server tests: `tests/conftest.py` defines `DevServerClient`, which starts `live_apps/run.py` as a subprocess on a port from `ephemeral_port_reserve.reserve`, logs into the test's `tmp_path`, and is marked `dev_server`; 35 such calls sit in `test_serving.py` and 4 in `test_debug.py`.

That reading answers the question the plugin cannot: is the suite safe to split across processes. The dev-server tests reserve a free port per start and write under the test's own `tmp_path`, so two workers never share a socket or a log file. The unix-socket test uses `tmp_path_factory.mktemp("sock")`, again per test. No test writes a fixed path in the repository, and the reloader tests copy their app into `tmp_path` before touching it. The one thing to watch is `filterwarnings = error` combined with xdist's own `DeprecationWarning` on some Python versions, which the agent checks by running once with the new flag and reading the first failure, if any.

The change is then small. `pytest-xdist` joins the `tests` group in `pyproject.toml`, `uv lock` updates `uv.lock`, which the `uv-lock` pre-commit hook checks, and `-n auto` joins the tox `commands` next to `--basetemp`, which xdist already makes per-worker. Nothing changes in the workflow: `--locked` installs the new pin, and `tox run` carries the flag to every matrix row. The `3.15t` row keeps running: xdist spawns processes, not threads, so it does not depend on the free-threading build's plugin support, and the `commands_pre` that uninstalls `watchdog` on that row is left as it is.

In its sandbox the agent runs `uv sync` and times `uv run --locked --no-default-groups --group dev tox run -e py3.13` before and after the flag, on the same machine, and quotes both wall-clock lines and the worker count pytest printed. It runs the parallel suite three times to see that it is stable, since a test that passes serially and fails once in three under xdist is the signal this change exists to surface. A `dev_server` test that proves flaky is quoted with its name and the agent's reading of why, and `--dist loadgroup` with an `xdist_group` mark is applied to that file only, rather than disabling parallelism for the suite.

## Start it with one API call

`POST https://api.phoenix.so/o/{org}/sessions`

```json
{
  "agent": "claude",
  "input": "Run this pytest suite in parallel with pytest-xdist. Read pyproject.toml, uv.lock, .pre-commit-config.yaml, .github/workflows/tests.yaml, tests/conftest.py and tests/test_serving.py first. Check that fixtures, ports, temporary directories and any module-level state are safe per worker process, and say what you found. Add pytest-xdist to the tests dependency group, run uv lock, and add -n auto to the tox pytest command. Do not change the workflow. Run uv sync, then time tox run -e py3.13 before and after the change on this machine and quote both. Run the parallel suite three times; if a test is flaky under xdist, name it, explain why from its code, and pin it with an xdist_group mark and --dist loadgroup rather than turning parallelism off. Open a pull request with the timings and the stability runs.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
```

1. **Get an API key.** In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
2. **Send the request.** POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
3. **Read what came back.** Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

## What comes back

What comes back is a pull request with two files changed, `pyproject.toml` and `uv.lock`: one line in the `tests` group, one flag in the tox command, and the lock entries for xdist and `execnet`. No test file changes unless a group mark was needed, and no workflow change. The description quotes two timings of the same tox environment on the same machine, serial and with `-n auto`, with the worker count pytest reported, and the three parallel runs all green.

The description also records what the agent checked and found safe: per-test ports from `ephemeral_port_reserve`, per-test `tmp_path` for the dev-server logs and sockets, and no shared files in the repository tree, so a reviewer sees the reasoning rather than a flag.

Before merging, a reviewer decides whether CI should pin a worker count instead of `auto`, since GitHub's runners report more cores than a dev-server test can use well, and whether the Windows row, where process start-up is slowest, wants `-n 2` on its own.

## Run it on every nightly

A parallel suite is where order-dependent tests show up, so a team runs it unattended. A `schedule` trigger with `cron` set to a quiet hour in the trigger's `timezone` starts a run that executes the suite with `-n auto` on the latest dependency versions, the way the tox `update-requirements` environment refreshes `uv.lock`, and opens a pull request only when a test fails, quoting the test's name, the worker it ran on and the stack.

A test that passes alone and fails beside another is reported with both names from pytest's `-v` output, which is the clue that a fixture leaks state. A green night produces no pull request at all; the run's `needs_review` state, when the suite does not pass, is the signal to read.

## Why run it on Rush?

Adding a worker flag is one line, and finding out it is safe is the whole job. A suite that starts servers, writes files and reserves ports has to be read before it is split, and the right time to read it is before the first flaky night in CI. In a Rush sandbox the agent reads the fixtures, runs the suite serially and in parallel on the same machine, and repeats the parallel run to prove it holds.

The record of the run keeps every run and both timings, and the pull request quotes them, so the reviewer reads the actual gain on this suite rather than a plugin's claim. Across every Python repository a team maintains, the same request runs in parallel, one sandbox and one pull request each.

## Frequently asked questions

### How does the agent know the dev-server tests are safe to parallelise?

By reading tests/conftest.py. DevServerClient reserves a free port with ephemeral_port_reserve for every start and writes its log and any unix socket under the test's own tmp_path, so two workers cannot collide. The pull request quotes those lines.

### What if a test is flaky only under xdist?

The agent runs the parallel suite three times and names any test that fails, with its reading of the cause from the test's code. Such a test gets an xdist_group mark so the tests that share state run on one worker, with --dist loadgroup, rather than switching parallelism off for everyone.

### Does this change the CI workflow?

No. The workflow runs tox with --locked, so it installs the new pin from uv.lock, and tox carries the -n auto flag to every matrix row, including the free-threaded and PyPy rows. The only files in the pull request are pyproject.toml and uv.lock.

### Will the suite be faster?

The pull request quotes the serial and parallel wall-clock times on the same machine so the reviewer can see. The gain comes from the dev-server tests, which each wait on a subprocess; the many fast tests gain little, and on a two-core runner the result is smaller than on the sandbox.

## More pytest jobs

- [Add Property-Based Tests with Hypothesis to a pytest Suite](https://getrush.ai/run/add-property-tests/pytest/)
- [Add Random Test Order to a pytest Suite](https://getrush.ai/run/add-random-test-order/pytest/)
- [Add Test Timeouts to a pytest Suite](https://getrush.ai/run/add-test-timeouts/pytest/)

## The same job on other stacks

- [Run a Jest Suite in Parallel Against One MongoDB](https://getrush.ai/run/add-parallel-tests/jest/)
- [Run a Mocha Suite in Parallel With --parallel](https://getrush.ai/run/add-parallel-tests/mocha/)

## More jobs like this: Write the missing tests

- [Add Property-Based Tests With fast-check to a Jest Suite](https://getrush.ai/run/add-property-based-tests/jest/)

## Keep reading

- [How a Rush run works on your repository, from request to pull request](https://getrush.ai/cloud/)
- [Compare cloud agents, agent sandboxes and AI code review](https://getrush.ai/compare/)
- [Rush pricing: Free, Pro and Max](https://getrush.ai/pricing/)
