# Add Test Timeouts to a pytest Suite

Yes. An agent adds pytest-timeout to your test dependencies and config in a form that works on every CI platform, proves a hung test fails, and opens a PR.

[Run this on Rush](https://cloud.getrush.ai/run/add-test-timeouts/pytest/). Opens the console with this prompt ready to send, in a Project that has your repository.

## What the agent does

The agent starts with how the suite is configured and run, because a timeout plugin is a dependency, a config key and a CI matrix question. In a project laid out like pallets/itsdangerous, `pyproject.toml` keeps the test tools in a `tests` dependency group (`freezegun` and `pytest`, locked in `uv.lock` at 1.5.2 and 8.3.5), runs pytest through tox with `uv-venv-lock-runner` and the flags `-v --tb=short --basetemp={env_tmp_dir}`, and sets `filterwarnings = ["error"]` under `[tool.pytest.ini_options]` so any warning fails a test. `.github/workflows/tests.yaml` runs `uv run --locked tox run -e py3.13` and friends on Ubuntu, plus a Windows row, a Mac row and PyPy. The 37 tests across five files in `tests/test_itsdangerous/` take fractions of a second, and `test_timed.py` freezes the clock with `freezegun` in an autouse fixture.

It then adds the plugin where the repository keeps such things. `pytest-timeout` joins the `tests` group in `pyproject.toml`, `uv lock` updates `uv.lock`, which the `uv-lock` pre-commit hook checks, and `timeout` goes under `[tool.pytest.ini_options]` next to `filterwarnings`, with a value well above the suite's longest test so it only fires for a hang. The tox commands need no change, since pytest loads the plugin by its entry point. Nothing is added to the workflow: `--locked` installs the new pin from the lockfile.

Two things come from reading rather than from the plugin's README. The CI matrix has a Windows row, and the plugin's `signal` method does not exist there, so the agent sets no `timeout_method` and lets the plugin pick `signal` on POSIX and `thread` on Windows, and says in the pull request why the config is silent on it. And `freezegun` patches `time.time` and `time.monotonic` in `test_timed.py`, which could make a naive timer think no time passes; the plugin's alarm is a real `SIGALRM`, which freezegun does not touch, so a hang inside a frozen test still trips. The agent proves that rather than asserting it.

In its sandbox the agent runs `uv sync` and `uv run --locked tox run -e py3.13` to see the suite green with the plugin loaded and the timeout in effect. It then writes a throwaway test that calls `time.sleep` for longer than the limit inside the `FreezeMixin` fixture, runs it once to read the `Failed: Timeout` line with its stack, and deletes it, quoting that output in the description so the reviewer sees the plugin fire under the frozen clock. A `@pytest.mark.timeout` on one test is left as an example only in the description, since no test here needs its own limit.

## Start it with one API call

`POST https://api.phoenix.so/o/{org}/sessions`

```json
{
  "agent": "claude",
  "input": "Add test timeouts to this pytest suite with pytest-timeout. Read pyproject.toml, uv.lock, .pre-commit-config.yaml, .github/workflows/tests.yaml and tests/test_itsdangerous/test_timed.py first. Add pytest-timeout to the tests dependency group, run uv lock, and set a timeout under [tool.pytest.ini_options] well above the slowest test. Do not set timeout_method, because the CI matrix includes Windows; say why in the pull request. Do not change the tox commands or the workflow. Run uv sync and uv run --locked tox run -e py3.13 and quote the result. Then write a temporary test that sleeps past the limit inside the freezegun fixture from test_timed.py, run it to capture the Timeout failure, delete it, and quote that output in the description. Open a pull request.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
```

1. **Get an API key.** In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
2. **Send the request.** POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
3. **Read what came back.** Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

## What comes back

What comes back is a pull request with two files changed, `pyproject.toml` and `uv.lock`: one line in the `tests` group, one `timeout` key in the pytest options, and the lock entries for the plugin. No test file changes and no workflow change. The description quotes the tox run with the plugin listed in pytest's header line and all tests passing, then the `Failed: Timeout` output from the temporary sleeping test under the frozen clock, with the stack pointing at the sleep.

The run's record keeps both pytest runs and the `uv lock` output, so a reviewer can confirm the lockfile change is only the plugin and its own dependencies.

Before merging, a reviewer sets the limit to taste, since the agent picked a number far above the suite's longest test, and decides whether to add `timeout_func_only` so slow fixtures are not counted against a test's own time.

## Run it on every nightly

The timeout earns its keep on the runs nobody watches. A `schedule` trigger with `cron` set to a quiet hour in the trigger's `timezone` starts a run that executes the suite on the latest dependency versions, the way the `tox` config's `update-requirements` environment refreshes `uv.lock`, and opens a pull request only when a test hangs or fails, quoting the `Timeout` line with the test's name and stack.

A hang that used to hold a CI runner until the job limit now ends in seconds with a named test. The run's `needs_review` state, when the suite does not pass, is the signal to read; a green night produces no pull request at all.

## Why run it on Rush?

A hung test is the one failure CI reports worst: the job sits until its own limit, the log ends mid-line, and nobody knows which test it was. Adding the plugin is a one-line change that is easy to get wrong on a mixed matrix, where a `signal` method set by hand breaks every Windows row. In a Rush sandbox the agent reads the matrix, keeps the config portable, and proves the alarm fires under the suite's frozen clock before anyone merges it.

The two pytest runs stay with the run and the pull request quotes them, so the reviewer reads the `Timeout` failure with its stack rather than taking the plugin's behaviour on faith. Across every repository a team maintains, the job runs in parallel, one sandbox and one pull request each.

## Frequently asked questions

### Will freezegun stop the timeout from firing?

No. freezegun patches the clock functions a test reads, but the plugin's default method on Linux and Mac arms a real alarm signal, which keeps running on wall-clock time. The pull request includes the output of a sleeping test under the `FreezeMixin` fixture failing with `Timeout` to show it.

### Why not set timeout_method explicitly?

Because the CI matrix includes Windows, where the `signal` method does not exist and pytest would error on collection. Left unset, the plugin uses `signal` where it can and `thread` on Windows, so one config serves every row.

### Does this slow the suite down?

No. The plugin arms a timer per test and cancels it when the test ends; the suite's runtime is unchanged. The limit only matters when a test does not end.

### Can one slow test have its own limit?

Yes, with `@pytest.mark.timeout(seconds)` on that test, or `timeout_func_only = true` in the config so fixture setup is excluded from the count. Nothing here needs either, so the pull request leaves them as notes in the description.

## More pytest jobs

- [Roll Back the Database After Each pytest Test in a FastAPI App](https://getrush.ai/run/add-transactional-test-fixtures/pytest/)
- [Fix "Fixtures are not meant to be called directly" in a pytest Suite](https://getrush.ai/run/fix-fixtures-are-not-meant-to-be-called-directly/pytest/)
- [Fix pytest Tests That Pass Once and Fail on the Next Run](https://getrush.ai/run/fix-flaky-tests/pytest/)

## The same job on other stacks

- [Set Test, Expect and Global Timeouts in a Playwright Suite](https://getrush.ai/run/add-test-timeouts/playwright/)

## More jobs like this: Write the missing tests

- [Add Unit Tests to a dbt Project](https://getrush.ai/run/add-unit-tests/dbt/)
- [Add Visual Regression Tests With Playwright](https://getrush.ai/run/add-visual-regression-tests/playwright/)

## Keep reading

- [How a Rush run works on your repository, from request to pull request](https://getrush.ai/cloud/)
- [Compare cloud agents, agent sandboxes and AI code review](https://getrush.ai/compare/)
- [Rush pricing: Free, Pro and Max](https://getrush.ai/pricing/)
