# Fail CI on Sphinx Warnings and Broken Links for a Python Library

Yes. An agent adds a CI job that builds your Sphinx docs with warnings as errors and checks links, pinned and scoped like your other workflows.

[Run this on Rush](https://cloud.getrush.ai/run/add-docs-build-check/python/). Opens the console with this prompt ready to send, in a Project that has your repository.

## What the agent does

The agent starts with what the repository already does for its documentation and its CI, because the new job must match both. In a library like psf/requests, `docs/` holds a Sphinx project with `conf.py` enabling `autodoc`, `intersphinx`, `todo` and `viewcode`, fifteen `.rst` files under `user/`, `dev/` and `community/`, a `Makefile` whose `html` target runs `sphinx-build`, and a `requirements.txt` that pins `Sphinx==7.2.6` for Read the Docs, which `.readthedocs.yaml` builds with the `dirhtml` builder on Python 3.12 after installing the package itself. The top-level `Makefile` has a `docs` target that runs `cd docs && make html`. Under `.github/workflows` there are eight workflows: `run-tests.yml` with a matrix of eight Python versions on three operating systems, `lint.yml` running pre-commit, `typecheck.yml` running pyright, `codeql-analysis.yml`, `publish.yml`, `zizmor.yml`, and two issue-management workflows. None of them builds the docs, so a broken cross-reference or an `autodoc` import failure is found on Read the Docs after merge, if anyone looks.

It then reads the conventions in those workflows so the new one is indistinguishable from them: every `uses:` is pinned to a full commit SHA with the version in a trailing comment (`actions/checkout@9c091bb… # v7.0.0`, `actions/setup-python@5fda3b9… # v7.0.0`), each job sets `permissions: contents: read` at the top, uses `persist-credentials: false` on checkout, and has `timeout-minutes: 10`. The pre-commit configuration in `.pre-commit-config.yaml` excludes `docs/` from Ruff, which tells the agent that the docs tree is deliberately outside the lint job and needs its own.

The agent writes `.github/workflows/docs.yml`, triggered on `push` and `pull_request` like `lint.yml`, with one job that checks out with the same pinned SHA and `persist-credentials: false`, sets up Python 3.12 to match `.readthedocs.yaml`, installs the package with `python -m pip install .` so `autodoc` can import `requests`, installs `-r docs/requirements.txt` so the Sphinx version is the one Read the Docs uses, and runs `sphinx-build -W --keep-going -b dirhtml docs docs/_build/dirhtml`, where `-W` turns warnings into a failing exit code and `--keep-going` reports all of them instead of the first. A second step runs `sphinx-build -b linkcheck` over the same tree, with `linkcheck_ignore` entries added to `docs/conf.py` for the hosts the agent finds rate-limiting or sign-in-only when it runs the checker itself. It also adds a `docs` entry to the top-level `Makefile` only if the existing `docs` target does not already match, which here it does, so the Makefile is left alone.

Before opening the pull request the agent runs exactly those commands in its sandbox on Python 3.12: install, `sphinx-build -W --keep-going`, `linkcheck`. If the strict build fails on existing warnings, as a docs tree that has never been built with `-W` often does, the agent fixes each one in the `.rst` or docstring it points at, in a separate commit from the workflow, and quotes the warning and the fix; it does not loosen `-W` to get green. It validates the workflow with `actionlint` and with `zizmor`, since the repository runs zizmor on its workflows and the new file must pass the same audit, then pushes, and the new job runs on the pull request alongside the existing eight.

## Start it with one API call

`POST https://api.phoenix.so/o/{org}/sessions`

```json
{
  "agent": "claude",
  "input": "Add a GitHub Actions job that builds this Python library's Sphinx documentation with warnings as errors and checks its links. Read every workflow under .github/workflows, .readthedocs.yaml, docs/conf.py, docs/requirements.txt, docs/Makefile and the top-level Makefile first, and match the existing workflows' conventions: SHA-pinned actions with version comments, contents: read permissions, persist-credentials false, a job timeout. Install the package and the pinned Sphinx version, run sphinx-build with -W --keep-going using the same builder Read the Docs uses, then run the linkcheck builder. Run both in the sandbox first; fix any existing warning at its source in a separate commit rather than relaxing the flags, and add linkcheck_ignore only for hosts that block automated checks. Validate the workflow with actionlint and zizmor and open a pull request.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
```

1. **Get an API key.** In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
2. **Send the request.** POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
3. **Read what came back.** Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

## What comes back

What comes back is a pull request on its own branch that adds `.github/workflows/docs.yml` and, when the strict build found warnings, a second commit touching only the `.rst` files or docstrings that caused them, plus any `linkcheck_ignore` lines in `docs/conf.py`. The eight existing workflows, `.readthedocs.yaml` and `docs/requirements.txt` are unchanged, so Read the Docs keeps building exactly as before.

The description quotes the `sphinx-build -W --keep-going` output and the `linkcheck` summary from the sandbox, lists each warning fixed with its file and line, explains each ignored host in a sentence, and shows the `actionlint` and `zizmor` results. The new job's status appears on the pull request itself, which is the first real run of the check.

Before merging, a reviewer scans the ignore list, since an over-broad pattern hides real dead links, and confirms the Python version and builder match `.readthedocs.yaml` so the job and the published docs cannot disagree. The transcript at `GET /o/{org}/sessions/{id}/messages` holds every build log.

## Run it on every terminal hand-off

A maintainer who just found a broken cross-reference on the published docs starts this from the terminal with this body and the repository in `resources`, then goes back to the issue queue. The run reads the real workflows and the real Read the Docs configuration, so the job it adds is pinned, scoped and versioned the way the others are, and it has already run the strict build once before anyone reviews it.

Send a `request_id`, a UUID, in the body so a retried start returns the run it already began. When the turn ends, `GET /o/{org}/sessions/{id}` names the pull request. The same body sent for each library you maintain adds the same check to each, every run reading that repository's own docs configuration in its own sandbox.

## Why run it on Rush?

The workflow is twenty lines, but the first strict build of a mature docs tree is where the work is: a handful of warnings that have been scrolling past for years, an `autodoc` import that needs the package installed, a link checker that fails on hosts that block bots. Done by hand, that is an afternoon of re-running builds; done loosely, `-W` gets dropped to make the job green and the check means nothing. The agent runs the strict build first, fixes causes rather than flags, and matches the repository's pinning and permission conventions so the new file passes its own zizmor audit.

On Rush the installs and builds happen in the agent's own sandbox on the Python version the docs are published with, and every build log stays in the transcript beside the diff. A maintainer reads a pull request whose claims they can check line by line, and the same job lands on the next library with one more request.

## Frequently asked questions

### What if the strict build already fails?

The agent fixes the warnings at their source, in the `.rst` or the docstring Sphinx points at, in a commit separate from the workflow so the two are reviewable apart, and quotes each warning in the description. It does not remove `-W` or add `suppress_warnings` to get a green run; if a warning cannot be fixed without a larger change, the pull request says which and why.

### Will linkcheck make CI flaky?

It can, which is why the agent runs it in the sandbox first and adds `linkcheck_ignore` patterns for hosts that return errors to automated requests, each with a comment. If you prefer, say so in the prompt and the agent puts `linkcheck` in a separate job with `continue-on-error: true`, so dead links are reported without blocking the merge.

### Does it change how Read the Docs builds?

No. `.readthedocs.yaml` and `docs/requirements.txt` are left alone; the new job reads them to use the same Python version, Sphinx version and builder, so a passing job means the published build will pass too. The job is a gate on pull requests, not a replacement for the hosted build.

### Why match the SHA pinning and permissions so carefully?

Because the repository runs `zizmor` over its workflows, and a new file with a floating tag or missing `persist-credentials: false` would fail that audit on the same pull request. The agent copies the pinned SHAs and version comments from `lint.yml`, sets `permissions: contents: read` and a timeout, and runs `zizmor` and `actionlint` locally before pushing.

## More Python jobs

- [Add the Latest Python Release to a Python Library's Support Matrix](https://getrush.ai/run/add-latest-python-support/python/)
- [Add OpenSSF Scorecard to a Python Project](https://getrush.ai/run/add-openssf-scorecard/python/)
- [Add pip-audit to a Python Project's CI](https://getrush.ai/run/add-pip-audit-to-ci/python/)

## The same job on other stacks

- [Add a Docs Build Check for a Sphinx Site on Every Pull Request](https://getrush.ai/run/add-docs-build-check/sphinx/)
- [Add a Build Check for a VitePress Docs Site on Every Pull Request](https://getrush.ai/run/add-docs-build-check/vitepress/)
- [Add a Build Check for a Docusaurus Site on Every Pull Request](https://getrush.ai/run/add-docs-build-check/docusaurus/)

## Keep reading

- [How a Rush run works on your repository, from request to pull request](https://getrush.ai/cloud/)
- [Compare cloud agents, agent sandboxes and AI code review](https://getrush.ai/compare/)
- [Rush pricing: Free, Pro and Max](https://getrush.ai/pricing/)
