# Add Link Checking to a Jekyll Site with html-proofer

Yes. An agent adds html-proofer to your Jekyll CI so pull requests check internal links and a weekly job checks external ones, fixes what it finds, and opens a PR.

[Run this on Rush](https://cloud.getrush.ai/run/add-link-checking/jekyll/). Opens the console with this prompt ready to send, in a Project that has your repository.

## What the agent does

The agent reads how the site is built before it checks what the build produces. In a site laid out like daattali/beautiful-jekyll, the `Gemfile` is one line, `gemspec`, plus the Windows time-zone gems, and `beautiful-jekyll-theme.gemspec` at version 6.0.1 depends on `jekyll >= 3.9.3`, `jekyll-paginate`, `jekyll-sitemap`, `kramdown` and `webrick`, with `appraisal` as a development dependency and an `Appraisals` file that pins Jekyll 3.9.4 and 4.3.3. `Gemfile.lock` is gitignored. `_config.yml` sets `permalink: /:year-:month-:day-:title/`, `paginate: 5`, the two plugins, a `navbar-links` map with three external addresses, `social-network-links` for seven networks, and an `exclude` list that keeps `CHANGELOG.md` and `README.md` out of the build. The content is two posts under `_posts/`, `aboutme.md`, `index.html`, `tags.html`, `feed.xml` and a `404.html` whose image comes from `assets/img/`.

`.github/workflows/ci.yml` runs on push and pull request: Ruby 3.3, `bundle install && bundle exec appraisal install`, `actions/configure-pages` to learn the `base_path`, a `_config_ci.yml` written with that `baseurl`, then `bundle exec appraisal jekyll build --future --config _config_ci.yml,_config.yml`, which builds the site once per Appraisal gemfile, and an upload of the Pages artifact. Nothing reads `_site` after the build. The agent counts what a checker would see: 81 distinct external URLs across the includes, layouts, posts and config, among them `ghbtns.com` badges, `raw.githubusercontent.com`, Wikipedia, a Twitter intent link and the demo site's own images, plus the internal links the navbar, pagination and tag pages generate under the Pages `baseurl`.

It adds `html-proofer` to the `Gemfile` in a `test` group rather than to the gemspec, so sites that consume the theme as a gem do not inherit it, and writes two jobs. The first runs in `ci.yml` after the build: `bundle exec htmlproofer ./_site --disable-external` with `--swap-urls` mapping the `baseurl` prefix back to `/`, since the CI build writes links under the Pages project path and html-proofer resolves them against `_site`, and `--ignore-files` for the `404.html` the layout renders without a page context. The second is `.github/workflows/links.yml`, weekly and on `workflow_dispatch`, which builds once with the Jekyll 4 gemfile and runs html-proofer with external checks on, a `--cache` of thirty days for external results stored under `tmp/.htmlproofer` and kept with `actions/cache`, `--ignore-status-codes` for the 999 that LinkedIn answers bots, and `--ignore-urls` for the `ghbtns.com` iframes, the Twitter intent and the `example.com` email address in `_config.yml`.

Before writing either job the agent installs Ruby 3.3 in its sandbox, runs the same `appraisal` build, and runs both html-proofer commands against `_site`. It fixes what is this repository's to fix, a relative link in a post or an image path in a layout, and leaves the ignore list to the hosts that answer a bot differently from a browser, with the reason in a comment beside each pattern.

## Start it with one API call

`POST https://api.phoenix.so/o/{org}/sessions`

```json
{
  "agent": "claude",
  "input": "Add link checking to this Jekyll site with html-proofer. Read Gemfile, beautiful-jekyll-theme.gemspec, Appraisals, _config.yml, .github/workflows/ci.yml, every file under _includes and _layouts, _posts, aboutme.md, index.html and 404.html first, and count the external URLs. Add html-proofer to the Gemfile in a test group, not to the gemspec. Add a step to ci.yml after the build that runs htmlproofer on _site with external checks disabled and the Pages baseurl swapped back to /. Add .github/workflows/links.yml that runs weekly and on workflow_dispatch, builds with the Jekyll 4 gemfile, runs htmlproofer with external checks on, a thirty day cache kept with actions/cache, ignored status code 999, and ignore patterns for the badge iframes, the Twitter intent link and the example.com address, each with a comment saying why. Install Ruby 3.3 in your sandbox, run the appraisal build and both htmlproofer commands, fix the links in this repository that are broken, and open a pull request with the htmlproofer output before and after. Do not change the gemspec, the theme layouts' structure or _config.yml values other than a broken link.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
```

1. **Get an API key.** In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
2. **Send the request.** POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
3. **Read what came back.** Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

## What comes back

The pull request changes `Gemfile` and `.github/workflows/ci.yml`, adds `.github/workflows/links.yml`, and touches whichever post, include or layout held a link that was actually broken. Its description pastes the html-proofer summary from the sandbox before the fixes and after, the list of ignored patterns with the reason for each, and the count of external URLs the weekly job will check.

The internal check runs on the pull request itself as a step of the existing `build` job, so the reviewer sees it green beside the Pages artifact. The weekly job appears under Actions with a manual trigger for its first run.

Before merging, a reviewer decides whether the `test` group in the `Gemfile` is the right home for the gem, whether the thirty day cache should be shorter, and which of the ignored hosts they would rather see fail than skip.

## Run it on every week

Once `links.yml` is merged, the weekly run reports dead external links, and the question becomes who fixes them. An automation on the Project with a `github` trigger on `workflow_run.failure` and the prompt "if the failed workflow is links, read the htmlproofer output, and for each dead external link find the page that carries it and a replacement or archive URL, then open a pull request" starts one run per failed check. The trigger adds the event after the prompt, and only work by an owner, member or collaborator fires it.

A team that maintains several Jekyll sites sends this body once per repository from its own server, each a `POST /o/{org}/sessions` with that repository in `resources` and a `request_id` that is a UUID v5 of the repository's full name, so a retried call returns the run already started. Each run reads its own build: a site on the `github-pages` gem gets the step after its `jekyll build`, and one with a `baseurl` in `_config.yml` gets that value in the swap.

## Why run it on Rush?

html-proofer is one gem and one command, and the command is wrong in the ways only a run shows: links written under a Pages `baseurl` that the checker cannot resolve, badge iframes and intent links that no host answers properly, an external pass that takes minutes and fails on someone else's outage. The agent builds the site the way CI does in its own sandbox, runs the checker both ways, and writes the ignore list from the output rather than from habit.

The before and after output stays with the run and is pasted into the pull request, so the reviewer sees which links were broken and which hosts were skipped. The same request runs on every Jekyll site a team maintains at once.

## Frequently asked questions

### Why add the gem to the Gemfile and not the gemspec?

The gemspec is what sites install when they use `beautiful-jekyll-theme` as a remote theme or gem, and a runtime or development dependency there reaches every one of them. A `test` group in the `Gemfile` is used only when this repository is built, which is where the check runs.

### Why are external links checked weekly and not on every pull request?

External checks take minutes of network traffic and fail on outages and rate limits that have nothing to do with the change. The pull request step checks internal links and images in seconds with `--disable-external`; the weekly job checks the 81 external addresses with a cache so a host that answered recently is not asked again.

### Why swap the `baseurl`?

`ci.yml` writes `baseurl` from `actions/configure-pages`, so a project site's links begin with its repository path, and html-proofer resolving them against `_site` would report every internal link as missing. `--swap-urls` maps that prefix back to `/` for the check only; the artifact that deploys is untouched.

### Will it fix the sample posts?

Only where a link is actually broken. The two sample posts and `aboutme.md` are the theme's demo content, and the agent changes a target in them only when html-proofer reports it; the description lists each change with the old and new URL so the maintainer can reject one.

## More Jekyll jobs

- [Add llms.txt to a Jekyll Site](https://getrush.ai/run/add-llms-txt/jekyll/)
- [Add Mermaid Diagrams to a Jekyll Blog on GitHub Pages](https://getrush.ai/run/add-mermaid-diagrams/jekyll/)
- [Add Open Graph and Twitter Card Tags to a Jekyll Blog on GitHub Pages](https://getrush.ai/run/add-open-graph-tags/jekyll/)

## The same job on other stacks

- [Add External Link Checking to an mdBook](https://getrush.ai/run/add-link-checking/mdbook/)
- [Add Link Checking to an MkDocs Site](https://getrush.ai/run/add-link-checking/mkdocs/)
- [Add Link Checking to a Pelican Blog](https://getrush.ai/run/add-link-checking/pelican/)

## Keep reading

- [How a Rush run works on your repository, from request to pull request](https://getrush.ai/cloud/)
- [Compare cloud agents, agent sandboxes and AI code review](https://getrush.ai/compare/)
- [Rush pricing: Free, Pro and Max](https://getrush.ai/pricing/)
