All jobs

Add Link Checking to a Pelican Blog

Yes. An agent builds a Pelican site in CI, runs lychee over the generated HTML weekly and on pull requests, and opens a PR with the workflow and a baseline.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

The agent reads how the site is built before it adds a job. In a Pelican blog like getpelican/pelican-blog, pelicanconf.py sets PATH = "content", THEME = "themes/notmyidea", SITEURL = "", DEFAULT_PAGINATION = 5, STATIC_PATHS for images and extra, and a MARKDOWN dict with the admonition, codehilite, extra, meta and toc extensions, so the Markdown extra must be installed. publishconf.py imports it and sets SITEURL = "https://blog.getpelican.com" and DELETE_OUTPUT_DIRECTORY = True. The Makefile's html target runs clean, which deletes inside output/, then pelican content -o output -s pelicanconf.py; publish uses publishconf.py; upload runs rsync to the blog's server, and the README says publishing is make upload. There is no .github/workflows directory, no requirements file and no lockfile, and output is gitignored. The content is reStructuredText under content/news/ and content/pages/, with one Markdown file, namespace-plugins.md, that uses Title:, Date: and Modified: metadata.

The links are the reason to check. The agent counts them in the content: about 130 outbound links, 73 to github.com, 26 to the Pelican docs, and single links to hosts such as gittip.com, crate.io, lesscss.org and jinja.pocoo.org from posts written between 2012 and 2021. Checking the sources directly would flag Pelican's own {filename} and {static} link syntax, so the agent checks the built site instead, with SITEURL empty so that internal links come out as root-relative paths that lychee resolves against the output directory.

The agent writes .github/workflows/links.yml with a schedule trigger for Monday at 07:00 UTC, workflow_dispatch, and pull_request on changes under content/, themes/ or the two config files. The job checks out the repository, sets up Python 3.12 with actions/setup-python, installs pelican[markdown] pinned to 4.12.0, the current release, runs pelican content -o output -s pelicanconf.py, and calls lycheeverse/lychee-action at v2.9.0, the current release, with args of --root-dir "$PWD/output" --no-progress --cache --max-cache-age 1d --accept 100..=103,200..=299,403,429 'output/**/*.html' so absolute links such as /pages/gratitude.html resolve to files and rate-limited hosts do not fail the run, --github-token from the workflow's token so the 73 GitHub links are not throttled, format set to markdown and fail true. A .lycheeignore holds one regular expression per line for localhost, example.com and the mailto addresses the theme prints, and an actions/cache step keeps .lycheecache between runs. A requirements.txt with the two pins makes the install reproducible and gives the Makefile a make links target that builds and runs lychee locally.

Then the agent runs it. In its own sandbox it installs the same versions, builds the site with pelicanconf.py, which succeeds with the stock theme, runs the lychee binary from the v0.24.2 release with the workflow's arguments and reads the report: which links are dead, which redirect, and which hosts answer 403 or 429. It does not edit posts; dead links in a ten-year-old news archive are an editorial call, so the report goes into the pull request as a baseline and the regular expressions in .lycheeignore cover only the addresses that are placeholders by design. The workflow file is checked with actionlint and the README gets a paragraph on the workflow and the make target.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "claude",
  "input": "Add link checking to this Pelican blog. Read pelicanconf.py, publishconf.py, the Makefile, develop_server.sh, the README, .gitignore, the content directory and the theme's templates first. Add .github/workflows/links.yml that runs every Monday morning, on manual dispatch and on pull requests touching content, the theme or the config: set up Python 3.12, install pelican[markdown] pinned to the current release, build the site with pelicanconf.py into output, and run lychee-action over output/**/*.html with the output directory as root-dir, caching, a markdown report, the workflow token for GitHub links, and fail set to true. Add a .lycheeignore for localhost, example.com and mailto links, a requirements.txt with the pins, and a make links target. Install pelican and lychee in the sandbox, build the site, run the check with the same arguments, and put the resulting list of dead and redirected links in the pull request as a baseline without editing any post. Check the workflow with actionlint and add a README paragraph. Open a pull request with the commands and their output.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

The pull request adds .github/workflows/links.yml, .lycheeignore and requirements.txt, and changes the Makefile and the README. Its description quotes the pelican build output and the lychee summary from the sandbox: the number of links found, the ones that failed with their status, the ones that redirected with their new address, and the hosts that answered 403 or 429 and are accepted for that reason. No file under content/ changes.

A reviewer decides what to do with the baseline, fixing links in a follow-up, pointing them at an archive, or adding an expression to .lycheeignore, and whether a failing scheduled run should block anything, since the workflow fails on dead links but nothing deploys from it. After merging, the Links workflow runs on the next Monday and on the next pull request that touches content.

Run it on every Monday morning

An automation in the Project that holds the blog takes a schedule trigger with cron set to Monday at 07:00 in the team's time zone. Its runs are read-only, so the saved prompt asks the run to build the site, run lychee with the workflow's arguments, and answer with the links that died since last week and a replacement for each it can find, which the key's session read or the console shows. A run that should edit the posts and open a pull request is started from the team's own server with this body and a request_id that is a UUID v5 of the ISO week, so a retried start returns the run already started.

A team with several Pelican sites sends one request per repository, each with its own resources entry.

The use case this job follows, with its automation and trigger: Hand off a long task.

Why run it on Rush?

A link checker is only trustworthy once it has run against the real build, with the real theme templates and the real content, and the agent does that in its own sandbox: it installs Pelican and lychee, builds the site, runs the check with the exact arguments the workflow will use and keeps every command and its output in the run's record. The pull request therefore ships with a baseline, not a promise.

The workflow fits the repository as it is: the pelicanconf.py build the Makefile already runs, root-relative links from an empty SITEURL, the output directory that is already gitignored, and a cache so the weekly run asks each host once a day at most. Ten blogs get ten pull requests, each with that site's own baseline.

Frequently asked questions

Why check the built site rather than the source files?

Pelican's {filename} and {static} syntax in reStructuredText and Markdown is not a URL until Pelican resolves it, so a checker reading the sources reports false failures. The built HTML has real paths, and --root-dir tells lychee where an absolute path such as /pages/gratitude.html lives on disk.

Will GitHub rate limits make the check flaky?

The workflow passes the run's own token to lychee, which lifts the anonymous limit for the 73 github.com links, and accepts 429 and 403 answers so a throttled or bot-blocking host does not fail the job. The cache keeps a host's answer for a day, so a rerun does not ask again.

Does the workflow fix dead links?

No. It reports them, in the job summary and as a failed check on a pull request. The agent's run puts the first report in the pull request as a baseline, and fixing or ignoring each link is a separate decision for whoever owns the archive.

Can I run the same check locally?

Yes. make links builds the site into output and runs lychee with the same arguments, using the pins in requirements.txt; install lychee once from its release page or a package manager. .lycheeignore applies locally and in CI alike.

More Pelican jobs

The same job on other stacks

Keep reading