What the agent does
The agent reads the pull request and the theme's own rules before it judges a template. In a theme like janraasch/hugo-bearblog, the layouts are a handful of files: layouts/_default/baseof.html, single.html and list.html, index.html, 404.html and the partials under layouts/partials/, with style.html holding the whole stylesheet inline and custom_head.html and custom_body.html left empty for a site to override. theme.toml sets min_version = "v0.110.0", so a template function newer than that release breaks a site that pins an older Hugo even if it builds on the newest. .github/workflows/ci.yml builds the example site with hugo --minify --gc --destination ../public --source ./exampleSite --themesDir ../.. on hugo-version: latest and deploys it to GitHub Pages on push, and that is the only check; package.json has npm run beautify, which runs js-beautify over layouts/**/*.html in place with the settings in .jsbeautifyrc, and nothing in CI runs it. exampleSite/hugo.toml disables the taxonomy kind, sets permalinks for blog and tags, and carries the [params] the templates read, description, favicon, images and title, with hideMadeWithLine, enablePostNavigator and dateFormat shown as commented-out examples.
The pull request's branch is what resources clones, so the agent reads its diff and the files it touches. For each changed template it asks what reads the value: a new .Params key needs a documented default in the README and the example config, a change in seo_tags.html reaches every page's head, and a change in style.html reaches the dark scheme through the prefers-color-scheme block as well as the light one. It checks the Hugo documentation for every function or method the diff introduces and notes the release each arrived in against min_version.
Then it builds twice. In its own sandbox it downloads the current Hugo release, hugo_0.169.0_linux-amd64.tar.gz, builds the example site on the base commit and on the head with CI's exact command into two directories, and runs diff -r between them, so the review lists every generated page whose HTML changed and quotes the changed lines, which is the fastest way to see a template edit that leaked into pages it was not meant for. It also downloads the v0.110.0 release named in theme.toml and builds the head with it, so a function newer than the floor shows up as a build error rather than a bug report from a pinned site. It serves the head build and opens the changed pages in its own browser in the light scheme and with the dark scheme emulated, and takes a screenshot of each.
Last it runs npm ci and npm run beautify on the head and reads git diff --stat, so a template the formatter would rewrite is called out with the lines it would change. The review it posts has a verdict, one finding per file with the line, the list of generated pages that changed, the two Hugo versions it built with, the formatter result and the screenshots, and it pushes nothing. A pull request from a fork is read the same way; only the review comment differs in who can see the sandbox record.
Start it with one API call
POST https://api.phoenix.so/o/{org}/sessions with this body:
{
"agent": "claude",
"input": "Review the pull request this run was started for, on a Hugo theme. Read the diff and every layout, partial and config it touches, plus theme.toml, exampleSite/hugo.toml, package.json, .jsbeautifyrc, the README and the CI workflow. Download the current Hugo release and build the example site with the command the workflow uses on the base commit and on the head, diff the two output directories and list every generated page whose HTML changed with the changed lines. Download the Hugo release named by min_version in theme.toml and build the head with it too. Open the changed pages in a browser in the light and the dark scheme. Run npm run beautify on the head and report whether it would change any file. Check that any new parameter is documented in the README and the example config. Post one review with a verdict, findings with file and line, the build results for both versions, the formatter result and screenshots. Push no commits. If no pull request follows this prompt, stop and say so.",
"resources": [
{
"url": "https://github.com/your-org/your-app/pull/42"
}
]
}- Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
- Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
- Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.
What comes back
The result is a review on the pull request: a verdict, findings each tied to a file and line, a table of generated pages whose HTML differs between base and head with the lines that changed, the build output for the current Hugo and for the min_version release, the beautify diff stat, and the screenshots. The branch is untouched, and the full build logs and the two output directories stay in the run's record for anyone who wants more than the summary.
A maintainer reads the table first, since it is the complete list of what the change does to rendered pages, then the findings. A pull request that adds a parameter without a README line, or that builds on the newest Hugo but not on the floor, gets that as a finding with the exact function that needs the newer release, and the author decides whether to raise min_version or rewrite.
Run it on every pull request opened
An automation on the Project that holds the theme takes a github trigger on pull_request.opened and pull_request.synchronize with this prompt, so every pull request and every push to one gets the same two builds and the same diff. The trigger adds the pull request after the prompt and fills nothing else in, its runs are read-only, which is all a review needs, and it fires only for pull requests from an owner, member or collaborator; a contributor's fork gets its review when a maintainer sends this body from their own server with the pull request URL in resources.
A theme author who maintains several themes adds the same automation to each Project, with the cap on runs per hour set for the pace of contributions.
The use case this job follows, with its automation and trigger: Review every pull request.
Why run it on Rush?
A theme change is judged by what it renders, and a diff of templates does not say that: a one-line edit to a partial can change every page, and a function that works on today's Hugo can fail on the one a site pinned two years ago. The agent builds the example site twice with two Hugo releases in its own sandbox, diffs the generated HTML, opens the pages in both colour schemes and keeps the whole record, so the review states what changed on the page rather than what the author meant.
It also checks the rules the repository keeps by hand, the formatter nobody runs in CI and the parameter documentation in the README, on every pull request and not only when a maintainer has time. Several themes with several example sites get the same review, each built with that theme's own floor version and config.
Frequently asked questions
Why build with two Hugo versions?
theme.toml promises the theme works from min_version on, and sites pin Hugo in their deploy workflow for years. A template function added after the floor builds fine on the newest release and fails on the pinned one, so the review builds with both and names the function when they disagree.
What does the HTML diff catch that the template diff does not?
The pages a change actually reaches. A partial edited for the post page may also render on the list, the 404 and the home page, and a seo_tags.html change touches every head. Diffing the two public directories lists each affected page with the changed lines, which the template diff cannot show.
Does the review change the branch?
No. The run reads the branch, builds it and posts a review; it pushes nothing, and when the run was started by an automation it is read-only in any case. A fix the author wants made is a separate run with a prompt that says so.
Will it run on pull requests from forks?
An automation's GitHub trigger fires only for pull requests from an owner, member or collaborator. For a fork, a maintainer sends this request from their own server with the pull request URL as the resource, and the same review is posted.
More Hugo jobs
The same job on other stacks
- Review Pull Requests on an Inertia.js App (Inertia.js)
- Review Pull Requests on a Ktor Server (Ktor)
- Review Pull Requests on a Laravel App (Laravel)