What the agent does
The agent checks out the pull request branch in its own sandbox and reads how the repository is wired before it looks at the diff. In a repository laid out like fluxcd/flux2-kustomize-helm-example, clusters/staging and clusters/production each hold three Flux objects: an ArtifactGenerator in artifacts.yaml that copies infrastructure/** and apps/base/** plus that cluster's overlay into two artifacts, and in infrastructure.yaml and apps.yaml the Kustomization objects infra-controllers, infra-configs and apps, chained with dependsOn, each with interval: 1h, retryInterval: 2m, timeout: 5m, prune: true and, on two of them, wait: true. The production copy of infra-configs carries a JSON patch that swaps the Let's Encrypt staging server in infrastructure/configs/cluster-issuers.yaml for the production one. Knowing that chain is what lets the review say which cluster a change reaches and in what order.
Then it runs what the repository's own CI runs. scripts/validate.sh installs nothing itself, so the agent fetches the Flux CLI, the flux-schema plugin and kustomize, and the script validates every YAML file with the ecosystem catalog named in .fluxschema.yml, skipping Secret:/sops, and builds each kustomization.yaml with --load-restrictor=LoadRestrictionsNone before validating the output. It repeats the second half of .github/workflows/test.yaml by hand: flux migrate -f . --yes followed by git diff --stat, which fails the check when a manifest still uses a deprecated API version. Both runs go into the review with their output.
The part CI does not do is the rendered diff. The agent builds apps/staging, apps/production, infrastructure/controllers and infrastructure/configs on the base branch and on the pull request branch and diffs the rendered objects, so a one-line change to apps/base/podinfo/release.yaml shows up as a change to both clusters, while an edit to apps/production/podinfo-values.yaml shows up only in production. On that output it checks the things a reviewer checks by eye: a HelmRelease whose chart.spec.version range admits a pre-release, which staging allows with >=1.0.0-alpha and production does not with >=1.0.0; an OCIRepository whose ref.semver widens, as >= 1.8.0-rc.0 on the Envoy Gateway chart does; a prune or wait flag flipped; a dependsOn removed so apps could reconcile before infra-configs has created the Gateway its HTTPRoute points at; an image tag such as the redis 8.6.2 pinned in the podinfo values moved to a floating tag; and a ClusterIssuer or GatewayClass changed in a base file without the matching cluster patch.
When the prompt asks for it, the agent also runs the e2e.yaml job's steps in the sandbox: it installs Docker with sudo apt-get install -y docker.io, starts dockerd, creates a kind cluster, runs flux install --components-extra source-watcher, points a GitRepository at the pull request branch with --ignore-paths="clusters/**/flux-system/", waits for infra-controllers and apps to become ready and for helmrelease/podinfo in the podinfo namespace, then port-forwards the Envoy service and sends a request with Host: podinfo.staging. It posts one review on the pull request that cites a file and line for every finding, and it pushes no commits.
Start it with one API call
POST https://api.phoenix.so/o/{org}/sessions with this body:
{
"agent": "claude",
"input": "Review this pull request to our Flux repository. Read clusters/, apps/ and infrastructure/ first so you know which Kustomization reaches which cluster and in what order. Run scripts/validate.sh with the Flux CLI, the flux-schema plugin and kustomize installed, then run `flux migrate -f . --yes` and report whether `git diff --stat` is empty. Build every overlay with kustomize on the base branch and on this branch and diff the rendered output per cluster. Flag any HelmRelease or OCIRepository version range that admits a pre-release in production, any prune, wait or dependsOn change, any floating image tag, and any base change that lacks its production patch. Post one review on the pull request citing file and line for each finding, with the validate and migrate output quoted. Do not push commits and do not change any cluster.",
"resources": [
{
"url": "https://github.com/your-org/your-app/pull/42"
}
]
}- Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
- Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
- Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.
What comes back
What comes back is one review on the pull request. It opens with the two checks the repository already defines, the validate.sh summary line with its valid and skipped counts and the flux migrate result, then lists findings in the order the clusters would apply them: controllers, configs, apps. Each finding names the file and line in the pull request, says which cluster's rendered output changed because of it, and quotes the rendered lines before and after. A pull request that only touches apps/staging gets a short review that says production is untouched.
The session keeps the record behind the review: every kustomize build, the full flux-schema output, the rendered diffs per overlay and, when the kind steps ran, the kubectl wait and flux tree ks flux-system output. GET /o/{org}/sessions/{id}/messages returns that transcript, so a reviewer who disagrees with a finding can see exactly which rendered object it came from.
Before merging, a reviewer checks that a finding about a version range is about the production overlay and not the staging one, that the review did not mistake the production JSON patch on infra-configs for a drift, and that the rendered diff covers every overlay the pull request touches. A follow-up message to the same session, such as asking for the diff of one object in full, continues from the same checkout.
Run it on every pull request into main
An automation in the Project that holds the Flux repository, with this prompt and a github trigger on pull_request.opened and pull_request.synchronize and branch_patterns set to main, reviews every push to a pull request. The trigger adds the pull request after the prompt and fills nothing else in. Automation runs are read-only, which is all a review needs: they can read the repository and comment, not push. Only pull requests by an owner, member or collaborator fire it, so a contributor's fork gets its review when a maintainer sends this body from their own server with the pull request URL in resources.
A platform team with one repository per cluster fleet adds the same automation to each Project. Set max_runs_per_hour to the pace of the repository: a run per push is the point, but a bot that rebases twenty branches at once would otherwise queue twenty reviews.
The use case this job follows, with its automation and trigger: Review every pull request.
Why run it on Rush?
Reviewing a Flux change by eye means holding the dependsOn chain, the overlay patches and the chart's version range in your head at once, and the mistake that gets through is the one nobody rendered. In a Rush sandbox the agent installs the Flux CLI, flux-schema, kustomize and, when asked, Docker and kind, renders both branches and diffs them, and none of that touches a laptop or a cluster.
The review carries its evidence with it: the validate and migrate output, the rendered diff per cluster, and the transcript of every command. Several pull requests get reviewed at the same time in separate sandboxes, each on its own checkout, so a Monday batch of dependency bumps to OCIRepository ranges does not wait on one person.
Frequently asked questions
Does it apply anything to my cluster?
No. The review renders manifests with kustomize build and validates them with flux-schema; it never talks to your cluster. When the prompt asks for the end-to-end step, the agent creates a throwaway kind cluster inside its own sandbox, which it has to install Docker for first, and that cluster is gone with the sandbox.
What about SOPS-encrypted secrets?
The repository's .fluxschema.yml already tells flux-schema to skip the /sops path on Secret objects, and the agent keeps that setting. It has no decryption key, so an encrypted secret appears in the rendered diff as changed ciphertext and nothing more; the review says the secret changed and does not guess at its contents.
Will it catch a chart version range that pulls a pre-release into production?
Yes. It reads each HelmRelease chart.spec.version and each OCIRepository ref.semver in the rendered production output and flags a range that admits a pre-release, the way >=1.0.0-alpha would. The staging overlay in the example repository allows pre-releases on purpose, and the review says so rather than flagging it.
Can it review a pull request from a fork?
A github trigger never fires for a fork, so the automation will not pick it up. A maintainer starts the review by sending this body with the pull request URL in resources; the agent checks out the pull request's branch through your GitHub App installation and posts the review the same way.