# Add Canary Rollouts to a Flux Repository

Yes. An agent installs Flagger through Flux, turns your Helm-released app into a Canary behind your Gateway, validates every manifest and opens a PR.

[Run this on Rush](https://cloud.getrush.ai/run/add-canary-rollouts/flux/). Opens the console with this prompt ready to send, in a Project that has your repository.

## What the agent does

The agent reads the repository's layout before it adds a controller, because in a Flux monorepo the order things reconcile is written in the files. In a repository like fluxcd/flux2-kustomize-helm-example, `clusters/staging/artifacts.yaml` holds an `ArtifactGenerator` that splits the monorepo into an `infrastructure` artifact from `infrastructure/**` and an `apps` artifact from `apps/base/**` and `apps/staging/**`. `clusters/staging/infrastructure.yaml` defines two Flux `Kustomization`s on that artifact, `infra-controllers` with `wait: true` and `infra-configs` that `dependsOn` it, and `clusters/staging/apps.yaml` defines `apps`, which depends on `infra-configs`. `infrastructure/controllers/` installs cert-manager and Envoy Gateway as `OCIRepository` plus `HelmRelease` pairs, and `infrastructure/configs/gateway.yaml` creates a `GatewayClass` and a `Gateway` named `envoy` in `envoy-gateway-system` with an HTTP listener on port 80. `apps/base/podinfo/release.yaml` is a `HelmRelease` for the podinfo chart with `httpRoute.enabled: true`, `parentRefs` pointing at that gateway and a hostname of `podinfo.local`, which `apps/staging/podinfo-values.yaml` and `apps/production/podinfo-values.yaml` patch to `podinfo.staging` and `podinfo.production`. `.github/workflows/test.yaml` validates every manifest with `fluxcd/flux-schema` and fails on deprecated API versions through `flux migrate`, `scripts/validate.sh` does the same locally, and `e2e.yaml` boots a kind cluster on every push and waits for `infra-controllers`, `apps` and the podinfo `HelmRelease` to become ready.

Flagger goes in beside the other controllers. The agent adds `infrastructure/controllers/flagger.yaml` with a `flagger-system` namespace, an `OCIRepository` for `oci://ghcr.io/fluxcd/charts/flagger` with `semver: "1.x"`, and a `HelmRelease` with `install.crds` and `upgrade.crds` set to `CreateReplace`, as Flagger's own Flux install guide shows, and values of `meshProvider: gatewayapi:v1`, since the chart lists that provider for any Gateway API implementation, and `prometheus.install: true`, which the chart's values offer for environments with no Prometheus yet and points `metricsServer` at the one it installs. A second file installs Flagger's load tester from the `oci://ghcr.io/fluxcd/flagger-manifests` artifact with a Flux `Kustomization` on its `tester` path, targeting the `podinfo` namespace. Both are listed in `infrastructure/controllers/kustomization.yaml`, so the existing `infra-controllers` Kustomization with `wait: true` registers the Canary CRD before `apps` reconciles, which is what the `dependsOn` chain already guarantees.

The app side is one new resource and one changed value. `apps/base/podinfo/canary.yaml` defines a `Canary` whose `targetRef` is the `podinfo` Deployment the chart creates under `releaseName: podinfo`, a `service` on port 9898 with `gatewayRefs` naming the `envoy` Gateway in `envoy-gateway-system` and `hosts` of `podinfo.local`, and an `analysis` with `interval: 1m`, `threshold: 5`, `maxWeight: 50` and `stepWeight: 10`. Flagger creates and owns the `HTTPRoute` for a Canary, so the agent sets `httpRoute.enabled: false` in `release.yaml` and moves the hostname patches: each cluster's `podinfo-values.yaml` keeps its `HelmRelease` patch, and a new `podinfo-canary.yaml` patch per overlay, registered in that overlay's `kustomization.yaml` with a `Canary` target, sets `podinfo.staging` or `podinfo.production` on the Canary's `hosts`. Two `MetricTemplate`s in `flagger-system` query the installed Prometheus for podinfo's own `http_request_duration_seconds` histogram, one for the share of 5xx responses and one for the p99 latency, and the analysis lists them with `thresholdRange` limits and a `load-test` webhook that asks the load tester to run `hey` against the canary's hostname during each step.

Validation is the repository's own. The agent installs the Flux CLI, the `flux-schema` plugin and `kustomize` in its sandbox and runs `scripts/validate.sh`, which builds every overlay with `LoadRestrictionsNone`, validates the output against the ecosystem schema catalog and reports how many resources were valid, and `flux migrate -f . --yes` followed by `git diff --stat` to prove no API version is deprecated. It builds `apps/staging` and `apps/production` with `kustomize build` and reads the Canary each produces to confirm the hostname patch landed. The kind end-to-end run needs Docker, which the sandbox does not ship; the agent installs `docker.io`, starts `dockerd` and runs the steps from `e2e.yaml` when the prompt asks for it, and otherwise notes that the workflow runs on the push of the branch. Pushing the branch starts that workflow in the reader's CI, where the new `HelmRelease`s and the Canary have to become ready for the job to pass.

## Start it with one API call

`POST https://api.phoenix.so/o/{org}/sessions`

```json
{
  "agent": "claude",
  "input": "Add canary rollouts with Flagger to this Flux repository. Read README.md, every file under clusters, infrastructure and apps, .fluxschema.yml, scripts/validate.sh and .github/workflows first. Install Flagger under infrastructure/controllers as an OCIRepository for oci://ghcr.io/fluxcd/charts/flagger at semver 1.x and a HelmRelease with CRDs set to CreateReplace, meshProvider gatewayapi:v1 and the chart's bundled Prometheus, plus Flagger's load tester from oci://ghcr.io/fluxcd/flagger-manifests into the podinfo namespace, and list both in the controllers kustomization. Add a Canary for the podinfo Deployment under apps/base/podinfo with gatewayRefs on the existing envoy Gateway, port 9898, an analysis of interval 1m, threshold 5, maxWeight 50 and stepWeight 10, MetricTemplates on podinfo's http_request_duration_seconds histogram for error share and p99 latency, and a load-test webhook; set httpRoute.enabled to false in the HelmRelease because Flagger owns the route, and add per-cluster patches so the Canary's hosts match the staging and production hostnames. Run scripts/validate.sh, flux migrate -f . --yes with git diff --stat, and kustomize build on both app overlays, and open a pull request quoting the output and explaining what the e2e workflow will check on push.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
```

1. **Get an API key.** In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
2. **Send the request.** POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
3. **Read what came back.** Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

## What comes back

The pull request adds `infrastructure/controllers/flagger.yaml`, a load tester file, `apps/base/podinfo/canary.yaml`, two `MetricTemplate`s and a `podinfo-canary.yaml` patch under `apps/staging` and `apps/production`, and changes `infrastructure/controllers/kustomization.yaml`, `apps/base/podinfo/kustomization.yaml`, `release.yaml` and both overlay `kustomization.yaml` files. Its description quotes the validation script's summary line, the empty `git diff --stat` after `flux migrate`, and the rendered Canary from each overlay with its hostname.

A reviewer checks the dependency chain, that Flagger lands in `infra-controllers` so its CRD exists before `apps` applies the Canary, and the analysis numbers, which decide how fast a bad release gets traffic and how many failed checks roll it back. The end-to-end workflow on the branch is the proof that the whole set reconciles on a fresh cluster; a `HelmRelease` that never turns ready fails that job before merge.

## Run it on every Flux repository with a Helm-released app

A platform team hands this off from the terminal once per repository: one `POST /o/{org}/sessions` with this body and the repository in `resources`, naming the application and its Gateway in `input` when the repository holds more than one. Each run reads its own layout, so a repository that already runs a Prometheus gets `metricsServer` pointed at it rather than a second one, and a repository with no Gateway API gets a note instead of a Canary. A `request_id` that is a UUID v5 of the repository's full name means a retried request returns the run it already started.

The key's `webhook_url` reports each run the key starts with the pull request URL in `result`, so a script that sends one request per cluster repository collects the pull requests without polling.

## Why run it on Rush?

A canary setup is one controller and one resource, but it has to land in the right order and agree with manifests that already exist: the Gateway it attaches to, the chart's own route, the hostname patch per cluster, and the validation the repository already enforces. The agent reads all of it in its own sandbox, runs the repository's validation script, the migration check and the overlay builds, and keeps every command and its output in the run's record.

Progressive delivery is also where numbers matter, and the agent writes them into the Canary where a reviewer can argue with them: the step size, the ceiling, the failure count and the metric thresholds. The pull request names each one and what it does, so the on-call engineer knows what a rollback will look like before the first release goes through it.

## Frequently asked questions

### Why turn off the chart's HTTPRoute?

Flagger creates and manages the `HTTPRoute` for a Canary, splitting weight between the primary and canary Services it also creates. Leaving `httpRoute.enabled: true` in the `HelmRelease` would give the Gateway two routes for one hostname, so the agent disables the chart's and moves the hostname into the Canary's `hosts`.

### Does this work with Envoy Gateway?

Yes. Flagger's `gatewayapi:v1` provider works with any Gateway API implementation, and the repository already installs Envoy Gateway and a `Gateway` named `envoy`, which is what the Canary's `gatewayRefs` names. The load tester sends its requests through that Gateway with the canary's `Host` header.

### Where do the metrics come from?

From the Prometheus the Flagger chart installs when `prometheus.install` is on, which scrapes pods that expose metrics, and from podinfo's own `http_request_duration_seconds` histogram. A repository with a Prometheus already gets `metricsServer` pointed at it instead, and the two `MetricTemplate`s query whichever one is configured.

### What happens when a release fails analysis?

Flagger stops advancing when a check fails, counts failures against `threshold`, and after five failed checks routes all traffic back to the primary and marks the Canary failed. The Deployment keeps the new image, the primary keeps the old one, and the next commit starts a new analysis.

### Can the end-to-end test run in the sandbox?

It needs a kind cluster, which needs Docker, and the sandbox has no Docker installed. Ask for it in `input` and the agent installs `docker.io`, starts `dockerd`, creates the cluster and runs the steps from `e2e.yaml`; otherwise it runs the schema validation and overlay builds and leaves the cluster test to the workflow on push.

## More Flux jobs

- [Add Cosign Signature Verification to Flux Helm Sources](https://getrush.ai/run/add-cosign-verification/flux/)
- [Add Drift Detection to Flux Helm Releases](https://getrush.ai/run/add-drift-detection/flux/)
- [Add Image Update Automation to a Flux Repository](https://getrush.ai/run/add-image-automation/flux/)

## The same job on other stacks

- [Convert a Blue-Green Argo Rollout to a Canary With Analysis in an Argo CD App](https://getrush.ai/run/add-canary-rollouts/argo-cd/)

## More jobs like this: Hand off a long task

- [Add cargo-deny to a Rust CLI's CI](https://getrush.ai/run/add-cargo-deny/rust-cli/)
- [Add cargo-semver-checks to CI for a Rust Library](https://getrush.ai/run/add-cargo-semver-checks-to-ci/rust-library/)

## Keep reading

- [How a Rush run works on your repository, from request to pull request](https://getrush.ai/cloud/)
- [Compare cloud agents, agent sandboxes and AI code review](https://getrush.ai/compare/)
- [Rush pricing: Free, Pro and Max](https://getrush.ai/pricing/)
