What the agent does
The agent reads how the clusters reconcile before it adds a flag, because the lockdown has to leave the platform team's own Kustomizations working. In a repository like fluxcd/flux2-kustomize-helm-example, clusters/staging/ and clusters/production/ each hold flux-system/ with the bootstrap-generated gotk-components.yaml and gotk-sync.yaml plus a kustomization.yaml that labels everything toolkit.fluxcd.io/tenant: sre-team and patches --concurrent=20 and --requeue-dependency=5s into the kustomize, helm and source controllers; an artifacts.yaml with an ArtifactGenerator that copies infrastructure/** and apps/base/** plus the cluster's overlay into two ExternalArtifacts; infrastructure.yaml with the infra-controllers and infra-configs Kustomizations, the second depending on the first and patching the letsencrypt ClusterIssuer's ACME server per cluster; and apps.yaml with an apps Kustomization depending on infra-configs. All six Kustomizations live in flux-system and none sets spec.serviceAccountName, so they reconcile with the controller's own cluster-admin rights. apps/base/podinfo/ creates the podinfo namespace labelled toolkit.fluxcd.io/tenant: dev-team, a HelmRepository and a HelmRelease for podinfo with an httpRoute on the envoy Gateway in envoy-gateway-system; infrastructure/controllers/ installs cert-manager and Envoy Gateway from OCI charts into namespaces labelled sre-team, and infrastructure/configs/ holds the Gateway, GatewayClass and ClusterIssuer. scripts/validate.sh validates every manifest and overlay with flux-schema using .fluxschema.yml, test.yaml runs it on pull requests and fails on deprecated API versions through flux migrate, and e2e.yaml boots a kind cluster, installs Flux and waits for infra-controllers, apps and the podinfo HelmRelease to become ready.
The labels already tell the story the RBAC should enforce: sre-team owns the Flux controllers and the infrastructure, dev-team owns podinfo, and today either team's manifest can reference a Flux object in any namespace and runs as cluster-admin. Flux's multi-tenancy lockdown is three controller flags, and the agent adds them to the existing patch in each clusters/<cluster>/flux-system/kustomization.yaml rather than in a second patch: --no-cross-namespace-refs=true on the kustomize, helm, notification, image-reflector and image-automation controllers and on source-watcher, which the e2e workflow installs with --components-extra, so a sourceRef or dependsOn must point inside the resource's own namespace; --no-remote-bases=true on the kustomize controller, so an overlay cannot pull a base from a URL; and --default-service-account=default on the kustomize, helm and source-watcher controllers, so a Kustomization or HelmRelease that names no service account runs with its namespace's default account, which has no permissions, instead of cluster-admin. A JSON patch on the flux-system Kustomization in gotk-sync.yaml sets /spec/serviceAccountName to kustomize-controller, which keeps the platform's own sync on the controller's cluster-admin account, and the two Kustomizations in infrastructure.yaml get the same service account explicitly, since they install CRDs and cluster-scoped Gateway resources that no namespaced account may create.
The tenant then gets what the default account lacks. The agent adds tenants/base/dev-team/ with a ServiceAccount named dev-team in the podinfo namespace, a RoleBinding of that account to the built-in admin ClusterRole in that namespace only, which Flux's aggregated flux-edit rules extend to Flux's own kinds, and moves the namespace manifest there from apps/base/podinfo/namespace.yaml so the namespace is created by the platform rather than by the tenant that lives in it. apps.yaml in each cluster is changed from one apps Kustomization in flux-system to a tenants Kustomization in flux-system that applies tenants/<cluster> with the controller's account, and a podinfo Kustomization in the podinfo namespace with serviceAccountName: dev-team that reconciles apps/<cluster> from an ExternalArtifact generated into that namespace; the ArtifactGenerator in artifacts.yaml gains the matching artifact, and dependsOn entries are rewritten to stay in-namespace, because the cross-namespace dependsOn: infra-configs is exactly what the first flag now refuses. The HelmRelease's httpRoute keeps pointing at the Gateway in envoy-gateway-system, since that is a Gateway API reference the Gateway's allowedRoutes: from: All admits, not a Flux reference. .sourceignore gets !/tenants/ so the new directory reaches the cluster.
Then it validates the way the repository already does and runs the end-to-end check it already has. In its own sandbox the agent installs the Flux CLI and flux-schema, runs scripts/validate.sh, which builds every overlay including the new tenants/ ones and validates them against the schemas in .fluxschema.yml, and runs flux migrate -f . --yes to confirm no deprecated API version appears, since test.yaml fails on either. For e2e.yaml's kind cluster the sandbox has no Docker, so the agent installs docker.io, starts dockerd, creates the kind cluster the workflow names and installs Flux with --components-extra source-watcher, then points a GitRepository at its working copy and waits for infra-controllers, tenants and the tenant's podinfo Kustomization to become ready. It then proves the lockdown bites: it applies a Kustomization in podinfo whose sourceRef names the flux-system artifact and reads the ACL denial in its status, and a HelmRelease with no service account in a fresh namespace that fails with the default account's forbidden error, and quotes both before deleting them.
Start it with one API call
POST https://api.phoenix.so/o/{org}/sessions with this body:
{
"agent": "claude",
"input": "Add tenant isolation to this Flux repository. Read every file under clusters, apps, infrastructure, .sourceignore, .fluxschema.yml, scripts/validate.sh and both GitHub workflows first, and list which team each namespace label names and which Kustomizations run with the controller's cluster-admin rights. In each cluster's flux-system kustomization.yaml extend the existing controller patch with the no-cross-namespace-refs flag on every controller that supports it including source-watcher, no-remote-bases on the kustomize controller, and the default service account flag on the kustomize, helm and source-watcher controllers, and patch the flux-system Kustomization's serviceAccountName to kustomize-controller. Give the infrastructure Kustomizations the same account explicitly. Create a tenants directory with a ServiceAccount per tenant namespace bound to the admin role in that namespace only, move the tenant namespace there, and replace the apps Kustomization with a platform-owned tenants Kustomization plus a tenant-owned one in the tenant namespace running as the tenant account from its own artifact, keeping every sourceRef and dependsOn in-namespace. Update .sourceignore. Run scripts/validate.sh and flux migrate as the test workflow does, then install Docker and kind in the sandbox, bootstrap the staging cluster from the working copy as the e2e workflow does, wait for the Kustomizations and the podinfo HelmRelease, and prove the lockdown with one cross-namespace sourceRef and one HelmRelease without a service account, quoting both denials. Open a pull request.",
"resources": [
{
"url": "https://github.com/your-org/your-app"
}
]
}- Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
- Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
- Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.
What comes back
A pull request that changes both clusters' flux-system/kustomization.yaml, infrastructure.yaml, apps.yaml and artifacts.yaml, adds tenants/ with a base and a per-cluster overlay, moves namespace.yaml out of apps/base/podinfo/, and edits .sourceignore. The description has a table of every Kustomization and HelmRelease with the service account it now runs as, quotes scripts/validate.sh and flux migrate output, the flux get kustomizations --all-namespaces listing from the kind cluster with every object ready, and the two denials: the ACL message on the cross-namespace sourceRef and the forbidden error from the default account.
The repository's test workflow runs the same validation on the pull request, and e2e runs the kind bootstrap on the branch push, so a reviewer sees both pass before reading the YAML. They then decide whether a Kyverno or other admission policy should enforce serviceAccountName on every tenant Kustomization, which the description recommends and leaves out, and whether to remove the flux-view and flux-edit aggregation so tenants need explicit Flux permissions.
Run it on every new tenant namespace
Once the lockdown is merged, onboarding a team is a change under tenants/: a namespace, a service account, a role binding and a Kustomization that runs as that account. An automation in the Project with the repository, with a github trigger on pull_request.opened and pull_request.synchronize and the prompt "if this pull request adds or changes a Flux Kustomization or HelmRelease outside flux-system, check that it names a serviceAccountName in its own namespace and that every sourceRef and dependsOn stays in that namespace, and say what is wrong in a comment", reviews each tenant change against the rules the flags enforce before Flux refuses it on the cluster. The trigger adds the pull request after the prompt and fires only for work by owners, members and collaborators.
For another Flux repository, send this body again with it in resources and a request_id that is a UUID v5 of the repository's full name, so a retried call returns the run already started. The run reads that repository's own cluster directories, so one bootstrapped by the Flux Operator with ResourceSets gets the flags through its FluxInstance instead of a kustomize patch.
The use case this job follows, with its automation and trigger: Hand off a long task.
Why run it on Rush?
Tenant isolation in Flux is three flags and some RBAC, and the risk is the order: turn on the flags before every platform Kustomization names a service account and the cluster stops reconciling its own infrastructure, or forget one dependsOn and the apps never become ready. The agent reads every reference in the repository, rewrites the ones the flags will refuse, and boots a real kind cluster in its own sandbox with Docker installed to prove that the platform objects reconcile and the tenant's do not escape their namespace, so the pull request carries flux get output and the two denials rather than a diagram.
Every command stays with the run, from scripts/validate.sh to the final kubectl deletes, and the pull request quotes them. The same lockdown across several Flux repositories costs no more attention than one, because each run reads that repository's own clusters and opens its own pull request.
Frequently asked questions
Will the platform's own Kustomizations keep working?
Yes. The flux-system Kustomization is patched to run as the kustomize-controller service account, and the two infrastructure Kustomizations name it explicitly, so cert-manager, Envoy Gateway and the ClusterIssuer still install with cluster-admin rights. The kind run in the sandbox waits for all of them to become ready before the lockdown is tested.
What does the tenant lose?
Cluster-admin. A tenant Kustomization or HelmRelease runs as its namespace's service account, bound to admin in that namespace, so it can create Deployments, Services and HTTPRoutes there and nothing cluster-scoped, and it cannot reference a Flux source or Kustomization in another namespace. The podinfo chart needs nothing more, which the kind run shows.
Why move the namespace out of the app directory?
Because a tenant cannot create its own namespace once it runs as a namespaced account, and because the namespace's toolkit.fluxcd.io/tenant label is the platform's statement of ownership. The namespace, the service account and the role binding live under tenants/, applied by the platform-owned Kustomization.
Does this need an admission controller?
Not for the flags to work, since the controllers enforce them. The description recommends a policy that requires serviceAccountName on every Kustomization and HelmRelease outside flux-system, because the default account fallback fails safe but fails late, after the object is committed and reconciled.
Why does the dependsOn on the apps change?
The apps Kustomization depended on infra-configs across namespaces, which --no-cross-namespace-refs refuses. The tenant's Kustomization now depends on nothing outside its namespace, and the platform's tenants Kustomization, which runs in flux-system, keeps the dependency on infra-configs so the Gateway exists before any tenant route.