All jobs

Validate Every Kustomize Build With kubeconform in CI

Yes. An agent adds a workflow that builds each kustomization, checks the output with kubeconform in strict mode, skips the one a plugin renders, and opens a PR.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

The agent starts by finding every kustomization and what it needs. In a GitOps repository laid out like argoproj/argocd-example-apps, there is no .github/workflows directory at all, so nothing checks a manifest before Argo CD tries to apply it. kustomize-guestbook/kustomization.yaml declares namePrefix: kustomize- and two resources, a Deployment and a Service for gcr.io/google-samples/gb-frontend:v5; sock-shop/kustomization.yaml lists twenty-nine files under base/, a Deployment and a Service for each of carts, catalogue, front-end, orders, payment, queue-master, rabbitmq, shipping and user with their databases, plus front-end-ingress.yaml on networking.k8s.io/v1; pre-post-sync/kustomization.yaml points at ../kustomize-guestbook through the deprecated bases: key and adds two Jobs annotated argocd.argoproj.io/hook. plugins/kustomized-helm/kustomization.yaml names ./all.yaml, a file the repository does not contain: the config management plugin renders it from the chart at sync time, so kustomize build cannot run there and the workflow must say so rather than fail on it.

It then writes .github/workflows/validate.yml, triggered on pull_request and on pushes to the default branch. One job checks out the repository, installs a pinned release of kustomize and of kubeconform from their GitHub releases, and loops over the three buildable directories with kustomize build <dir> | kubeconform -strict -summary -kubernetes-version <version>. -strict rejects fields the schema does not know, which catches a misspelled replicas or containerPort that Kubernetes would otherwise drop silently. The version is the one the reader's clusters run, passed in from a workflow variable so an upgrade is one edit. Every resource in the three builds is a core kind, Deployment, Service, Job and Ingress, so the default schema source covers them and no CRD catalog is needed yet; the agent leaves a commented line showing the -schema-location form for when a Rollout or an Application joins a kustomization.

Two details from the files go into the job. The bases: key in pre-post-sync makes kustomize build print a deprecation warning on stderr; the job keeps the build's exit code as the verdict and does not treat the warning as failure, and the pull request description points the team at the one-line resources: replacement. kustomize-guestbook/guestbook-ui-svc.yaml ends each line with a tab character, which the YAML parser accepts and kubeconform does not mind, and the agent leaves it alone.

In its sandbox the agent downloads the same two binaries, runs the loop against the working tree, and quotes the -summary line for each directory. It then breaks a manifest on purpose in a scratch copy, misspelling containerPort in sock-shop/base/front-end-dep.yaml, to show the strict check rejects it with the file and field named, and reverts the change before committing.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "claude",
  "input": "Validate every Kustomize build with kubeconform in CI. Read every kustomization.yaml in this repository and the files they reference first. Add .github/workflows/validate.yml that runs on pull requests and pushes to the default branch, installs pinned releases of kustomize and kubeconform, and for each directory whose kustomization can be built from the repository alone runs kustomize build piped into kubeconform with -strict and -summary, with the Kubernetes version taken from a workflow variable. Skip a kustomization whose resources are rendered by a plugin and say which in the workflow. Do not fail on the bases deprecation warning. Run the same loop in the sandbox, quote the summary per directory, show that a misspelled field is rejected, and open a pull request.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

What comes back is a pull request that adds one file, .github/workflows/validate.yml, and changes no manifest. The workflow names each directory it builds and the one it skips with the reason, pins both tool versions, and reads the Kubernetes version from a variable with a default.

The description quotes the kubeconform summary for kustomize-guestbook, sock-shop and pre-post-sync from the agent's sandbox run, the stderr warning about bases: so the team sees it is a warning and not a failure, and the rejection the strict check produced for the deliberately misspelled field, with the file and path kubeconform printed. Because the workflow is new, the pull request itself is the first run of it, and its check appears on the pull request alongside the agent's record.

Before merging, a reviewer sets the Kubernetes version variable to the clusters' real version, decides whether the bases: key should be replaced in the same change or in a follow-up, and whether apps/ and blue-green/, which are Helm charts rather than kustomizations, should get a helm template step with the Argo CD and Rollouts CRD schemas next.

Run it on every failed validate workflow

Once the workflow is in the repository it runs on every pull request, so the run worth starting on Rush is the one that reads a failure. In the Project that holds the repository, create an automation with a github trigger on workflow_run.failure and a prompt that reads the failed job's log, finds the manifest and field kubeconform named, fixes the manifest when the field is a typo or a wrong API version, and opens a pull request against the branch, or comments on the pull request naming the field when the fix is a decision. The trigger puts the failed run after the prompt and fills nothing in.

A failure in a workflow other than validate is not this automation's; the prompt says to open nothing for it. Pull requests from forks never fire the trigger.

The use case this job follows, with its automation and trigger: Hand off a long task.

Why run it on Rush?

Checking a manifest change means building every kustomization with the same tool versions CI will use and reading the strict output, which on a laptop is two downloads and a loop that few people run before pushing. The sandbox does it from a clean start, keeps each directory's summary, and proves the check rejects a bad field before the workflow is ever merged.

The run's record keeps the build output, the warning text and the rejection message, so a reviewer reads what kubeconform said rather than a line claiming the manifests are valid. Across a platform team's repositories the same request runs at once, one sandbox each, and each pull request lists that repository's own directories and the ones it had to skip.

Frequently asked questions

Why kubeconform and not kubectl apply --dry-run?

A dry run needs a cluster and its credentials in CI; kubeconform checks against published schemas with no cluster at all. Its -strict flag also rejects unknown fields, which a server-side dry run reports only for the API versions the cluster has.

What about the Argo CD Application and Rollout kinds?

They live in the Helm charts under apps/ and blue-green/, not in a kustomization, so the loop never sees them. When one appears in a kustomization, the -schema-location line the agent left commented points kubeconform at a CRD schema catalog in addition to the default source.

Will the deprecated bases key fail the job?

No. kustomize build prints a warning and still exits zero, and the job uses the exit code. The pull request notes the one-line change to resources: so the warning can be removed when the team chooses.

Does this change what Argo CD deploys?

No. The workflow only reads the repository. Argo CD keeps syncing from the branch it watches; the check tells people before a merge that a manifest would not apply.

More Kustomize jobs

The same job on other stacks

More jobs like this: Hand off a long task

Keep reading