All jobs

Add Kyverno Policies to a GKE Deployment

Yes. An agent tests Kyverno's library policies against your manifests, writes the ones that fit as a kustomize component, adds a CI check and opens a PR.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

The agent reads the deployment before it reads any policy. In a GKE sample like GoogleCloudPlatform/microservices-demo, terraform/main.tf creates a google_container_cluster with enable_autopilot = true and applies the manifests with a null_resource; kustomize/kustomization.yaml builds base/, where twelve Deployments live in eleven files, one per service plus the Redis the cart uses, and lists the optional components under components/, all commented out, from network-policies to container-images-registry. release/kubernetes-manifests.yaml is the rendered copy the README applies with kubectl apply -f. CI already validates shape: .github/workflows/kustomize-build-ci.yaml runs kubectl kustomize . plus kustomize build over each scenario in kustomize/tests/, and kubevious-manifests-ci.yaml runs kubevious/cli@v1.0.64 over kubernetes-manifests, helm-chart and kustomize with container-latest-image skipped. Nothing checks the pod security fields against a policy.

The manifests are already careful, which is what makes a policy worth writing: every Deployment under kustomize/base/ sets runAsNonRoot: true, allowPrivilegeEscalation: false, readOnlyRootFilesystem: true, dropped capabilities, requests and limits, and readiness and liveness probes, and every service image is pinned to us-central1-docker.pkg.dev/online-boutique-ci/microservices-demo/<service>:v0.10.7. The gaps are specific. No pod sets seccompProfile, while helm-chart/values.yaml has a seccompProfile block with enable: false and type: RuntimeDefault, so the chart knows about it and the kustomize base does not. cartservice.yaml runs redis:alpine with no tag pin and loadgenerator.yaml pins busybox:1.38.0 by digest, so two images come from Docker Hub while the rest come from Artifact Registry. The frontend-check init container in loadgenerator.yaml has no resources, and the loadgenerator has no probes, which is correct for a traffic generator. No workload carries an app.kubernetes.io/name label; they use app.

The agent installs the Kyverno CLI, 1.19.2 at the time of writing, clones kyverno/policies, and applies ten library policies in the current policies.kyverno.io/v1 ValidatingPolicy form to the output of kustomize build: require-requests-limits, require-pod-probes, require-ro-rootfs, disallow-latest-tag, restrict-image-registries, require-labels, and the restricted pod-security set, disallow-privilege-escalation, require-run-as-nonroot, disallow-capabilities-strict and restrict-seccomp-strict. The CLI reports 82 passes and 38 failures over the twelve Deployments, and the failures are exactly the gaps it read: all twelve fail restrict-seccomp-strict and require-labels, all twelve fail restrict-image-registries because the sample allows only eu.foo.io and bar.io, and only loadgenerator fails require-requests-limits, for its init container, and require-pod-probes. The five other policies pass everywhere.

From that table it writes kustomize/components/kyverno-policies/, a Component like network-policies with a README and one file per policy, and it changes the policies rather than the manifests where the manifest is right: restrict-image-registries allows us-central1-docker.pkg.dev/ and docker.io/, require-labels is left out with a note, since the repository's selectors use app and relabelling twelve Deployments is not this job, and require-pod-probes gets a Kyverno PolicyException for loadgenerator. Where the manifest is wrong it changes the manifest: seccompProfile: {type: RuntimeDefault} goes into each pod's securityContext, matching the chart, and the init container gets requests and limits. Every policy ships with validationActions: [Audit] and the README says how to switch to Enforce once the cluster, with Kyverno installed by Helm into its own kyverno namespace, reports no audit results. A new workflow, kyverno-policies-ci.yaml, runs kyverno apply against kustomize build for the base and each scenario in kustomize/tests/ on every pull request that touches kustomize/, so a future manifest that drops runAsNonRoot fails before merge.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "claude",
  "input": "Add Kyverno policies to this GKE deployment. Read the terraform, the kustomize base and components, the release manifests, the Helm chart values and every workflow first, and list what the pod specs already set: securityContext fields, seccomp, probes, resources, image registries and tags, labels. Install the Kyverno CLI and clone kyverno/policies, then run kyverno apply with the best-practices and restricted pod-security ValidatingPolicy samples against kustomize build output and record pass and fail counts per policy. Add a kustomize component named kyverno-policies with one file per policy that fits, with validationActions Audit, a README that explains each choice, and the registry list set to the registries the manifests use. Where a failure shows a real gap, fix the manifest; where the manifest is right, adjust the policy or add a PolicyException and say why. Add a workflow that runs kyverno apply over the base and every scenario under kustomize/tests on pull requests. Open a pull request with the before and after table.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

A pull request with the new kustomize/components/kyverno-policies/ directory, its README, the manifest edits for seccomp and the init container's resources, the PolicyException for the load generator, and .github/workflows/kyverno-policies-ci.yaml. The description holds the table of policies with their pass and fail counts before the change and after, one line per policy saying whether the agent changed the policy, the manifest or neither, and the kyverno apply output for the final state; the full CLI output for every scenario is in the run's record.

A reviewer checks the table's second column against their intent. A policy the agent tuned, such as the registry list, is a decision the reviewer confirms; a manifest edit, such as seccompProfile, is a change to running pods once applied, so the reviewer reads it like any deployment change. The Audit default means merging adds reports and blocks nothing, and the README says what to flip and when.

Run it on every terminal hand-off

A platform engineer asked for admission policy before the next security review starts this from the terminal with this body and the deployment repository in resources, and goes back to the cluster. The run reads the real pod specs and tests real policies against them, so the component it writes reflects what the manifests already do rather than a generic baseline.

Send a request_id, a UUID, with the body so a retried start returns the run it already began. When the turn ends, GET /o/{org}/sessions/{id} names the pull request, and a follow-up to the same session can ask it to switch one policy to Enforce or to relabel the Deployments with app.kubernetes.io/name in a second pull request.

The use case this job follows, with its automation and trigger: Hand off a long task.

Why run it on Rush?

Policy work is only credible with the failures in hand: which policy, which workload, which field, and whether the manifest or the policy is the one that should move. The agent gets that table by running the Kyverno CLI over the built manifests in its own sandbox, with the policy library checked out beside them, and the pull request carries the counts before and after, so a reviewer judges each decision against the output rather than against a description of it.

The same run fits any Kubernetes repository that builds with kustomize or Helm, since it starts from the rendered output and the library policies, not from a list of rules written for one cluster. Each repository gets a component derived from its own gaps, a workflow that keeps them closed, and the full record of every policy it tried.

Frequently asked questions

Does it install Kyverno on my cluster?

No. The run has no cluster access; it tests policies with the Kyverno CLI against the manifests kustomize builds and commits the policies as a component. The README it writes gives the Helm install into a kyverno namespace and the order to apply things in; a person runs that.

Why do the policies start in Audit mode?

Because the agent cannot see what else runs in the cluster. Audit records violations in policy reports without blocking admission, so the first days show whether anything outside this repository would be refused. The README names the validationActions line to change to Enforce per policy.

What about the sample policies that do not fit my manifests?

The agent changes the policy when the manifest is right, such as setting the allowed registries to the ones the images use, and adds a PolicyException where one workload is a deliberate exception, such as a load generator without probes. Each case is one line in the pull request's table with the reason.

Will the CI check slow down pull requests?

The workflow runs only when kustomize/ changes, like the existing kustomize build check, and kyverno apply over a built manifest takes seconds. It fails a pull request only on a policy marked for enforcement, so audited policies report without blocking.

More GKE jobs

The same job on other stacks

More jobs like this: Hand off a long task

Keep reading