All jobs

Add chart-testing and a kind Install Test to a Helm Chart's CI

Yes. An agent adds a ct config and test values to your chart, runs ct lint in its sandbox, writes a CI job that installs the chart on a kind cluster, and opens a PR.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

The agent clones the repository into its own cloud sandbox and reads how the chart is tested today. In a repository like GoogleCloudPlatform/microservices-demo the chart lives in helm-chart/, with Chart.yaml at version 0.10.7, a values.yaml where every one of the eleven services has a create flag and a resources block, and templates such as frontend.yaml, cartservice.yaml and loadgenerator.yaml. The existing .github/workflows/helm-chart-ci.yaml runs helm lint --strict and then helm template five times with different --set combinations, for gRPC health probes, Spanner, a service mesh and TLS origination to an external Redis, and pipes each rendering through kustomize build to check the YAML is well-formed. Nothing ever installs the chart, so a template that renders but fails admission, a probe that never passes or a Service with no matching pod would not be caught before a release to oci://us-docker.pkg.dev/online-boutique-ci/charts/onlineboutique.

It adds the chart-testing configuration the ct tool reads: a ct.yaml at the repository root naming helm-chart as the chart directory, the target branch, and validate-maintainers: false because Chart.yaml lists no maintainers, plus a helm-chart/ci/ directory with one values file per scenario the current workflow exercises. default-values.yaml is empty on purpose so ct install runs the chart as shipped; grpc-probes-values.yaml sets nativeGrpcHealthCheck: true; mesh-values.yaml turns on networkPolicies.create, sidecars.create, authorizationPolicies.create and the frontend virtualService. Scenarios that need Google Cloud, the Spanner connection string and the Workload Identity annotation, stay as helm template steps, because a kind cluster has no Spanner to connect to; the agent says so in the file's header comment rather than writing a values file that can only fail.

Two things in this chart would make a bare ct install fail, and the agent handles both. The loadgenerator deployment has an init container that polls frontend:80 twelve times and exits non-zero if it never answers, and the frontend's externalService creates a LoadBalancer Service that never gets an address on kind; so the test values set loadGenerator.create: false and frontend.externalService: false, and the agent explains in the pull request that the load generator is a demo client, not part of the chart under test. It also notes that the chart's images come from us-central1-docker.pkg.dev/online-boutique-ci/microservices-demo at the tag appVersion names, so the install test pulls public images and needs no registry secret.

It writes the new workflow job in helm-chart-ci.yaml after the existing lint and template steps: helm/chart-testing-action to install ct, helm/kind-action to create a cluster, ct lint --config ct.yaml and ct install --config ct.yaml, with a ct list-changed step so the install only runs when the chart changed and the job stays cheap on unrelated pull requests. In the sandbox the agent installs helm and ct and runs ct lint --config ct.yaml --all against the real chart, which also validates Chart.yaml and values.yaml against their schemas and enforces a version bump, so it bumps version to 0.10.8 as ct requires for a changed chart. The sandbox has no Kubernetes cluster and installing one is not what the job needs, so ct install is run by the workflow, not the agent, and the pull request says exactly that.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "claude",
  "input": "Add chart-testing to this repository's Helm chart. Read Chart.yaml, values.yaml, every template under the chart's templates directory, the chart README and the existing GitHub Actions workflow that lints or templates the chart first. Write a ct.yaml at the repository root and a ci/ directory inside the chart with one values file per scenario the current workflow renders, leaving out scenarios that need cloud services a local cluster cannot provide and saying why in a comment. Read the templates for anything that would fail an install on a kind cluster, such as init containers that wait for a service or LoadBalancer Services, and set test values so the install can pass, explaining each choice. Add a job to the workflow that installs ct and kind, runs ct list-changed, ct lint and ct install, and bump the chart version as ct requires. Install helm and ct in your sandbox and run ct lint against the chart; do not try to run ct install, and say in the pull request that the install test runs in CI. Open a pull request with the ct lint output and the list of scenarios covered.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

What comes back is a pull request adding ct.yaml, a ci/ directory inside the chart with three small values files, a new chart-testing job in the chart's workflow, and a one-line version bump in Chart.yaml. The templates are untouched, so helm template output on the default values is identical to before, and the description includes the ct lint output from the sandbox showing the chart, its values schema check and the version bump passing.

The description lists each scenario from the old workflow and where it now lives: three as ci/ values files that ct install will deploy on kind, and the Spanner and Memorystore scenarios still as helm template steps, with the reason. It also names the two template behaviours the test values work around, the load generator's frontend check and the LoadBalancer Service, so a reviewer knows the install test covers the services and not the demo client.

Before merging, a reviewer checks the first run of the new job on the pull request itself, since the install test can only run in CI, decides whether the job should also run on a schedule against the published chart, and confirms the version bump matches the team's release convention.

Run it on every pull request that touches the chart

In the console, an automation in the Project that has the repository, with a github trigger on pull_request.opened and pull_request.synchronize, starts one run per pull request with the event after the prompt. The prompt asks the run to finish at once unless a file under the chart directory changed, and otherwise to read the new or changed values and templates, add a ci/ values file when the change introduces a flag no existing scenario exercises, and leave a review comment naming the scenario it added, so the install test keeps up with the chart as options are added.

From your own server, send this body as POST /o/{org}/sessions with the repository in resources for each repository that ships a chart, with a request_id that is a UUID v5 of the repository name so a retried call returns the run it already started. Set a webhook_url on the key to receive run.completed with data.session_url when each pull request is open.

The use case this job follows, with its automation and trigger: Write the missing tests.

Why run it on Rush?

A chart that only ever gets rendered is tested for syntax, not for behaviour, and the gap shows up at install time in someone else's cluster. Adding the install test means reading every template for what will not work on a throwaway cluster, which is slow by hand and exactly the kind of reading an agent does well; in its sandbox it installs helm and ct, runs the lint against the real chart and keeps the output in the run's record, so the pull request arrives with proof that the configuration is valid before CI has run once.

The same request runs across every repository that ships a chart, each in its own sandbox reading its own templates, which is how a platform team gets one consistent ct setup across a dozen charts instead of a copied workflow that assumes every chart has the same shape.

Frequently asked questions

Why does the agent not run ct install itself?

It needs a Kubernetes cluster, and the point of the job is to add the test to CI where a kind cluster is created per run. The agent runs ct lint, which checks the chart, its schemas and the version bump without a cluster, and leaves ct install to the workflow. The pull request says so rather than claiming a test that did not run.

What happens to scenarios that need cloud services?

They stay as helm template steps in the workflow, as they are today. A kind cluster cannot connect to Spanner or a managed Redis, so a values file for those scenarios could only fail. The agent lists them in the pull request so the gap is a known one.

Why turn the load generator off in the test values?

Its init container polls the frontend Service and fails the pod if it never gets a 200, and ct install waits for every pod to be ready. The load generator is a demo client that drives traffic at the shop; disabling it tests the eleven services the chart is for. The default values still ship it.

Does ct require a version bump on every change?

Yes, ct lint fails when a changed chart keeps its version, which is the convention it enforces. The agent bumps the patch version in this pull request; you can set check-version-increment: false in ct.yaml if your release process bumps versions elsewhere.

Will the job slow down every pull request?

No. The job runs ct list-changed first and skips the install when no chart file changed, and the workflow already limits itself to paths under the chart directory. A full lint and install on kind runs only when the chart itself moves.

More Helm jobs

More jobs like this: Write the missing tests

Keep reading