What the agent does
The agent starts in the chart directory instead of inventing a generic suite. In a repository like prometheus-community/helm-charts it opens charts/prometheus-blackbox-exporter/Chart.yaml and reads name: prometheus-blackbox-exporter, version: 11.20.0, appVersion: v0.29.0 and kubeVersion: ">=1.21.0-0". It lists templates/ (deployment.yaml, daemonset.yaml, ingress.yaml, service.yaml, configmap.yaml, _helpers.tpl and the rest) and confirms there is no unittests/ or tests/ tree under this chart, even though sibling charts such as prometheus-node-exporter and kube-prometheus-stack already ship helm-unittest YAML under unittests/.
It reads values.yaml for the knobs the prompt names. ingress.enabled defaults to false with hosts[0].host: chart-example.local and a path /; extraVolumes and extraVolumeMounts are empty lists with commented examples; securityContext sets runAsUser: 1000, runAsGroup: 1000, runAsNonRoot: true, readOnlyRootFilesystem: true, allowPrivilegeEscalation: false and capabilities.drop: ["ALL"]; kind defaults to Deployment. The ci/ folder already holds render fixtures (default-values.yaml, ingress-values.yml, daemonset-values.yml) that chart-testing uses, which the agent uses as a hint for realistic set: blocks without copying them blindly.
Matching the layout used by charts/prometheus-node-exporter/unittests/, it adds suites under charts/prometheus-blackbox-exporter/unittests/ with the helm-unittest schema header. One suite targets deployment.yaml (and daemonset.yaml when kind: DaemonSet) for default values: image tag from appVersion, the default securityContext fields, and probes on /-/healthy. Another suite turns ingress.enabled on and off against ingress.yaml, asserting the Ingress appears only when enabled and that host and path match the values. A third mounts an extraVolumes secret volume and checks the volume and mount land on the pod template. It does not change production templates.
Before opening the pull request it installs the same plugin the repository's .github/workflows/ci.yaml installs (helm plugin install https://github.com/helm-unittest/helm-unittest.git), runs helm unittest charts/prometheus-blackbox-exporter and helm lint charts/prometheus-blackbox-exporter, and fixes assertion paths until both commands exit clean. Then it opens a pull request whose description lists each suite file and the commands that passed.
Start it with one API call
POST https://api.prix.dev/o/{org}/sessions with this body:
{
"agent": "claude",
"input": "Write helm-unittest tests for the prometheus-blackbox-exporter chart's templates: default values, ingress on and off, extra volumes, and pod security settings. Run helm unittest and helm lint, and open a pull request.",
"resources": [
{
"url": "https://github.com/your-org/your-charts"
}
]
}- Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
- Send the request. POST the body in request.json to https://api.prix.dev/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
- Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.
What comes back
What comes back is a pull request on a new branch. For a chart shaped like prometheus-blackbox-exporter it adds YAML under charts/prometheus-blackbox-exporter/unittests/ (for example suites that cover deployment.yaml / daemonset.yaml and ingress.yaml), leaves templates/ and default values.yaml unchanged, and may add a short note in the chart README only if the repository already documents how sibling charts run unittest.
The pull request description names the helm unittest and helm lint commands that were run and which suites cover ingress, volumes and securityContext. The session trajectory keeps every helm command and its stdout, so a reviewer can see a failed assertion path that was corrected before the push.
Before merging, a reviewer checks that the suites follow the same directory layout as other charts in the repo, that set: blocks match real keys in values.yaml, and that no template logic was rewritten just to make a test pass. Chart-testing CI continues to use the existing ci/*-values files; the new unittest suites are an extra check on render shape.
Run it on every chart values or templates changed
Point a GitHub pull_request webhook (paths under charts/) at your server and, when the diff touches a chart that still lacks unittests/, POST this same session body with that repository in resources. Each chart gets its own sandbox run and its own pull request, so a monorepo of exporters can fill gaps without one giant branch.
Set a webhook URL on the API key so Rush POSTs run.completed or run.failed when the turn finishes, with the session URL, and your server can paste the pull request link into the chart owners' channel. A key runs at most max_concurrent sessions at once (4 by default) and answers 429 past that, so queue chart-by-chart backfills.
The use case this job follows, with its automation and trigger: Write the missing tests.
Why run it on Rush?
helm-unittest needs the helm binary, the plugin, and repeated helm template renders for every set: combination. Doing that on a laptop means installing the plugin globally, fighting version skew with CI, and leaving half-finished YAML in your working tree while you iterate. On Rush the agent installs the plugin inside its own sandbox, runs the same commands the repo's CI workflow names, and only pushes when helm unittest and helm lint are clean.
The proof is in the pull request and the trajectory: reviewers see the suite files next to charts that already use this pattern, and they can re-run the same two commands locally. Many charts can be covered in parallel, each in its own sandbox, without blocking a single engineer's machine on render loops.
Frequently asked questions
Will it modify my chart templates?
The job is to add tests, not to rewrite templates. It asserts on the rendered objects from the current templates and values. If a template cannot express what the prompt asks to assert, it says so in the pull request instead of silently changing production YAML.
Where do the test files go?
Alongside the chart, typically charts/<name>/unittests/*.yaml, matching how prometheus-community charts such as prometheus-node-exporter already store suites. It does not put tests under templates/tests unless that chart already uses the older in-template test pod pattern.
Do I need a Kubernetes cluster?
No. helm-unittest renders with the Helm engine and checks the YAML structure. helm lint is static too. Cluster install and chart-testing remain separate from this job.
What Helm and plugin versions does it use?
It follows the repository's CI where present. In prometheus-community/helm-charts the workflow installs helm-unittest from GitHub with a pinned VERSION and --verify=false. The agent uses that same install line so local and CI stay aligned.