All jobs

Add a Horizontal Pod Autoscaler to Kustomize Deployments

Yes. An agent adds an autoscaling/v2 HorizontalPodAutoscaler per Deployment with CPU requests, as an opt-in Kustomize component, renders the build and opens a PR.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

The agent clones the repository into its own cloud sandbox and reads how the Kustomize tree is put together before it adds anything. In a tree like the kustomize/ directory of GoogleCloudPlatform/microservices-demo, kustomize/kustomization.yaml lists base as its only resource and a commented list of components (cymbal-branding, network-policies, memorystore, without-loadgenerator and others), each a directory under kustomize/components/ with kind: Component and apiVersion: kustomize.config.k8s.io/v1alpha1. kustomize/base/kustomization.yaml names eleven files, one Deployment and Service per microservice from adservice.yaml to shippingservice.yaml. Every container sets resources.requests and limits: frontend asks for 100m CPU and 64Mi with limits of 200m and 128Mi, adservice for 200m and 180Mi, recommendationservice for 100m and 220Mi, and loadgenerator for 300m and 256Mi. Only loadgenerator.yaml sets replicas, and no file anywhere in kustomize/, helm-chart/ or kubernetes-manifests/ defines a HorizontalPodAutoscaler.

CPU requests are what a utilization target scales against, so every Deployment in the base qualifies, and the agent follows the repository's own convention rather than editing the base: a new kustomize/components/autoscaling/ directory with a kustomization.yaml of kind: Component whose resources list one hpa-<service>.yaml per Deployment, the way network-policies keeps one network-policy-<service>.yaml per service. Each file is an autoscaling/v2 HorizontalPodAutoscaler whose scaleTargetRef names the Deployment by its metadata.name, with minReplicas, maxReplicas and a Resource metric on cpu with averageUtilization taken from the prompt. The load generator is left out on purpose, since scaling it would scale the load it generates, and the component's README.md says so in the style of the other component READMEs, with the kustomize edit add component components/autoscaling line the main README uses for every variation.

The agent then puts the component where the repository expects a new one: a commented line in kustomize/kustomization.yaml beside the other components so it stays opt-in, a row in the variations list of kustomize/README.md, and a new folder under kustomize/tests/ with a kustomization.yaml that takes ../../base and the autoscaling component together with the others, because .github/workflows/kustomize-build-ci.yaml runs kubectl kustomize . on the base and then kustomize build on every subfolder of kustomize/tests/ whenever anything under kustomize/ changes.

To check its work the agent installs kubectl and the kustomize binary in the sandbox, enables the component in a scratch copy of kustomize/kustomization.yaml and runs kubectl kustomize kustomize/, then reads the output to confirm one HorizontalPodAutoscaler per Deployment, each scaleTargetRef.name matching a Deployment rendered in the same output, and runs kustomize build on the new test folder. It also confirms that none of the targeted Deployments pins replicas, because a fixed replica count and an HPA would fight over the same field. No cluster is involved: the run renders manifests and pushes nothing to Kubernetes.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "claude",
  "input": "Add a Kustomize component that gives every Deployment in kustomize/base a HorizontalPodAutoscaler, except loadgenerator. Read kustomize/kustomization.yaml, kustomize/base and the existing directories under kustomize/components first and follow their layout: one component directory with a kustomization.yaml of kind Component, one HPA file per service and a README. Use autoscaling/v2 with a CPU utilization target of 70 percent, minReplicas 2 and maxReplicas 10, and point each scaleTargetRef at the Deployment by name. Leave the component commented out in kustomize/kustomization.yaml like the others, add a test overlay under kustomize/tests that enables it, list it in kustomize/README.md, run kubectl kustomize on the base with the component enabled and kustomize build on the new test folder, and open a pull request.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

What comes back is a pull request that adds kustomize/components/autoscaling/ with its kustomization.yaml, one hpa-<service>.yaml per Deployment and a README.md, adds a test folder under kustomize/tests/, and changes two existing files by one line each: kustomize/kustomization.yaml and kustomize/README.md. The description lists the ten targets with their request values from the base, pastes the rendered HPA section of the kubectl kustomize output, and names the test folder the CI workflow will now build. Because the paths match the workflow's filter, the kustomize-build-ci check runs on the pull request itself.

The run's record keeps every command with its output, so a reviewer can see the kustomize build of the new test folder and the grep that matched each scaleTargetRef to a rendered Deployment, without checking the branch out.

Before merging, a reviewer decides whether a single CPU target fits every service, since adservice and recommendationservice request far more memory than the others and may want a memory metric too, checks that the clusters this tree deploys to run metrics-server, and looks at how the new minimum replica count interacts with any PodDisruptionBudget or node capacity the base does not know about.

Run it on every new Deployment added to the base

In the console, an automation in the Project that has the repository, with a github trigger on pull_request.opened and pull_request.synchronize and branch_patterns for your default branch, starts one run per pull request with the pull request as its context. The prompt tells the run to look for a Deployment under kustomize/base/ that the autoscaling component does not cover yet, add its HPA file and the matching resources entry on the pull request's branch, and do nothing when every Deployment is covered.

From your own server, the same job is one POST /o/{org}/sessions per repository with this body and the repository in resources. Send a request_id that is a UUID v5 of the repository name and the week, so a retried call returns the run already started, and set a webhook URL on the API key so Rush posts run.completed with the session_url to read the pull request link from.

The use case this job follows, with its automation and trigger: Hand off a long task.

Why run it on Rush?

Writing ten near-identical manifests by hand is where typos live: a scaleTargetRef that names the Service instead of the Deployment, a metrics block in the autoscaling/v1 shape under a v2 header, a component whose kustomization.yaml forgets a file. The agent renders the whole tree with the real kustomize binary before the pull request exists, and the rendered output is part of the run, so the reviewer reads what the cluster would receive instead of guessing from YAML.

The sandbox is its own computer with kubectl and kustomize installed for the job, and a platform team with several application repositories on the same component layout can start one run per repository at once, each ending in its own pull request with its own build output.

Frequently asked questions

Will the HPA conflict with a replicas field in the base?

It would, which is why the agent checks. In this tree only loadgenerator.yaml sets replicas, and the load generator is excluded. If a Deployment it targets does pin replicas, the agent removes that line in the same pull request and says so, because the autoscaler owns the count from then on.

Why a component instead of editing the base manifests?

Because the repository models every variation as a Kustomize component that a reader enables with kustomize edit add component. An HPA in the base would apply to every environment, including a local cluster without metrics-server, where the HPA would sit in an unknown state. The component keeps it opt-in and keeps the diff to the base at zero.

What about scaling on memory?

The prompt asks for a CPU target, so each HPA carries one Resource metric on cpu. Every container in the base also sets a memory request, so a second metric on memory is a few lines in the same files; the pull request points out the two services whose memory requests are several times the others as the ones to consider first.

Does the run need access to a cluster?

No. It renders manifests with kubectl kustomize and kustomize build in its sandbox and reads the result. Whether the target cluster has metrics-server, which an HPA on CPU utilization needs, is for the reviewer to confirm before enabling the component.

Does it change the Helm chart in the same repository?

Not in this run. The prompt scopes the work to the Kustomize tree, and the chart under helm-chart/ has no autoscaling template either. A second run with a prompt for the chart adds a templated HPA and values.yaml keys there, as a separate pull request.

More Kustomize jobs

The same job on other stacks

Keep reading