All jobs

Add Terraform Drift Detection to an EKS Configuration

Yes. An agent adds a scheduled GitHub Actions workflow that runs terraform plan with -detailed-exitcode against your state and opens an issue when drift appears.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

The agent clones the repository into its own cloud sandbox and reads the root module before it writes a workflow. In a configuration like hashicorp-education/learn-terraform-provision-eks-cluster, terraform.tf pins required_version = "~> 1.3" and four providers (hashicorp/aws ~> 5.47.0, random ~> 3.6.1, tls ~> 4.0.5, cloudinit ~> 2.3.4), and a committed .terraform.lock.hcl resolves aws to 5.47.0 with its hashes. main.tf declares a random_string.suffix, a module "vpc" from terraform-aws-modules/vpc/aws 5.8.1 with one NAT gateway, a module "eks" from terraform-aws-modules/eks/aws 20.8.5 with two managed node groups (one and two, both t3.small) and the aws-ebs-csi-driver add-on, and a module "irsa-ebs-csi" from the IAM module's iam-assumable-role-with-oidc. variables.tf has one variable, region, defaulting to us-east-2; outputs.tf exposes cluster_endpoint, cluster_security_group_id, region and cluster_name. There is no .github directory, and the cloud {} block in terraform.tf is commented out, so nothing in the repository says where state lives.

That last fact decides the design. Drift is the difference between state and the live account, so a plan in CI needs the same state the last apply wrote. The agent writes .github/workflows/drift.yml with a schedule cron and workflow_dispatch, permissions of id-token: write, contents: read and issues: write, aws-actions/configure-aws-credentials assuming a role through OIDC (the role ARN as a repository variable, the region from variables.tf's default), and hashicorp/setup-terraform pinned inside the ~> 1.3 constraint. The job runs terraform init -input=false and then terraform plan -input=false -lock=false -detailed-exitcode -no-color -out=drift.tfplan. Exit code 0 means no drift, 1 an error, 2 a difference; the step is marked continue-on-error so the job can branch on the code instead of failing on it. The agent adds a backend "s3" block only when the prompt names a bucket and table; otherwise the pull request says in its first line that the workflow needs a remote backend or the uncommented cloud {} block before it can see state.

On exit code 2 the workflow renders terraform show -no-color drift.tfplan into the job summary and runs gh issue list --label drift to find an open drift issue; it updates that issue with the new plan or creates one with the drift label, so a week of drift is one issue with a history rather than seven. On exit code 1 the job fails, which is the right outcome for an expired role or a provider error. The agent notes in the workflow why data.aws_availability_zones.available never counts as drift (a data source is read fresh on every plan) and why random_string.suffix never changes (its value is in state, and random only regenerates when its arguments change).

Before opening the pull request the agent checks what it can without an AWS account: terraform fmt -check, terraform init -backend=false, which fetches the providers the lock file pins, terraform validate, and actionlint on the workflow. It does not run the plan itself, because the sandbox has no credentials and no state; the pull request says so rather than describing a result it did not see.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "claude",
  "input": "Add drift detection to this Terraform repository. Read terraform.tf, main.tf, variables.tf, outputs.tf and .terraform.lock.hcl first. Write .github/workflows/drift.yml that runs on a schedule and on workflow_dispatch, assumes an AWS role through OIDC using a repository variable for the role ARN, installs the Terraform version required_version allows, runs terraform init and terraform plan -detailed-exitcode -lock=false, and on exit code 2 writes the plan to the job summary and opens or updates a GitHub issue labelled drift. Never run terraform apply in the workflow. If the configuration has no backend, say so at the top of the pull request and explain what to add. Check the workflow with actionlint and the configuration with terraform fmt -check, terraform init -backend=false and terraform validate, then open a pull request that explains each permission the workflow needs.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

What comes back is a pull request on its own branch with one new file, .github/workflows/drift.yml, and a short section added to README.md naming the repository variable for the role ARN and the IAM trust policy the role needs for GitHub's OIDC provider. If the prompt named a backend, terraform.tf gains a backend block as a second change; if not, the pull request's first paragraph says the workflow cannot see state until one exists. The description quotes the terraform validate and actionlint output from the sandbox and explains the three exit codes the plan step handles.

The run's record keeps every command the agent ran with its output, so a reviewer can see that terraform init -backend=false resolved the providers the lock file pins and that validate passed, and that no plan or apply ran against any account.

Before merging, a reviewer checks that the schedule is frequent enough for how the account is touched, that -lock=false is acceptable for a read-only plan (it is the usual choice, because a scheduled plan must not block a human apply), that the role's permissions are read-only, and that the drift label exists in the repository.

Run it on every new Terraform root module

A platform team with many Terraform repositories starts one run per repository from its own server: one POST /o/{org}/sessions with this body and the repository in resources, with a request_id that is a UUID v5 of the repository name so a retried call returns the first run instead of opening a second pull request. Each run reads that repository's own terraform.tf and lock file, so the pinned versions in each workflow match its module.

A webhook URL on the API key receives run.completed or run.failed for every run the key started, with session_url, so the server can collect the pull request links in one place and post them to the channel where infrastructure changes are reviewed.

The use case this job follows, with its automation and trigger: Hand off a long task.

Why run it on Rush?

A drift workflow is small but easy to get subtly wrong: the wrong exit-code handling turns every plan with changes into a red build, a missing -lock=false blocks a colleague's apply at the moment the schedule fires, and a workflow written without reading required_version installs a Terraform that refuses the configuration. The agent reads the pins and the modules first, validates the configuration with the exact providers the lock file names, and keeps that output in the run's record.

It also stays honest about what it could not check. The sandbox has no AWS role and no state, so the pull request says the plan was not run, names the precondition (a backend) if it is missing, and leaves the first real plan to the workflow itself. A reviewer gets a change they can read in five minutes with the proof attached, rather than a workflow pasted from a blog post.

Frequently asked questions

Does the agent need AWS credentials to do this?

No. Writing and validating the workflow needs only the repository: terraform init -backend=false downloads providers and terraform validate checks the configuration without an account. The scheduled plan is what needs the OIDC role, and that runs in GitHub Actions, not in the sandbox.

Will the workflow ever apply changes?

No. The workflow runs terraform plan only, and the prompt forbids terraform apply. A reviewer can confirm this by reading the one workflow file in the pull request; there is no apply step to enable later by accident.

What if state is in HCP Terraform instead of S3?

If the cloud {} block in terraform.tf is uncommented, the agent passes an API token to hashicorp/setup-terraform through cli_config_credentials_token and lets the plan run remotely in that workspace. The exit-code logic is the same; only the credentials step changes.

Why open an issue rather than fail the job?

Drift is a fact about the account, not a defect in the commit, and a failed scheduled job is easy to ignore. One issue labelled drift, updated with each new plan, keeps the history in one place and can be assigned to whoever owns the cluster.

Can it run for several root modules in one repository?

Yes. Say so in input and the agent writes a matrix over the directories it finds with a terraform.tf or versions.tf, with one plan and one issue per directory.

More Terraform jobs

The same job on other stacks

More jobs like this: Hand off a long task

Keep reading