All jobs

Split a Pulumi Program Into Stacks Linked by StackReference

Yes. An agent splits one Pulumi program into layered projects, exports the shared outputs, reads them with StackReference, and opens a PR that compiles and previews.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

The agent reads the program as one piece before it draws a line through it. It starts with how the repository already does this: in pulumi/examples, aws-ts-stackreference is three projects, company, department and team, each with its own Pulumi.yaml (aws-ts-stackreference-company and so on, runtime: nodejs) and package.json pinning @pulumi/pulumi 3.228.0 and @pulumi/aws 7.23.0. The first two export one config value each; team/index.ts builds new pulumi.StackReference(config.require("companyStack")) and departmentStack the same way, reads getOutput("companyName") and getOutput("departmentName") into the tags of an aws.ec2.Instance, and its Pulumi.yaml documents companyStack and departmentStack as config in the <organization>/<project>/<stack> form. The larger aws-ts-stackreference-architecture does the same for real infrastructure: networking-layer exports appVpcId, appVpcPrivateSubnetIds, appVpcPublicSubnetIds, dataVpcPrivateSubnetIds and peeredSecurityGroupId from a Vpc component; database-layer reads networkingStack from config, passes networkingStack.getOutput("dataVpcPrivateSubnetIds") to an RdsInstance, takes dbPassword with requireSecret and exports dbEndpoint, dbPort, dbAddress, dbUsername and dbPassword; application-layer reads both and feeds the VPC, subnets and database values into an Application component behind an ALB.

Then it reads your program and finds the seams. A single index.ts that creates a VPC, a database and a service usually has three lifecycles in one stack: the network changes rarely, the database must survive an application rollback, and the service deploys daily. The agent lists every value that crosses those lines, the VPC id, subnet ids, security group id, database address, port and credentials, and that list becomes the outputs of the upstream projects and the getOutput calls of the downstream ones. Each new project gets a Pulumi.yaml with the upstream stack names as documented config, a package.json and tsconfig.json copied from the original, and an index.ts holding only its own resources. Secret outputs stay secret: a requireSecret value exported upstream comes through getOutput still marked secret, and the agent uses getOutputDetails only where a plain value is needed at deploy time.

Existing resources are the part that goes wrong when this is done by hand, so the pull request handles it explicitly. For a program that is already deployed, moving a resource's code to another project would destroy and recreate it on the next pulumi up. The agent writes the pulumi state move commands, one per resource, that move each resource's state from the original stack to its new stack, in dependency order, and puts them in a MIGRATION.md beside a checklist: deploy the upstream stacks first, set the networkingStack and databaseStack config on the downstream ones, then preview and confirm every resource shows as unchanged. For a program not yet deployed, that section says so and the order of the first pulumi up is enough.

The proof is what can be proved without cloud credentials. The agent runs npm ci and tsc --noEmit in every new project and includes the output. Where the sandbox has credentials and a backend holding the upstream stacks, it runs pulumi preview on each project in order and shows the downstream preview resolving the references; where it does not, the description says the previews are the reviewer's step and lists the exact commands. It does not run pulumi up and does not touch a real backend's state.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "claude",
  "input": "Split the Pulumi program in this repository into separate projects linked by StackReference. Read the existing program, its Pulumi.yaml and stack config files, and any StackReference usage already in the repository first. Separate the resources by lifecycle, such as networking, data and application, into their own projects, export every value that crosses a boundary from the upstream project, and read it downstream with StackReference from a documented config value in <organization>/<project>/<stack> form. Keep secret outputs secret. For resources that already exist, write the pulumi state move commands that move their state into the new stacks without recreating them, with the deploy order, in a migration note. Run the TypeScript compiler in every project, run pulumi preview if credentials and a backend are available, and include the output. Do not run pulumi up. Open a pull request.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

What comes back is a pull request that replaces one project with two or three: each with its own Pulumi.yaml, package.json, tsconfig.json and index.ts, the upstream ones ending in export const lines for every shared value and the downstream ones starting with new StackReference(config.require(...)). A MIGRATION.md lists the pulumi state move commands and the deploy order; the original stack config files are split so each project keeps the settings it uses.

The description quotes the tsc --noEmit result for each project and the pulumi preview output when it ran, with the downstream resources showing their inputs resolved from the upstream outputs. It names which values are secrets and confirms they stay secret across the reference. The full commands and output are in the transcript at GET /o/{org}/sessions/{id}/messages.

Before merging, a reviewer checks the boundary: a value that should have stayed internal is now an output, or a resource landed on the wrong side of the line. They then run the previews in order against the real backend and confirm every existing resource reads as unchanged before the first pulumi up. A follow-up message to the same session can add a fourth project or move a resource across the line.

Run it on every Pulumi program that has outgrown one stack

A platform team starts one run per repository with one POST /o/{org}/sessions each, the repository in resources and this prompt as input, when a program's pulumi up has become slow or risky because unrelated resources share a stack. Each run reads its own program and opens its own pull request with its own migration note.

Send a request_id, a UUID v5 of the repository's full name, so a retried call returns the run it already started. The key's webhook_url reports when each run the key starts finishes, with the run's final answer in result.

The use case this job follows, with its automation and trigger: Hand off a long task.

Why run it on Rush?

Splitting a stack is mostly reading: which values cross the line, which are secrets, which resources already exist and must be moved rather than recreated. A mistake shows up as a destroyed database. The agent reads the whole program and the repository's own StackReference examples, writes the split, and proves what it can with the compiler and previews in a sandbox of its own, with every command's output kept.

The pull request carries the migration commands and the preview output, so the reviewer's job is to check the boundary and run the previews, not to rediscover the plan. Several programs can be split at once, each in its own sandbox, each with its own pull request.

Frequently asked questions

Will existing resources be recreated?

Not if the migration note is followed. Moving a resource's code between projects without moving its state makes Pulumi destroy and recreate it. The pull request lists the pulumi state move commands per resource in dependency order so each resource's state lands in the new stack and the first preview shows it unchanged.

How do the downstream stacks find the upstream ones?

Through a config value such as networkingStack holding the fully qualified <organization>/<project>/<stack> name, documented in the project's Pulumi.yaml as the repository's own examples do, and read with config.require into new StackReference(...). One value per environment keeps dev referencing dev and prod referencing prod.

Do secret outputs stay secret?

Yes. A value exported from a requireSecret config or a secret resource property is stored encrypted in the upstream stack's outputs, and getOutput on it downstream returns an output still marked secret. The agent uses getOutputDetails only for plain values that must be known at deploy time, and says which.

Can it preview without AWS credentials?

It compiles every project with tsc regardless. A pulumi preview needs provider credentials and a backend that holds the upstream stacks, so the agent runs it when the sandbox has them and otherwise lists the commands for the reviewer. It never runs pulumi up.

More Pulumi jobs

More jobs like this: Hand off a long task

Keep reading