All jobs

Add a Release Workflow to a Custom GitHub Action

Yes. An agent adds a workflow that runs on a published release, checks the tag against package.json and dist/, moves the major tag like v1, and opens a PR.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

The agent reads how the action is built and shipped before it writes a workflow. In an action bootstrapped from actions/typescript-action, action.yml declares one input, milliseconds with a default of '1000', one output, time, and runs with using: node24 and main: dist/index.js. package.json is at version 0.0.0, type: module, with @actions/core as the only runtime dependency and scripts for bundle (format then package, which runs rollup from rollup.config.ts into dist/index.js with a source map), ci-test (Jest under --experimental-vm-modules), lint, coverage and all. .node-version pins 24.4.0. Five workflows exist under .github/workflows/: ci.yml runs format, lint and tests and then uses the action from ./ with milliseconds: 2000; check-dist.yml deletes dist/, rebuilds it with npm run bundle and fails if git diff shows a change; codeql-analysis.yml, linter.yml and a licensed.yml that only runs on workflow_dispatch. .github/dependabot.yml groups weekly npm and actions updates.

Releasing is a shell script. script/release reads the latest v* tag, prompts for a new vX.X.X, asks the maintainer to confirm package.json was bumped, creates the annotated tag, then either creates a new major tag vN and a releases/vN branch or force-moves the existing vN tag with git tag --force and git push --force, and README.md explains that the maintainer then creates a GitHub release by hand. That is the sequence the GitHub Actions toolkit's versioning guide describes (a release per version, a major tag moved to the latest release), but every step depends on one person's terminal, and nothing checks that the commit being tagged has a dist/index.js that matches its sources, which is what uses: owner/action@v1 actually runs.

The agent adds .github/workflows/release.yml, triggered by release with types: [published], with permissions: contents: write and nothing else. Its first job checks out the release tag, sets up Node from .node-version with npm caching, runs npm ci, and verifies three things in order: the tag matches ^v[0-9]+\.[0-9]+\.[0-9]+$, the version field in package.json equals the tag without its v, and npm run bundle followed by git diff --ignore-space-at-eol --text dist/ produces no output, the same comparison check-dist.yml makes, so a tag whose bundle is stale fails here instead of in every consumer's workflow. It then runs npm run ci-test. The second job, needing the first, computes the major tag from the release tag, moves it with git tag -fa v1 -m "Update v1 tag" and git push origin v1 --force, the two commands the toolkit guide gives, using the workflow's GITHUB_TOKEN, and skips with a notice when the release is marked as a prerelease so a v2.0.0-beta never moves v2.

A release event cannot be fired from a pull request, so the agent proves each step in its sandbox instead. It runs npm ci, npm run ci-test and npm run bundle and shows git diff dist/ is empty on the current sources; runs the version comparison as a shell snippet against 0.0.0 with a matching tag and a mismatching one to see it pass and fail; creates a throwaway tag in a local clone and runs the major-tag commands against a local bare remote to show v0 moves and that a prerelease tag is skipped; and runs actionlint against the new file, since the repository ships an actionlint.yml and its linter.yml lints workflows. It also updates script/release to stop moving the major tag itself and to say the workflow does it after the release is published, and rewrites the README's release section as three steps: bump package.json, tag and push with the script, publish the release.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "claude",
  "input": "Add a release workflow to this GitHub Action and open a pull request. Read action.yml, package.json, .node-version, rollup.config.ts, script/release, README.md, actionlint.yml and every workflow under .github/workflows first. Write .github/workflows/release.yml triggered on release published with contents: write as its only permission. First job: check out the release tag, set up Node from .node-version with npm cache, npm ci, fail unless the tag is vX.Y.Z, fail unless the package.json version equals the tag without v, run npm run bundle and fail if git diff on dist/ shows any change, then run npm run ci-test. Second job, after the first: compute the major tag from the release tag, move it with git tag -fa and git push --force using GITHUB_TOKEN, and skip with a notice when the release is a prerelease. Update script/release so it no longer moves the major tag and says the workflow does, and rewrite the README release section as bump, tag, publish. Do not change ci.yml or check-dist.yml. In the sandbox run npm ci, npm run ci-test and npm run bundle with git diff dist/, run the version check with a matching and a mismatching tag, exercise the major-tag step against a local bare remote with a normal and a prerelease tag, and run actionlint on the new workflow. Paste every result in the pull request and say plainly that the workflow has not yet run on a real release.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

What comes back is a pull request with one new file, .github/workflows/release.yml, a smaller script/release, and a rewritten release section in README.md. action.yml, src/, dist/, ci.yml and check-dist.yml are untouched. The description walks the workflow step by step, names contents: write as its only permission, and says in plain words that it has not run on a release yet because no release exists on the branch.

The run's record carries what was verified: npm ci, npm run ci-test and npm run bundle with an empty git diff dist/, the version check passing and failing, the local major-tag rehearsal showing v0 moved for a normal tag and skipped for a prerelease, and a clean actionlint run. The existing ci.yml, check-dist.yml and linter.yml run on the pull request as they do on every pull request to main.

Before merging, a reviewer confirms the trigger they want (a published release, rather than a pushed tag, so a draft never moves v1), that a prerelease should indeed be skipped, and that GITHUB_TOKEN is allowed to push tags in the repository's Actions settings. After merging, they publish a release for the next tag and watch the major tag move.

Run it on every new action repository

A team that owns several actions starts one run per repository from the terminal: POST /o/{org}/sessions with this prompt and the repository in resources. GET /o/{org}/github lists every repository the Rush GitHub App reaches, so a script can pick the ones with an action.yml and no release.yml and start them in a loop, each in its own sandbox on its own branch.

Set a webhook_url on the key and each run POSTs run.completed with its session_url when the turn ends, so the pull request links collect without polling. Once merged, the workflow runs on every published release with no further run.

The use case this job follows, with its automation and trigger: Hand off a long task.

Why run it on Rush?

A release workflow fails on release day or not at all, usually because a tag went out with a stale dist/index.js or the major tag moved to a prerelease. On Rush the agent rehearses every step in its own sandbox: the bundle diff, the version check both ways, and the tag move against a local remote, and keeps each command and its output in the run's record, so the first real release runs steps that have already been seen to work.

The sandbox has the repository, Node 24 and git, and leaves nothing behind on a maintainer's machine, including no stray tags. The same request works on every action a team maintains, and the pull request says exactly what could only be proven by a real release.

Frequently asked questions

Why trigger on a published release rather than a pushed tag?

The README already asks the maintainer to create a GitHub release after tagging, and the toolkit's versioning guide recommends a release per version. Running on release: published means a draft or a deleted tag never moves the major tag, and the release page is the one place a version is announced.

Does this replace check-dist.yml?

No. check-dist.yml keeps running on every pull request to main. The release workflow repeats the same npm run bundle and git diff dist/ comparison on the tagged commit, because that is the commit consumers run.

What happens to script/release?

It still validates the version and creates and pushes the annotated tag, which is the part that needs a person. Moving the major tag moves to the workflow, so the script no longer force-pushes anything.

Can the workflow push a tag with GITHUB_TOKEN?

Yes, with contents: write, provided the repository's Actions settings allow the token write access and no tag protection rule blocks it. A tag moved by the token does not trigger other workflows, which is the behaviour you want here.

More GitHub Actions jobs

The same job on other stacks

Keep reading