What the agent does
The agent reads the workflow before the annotation, since the fix is per action and per major. In a repository like mailcheck/mailcheck, .github/workflows/ci.yml has a compatibility job on a Node 22 and 24 matrix that uses actions/checkout@v4 with persist-credentials: false, actions/setup-node@v4 with cache: npm, then npm ci --ignore-scripts --no-audit --no-fund, npm run build:check, npm test, npm run test:types and npm run test:package; a browser job that installs Playwright and uses actions/upload-artifact@v4 on failure with retention-days: 7; and a notify job with no actions at all. The workflow already sets permissions: contents: read and a concurrency group.
The annotation on such a run reads Node.js 20 actions are deprecated. The following actions are running on Node.js 20 and may not work as expected: followed by the action references, then Please check if updated versions of these actions are available that support Node.js 24. GitHub's changelog says runners started defaulting to Node 24 on June 16, 2026 and removed Node 20 on September 23, 2026, so a v4 action now runs on a Node it was not released for. The agent confirms which majors are affected by reading each tag's action.yml: checkout@v4, setup-node@v4 and upload-artifact@v4 all declare runs.using: node20.
Then it reads the release notes of every major in between, because the first Node 24 major is not always the right stop. checkout v5 moved to Node 24 and needs runner 2.327.1 or newer; setup-node v5 moved to Node 24 and turned on automatic package-manager caching, which package-manager-cache: false disables, and v6 limited that automatic caching to npm; upload-artifact v5 added preliminary Node 24 support but still ran on Node 20, and v6 is the first to declare node24. The current majors, checkout v7, setup-node v7 and upload-artifact v7, all declare node24, and setup-node v7 is an ESM rewrite with new cache outputs. Since this workflow sets cache: npm explicitly, none of the caching changes alters it.
The change is three uses: lines, each moved to the current major, with every with: block kept. The agent cannot run GitHub's runner in its sandbox, so it proves what it can: npm ci --ignore-scripts --no-audit --no-fund installs the package, npm run build:check prints Minified build is current., npm test passes under the Node test runner, npm run test:types compiles both type specs, and npm run test:package reports that the packed consumer checks for CommonJS, ESM, minified imports and jQuery types passed. The pull request's own workflow run on GitHub is the final check: its annotations are where the deprecation would reappear.
Start it with one API call
POST https://api.phoenix.so/o/{org}/sessions with this body:
{
"agent": "claude",
"input": "Our workflow runs are annotated \"Node.js 20 actions are deprecated\". Read every workflow under .github/workflows, list each action reference, and read the action.yml at each pinned tag to find which declare runs.using node20. For each one, read the release notes of every major between the pinned one and the current one and note any breaking change, especially around caching and runner version. Move each to the current major, keep every with: block, and run the workflow's own install, build and test commands locally. Open a pull request listing each bump with the release notes you read and the local command output, and say that the pull request's own workflow run is where the annotation should no longer appear.",
"resources": [
{
"url": "https://github.com/your-org/your-app"
}
]
}- Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
- Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
- Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.
What comes back
A pull request that changes three uses: lines in ci.yml and nothing else. The description has one row per action: the pinned major, the runs.using it declares, the current major, and the release-note lines for each major in between, with the caching behaviour change in setup-node v5 and v6 and the runner version each Node 24 release requires.
A reviewer checks that persist-credentials: false, cache: npm, the node-version matrix expression and the artifact retention-days are unchanged, then reads the pull request's own workflow run on GitHub. A run with no deprecation annotation and green jobs is the proof the sandbox cannot produce.
The description also quotes the local run of build:check, test, test:types and test:package, so the reviewer knows the project itself is fine on the Node the matrix uses, independent of the action bumps.
Run it on every workflow run with a deprecation annotation
The annotation appears on every run until the actions move, so the trigger is a run. A handler on your server that receives GitHub's workflow-run webhook and reads the run's annotations starts this request with POST /o/{org}/sessions, sending a UUID v5 of the repository name as request_id so one pull request results no matter how many runs report it.
For an org-wide sweep, your server sends this request once per repository that has a .github/workflows directory, each with its own UUID as request_id, and reads the run.completed webhooks; a key starts at most max_concurrent runs at once and queues the rest.
The use case this job follows, with its automation and trigger: Fix a bug from an alert.
Why run it on Rush?
Bumping a major without reading its notes is how a caching default or a runner requirement slips into a two-line pull request. In a Rush sandbox the agent fetches each tag's action.yml and each release's notes, records what it read, and keeps the record with the run, so the pull request carries the evidence for every bump rather than a changelog link.
The repository's own test commands are the part of the workflow the sandbox can run. The agent runs them on the Node version the matrix names and quotes their output, so a failure on GitHub after the bump points at the action change and not at the project.
Frequently asked questions
Which action versions still run on Node 20?
The ones whose action.yml declares runs.using: node20. For the three actions in this workflow that is every v4 release, and for upload-artifact also v5, which added Node 24 support but kept Node 20 as its default. The agent reads the file at each pinned tag rather than assuming.
Does moving to the latest major change caching?
setup-node v5 turned on automatic package-manager caching and v6 limited it to npm, so a workflow that relied on the old default would change. A workflow that sets cache: npm explicitly, as this one does, keeps the same behaviour across the bump.
Why can't the sandbox prove the annotation is gone?
The annotation is produced by GitHub's runner, which the sandbox does not run. The agent proves what it can, the project's own commands on the matrix's Node, reads runs.using at each new tag, and names the pull request's own workflow run as the place to check.
What about self-hosted runners?
Every Node 24 action release names a minimum runner, 2.327.1 for the three here. GitHub-hosted runners meet it; the pull request lists the requirement so an org with self-hosted runners updates them first.