All jobs

Migrate a Python Package to PyPI Trusted Publishing

Yes. An agent rewrites your PyPI release workflow to publish with a short-lived OIDC token instead of a stored API token, keeps your checks and opens a PR.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

The agent reads the whole release path before touching it. In a package shaped like encode/httpx, .github/workflows/publish.yml runs on any tag push, in a GitHub environment named deploy, checks out with actions/checkout@v4, sets up Python 3.9 with actions/setup-python@v6, then runs three shell scripts: scripts/install (pip installs requirements.txt), scripts/build (python -m build, twine check dist/*, mkdocs build) and scripts/publish. That last script compares the version in httpx/__version__.py with GITHUB_REF and exits if the tag does not match, then runs twine upload dist/* with TWINE_USERNAME: __token__ and TWINE_PASSWORD: ${{ secrets.PYPI_TOKEN }}, and finally mkdocs gh-deploy --force. requirements.txt pins build==1.3.0 and twine==6.1.0, pyproject.toml builds with hatchling and reads the version from httpx/__version__.py, and docs/contributing.md tells maintainers that creating a GitHub release uploads to PyPI automatically, with scripts/publish as the manual fallback.

The agent then rewrites publish.yml into two jobs. A build job with contents: read runs scripts/install and scripts/build as before, keeps the tag-versus-version check as its own step before building (lifted out of scripts/publish so a mismatched tag fails before anything is uploaded), and uploads dist/ as a workflow artifact. A publish job that needs: build, runs in the deploy environment and has permissions: id-token: write downloads the artifact and publishes it with the PyPA publish action pinned to a commit, with attestations enabled. No TWINE_PASSWORD, no PYPI_TOKEN. The docs deploy becomes a third job, since mkdocs gh-deploy pushes to a branch and needs contents: write, which the publish job should not have.

It keeps scripts/publish for the manual fallback the contributing guide describes, but changes it so twine upload is only reached locally, and updates docs/contributing.md to say that releases publish through trusted publishing and that the fallback still needs a token on the maintainer's machine. It also raises actions/checkout to the same major line as the other actions in the repository and pins every action in the workflow to a commit with a version comment, because a workflow that mints OIDC tokens is the one most worth pinning.

What the agent cannot do is the PyPI side. Trusted publishing is configured on pypi.org by a project owner, naming the GitHub repository, the workflow file name publish.yml and the environment deploy. The agent writes those exact values into the pull request description as a checklist, and notes that until that step is done the first tag push fails at the publish step with an authentication error, which is the safe failure. In the sandbox it validates the workflow with actionlint, runs scripts/install and scripts/build to show the build and twine check still pass, and runs the test suite the way test-suite.yml does.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "claude",
  "input": "Migrate this package's PyPI release to trusted publishing and open a pull request. Read .github/workflows/publish.yml, .github/workflows/test-suite.yml, scripts/install, scripts/build, scripts/publish, requirements.txt, pyproject.toml and docs/contributing.md first. Rewrite publish.yml into a build job (contents: read) that keeps the current tag-versus-version check as an explicit step, runs scripts/install and scripts/build, and uploads dist/ as an artifact; a publish job that needs build, runs in the existing deploy environment with permissions id-token: write, downloads the artifact and publishes with the PyPA publish action pinned to a commit with attestations enabled, and no PyPI token anywhere; and a separate docs job with contents: write for mkdocs gh-deploy. Pin every action in the workflow to a commit SHA with a version comment. Keep scripts/publish as the documented manual fallback and update docs/contributing.md to describe the new path and the one-time PyPI setup. In the sandbox run actionlint on the workflow, run scripts/install and scripts/build to show twine check passes, and run the test suite. In the pull request description, list the exact values a project owner must enter on PyPI for the trusted publisher (repository, workflow file name, environment name) and paste the build and test output. Do not publish anything.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

What comes back is a pull request that replaces .github/workflows/publish.yml with the three-job version, edits scripts/publish so it remains a local fallback, and updates the releasing section of docs/contributing.md. The diff removes every reference to PYPI_TOKEN and TWINE_PASSWORD from CI. The description carries the PyPI checklist with the repository, publish.yml and deploy filled in, the actionlint result, the twine check output from the sandbox build and the test run.

The test suite workflow runs on the pull request as usual, since the package code is untouched. The publish workflow does not run, because it is triggered by tags only, which is why the agent validates it with actionlint and a sandbox build rather than by publishing.

A reviewer checks that the publish job has id-token: write and nothing more, that the docs job is the only one with contents: write, that the version check still runs before the build, and that the secret PYPI_TOKEN can be deleted from the repository settings once the PyPI trusted publisher is in place. Then they do the PyPI step and tag the next release.

Run it on every release workflow still using a long-lived token

An organisation with several Python packages has this workflow several times with small differences. A POST /o/{org}/sessions with this prompt and one repository in resources, sent from the terminal for each package, starts one run per repository; each run reads that package's own scripts and environment names and writes a checklist with that package's values. The sandbox parks with its disk when the turn ends, so a follow-up such as also publish to TestPyPI on workflow_dispatch wakes it with the build already done.

With a webhook_url on the API key, Rush posts run.completed or run.failed with the session_url for each run as it finishes, so the pull requests and their PyPI checklists arrive in one place.

The use case this job follows, with its automation and trigger: Hand off a long task.

Why run it on Rush?

Release workflows are edited rarely and tested only by releasing, which is why they keep long-lived tokens for years. Getting the new shape right means knowing which job needs which permission, where the artifact crosses between jobs, and that the version check must stay in front of the build. The agent works that out with the real scripts in front of it, runs the build and twine check in its own sandbox, and leaves the output in the run's record so the reviewer is not the first person to find out whether dist/ is built.

The one step a machine cannot do, registering the trusted publisher on PyPI, is handed over as an exact checklist rather than a vague instruction, so the maintainer's part takes a minute and the next tag push is the real test, failing safely if the PyPI side is not done yet.

Frequently asked questions

Does the agent publish a release to check its work?

No. The prompt forbids it, and the workflow only runs on a tag push. The agent checks the YAML with actionlint, runs scripts/install and scripts/build in the sandbox so the artifact and twine check are shown to work, and leaves the first real publish to the next tag the maintainers push.

What has to happen on PyPI, and who does it?

A project owner adds a trusted publisher on pypi.org naming the GitHub repository, the workflow file publish.yml and the environment deploy. The agent writes those values into the pull request. Until that is done, the publish job fails with an authentication error and uploads nothing.

Why split the workflow into three jobs?

Because the permissions differ. Building needs only contents: read, publishing needs id-token: write and should have nothing else, and mkdocs gh-deploy --force pushes a branch and needs contents: write. One job would have to hold all three at once, which is the opposite of what the migration is for.

Is scripts/publish deleted?

No. The contributing guide names it as the fallback when the CI job fails, and that remains true, with a token on the maintainer's machine. The agent keeps the script, removes the CI-only branch that read GITHUB_REF, since the version check now lives in the workflow, and updates the guide to match.

Why pin the actions to commits in this workflow only?

The agent pins the actions in the file it rewrites, because this workflow is the one that mints an OIDC token and uploads to PyPI, so a compromised tag on an action it uses would be the worst case. Pinning the other workflows is a separate change the pull request mentions but does not make.

More Python jobs

More jobs like this: Hand off a long task

Keep reading