All jobs

Fix "No module named 'distutils'" on Python 3.12

Yes. An agent reproduces the import error on Python 3.12, replaces each distutils call with a supported equivalent and shows the tool starting again.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

Python 3.12 removed distutils from the standard library, so any from distutils... line that runs at import time stops the program with ModuleNotFoundError: No module named 'distutils'. The agent does not start by searching and replacing. It starts by making the error happen, because where it happens decides the fix. In a command-line tool like hellofresh/eks-rolling-update the import is in runtime code, not in packaging: eksrollup/config.py begins with from distutils.util import strtobool and uses it in a one-line helper, str_to_bool, that parses seven environment variables such as DRY_RUN, TAINT_NODES and K8S_AUTOSCALER_ENABLED when the module loads. setup.py here is clean; it imports setuptools only.

The agent creates a Python 3.12 environment, installs the runtime requirements setup.cfg declares (boto3, kubernetes, python-dotenv, each with only a lower bound), and runs eks_rolling_update.py -h. That fails on line 2 of config.py. It then explains why nobody saw it sooner, from the repository's own files. python_requires is >=3.7, so pip installs the package on 3.12 without complaint. Both workflows pin Python 3.7. The Dockerfile builds from python:3-alpine3.10, a tag that reads as the latest Python 3 but stopped moving when that Alpine release did, so the image still carries an interpreter that has distutils.

There is a second reason these bugs hide, and the agent tests for it. A recent setuptools installs a shim that makes import distutils work again by pointing it at a copy inside setuptools. An environment that contains setuptools for any reason, for example as a dependency of some other package, runs this tool fine on 3.12, and one without it does not. So the agent reproduces the failure in an environment with no setuptools, and checks the fix there too. Adding setuptools to install_requires would make the error go away and is not the fix it takes.

The replacement depends on the function. For strtobool there is no standard library successor, so the agent writes the logic into str_to_bool itself with the same rules: y, yes, t, true, on and 1 are true, n, no, f, false, off and 0 are false, case is ignored, and anything else raises ValueError. It keeps the error on purpose. DRY_RUN=maybe should stop a tool that replaces cluster nodes, not quietly mean false. For other imports it uses the usual mappings: shutil.which for distutils.spawn.find_executable, packaging.version for LooseVersion and StrictVersion, shutil.copytree for dir_util.copy_tree, sysconfig for distutils.sysconfig, and setuptools for distutils.core.setup. Then it runs the test target on a Python version the pinned test requirements still install on, before and after, and the entry point on 3.12.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "claude",
  "input": "This Python project fails on Python 3.12 with ModuleNotFoundError: No module named 'distutils'. Fix it without depending on setuptools at run time. First find every import of distutils in the repository and sort them into packaging code (setup.py, build scripts) and code that runs when the package is imported or used. Read setup.py, setup.cfg or pyproject for python_requires and classifiers, the CI workflows for the Python versions tested, and any Dockerfile for its base image, and say which Python each of those actually runs. Reproduce the error: create a Python 3.12 virtual environment that has no setuptools in it, install the project's declared runtime requirements, and run its entry point with --help; paste the traceback. Check whether any installed dependency pulls in setuptools, because its distutils shim hides the error, and say what you found. Then replace each use: distutils.util.strtobool with a local function that accepts exactly the same strings and still raises ValueError on anything else; distutils.spawn.find_executable with shutil.which; LooseVersion and StrictVersion with packaging.version, noting any version strings in the code or tests that packaging rejects; distutils.dir_util.copy_tree with shutil.copytree(dirs_exist_ok=True); distutils.sysconfig with sysconfig; distutils.core.setup and Extension with setuptools. Add a dependency only when a replacement needs one, in every file that lists requirements. Add unit tests for any helper you write, covering every accepted spelling, mixed case, the empty string and an invalid value. Run the existing test command before and after on a Python version the pinned test requirements install on, and report failures that are present both times separately. Run the entry point again on 3.12 without setuptools. If the pinned requirements themselves cannot be installed or imported on 3.12, report that with the error and do not change the pins. Open a pull request.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

A pull request that changes eksrollup/config.py and adds a test module for str_to_bool. The import line is gone, the helper is a few lines longer, and no requirement file changes because the replacement needs no package. The description shows the traceback from the 3.12 environment before the change and the usage text from eks_rolling_update.py -h in the same environment after it.

The reviewer reads the helper against the list of accepted strings and checks the new tests cover the awkward cases: the empty string, which raises as it did before, a value with surrounding spaces, and the non-string default. os.getenv('DRY_RUN', False) returns the boolean False when the variable is unset, and the helper passes a boolean through untouched, so that branch must survive the rewrite.

The description also lists what the agent found and left alone. requirements.txt and requirements-tests.txt pin 2019 releases of boto3, kubernetes and moto. With those pins the unit tests cannot even be imported on 3.12, where an old vendored library fails with ImportError: cannot import name 'Mapping' from 'collections'. The tests are therefore run on an older interpreter, with the same result before and after, and any failure present in both runs is named as pre-existing. Raising the pins, the CI version and the Dockerfile base tag is a larger change that the pull request recommends and does not make.

Run it on every issue that mentions distutils

Users report this error the day their distribution or CI image moves to Python 3.12, usually as an issue with the traceback pasted in. Add an automation to the Project with this prompt and a github trigger on issues.labeled, filtered to a label such as python-3.12. A trigger fires only for work by an owner, member or collaborator, so an outside reporter's issue starts nothing by itself: a maintainer adds the label and the run starts. The issue is added after the prompt as event data, so the agent sees the reporter's traceback and environment; the saved prompt stays the only instruction.

For a one-off sweep, call POST /o/{org}/sessions with this body for each repository where a search finds distutils outside setup.py. Each answers 202, and a request_id that is a UUID v5 of the repository name keeps a rerun from starting duplicates.

The use case this job follows, with its automation and trigger: Upgrade dependencies weekly.

Why run it on Rush?

Whether this bug shows up depends on what else is installed, which makes a developer's own machine a poor witness: an old virtual environment, a globally installed setuptools or a 3.11 interpreter all make the tool appear to work. The run builds its environments from nothing, one with setuptools absent on purpose, and the transcript keeps the traceback from the broken one and the help text from the fixed one.

It also needs more than one interpreter. The failure is on 3.12 and the pinned test suite only loads on an older release, so the agent installs both in its sandbox and reports each result against the version that produced it. A reviewer gets a pull request where every claim about a Python version has the command output behind it.

Frequently asked questions

Can I just install setuptools to fix it?

It often works, because setuptools ships a distutils shim, and for a setup.py that is the right answer. For code that runs in production it leaves you depending on a compatibility layer inside a build tool. Replacing the few functions you use is small and removes the dependency for good.

What replaces distutils.util.strtobool?

Nothing in the standard library. Write a short function that maps y, yes, t, true, on, 1 to true and n, no, f, false, off, 0 to false after lowercasing, and raises ValueError otherwise. Keep the exception; code that calls it may rely on bad configuration failing loudly.

Why does it work in our Docker image but not for a user who ran pip install?

The image pins an interpreter, sometimes without meaning to: a tag that names an old base distribution stops receiving new Python releases. A user installing with pip gets whatever Python they have, and if your package metadata allows 3.12 nothing warns them. The agent reads the base tag and python_requires and reports the gap.

Will it also upgrade my pinned dependencies so the tests run on 3.12?

Not in this run. The prompt keeps the change to the distutils imports and tells the agent to report pins that block 3.12 with the error they produce. That list is the starting point for a separate dependency upgrade, which has its own risks and deserves its own review.

More jobs like this: Upgrade dependencies weekly

Keep reading