# Remove six and Python 2 Compatibility Code From a Python Project

Yes. An agent replaces every six import with its Python 3 equivalent, runs the tests on a current interpreter and reports what still needs six.

[Run this on Rush](https://cloud.getrush.ai/run/remove-six/six/). Opens the console with this prompt ready to send, in a Project that has your repository.

## What the agent does

Removing `six` is two jobs that look like one: rewriting imports, and admitting in the packaging that Python 2 is gone. The agent reads the packaging first. In a project like coursera-dl/edx-dl, `setup.py` still carries the classifiers `Programming Language :: Python :: 2` and `2.7`, `.github/workflows/pythonapp.yml` tests `[2.7, 3.5, 3.6, 3.7, 3.8]`, `tox.ini` lists `py27,py34,pypy`, and the `Dockerfile` starts from `python:3.4-slim`. None of that can stay if the compatibility layer goes, so the pull request changes the classifiers, adds a `python_requires`, and rewrites the matrix. The agent states the new floor it chose and why; your prompt can fix it instead.

The imports are mechanical. `edx_dl/edx_dl.py` takes `CookieJar` from `six.moves.http_cookiejar` and its URL helpers from `six.moves.urllib.error`, `.parse` and `.request`; these become `http.cookiejar` and `urllib.error`, `urllib.parse` and `urllib.request`. `test_utils.py` loops with `six.iteritems(strings)` three times, which becomes `strings.items()`. `edx_dl/utils.py` and `edx_dl/parsing.py` both have `from six.moves import html_parser`, and that one is not mechanical.

Those two files call `html_parser.HTMLParser().unescape(...)`, once in `clean_filename` and twice on the metadata string in `parsing.py`. On Python 3, `six.moves.html_parser` is the standard `html.parser` module, and its `HTMLParser.unescape` method was removed in Python 3.9. So before the agent changes anything, it runs the suite on a current interpreter and records the failure: `AttributeError: 'HTMLParser' object has no attribute 'unescape'`, in every test that reaches a filename or a metadata block. The package has been broken on modern Python with `six` installed and working. The fix is `html.unescape(s)`, a function that has existed since Python 3.4, and the agent shows the same tests passing after it.

The last step is the one a search-and-replace cannot do: find out whether `six` can leave the environment. The agent deletes it from `requirements.txt` and from the `deps` in `tox.ini`, resolves the remaining requirements, and reads what still asks for it. Here `html5lib>=1.0.1` does, since html5lib 1.1 declares `six>=1.9`. That matters more than it looks. `parsing.py` calls `BeautifulSoup(page)` four times without naming a parser, so Beautiful Soup uses the best one it can import. With `six` forced out, `import html5lib` raises `ModuleNotFoundError`, Beautiful Soup falls back to `html.parser` without an error, and the tests still pass on a different parser. The agent leaves `six` installed as html5lib's dependency and says so.

## Start it with one API call

`POST https://api.phoenix.so/o/{org}/sessions`

```json
{
  "agent": "claude",
  "input": "Remove the six compatibility library from this Python project's own code. Read setup.py or pyproject.toml, the requirements files, tox.ini, the CI workflows and the Dockerfile, and list every place that still declares or tests Python 2. Unless a file says otherwise, set the minimum to the oldest Python 3 release that still receives security fixes, state it, and update the classifiers, python_requires, tox envlist and CI matrix to match. Before editing any code, create an environment on a current Python release, install the requirements and run the existing tests; record every failure with its exception. List each six use: six.moves imports, iteritems, itervalues, string_types, text_type, PY2 or PY3 branches, add_metaclass, and any from __future__ import. Replace each with the Python 3 form. Where a six.moves name maps to a standard library attribute that later Python versions removed, such as HTMLParser.unescape, use the current replacement and say so. Leave __future__ imports alone and report them. Then remove six from the project's requirement files, resolve the remaining requirements and print which installed packages still require six; do not uninstall it if another dependency needs it, and explain what would change if it were removed. Run the tests again on the oldest and newest supported versions and show before and after. Change no behaviour beyond what the port requires. Open a pull request.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
```

1. **Get an API key.** In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
2. **Send the request.** POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
3. **Read what came back.** Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

## What comes back

A pull request with the import changes in `edx_dl/edx_dl.py`, `edx_dl/utils.py`, `edx_dl/parsing.py` and `test_utils.py`, the `html.unescape` fix, and the packaging edits in `setup.py`, `requirements.txt`, `tox.ini` and the workflow. Its description has a before and after for the test suite on a current Python: the `AttributeError` failures first, the passing output second. A table maps each `six` name to what replaced it.

A separate section answers whether `six` is gone. For this repository the answer is no: it is out of the project's own imports and requirement lines, and still installed because html5lib requires it. The description quotes the resolver output that shows this and notes the four `BeautifulSoup(page)` calls that pick their parser by what is importable, as a risk to fix by naming the parser.

The reviewer checks the chosen Python floor against who still installs the tool, and checks that `.travis.yml` and the `Dockerfile` were either updated or listed as stale. The agent also reports what it saw and left alone, such as the regular expression in `parsing.py` that Python now warns about for its `\d` in a non-raw string.

## Run it on every Monday morning

Compatibility code comes out best in small, regular pieces. Add an automation to the Project with a `schedule` trigger, a `cron` of `0 7 * * 1` and your `timezone`, and save the prompt above. Each Monday a run starts with that prompt and opens a pull request if anything is left to remove. After the pull request merges, the next run finds no `six` imports, changes nothing and says so in its answer.

To keep the weekly run useful after that, widen the prompt's first sentence to name the next layer, for example `future` or `mock`, and keep the rest. A schedule fires at most hourly, and turning the automation back on after a pause moves it to its next slot instead of firing the Mondays it missed.

## Why run it on Rush?

The useful discoveries in this job only appear when the code is executed on an interpreter newer than the one it was written for. The run creates that interpreter in its own sandbox, runs the suite before touching a line, and keeps the failing output. That is how a removed standard library method shows up as a fact in the pull request and not as a bug report from a user three weeks later.

The dependency question is settled the same way. Resolving the requirements and then importing each package with `six` absent takes a throwaway environment, which is what a sandbox is. The reviewer gets the resolver's own words on why `six` stays installed, next to a diff small enough to read in one sitting.

## Frequently asked questions

### What replaces six.moves.urllib?

The standard library modules it pointed to on Python 3: `urllib.request` for `urlopen`, `Request` and the opener classes, `urllib.parse` for `urlencode` and `urlparse`, and `urllib.error` for `HTTPError` and `URLError`. `six.moves.http_cookiejar` is `http.cookiejar`. The names inside those modules are unchanged.

### Why does HTMLParser().unescape fail after the port?

It fails before the port too, on Python 3.9 or newer. The method was deprecated in Python 3.4 and removed in 3.9, and `six.moves.html_parser` only forwards to the standard module. Use the function `html.unescape`, which takes the string and returns it with entities decoded.

### Can I uninstall six once my code no longer imports it?

Only if nothing else you install needs it. Libraries such as html5lib 1.1 still declare it, so a resolver puts it back. The agent removes it from your own requirement lines and prints which packages keep it in the environment.

### Does the agent drop Python 2 without asking?

It treats removing `six` as the decision to drop Python 2, because the two cannot be separated, and it makes every packaging change that follows from it in the same pull request. If the prompt names a floor, it uses that. Nothing merges without your review, and the description lists each file that declared Python 2.

## More jobs like this: Upgrade dependencies weekly

- [Replace passlib in a FastAPI App Without Breaking Stored Hashes](https://getrush.ai/run/replace-passlib/passlib/)
- [Update Gin Dependencies Every Week](https://getrush.ai/run/update-dependencies/gin/)
- [Update GitHub Actions in a CMake Project's CI](https://getrush.ai/run/update-github-actions/cmake/)
- [Update the GitHub Actions in a Cobra CLI's Workflow](https://getrush.ai/run/update-github-actions/cobra/)
- [Upgrade Pinned GitHub Actions to the Latest Releases](https://getrush.ai/run/update-github-actions/github-actions/)
- [Update the GitHub Actions in a Gradle Build's Workflow](https://getrush.ai/run/update-github-actions/gradle/)

## Keep reading

- [How a Rush run works on your repository, from request to pull request](https://getrush.ai/cloud/)
- [Compare cloud agents, agent sandboxes and AI code review](https://getrush.ai/compare/)
- [Rush pricing: Free, Pro and Max](https://getrush.ai/pricing/)
