# Migrate From boto to boto3 in a Python Service

Yes. An agent ports S3 code from boto 2 to boto3, compares the requests both libraries send to a local stub, and opens a pull request.

[Run this on Rush](https://cloud.getrush.ai/run/migrate-boto-to-boto3/boto/). Opens the console with this prompt ready to send, in a Project that has your repository.

## What the agent does

boto 2 has a hard stop date built into Python itself. Its last release, 2.49.0, is from July 2018, its repository is archived, and on Python 3.12 `import boto` fails with `ModuleNotFoundError: No module named 'boto.vendored.six.moves'`. On 3.11 the same import works, so the break arrives with the interpreter upgrade and not before. The agent first finds every boto call. In a service like propublica/politwoops-tweet-collector, `requirements.txt` pins `boto==2.45.0` and one file uses it: `bin/screenshot-worker.py` imports `S3Connection` and `Key`, and a single method, `upload_image`, does the work for both screenshots and tweet images.

That method is short and nearly every line of it carries a decision. It reads `access_key`, `secret_access_key`, `bucket_name` and `url_prefix` from the `[aws]` section of the worker's ini file and passes the keys to `S3Connection`. It calls `conn.create_bucket(bucket_name)` on every upload to get a bucket object. It uploads with `key.set_contents_from_filename(tmp_path, policy='public-read', headers={'Content-Type': content_type, 'Max-Age': 604800})`, returns a URL it builds by hand as `http://s3.amazonaws.com/<bucket>/<path>`, and catches only `IOError`. The caller stores that URL in MySQL.

The worker cannot run whole in a sandbox: it needs beanstalkd, MySQL and a `phantomjs` binary, and there are no tests. So the agent isolates the upload. It starts a small HTTP server on localhost that accepts S3-style `PUT` requests and logs the path and headers, points boto at it, and runs the method's exact calls on Python 3.11. The log shows two requests for one image. The first is `PUT /<bucket>/`, the bucket creation, sent every time. The second is the object, with `x-amz-acl: public-read`, the content type, and a header literally named `Max-Age`, which is not a caching header. S3 takes caching instructions from `Cache-Control`, so by Amazon's documentation the week of caching this code asks for is not being applied. Both requests are signed with the older `AWS key:signature` scheme.

The boto3 version is one call, `upload_file(tmp_path, bucket_name, dest_path, ExtraArgs={...})`, and the same server shows one `PUT`, signed with `AWS4-HMAC-SHA256`. Two details need a human decision and the agent raises both. boto3 refuses unknown upload arguments, so `Max-Age` cannot be carried over as it is: `ExtraArgs={'Max-Age': ...}` raises `ValueError: Invalid extra_args key 'Max-Age'`. The agent sends `CacheControl='max-age=604800'`, which is what the original meant, and flags it as a behaviour change, since objects will now be cached. And the error handling changes shape. A missing local file still raises an `IOError` subclass, but a failed upload raises a boto3 or botocore exception that the existing `except IOError` does not catch, so the agent widens that clause to keep the method returning `None` on failure as its caller expects.

## Start it with one API call

`POST https://api.phoenix.so/o/{org}/sessions`

```json
{
  "agent": "claude",
  "input": "Port this Python project from boto (version 2) to boto3. Read the requirements files and every module that imports boto. For each use, write down the connection arguments and where credentials come from, every bucket and key call with its arguments, headers and ACL, what the function returns, and which exceptions it catches. Confirm on the Python versions you can install whether `import boto` works. Do not use real AWS credentials. Start a local HTTP server that accepts S3-style requests and logs method, path and headers; run the existing boto calls against it and save the log. Port to boto3 with the same credential source passed explicitly, the same bucket, key, ACL and content type. Do not create the bucket on each upload. Where the old code sent a header boto3 will not accept, say what the header did, choose the closest supported argument, and flag it as a behaviour change. Keep return values, including any URL the code builds and stores, exactly as they are. Make sure every failure the old except clause handled is still handled, and list the boto3 exception types involved. Run the boto3 version against the same server and show both request logs in the pull request. State which Python versions the new dependency supports and whether the rest of the project runs on them.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
```

1. **Get an API key.** In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
2. **Send the request.** POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
3. **Read what came back.** Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

## What comes back

A pull request that changes one method in `bin/screenshot-worker.py`, its two imports, and one line of `requirements.txt`. The description shows the two request logs next to each other: two requests against one, the old signature scheme against the new, `Max-Age` against `Cache-Control`. It states that the stored URL format is untouched, so rows already in the database and rows written after the change look the same.

Three notes are for the person who owns the AWS account, because a local stub cannot answer them. The first is permissions: the worker's credentials no longer need to create buckets, and the bucket must already exist. The second is the ACL. The upload still asks for `public-read`, and Amazon's documentation says buckets created with the current defaults have ACLs disabled and reject such a request, so a move to a new bucket would need a bucket policy instead. The third is the region. boto3 signs for a region, and the ini file has no region setting, so the agent adds one with a default and says which.

The description also reports what the port does not solve. The current boto3 requires Python 3.10 or newer, which suits a 3.12 upgrade, but three other scripts under `bin/` are still Python 2 syntax and fail to compile on any Python 3. They are outside this change and are listed so nobody assumes the repository as a whole is ready.

## Run it on every repository on your Python 3.12 upgrade list

boto tends to survive in the oldest corners of a codebase: cron jobs, backup scripts, workers nobody has deployed in a year. When a Python upgrade is planned, search for `boto==` and `import boto` across your repositories and send the body above to `POST /o/{org}/sessions` once for each hit. Use a `request_id` that is a UUID v5 of the repository name, so a second pass of the script returns the earlier runs instead of starting new ones.

With a `webhook_url` set on the key, each run posts `run.completed` or `run.failed` when it ends. Runs past the key's `max_concurrent` wait as `queued` and start in order.

## Why run it on Rush?

Code that talks to S3 is risky to test where real credentials are within reach. A laptop often has an AWS profile configured, and the quickest manual test of an upload function is to upload something. The agent's sandbox has no AWS access unless you give it one, and the prompt tells it to prove the port against a server it starts itself. What that server logs is better evidence than a successful real upload would be, because it shows the exact headers each library sent.

It is also the only way most teams will ever see what their boto code does on the wire. Few people reading `headers={'Max-Age': 604800}` would stop to ask whether S3 knows that header. A request log makes it obvious, and puts the choice between fixing it and preserving it in front of a reviewer.

## Frequently asked questions

### Why does import boto fail on Python 3.12?

boto bundles an old copy of the `six` compatibility library, and the way that copy registers its `moves` module stopped working with Python 3.12's import system. The error is `No module named 'boto.vendored.six.moves'`. No boto release fixes it; the last one came out in 2018.

### Can boto and boto3 be installed side by side during the migration?

Yes. They are separate packages with separate import names, and a large project can port one module per pull request. They only have to share an interpreter that boto still imports on, which means Python 3.11 or older until the last module is done.

### What replaces Key.set_contents_from_filename?

`upload_file` on the S3 client or on a `Bucket` resource. The ACL moves from `policy='public-read'` to `ExtraArgs={'ACL': 'public-read'}`, and content type and caching become `ContentType` and `CacheControl` in the same dictionary. Arbitrary HTTP headers are not accepted there.

### Do I still need to call create_bucket or get_bucket first?

No. boto3's client calls take the bucket name as an argument and make no request until the operation itself. Code that called `create_bucket` only to obtain a bucket object, as this worker does, loses a request per upload and no longer needs permission to create buckets.

### Will my S3 URLs change?

Only if your code asks the library for them. This worker formats its own URL string and stores it, so the agent leaves it alone. Where code used `generate_url`, the agent ports it to `generate_presigned_url` and shows an old and a new URL, because the query parameters differ between the two signature versions.

## More jobs like this: Upgrade dependencies weekly

- [Convert a CoffeeScript Project to Modern JavaScript With decaffeinate](https://getrush.ai/run/migrate-coffeescript-to-javascript/coffeescript/)
- [Migrate a Go Project From dep (Gopkg.toml) to Go Modules](https://getrush.ai/run/migrate-dep-to-go-modules/dep/)
- [Migrate From drf-yasg to drf-spectacular in Django REST Framework](https://getrush.ai/run/migrate-drf-yasg-to-drf-spectacular/drf-yasg/)
- [Migrate From faker to @faker-js/faker in a Node.js Project](https://getrush.ai/run/migrate-faker-to-faker-js/faker/)
- [Migrate From Flask-Cache to Flask-Caching](https://getrush.ai/run/migrate-flask-cache-to-flask-caching/flask-cache/)
- [Migrate From Flask-JWT to Flask-JWT-Extended](https://getrush.ai/run/migrate-flask-jwt-to-flask-jwt-extended/flask-jwt/)

## Keep reading

- [How a Rush run works on your repository, from request to pull request](https://getrush.ai/cloud/)
- [Compare cloud agents, agent sandboxes and AI code review](https://getrush.ai/compare/)
- [Rush pricing: Free, Pro and Max](https://getrush.ai/pricing/)
