All jobs

Fix "ImportError: cannot import name 'mock_s3' from 'moto'" in a pytest suite

Yes. An agent replaces the per-service moto decorators with mock_aws, fixes what moto 5 now validates, and runs the test suite to prove it.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

The agent starts from where the test run stops. In MITLibraries/ppod, a small AWS Lambda that posts MARCXML files from an S3 bucket to a remote service, conftest.py opens with from moto import mock_s3, mock_ssm on line 6. Two session-scoped fixtures use them: mocked_s3 enters with mock_s3(): on line 45 and creates the buckets ppod, empty_tar, no_files and a_lot_of_files, the last one with 1001 objects under upload/, and mocked_ssm enters with mock_ssm(): on line 75 and puts the parameter /apps/ppod/stream-name. The Pipfile lists moto = "*" under dev-packages with no bound, the Pipfile.lock holds moto 3.1.11, boto3 1.24.0 and pytest 7.1.2, and the Makefile has an update target that runs pipenv update --dev. The Dockerfile builds on public.ecr.aws/lambda/python:3.9 and .python-version says 3.9.12.

A fresh install of the Pipfile's packages resolves moto 5.2.5 today, and pytest ends before collecting a single test: ImportError while loading conftest, pointing at line 6. The agent then finds the boundary by installing releases: moto 5.0.0 has mock_aws and no mock_s3, while 4.2.14, published the same day, has mock_s3 and no mock_aws. So the suite breaks the first time the lock is refreshed past 5.0.0, which is exactly what make update does, and a pin to moto<5 is the only way to keep the old names.

The rename itself is three lines: the import becomes from moto import mock_aws, and both fixtures enter with mock_aws():. One context covers S3 and SSM alike, and the two session fixtures can each enter their own, since the tests pass with both of them active. But the rename is not the end. With it in place every one of the 16 tests errors in the SSM fixture with a ValidationException from PutParameter: A parameter type is required when you create a parameter. moto 3.1.11 accepted put_parameter with only Name and Value; 4.2.14 and 5.2.5 both refuse it. Adding Type="String" to that call is the second change, and with it the suite reports 16 passed, through the same coverage run --include=ppod.py -m pytest the Makefile's test target uses.

The agent also checks what else moved in the same refresh, because a lock refresh never bumps one package. boto3 goes from 1.24.0 to the current release, pytest from 7 to 9, and requests-mock and responses move with them. Here none of those produce a failure once the two moto changes are in, which the pull request states plainly rather than leaving the reviewer to wonder.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "claude",
  "input": "Our test suite fails before collection with \"ImportError: cannot import name 'mock_s3' from 'moto'\" after refreshing the lock file. Find every import and every use of a per-service moto decorator or context manager (mock_s3, mock_ssm, mock_dynamodb and the rest) in the tests and fixtures. Read the dependency files and the lock to say which moto release was locked and which a fresh install resolves now, then install releases to find the first moto release without the old names and the last one with them. Replace each old name with mock_aws, keeping every fixture's scope and what it creates. Run the whole test suite. If tests then fail on stricter validation in moto, such as a missing parameter type, fix the test setup to match what the real service requires and say so. Open a pull request with the version findings, the full test output before and after, and anything else the refresh changed that affects the tests.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

A pull request that changes conftest.py in four places: the import on line 6, the two with statements on lines 45 and 75, and the put_parameter call that gains Type="String". The description names the locked moto release, the one a fresh install resolves, and the two releases on either side of the rename, with the install commands that showed it. Below that is the pytest output from before the change, the ImportError and the 16 ValidationException errors that follow the rename alone, and the run after, with 16 passed and the coverage report.

What the reviewer checks is small and specific. Every mock_<service> name in the repository is in the diff and none was left for a service the tests do not touch. The fixture scopes are unchanged, so the buckets and the parameter are still created once per session. The Type="String" addition matches what the Lambda reads in production, which is a plain string parameter, and does not paper over a real difference. And the lock file, if it is part of the change, moves moto past 5.0.0 rather than pinning it below.

The pull request does not claim anything about the deployed Lambda. The sandbox has no AWS account, so the only proof is the suite under moto, which is what this repository's CI runs as well.

Run it on every failed test job after a dependency refresh

This error arrives through CI, the first time a refreshed lock is tested. A step in that workflow that runs only when the test job fails can send the job's log tail to a Rush webhook trigger's URL, with the trigger's secret in Authorization: Bearer. The log text lands after the automation's saved prompt as the event, so the prompt tells the agent to read the import error from it before looking at the code. Send an Idempotency-Key header to the trigger's URL set to the workflow run id, so a re-run of the same job starts nothing new.

A repository that refreshes its lock on a schedule can instead start the refresh itself. An automation with a schedule trigger in the Project that holds the repository can run the Makefile's update target, run the tests, and fix what the refresh breaks, once a week. One run per week is far under the automation's runs-per-hour cap, and the week's pull request carries the test output either way.

The use case this job follows, with its automation and trigger: Fix a bug from an alert.

Why run it on Rush?

Finding which release renamed a function means installing several of them, and nobody wants four versions of moto cycling through the environment where they work. The sandbox is disposable: the agent installs 3.1.11, 4.2.14, 5.0.0 and the current release in turn, keeps the one import line that shows the difference for each, and throws the environment away when the run ends. Every command and its output stays in the session's record.

The rename is also only half the change, and the half that looks finished is the one that is not. A diff that swaps three names reads as complete, and only a test run shows the 16 errors that follow it. The pull request carries that run, before and after, so the reviewer judges the suite's output rather than the plausibility of the diff.

Frequently asked questions

Which moto release removed mock_s3?

moto 5.0.0, published on 2024-01-27, has mock_aws and no mock_s3. moto 4.2.14, published the same day, has mock_s3 and no mock_aws. A dependency file with moto and no upper bound picks up 5.x on its next refresh.

Can I pin moto below 5 instead of renaming?

Yes, moto<5 keeps mock_s3 working, and the pull request says where to put it. It also freezes the suite on a release that stops receiving the service changes AWS makes, and the rename is three lines, so the pin only makes sense as a stopgap while something larger blocks the refresh.

Why do my tests fail with ValidationException after the rename?

Because newer moto checks what the real service checks. In this fixture, put_parameter without a Type was accepted by moto 3.1.11 and is refused by 4.2.14 and 5.2.5 with A parameter type is required when you create a parameter. The fix is to add Type="String" to the test setup, which is what the real API has always required.

Do I need one mock_aws around everything, or one per fixture?

Either works. In this fixture the S3 and SSM fixtures each enter their own with mock_aws(): at session scope, nested, and the 16 tests pass that way. A single context covering both would pass too; keeping two leaves the fixture boundaries where the authors put them.

More jobs like this: Fix a bug from an alert

Keep reading