What the agent does
Turning on --strict in an existing codebase produces one long list of errors, and the agent's first task is to split that list by where the errors come from. It reads the current setup. In a library like vangorra/python_withings_api the type checker is pinned in pyproject.toml as mypy = "==0.790", scripts/build.sh runs mypy ., and the [mypy] section of setup.cfg already holds more than a dozen settings written out by hand: disallow_untyped_defs, disallow_untyped_calls, disallow_incomplete_defs, check_untyped_defs, warn_return_any, warn_unused_ignores, strict_equality and others, plus the Pydantic plugin. It also sets ignore_missing_imports = True and implicit_reexport = True. The project is most of the way to strict already. What nobody knows is how far the old pin has drifted from what a current mypy reports.
So the agent measures in two steps. First it runs a current mypy with the repository's own configuration, changing nothing else. When this page was written that was mypy 2.4.1, with stub packages for requests, dateutil and requests-oauthlib installed, and the answer was 39 errors in 3 files. None of them is about strictness. Thirty-seven are in tests/test_init.py: the url of a recorded request is typed str | None, and two test helpers that take str are called with it 37 times. One is in withings_api/__init__.py, where an optional refresh URL is passed to a parameter typed str. One is in withings_api/common.py, where a field annotated with a subclass gets a default factory that returns the parent class.
Second, it adds --strict and subtracts. The total becomes 57 errors in 4 files, so strict mode itself accounts for 18, and they fall into three kinds: two from flags the hand-written list lacked and one from a flag it had switched off. Twelve are redundant-cast: calls to cast() that were needed for the type information of 2020 and are now no-ops. Five are type-arg: bare Tuple and dict annotations with no parameters. One is attr-defined in scripts/integration_test.py, a name imported through a module that does not re-export it, which the explicit implicit_reexport = True had been permitting.
The fixes follow the split, one commit per group, and the agent runs mypy and the tests after each. The helper signatures in the tests accept the optional type and assert it is present. The optional URL gets a guard that raises the library's own error. Redundant casts are deleted, generics get their parameters, the import is pointed at the defining module. Then the hand-listed flags in setup.cfg are replaced by strict = True, the settings that contradict it are removed, and the mypy pin moves to the version the work was done on. ignore_missing_imports is narrowed to per-module overrides for the packages that still ship no types, so a typed dependency can no longer be silently treated as Any. No # type: ignore is added without an error code and a reason.
Start it with one API call
POST https://api.phoenix.so/o/{org}/sessions with this body:
{
"agent": "claude",
"input": "Enable mypy strict mode in this repository. Read the mypy configuration wherever it lives (setup.cfg, mypy.ini, pyproject.toml), the pinned mypy version in the manifest and lockfile, any mypy plugins, and how CI and scripts invoke mypy. Build an environment with the project's dependencies and a current mypy, and install stub packages for third-party libraries that have them. Measure in two steps and keep both outputs. Step one: run the current mypy with the repository's existing configuration and no other change; group the errors by file and error code and explain each group, since these come from the old pin and newer dependency types, not from strictness. Step two: run with --strict and report only the additional errors, grouped by error code, and name the strict flag responsible for each group. List which flags strict implies that the existing configuration already sets, which it lacks, and which settings contradict it. Fix the errors in separate commits: first the ones the existing configuration reports on current mypy, then each strict group. Prefer real fixes: narrow an Optional with a check that raises a meaningful error, add type parameters, delete casts mypy calls redundant, import names from the module that defines them. Do not change runtime behaviour beyond such guards, do not weaken a public signature, and do not add a type: ignore without an error code and a one-line reason; list every ignore you add. Run the test suite after each commit. Finally set strict = True in the configuration, remove flags it makes redundant and settings that contradict it, replace a global ignore_missing_imports with per-module overrides for packages that have no types, and update the mypy pin. Open a pull request with the two measurements, the commits by group, and mypy's final output.",
"resources": [
{
"url": "https://github.com/your-org/your-app"
}
]
}- Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
- Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
- Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.
What comes back
A pull request whose commits mirror the two measurements. The first commit clears what a current mypy reports under the old configuration; in a library like the one above that is mostly two test helpers and one guard in the client. The next commits each remove one kind of strict error: redundant casts, missing type parameters, the indirect import. The last changes configuration only: strict = True, the redundant flags removed, the import override narrowed, and the pin raised.
The description gives both counts with their breakdown by file and error code, so a reviewer can see that 39 of the 57 errors had nothing to do with strict mode. That changes the conversation: the question of whether strict mode is worth it is separated from the bill for years on an old pin. The final section is mypy's own last line, reporting no issues, and the test totals after each commit.
Reviewers look at two places. Any guard the agent added for an optional value is a small behaviour change, a clear error where there used to be a confusing one deeper in a dependency, and each is listed. And the per-module overrides name every dependency still treated as untyped, which is a to-do list for later: when one of those packages ships types, deleting its line turns the checking on.
Run it on every repository without strict typing
A team decides once to hold a repository to strict typing, and a developer hands the work off. They send this body to POST /o/{org}/sessions from a terminal or an MCP client with a request_id that is a UUID made for the job, and the pull request is on the session when the run completes. If the reviewer disagrees with a fix, a follow-up to POST /o/{org}/sessions/{id}/messages wakes the same sandbox, where the environment and mypy's cache are still in place.
To learn how much work strict mode would be across many repositories before committing to any of it, send a shorter prompt that stops after the two measurements, once per repository, each with a request_id that is a UUID v5 of the repository's full name. The key queues what it cannot run at once and its webhook_url receives run.completed for each run it started. The counts that come back say which repositories are an afternoon and which are a project.
The use case this job follows, with its automation and trigger: Hand off a long task.
Why run it on Rush?
The measurement is only trustworthy in a clean environment. On a laptop, mypy sees whichever stubs and package versions happen to be installed, and its incremental cache can hide an error that a fresh run reports. The run's sandbox starts empty, the agent builds the environment from the manifest, and both mypy runs with their full output are kept in the transcript at GET /o/{org}/sessions/{id}/messages.
The work after the measurement is dozens of small edits, each followed by a type check and a test run. It is easy, slow and unrewarding, which is why strict mode tends to stay on a wish list. A run does the edits in order, commits by group, and leaves the developer to review a pull request where every change has a stated reason.
Frequently asked questions
How many errors will --strict produce in my codebase?
Fewer than the first run suggests, once you separate causes. In vangorra/python_withings_api a current mypy with --strict reports 57 errors, but 39 of them appear without the flag and come from an old mypy pin and newer dependency types. Strict mode itself adds 18. The agent always reports the two numbers apart.
Why replace ignore_missing_imports?
Set globally, it turns every import mypy cannot resolve into Any, including a typed package whose stubs are simply not installed. Errors in how you call that package then vanish. Per-module overrides keep the escape hatch for packages that have no types and nothing else.
Will it sprinkle type: ignore comments to get to zero?
The prompt allows an ignore only with an error code and a written reason, and requires a list of every one in the pull request. Most strict errors have a real fix that is one line long. An ignore is for the rare case where a third-party type is wrong, and the reason says so.
The same job on other stacks
- Turn On TypeScript Strict Mode and Fix the Errors (TypeScript)
- Enable TypeScript Strict Mode in a discord.js Bot (discord.js)