What the agent does
The agent reads the app and the checks that already run. In an app laid out like streamlit/llm-examples, the entry page is Chatbot.py and five more pages live under pages/ with Streamlit's numbered names, such as 1_File_Q&A.py and 5_Chat_with_user_feedback.py. requirements.txt lists streamlit>=1.28, openai>=1.2, langchain==0.0.350, langchain-community==0.0.3, anthropic>=0.3.0, duckduckgo-search, trubrics>=1.4.3 and streamlit-feedback; requirements-dev.txt already pins mypy==1.4.1 next to black, pre-commit and pytest, but nothing runs it and no configuration exists. .ruff.toml sets a line length of 105, .pre-commit-config.yaml runs ruff and the standard hooks, and .github/workflows/app-testing.yml runs on Python 3.11 through streamlit/streamlit-app-action, which installs the requirements, runs ruff and runs the AppTest cases in app_test.py, then publishes the JUnit results. The devcontainer uses Python 3.11 as well.
The agent writes a [tool.mypy] table in a new pyproject.toml, keeping the ruff settings where they are: python_version 3.11 to match CI, files listing Chatbot.py, app_test.py and every page under pages/ by path, since those file names are not importable module names, check_untyped_defs on, and ignore_missing_imports only for the packages that ship no type information, each named in an overrides entry so a package that later gains stubs is noticed. Then it runs mypy and fixes what it reports in the app code rather than silencing it.
The first fixes are the ones every OpenAI-based chat page hits: response.choices[0].message.content is typed str | None in the openai package, and Chatbot.py appends it to st.session_state.messages and passes it to st.chat_message().write(), so the agent narrows it with an explicit check and a short message when the model returns nothing. Pages that read st.session_state keys get a typed accessor, and the app_test.py patches of openai.resources.chat.Completions.create keep their fake ChatCompletion objects, which already satisfy the types. Each fix is small and its reason is in the commit message.
CI gets one step after the Streamlit action in app-testing.yml: install requirements-dev.txt and run mypy, so the pinned version in that file is the one CI uses. .pre-commit-config.yaml gets the mirrors-mypy hook pinned to the same version with the same files. The agent runs ruff check, mypy, pytest app_test.py and pre-commit run --all-files in its sandbox and opens the pull request when all four are clean.
Start it with one API call
POST https://api.phoenix.so/o/{org}/sessions with this body:
{
"agent": "claude",
"input": "Add type checking to this Streamlit app. Read Chatbot.py, every file under pages/, app_test.py, requirements.txt, requirements-dev.txt, .ruff.toml, .pre-commit-config.yaml and every file under .github/workflows first. Add a [tool.mypy] table in pyproject.toml with the Python version CI uses, the entry page, the test file and every page listed by path, check_untyped_defs on, and ignore_missing_imports only through per-package overrides for packages without type information. Run mypy with the version requirements-dev.txt pins and fix each error in the app code; never add a blanket ignore. Add a step to the testing workflow that installs the dev requirements and runs mypy, and add the mirrors-mypy hook to pre-commit at the same version. Run ruff, mypy, the AppTest suite and pre-commit on all files in the sandbox, and open a pull request that lists each fix with the error it resolved.",
"resources": [
{
"url": "https://github.com/your-org/your-app"
}
]
}- Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
- Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
- Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.
What comes back
The pull request adds pyproject.toml with the mypy table, edits Chatbot.py and the pages under pages/ where a type error was real, adds a step to .github/workflows/app-testing.yml and a hook to .pre-commit-config.yaml. Its description lists each mypy error with the file, the line and the one-line fix, the packages covered by ignore_missing_imports overrides and why, and the output of ruff, mypy, the AppTest suite and pre-commit from the sandbox.
After the merge, every push and pull request runs mypy on Python 3.11 after the app tests, and a new page that passes a possibly-missing model reply straight to a widget fails the check before it ships.
Before merging, a reviewer reads the narrowing added around the model reply to confirm the fallback message is the one they want users to see, checks the overrides list against the requirements for any package that does ship stubs, and decides whether disallow_untyped_defs should come next.
Run it on every new repository
A team that owns several Streamlit apps sends this body once per repository from its own server, each a POST /o/{org}/sessions with that repository in resources and a request_id that is a UUID v5 of the repository's full name, so a retried call returns the run already started. Each run reads its own requirements: an app on a newer mypy pin runs that version, one without pre-commit gets only the workflow step, and one whose pages import a typed client gets no override for it.
Once the check exists, an automation on the Project with a github trigger on pull_request.opened and the prompt "if the mypy step failed on this pull request, read its output and comment with each error and a suggested fix" starts one run per pull request. The trigger adds the pull request after the prompt, and the run comments without pushing.
The use case this job follows, with its automation and trigger: Hand off a long task.
Why run it on Rush?
Turning on mypy in an app that never ran it produces a wall of errors, most of them from untyped packages, and the usual shortcut is a blanket ignore that checks nothing. The agent separates the packages without stubs from the real errors in the app, fixes the latter one by one, and runs the app's own AppTest suite in its sandbox to show the fixes changed no behaviour.
The mypy runs before and after, the test output and the pre-commit run stay with the run, so a reviewer sees each error and the line that resolved it. A team with many apps starts one run per repository at once, each in its own sandbox with its own interpreter.
Frequently asked questions
Why list the pages by path?
Streamlit page files carry names like 1_File_Q&A.py, which no import statement could name. Passing them to mypy as paths checks them without treating them as a package, and the list in pyproject.toml is what both the CI step and the pre-commit hook use.
Will it add type: ignore everywhere?
No. Packages without type information get a per-package ignore_missing_imports override, named so a reviewer can see the list, and errors in the app code are fixed in the code. A blanket ignore is not used.
Which mypy version does CI run?
The one requirements-dev.txt pins, installed by the new workflow step, and the same version in the pre-commit hook. The agent does not change the pin; raising it is a separate change.
Does this change what the app does?
Only where a value could be missing: a model reply that comes back empty now shows a short message instead of writing nothing. The AppTest cases in app_test.py pass unchanged, which the pull request shows.
More Streamlit jobs
The same job on other stacks
More jobs like this: Hand off a long task
- Generate an API Reference for a TypeScript Library With TypeDoc (TypeScript library)