What the agent does
The agent reads every page of the app to find where model calls happen and what happens when they fail. In an app like streamlit/llm-examples, Chatbot.py reads an OpenAI key from st.text_input(key="chatbot_api_key", type="password"), stops with st.info when it is empty, then calls client.chat.completions.create(model="gpt-3.5-turbo", ...) with no try around it; a wrong key raises openai.AuthenticationError and Streamlit renders the full traceback in the page. pages/1_File_Q&A.py calls anthropic.Client(...).completions.create(...) with anthropic.HUMAN_PROMPT and model="claude-v1", pages/2_Chat_with_search.py runs a LangChain initialize_agent with DuckDuckGoSearchRun and StreamlitCallbackHandler, and pages/5_Chat_with_user_feedback.py repeats the OpenAI call and then logs feedback to Trubrics when TRUBRICS_EMAIL is in st.secrets. requirements.txt pins langchain==0.0.350 and langchain-community==0.0.3 and floors openai>=1.2 and anthropic>=0.3.0; .ruff.toml sets line-length = 105; app_test.py already uses streamlit.testing.v1.AppTest with unittest.mock.patch on openai.resources.chat.Completions.create.
Rather than scatter try blocks, the agent adds a small utils/errors.py with one function, show_api_error(exc), that maps exception classes to messages a user can act on: openai.AuthenticationError and anthropic.AuthenticationError to "check the key in the sidebar", openai.RateLimitError to a retry hint, openai.APITimeoutError and openai.APIConnectionError to a connectivity message, openai.BadRequestError to the provider's own message (which is where a model name the key cannot use shows up), and anything else to a short st.error plus st.exception behind an expander so the traceback is still available. Each page wraps only the model call in try/except, calls that function and then st.stop(), so partially appended messages do not linger in st.session_state.messages. In Chatbot.py and page 5 the user's message is appended before the call, so on failure the agent pops it, which keeps the transcript consistent with what the model actually saw.
The LangChain page is different: search_agent.run(...) wraps the OpenAI error in its own exception types, so the agent catches openai errors and LangChain's OutputParserException separately and keeps handle_parsing_errors=True. For the Anthropic page it also notes in the pull request that anthropic.HUMAN_PROMPT, completions.create and claude-v1 are the legacy Text Completions interface, and leaves that migration out of this change so the diff stays about errors.
Then it extends app_test.py. Following the existing pattern, it patches openai.resources.chat.Completions.create with side_effect=openai.AuthenticationError(...) constructed with a fake httpx.Response, runs AppTest.from_file("Chatbot.py"), sets the key and a prompt, and asserts at.error[0].value contains the key message and not at.exception; it does the same for RateLimitError and for page 5. It runs ruff check and ruff format --check at the pinned pre-commit version and pytest -v, the same commands .github/workflows/app-testing.yml runs through streamlit/streamlit-app-action with ruff: true.
Start it with one API call
POST https://api.phoenix.so/o/{org}/sessions with this body:
{
"agent": "claude",
"input": "Add error handling for LLM API calls to this Streamlit app and open a pull request. Read every page under pages/, the root app file, app_test.py, requirements.txt and .ruff.toml first. Add one helper that maps OpenAI and Anthropic authentication, rate-limit, timeout, connection and bad-request errors to short st.error messages a user can act on, with the traceback in an expander for anything unknown. Wrap only the model calls in each page, call the helper and st.stop() on failure, and remove the user message appended to session state before a failed call so the transcript stays consistent. Do not migrate any SDK or change models. Extend app_test.py with AppTest cases that patch the OpenAI create call to raise AuthenticationError and RateLimitError and assert the message and no exception. Run ruff check, ruff format --check and pytest -v in the sandbox and quote the output.",
"resources": [
{
"url": "https://github.com/your-org/your-app"
}
]
}- Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
- Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
- Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.
What comes back
What comes back is a pull request that adds utils/errors.py, edits the model-call block in Chatbot.py and each page under pages/, and adds test cases to app_test.py. Each page's diff is a try/except around one call plus a session-state pop; no model name, prompt or SDK import changes. requirements.txt is unchanged because every exception class used already ships in the pinned SDKs.
The description quotes ruff check, ruff format --check and pytest -v output from the sandbox, including the new test names, and lists the error classes handled per page with the message each shows. The repository's own App testing workflow runs on the pull request through streamlit-app-action, so the same checks run again on GitHub.
Before merging, a reviewer reads the messages for tone, decides whether the traceback expander should be shown to end users or only when a DEBUG secret is set, and runs streamlit run Chatbot.py with a deliberately wrong key to see the new message in the sidebar flow. They also check that the Anthropic legacy-API note is tracked somewhere, since this pull request leaves it alone on purpose.
Run it on every Streamlit app with unguarded model calls
A team with several internal Streamlit tools sends one POST /o/{org}/sessions per repository with this prompt and the repository in resources. GET /o/{org}/github lists the repositories the Rush GitHub App installation reaches, so a script can grep each clone for completions.create or messages.create outside a try block and start a run only where one is found. Each run reads that app's pages and SDK versions, so an app on the Anthropic Messages API gets the matching exception classes rather than a copy of this one.
Set a webhook URL on the API key and Rush POSTs run.completed or run.failed when the turn ends, with session_url to read the pull request link. Send a fresh UUID as request_id with each request so a retried request returns the run already started.
The use case this job follows, with its automation and trigger: Hand off a long task.
Why run it on Rush?
Error handling for model APIs is the kind of change that looks trivial and goes wrong in the details: which exception class a given SDK version raises, what a LangChain agent wraps it in, and what is left in session state after a failure. In its own sandbox the agent installs the pinned requirements.txt, imports the real classes, raises them under AppTest and reads what the page shows, so the pull request is tested behaviour and not a guess. Every command and its output are kept in the run's record.
The diff touches one block per page and ships with tests the repository's existing CI already knows how to run. The same request runs across every Streamlit app a team owns at once, each in its own sandbox, each with its own pull request.
Frequently asked questions
Will it change which models or SDK methods the app uses?
No. The prompt rules that out so the diff is about error handling only. The agent does note in the description where a page uses a legacy interface, such as the Anthropic Text Completions call in the File Q&A page, so the team can plan that migration separately.
How can AppTest exercise a real API error without a key?
The repository's app_test.py already patches openai.resources.chat.Completions.create. The agent keeps that approach and sets side_effect to an openai.AuthenticationError built with a fake httpx.Response, so the page runs exactly the code path a bad key triggers with no network call.
What happens to the chat history when a call fails?
The user's message is appended to st.session_state.messages before the call in Chatbot.py and page 5. On failure the agent removes it again before st.stop(), so the next successful call sends the model the same transcript the user sees and no half-turn stays behind.
Does the LangChain page get the same treatment?
Yes, with one difference. search_agent.run can raise LangChain's OutputParserException as well as the OpenAI errors, so both are caught and shown through the same helper, and handle_parsing_errors=True stays on so recoverable parse errors are still retried by the agent itself.
More Streamlit jobs
The same job on other stacks
- Add Global Error Handling to TanStack Query (TanStack Query)
- Add API Error Handling to a tRPC App (tRPC)
- Add Consistent API Error Responses to a Vapor App (Vapor)