# Stream Chat Responses in a Streamlit App

Yes. An agent switches your Streamlit chatbot to streamed replies rendered with st.write_stream, keeps the AppTest suite green with a mocked stream, and opens a PR.

[Run this on Rush](https://cloud.getrush.ai/run/add-streaming-responses/streamlit/). Opens the console with this prompt ready to send, in a Project that has your repository.

## What the agent does

The agent reads how the chat is wired before it changes a call. In an app shaped like streamlit/llm-examples, `Chatbot.py` keeps the history in `st.session_state["messages"]`, replays it with `st.chat_message(role).write(content)`, reads the key from a sidebar `text_input` with key `chatbot_api_key`, and on each `st.chat_input` calls `client.chat.completions.create(model="gpt-3.5-turbo", messages=...)`, then writes `response.choices[0].message.content` in one go. `pages/5_Chat_with_user_feedback.py` makes the same non-streaming call and appends the whole reply before `st.write`. `pages/2_Chat_with_search.py` is the one place that already streams, by passing `streaming=True` to `ChatOpenAI` and a `StreamlitCallbackHandler`, so the app's users already see partial output on one page and not on the others.

`requirements.txt` pins `streamlit>=1.28`, `openai>=1.2`, `langchain==0.0.350` and `langchain-community==0.0.3`, and `.github/workflows/app-testing.yml` runs `streamlit/streamlit-app-action` with `ruff: true` and `pytest-args` on `Chatbot.py`. `app_test.py` is the test the change must keep green: it loads the app with `AppTest.from_file("Chatbot.py")`, patches `openai.resources.chat.Completions.create` to return a `ChatCompletion` built by `create_chat_completion`, sets the key input and the chat input, and asserts that `at.chat_message[2].markdown[0].value` equals the joke. A streamed call returns an iterator of chunks, not a `ChatCompletion`, so the fixture has to change with the code.

The agent passes `stream=True`, wraps the result in a generator that yields `chunk.choices[0].delta.content or ""`, and inside `with st.chat_message("assistant"):` calls `st.write_stream(generator)`, which renders the tokens as they arrive and returns the full string; that string is what goes into `st.session_state.messages`, so the replay loop on the next rerun is unchanged. It applies the same shape to `pages/5_Chat_with_user_feedback.py`, keeping the feedback widget keyed on the message count, and raises the `streamlit` floor in `requirements.txt` to a release that has `st.write_stream`, which the pin at `>=1.28` does not guarantee.

In `app_test.py` the agent adds a `create_chat_completion_chunks` helper that builds `ChatCompletionChunk` objects from `openai.types.chat` with a `ChoiceDelta` per word and a final chunk with `finish_reason` `stop`, makes the patched `create` return an iterator over them, and keeps the existing assertion on the assembled markdown. It runs `ruff check .` with the repository's `.ruff.toml` line length of 105 and `pytest -v`, the two things the workflow runs, and keeps both outputs.

## Start it with one API call

`POST https://api.phoenix.so/o/{org}/sessions`

```json
{
  "agent": "claude",
  "input": "Make this Streamlit chatbot stream its replies and open a pull request. Read Chatbot.py, pages/5_Chat_with_user_feedback.py, pages/2_Chat_with_search.py, app_test.py, requirements.txt, .ruff.toml and .github/workflows/app-testing.yml first. In Chatbot.py and the feedback page, call client.chat.completions.create with stream=True, wrap the chunks in a generator that yields each delta's content, render it inside st.chat_message with st.write_stream, and store the string it returns in st.session_state.messages so the history replay is unchanged. Raise the streamlit floor in requirements.txt to a release that has st.write_stream and say which one in the pull request. Update app_test.py so the patched Completions.create returns an iterator of ChatCompletionChunk objects ending with finish_reason stop, and keep the existing assertion on the assembled assistant message. Run ruff check . and pytest -v and paste the output in the pull request. Do not change the LangChain pages or the sidebar.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
```

1. **Get an API key.** In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
2. **Send the request.** POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
3. **Read what came back.** Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

## What comes back

What comes back is a pull request with a few lines changed in `Chatbot.py` and `pages/5_Chat_with_user_feedback.py`, a new chunk helper and an updated patch in `app_test.py`, and one line in `requirements.txt`. The description shows the generator, names the Streamlit release the floor now points at, and pastes the `ruff check` and `pytest -v` output from the sandbox.

`app-testing.yml` runs the same ruff and pytest steps on the pull request through `streamlit-app-action`, with the results summarised on the check, so the mocked stream is exercised there as well.

A reviewer runs `streamlit run Chatbot.py` with a real key once and watches a reply arrive word by word, then sends a second message to confirm the first reply is still in the history. They check that the feedback page still records feedback for the streamed reply and that nothing on the LangChain pages changed. Then they merge.

## Run it on every chat page that writes replies in one go

A team with several Streamlit apps sends one `POST /o/{org}/sessions` per repository with this prompt and the repository in `resources`. Each run reads its own pages and test file, so an app that uses a different client gets the generator written for that client's chunk shape, and each opens its own pull request.

Give the key a `webhook_url` and Rush POSTs `run.completed` when each turn ends, with `session_url` to read the pull request from. Send a `request_id` with each call so a retried request returns the run it already started.

## Why run it on Rush?

Streaming is a small code change and a larger testing change: the mocked response has to become a mocked stream, and the assertion on the assembled message has to keep passing. In a Rush sandbox the agent runs the app's own `AppTest` suite and ruff before pushing, and the output stays in the run's record beside the pull request, so the reviewer sees the suite passing on the new fixture rather than a note saying it should.

Doing it by hand means reading the chunk types, rewriting the fixture and then checking two pages by eye. The agent does that in one pass, and the same request works unchanged on the next Streamlit app whose replies still arrive all at once.

## Frequently asked questions

### Will the history replay change?

No. `st.write_stream` returns the full text once the generator is exhausted, and that string is appended to `st.session_state.messages` exactly as the non-streaming reply was, so the loop at the top of the page renders it the same way on the next rerun.

### Why does requirements.txt change?

Because `streamlit>=1.28` allows releases that do not have `st.write_stream`. The agent raises the floor to the first release that does and names it in the pull request, so `pip install -r requirements.txt` cannot produce an app that fails at the new call.

### What does the test mock now?

An iterator of `ChatCompletionChunk` objects, one `ChoiceDelta` per word and a final chunk with `finish_reason` set to `stop`, returned by the same patched `openai.resources.chat.Completions.create`. The assertion still compares `at.chat_message[2].markdown[0].value` to the joke.

### Does it touch the LangChain pages?

No. `pages/2_Chat_with_search.py` already streams through `StreamlitCallbackHandler`, and the quickstart and prompt-template pages are left alone; the prompt says so.

## More Streamlit jobs

- [Show Token Usage per Reply in a Streamlit Chat App](https://getrush.ai/run/add-token-usage-display/streamlit/)
- [Add Type Checking to a Streamlit App](https://getrush.ai/run/add-type-checking/streamlit/)
- [Dockerize a Streamlit App](https://getrush.ai/run/dockerize-app/streamlit/)

## The same job on other stacks

- [Add Streaming Responses to a Hono API](https://getrush.ai/run/add-streaming-responses/hono/)

## More jobs like this: Hand off a long task

- [Add Stripe Checkout to a Medusa Store](https://getrush.ai/run/add-stripe-checkout/medusa/)
- [Add JSON-LD Structured Data to an Eleventy Blog](https://getrush.ai/run/add-structured-data/eleventy/)

## Keep reading

- [How a Rush run works on your repository, from request to pull request](https://getrush.ai/cloud/)
- [Compare cloud agents, agent sandboxes and AI code review](https://getrush.ai/compare/)
- [Rush pricing: Free, Pro and Max](https://getrush.ai/pricing/)
