# Add Caching to a Streamlit App

Yes. An agent finds what each rerun rebuilds or re-sends, caches clients with st.cache_resource and repeat answers with st.cache_data, and tests it with AppTest.

[Run this on Rush](https://cloud.getrush.ai/run/add-caching/streamlit/). Opens the console with this prompt ready to send, in a Project that has your repository.

## What the agent does

The agent reads each page as Streamlit runs it, top to bottom on every interaction, and lists what that repeats. In streamlit/llm-examples, `Chatbot.py` builds `OpenAI(api_key=openai_api_key)` inside the `if prompt := st.chat_input()` block, so a new client object is made for every message; `pages/5_Chat_with_user_feedback.py` does the same. `pages/1_File_Q&A.py` reads the uploaded file with `uploaded_file.read().decode()` and sends the whole article to Anthropic each time the question input changes, so editing one word of the question re-sends the document. `pages/2_Chat_with_search.py` constructs `ChatOpenAI`, `DuckDuckGoSearchRun` and `initialize_agent` on every message. `requirements.txt` pins nothing below `streamlit>=1.28`, `openai>=1.2` and `anthropic>=0.3.0`, and `app_test.py` already tests two pages with `streamlit.testing.v1.AppTest`, patching `openai.resources.chat.Completions.create`.

It separates the two kinds of thing Streamlit caches. Objects that should exist once per process and are not data, the API clients and the search tool, go behind `st.cache_resource`: a `get_openai_client(api_key)` in a new `utils.py` returns one client per key, and `get_search_tool()` returns one `DuckDuckGoSearchRun`. Values that are the result of a computation go behind `st.cache_data`, which pickles and copies them: in File Q&A, an `answer_question(article, question, _api_key)` function whose cache key is the article text and the question, with the key argument named with a leading underscore so Streamlit leaves it out of the hash and a key rotation does not throw away answers, plus a `ttl` so a changed model does not serve old text for ever. The chat completions in `Chatbot.py` are left uncached on purpose: the request includes the whole conversation, so no two calls share a key, and caching would only hide the cost.

The pages change by a few lines each. The client construction moves out of the chat block into the cached helper; File Q&A calls the cached function and shows the answer, and gains a small `Clear cached answers` button that calls `st.cache_data.clear()` in the sidebar, since a cached wrong answer is the first thing a user will ask about. The agent keeps the API-key-missing checks exactly where they are, before any cached call, so a missing key still shows the `st.info` message the tests assert on. Line length stays within the `105` that `.ruff.toml` sets, and the pre-commit hooks in `.pre-commit-config.yaml` run `ruff` and `ruff-format`.

Then it proves both the speed-up and that nothing broke. `app_test.py` gains a case that drives File Q&A twice with the same upload and question through `AppTest` with `anthropic.Client` patched, and asserts the patched method was called once; and a case that two runs of `Chatbot.py` with the same key return the same client object from the cache. The existing `test_Chatbot` keeps passing because the patch is on the class method, which the cached client still calls. `pytest` runs them, `ruff check .` and `ruff format --check .` pass, and `.github/workflows/app-testing.yml` runs the same through `streamlit-app-action` with `ruff: true`.

## Start it with one API call

`POST https://api.phoenix.so/o/{org}/sessions`

```json
{
  "agent": "claude",
  "input": "Add caching to this Streamlit app. Read requirements.txt, .ruff.toml, app_test.py and every page under the root and pages/ first, and list what each rerun rebuilds or re-sends. Put objects that should exist once per process (API clients, tools) behind st.cache_resource in a shared module, and put repeatable computations (the File Q&A answer, keyed on the article text and question) behind st.cache_data with a ttl and with the API key passed as an underscore-prefixed argument so it is not part of the cache key. Do not cache chat completions whose input is the whole conversation; explain why in the pull request. Keep every missing-API-key check in place before any cached call. Add a sidebar button that clears st.cache_data. Extend app_test.py with AppTest cases that show the patched provider call happens once for a repeated question and that the cached client is reused. Run pytest, ruff check and ruff format --check, paste the output in the pull request and open it.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
```

1. **Get an API key.** In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
2. **Send the request.** POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
3. **Read what came back.** Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

## What comes back

A pull request that adds `utils.py`, changes `Chatbot.py`, `pages/1_File_Q&A.py`, `pages/2_Chat_with_search.py` and `pages/5_Chat_with_user_feedback.py` by a handful of lines each, and extends `app_test.py`. The description lists each page with what it used to rebuild and what is now cached, quotes the `pytest` run with the new cases, and shows `ruff` passing.

A reviewer checks the cache keys more than the code: that the File Q&A key is the article text and the question and nothing else, that the `ttl` matches how often the model or prompt changes, and that no `st.cache_data` function returns an object that should not be copied. They also decide whether the uncached chat path should stay uncached, which the pull request argues for.

The new tests are the proof the behaviour holds: a second identical question reaches the provider zero times, and the existing assertions about the missing-key message still pass.

## Run it on every Linear issue labelled performance

The next slow page gets the same treatment from an issue. In the console, add an automation on the Project that holds the repository with a `linear` trigger on `linear.issue.labeled` and the label `performance`, and a prompt that says: read the page the issue names, cache what each rerun rebuilds with the right decorator, keep the API-key checks, add an AppTest case, run pytest and ruff, and open a pull request. The issue is placed after the prompt as the event and fills in nothing, so the page comes from the issue text. The trigger needs a Linear connection for the automation's runner and fires only when the label is applied by someone in your Rush org.

## Why run it on Rush?

Caching in Streamlit is easy to add and easy to get wrong: a resource cached as data gets copied on every read, a key cached with the secret in it empties when the secret rotates, and a cached chat answer looks fine until two users share one. The agent runs each page through `AppTest` in its own sandbox, with the provider calls patched, so the pull request can say how many times the provider was reached before and after, which a code read cannot.

Those runs, with their output, stay in the run's transcript at `GET /o/{org}/sessions/{id}/messages`, and every changed file is listed with its diff at `GET /o/{org}/p/{project}/sessions/{id}/changes`. The reviewer reads the test that proves the repeated question was served from cache, instead of trusting a sentence that says so.

## Frequently asked questions

### When is it st.cache_resource and when st.cache_data?

`st.cache_resource` is for things that should exist once and be shared, such as an API client or a tool object; it returns the same object every time. `st.cache_data` is for results, such as an answer string; it stores a pickled copy and hands each caller a fresh one. Putting a client behind `cache_data` copies it on every read, and putting a result behind `cache_resource` lets one caller mutate what another sees.

### Why is the API key not part of the cache key?

Because the answer to a question about an article does not depend on which key paid for it, and keys rotate. The agent names the parameter with a leading underscore, which tells Streamlit to leave it out of the hash, so a new key reuses the cached answers. For the client itself the key is the argument, since a client is only valid for one key.

### Why not cache the chatbot replies?

Each call sends the whole conversation, so no two calls have the same input and a cache would never hit. It would also store user conversations in process memory. The agent caches the client only and says so in the pull request.

### Do the existing tests still pass?

Yes. `app_test.py` patches `openai.resources.chat.Completions.create` on the class, and the cached client calls that method, so the patch still applies. The missing-key checks stay before any cached call, so the `st.info` message the tests assert on is unchanged.

## More Streamlit jobs

- [Add Chat History Export to a Streamlit App from a Linear Issue](https://getrush.ai/run/add-chat-export/streamlit/)
- [Add Test Coverage to CI for a Streamlit App](https://getrush.ai/run/add-coverage-to-ci/streamlit/)
- [Set Up Dependabot for a Streamlit App](https://getrush.ai/run/add-dependabot/streamlit/)

## The same job on other stacks

- [Add Caching to a Vapor App](https://getrush.ai/run/add-caching/vapor/)
- [Add Caching to a Flask App](https://getrush.ai/run/add-caching/flask/)
- [Add Response Caching to a Hono API](https://getrush.ai/run/add-caching/hono/)

## Keep reading

- [How a Rush run works on your repository, from request to pull request](https://getrush.ai/cloud/)
- [Compare cloud agents, agent sandboxes and AI code review](https://getrush.ai/compare/)
- [Rush pricing: Free, Pro and Max](https://getrush.ai/pricing/)
