All jobs

Replace streamlit-feedback With st.feedback in a Streamlit App

Yes. An agent swaps the streamlit-feedback component and Trubrics upload for the built-in st.feedback widget, stores ratings locally, keeps tests green, in a PR.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

The agent reads the app before changing the dependency. In a project like streamlit/llm-examples, pages/5_Chat_with_user_feedback.py is the page in question: it keeps the chat in st.session_state.messages, calls client.chat.completions.create with the key from the sidebar, and after each reply renders streamlit_feedback(feedback_type="thumbs", optional_text_label=...) from the streamlit-feedback package with a key of feedback_{len(messages)}. When a rating comes back and TRUBRICS_EMAIL is in st.secrets, it calls trubrics.init, trubrics.collect and trubrics.save and shows a toast. requirements.txt pins streamlit>=1.28, trubrics>=1.4.3 and streamlit-feedback beside langchain==0.0.350, openai>=1.2 and anthropic>=0.3.0. app_test.py drives Chatbot.py and the LangChain quickstart with AppTest, patching openai.resources.chat.Completions.create through a create_chat_completion helper, and .github/workflows/app-testing.yml runs those tests with ruff through streamlit/streamlit-app-action. The feedback page has no test, and the two feedback packages are the only reason it needs a third-party component and an external account.

It rewrites the page around st.feedback. Each assistant message gets its own widget rendered right under it, keyed by the message index, with "thumbs" as the option set, so a rating survives reruns and a user can rate any earlier answer, not only the last one. The widget returns the selected index, which the agent maps to a label in a small record_feedback function and stores in st.session_state.feedback keyed by message index, alongside the prompt, the reply and a timestamp. A thumbs-down reveals an st.text_input for the reason, since the built-in widget has no free-text field, and the input's on_change writes the comment into the same record. The page's intro text and title drop the Trubrics mention, and the sidebar keeps the API key field and the source link as the other pages have them.

Storage stays in the reader's hands. record_feedback appends one JSON line per rating to a feedback.jsonl file next to the app, and the function has a short docstring saying it is the place to call a database or an analytics endpoint instead. trubrics and streamlit-feedback leave requirements.txt, and the streamlit floor rises to the first release that ships st.feedback, which the agent reads from the Streamlit changelog rather than guessing. The st.secrets lookup for Trubrics credentials goes, so a fresh deployment on Community Cloud no longer needs those two secrets.

Then it proves the page still works. A new test in app_test.py loads pages/5_Chat_with_user_feedback.py with AppTest.from_file, patches the completion the same way test_Chatbot does, sends a prompt through chat_input, asserts the mocked reply appears as the assistant message and that the run raised no exception; a second test calls record_feedback directly and reads the JSON line back. pytest runs the whole file, ruff check passes under the 105-character line length in .ruff.toml, and the agent starts the app once with streamlit run in the sandbox to confirm the page loads without the removed imports.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "claude",
  "input": "Replace the streamlit-feedback component and the Trubrics upload in this Streamlit app with the built-in st.feedback widget. Read pages/5_Chat_with_user_feedback.py, Chatbot.py, requirements.txt, requirements-dev.txt, app_test.py, .ruff.toml, .pre-commit-config.yaml, .github/workflows/app-testing.yml and README.md first. Render a thumbs widget under every assistant message keyed by message index, show a text input for a reason after a thumbs-down, and write each rating with the prompt, reply, label, optional comment and timestamp as one JSON line to feedback.jsonl through a record_feedback function whose docstring says where to plug in another store. Remove trubrics and streamlit-feedback from requirements.txt, raise the streamlit floor to the first release with st.feedback, and drop the st.secrets lookup. Add an AppTest test for the page that patches the OpenAI completion like test_Chatbot and a unit test for record_feedback. Run pytest and ruff check, start the app once to confirm the page loads, and open a pull request that notes the two secrets that are no longer needed.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

What comes back is a pull request that rewrites pages/5_Chat_with_user_feedback.py, edits requirements.txt and README.md, and adds two tests to app_test.py. The description lists the two removed packages, the new streamlit floor, the shape of a feedback.jsonl line and the TRUBRICS_EMAIL and TRUBRICS_PASSWORD secrets that can be deleted from the deployment.

The run's record keeps the pytest output with each test named, the ruff check result and the lines streamlit run printed when the page was opened, so a reviewer can see the page renders without the component and the test covers the chat path end to end.

Before merging, a reviewer runs the app, rates a reply down, types a reason, and reads the new line in feedback.jsonl; then checks the deployment's secrets page and removes the two Trubrics entries the page no longer reads.

Run it on every page that still imports a third-party feedback component

The setup is one POST /o/{org}/sessions with this body and the repository in resources, started from a terminal; the run continues in its sandbox after the window closes, and a follow-up through POST /o/{org}/sessions/{id}/messages can ask for a different store than a JSON file or for a faces widget instead of thumbs without starting over. A team with several Streamlit apps sends one request per repository, each with a fresh UUID as request_id, so a retried script returns the run it already started.

Set a webhook_url on the API key and Rush posts run.completed or run.failed when a run's turn finishes, with session_url so your server can read the pull request link and the test output.

The use case this job follows, with its automation and trigger: Hand off a long task.

Why run it on Rush?

Swapping a widget looks like a ten-line change until the reruns bite: a feedback key that changes with the message count loses the rating on the next rerun, a callback that writes state during render double-counts, and a removed import breaks a page the test suite never loaded. The agent runs the page under AppTest and under streamlit run in its own sandbox and reads what happens, instead of trusting the diff.

The commands and their output are kept with the run, so the pull request can point at the test that sends a prompt and finds the reply, and at the clean ruff result, rather than asking a reviewer to install the app locally to find out whether the page still opens.

Frequently asked questions

What happens to feedback that was already going to Trubrics?

Nothing in the repository stores it, so there is no migration; the page sent each rating to the Trubrics backend at the time. From this change on, ratings land in feedback.jsonl, and record_feedback is the one function to change if they should go to a database or an analytics endpoint instead.

Why one widget per message instead of one under the latest reply?

The old page keyed a single widget by the message count, so it appeared only for the newest answer and vanished on the next turn. A widget per assistant message, keyed by its index, lets a user rate any answer in the conversation and keeps the rating visible after later turns.

Does st.feedback support a free-text comment?

No, it returns only the selected option. The agent shows an st.text_input after a thumbs-down and merges its value into the same record, which keeps the optional explanation the old component offered without a third-party package.

Will the other pages change?

No. Chatbot.py, the File Q&A page and the LangChain pages keep their imports and their tests. Only the feedback page, requirements.txt, README.md and app_test.py are touched, and the agent runs the existing tests to show they still pass.

More Streamlit jobs

More jobs like this: Hand off a long task

Keep reading