What the agent does
The agent clones the repository into its own cloud sandbox and reads the analysis code first. In a Kaggle-style project like GlaIZier/titanic-prediction, requirements.txt lists numpy, pandas, matplotlib, seaborn, scikit-learn, xgboost and jupyterlab with no versions, so a fresh install gets whatever is current. python/data_analysis.py defines analyze_training_data(train), which at line 43 calls train.groupby('Sex').agg('sum')[['Survived', 'Died']], at line 47 train.groupby('Sex').agg('mean')[['Survived', 'Died']] and at line 76 train.groupby('Pclass').mean()['Fare'], each feeding a bar chart. data/train.csv has twelve columns, and Name, Sex, Ticket, Cabin and Embarked hold text. The model scripts under python/ keep the call commented out as # da.analyze_training_data(raw_train), which is how the break went unnoticed.
Then it reproduces the error at each pandas release that matters. A fresh install resolves pandas 3.1.1 with numpy 2.4.6; the text columns load as str dtype and line 47 raises TypeError: dtype 'str' does not support operation 'mean'. On pandas 2.2.3 the same line raises TypeError: agg function failed [how->mean,dtype->object], the message most people paste into a search box. On 2.0.3 it is TypeError: Could not convert followed by every passenger name joined together and to numeric. On 1.5.3 the call works and prints FutureWarning: The default value of numeric_only in DataFrameGroupBy.mean is deprecated, which named the change two releases before it landed. Line 43's agg('sum') is worse than an error: on every version it concatenates the names, tickets and cabins per group and raises nothing.
The fix is to say which columns to aggregate. For the fare chart, train.groupby('Pclass')['Fare'].mean() gives the same three means on 1.5.3, 2.2.3 and 3.1.1 (84.15, 20.66 and 13.68 for the three classes), and for the survival charts train.groupby('Sex')[['Survived', 'Died']].sum() and .mean() select before aggregating, so no text column reaches the reducer. The agent prefers column selection over numeric_only=True: the keyword also works, but it silently drops any column that is not numeric, including an id column a future edit might add, where the explicit list says what the chart is of.
It then runs analyze_training_data once under a non-interactive Matplotlib backend to show the three charts render without error on the current pandas, and runs it again on the pinned 1.5.3 to show the numbers did not move. It leaves the model scripts alone, since they never call the function, and does not pin pandas in requirements.txt; the pull request describes the pin as a separate decision for the owner.
Start it with one API call
POST https://api.phoenix.so/o/{org}/sessions with this body:
{
"agent": "claude",
"input": "This project's analysis code fails with \"TypeError: agg function failed [how->mean,dtype->object]\" on current pandas. Read requirements.txt and every Python file that calls groupby first. Install the current pandas, run the failing function with a non-interactive Matplotlib backend and quote the error with its file and line; then install the last pandas 1.x and show the same call works, and quote any FutureWarning it prints. Fix each groupby by selecting the columns the chart actually uses before aggregating, keep the output values identical to the 1.x run and show them side by side, and point out any aggregation over text columns that returns nonsense without raising. Do not pin pandas and do not change the model scripts. Open a pull request that quotes the failing and passing runs.",
"resources": [
{
"url": "https://github.com/your-org/your-app"
}
]
}- Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
- Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
- Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.
What comes back
What comes back is a pull request touching python/data_analysis.py: the three groupby calls rewritten to select ['Survived', 'Died'] or 'Fare' before .sum() and .mean(), nothing else in the function changed. The description quotes the TypeError from pandas 3.1.1 and from 2.2.3 with the line number, the FutureWarning from 1.5.3, and a small table of the per-class fare means and per-sex survival means on the old and new pandas, which match.
A reviewer checks that the selected columns are the ones each chart plots, that the agg('sum') on text columns is gone rather than merely silenced, and that no numeric_only=True was added where an explicit column list was possible. The run's record has every command with its full output, so the before and after can be read rather than taken on trust.
Run it on every Monday morning
Add an automation to the Project with this prompt and a schedule trigger, cron set to 0 9 * * 1 in your timezone. Each Monday one run installs the current pandas, exercises the analysis code and opens one pull request on its own branch when something broke; a week where nothing changed opens nothing and ends with a one-line answer, and a week where the fix cannot keep the numbers identical ends needs_review with the two outputs attached.
The same body works from your own code with POST /o/{org}/sessions and an API key when a dependency bot's pandas bump turns a notebook job red; send a request_id that is a UUID v5 of the bot's pull request number so a retried start returns the same run, and the key's webhook_url receives run.completed or run.failed with the session URL when it ends.
The use case this job follows, with its automation and trigger: Upgrade dependencies weekly.
Why run it on Rush?
The failure depends on which pandas is installed and on the data, so a laptop with an old environment never sees it, and a notebook that already ran the cell never re-runs it. The agent's sandbox starts from requirements.txt with nothing cached, installs each pandas release in turn, and keeps every command and its output in the run's record, so the pull request names the exact version boundary rather than guessing at it.
Correctness here is numeric, not just a green run. Selecting columns before aggregating changes the code path, and the only proof that the chart still shows the same thing is the same numbers from both versions. The agent computes them in the sandbox and puts them in the pull request, which is the check a reviewer would otherwise have to reproduce by hand.
Frequently asked questions
Why did this work before and break now?
Until pandas 2.0, groupby(...).mean() quietly dropped columns it could not average. 1.5 warned that the default of numeric_only would change, 2.0 made the reducer raise on text columns, and 2.1 gave the error the agg function failed wording; pandas 3 reads text as a str dtype and words the error differently again. The call was always averaging the wrong set of columns; the newer versions say so.
Is numeric_only=True a fine fix?
It makes the error go away and keeps the old behaviour of dropping non-numeric columns. The agent uses explicit column selection instead where it can, because the selection says what the chart is of and keeps working when a numeric column that should not be averaged, such as an id, is added later. The pull request says which it chose and why.
Will it pin pandas in requirements.txt?
Not unless asked. Pinning hides the break rather than fixing it, and the prompt tells the agent to leave the file alone. If you want a pin, say so in the prompt and the agent adds one with the version it tested.
What about the sum over text columns that did not raise?
The prompt asks the agent to report any aggregation over text that returns nonsense silently. Summing a group of names concatenates them on every pandas version, so the agent rewrites that call to select the two numeric columns too and quotes the concatenated output in the pull request as evidence.