All jobs

Fix the delim_whitespace Keyword in pd.read_csv Is Deprecated Warning

Yes. An agent reproduces the pandas FutureWarning, shows the TypeError pandas 3 raises, swaps delim_whitespace for sep=r"\s+" and opens a PR with the reader checked.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

The agent clones the repository into its own cloud sandbox and finds every reader first. In a small training script like servomac/Human-Activity-Recognition, data.py wraps pd.read_csv(filename, delim_whitespace=True, header=None) in _read_csv, which load_signals calls once per signal file for the nine Inertial Signals of the UCI HAR dataset and load_y calls for the label files before pd.get_dummies. requirements.txt pins pandas==1.4.3, numpy==1.23.1, keras==2.9.0 and tensorflow==2.9.1, the Dockerfile starts FROM python:3.8, the README downloads the dataset zip with wget and runs python3 main.py in that image, and main.py trains an LSTM on what load_data returns.

Then it reproduces the warning across releases, on a two-line whitespace-separated file so the dataset download is not needed for the check. pandas 1.5 reads it silently as a (2, 3) frame. pandas 2.2 reads the same frame and prints FutureWarning: The 'delim_whitespace' keyword in pd.read_csv is deprecated and will be removed in a future version. Use sep='\s+' instead. pandas 3 refuses the call outright: TypeError: read_csv() got an unexpected keyword argument 'delim_whitespace'. With sep=r"\s+" both 2.2 and 3 return the identical (2, 3) frame under -W error. The pinned 1.4.3 has no wheel for a current Python and fails to build, which the agent notes as the reason the comparison starts at 1.5.

The fix is one argument in one function: pd.read_csv(filename, sep=r"\s+", header=None), in _read_csv, so every signal and label file goes through the one changed call. Because the regex separator is accepted by every pandas release the repository could pin, the change is safe before and after any bump, and the agent leaves pandas==1.4.3 in requirements.txt unless asked to move it, noting that tensorflow==2.9.1 and numpy==1.23.1 are pinned beside it and a pandas bump should be checked against them in the Docker image the README names.

To show the reader still produces what the model expects, the agent downloads the dataset from the README's URL when the sandbox can reach it, runs load_data() and prints the four array shapes, which the docstring in data.py describes as 7352 train and 2947 test samples of 128 timesteps by 9 signals; without the download it checks the small file only and says so.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "claude",
  "input": "This Python project prints \"FutureWarning: The 'delim_whitespace' keyword in pd.read_csv is deprecated\" and will break on pandas 3. Read requirements.txt and every file that calls pandas first, and list each read_csv or read_table call that passes delim_whitespace with its file and line. Reproduce the warning under python -W default with the pandas 2 release pip resolves today, and the error with the current pandas 3 release, on a small whitespace-separated file. Replace delim_whitespace=True with sep=r\"\\s+\" in each call, keeping every other argument, and show the same frame comes back on both releases under -W error. If the repository's data is downloadable from a URL in its README, load it through the changed reader and print the resulting shapes. Do not change the pinned pandas version in this change; say whether it should move and what else is pinned beside it. Open a pull request that quotes the warning, the error and the passing checks.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

What comes back is a pull request touching data.py only: delim_whitespace=True replaced by sep=r"\s+" in _read_csv, with header=None and the rest unchanged. The description quotes the FutureWarning from pandas 2.2 and the TypeError from pandas 3 with the call that produced them, the identical frame shape from the fixed call on both, and, when the dataset download succeeded, the four array shapes load_data() returned.

A reviewer checks that every reader in the repository went through the one helper so none was missed, that \s+ is what the files really need, and reads the note on whether to bump pandas next to the tensorflow and numpy pins. The run's record holds each pandas version's output in full.

Run it on every Monday morning

Add an automation to the Project with this prompt and a schedule trigger, cron set to 0 9 * * 1 in your timezone. Each Monday one run installs the current pandas releases beside the pinned one, runs the readers under -W error and opens one pull request on its own branch when a deprecated keyword shows up; a week where the frames differ after the rewrite ends needs_review with both outputs attached, and a clean week opens nothing.

The same body works from your own code with POST /o/{org}/sessions and an API key when a notebook or a CI log first shows the FutureWarning; the key's webhook_url receives run.completed or run.failed with the session URL and the pull request when the run ends.

The use case this job follows, with its automation and trigger: Upgrade dependencies weekly.

Why run it on Rush?

A pandas deprecation is a countdown with a hidden end date. The warning appears on 2.x and the call stops working on 3.x, and whether your project is on one or the other depends on a pin in a file nobody has opened since the model was trained. The agent installs each release in its own sandbox, runs your reader under -W error and keeps every output in the run's record, so the pull request shows the warning, the error and the fixed call side by side instead of asking you to trust a changelog.

It also keeps the change proportional. One keyword in one helper is replaced, the pins stay where they are, and the bump that would actually change behaviour is named as the next step together with the other pins it must be checked against. What comes back is a one-line pull request with the proof attached, ready to merge before the next pip install surprises a training run.

Frequently asked questions

Is sep=r"\s+" exactly what delim_whitespace=True did?

Yes. delim_whitespace=True was an alias for the regex separator \s+, which splits on one or more whitespace characters, and the pandas warning text itself names that replacement. The agent shows the same frame shape from both calls on the same file to prove it.

Does the fix work on the pinned pandas 1.4.3 as well?

The regex separator has been accepted for years, so the change is safe on 1.x too. The pinned 1.4.3 does not build on a current Python, so the agent checks 1.5 and the 2.x and 3.x releases and says so; the repository's own python:3.8 image is where 1.4.3 still installs.

Will it bump pandas while it is there?

Not in this change. tensorflow==2.9.1 and numpy==1.23.1 are pinned beside it, and a pandas bump has to be checked against those in the Docker image the README builds. The pull request names that as the next run.

What if the data is not downloadable?

Then the agent proves the reader on a small whitespace-separated file it writes itself and says in the pull request that the dataset was not loaded. Attach the files to the run as input blocks, or point the prompt at a URL, and it runs the full load_data() and prints the shapes.

More pandas jobs

More jobs like this: Upgrade dependencies weekly

Keep reading