# Fix "TypeError: read_csv() got an unexpected keyword argument 'error_bad_lines'" in pandas 2

Yes. An agent replaces error_bad_lines with on_bad_lines, counts the rows the old flag was skipping, and fixes the positional separator that fails next.

[Run this on Rush](https://cloud.getrush.ai/run/fix-read-csv-error-bad-lines-type-error/pandas/). Opens the console with this prompt ready to send, in a Project that has your repository.

## What the agent does

pandas 2.0 removed two `read_csv` arguments, `error_bad_lines` and `warn_bad_lines`, in favour of one: `on_bad_lines`, which takes `'error'`, `'warn'` or `'skip'`. Code written for older pandas that passes `error_bad_lines=False` now stops with the `TypeError` in the title before a single row is read. The edit is short. The part worth an agent's time is the question the old flag buried: how many lines of this file were being thrown away, and did anyone know?

faizann24/Using-machine-learning-to-detect-malicious-URLs has the call in its plainest form. `AIserver.py` loads its training set with `pd.read_csv(allurls,',',error_bad_lines=False)`, where `allurls` is `./data/data.csv`. The repository's `REQUIREMENTS` file pins `pandas==0.19.2`, so the code predates the replacement argument entirely, and anyone who installs a current pandas instead of that pin meets the error at startup: the script trains its model inside `TL()` before `app.run`, so the Flask server never comes up.

The agent's first step is to translate the flag faithfully. `error_bad_lines=False` on its own meant skip the malformed line and print a warning, because `warn_bad_lines` defaulted to true. The matching value is therefore `on_bad_lines='warn'`, not `'skip'`, which would silence warnings the old code produced. Its second step is to run the new call, and here this file teaches something: the same line then fails with `read_csv() takes 1 positional argument but 2 positional arguments (and 1 keyword-only argument) were given`. pandas 2 made every argument after the path keyword-only, and the bare `','` in second position is the separator. It becomes `sep=','`, or is dropped, since a comma is the default.

Then it measures. With the committed `data/data.csv`, a `url,label` file, the agent reads the file under `on_bad_lines='error'` and under `'warn'` and compares row counts. If strict parsing succeeds, the flag was never skipping anything in this data and the honest change is to remove it, so that a future malformed line stops the job instead of shrinking the training set. If strict parsing fails, the agent reports each rejected line number and what is wrong with it, and keeps `'warn'`. Reading on, it lists what else it saw without changing it: `random.shuffle(allurlsdata)` is applied to a two-dimensional NumPy array, which duplicates some rows and drops others instead of permuting them, and `REQUIREMENTS` lists `sklearn==0.0`, a placeholder package and not scikit-learn itself.

## Start it with one API call

`POST https://api.phoenix.so/o/{org}/sessions`

```json
{
  "agent": "claude",
  "input": "This project fails on pandas 2 with \"TypeError: read_csv() got an unexpected keyword argument 'error_bad_lines'\". Find every call to pandas read_csv, read_table and read_fwf, and every pd.io.parsers call, that passes error_bad_lines or warn_bad_lines. Replace them with on_bad_lines, translating exactly: error_bad_lines=False with warn_bad_lines unset or True becomes on_bad_lines='warn'; error_bad_lines=False with warn_bad_lines=False becomes on_bad_lines='skip'; error_bad_lines=True is the default and the argument is removed. In the same calls, convert any positional argument after the file path to its keyword (the second positional argument is sep), because pandas 2 accepts only the path positionally. Then measure what the flag was doing. For every data file committed to the repository that one of these calls reads, parse it once with on_bad_lines='error' and once with 'warn', and report the row counts and the line numbers of any rejected lines with the reason. Where strict parsing succeeds, remove on_bad_lines so a malformed line raises in future, and say so. Where it fails, keep 'warn' and list the lines. Do not edit any data file. Install the pandas version the project pins, if it still installs, and the newest pandas, run the changed code paths under both, and include the commands and output. If the pinned version cannot be installed on a current Python, say which Python it needs. Report, without changing, any other pandas 2 or NumPy incompatibility you can see in the files you touched. Open a pull request.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
```

1. **Get an API key.** In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
2. **Send the request.** POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
3. **Read what came back.** Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

## What comes back

A pull request with a small diff and a table. The diff changes each affected `read_csv` call: the removed arguments are gone, positional extras are keywords, and `on_bad_lines` appears only where a file needs it. The table has one row per data file: rows read under strict parsing, rows read under `'warn'`, and the rejected line numbers when the two differ.

The reviewer's decision sits in that table. Equal counts mean the old flag was decoration and the pull request removes it, which makes the loader stricter than before; a reviewer who knows that upstream data is sometimes ragged may prefer to keep `'warn'`, and that is one word to change. Unequal counts mean rows have been vanishing, and the listed line numbers let someone open the file and decide whether those rows are junk or records worth repairing.

Findings outside the task are in a separate section of the description with the file and line for each. They are described, with a one-line reproduction where the agent wrote one, and left for their own change.

## Run it on every Sentry issue

A loader that raises at import or startup shows up as a new issue in error tracking the first time a rebuilt image picks up pandas 2. Sentry signs the webhooks it sends and cannot add an `Authorization` header of your choosing, so it cannot call a Rush `webhook` trigger directly. A small handler on your own server receives Sentry's webhook, checks that the title contains `error_bad_lines`, and sends the request above with your API key.

Sentry delivers again when an issue regresses or a rule re-fires. Have the handler send a `request_id` that is a UUID v5 of the Sentry issue id, so a repeated delivery returns the run already started instead of opening a second pull request. The handler can also add the issue's stack trace to the end of `input`; the agent then starts from the failing call and still searches the rest of the repository.

## Why run it on Rush?

The answer depends on running the parser, twice, on the real file, under two pandas versions. That is a few minutes of setup nobody wants to do by hand for a one-argument change: an environment with the old pin, another with the current release, both reads, both counts. The agent builds both in its sandbox and the session keeps every command with its output, so the row counts in the pull request are numbers it observed and a reviewer can trace each to the call that printed it.

It also keeps the change from being a blind rename. Swapping the flag for `on_bad_lines='skip'` makes the error go away and preserves a loader that discards input without saying so. Because the run checks whether anything is actually discarded, the result is either a stricter loader or a list of the lines being lost, and both are more useful than a green build.

## Frequently asked questions

### What is the exact replacement for error_bad_lines=False?

`on_bad_lines='warn'` if you never set `warn_bad_lines`, because the old default printed a warning for each skipped line. Use `on_bad_lines='skip'` only where the code also passed `warn_bad_lines=False`. `on_bad_lines` has been available since pandas 1.3, so the new spelling also runs on the last 1.x releases.

### I replaced the argument and now get "takes 1 positional argument but 2 positional arguments". Why?

pandas 2 accepts only the file path positionally. A call such as `pd.read_csv(path, ',', ...)` passed the separator in second position; write `sep=','` instead. The agent converts these in the same pass, since they fail one after the other on the same line.

### Can I see which lines are being skipped?

Yes. `'warn'` reports each skipped line with its number and the field count it found. For more control, `on_bad_lines` also accepts a function when `engine='python'`: it receives the split fields of the bad line and returns a repaired list or `None` to drop it.

### Should I just pin pandas below 2 instead?

A pin is a fair stopgap for a job that must run tonight, and the agent says so when the old version still installs on your Python. It does not remove the problem, because older pandas releases stop getting wheels for new Python versions. The code change is two arguments per call and works on both sides of the boundary.

## More pandas jobs

- [Upgrade a Python Library from pandas 1.x to pandas 3](https://getrush.ai/run/upgrade-pandas/pandas/)
- [Fix agg function failed [how->mean,dtype->object] After a pandas Upgrade](https://getrush.ai/run/fix-agg-function-failed-how-mean-dtype-object/pandas/)
- [Fix "'DataFrame' object has no attribute 'append'" in pandas 2](https://getrush.ai/run/fix-dataframe-append-attribute-error/pandas/)

## More jobs like this: Fix a bug from an alert

- [Fix "Error: req#logout requires a callback function" in Passport 0.6 and later](https://getrush.ai/run/fix-req-logout-requires-a-callback-function/passport/)
- [Fix "LookupError: Resource punkt_tab not found" in word_tokenize](https://getrush.ai/run/fix-resource-punkt-tab-not-found/nltk/)
- [Fix "ImportError: cannot import name 'SafeConfigParser' from 'configparser'" on Python 3.12](https://getrush.ai/run/fix-safeconfigparser-import-error/configparser/)

## Keep reading

- [How a Rush run works on your repository, from request to pull request](https://getrush.ai/cloud/)
- [Compare cloud agents, agent sandboxes and AI code review](https://getrush.ai/compare/)
- [Rush pricing: Free, Pro and Max](https://getrush.ai/pricing/)
