All jobs

Fix "AttributeError: 'DataFrame' object has no attribute 'ix'" in an Old pandas Script

Yes. An agent replaces each .ix call with .loc or .iloc according to what the indexer selects, then runs the script on current pandas and fixes what breaks next.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

pandas deprecated the .ix indexer in 0.20 and removed it in 1.0. Scripts written before that still circulate, in course projects, in notebooks attached to papers, in a colleague's repository you were asked to rerun. yazanobeidi/fraud-detection is a typical case: a 2017 machine learning project whose adasyn.py rebalances a credit card dataset before training. Line 24 selects the feature columns with X_train.ix[:, X_train.columns != 'Class']. On any current pandas that line raises the error in the title.

The agent first tries to recreate the original environment, and reports why it cannot. requirements.txt pins pandas==0.19.2, but the file is a pip freeze of a Python 2 desktop: it includes functools32, subprocess32, unity-lens-photos and adium-theme-ubuntu, and the notebook's kernel is named python2. Those pins do not install on a current Python, so going back is not an option and the script has to move forward. The agent installs current pandas, NumPy, scikit-learn and imbalanced-learn and runs the script as the README says.

The .ix line is the fourth thing that fails, and the agent meets the first three on the way to it. The README says to run unzip data/creditcardfraud.zip from the repository root, the archive holds creditcard.csv at its top level, and the script reads data/creditcard.csv, so the first error is a missing file. Line 17 is ADASYN(n_jobs=3), and current imbalanced-learn no longer accepts n_jobs. Line 18 writes to ada.logger, an attribute the class no longer has. Past the .ix line, fit_sample has become fit_resample, and the script writes its output into a pickle/ directory the repository does not contain. A developer who searched for the .ix message has probably already worked around the earlier ones by hand; the agent fixes all of them and lists them in order.

For .ix itself the replacement depends on what the indexer holds, because .ix accepted labels and positions alike. Here the row part is a full slice and the column part is a boolean mask, so X_train.loc[:, X_train.columns != 'Class'] selects the same columns, and X_train.drop(columns='Class') says it more plainly. Where an old script passes integers, the agent checks whether the index is itself made of integers before choosing between .loc and .iloc, since that is the case where the two give different rows. It also reads project.ipynb, where the same .ix call sits in commented-out lines of one cell and a later cell loops over a plain dict with dataset.iteritems(), a Python 2 method that needs items() on Python 3.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "claude",
  "input": "Running this project's Python scripts on a current pandas fails with \"AttributeError: 'DataFrame' object has no attribute 'ix'\". First read the requirements file and the notebook metadata and say which Python and pandas the project was written for; try to install those pins and, if they do not install on a current Python, say which packages block it and move forward instead. Install current pandas and the current versions of the other libraries the scripts import. Follow the README's steps to prepare the data, then run each script and record every error in the order it appears, including ones that come before the .ix line (missing files or directories, arguments or methods that other libraries have removed). Find every use of .ix in the .py files and in the code cells of any notebook, including commented-out code. For each one, decide from the indexer whether it selects by label or by position, replace it with .loc or .iloc, and where the index holds integers explain why you chose the one you did. Fix the other removed pandas APIs you meet (iteritems, as_matrix, append, sort) and the calls into other libraries that no longer exist. Do not change model parameters or the sampling logic. Check that the selection returns the same columns and shape before and after on a small sample. Replace the frozen requirements with the direct dependencies and the versions you ran. Open a pull request listing each error in order, each replaced call with its line, and the output of the script after the changes.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

A pull request that changes the script, the notebook's affected cells, the README's data step and requirements.txt. Each replaced indexer appears in the description with its file and line and the reason for .loc or .iloc. The other fixes are listed in the order the script reached them: the data path, the ADASYN constructor, the logger lines, fit_resample, the output directory. The requirements file shrinks from a full desktop freeze to the handful of packages the code imports, at the versions the agent ran.

A reviewer looks at two things. The first is any .ix call with integer arguments, where the description should say what the index was and why the chosen replacement selects the same rows. The second is the claim that nothing statistical changed: the diff should show no edited sampling fractions, model settings or random seeds. If the full dataset is too large or too slow to process inside the session, the agent says how far the script ran and on what sample, and does not describe the unrun part as verified.

Run it on every Linear issue labelled legacy-script

Old analysis code tends to surface one request at a time: someone needs last year's numbers rerun, opens an issue and finds that the script no longer starts. Give the Project an automation with a linear trigger on linear.issue.labeled for the label your team uses, and a prompt that reads the issue, runs the script it names on current libraries and fixes what fails. The issue is added after the prompt as event text, so the prompt itself stays general. The trigger needs a Linear connection for the member the automation runs as, made in the console under Settings → Connections.

To clear a backlog instead, list the repositories that still call .ix and send one request per repository from your own script, with the resources URL changed each time. Runs past your key's concurrency limit are created queued and start as slots free, so the script needs no retry loop.

The use case this job follows, with its automation and trigger: Fix a bug from an alert.

Why run it on Rush?

Porting an old data script is mostly waiting: install, run, read the traceback, fix one line, run again. Each step is small and none can be skipped, because the errors hide behind one another. The agent does that loop in its own sandbox with a clean Python, so nothing on your machine is upgraded or downgraded to make an old project run, and it works on the real dataset where the session has room for it. The command and output of each attempt stay in the session transcript.

The pull request also records a judgement that is easy to make silently and get wrong. .ix with integers meant different things on different indexes, and a mechanical replacement can return other rows without raising anything. The run states its reasoning per call, so a reviewer who knows the data can confirm it in a minute.

Frequently asked questions

Should .ix become .loc or .iloc?

.loc when the code selects by label or with a boolean mask, .iloc when it selects by position. They differ only when the index is made of integers that are not 0, 1, 2 and so on, for example after filtering rows. The agent inspects the index at that point in the script before choosing.

Can I pin an old pandas instead of changing the code?

Only if you also keep an interpreter old enough to install it. .ix last existed in the 0.25 releases, which predate current Python versions. For a project frozen on Python 2 packages, as in this example, the pins no longer install at all.

Does it fix the notebook as well as the .py files?

Yes. It reads the code cells of each .ipynb file, applies the same replacements and leaves the stored outputs alone unless it reran the cell. Calls in commented-out cells are updated too and listed separately, since they were not executed.

What about errors from scikit-learn or imbalanced-learn in the same script?

They are part of the job. A script old enough to use .ix usually calls other libraries the way they worked that year, such as fit_sample on a sampler. The agent fixes each call it reaches and names the library and the replacement in the pull request.

More pandas jobs

More jobs like this: Fix a bug from an alert

Keep reading