All jobs

Fix "NotImplementedError: interp2d has been removed in SciPy 1.14.0"

Yes. An agent replaces each interp2d with RectBivariateSpline or RegularGridInterpolator and proves the new values equal the old ones to the last digit.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

The agent starts from the function that calls it. In biorack/omaat, a Jupyter toolkit for mass spectrometry imaging, omaat_lib.py defines barycentric_trapezoidial_interpolation(Nx, Ny, p, hexagonalOffset=0.5) at line 1303: it takes the four corner points of a trapezoid and returns the Nx by Ny grid of sample positions inside it, with every other row shifted by a hexagonal offset. Lines 1338 and 1340 build interpolate.interp2d([0,1], [0,1], px, kind='linear') and the same for py, where px and py are the corner coordinates as two-by-two lists, and then evaluate each on x_basis and y_basis, two linspace vectors, and flatten the result. Six call sites use the function, three in omaat_lib.py and three in the offline copy omaat_lib_offline.py, including the draggable-corner widget that regrids a sample array. setup.py lists scipy with no version, and .travis.yml tests on Python 2.7 and 3.5.

A fresh install today resolves SciPy 1.17.1, and the first call raises the error in the title with the rest of SciPy's message: for legacy code, RectBivariateSpline on regular grids and bisplrep/bisplev for scattered data are nearly bug-for-bug compatible, and for new code on regular grids, RegularGridInterpolator. The agent fixes the boundary by installing releases: 1.9.3 runs the call silently, 1.10.1 prints DeprecationWarning: interp2d is deprecated, 1.13.1 prints a warning naming 1.14.0 as the removal, and 1.14.0, published 2024-06-24, removes it. The repository also fails one line further on Python 3.11 or newer, where inspect.getargspec on line 1360 no longer exists, which the agent notes as a separate change.

The replacement is where care goes, because the two APIs disagree on axis order. interp2d(x, y, z) took z indexed as z[y][x] and f(xs, ys) returned a grid of shape (len(ys), len(xs)); RectBivariateSpline(x, y, z, kx=1, ky=1) takes z indexed as z[x][y] and its call returns (len(xs), len(ys)). So the port passes the transposed corner array and transposes the result back before flatten, with kx=1, ky=1 for the linear kind the code used. The RegularGridInterpolator form takes the same transposed array and a list of points built from a meshgrid of the two basis vectors, and returns the flattened values directly.

The proof is numeric. The agent runs the function with the docstring's example, a unit square whose corners are each moved by a seeded random offset, on Nx=20 and Ny=15, three times: with interp2d on SciPy 1.13.1, and with each replacement on 1.17.1. All three give arrays of 300 positions, and the maximum absolute difference between the old output and either new one is 0.0, so the regridded sample positions that every downstream step in the notebook reads are unchanged. The pull request carries that comparison, and it uses RectBivariateSpline in the code since it is the form SciPy names for legacy code and needs no meshgrid.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "claude",
  "input": "Our analysis code fails with \"NotImplementedError: `interp2d` has been removed in SciPy 1.14.0\". Find every interp2d call in the repository with its file, line, kind argument and the shape of the data it interpolates, and every caller of each function that contains one. Read the dependency files to say how scipy is pinned and which release a fresh install resolves, and install releases to find when interp2d started warning and when it was removed, quoting the messages. Replace each call with RectBivariateSpline for regular grids, with kx and ky set to match the old kind, taking care of the transposed data and result that the two APIs differ on; mention RegularGridInterpolator as the alternative. Prove equivalence numerically: run each changed function with its documented example on the last SciPy release that still has interp2d and on the current release with your replacement, and report the maximum absolute difference of the outputs. Note any other call in the same files that the current Python or SciPy also removes. Open a pull request with the diff, the version findings and the comparison.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

A pull request that changes lines 1338 to 1341 of omaat_lib.py and the matching lines of omaat_lib_offline.py: two RectBivariateSpline objects built from the transposed corner arrays with kx=1, ky=1, evaluated on the two basis vectors, transposed back and flattened, with the offset loop below untouched. The description names the resolved SciPy, the three releases where interp2d ran silently, warned and was removed, and the warning and error texts. Below that is the comparison: the example corners, the three runs, the output shapes and a maximum absolute difference of 0.0 against the old implementation on 1.13.1.

The reviewer checks the transposition, since that is the one place a port of interp2d goes wrong without an error: the .T on the corner arrays going in and the .T on the grid coming out, before flatten, so the flattened order matches what the hexagonal-offset loop below expects. The reviewer also reads the note about inspect.getargspec on line 1360, which fails on Python 3.11 in a different function, and decides whether it rides in the same pull request.

What the run does not exercise is the notebook. The function's callers include a widget that redraws on mouse drags, and the sandbox runs the function and the comparison script, not the Jupyter session; the pull request names the six call sites and leaves the interactive check to the reviewer.

Run it on every notebook or pipeline run that fails on a removed SciPy function

This error appears when an environment is rebuilt, usually in a scheduled pipeline or the first notebook session on a new machine, as a traceback in a job log. A pipeline runner that can send a custom header posts the log to a Rush webhook trigger's URL with the trigger's secret in Authorization: Bearer; a hosted notebook service that cannot goes through a small handler on your server that forwards it. The traceback lands after the saved prompt as the event, so the prompt tells the agent which file and line to open. Send an Idempotency-Key header to the trigger's URL set to the job's run id, so a retried job starts one run.

Code that is run by hand has no job to fail. For a set of research repositories, search each for interp2d( and start one run per repository through the API with the request above, each with a request_id that is a UUID v5 of the repository name, so a repeated sweep returns the runs it already started.

The use case this job follows, with its automation and trigger: Fix a bug from an alert.

Why run it on Rush?

A numeric port is only right if the numbers match, and matching them means running the old function on an old SciPy and the new one on the new SciPy, with the same inputs, in one place. A Rush sandbox holds both environments at once, one with 1.13.1 and one with 1.17.1, runs the function in each, and compares the arrays, with the seed, the shapes and the difference kept in the session. Nobody keeps two SciPys on a laptop for that.

The release history is part of the answer too: a team that sees a warning on 1.10 and an error on 1.14 can tell which of their environments will break next. Finding that by installing 1.9.3, 1.10.1, 1.13.1 and 1.14.0 in turn is a few minutes of sandbox time and a few lines in the pull request.

Frequently asked questions

Which SciPy release removed interp2d?

SciPy 1.14.0, published 2024-06-24. Installed one at a time, 1.9.3 runs the call silently, 1.10.1 prints a deprecation warning, and 1.13.1 prints one that names 1.14.0 as the removal. A scipy requirement with no bound resolves past 1.14 on every fresh install.

Is RectBivariateSpline a drop-in replacement?

Nearly, which is the word SciPy's own error message uses. The data array and the returned grid are transposed relative to interp2d, and the spline degrees are set with kx and ky instead of a kind string, so kind='linear' becomes kx=1, ky=1. With the two transposes in place the values match the old ones exactly in this fixture.

When would I use RegularGridInterpolator instead?

For new code on a regular grid, which is what SciPy's message recommends. It takes the same transposed data and a list of points rather than two axis vectors, so the caller builds the points with meshgrid. On this function it gives the same 300 values as the spline form; the pull request uses the spline because it keeps the call shape closest to the original.

What else breaks in this file on a current Python?

inspect.getargspec on line 1360, in init_default_params, which Python 3.11 no longer has. It is unrelated to SciPy and fails only when that function is called, so the pull request lists it with the line and leaves the reviewer to take it in the same change or the next.

Does the agent run the notebook?

No. It runs the changed function and the comparison script in the sandbox. The six call sites include a widget that redraws on mouse drags in a Jupyter session, and the pull request names them so the reviewer can open the notebook once and drag a corner.

More SciPy jobs

More jobs like this: Fix a bug from an alert

Keep reading