All jobs

Fix "ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape"

Yes. An agent finds the ragged list behind the NumPy 1.24 error, traces it to the np.matrix that made each entry a 1 by 1 array, and reruns the script cleanly.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

NumPy 1.24 turned a warning into an error. Building an array from nested sequences whose parts have different shapes used to produce an object array with VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences; since December 2022 the same call raises ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after N dimensions. The message names the depth at which the shapes stopped agreeing, which is the clue: a (3, 3) + inhomogeneous part means a 3 by 3 literal whose entries are not scalars.

A geometry script like alyssaq/3Dreconstruction shows how that happens without any list of different lengths in sight. structure.py defines skew(x) on line 104, which returns np.array([[0, -x[2], x[1]], [x[2], 0, -x[0]], [-x[1], x[0], 0]]), the usual cross-product matrix. cube_reconstruction.py calls it on line 104 with translation_mat_wrt_c1, which line 91 builds as np.matrix([3, 0, 1]).T. Indexing a 3 by 1 matrix with x[2] gives a 1 by 1 matrix, not a number, so the nine entries are tiny arrays and the literal is ragged. On NumPy 1.23 the script runs with the warning at structure.py line 110 and then prints a wrong True essential matrix full of matrix([[0.]]) entries before dying later with shapes (3,3) and (1,1) not aligned; on 1.24.0 and on the current 2.x release it stops at skew with the error in the title.

The agent reproduces both versions in clean environments with the script's own requirements.txt, numpy and matplotlib unpinned, and reads the traceback upward from the array literal to the call site to the np.matrix constructor, which is the cause. np.matrix is a legacy class whose indexing never returns scalars, and the same file builds translation_mat on line 78 the same way. The fix goes in skew, which should accept whatever a caller passes: x = np.ravel(x) as its first line turns the matrix, a column vector or a plain list into three scalars, and the two other callers in structure.py, reconstruct_one_point on lines 20 and 21 and the epipole on line 142, keep working because they already pass flat arrays.

Then the agent reruns the script on the current NumPy. One more change is needed to reach the end on current libraries, fig.gca(projection='3d') in the plotting helper, which newer matplotlib refuses; it becomes fig.add_subplot(projection='3d'). With both, cube_reconstruction.py runs through and prints the true and computed essential matrices and the recovered camera parameters with no warning. The agent greps the repository for every other np.matrix and for np.int, which NumPy 1.24 also removed and which example.py uses on line 45, and lists them in the pull request with replacements, running the ones it can and marking the rest.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "claude",
  "input": "This project fails on current NumPy with ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape. In a clean virtual environment install the project's requirements with the current NumPy, run the failing script and quote the traceback. In a second environment install NumPy 1.23 and run the same script with warnings enabled; quote the VisibleDeprecationWarning with its file and line, and say whether the script then produced correct output or merely continued. Trace the ragged array to its cause: find what the elements of the literal are, where they come from, and whether np.matrix, a column vector or lists of unequal length is responsible. Fix it at the function that builds the array so that any caller shape works, without converting to dtype=object unless the data is genuinely ragged. Rerun the script on the current NumPy and quote its output. Grep the repository for every other np.matrix use and for np.int, np.float, np.bool and np.object, list each with a replacement, apply the ones in code you can run, and mark the rest as not run. If the script also needs a change for current matplotlib, make it and say so. Open a pull request with both tracebacks, the diff and the rerun output.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

A pull request that changes the array-building function and the plotting call, and pins nothing, because the fix makes the code correct on every NumPy rather than holding it on an old one. The description opens with the two tracebacks: the ValueError from the current release and the warning from 1.23 with the wrong matrix the old run printed after it, so a reviewer sees that the error is not new breakage but an old silent bug surfacing.

Below that is the cause in one paragraph, np.matrix indexing returning 1 by 1 matrices, and the rerun output on the current NumPy with the essential matrices and camera parameters the script now prints. The session's record keeps both environments and every command.

The description ends with the table of remaining np.matrix and removed-alias uses, each with file, line and replacement, split into applied and not run. A reviewer who knows the repository decides whether to retire np.matrix everywhere in a follow-up, which the pull request recommends and does not do.

Run it on every failed workflow run on a NumPy upgrade

This error arrives with a dependency bump, usually the one that moves NumPy past 1.24 or to 2.0, and shows up as a failed job rather than a bug report. Add an automation to the Project with the prompt above and a github trigger on workflow_run.failure, so the failed workflow is placed after the prompt as the event. The saved prompt is the only instruction, so a failure with another cause gets a short report and no pull request.

For an update opened by a bot, the github trigger does not fire, since bot senders are excluded; add a webhook trigger to the same automation and have a workflow step post the failing lines to its URL with Authorization: Bearer and the trigger's secret when the log contains inhomogeneous shape. Send an Idempotency-Key header to the trigger's URL set to the workflow run id, so a re-run of the failed job starts nothing new.

The use case this job follows, with its automation and trigger: Fix a bug from an alert.

Why run it on Rush?

The tempting fix is dtype=object, which the old warning itself suggests, and it is wrong here: it reproduces the object array that made the 1.23 run print a nonsense matrix and fail three lines later. Telling a genuinely ragged dataset from a scalar that is not a scalar takes running the script on both releases and reading what each produced. A Rush sandbox holds both environments, and the session keeps the outputs side by side.

Numerical code also fails late and quietly. The run proves the fix by executing the script to its last print on the current release, not by making the exception go away, and it reports the other removed names it found so the next NumPy bump does not start the same loop. A team with several research repositories pinned to old NumPy starts one run per repository with the same request.

Frequently asked questions

Should I add dtype=object like the old warning said?

Only when the data really is ragged, such as a list of sequences of different lengths that you mean to keep as objects. When the entries are supposed to be numbers and happen to be 1 by 1 arrays, dtype=object hides the bug and breaks the next matrix product. The agent checks which case applies by reading what the elements are.

Why did this work on the old NumPy?

It did not, quite. NumPy 1.23 and earlier built an object array and warned; arithmetic on it either produced wrong values or failed later with a shape error. 1.24 moved the failure to the point where the ragged array is built, which is the right place to find the cause.

What is wrong with np.matrix?

Indexing it never returns a scalar: m[2] on a 3 by 1 matrix is a 1 by 1 matrix, and its multiplication operator differs from ndarray. NumPy documents the class as legacy and recommends ndarray with @ for products. The agent fixes the function that received the matrix and lists the other constructors for a follow-up.

Does np.ravel change results for callers that already pass arrays?

No. On a one-dimensional array of three values it returns the same values, and on a 3 by 1 column it returns them flattened, which is what the formula needs. The agent runs every caller it can reach to show the outputs are unchanged, and names any caller it could not run.

More NumPy jobs

More jobs like this: Fix a bug from an alert

Keep reading