What the agent does
The two numbers in this message are the giveaway. A compiled Python package, built when NumPy 1.x was current, expects NumPy's dtype object to be 96 bytes. NumPy 2 made it 88. The package imports NumPy, compares sizes, and raises before any of your code runs. It happens to projects that pin the compiled package and leave NumPy open, because every fresh install then pairs an old binary with the newest NumPy. Nothing in the repository changed. The lockless requirements file simply resolved differently this month than it did last year.
ajosegun/HealthQA_API has exactly that file. Its requirements.txt is five lines: sentence-transformers, pandas==1.3.5, numpy, fastapi, uvicorn. The first statement in app.py is import pandas as pd. On Python 3.10, installing that file today brings in a NumPy 2 release next to pandas 1.3.5, and the import fails inside pandas/_libs/interval.pyx with the error in the title. The API never starts. vercel.json deploys that same app.py, so a redeploy that reinstalls dependencies is exposed to the same resolution.
The agent starts by identifying the package at fault instead of assuming. The traceback's last frame inside site-packages names it, and here that is pandas. In a project with several compiled dependencies there can be more than one, so it imports each top-level module the code uses, one per process, and records which fail. Then it builds the two possible repairs side by side. The first adds an upper bound, numpy<2, and changes nothing else: the old pandas gets the NumPy it was built against. The second raises the package to a release built for NumPy 2. For pandas that boundary is specific: 2.0.3 installs beside NumPy 2 and fails at import with this same message, 2.1.4 and 2.2.1 declare numpy<2 so an installer refuses the pair, and 2.2.2 imports cleanly.
Choosing between them takes more reading. This application loads its data with pd.read_pickle('df_health_ds.pkl'), a file it downloads at startup and that is not in the repository. Pickled DataFrames are tied to the pandas that wrote them more tightly than CSV files are, so jumping from pandas 1.3 to 2.2 could trade an import error for an unpickling one. The agent cannot see that file, says so, and makes the bound its default proposal, with the upgrade described as a follow-up that needs the real data. It also checks the Python side of the pin: pandas 1.3.5 publishes no wheel for Python 3.11, so the bound is a clean install only while the runtime stays on 3.10 or older; past that, pip has to compile pandas from source. Along the way it lists what it noticed and left alone: app.py imports requests, which the requirements file gets only through another package, and the README's start command names main:app while the file is app.py.
Start it with one API call
POST https://api.phoenix.so/o/{org}/sessions with this body:
{
"agent": "claude",
"input": "This project fails at import with \"ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject\". Find how dependencies are installed (requirements files, pyproject.toml, setup.py, a lockfile, a Dockerfile, deployment config) and which Python version the project targets; if none is stated, find the newest Python for which every pinned package has a wheel and say that you chose it. Create a fresh virtual environment with that Python, install exactly as the project does, and import every third-party top-level module the code uses, each in its own process. Record which imports fail and, from each traceback, which installed package contains the compiled extension that raised. Report the installed versions of NumPy and of each failing package. Then build two candidate fixes in separate fresh environments. Candidate A: add the tightest NumPy upper bound that makes all imports succeed and change nothing else. Candidate B: raise each failing package to the oldest release that imports correctly with the newest NumPy, change nothing else, and note every release in between that fails or cannot be installed. For both, repeat the import check, run the test suite if there is one, and start the application's entry point far enough to pass its imports without calling external services or downloading data. Check the code for uses that tie it to the old package version, especially pickle files, read_pickle, joblib dumps and saved models, and say whether candidate B could break loading them. Recommend one candidate and explain why; implement it in the dependency files. If the recommended fix only works up to some Python version, state it. List anything else you noticed in the dependency files without changing it. Open a pull request with both candidates' output.",
"resources": [
{
"url": "https://github.com/your-org/your-app"
}
]
}- Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
- Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
- Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.
What comes back
A pull request that edits the dependency file by one line: an upper bound on NumPy, or a higher version of the package that was built against the old one. The description names the package the traceback pointed to, the versions that were installed when it failed, and the import check before and after, module by module.
Both candidates are written up even though only one is in the diff. For the one not chosen, the description gives the exact line to use and what the agent saw when it tried it, including any intermediate versions that fail or will not install. A team that wants the upgrade later starts from a tested version number instead of trial and error.
The reviewer weighs one thing the agent cannot settle alone: data written by the old version. If the service reads pickles, saved models or other version-bound files that are not in the repository, the bound is the safe merge and the upgrade needs a test against real data first. The description says which such reads it found and where, and states the Python version beyond which the bound stops being installable.
Run it on every failed build
This failure shows up in whatever builds your image or environment from scratch, so let that job report it. Create an automation in the Project with a webhook trigger and the prompt above, and store the trigger's secret when the console shows it. In your CI job's failure step, call the trigger's URL with Authorization: Bearer <secret>, and send the trigger an Idempotency-Key header set to the commit SHA so a retried job starts nothing new. Send the last lines of the build log as the POST data; up to 64 KB of it becomes the event text after the prompt.
The call answers 202 with a status: queued when a run was created, deduped when that commit already started one. Not every failed build is this error, so begin the saved prompt with an instruction to stop and say so in one sentence when the log does not contain numpy.dtype size changed.
The use case this job follows, with its automation and trigger: Fix a bug from an alert.
Why run it on Rush?
The error cannot be reproduced in an environment that already works. A developer's virtual environment was resolved before NumPy 2 existed and still imports fine, which is why the report often comes from a new teammate or a rebuilt container. The agent's sandbox is empty on every run, so the install it performs is the fresh install that fails, on the Python version the project names and not the one a laptop happens to have.
Finding the version boundary is a search, and each probe is a new environment: install, import, record, discard. Trying four releases of one package against the newest NumPy is tedious by hand and quick for an agent that can create environments freely. Every probe's command and result stays in the session record, so the claim that a given release is the first to work is backed by the failures just below it.
Frequently asked questions
What do the numbers 96 and 88 mean?
They are the size in bytes of NumPy's dtype structure. The compiled package was built when it was 96, and the NumPy now installed reports 88, which is NumPy 2. The mismatch tells you the direction: an older binary is meeting a newer NumPy, so either NumPy comes down or the package goes up.
Is pinning numpy<2 a real fix or a workaround?
It is a correct fix for the versions you have pinned, since it restores the combination the package was built for. It has a shelf life: old package versions stop publishing wheels for new Python releases, and at that point the pin cannot be installed. The run tells you which Python version that is for your project.
The traceback mentions a package I never import. Why?
Compiled extensions are often pulled in by a dependency of a dependency. The agent reads the traceback for the file that raised, maps it to the installed distribution, and then finds which of your declared requirements brought it in. The fix goes on the package that owns the extension, not on your import.
I upgraded pandas and still get the error. How?
Some releases that predate NumPy 2 do not declare an upper bound on it, so an installer pairs them with NumPy 2 and they fail when imported. pandas 2.0.3 behaves this way. The first pandas release that imports with NumPy 2 is 2.2.2, and the agent verifies the boundary for whichever package is failing in your project.
Will a package built for NumPy 2 still work if I keep NumPy 1?
Generally yes. Extensions compiled against NumPy 2 are designed to run on NumPy 1.x as well, which is why raising the package is the durable repair. The agent confirms it for your pins by importing under both.