All jobs

Fix "Error: pg_config executable not found" When Installing psycopg2

Yes. An agent works out whether your image should compile psycopg2 or use psycopg2-binary, edits the Dockerfile or requirements to match, and builds it.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

pip install stops with Error: pg_config executable not found. followed by pg_config is required to build psycopg2 from source. The package named psycopg2 on PyPI is published for Linux as source only. Installing it compiles a C extension against libpq, PostgreSQL's client library, and pg_config is the program that tells the build where libpq's headers are. A machine with PostgreSQL's development package has it. A slim container image does not, and usually has no C compiler either. The error text itself names the other route: the psycopg2-binary package, which ships a compiled extension with libpq included.

rubinelli/sheetdrop has the failing combination in two short files. src/Dockerfile is six instructions: FROM python:3.11-slim-buster, a WORKDIR, COPY requirements.txt ., RUN pip install -r requirements.txt, then the copy and the uvicorn command. There is no apt-get line. src/requirements.txt lists psycopg2==2.9.9 under the comment # PostgreSQL / Redshift, beside PyMySQL==1.1.1, cx_Oracle==8.3.0, pyodbc==5.1.0, ibm_db==3.2.3 and fdb==2.0.2. The comment above them tells the user to keep only the driver for the database they plan to use, and the README repeats it. Someone who builds the image without trimming that list meets pg_config first.

The agent reproduces the failure without guessing at the environment. On Python 3.11 with no PostgreSQL packages, pip install psycopg2==2.9.9 ends in the pg_config error, and pip download --only-binary :all: psycopg2==2.9.9 finds no distribution at all, which confirms there is no wheel to fall back on. psycopg2-binary==2.9.9 installs and imports. Then it reads how the driver is used. src/sheetdrop/db.py passes a URL to sqlalchemy.create_engine, and src/config.yaml.sample gives postgresql://user:password@localhost/sheetdrop as the PostgreSQL form. Both packages install the same psycopg2 module, so that URL and every import keep working whichever is chosen.

There are two correct fixes and the agent picks by what the image is for. For a development image, or a project whose README builds once and runs, it swaps the line to psycopg2-binary at the same version: one changed line, no compiler in the image. For a production image it keeps the source package and adds the build dependencies, libpq-dev and gcc, in a build stage, copying the installed packages into a final stage that has only the runtime library libpq5. To prove either one it builds the image. The sandbox has no Docker, so the agent installs it first with sudo apt-get install -y docker.io and starts dockerd, then runs docker build and an import of psycopg2 inside the result. It also checks the base image before adding an apt-get line, because a tag for an old Debian release may no longer reach its package mirrors.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "claude",
  "input": "Building this project fails with \"Error: pg_config executable not found. pg_config is required to build psycopg2 from source.\" Find every place psycopg2 or psycopg2-binary is declared (requirements and constraints files, pyproject.toml, setup.py, lockfiles) and every place dependencies are installed (Dockerfiles, compose files, CI workflows, Makefiles, deploy scripts), with the base image or OS and the Python version of each. Reproduce the error in a clean environment on the same Python version without PostgreSQL development packages and keep the output. Check whether a wheel exists for the pinned version on that Python; if psycopg2-binary is pinned and still tries to compile, the pin predates wheels for that Python, so raise it to the lowest release that has one. Read how the driver is used (SQLAlchemy URLs, Django settings, direct imports) and confirm the change you choose keeps every import and connection URL working. Choose one fix and explain the choice. Option A, for development, test and CI environments: depend on psycopg2-binary at the same version. Option B, for production images: keep psycopg2 and install the build dependencies (libpq-dev and a C compiler, or postgresql-dev, gcc and musl-dev on Alpine), in a build stage when the Dockerfile is or can be multi-stage, with only the runtime library libpq5 in the final stage. Do not do both. Before adding package manager commands to a Dockerfile, check that the base image's package repositories still work; if they do not, say so and propose a base image tag on the same Python version. Apply the change everywhere dependencies are installed. Install Docker in the sandbox (sudo apt-get install -y docker.io, then start dockerd), build each Dockerfile you changed, and run python -c \"import psycopg2; print(psycopg2.__version__)\" in the built image. If other packages in the same file fail to build, list them with their errors and do not fix them. Open a pull request with the failing output, the build output and the result of the import.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

A pull request that is either one line in a requirements file or a few lines in a Dockerfile, never both. The description says which fix was chosen and the reasoning in two or three sentences: what the image is used for, whether a wheel exists for the pinned version on the image's Python, and what the alternative would have cost in image size or build time.

The evidence follows. First the original failure with the pg_config message. Then the tail of docker build for each changed Dockerfile and the output of importing psycopg2 inside the image, which prints the version and the features it was compiled with. If the build stopped later on a different package, that failure is shown too. In a requirements file like the fixture's, the Oracle, ODBC, DB2 and Firebird drivers on the following lines each need native libraries of their own, and the agent lists them as separate findings.

A reviewer checks that the choice fits the deployment. If the pull request switches to psycopg2-binary and the service runs in production, read the note on that trade-off and decide whether you want the compiled-from-source variant instead. If it adds build dependencies, look at the final image stage and confirm the compiler did not stay in it.

Run it on every failed Docker build

The build pipeline already knows when this happens. Add a step that runs when the image build fails, checks the log for pg_config executable not found, and sends the request above with your API key and the branch that failed as the resources URL. Give it a request_id that is a UUID v5 of the commit SHA, and a retried pipeline returns the run that already exists.

Organisations with many small services often have the same Dockerfile pattern in all of them. One script can send the request for each repository in turn. Each call answers 202 at once with the session, a key runs its max_concurrent limit at a time, and a webhook_url on the key collects a run.completed event per repository so you can gather the pull request links in one place.

The use case this job follows, with its automation and trigger: Fix a bug from an alert.

Why run it on Rush?

This error is about what is missing from a machine, so it cannot be studied on one that has everything. A developer's laptop usually has PostgreSQL installed, pg_config on the path, and a compiler, and pip install psycopg2 simply works there. The agent's sandbox starts without those, like the container that failed, and the agent builds the real image to check the fix instead of trusting a local virtualenv.

The run keeps the build logs. Docker output scrolls past and is gone on a workstation; in a session, every command and its output is stored in the transcript, including the failed build and the successful one. When the pull request claims the image builds and the driver imports, the lines that show it are attached to that claim.

Frequently asked questions

Should I use psycopg2 or psycopg2-binary?

They provide the same module. psycopg2-binary is a precompiled wheel with its own copy of libpq, which makes it the easy choice for development, tests and CI. psycopg2 compiles against the libpq on your system, and the psycopg project recommends that for production. The agent chooses by how the image is used and states the reason.

What do I need to install to build psycopg2 from source?

On Debian and Ubuntu images, libpq-dev for pg_config and the headers, plus a C compiler such as gcc. On Alpine the equivalents are postgresql-dev, gcc and musl-dev. The final image needs only the runtime library, libpq5 on Debian, if you build in a separate stage.

I already use psycopg2-binary and still get this error. Why?

Your pinned release has no wheel for your Python version, so pip falls back to compiling it. On Python 3.11, psycopg2-binary==2.9.3 fails with the same pg_config message while 2.9.9 installs. Raising the pin to a release published after your Python version came out fixes it.

Do I have to change my code or connection string?

No. Both packages install a module named psycopg2, so imports stay the same, and an SQLAlchemy URL beginning postgresql:// continues to select it. Only the dependency name or the image's system packages change.

Why does it install fine on my Mac and fail in Docker?

A machine with PostgreSQL installed through a package manager has pg_config and a compiler, so the source build succeeds without you noticing it ran. A slim base image has neither. The container is showing you what the package really requires.

More psycopg2 jobs

More jobs like this: Fix a bug from an alert

Keep reading