# Migrate From Requests to HTTPX in a Python Project

Yes. An agent ports Python code from Requests to HTTPX, restores redirect following and error labels, and proves it against a local test server.

[Run this on Rush](https://cloud.getrush.ai/run/migrate-requests-to-httpx/requests/). Opens the console with this prompt ready to send, in a Project that has your repository.

## What the agent does

The agent starts with every place the code touches Requests, including the places that only name its exceptions. In a project like public-apis/public-apis the dependency lives in `scripts/requirements.txt` and one file uses it, `scripts/validate/links.py`, the checker that visits every link in the README each night. Its core is `check_if_link_is_working`: one `requests.get(link, timeout=25, headers=...)` with a random user agent and an explicit `host` header, followed by five `except` clauses that turn exceptions into labels. `SSLError` becomes `ERR:SSL`, `ConnectionError` becomes `ERR:CNT`, a connect timeout becomes `ERR:TMO`, `TooManyRedirects` becomes `ERR:TMR`, and anything else becomes `ERR:UKN`. The file also imports `Response` from `requests.models` as a type hint for the Cloudflare check, which reads `status_code`, `headers` and `text`.

The existing tests cannot judge this change. `python -m unittest discover tests/` runs 31 tests, and none of them makes a request: the link tests use a `FakeResponse` class of their own. So the agent builds the missing check. It starts a small HTTP server on localhost with one path per behaviour: a 200, a 404, a 301 that leads to a 404, a 302 that leads to a 200, a redirect to itself, a 403 with a Cloudflare-looking body, and a path that never answers. It adds a closed port, an unresolvable host name and an `https` URL aimed at the plain HTTP port, then records what the unchanged function returns for all ten.

The first surprise is in the baseline. The path that never answers comes back after 25 seconds as `ERR:UKN`, with Requests' read-timeout text in the message. The `ERR:TMO` branch catches only `ConnectTimeout`, and since that class is also a `ConnectionError`, the clause above it gets there first. The agent records this as existing behaviour; it is about to change.

Swapping the call for `httpx.get` and stopping there produces a checker that passes its tests and is wrong in three ways the local server exposes. HTTPX does not follow redirects unless asked, so the 301 that leads to a 404 is reported as a working link and the redirect loop as well. The `except` clauses still name `requests.exceptions`, so the first unreachable link raises `NameError: name 'requests' is not defined`. And HTTPX has no separate TLS exception: a failed handshake arrives as `httpx.ConnectError` with the OpenSSL text inside. The finished port passes `follow_redirects=True`, maps `httpx.TimeoutException` to `ERR:TMO`, `httpx.TooManyRedirects` to `ERR:TMR` and `httpx.ConnectError` to `ERR:CNT`, and keeps `ERR:SSL` by checking whether the underlying cause is an `ssl.SSLError`. The requirements file changes too. The pins on `urllib3` and `charset-normalizer` exist for Requests and go away; HTTPX brings `httpcore`, `h11` and `anyio` and still uses `certifi` and `idna`.

## Start it with one API call

`POST https://api.phoenix.so/o/{org}/sessions`

```json
{
  "agent": "claude",
  "input": "Port this Python project from requests to httpx. Read the requirements files, every module that imports requests or anything under requests.*, the tests, and the workflows that run them. List each call with its arguments (timeout, headers, allow_redirects, verify, proxies, auth, sessions) and each except clause that names a requests exception, in order. Run the existing tests and say whether any of them makes a real HTTP request. Then start a local HTTP server and record what the unchanged code returns for: 200, 404, a redirect that ends in 404, a redirect that ends in 200, a redirect loop, a response that looks like a bot challenge, a server that never answers, a closed port, an unresolvable host, and a TLS failure. Port the code: httpx with follow_redirects=True wherever requests followed them by default, the same timeout, the same headers, and an exception mapping that keeps every existing error label. Where httpx has no matching exception class, inspect the cause. Do not switch to async or add concurrency. Update type hints and the requirements file, dropping pins that existed only for requests. Re-run the tests and the ten local cases, and show old and new results in the pull request. List every case whose label or message text changed and why.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
```

1. **Get an API key.** In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
2. **Send the request.** POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
3. **Read what came back.** Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

## What comes back

A pull request that edits `scripts/validate/links.py` and `scripts/requirements.txt` and adds the local-server check as a test, so the next change to this function has something to fail. The description holds a table of ten rows and three columns: the unchanged function, the naive swap, and the finished port. The middle column is there on purpose. It shows the two redirect rows going quiet, which is the argument for the `follow_redirects` line that a reader of the diff might otherwise question.

Two rows change between the first and last column, and the reviewer has to approve both. The server that never answers moves from `ERR:UKN` to `ERR:TMO`, because one HTTPX class covers connect and read timeouts; that is arguably the label the authors intended. And every message gets shorter. Requests reports a refused connection as `HTTPConnectionPool(host=..., port=...): Max retries exceeded with url: ...`, where HTTPX says `[Errno 111] Connection refused`. The redirect limit reads `Exceeded maximum allowed redirects.` in place of `Exceeded 30 redirects.` Anything that parses the nightly job's output needs those strings.

The pull request does not make the checker faster. Links are still visited one at a time with a 25 second timeout, as before. HTTPX makes a concurrent version possible, and the description says so in one sentence and leaves it for a change that can be reviewed on its own.

## Run it on every Linear issue labelled httpx

Teams usually adopt HTTPX one script or service at a time, as each needs HTTP/2 or async. Connect Linear under Settings → Connections, add an automation to the Project with a `linear` trigger on `linear.issue.labeled` and the label you choose, and save the prompt above. The run starts in the Project's repository when someone in your Rush org applies the label, and the issue follows the prompt as event text. Use the issue for specifics the prompt cannot know, such as which error labels downstream tooling depends on.

A Linear trigger needs a Linear connection for the member the automation runs as. If that member leaves the org, the fire is skipped with the reason `not_member` instead of running as someone else.

## Why run it on Rush?

The two libraries look alike on purpose, and that is the hazard. A port that reads correctly can turn a link checker into one that approves dead links, and the project's own tests will pass because they never open a socket. Telling the difference takes a server with awkward paths, a slow one included, and patience for a 25 second timeout per timeout case. The agent builds that server in its sandbox, runs the cases, and leaves the request log and results in the run's transcript.

Because the sandbox is its own machine, the checks cannot disturb anything: no local port clashes, no proxy settings from a developer's shell changing how either library connects. The same ten cases give the same answers for whoever reruns them, which is what makes the table in the pull request evidence.

## Frequently asked questions

### Why does HTTPX return 301 where Requests returned the final page?

Requests follows redirects by default and HTTPX does not. Pass `follow_redirects=True` on the call or on the client. HTTPX also stops after 20 redirects where Requests allows 30, so a very long chain can fail that used to pass.

### Which HTTPX exceptions replace the Requests ones?

`httpx.TimeoutException` covers connect, read, write and pool timeouts. `httpx.ConnectError` covers refused connections, DNS failures and failed TLS handshakes. `httpx.TooManyRedirects` keeps its name, and `httpx.HTTPError` is the base class to catch last. There is no `SSLError`, so code that reported TLS problems separately has to look at the exception's cause.

### Do my mocked tests keep working?

Tests that pass hand-made response objects into your own functions, as this project's do, keep working as long as the fake has the attributes you read. Tests that patch `requests.get` or use a Requests mocking library must be rewritten, and the agent lists each one. HTTPX accepts a custom transport, which lets a test answer requests without a network.

### Is the timeout behaviour the same?

A single number means roughly the same thing in both: a limit on each phase, not on the whole request. The default differs. Requests waits forever when no timeout is given and HTTPX stops after 5 seconds, so calls that passed no timeout can start failing on slow endpoints. The agent lists every call without one.

## More jobs like this: Turn a Linear label into a PR

- [Replace shortid With nanoid in a React App](https://getrush.ai/run/migrate-shortid-to-nanoid/shortid/)
- [Migrate a Blazor App from SQL Server to PostgreSQL](https://getrush.ai/run/migrate-sql-server-to-postgres/blazor/)
- [Migrate a Streamlit App to the Anthropic Messages API from a Linear Issue](https://getrush.ai/run/migrate-to-anthropic-messages-api/streamlit/)
- [Migrate a Go CLI From urfave/cli v1 to v2](https://getrush.ai/run/migrate-urfave-cli-v1-to-v2/urfave-cli/)
- [Migrate a Vue 3 App From Vuex to Pinia](https://getrush.ai/run/migrate-vuex-to-pinia/vuex/)
- [Migrate From Winston to Pino in a Node.js Service](https://getrush.ai/run/migrate-winston-to-pino/winston/)

## Keep reading

- [How a Rush run works on your repository, from request to pull request](https://getrush.ai/cloud/)
- [Compare cloud agents, agent sandboxes and AI code review](https://getrush.ai/compare/)
- [Rush pricing: Free, Pro and Max](https://getrush.ai/pricing/)
