# Upgrade Python Selenium Tests to Selenium 4 (find_element_by_* and PhantomJS Removed)

Yes. An agent ports find_element_by_* calls and the PhantomJS driver to Selenium 4, runs the tests in a real headless browser, and opens a pull request.

[Run this on Rush](https://cloud.getrush.ai/run/upgrade-selenium/selenium/). Opens the console with this prompt ready to send, in a Project that has your repository.

## What the agent does

Two removals break old Selenium suites. `find_element_by_id` and its siblings are gone from Selenium 4, so a test fails with `AttributeError: 'WebDriver' object has no attribute 'find_element_by_id'`, and `webdriver.PhantomJS` is gone too. The agent begins with what a fresh install gives the project. In a Django package like batiste/django-page-cms, `setup.py` lists `selenium>=3.0.1` under `tests_require` with no upper bound, and the workflow in `.github/workflows/django.yml` runs `pip install .[tests]` and then `python pages/test_runner.py`. That install resolves to a current Selenium 4 release, in which `selenium.webdriver` has no `PhantomJS` attribute at all.

That repository shows how such suites usually end up. Every line of `pages/tests/test_selenium.py` is commented out: a `SeleniumTestCase` with three browser tests of the admin (`test_admin_select`, `test_admin_move_page`, `test_admin_export_json`) that start `webdriver.PhantomJS()`. The remaining suite passes, so nothing looks wrong, and the admin's browser behaviour has had no test since. The agent restores the file and ports it. The driver becomes `webdriver.Chrome` with a headless option. Direct calls such as `select.find_elements_by_tag_name('option')` become `select.find_elements(By.TAG_NAME, 'option')`.

A search for `.find_element_by_` would miss half of the work in this file. Its wait helper takes the method name as a string, `self.timeout('find_element_by_css_selector', selector)`, and calls `getattr(b, command)(param)` inside a `WebDriverWait`. Those strings match no method-call pattern and fail only at run time. The agent changes the helper to take a locator strategy and a value, and passes `By.CSS_SELECTOR` or `By.ID` from each caller. It confirms each removed name against the installed package (`hasattr` on the `WebDriver` and `WebElement` classes) so the list of what changed comes from the library and not from a migration guide.

Then it runs the three tests in a real browser, and reads the result past the summary line. Ported this way and executed on Python 3.12 with Chromium in headless mode, the browser starts and the suite ends `OK (skipped=3)`. That is not a pass. The helper converts every `TimeoutException` into `SkipTest`, and the log above the summary shows why the waits timed out: the live server answered `/admin/login/` with a 500 and `database table is locked: django_site`. The test class inherits from the project's `TestCase`, which is Django's transaction-wrapped one, and from `LiveServerTestCase`, whose server runs in a second thread against the same SQLite test database. The agent reports the port as done and the tests as not yet passing, and proposes the follow-up: base the class on the live-server test case alone and make a timeout fail instead of skip.

## Start it with one API call

`POST https://api.phoenix.so/o/{org}/sessions`

```json
{
  "agent": "claude",
  "input": "Upgrade this project's Selenium tests to Selenium 4. Read how selenium is declared (requirements files, setup.py extras, pyproject.toml) and which version a fresh install resolves to, and read how CI installs and runs the tests. Find every use of the removed APIs: find_element_by_* and find_elements_by_* called directly, the same names passed as strings to getattr or to helper functions, webdriver.PhantomJS, desired_capabilities, executable_path, and the *_options keyword renames. Include tests that are commented out or skipped, and say when and why they were disabled if git history shows it. Confirm each removed name against the installed package and list them. Port the code: find_element(By.X, value) for locators, a headless Chrome or Firefox driver in place of PhantomJS, Service and Options objects in place of removed keyword arguments. Change helper signatures where a method name was passed as a string. Do not change what the tests assert. Install a browser if the sandbox has none, run the Selenium tests, and report each test as passed, failed or skipped with the reason from the log; a skipped test is not a pass. If a test helper turns timeouts or errors into skips, say so and show what was hidden. If tests fail for a reason unrelated to the Selenium API, such as the live server or the database, diagnose it from the log, propose the fix, and apply it only if it is confined to the test files. Run the remaining test suite before and after to show it is unaffected. Set a lower bound of selenium 4 in the dependency declaration. Open a pull request.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
```

1. **Get an API key.** In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
2. **Send the request.** POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
3. **Read what came back.** Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

## What comes back

A pull request that restores and ports the Selenium test file, raises the declared minimum to `selenium>=4`, and states plainly where the tests stand. For django-page-cms the description has a three-row table, one per browser test, each marked skipped with the line from the log that explains it, next to the count of the other tests in the suite before and after the change. It quotes the helper that raises `SkipTest` on timeout, because that helper is the reason a broken admin page would have shown up as a skip and not a failure even when PhantomJS still worked.

The reviewer decides what to merge. The API port is safe to take alone: it changes names and the driver, and the remaining tests are untouched. Whether to take the proposed change of base class in the same pull request is a judgement about the test setup, and the agent keeps it as a separate commit so it can be dropped. Before merging they check that the workflow's runner image has a browser, since the tests now need Chrome or Firefox where they once needed a PhantomJS binary.

## Run it on every test module still on Selenium 3 calls

A large suite is better ported one module at a time, and the expensive setup, a virtual environment, a browser and its driver, should be paid once. Start the run with this body narrowed to the first module. When that turn ends the sandbox parks with its disk and memory and costs nothing while parked. Send the next module as a follow-up, `POST /o/{org}/sessions/{id}/messages` with `{"input": "Now port the checkout tests the same way."}`, and the same sandbox wakes with everything still installed.

The follow-up answers `202` with the session and the turn it opened. Wait on that turn and read its result when it ends `completed`; the session's own status can still show the previous answer while the message waits in line. A parked sandbox is kept for 7 idle days, so the port can follow the pace of review.

## Why run it on Rush?

Browser tests are the kind people port by search and replace and never run, because running them needs a browser, a driver that matches it, and often a display. The run has its own sandbox with a computer and a browser, so the ported tests execute against a live server and a real page, and the full log is kept in the session transcript. That log is where this job's real finding sits: a summary line that says OK above three skipped tests and a database lock.

It also keeps the two questions apart. Whether the code now uses the Selenium 4 API is one question, answered by the diff and by checking the removed names against the installed package. Whether the tests pass is another, answered only by running them. A report that says the first is done and the second is not, with evidence for both, is more useful than a green tick.

## Frequently asked questions

### What replaces find_element_by_id and find_element_by_css_selector?

`driver.find_element(By.ID, 'name')` and `driver.find_element(By.CSS_SELECTOR, 'div.x')`, with `from selenium.webdriver.common.by import By`. The plural forms become `find_elements(By..., ...)`. The same methods exist on elements, so `row.find_elements(By.TAG_NAME, 'td')` replaces `row.find_elements_by_tag_name('td')`.

### What do I use instead of PhantomJS?

A headless mode of a real browser. Create `webdriver.ChromeOptions()`, add the `--headless=new` argument and pass it as `options=` to `webdriver.Chrome`, or do the equivalent with Firefox. PhantomJS development stopped years ago and Selenium 4 has no driver class for it.

### Do I still need to download chromedriver myself?

Usually not. Recent Selenium 4 releases include Selenium Manager, which finds or downloads a driver matching the installed browser when you create the driver object. On a machine with no outbound network, install the driver with your system packages and pass its path through a `Service` object.

### Can I pin selenium below 4 instead?

Only as a short stopgap. Selenium 3 predates current browser releases and current Python versions, and a pin does not bring PhantomJS back to life. The locator change is mechanical, and the driver change is needed in any case.

### Why did my suite pass while the Selenium tests were broken?

Check for two things: test files that were commented out or excluded, and helpers that catch a timeout and raise a skip. Both turn a failing browser test into silence. The agent lists skipped tests with their reasons for that purpose, and treats a skip as a result that needs explaining.

## More Selenium jobs

- [Fix "TypeError: WebDriver.__init__() got an unexpected keyword argument 'executable_path'"](https://getrush.ai/run/fix-webdriver-executable-path-unexpected-keyword-argument/selenium/)

## More jobs like this: Upgrade dependencies weekly

- [Upgrade SQLx From 0.5 to the Current Release in a Rust API](https://getrush.ai/run/upgrade-sqlx/sqlx/)
- [Upgrade Storybook to the Latest Version](https://getrush.ai/run/upgrade-storybook/storybook/)
- [Upgrade Strapi to the Latest Version](https://getrush.ai/run/upgrade-strapi/strapi/)
- [Upgrade Symfony and Its Dependencies](https://getrush.ai/run/upgrade-symfony/symfony/)
- [Upgrade Terraform Providers in a Terraform Module](https://getrush.ai/run/upgrade-terraform-providers/terraform/)

## Keep reading

- [How a Rush run works on your repository, from request to pull request](https://getrush.ai/cloud/)
- [Compare cloud agents, agent sandboxes and AI code review](https://getrush.ai/compare/)
- [Rush pricing: Free, Pro and Max](https://getrush.ai/pricing/)
