All jobs

Migrate a Python Project From oauth2client to google-auth

Yes. An agent replaces oauth2client's service account class with google-auth, rewrites token and signing calls, and tests them offline with a throwaway key.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

oauth2client is deprecated and its last release, 4.1.3, is from September 2018. The supported library is google-auth, and the two are not drop-in compatible, so the agent maps each use before it edits. In a wrapper like thisbejim/Pyrebase everything is in pyrebase/pyrebase.py. The import is from oauth2client.service_account import ServiceAccountCredentials. Firebase.__init__ builds the object from config["serviceAccount"] with from_json_keyfile_name when it is a path and from_json_keyfile_dict when it is a dict, passing three scopes. The object is then used in four ways: Database.build_headers calls self.credentials.get_access_token().access_token, Auth.create_custom_token reads service_account_email and the private attribute _private_key_pkcs8_pem, and Storage.__init__ hands the object to storage.Client(credentials=credentials, project=storage_bucket) from the gcloud package.

The constructors translate directly: service_account.Credentials.from_service_account_file(path, scopes=scopes) and from_service_account_info(info, scopes=scopes) from google.oauth2. service_account_email keeps its name. The other three uses do not survive. A google-auth credential has no get_access_token method; it starts with token set to None, and the caller refreshes it with credentials.refresh(google.auth.transport.requests.Request()) when credentials.valid is false, then reads credentials.token. The agent puts that in one small helper so build_headers does not refresh on every request.

The private key attribute is gone as well, and that is an improvement the agent uses. The old code pulls the PEM out of the credential, loads it with RSA.importKey and signs a JWT with python_jwt. google-auth exposes credentials.signer, and google.auth.jwt.encode(credentials.signer, payload) produces an RS256 token with the key id in its header. After the rewrite nothing in the file needs python_jwt or Crypto.PublicKey, so both imports and both requirements come out. That also settles a disagreement in the repository, where requirements.txt pins pycrypto==2.6.1 and setup.py asks for pycryptodome==3.4.3.

The fourth use is a blocker the agent has to resolve in the same change. gcloud==0.17.0 is the long-renamed predecessor of google-cloud-storage and expects an oauth2client object: given a google-auth credential, storage.Client raises AttributeError: 'Credentials' object has no attribute 'authorize'. So the pull request replaces from gcloud import storage with from google.cloud import storage, which accepts the new credential type. The repository's tests cannot referee any of this without a real project, since tests/tools.py builds every client from a tests/config.py that holds a live database URL and a secret.json. The agent adds offline tests instead: it generates an RSA key, builds a fake service account dict around it, and checks the constructors, the email, a custom token verified against the public key, and the storage client's construction.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "claude",
  "input": "Migrate this Python project from oauth2client to google-auth. First list every import from oauth2client and every attribute or method used on the objects it returns, including private ones (names starting with an underscore) and any place the credential object is passed to another library. Map each to google-auth and say so in a table: from_json_keyfile_name and from_json_keyfile_dict become google.oauth2.service_account.Credentials.from_service_account_file and from_service_account_info with scopes as a keyword; get_access_token() becomes a refresh with google.auth.transport.requests.Request() when credentials.valid is false, then credentials.token; authorize(http) becomes google.auth.transport.requests.AuthorizedSession or the transport the project already uses; reading the private key to sign something becomes credentials.signer with google.auth.jwt.encode; oauth2client.file.Storage and flow classes become google-auth-oauthlib. For every library that receives the credential object, install it at the pinned version next to google-auth and construct it with a google-auth credential; if it fails, replace it with its maintained successor in the same change and say which one. Do not use any real key or call Google. Write tests that run offline: generate an RSA key in the test, build a service account info dict around it, and check both constructors, the service account email, that a token signed through credentials.signer verifies with the public key and carries the expected claims, and that each client object can be constructed. Keep the project's public function signatures and config keys unchanged. Update requirements files and setup.py or pyproject together, remove oauth2client and any package that is no longer imported after the change, and report where the requirement files disagreed with each other before you started. Run the new tests and state plainly which existing tests you could not run because they need live credentials. Open a pull request with the mapping table and the test output.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

A pull request that edits pyrebase/pyrebase.py, requirements.txt and setup.py and adds one offline test module. In the requirement files oauth2client, gcloud, python-jwt and the pycrypto line leave, and google-auth and google-cloud-storage arrive. The description opens with the mapping table: each old call, the file and line it was on, and what replaced it.

A reviewer checks three things. First, the token helper: that it refreshes only when the credential is not valid, and that a long-lived Database object keeps working after the first token expires. Second, the custom token: the payload still carries the same iss, sub, aud and uid, the lifetime is still sixty minutes, and the header now includes a key id, which Firebase accepts and the old code never sent. Third, the storage client, where the class comes from a different package and upload and download calls should be read against its current method names.

The description is explicit about its limits. The offline tests prove object construction and signing with a generated key. They do not prove that Google accepts the token, and the sixteen existing tests under tests/ are listed as not run, with the reason. Someone with a test Firebase project runs pytest -s tests once before release, as DEV.md describes.

Run it on every repository that pins oauth2client

Internal scripts that talk to Google Sheets, Drive, BigQuery or Firebase were often started from the same snippet, so one organisation can have many of them on oauth2client. Search your requirement files for the name and start one run each: send this body to POST /o/{org}/sessions with the repository in resources, and a request_id that is a UUID v5 of the repository name so a retry of your script starts nothing twice.

Each call returns 202 and the session id. No key or secret goes into the run, because the prompt forbids real credentials and the tests generate their own. When a run completes, the session read shows the pull request; with a webhook_url on the API key, run.completed arrives for every run that key started.

The use case this job follows, with its automation and trigger: Upgrade dependencies weekly.

Why run it on Rush?

The dangerous step in this migration on a laptop is the test: the quickest way to see whether a token works is to point the code at a real service account file, and that file then sits in a working directory, a shell history or a paste. The agent's sandbox never receives one. It proves what can be proved offline, with a key that exists only for the length of a test, and writes down what remains for a person with access.

The compatibility checks need old and new packages installed together: the pinned gcloud next to google-auth to see the authorize error, then its successor to see the client construct. That is a clean environment built for one question and discarded, and the commands and tracebacks stay in the session's transcript. A reviewer sees why the storage package had to change in a pull request whose title is about authentication.

Frequently asked questions

What replaces get_access_token() in google-auth?

There is no single method. Call credentials.refresh(Request()) with a google.auth.transport.requests.Request, then read credentials.token; check credentials.valid first so you only refresh when needed. If you make HTTP calls with requests, an AuthorizedSession does the refresh and adds the header for you.

Can I still read the service account's private key from the credential?

Not through a supported attribute. google-auth gives you credentials.signer, an object that signs bytes, and helpers such as google.auth.jwt.encode that take it. That covers signing a custom token or a URL without the key ever leaving the credential object.

Why did my Google client library stop accepting the credentials?

Older client libraries call credentials.authorize(http), an oauth2client method that google-auth credentials do not have. The fix is to upgrade that library to a release built on google-auth. The agent tests each such library against the new credential type and replaces the ones that fail.

Does this change how users configure the library?

No. The prompt keeps public signatures and config keys, so a path or a dict under serviceAccount still works. Only the object stored on the instance changes type, which matters to anyone who reached into it directly.

More jobs like this: Upgrade dependencies weekly

Keep reading