All jobs

Migrate From github.com/golang/protobuf to google.golang.org/protobuf

Yes. An agent moves hand-written Go code off the deprecated golang/protobuf helpers, tests the nil and overflow cases that change, and opens a pull request.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

github.com/golang/protobuf is deprecated in favour of google.golang.org/protobuf, and a linter such as staticcheck reports every use as SA1019. The agent first separates two kinds of use, because they are fixed in different ways: code a person wrote, and code protoc-gen-go generated. In a library like collectd/go-collectd, go.mod requires github.com/golang/protobuf v1.5.3 directly and lists google.golang.org/protobuf as indirect. The hand-written use is four calls in rpc/marshalling.go: ptypes.TimestampProto, ptypes.DurationProto, ptypes.Timestamp and ptypes.Duration. The generated use is in rpc/proto/collectd.pb.go and rpc/proto/types/types.pb.go, which import the old proto package and the old ptypes/duration and ptypes/timestamp packages.

The rpc package has no test files, so before changing anything the agent writes two: a round trip of an api.ValueList through MarshalValueList and UnmarshalValueList, and a case that sets the time or the interval to nil and expects an error. Both pass on the old code. Then it rewrites the four calls. ptypes.TimestampProto(t) becomes timestamppb.New(t) followed by CheckValid(), ptypes.DurationProto(d) becomes durationpb.New(d), and the two readers become CheckValid() followed by AsTime() or AsDuration(). The message types need no change: since golang/protobuf 1.4 the old timestamp.Timestamp is an alias for timestamppb.Timestamp, so the fields in the old generated structs accept the new values.

The CheckValid calls are the part that is easy to get wrong, and the agent shows why with a small program that runs both APIs on the same inputs. The old ptypes.Timestamp(nil) returns the error timestamp: nil Timestamp. The new AsTime() on a nil message returns 1970-01-01 with no error, and AsDuration() returns zero. A rewrite that drops the check turns a rejected message into a metric dated at the Unix epoch. With the check, the nil test still passes and the error text becomes proto: invalid nil Timestamp. One difference remains even with the check: a Duration too large for time.Duration was an out-of-range error under ptypes.Duration, while CheckValid accepts it and AsDuration returns the largest representable duration. The agent states that in the pull request and, where it matters, adds an explicit bound.

Last, it runs go mod tidy and go mod why -m github.com/golang/protobuf. The old module does not leave go.mod: the generated package collectd.org/rpc/proto still imports it. Removing it means regenerating the .pb.go files with a current protoc-gen-go, and here the agent stops and reports. This repository contains no .proto files; the header of types.pb.go names types.proto as its source, and that file lives in the collectd daemon's own repository. Current generators also put the gRPC service in a separate file with a different server interface, so rpc/server.go would change too. The agent proposes that as a second pull request and keeps this one to the hand-written code.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "claude",
  "input": "Migrate this Go module's hand-written code from github.com/golang/protobuf to google.golang.org/protobuf. Read go.mod and list every import of github.com/golang/protobuf by package (proto, jsonpb, ptypes and its subpackages, descriptor), split into generated files (a 'Code generated' header) and hand-written files. Do not edit generated files by hand. For hand-written code, map each call to its replacement: ptypes.TimestampProto and ptypes.Timestamp to timestamppb.New, CheckValid and AsTime; ptypes.DurationProto and ptypes.Duration to durationpb.New, CheckValid and AsDuration; ptypes.MarshalAny and UnmarshalAny to anypb; jsonpb to protojson; proto.Marshal and friends to the new proto package. Before editing, make sure each touched function has a test, adding round-trip tests and nil-input tests where there are none, and run them on the old code. Write a small program that runs the old and new helpers on the same inputs, including a nil message, a time before year 1 and a duration that overflows time.Duration, and include its output in the pull request. Keep every error return the old code had by calling CheckValid, and say where behaviour still differs. If jsonpb is used, compare the JSON bytes old and new for a sample message and report differences. Run gofmt on the files you touch, go build, go vet and go test for every package before and after, noting packages that could not build before your change. Run go mod tidy, then go mod why -m github.com/golang/protobuf, and report what still requires it. If only generated code does, say what regenerating needs (the .proto sources, protoc, protoc-gen-go and protoc-gen-go-grpc versions) and whether the sources are in this repository; do not regenerate in this change. Open a pull request.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

A pull request that changes rpc/marshalling.go, adds rpc/marshalling_test.go, and moves google.golang.org/protobuf from the indirect block of go.mod to the direct one. The description opens with the comparison program's output: for each input, what the old helper returned and what the new methods return. It then lists what was not done and why, which for go-collectd is the two generated files, with the go mod why output that shows they are the only remaining reason the old module is required.

The reviewer checks each AsTime and AsDuration call for a CheckValid before it, since a missing one compiles and passes a round-trip test. They read the note on oversized durations and decide whether a bound is wanted. They also look at the test section: in go-collectd the plugin package does not build without cgo and the collectd headers, before or after this change, and the description says so instead of counting it as a failure of the migration.

Run it on every Monday morning

A large module is easier to move one package a week than in one pull request nobody can review. Create an automation in the Project that holds the repository with a schedule trigger, a five-field cron such as 0 7 * * 1 and a timezone. Save this prompt with one sentence added at the top: pick the first package, in alphabetical order, whose hand-written files still import github.com/golang/protobuf, migrate only that package, and if none is left, answer that the migration is finished and change nothing.

Each Monday then produces one small pull request, or a one-line answer once the work is done. The automation runs as the member who last changed it, and its max_runs_per_hour setting caps it along with anyone pressing Run now. If last week's pull request is still open, the new run sees the old imports on the default branch and would pick the same package, so add a line telling it to list open pull requests first and skip a package that already has one.

The use case this job follows, with its automation and trigger: Upgrade dependencies weekly.

Why run it on Rush?

The dangerous part of this migration compiles cleanly. AsTime on a nil message returns a valid-looking time, so neither the compiler nor a happy-path test objects to a rewrite that dropped an error check. The run executes old and new helpers side by side in its own sandbox and keeps the output, so the pull request can show the exact inputs where they differ instead of summarising the documentation.

It also settles the question of what can be removed. go mod tidy and go mod why run against the real module graph, and the answer for a given repository, whether the old module goes or stays and which package holds it there, is in the transcript with the commands that produced it. That is a better basis for planning the regeneration step than a guess.

Frequently asked questions

Can I just replace the import path?

For the message types, often yes, since the old well-known type packages are aliases of the new ones. For functions, no. ptypes.Timestamp returned a time and an error, while AsTime returns only a time, so the error handling has to be rebuilt with CheckValid. jsonpb and protojson also differ in options and in output spacing.

Why is github.com/golang/protobuf still in go.mod afterwards?

Usually because generated .pb.go files import it. Files produced by an old protoc-gen-go reference github.com/golang/protobuf/proto, and only regenerating them removes that import. go mod why -m github.com/golang/protobuf names the package responsible. A dependency that has not migrated can keep it as an indirect requirement too.

Do old and new generated messages work together?

Yes. From version 1.4 on, github.com/golang/protobuf is implemented on top of google.golang.org/protobuf, and a message from old generated code can be passed to the new API through the old module's proto.MessageV2 function. That is why the migration can be done in steps and does not need one large change.

Does the wire format change?

No. Both modules read and write the same protobuf binary encoding, so services on either side of the change keep talking to each other. What can change is text and JSON output, which the new packages deliberately do not keep byte-stable, so tests that compare such strings may need updating.

Will the agent regenerate my .pb.go files?

Not in this change. Regenerating depends on having the .proto sources and agreeing on generator versions, and newer gRPC generators change the service interfaces your code implements. The agent reports what is needed and whether the sources are in the repository, so that step can be its own reviewed pull request.

More jobs like this: Upgrade dependencies weekly

Keep reading