All jobs

Migrate golang.org/x/exp/slices and maps to the Go Standard Library

Yes. An agent moves the imports to slices and maps, fixes the calls whose signatures changed, runs your tests and reports the one behaviour they do not cover.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

The import paths differ by a prefix, and the packages behind them do not match, so the agent starts by letting the compiler list the differences. philhassey/goatlang, an interpreter for a subset of Go, pins golang.org/x/exp v0.0.0-20230224173230-c95f2b4c22f2 in a go.mod that says go 1.20, and imports slices or maps from it in six files. Its suite passes as it stands. Swapping both paths for the standard library's and running go build gives nine errors in three files, and they fall into two groups, with a tenth of the first kind in load_test.go. The agent also runs the step a scheduled upgrade would take, go get golang.org/x/exp@latest, and shows it ends at the same first error while raising the go directive along the way. The old pin was holding back both routes.

The first group is the sort comparator. At lines 324, 334 and 344 of builtins.go the code passes func(a, b Value) bool, a less function, to SortFunc and SortStableFunc. The standard library wants func(a, b Value) int. Wrapping less as a three-way compare means calling it twice whenever the first call says no, and here the less function can be a function written in the interpreted script, called through g.Func. On a 50-element input, a counter shows 291 calls with the old package, 396 through such an adapter and 286 with sort.Slice. So the agent uses sort.Slice and sort.SliceStable, which still take a less function, for the three interpreter builtins, and writes a real comparator with strings.Compare only for the plain name sort in the test file.

The second group is maps.Keys. In x/exp it returns a slice; in the standard library it returns an iterator. load.go sorts the keys, indexes into them and deletes from them, and value.go assigns them to a []string field at lines 853 and 933, hence errors such as S (type iter.Seq[string]) does not satisfy ~[]E. The agent writes slices.Sorted(maps.Keys(packages)) where a sort followed, and slices.Collect(maps.Keys(m.data)) where the code needs a slice. Both helpers need a newer language version, so it sets go 1.23 in go.mod and says so first in the pull request, since that is a decision for every consumer of the module. go mod tidy then removes the golang.org/x/exp requirement entirely.

After that the build is clean and go test ./... reports the same 1,401 passing results as before, with none failing. The agent does not stop at green, because one function changed behaviour without changing its signature. The standard library's slices.Delete zeroes the elements it removes from the tail of the backing array, and the pinned x/exp version did not. Goatlang exposes Delete to scripts, and a three-statement script that keeps a second reference to the original slice prints [1 4 3 4] before the migration and [1 4 nil nil] after. No existing test notices. The agent adds one that pins the behaviour it measured and puts the question to the maintainers. It leaves alone the names scripts import, strings such as "golang.org/x/exp/slices.SortFunc" in builtins.go, because those are the interpreter's own interface and not Go imports.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "claude",
  "input": "Migrate this Go module from golang.org/x/exp/slices and golang.org/x/exp/maps to the standard library slices and maps packages. Run the test suite first and record the number of passing and failing results. Then replace the import paths everywhere, including test files, run go build and go vet, and list every error grouped by cause before fixing any. Separately, on a scratch copy, run go get golang.org/x/exp@latest without code changes and report what it does to the build and to go.mod. For sort calls that pass a less function: if the comparator is cheap, rewrite it as a three-way compare; if it calls user code, a callback or anything with side effects, do not wrap less in an adapter that calls it twice, and use sort.Slice or sort.SliceStable instead; measure comparator calls on a fixed input to justify the choice. For maps.Keys and maps.Values, use slices.Collect or slices.Sorted where a slice is needed. Set the go directive to the lowest version that provides everything you used, state it at the top of the pull request, and run go mod tidy. Then check functions whose signature is the same but whose behaviour differs between the pinned x/exp version and the standard library, starting with slices.Delete, Compact, Insert and Clip: write a small program that compares them, and if this module's public behaviour changes as a result, add a test that shows it and report it. Do not rename identifiers that are part of this project's own public or scripting interface even if they contain the old import path. Finish with go vet and the full test suite, and open a pull request.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

A pull request touching six Go files and go.mod, with go.sum shrinking as the experimental module drops out. Its first line states the new go directive. Then come the test counts before and after, the compile errors from the plain import swap grouped as comparator and iterator, the comparator call counts behind the choice of sort.Slice, and the before and after output of the script that shows Delete now clearing the tail. The new test for that behaviour is in the diff.

A reviewer has three decisions the agent prepared and did not make. Whether requiring Go 1.23 is acceptable for a library others import. Whether scripts depending on the old contents of a deleted slice's tail matter, in which case the builtin can copy before deleting. And whether the interpreter should also accept "slices" as an import name in scripts. Everything else is mechanical and checked by the suite, which the pull request shows passing on the toolchain it names. If the suite fails after the change, the session ends for review with the failing tests listed.

Run it on every weekly upgrade run

Pinned pseudo-versions of golang.org/x/exp are a common reason a weekly go get -u pull request fails to build, since the experimental packages changed signatures after the standard library adopted them. Put this prompt in an automation with a schedule trigger, such as the cron 0 9 * * 1, on the Project that has the repository. The first run does the migration; later runs find nothing to migrate and say so in one line.

If you already have an upgrade job and only want help when it breaks, use a github trigger on workflow_run.failure, and word the prompt so the run stops with a one-line note when the failed workflow is not the upgrade job. The failure event is placed after the saved prompt, and the agent starts from what it says failed. Runs started by an automation count against its max runs per hour, which you set when you create it.

The use case this job follows, with its automation and trigger: Upgrade dependencies weekly.

Why run it on Rush?

The migration is easy to do badly in a way that still compiles. A less-to-compare adapter builds, passes the tests and quietly calls user code more often. A deleted slice's tail changes and nothing fails. Finding those takes small throwaway programs and a scratch copy of the module, which is the kind of side work a Rush run does freely in its sandbox: the agent installs the Go toolchain the module asks for, and the output of every build, count and test is kept in the session.

It also separates the mechanical from the judgement calls. The reviewer does not have to rediscover that the standard library's maps.Keys returns an iterator or what go mod tidy removed. They get the grouped errors, the measurements and three named decisions, and can check each number against the transcript of the commands that produced it.

Frequently asked questions

Is slices in the standard library a drop-in for golang.org/x/exp/slices?

For most calls, such as Contains, Index and EqualFunc, yes. The sort functions that take a function are the exception: they want a comparison returning a negative number, zero or a positive number, where older x/exp versions took a less function returning a bool. Code pinned to an old x/exp will not compile until those are rewritten.

Why does maps.Keys no longer return a slice?

The standard library's maps.Keys and maps.Values return iterators. Use slices.Collect(maps.Keys(m)) to get a slice, or slices.Sorted(maps.Keys(m)) for a sorted one. Ranging directly over the iterator also works when you do not need the slice.

Which Go version does the migration need?

The slices and maps packages arrived in Go 1.21, and the iterator helpers used here, slices.Collect and slices.Sorted, need 1.23. The agent sets the go directive to the lowest version covering what it used and puts that at the top of the pull request.

Can I just upgrade golang.org/x/exp instead?

It does not avoid the work. In the fixture, upgrading to the latest x/exp stops at the same comparator error as switching to the standard library, and it also raises the go directive. Since the standard packages are stable and the experimental ones are not, the migration removes a dependency for the same effort.

Does the agent change behaviour my tests do not cover?

It looks for exactly that. For functions that kept their signature, it compares the old and new implementations in a small program and, when the difference is visible through your module, adds a test and reports it instead of deciding silently.

More jobs like this: Upgrade dependencies weekly

Keep reading