All jobs

Write Golden Tests for a Flutter App

Yes. An agent adds flutter_test, loads your app's real fonts, writes golden tests for its shared widgets, and opens a pull request with the generated PNGs.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

The agent reads pubspec.yaml first, because a golden test is only as stable as the environment it renders in. In an app like gskinnerTeam/flutter-wonderous-app the SDK constraint is ^3.11.5 with Flutter ^3.41.9, and .github/workflows/build_web.yml installs exactly 3.41.9 through subosito/flutter-action, so the agent installs that version in the sandbox rather than whatever is newest. dev_dependencies has icons_launcher, flutter_lints and dependency_validator and no flutter_test, and there is no test/ directory, so the first edit is adding flutter_test from the SDK. The fonts section declares Cinzel, Yeseva, Tenor, Raleway with its 500, 700 and 800 weights and italic, MaShanZheng and B612Mono from assets/fonts/; without loading them, every golden would render text as the test framework's placeholder boxes.

Next it learns how a widget gets its styling. lib/main.dart registers lazy singletons in GetIt for AppLogic, WondersLogic, TimelineLogic, SettingsLogic, LocaleLogic and the rest, and every widget reaches theme values through $styles, a getter on WondersAppScaffold.style in lib/ui/app_scaffold.dart, and strings through $strings, which reads localeLogic.strings. AppStyle in lib/styles/styles.dart scales text and insets by the screen's shortest side, 1.0 up to 800 logical pixels, 1.1 up to 1000 and 1.2 above, and takes disableAnimations from MediaQuery. So the agent writes a pumpWonders helper under test/ that calls registerSingletons(), runs wondersLogic.init(), loads English strings through localeLogic.loadIfChanged(const Locale('en')), wraps the widget in a MaterialApp with the AppLocalizations delegates from lib/l10n/, a MediaQuery with disableAnimations on and WondersAppScaffold, sets the surface to a fixed phone size, and calls GetIt.I.reset() in tearDown so one test's singletons never leak into the next.

Fonts load once, in test/flutter_test_config.dart, which flutter test runs before every file: a FontLoader per family reading the .ttf files the manifest lists. The goldens target the shared widgets under lib/ui/common/ rather than the animated screens: AppBtn from controls/buttons.dart in its primary, secondary and circular forms, AppHeader from controls/app_header.dart with and without a back button, TimelineEventCard, CompassDivider, WonderousLogo and AppPageIndicator, each with a matchesGoldenFile against a PNG under test/goldens/, and one per WonderType for the wonder title text, since the enum in lib/logic/data/wonder_type.dart lists eight. The agent generates the files once with flutter test --update-goldens, then runs flutter test again without the flag to prove they match, and flutter analyze to satisfy analysis_options.yaml, whose rules include always_use_package_imports and prefer_single_quotes.

Because goldens differ by platform rasterizer, the agent also adds a test job to .github/workflows/, on Ubuntu with the same flutter-version: '3.41.9' the web build uses, running flutter test, so the PNGs are checked where they were produced. It changes nothing under lib/, pushes a branch, and opens a pull request with the test output and the images attached.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "claude",
  "input": "This Flutter app has no tests. Add golden tests for its shared widgets. Read pubspec.yaml, lib/main.dart, lib/ui/app_scaffold.dart, lib/styles/styles.dart and the workflows under .github/workflows first, install the Flutter version the workflows pin, and add flutter_test to dev_dependencies if it is missing. Write test/flutter_test_config.dart that loads every font family pubspec.yaml declares from assets/fonts with FontLoader. Write a pump helper that registers the app's singletons the way main.dart does, loads English strings, wraps the widget in MaterialApp with the app's localization delegates, a MediaQuery with disableAnimations on and the app's scaffold, uses a fixed surface size, and resets GetIt in tearDown. Add golden tests for the buttons, header, timeline card, divider, logo and page indicator under lib/ui/common, one golden PNG each under test/goldens. Generate the goldens once with flutter test --update-goldens, then run flutter test without the flag and flutter analyze. Add a GitHub Actions job that runs flutter test on Ubuntu with the pinned Flutter version. Do not change code under lib/. Open a pull request with the test output and the generated PNGs.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-flutter-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

What comes back is a pull request on its own branch. pubspec.yaml gains flutter_test under dev_dependencies. A new test/ directory holds flutter_test_config.dart, the pump helper, one test file per widget and a goldens/ folder of PNGs rendered at the fixed surface size. A new workflow file runs flutter test on the pinned Flutter version. Nothing under lib/ changes, and the description lists each widget covered and the states rendered for it.

The pull request carries its own proof. The new workflow runs on the branch and either matches every golden or fails with the diff images flutter test writes to test/failures/. The session transcript, at GET /o/{org}/sessions/{id}/messages, keeps the flutter --version output, the --update-goldens run, the plain flutter test run that followed it and flutter analyze, so a reviewer can see the goldens were verified, not just generated.

Before merging, a reviewer opens the PNGs and checks they show the real fonts rather than placeholder boxes, confirms the surface size matches a device the team cares about, and decides whether the Ubuntu rasterizer is the one to pin, since a golden generated on macOS will not match it byte for byte. If a widget is missing, a follow-up message to the same session adds it on the same branch.

Run it on every nightly schedule

Create an automation in the Project with a schedule trigger, cron 0 2 * * * in your timezone, and a prompt that asks for goldens on widgets under lib/ui/common/ that have none yet, changed in the last week according to git log. Each night's run opens one pull request on its own branch, or none when every shared widget already has a golden, and the Project lists the run with its result.

To review the pull requests from your own tooling, start the nightly run from your scheduler with POST /o/{org}/sessions using an API key that has a webhook_url. Rush POSTs run.completed with the session_url when the turn ends, so a handler can post the link to the channel where the mobile team reads in the morning.

The use case this job follows, with its automation and trigger: Write the missing tests.

Why run it on Rush?

Golden tests are the test type teams most often skip, because the first one costs the most: a Flutter install that matches CI, fonts loaded correctly, a stable way to boot the app's dependency container, and a rasterizer decision. On a laptop that setup competes with feature work and the result depends on which machine generated the images. In a Rush sandbox the agent installs the exact Flutter version the workflow pins, renders on Ubuntu as the new job will, and leaves your checkout untouched.

What arrives is checkable. The PNGs are in the pull request to look at, the workflow runs the comparison on the branch, and the transcript shows the generate-then-verify pair of flutter test runs. Several apps or several widget groups can run at once in separate sandboxes, and a widget that proves too animated for a stable golden is listed in the description rather than silently skipped.

Frequently asked questions

Why target the common widgets instead of whole screens?

Screens like HomeScreen in lib/ui/screens/home/ drive parallax page views, animated clouds and flutter_animate sequences, and they load wonder images and data on bootstrap. A golden of one of them changes whenever any of that moves. The widgets under lib/ui/common/controls/ and the cards and dividers beside them are what the screens are built from, render deterministically with animations disabled, and are what a designer would want pinned.

Will the goldens fail on my Mac?

Possibly, by a few pixels. Font rasterization differs between macOS and Linux, so a golden generated in the Ubuntu sandbox can fail locally on macOS. The pull request adds an Ubuntu flutter test job as the source of truth. If you want local runs to pass too, the usual fix is to run the tests in Docker or to accept a small tolerance with a custom comparator, and the agent can add either in a follow-up.

How does it load fonts and strings without the real app running?

Fonts come from test/flutter_test_config.dart, which reads each family from assets/fonts/ with FontLoader before any test runs. Strings come from the generated AppLocalizations in lib/l10n/, loaded for English through the app's own LocaleLogic. Singletons are registered with the same registerSingletons() that main.dart calls, then reset between tests so state cannot leak.

Does it change any production code?

No. The prompt forbids changes under lib/, and the only manifest change is adding flutter_test to dev_dependencies. If a widget cannot be rendered in isolation without a code change, the agent says so in the pull request and leaves that widget out rather than editing it.

More Flutter jobs

More jobs like this: Write the missing tests

Keep reading