What the agent does
The agent reads how the app tests before it measures anything, because the coverage number depends on which tests run and which files count. In an app shaped like simplezhli/flutter_deer, pubspec.yaml asks for Dart >=3.4.0 <4.0.0 and Flutter >=3.19.0, lists flutter_test, flutter_driver, integration_test and test as dev dependencies, and turns on generate: true so l10n.yaml produces deer_localizations.dart from the .arb files in lib/l10n. The test/ directory holds widget_test.dart (pumps MessagePage and expects one text), accessibility_test.dart, which aggregates eight per-module files such as login/login_accessibility_test.dart that pump each page inside MyApp and assert meetsGuideline(labeledTapTargetGuideline), and net/dio_test.dart, which sets baseUrl to https://api.github.com/ and makes a real request. integration_test/ and test_driver/ need a device and are not part of a unit coverage run.
In the sandbox it installs the Flutter SDK at the version the repository's workflows already pin, 3.47.4 in .github/workflows/flutter-drive.yml, runs flutter pub get and flutter gen-l10n, and then flutter test --coverage, excluding the network test by tagging it: a dart_test.yaml with a network tag on test/net/dio_test.dart and --exclude-tags network on the command, so the coverage job does not depend on GitHub's API answering. The run writes coverage/lcov.info. The agent then removes what should not count: lcov --remove coverage/lcov.info 'lib/l10n/*' 'lib/generated/*' 'lib/widgets/bezier_chart/*', the same paths analysis_options.yaml already excludes from analysis, so a generated localization class does not pad or drag the figure. lcov --summary on the result gives the line percentage, and the agent rounds it down to a whole number as the floor.
Where the job goes depends on what exists. flutter-drive.yml runs flutter drive on a macOS runner with an iPhone simulator and on an Android emulator, and a third job, accessibility_test, runs flutter test test/accessibility_test.dart on macos-latest; flutter-web-deploy.yml triggers on a file name that does not exist. None runs the whole test/ directory on Linux, so the agent adds a coverage job on ubuntu-latest: subosito/flutter-action@v2 at the pinned version with caching, flutter pub get, flutter gen-l10n, the same flutter test --coverage --exclude-tags network, the lcov --remove step, a shell step that reads the line percentage from lcov --summary and fails when it is below the floor, and actions/upload-artifact for coverage/lcov.info. If a repository already has a job that runs flutter test, the agent extends that job rather than adding a second one. It adds coverage/ to .gitignore, which lists /build/ but not the coverage directory.
It does not write tests to raise the number. The floor is today's figure, so the job catches a pull request that lowers coverage without asking anyone to lift it first; raising the floor is a separate request once the team has written tests. It also leaves the macOS and emulator jobs as they are, because they test what a Linux coverage run cannot, and it leaves analysis_options.yaml alone. The pull request body shows the lcov --summary output before and after the --remove step, so a reviewer sees how much the generated files were worth, and the percentage it chose as the floor.
Start it with one API call
POST https://api.phoenix.so/o/{org}/sessions with this body:
{
"agent": "claude",
"input": "Add test coverage measurement with a floor to this Flutter app's GitHub Actions and open a pull request. Read pubspec.yaml, l10n.yaml, analysis_options.yaml, every workflow under .github/workflows, .gitignore and every file under test/ first. Install the Flutter SDK at the version the existing workflows pin, run flutter pub get and flutter gen-l10n, then flutter test --coverage, excluding any test that reaches the network by tagging it in a dart_test.yaml and passing --exclude-tags. Remove generated and excluded paths from coverage/lcov.info with lcov --remove, using the paths analysis_options.yaml excludes, and read the line percentage with lcov --summary; round it down to a whole number and use it as the floor. If a workflow already runs flutter test, add the coverage steps to that job; otherwise add a coverage job on ubuntu-latest with subosito/flutter-action at the pinned version, the same commands, a step that fails when the percentage is under the floor, and an upload of coverage/lcov.info as an artifact. Add coverage/ to .gitignore. Do not write new tests and do not change the device-based jobs. Open a pull request with the lcov summary before and after the removal and the floor you set.",
"resources": [
{
"url": "https://github.com/your-org/your-app"
}
]
}- Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
- Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
- Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.
What comes back
The pull request adds a dart_test.yaml that tags the network test, a job or steps in .github/workflows that run flutter test --coverage on Linux, strip the generated paths, enforce the floor and upload lcov.info, and one line in .gitignore. The body shows two lcov --summary outputs, before and after --remove, and names the floor it set.
The new job runs on the pull request itself, so the first evidence that the workflow is correct is its own green check beside the existing driver and accessibility jobs. The uploaded artifact is the full lcov.info, which any coverage viewer opens, so a reviewer can see which pages the accessibility tests touch and which lib/ modules have no test at all.
Before merging, a reviewer checks that the excluded paths are the generated ones and nothing hand-written, that the floor matches the summary in the body, and that the network test is tagged rather than deleted. The accessibility and widget tests still run on macOS where they ran before; the coverage job is in addition, not instead.
Run it on every coverage job that turned red
Once the floor exists, add an automation to the Project with a github trigger on workflow_run.failure and a second prompt: read the failed coverage job's log, find which files in lib/ lost covered lines in this pull request's diff, and add widget tests for them until the job passes. The trigger appends the failed run to the prompt, so the agent reads the branch and the run id from it, and max_runs_per_hour keeps a busy day from starting a run per push.
For the first pass across many apps, a script on your server lists repositories with GET /o/{org}/github, picks the ones whose pubspec.yaml has a flutter dependency and whose workflows run no flutter test --coverage, and sends one POST /o/{org}/sessions per repository with this prompt and a request_id that is a UUID v5 of the repository name, so a retry does not open a second pull request.
The use case this job follows, with its automation and trigger: Write the missing tests.
Why run it on Rush?
Measuring coverage once is easy; wiring it so it stays measured is the part that gets postponed, because it means a Linux runner, the right SDK pin, lcov on the path and a floor nobody wants to argue about. In a Rush sandbox the agent installs the SDK at the version your workflows already pin, runs the suite, and keeps the lcov --summary numbers in the transcript, so the floor it proposes comes from a run you can read.
Each app gets its own run and its own pull request, so a team with a handful of Flutter apps gets a floor per app rather than one number imposed on all of them. The check that counts is the job itself, green on the pull request that adds it.
Frequently asked questions
Why exclude the generated localization files?
l10n.yaml generates deer_localizations.dart from the .arb files, and analysis_options.yaml already excludes lib/l10n/** and lib/generated/json/** from analysis. Counting those lines would move the figure with every translation string rather than with tests, so the agent removes the same paths from lcov.info.
Does it run the integration tests or the driver tests?
No. integration_test/ and test_driver/ need a simulator or emulator, which the existing flutter-drive.yml jobs provide on macOS and Android. The coverage job runs the test/ directory on Linux and leaves those jobs as they are.
What happens to the test that calls api.github.com?
test/net/dio_test.dart makes a real request, so a network blip would fail the coverage job. The agent tags it network in dart_test.yaml and passes --exclude-tags network; the test still exists and still runs when you run flutter test without the flag.
Will it raise the floor over time?
Not on its own. The floor is today's percentage rounded down, and raising it is a decision for the team once tests are added. The job's purpose is to stop a pull request from lowering it unnoticed.