What the agent does
The agent reads how the app is built and tested before writing a workflow, because this one has needs a generic Elixir template would miss. In an app laid out like fly-apps/live_beats there is no .github directory at all. mix.exs asks for Elixir ~> 1.12 and pulls phoenix_live_view and bumblebee straight from GitHub rather than Hex, and the test alias runs ecto.create --quiet and ecto.migrate --quiet before test. config/test.exs connects to Postgres on localhost as postgres with password postgres, names the database live_beats_test with a MIX_TEST_PARTITION suffix, and sets the logger to warnings only. The Dockerfile pins the image the app actually ships on, hexpm/elixir:1.14.0-erlang-24.0.1, which is the version pair the workflow uses, with erlef/setup-beam and a cache of deps and _build keyed on mix.lock.
The need that is easy to miss is in lib/live_beats/application.ex. On start the supervisor decides whisper_serving? from the FLAME backend, and outside production that backend is the local one, so every start, the test run included, calls Bumblebee.load_model({:hf, "openai/whisper-tiny"}) and compiles the serving with EXLA. In CI that means a model download from Hugging Face and an EXLA build on every run unless the directories are cached. fly.toml already names BUMBLEBEE_CACHE_DIR, so the workflow sets it under the workspace and caches it keyed on the model name, and caches the directory EXLA downloads its archive into, keyed on the exla entry in mix.lock. The first run pays for both; later runs restore them. The pull request names this as the reason the workflow is longer than a template's.
The workflow itself is .github/workflows/ci.yml, on pushes to the default branch and on pull requests: a postgres service with the credentials config/test.exs expects and a pg_isready health check, erlef/setup-beam with the pinned pair, the caches, mix deps.get, mix format --check-formatted, which honours the Phoenix.LiveView.HTMLFormatter plugin from .formatter.exs, and mix test. MIX_ENV is set to test for every step so the dependencies compile once. Nothing builds assets: esbuild and tailwind run only through the assets.deploy alias, and the tests under test/live_beats_web/live drive LiveViews through Phoenix.LiveViewTest without a browser.
In its sandbox the agent starts a Postgres, runs mix deps.get and mix test with the same environment the workflow sets, and reads the result, including the MP3Stat tests that parse test/support/fixtures/silence1s.mp3 in pure Elixir. If the GitHub-sourced dependencies refuse the pinned Elixir, the agent moves to the lowest pair they accept and says so in the description, with the line from the dependency's mix.exs that forced it, rather than silently picking the newest.
Start it with one API call
POST https://api.phoenix.so/o/{org}/sessions with this body:
{
"agent": "claude",
"input": "Set up GitHub Actions CI for this Phoenix LiveView app. Read mix.exs, mix.lock, config/test.exs, config/config.exs, lib/live_beats/application.ex, .formatter.exs, Dockerfile and fly.toml first. Write .github/workflows/ci.yml that runs on push to the default branch and on pull requests with a Postgres service matching config/test.exs, erlef/setup-beam at the Elixir and OTP versions the Dockerfile pins, a cache of deps and _build keyed on mix.lock, a cache of the Bumblebee model directory and the EXLA archive directory because the application loads a Whisper model on start, then mix deps.get, mix format --check-formatted and mix test with MIX_ENV=test. Do not build assets. Run mix test in your sandbox against a local Postgres with the same environment and quote the result; if the pinned versions fail to compile a dependency, use the lowest versions that work and say which dependency forced it. Open a pull request.",
"resources": [
{
"url": "https://github.com/your-org/your-app"
}
]
}- Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
- Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
- Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.
What comes back
What comes back is a pull request adding one file, .github/workflows/ci.yml, and nothing else: no change to mix.exs, config/test.exs or the application. The description quotes the mix test run from the sandbox, names the Elixir and OTP pair the workflow pins and where it read them, and explains the two extra caches with the line in application.ex that loads the model.
The workflow runs on the pull request itself, so the first real result is on the checks tab, with the first run slower than the rest while the model and the EXLA archive are fetched and saved. The run's record keeps the sandbox test output and the files the agent read.
Before merging, a reviewer decides whether the model load belongs in the test environment at all, since a config switch that skips the serving under test would remove the largest cache, and whether mix compile --warnings-as-errors should be added once the existing warnings are cleared.
Run it on every new Phoenix repository
A team that keeps several Phoenix apps sends one POST /o/{org}/sessions per repository with this body and the repository in resources, from a script. Each run reads its own mix.exs and config/test.exs, so an app without a model on start gets no model cache and an app on MySQL gets that service instead, and each opens its own pull request.
Send a request_id, a UUID, with each call so a retried request returns the run it already started. Give the key a webhook_url and Rush POSTs run.completed when a run's turn ends, with session_url to read the pull request from; a run whose test suite did not pass ends needs_review rather than opening a pull request that claims a green workflow.
The use case this job follows, with its automation and trigger: Hand off a long task.
Why run it on Rush?
A CI workflow for this app is not the Elixir template: the application downloads a speech model on start, two dependencies come from GitHub, and the test database has a partition suffix. Each of those is a red first run when the workflow is written from memory. The agent reads them out of the files and runs the suite in its own sandbox before the workflow exists, so the pull request carries a result rather than a guess.
The sandbox run stays with the run and the pull request quotes it, so the reviewer sees which tests ran and how long the model load took rather than taking the workflow's shape on trust. Across every Phoenix app a team maintains, the same request runs in parallel, one sandbox and one pull request each.
Frequently asked questions
Why does a test workflow need a model cache?
Because LiveBeats.Application starts an Nx.Serving with a Whisper model whenever the FLAME backend is not the Fly one, which includes the test environment. Without a cached BUMBLEBEE_CACHE_DIR and EXLA archive, every run downloads the model and compiles the serving before the first test.
Where do the Elixir and OTP versions come from?
From the Dockerfile, which pins hexpm/elixir:1.14.0-erlang-24.0.1. mix.exs only asks for ~> 1.12, and the dependencies fetched from GitHub may need more, so the agent compiles on the pinned pair first and moves only if a dependency refuses it.
Does the workflow build the JavaScript and CSS?
No. The LiveView tests use Phoenix.LiveViewTest and Floki against rendered HTML, and esbuild and tailwind only run through the assets.deploy alias for a release. Building assets in CI would add time and prove nothing the tests check.
Why is the database name partitioned?
config/test.exs appends MIX_TEST_PARTITION to live_beats_test so several test runs can share one Postgres. The workflow leaves the variable unset for a single job, and a later matrix can set it per job without touching the config.