What the agent does
The agent finds out what CI runs today. In a Phoenix app like thechangelog/changelog.com, .github/workflows/ship_it.yml runs on pull requests and pushes to master and calls github.yml, whose build step is mise run ci. mise-tasks/ci builds the runtime image and then runs docker:test, which starts docker/compose.test.yml, and the test container's command is mise-tasks/docker/test-cmd: mix deps.get --only test, mix deps.compile when _build is empty, mix compile and mix test. Credo is in mix.exs as {:credo, "~> 1.7", only: [:dev, :test], runtime: false} and .credo.exs and .formatter.exs are both checked in, but neither mix credo nor mix format --check-formatted runs anywhere in CI.
Before touching CI, the agent measures. It installs the Erlang and Elixir versions mise.toml pins, runs mix deps.get, then mix format --check-formatted and mix credo on the code as it is, and keeps both outputs. .credo.exs runs with strict: false and already turns off several checks, among them Refactor.ABCSize, Refactor.VariableRebinding and Readability.Specs, so the agent reads that list before judging any finding. Formatting differences go in a commit of their own made by mix format and nothing else, so a reviewer can skip it. Credo findings are fixed in code where the fix is local and clear; where a check disagrees with how the whole codebase is written, the agent turns it off in .credo.exs with a comment saying why, instead of rewriting dozens of files.
Then it adds the checks where the tests already run. Because CI runs everything inside the test container, the right place is mise-tasks/docker/test-cmd, after mix compile and before mix test: mix format --check-formatted and mix credo. Both exit non-zero on a finding, which fails the step and the workflow, and both are available in the test environment the container sets with MIX_ENV=test. The agent does not add a new workflow or job, because the image build is the slow part and a second job would build it again.
To prove the change, the agent installs PostgreSQL with the postgres user and password config/test.exs expects and runs mise-tasks/docker/test-cmd directly with MIX_ENV=test, so the same script CI runs passes from start to finish in the sandbox. It then breaks formatting in one file and leaves an IO.inspect call in another on a scratch commit, runs the script again to show each check failing with its message, and drops that commit before opening the pull request.
Start it with one API call
POST https://api.phoenix.so/o/{org}/sessions with this body:
{
"agent": "claude",
"input": "Add Credo and a format check to CI. Read .github/workflows/ship_it.yml, github.yml, mise-tasks/ci, mise-tasks/docker/test and test-cmd, docker/compose.test.yml, .credo.exs, .formatter.exs and mix.exs first. Install the Erlang and Elixir in mise.toml, run mix format --check-formatted and mix credo on the current code and keep the output. Put formatting fixes in their own commit made by mix format. Fix local Credo findings in code and disable a check in .credo.exs only with a comment saying why. Add both commands to mise-tasks/docker/test-cmd after mix compile. Run that script with Postgres, show it passing and failing on a deliberate violation, and open a pull request.",
"resources": [
{
"url": "https://github.com/your-org/your-app"
}
]
}- Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
- Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
- Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.
What comes back
What comes back is a pull request with up to three commits: one made only by mix format, one with Credo fixes and any .credo.exs changes, and one that adds two lines to mise-tasks/docker/test-cmd. The description lists the counts mix credo and mix format --check-formatted reported before the change, each Credo check that was fixed in code or turned off with its reason, and the output of the test script passing at the end.
It also shows the deliberate failures: the format check naming the unformatted file and Credo naming the IO.inspect call with its line from Warning.IoInspect, from the scratch commit that is not in the branch. The repository's Ship It! workflow runs on the pull request and now runs both checks inside the test container.
A reviewer reads the .credo.exs diff most carefully, since every check turned off there stays off for the next contributor, and decides whether the team wants strict: true later as a separate change.
Run it on every pull request labelled lint-fix
Once the checks are in CI, the agent is for the pull request that fails them. An automation in the Project with a github trigger on pull_request.labeled and lint-fix in its labels adds the pull request after the prompt, so the prompt says to check out the branch of the pull request that follows, run mix format and mix credo, fix what they report without changing behaviour and push to that branch.
The trigger fires only for a label added by a repository owner, member or collaborator. An automation run has its own max_runs_per_hour cap, so a contributor who labels ten old branches at once gets them worked through in turn.
The use case this job follows, with its automation and trigger: Turn a Linear label into a PR.
Why run it on Rush?
Turning on a linter in an existing codebase is mostly triage: which findings are real, which are taste, and which checks to keep. That needs the pinned Erlang and Elixir, the dependencies and a Postgres for the test run, and the sandbox has all of them. Every command and its output stays in the run's record, so the counts in the pull request are the ones the agent saw.
The agent also proves the CI change by running the same script CI runs, and by making each check fail on purpose, which is the step people skip when they add a lint line in a hurry. The pull request arrives with the formatting churn separated from the real fixes, so the review takes as long as the real fixes need.
Frequently asked questions
Why add the checks to test-cmd instead of a new GitHub Actions job?
This repository runs its tests inside a Docker image built from the runtime image, and the build is the slow part. Adding two commands to mise-tasks/docker/test-cmd reuses that image and its compiled dependencies, while a separate job would need Erlang and Elixir set up on its own.
Will it reformat the whole codebase?
Only if mix format --check-formatted reports differences, and then in a commit made by mix format alone with no other edits. The pull request says how many files that commit touches so the team can decide to land it separately.
Do the new checks need the database?
No. mix format --check-formatted and mix credo read the source files and neither talks to Postgres, so they fail fast before mix test starts. The pull request shows the script's output so the team can see how long each step takes in their CI.
What if we want strict mode?
strict: false in .credo.exs hides lower-priority findings. Switching it to true is a separate decision with many more findings, so the agent leaves it as is and reports how many more issues mix credo --strict shows.