What the agent does
The agent checks out the pull request's branch and reads the role's own contract first. In a role shaped like geerlingguy/ansible-role-docker, meta/main.yml declares role_name: docker, min_ansible_version: 2.15.1 and platforms from Debian buster through trixie, Ubuntu bionic through noble, Fedora, Alpine, Arch, SLES and openSUSE; defaults/main.yml is the public interface, from docker_edition and the docker_packages list through docker_service_manage, docker_install_compose_plugin, docker_compose_version, docker_apt_release_channel, docker_users and docker_daemon_options; and README.md documents each of those variables with its default. tasks/main.yml loads vars/<distribution>.yml or vars/<os_family>.yml with first_found, includes setup-Debian.yml, setup-RedHat.yml or setup-Suse.yml, installs packages with allow_downgrade on Ansible 2.12 and later, writes /etc/docker/daemon.json only when docker_daemon_options has keys, patches the unit's ExecStart when docker_service_start_command is set, starts the service, flushes handlers, and then handles Compose and the docker group.
Then it runs what .github/workflows/ci.yml runs. The lint job installs yamllint and lints the tree under .yamllint, which extends default with a 200-character line-length warning. The molecule job runs molecule test once per distro in a seven-way matrix: rockylinux10, ubuntu2404, ubuntu2204, debian13, debian12, fedora43 and opensuseleap15, each selecting geerlingguy/docker-${MOLECULE_DISTRO}-ansible:latest in molecule/default/molecule.yml, which uses the Docker driver with privileged: true, cgroupns_mode: host and the cgroup volume. The agent runs yamllint . and ansible-lint (with the three rules .ansible-lint skips) in every case, and molecule test for the distros the diff concerns when a Docker daemon is available to the sandbox; when it is not, it runs ansible-playbook --syntax-check against molecule/default/converge.yml and says plainly in the review that converge did not run.
The diff is read against three things Molecule's idempotence step and the role's structure make visible. Idempotence: molecule test converges twice and fails if the second pass changes anything, so a new command or shell task needs changed_when, the way tasks/docker-compose.yml marks the version check with changed_when: false and check_mode: false, and a get_url or copy needs a stable dest and mode. Privilege escalation: molecule/default/converge.yml leaves become: true commented out because the test container runs as root, so a task that only works because of that, or a user task that adds to the docker group without append: true as tasks/docker-users.yml does, is a finding. Variable precedence: vars/Debian.yml and vars/RedHat.yml each redefine docker_obsolete_packages, and vars/ outranks defaults/, so a pull request that edits the list in defaults/main.yml expecting Debian to pick it up has changed nothing on Debian, and the review says so with both file paths.
Two more checks come from the role's conventions. A new variable needs a line in defaults/main.yml and a paragraph in README.md, since the README is the role's documentation on Galaxy; the diff is checked for both. A change to molecule/default/verify.yml, which today runs docker version, checks systemctl is-active docker under systemd and pulls hello-world, is read for whether it would pass on every distro in the matrix, not only the one the author tried. The review cites file and line for each finding, separates blocking ones from suggestions, quotes the lint and Molecule output with the distros it actually converged, and pushes nothing.
Start it with one API call
POST https://api.prix.dev/o/{org}/sessions with this body:
{
"agent": "claude",
"input": "Review the open pull request on this Ansible role. Check out its branch and read meta/main.yml, defaults/main.yml, tasks/main.yml, the files under vars/ and the Molecule scenario under molecule/default to learn how the role is structured and tested. Run yamllint and ansible-lint. Run molecule test with MOLECULE_DISTRO set to the distributions the diff concerns if a Docker daemon is available; if it is not, run ansible-playbook --syntax-check on the converge playbook and say in the review that converge did not run. Read the diff for idempotence (command and shell tasks without changed_when, handlers notified on every run), privilege escalation (tasks that assume root because the Molecule container runs as root), and variable precedence (values set in vars/ that override defaults/). Check that a new variable is documented in README.md. Post one review that cites file and line for every finding, separates blocking findings from suggestions, and quotes the output of each check with the distributions it ran on. Do not push commits.",
"resources": [
{
"url": "https://github.com/your-org/your-app/pull/1"
}
]
}- Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
- Send the request. POST the body in request.json to https://api.prix.dev/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
- Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.
What comes back
What comes back is one review on the pull request. It opens with what ran: the yamllint and ansible-lint results, and for Molecule the distros converged, whether the idempotence pass was clean, and the verify output, or a plain statement that only the syntax check ran. Findings follow as file and line with a reason: a shell task without changed_when, a default that vars/Debian.yml shadows, a new variable missing from the README.
A reviewer checks the distro list against the seven in ci.yml before trusting a green converge, and reads any precedence finding with both files open, since that is the kind of bug that passes on one distro and fails on another. The session's transcript keeps every command and its output, including the second converge pass that proves idempotence.
Nothing is pushed. After the turn has finished, a follow-up message to the same session can ask the agent to make the change, and it commits from the sandbox that already has the collection and the Molecule images.
Run it on every pull request opened or pushed
In the console, add an automation to the Project with this prompt and a GitHub trigger on pull_request.opened and pull_request.synchronize. Each push to the pull request starts a run against that commit, and a redelivered event starts nothing. Only pull requests by an owner, member or collaborator fire it; pull requests from forks never do, which matters for a role that gets many drive-by contributions.
Mark the automation read-only so review runs can comment but never push. To start reviews from your own server instead, send this request with the pull request URL in resources and the delivery id as Idempotency-Key, and set a webhook_url on the API key to receive run.completed with the session_url once the review is up.
The use case this job follows, with its automation and trigger: Review every pull request.
Why run it on Rush?
A role review is only worth something if the role converged, and converging this one means a privileged Docker container per distro with systemd inside it. That is slow and messy on a laptop and leaves containers behind. The sandbox runs the scenario for the distros the diff concerns, converges twice, and keeps the output, so the review quotes an idempotence result instead of asserting one.
Precedence and become bugs are invisible in a diff viewer because the shadowing file is not in the diff. The agent reads defaults/, vars/ and the setup tasks together for every change, cites both sides, and does it for several pull requests at once, each in its own sandbox.
Frequently asked questions
Does it run all seven distros in the CI matrix?
Not unless you ask. It picks the distros the diff concerns, such as the Debian family for a change to tasks/setup-Debian.yml, and names them in the review. Name the full matrix in the prompt when a change to tasks/main.yml or defaults/main.yml should be converged everywhere.
What if Molecule cannot start its container in the sandbox?
The review says so. The agent falls back to yamllint, ansible-lint and ansible-playbook --syntax-check on molecule/default/converge.yml, and states that converge, idempotence and verify did not run, so nobody reads a lint pass as a converge pass.
Will it enforce ansible-lint rules the role skips?
No. .ansible-lint lists yaml, risky-shell-pipe and role-name under skip_list, and the agent runs with that file. A rule the role has chosen to skip is not a finding, though a suggestion may mention it when the diff adds exactly the pattern the rule exists for.
Does it check the README?
Yes, for variables. A new key in defaults/main.yml without a matching entry in the Role Variables section of README.md is a suggestion with both file paths, because the README is what Galaxy shows and what users read before setting the variable.