# Review Pull Requests on an Ansible Role

Yes. An agent checks out the pull request to your Ansible role, runs yamllint, ansible-lint and Molecule, and posts one review on idempotence, become and precedence.

[Run this on Rush](https://cloud.getrush.ai/run/review-pull-requests/ansible/), in a Project that has your repository.

## What the agent does

The agent checks out the pull request's branch and reads the role's own contract first. In a role shaped like geerlingguy/ansible-role-docker, `meta/main.yml` declares `role_name: docker`, `min_ansible_version: 2.15.1` and platforms from Debian buster through trixie, Ubuntu bionic through noble, Fedora, Alpine, Arch, SLES and openSUSE; `defaults/main.yml` is the public interface, from `docker_edition` and the `docker_packages` list through `docker_service_manage`, `docker_install_compose_plugin`, `docker_compose_version`, `docker_apt_release_channel`, `docker_users` and `docker_daemon_options`; and `README.md` documents each of those variables with its default. `tasks/main.yml` loads `vars/<distribution>.yml` or `vars/<os_family>.yml` with `first_found`, includes `setup-Debian.yml`, `setup-RedHat.yml` or `setup-Suse.yml`, installs packages with `allow_downgrade` on Ansible 2.12 and later, writes `/etc/docker/daemon.json` only when `docker_daemon_options` has keys, patches the unit's `ExecStart` when `docker_service_start_command` is set, starts the service, flushes handlers, and then handles Compose and the `docker` group.

Then it runs what `.github/workflows/ci.yml` runs. The `lint` job installs `yamllint` and lints the tree under `.yamllint`, which extends `default` with a 200-character line-length warning. The `molecule` job runs `molecule test` once per distro in a seven-way matrix: rockylinux10, ubuntu2404, ubuntu2204, debian13, debian12, fedora43 and opensuseleap15, each selecting `geerlingguy/docker-${MOLECULE_DISTRO}-ansible:latest` in `molecule/default/molecule.yml`, which uses the Docker driver with `privileged: true`, `cgroupns_mode: host` and the cgroup volume. The agent runs `yamllint .` and `ansible-lint` (with the three rules `.ansible-lint` skips) in every case, and `molecule test` for the distros the diff concerns when a Docker daemon is available to the sandbox; when it is not, it runs `ansible-playbook --syntax-check` against `molecule/default/converge.yml` and says plainly in the review that converge did not run.

The diff is read against three things Molecule's idempotence step and the role's structure make visible. Idempotence: `molecule test` converges twice and fails if the second pass changes anything, so a new `command` or `shell` task needs `changed_when`, the way `tasks/docker-compose.yml` marks the version check with `changed_when: false` and `check_mode: false`, and a `get_url` or `copy` needs a stable `dest` and `mode`. Privilege escalation: `molecule/default/converge.yml` leaves `become: true` commented out because the test container runs as root, so a task that only works because of that, or a `user` task that adds to the `docker` group without `append: true` as `tasks/docker-users.yml` does, is a finding. Variable precedence: `vars/Debian.yml` and `vars/RedHat.yml` each redefine `docker_obsolete_packages`, and `vars/` outranks `defaults/`, so a pull request that edits the list in `defaults/main.yml` expecting Debian to pick it up has changed nothing on Debian, and the review says so with both file paths.

Two more checks come from the role's conventions. A new variable needs a line in `defaults/main.yml` and a paragraph in `README.md`, since the README is the role's documentation on Galaxy; the diff is checked for both. A change to `molecule/default/verify.yml`, which today runs `docker version`, checks `systemctl is-active docker` under systemd and pulls `hello-world`, is read for whether it would pass on every distro in the matrix, not only the one the author tried. The review cites file and line for each finding, separates blocking ones from suggestions, quotes the lint and Molecule output with the distros it actually converged, and pushes nothing.

## Start it with one API call

`POST https://api.prix.dev/o/{org}/sessions`

```json
{
  "agent": "claude",
  "input": "Review the open pull request on this Ansible role. Check out its branch and read meta/main.yml, defaults/main.yml, tasks/main.yml, the files under vars/ and the Molecule scenario under molecule/default to learn how the role is structured and tested. Run yamllint and ansible-lint. Run molecule test with MOLECULE_DISTRO set to the distributions the diff concerns if a Docker daemon is available; if it is not, run ansible-playbook --syntax-check on the converge playbook and say in the review that converge did not run. Read the diff for idempotence (command and shell tasks without changed_when, handlers notified on every run), privilege escalation (tasks that assume root because the Molecule container runs as root), and variable precedence (values set in vars/ that override defaults/). Check that a new variable is documented in README.md. Post one review that cites file and line for every finding, separates blocking findings from suggestions, and quotes the output of each check with the distributions it ran on. Do not push commits.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app/pull/1"
    }
  ]
}
```

1. **Get an API key.** In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
2. **Send the request.** POST the body in request.json to https://api.prix.dev/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
3. **Read what came back.** Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

## What comes back

What comes back is one review on the pull request. It opens with what ran: the `yamllint` and `ansible-lint` results, and for Molecule the distros converged, whether the idempotence pass was clean, and the verify output, or a plain statement that only the syntax check ran. Findings follow as file and line with a reason: a `shell` task without `changed_when`, a default that `vars/Debian.yml` shadows, a new variable missing from the README.

A reviewer checks the distro list against the seven in `ci.yml` before trusting a green converge, and reads any precedence finding with both files open, since that is the kind of bug that passes on one distro and fails on another. The session's transcript keeps every command and its output, including the second converge pass that proves idempotence.

Nothing is pushed. After the turn has finished, a follow-up message to the same session can ask the agent to make the change, and it commits from the sandbox that already has the collection and the Molecule images.

## Run it on every pull request opened or pushed

In the console, add an automation to the Project with this prompt and a GitHub trigger on `pull_request.opened` and `pull_request.synchronize`. Each push to the pull request starts a run against that commit, and a redelivered event starts nothing. Only pull requests by an owner, member or collaborator fire it; pull requests from forks never do, which matters for a role that gets many drive-by contributions.

Mark the automation read-only so review runs can comment but never push. To start reviews from your own server instead, send this request with the pull request URL in `resources` and the delivery id as `Idempotency-Key`, and set a `webhook_url` on the API key to receive `run.completed` with the `session_url` once the review is up.

## Why run it on Rush?

A role review is only worth something if the role converged, and converging this one means a privileged Docker container per distro with systemd inside it. That is slow and messy on a laptop and leaves containers behind. The sandbox runs the scenario for the distros the diff concerns, converges twice, and keeps the output, so the review quotes an idempotence result instead of asserting one.

Precedence and become bugs are invisible in a diff viewer because the shadowing file is not in the diff. The agent reads `defaults/`, `vars/` and the setup tasks together for every change, cites both sides, and does it for several pull requests at once, each in its own sandbox.

## Frequently asked questions

### Does it run all seven distros in the CI matrix?

Not unless you ask. It picks the distros the diff concerns, such as the Debian family for a change to `tasks/setup-Debian.yml`, and names them in the review. Name the full matrix in the prompt when a change to `tasks/main.yml` or `defaults/main.yml` should be converged everywhere.

### What if Molecule cannot start its container in the sandbox?

The review says so. The agent falls back to `yamllint`, `ansible-lint` and `ansible-playbook --syntax-check` on `molecule/default/converge.yml`, and states that converge, idempotence and verify did not run, so nobody reads a lint pass as a converge pass.

### Will it enforce ansible-lint rules the role skips?

No. `.ansible-lint` lists `yaml`, `risky-shell-pipe` and `role-name` under `skip_list`, and the agent runs with that file. A rule the role has chosen to skip is not a finding, though a suggestion may mention it when the diff adds exactly the pattern the rule exists for.

### Does it check the README?

Yes, for variables. A new key in `defaults/main.yml` without a matching entry in the Role Variables section of `README.md` is a suggestion with both file paths, because the README is what Galaxy shows and what users read before setting the variable.