# Review Pull Requests Automatically in a Rails App

Yes. An agent checks out the Rails pull request, runs bin/rails test, and posts one review of the issue-assignment code with file and line citations.

[Run this on Rush](https://cloud.getrush.ai/run/review-pull-requests/rails/), in a Project that has your repository.

## What the agent does

The agent checks out the pull request and reads the Ruby toolchain before Bundler. In an app shaped like codetriage/CodeTriage, .ruby-version is 4.0.5, the Gemfile pins ruby file: .ruby-version and rails 8.1.3.1, and Gemfile.lock records ruby 4.0.5 and BUNDLED WITH 4.0.12. Sidekiq is the background-job gem. There is no config/database.yml in the tree. config/database.ci.yml is the committed Postgres config: adapter postgresql, username postgres, password password, database triage_test for the test environment. .github/workflows/main.yml copies that file to config/database.yml, loads the schema with RAILS_ENV=test bundle exec rake db:create db:schema:load against a postgres:16 service, runs bundle exec standardrb, then bin/rake test, then RAILS_ENV=test bin/rails db:seed.

Issue assignment is not a class under app/jobs. app/models/issue_assigner.rb is a 79-line PORO. Its bang assign method walks subscriptions and calls assign_issues_for_sub, which loads open issues with a find_by_sql that selects from issues where repo_id and state match, excludes ids already in issue_assignments for that subscription, orders by random(), and limits to email_limit. A valid issue becomes an assignment with delivered left unset and increments the count. An invalid one is inserted with delivered: true so the next select skips it. The method calls itself until stop? sees the subscription at its email_limit. can_access_network defaults to the opposite of Rails.env.test?, so a test run does not call GitHub unless the caller passes true. app/jobs/send_single_triage_email_job.rb calls that method with IssueAssigner.new(repo_sub.user, [repo_sub]). User#issue_assignments_to_deliver calls issue_assigner, which memoizes IssueAssigner.new(self, subs) for active repo subscriptions.

bin/rails test is the command in the request, and it is the Minitest suite CI runs via bin/rake test. The agent copies database.ci.yml into place the way the workflow does, then runs the suite. test/test_helper.rb starts SimpleCov, sets RAILS_ENV to test, loads fixtures :all, and includes Capybara, Warden and WebMock. test/unit/issue_assignment_test.rb uses the users(:schneems) fixture. One example deletes undelivered assignments and expects that method to leave 3. Another stubs email_limit to 1 on two subscriptions and expects 2. The GitHub example stubs api.github.com for bemurphy/issue_triage_sandbox and passes can_access_network: true. test/functional/issue_assignments_controller_test.rb covers the controller. The review quotes this run's output.

The review cites file and line and splits blocking from suggestions. A blocking note is a change to the NOT IN clause, the random() order, or the delivered: true insert, because those decide which issues a subscriber receives. A suggestion is a new network call that forgets the can_access_network flag and would hang under WebMock, or a migration that assumes database.yml exists. IssueAssignment validates issue_id uniqueness scoped to repo_subscription_id. The agent does not push. standardrb is what CI lints with, so the review mentions it when the diff would fail that step, and the request's test command remains bin/rails test.

## Start it with one API call

`POST https://api.prix.dev/o/{org}/sessions`

```json
{
  "agent": "claude",
  "input": "Review this pull request on the CodeTriage Rails app. Check it out, install with Bundler (Ruby 4.0.5 from .ruby-version, Rails 8.1.3.1), copy config/database.ci.yml to config/database.yml, and run bin/rails test. Read the diff with app/models/issue_assigner.rb, app/models/issue_assignment.rb, app/jobs/send_single_triage_email_job.rb and test/unit/issue_assignment_test.rb. Post one GitHub review that cites file and line for every finding, separates blocking issues from suggestions, and includes the test result. Do not push commits.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app/pull/1"
    }
  ]
}
```

1. **Get an API key.** In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
2. **Send the request.** POST the body in request.json to https://api.prix.dev/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
3. **Read what came back.** Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

## What comes back

What comes back is a review on the pull request, plus the session record of Bundler and bin/rails test. The session read names the repository and the pull request. The review's test section is the Minitest summary from that run. Findings point at lines in issue_assigner.rb, the job that calls it, and the fixture test that locks the counts. The branch is not updated by the agent. If Postgres was not reachable and the suite could not boot, that error is the test result, written as a setup failure rather than as a product defect.

A reviewer checks three things before relying on the review. The database file used was database.ci.yml, not a guessed SQLite config. The counts in the review match the asserts in issue_assignment_test.rb only as a description of the existing tests, not as a claim about this diff's new behavior, unless the diff changed those asserts. And no commit was added. The transcript keeps the commands. Send a follow-up after status leaves running if you want the same checkout to re-run one test file.

## Run it on every pull request opened

When GitHub sends pull_request.opened, POST the session with that pull request's URL in resources. The sandbox clones the head branch through the Rush GitHub App. Put the delivery id in the Idempotency-Key header so GitHub's retry does not start a second Bundler install. A repository the App cannot see is 422 RESOURCE_NOT_ACCESSIBLE, and any URL that is not a GitHub repo or a Linear issue is 422 RESOURCE_UNSUPPORTED.

The key's webhook receives run.completed when the review turn finishes. result on that payload is the answer, truncated past 16 KiB, with session_url for the rest. Verify webhook-signature against the raw body and the whsec_ secret. Rails reviews are heavy because Bundler and the suite take the sandbox, so queue them inside max_concurrent instead of starting one per opened pull request in the same second.

## Why run it on Rush?

Running this review on a laptop means installing Ruby 4.0.5, Bundler 4.0.12, and Postgres with the triage_test role from database.ci.yml, then remembering that database.yml is not in git. The sandbox does the copy and bin/rails test on its own clone. WebMock and the fixture users stay inside that run. Your checkout is not switched to the pull request branch, and the turn continues if you close the machine.

The value of the review is the test output next to the SQL. random() and the delivered flag only make sense against the Minitest that stubs email_limit and the GitHub calls. A comment written from the diff cannot show whether bin/rails test booted. Each pull request gets a separate sandbox, so two assignment changes do not share a Postgres cluster or a Bundler path, and a person still merges.

## Frequently asked questions

### bin/rails test needs Postgres. Does the agent install a database server?

It prepares the app the way CI does: copy config/database.ci.yml to config/database.yml and run the test task. The sandbox has to be able to reach Postgres for the suite to boot. If it cannot, the review quotes that failure and does not switch the adapter to SQLite to force a pass.

### Will it change Gemfile or the workflow?

It does not push, so Gemfile, Gemfile.lock and .github/workflows/main.yml stay as the pull request left them. A finding can say the workflow's bin/rake test and the request's bin/rails test are the same Minitest run, and that standardrb runs first in CI.

### Issue assignment lives in a model, not a job. What does it review?

app/models/issue_assigner.rb, the IssueAssignment validations, and the caller in app/jobs/send_single_triage_email_job.rb. The jobs directory also has send_daily_triage_email_job.rb and populate_issues_job.rb. Those are in the review only when the diff touches them.

### The tests stub GitHub. Will the agent call the real API?

bin/rails test runs with can_access_network off by default, and the one example that turns it on stubs api.github.com with WebMock. The agent does not remove those stubs. A diff that calls GitHub without a stub is a finding, because the suite would try the network.

### Can I point it at one test file?

Yes. Replace bin/rails test in the input with bin/rails test test/unit/issue_assignment_test.rb when that is all you want quoted. The review should still say the full suite is what CI runs, so a green single file is not described as a green workflow.