What the agent does
Each customer repository gets its own session, started by your backend, and the agent begins with the Ruby side of the lockfile. In a Rails app like codetriage/CodeTriage the Gemfile reads its Ruby from .ruby-version (4.0.5) with ruby file: ".ruby-version", pins rails to exactly 8.1.3.1, and carries an older asset stack beside it: coffee-rails ~> 5.0.0, neat ~> 1.7 (locked at 1.7.4 in Gemfile.lock), bourbon, sassc and sprockets. Background work is Sidekiq, sign-in is Devise with omniauth-github, and the database is Postgres through pg. The agent installs that Ruby, runs bundle install, then bundle outdated --only-explicit --major so only gems named in the Gemfile and a major version behind make the list.
There is no config/database.yml in the repository, which is the first thing a naive test run trips on. The agent reads .github/workflows/main.yml and does what CI does: starts Postgres, copies config/database.ci.yml to config/database.yml inside the sandbox, runs RAILS_ENV=test bundle exec rake db:create db:schema:load, then bin/rake test. It also runs bundle exec standardrb, the workflow's lint step, and RAILS_ENV=test bin/rails db:seed, which CI runs to make sure seeding still works. test/test_helper.rb configures VCR with allow_http_connections_when_no_cassette = false against the cassettes in test/vcr_cassettes/, so the suite talks to GitHub only through recordings; a test that tries to reach the network without one fails, and the agent reports it as such rather than as a flaky failure.
For risky patterns it reads each candidate instead of counting grep hits. In this repository html_safe appears in app/controllers/users/omniauth_callbacks_controller.rb, app/controllers/topics_controller.rb, app/views/users/show.html.slim and app/views/repos/_docs.html.slim, and in each case it is called on a string literal the code builds itself, with no user value inside, so none of them is a finding. Issue.find_by_sql in app/models/issue_assigner.rb passes named binds (sub_id:, repo_id:, email_limit:), so it is not an injection either. The agent goes on to what does matter in a Rails app: where strings with interpolation, params.permit!, constantize or send on request input, skipped CSRF checks, and secrets committed in config/. Brakeman is not in this Gemfile, so the agent installs it in the sandbox with gem install brakeman, runs it, and reads each warning before deciding whether it belongs in the ten.
The prompt says not to change any file. The copied database.yml, the test database and the Brakeman install exist only in the sandbox; the repository is untouched, and the final answer is the JSON.
Start it with one API call
POST https://api.prix.dev/o/{org}/sessions with this body:
{
"agent": "claude",
"input": "Audit this Rails repository without changing any tracked file. Return only JSON with three arrays. outdated_dependencies: gems named in the Gemfile whose latest release is at least one major version ahead of Gemfile.lock, with name, locked and latest. tests: each test, lint and seed command the CI workflows run, with the command, passed true or false, and the last 20 lines of output when it failed; set up the database the way CI does, inside the sandbox. risky_patterns: up to ten patterns such as SQL strings with interpolated input, params.permit!, constantize or send on request input, skipped CSRF protection, html_safe or raw on user data, or committed secrets, each with file, line, pattern and a one-sentence reason. You may install Brakeman in the sandbox to help, but read each warning before listing it.",
"resources": [
{
"url": "https://github.com/your-org/your-app"
}
]
}- Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
- Send the request. POST the body in request.json to https://api.prix.dev/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
- Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.
What comes back
The audit arrives as the session's result and, when it is 16 KiB or less, as data.result in the run.completed webhook; otherwise the webhook omits it and your server reads data.session_url with the key. For a repository set up like this one, each tests entry names the CI step it mirrors (standardrb, bin/rake test, db:seed) so the customer can match the audit to their own pipeline. Gems are listed with the version Gemfile.lock installs and the latest major, which lets your product group them, for example separating the asset pipeline gems from the framework ones.
Before trusting the audit, your product checks that the JSON parses into the three arrays. A reviewer sampling audits looks at the risky patterns: each should involve input the code does not control, and a Brakeman warning should only appear when the agent explains why it holds for this code. If a test failed, the tail of its output in the entry should show a real assertion, not a database the sandbox failed to create. The transcript has each command and its full output.
Run it on every Monday morning
A weekly audit catches what a one-time audit misses: new major releases, a test that started failing, a pattern introduced last week. A scheduler on your side, a cron job or your queue's delayed jobs, walks the list of connected customer repositories each Monday and starts one session for each, spacing them so no more than the key's max_concurrent runs are in progress at once.
Each finished audit POSTs run.completed to the key's webhook, signed with its whsec_ secret. Store the JSON with the date and compare it with last week's: a gem that newly fell a major version behind, or a test that passed and now fails, is what your product shows the customer first.
The use case this job follows, with its automation and trigger: Run an agent per customer task.
Why run it on Rush?
A Rails test run needs the exact Ruby from .ruby-version, native gems compiled for it, a Postgres server and the customer's own code executing against it. A Rush session brings all of that up in a sandbox that belongs to one audit, so customers' databases and gems never share a machine and nothing has to be cleaned up between them.
Each answer is backed by what ran. The outdated list comes from bundle outdated, the test results from the suite CI runs, and the risky patterns from files the agent opened and reasoned about, including the ones it decided were safe. That record stays in the transcript, so your support team can answer a customer's question about one line without re-running the audit.
Frequently asked questions
Will a Brakeman warning always show up as a risky pattern?
No. The agent reads each warning in context and lists only the ones it can explain for this code, up to ten. Brakeman flags patterns; some are safe in practice, like html_safe on a string literal, and the audit is more useful to your customer without them.
What if the repository needs Redis or Sidekiq for its tests?
The agent reads the CI workflow and the test helper to find out. If the suite needs Redis, it starts Redis in the sandbox; if jobs run inline in tests, as many Rails test configs set, nothing else is needed. The tests entry records the setup it used.
Does the audit open a pull request with fixes?
No. The prompt forbids changes and the result is JSON. If a customer wants one finding fixed, send POST /o/{org}/sessions/{id}/messages to the same session asking for a pull request; its sandbox still has the clone and the test database.
How do I match a webhook back to the customer?
Save the execution_id from the 202 that started the run against the customer's id. The webhook's data.execution_id is the same value, so your handler looks it up. The request body has no field for your own ids, so this mapping lives in your database.