What the agent does
The agent clones the application into its own cloud sandbox and sets it up the way its own CI does, because an audit of code that does not compile is a guess. In a Phoenix app like hexpm/hexpm, mix.exs asks for Elixir ~> 1.20, .github/workflows/main.yml installs OTP 29.1 and Elixir 1.20.4 with a postgres:18 service, and the test alias runs ecto.create, ecto.load --skip-if-loaded and ecto.migrate before mix test. The agent starts the same Postgres, runs mix deps.get and mix compile --warnings-as-errors as the format job does, and runs the suite once so later claims rest on a working build.
Then it runs the three tools the repository does not run itself. mix hex.audit ships with Hex and lists retired packages in mix.lock. mix_audit and sobelow are not in deps(), so the agent installs both as escripts in the sandbox, leaving mix.exs and mix.lock untouched, and runs mix_audit against the lockfile for advisories and sobelow against the Phoenix code for the usual classes: config, CSRF, XSS, SQL, traversal and secrets. Each tool's full output goes into the report; the agent marks which findings it confirmed by reading the code and which it could not.
The reading is the larger half. lib/hexpm_web/router.ex sets a content security policy through HexpmWeb.Plugs.ContentSecurityPolicy with nonces for scripts and styles, script_src 'strict-dynamic', frame_ancestors 'none', form_action 'self' and object_src 'none', puts protect_from_forgery and put_secure_browser_headers in the browser pipeline, and places HexpmWeb.Plugs.Attack in every pipeline; that plug, built on plug_attack, throttles by user, organization, workload identity and IP with ETS storage and a database-backed Hexpm.RateLimits for limits that must survive a deploy. The API pipeline adds Corsica with origins: "*" for HEAD and GET only. The agent checks each of these against what the routes actually do, and reads config/runtime.exs, where secret_key_base comes from HEXPM_SECRET_KEY_BASE, beside config/dev.exs and config/test.exs, where a fixed key is checked in, to say which keys are development-only and which would matter if copied.
It also records what the repository already does so the report does not recommend it twice: codeql.yml scans the actions language only, dependency-review.yml runs on pull_request_target when mix.exs, mix.lock or a workflow changes, and dependabot.yml groups weekly mix and github-actions updates behind a seven-day cooldown. The Dockerfile builds from hexpm/elixir with MIX_ENV=prod and copies config/runtime.exs in last. Each finding in security-audit.md names the file and line, says what the agent did to confirm it, gives a severity and a fix. The agent changes no code and pushes nothing.
Start it with one API call
POST https://api.prix.dev/o/{org}/sessions with this body:
{
"agent": "claude",
"input": "Audit this Elixir and Phoenix application and write security-audit.md. Set it up as .github/workflows/main.yml does: the Postgres service, mix deps.get, mix compile --warnings-as-errors and mix test once. Run mix hex.audit; install mix_audit and sobelow as escripts in the sandbox without editing mix.exs or mix.lock, and run both. Then read lib/hexpm_web/router.ex and every plug under lib/hexpm_web/plugs, especially attack.ex, content_security_policy.ex and read_only.ex, the auth and session modules, config/runtime.exs against config/dev.exs and config/test.exs for checked-in secrets, the Dockerfile, and .github for what CodeQL, dependency review and Dependabot already cover. For each finding write the file and line, what you ran or read to confirm it, severity and a fix. Say which tool findings you could not confirm. Do not paste secrets into the report. Change no code.",
"resources": [
{
"url": "https://github.com/your-org/your-app"
}
]
}- Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
- Send the request. POST the body in request.json to https://api.prix.dev/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
- Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.
What comes back
security-audit.md comes back as an artifact on the session, with a name, a mime_type and a url on api.prix.dev that downloads it with no key until url_expires_at; reading the session again gives a fresh link, and GET /o/{org}/sessions/{id}/artifacts/security-audit.md with the key also works. The repository has no new commit, and git status in the transcript is clean.
The report has four sections: the build and test run, the tool output from mix hex.audit, mix_audit and sobelow with each item marked confirmed or unconfirmed, the findings from reading the router, plugs and config with file and line, and a last section listing what the repository already covers so nobody files a ticket for CodeQL that is already there. A finding the agent could not reproduce is left as unconfirmed rather than padded into a severity.
A reviewer judges each finding by opening the cited line and the recorded command rather than by its severity word. The fix sentences are suggestions and were not applied; the usual next step is a follow-up on the same session that turns one accepted finding into a pull request.
Run it on every Monday morning
In the console, add an automation to the Project with this prompt and a schedule trigger, cron set to an early Monday hour in your timezone. Each fire starts a run in the repository that compiles the current default branch, reruns the three tools against the current mix.lock and writes a fresh report; the run is read only, which suits an audit that changes nothing, and it appears in the Project's run list with its artifact.
To start one from your release script instead, POST the same body to https://api.prix.dev/o/{org}/sessions with the repository in resources and a request_id made from the release tag. Give the key a webhook_url and Rush POSTs run.completed when the turn finishes; the report is an artifact rather than the result text, so the handler reads data.session_url and downloads it from the session.
The use case this job follows, with its automation and trigger: Produce a report or file, no repo.
Why run it on Rush?
An audit of a Phoenix app needs a compiled build, a database, two tools the project does not install and a careful read of the router, which is an afternoon on a laptop and is rarely repeated. In its own sandbox the agent does the setup the CI way, runs the tools with their full output kept, and reads the code, so every finding carries the command or the line that supports it.
Because nothing is pushed and the result is a file, the run is safe to repeat weekly and to compare: the next report's tool sections can be diffed against this one. Several services can be audited at once, each in its own sandbox, and the transcript is the proof that the test suite passed before any claim was made.
Frequently asked questions
Does it add mix_audit and sobelow to my mix.exs?
No. Both run as escripts installed in the sandbox, so mix.exs and mix.lock are unchanged and git status stays clean. If you want them in CI, the report's last section says so as a suggestion, and a follow-up on the same session can open that pull request.
What does it do with the secret_key_base in config/dev.exs?
It reports that a fixed key is checked in for the dev, test and hex configs while config/runtime.exs reads the production key from HEXPM_SECRET_KEY_BASE, names the lines, and rates it for what it is: harmless unless a deployment reuses the dev config. The value itself is not pasted into the report.
Will it flag things CodeQL or Dependabot already cover?
It reads .github/workflows and .github/dependabot.yml first and lists what they cover in their own section. CodeQL here scans the actions language only, so Elixir code is not covered by it, which the report says plainly rather than assuming the badge means more than it does.
Does it test the running app, or only read code?
It runs mix test once to prove the build, and sobelow and the reads are static. Where a plug's behaviour matters, such as the Attack throttles or the ReadOnly write routes, it says whether it confirmed the behaviour with an existing test under test/hexpm_web or only by reading. It does not probe a deployed host.