# Add Sobelow Security Scanning to a Phoenix App

Yes. An agent adds Sobelow to your Phoenix app, triages its first findings against the code, commits a config with reasons, runs the scan in CI and opens a PR.

[Run this on Rush](https://cloud.getrush.ai/run/add-sobelow/elixir/). Opens the console with this prompt ready to send, in a Project that has your repository.

## What the agent does

The agent clones the application into its own cloud sandbox and reads how it is built and checked before it adds a tool. In a Phoenix app like hexpm/hexpm, `mix.exs` requires Elixir `~> 1.20`, lists Phoenix, Plug, Ecto, Bandit, Oban, Sentry and a long tail of syntax-highlighting packages, and defines a `test` alias that creates, loads and migrates the database first. `.github/workflows/main.yml` runs `mix compile --warnings-as-errors`, `mix format --check-formatted` and `mix ecto.setup` in a format job and `mix test` in a test job, both on Elixir 1.20.4 and OTP 29.1 against a `postgres:18` service, with every action pinned to a commit SHA. A separate `codeql.yml` already has `security-events: write` and uploads results through the CodeQL action, and a `.credo.exs` is checked in, but nothing scans for Phoenix-specific security problems.

The agent adds `:sobelow` to `deps` for `:dev` and `:test` with `runtime: false`, runs `mix deps.get`, then runs `mix sobelow --verbose` once and reads every finding against the code it points at. The checks it runs for configuration pass on this app for reasons the agent can name: `lib/hexpm_web/router.ex` puts `:protect_from_forgery` and `:put_secure_browser_headers` in both the `:browser` and `:browser_api` pipelines, `config/prod.exs` sets `force_ssl` with `rewrite_on: [:x_forwarded_proto]`, and `config/runtime.exs` reads `HEXPM_SECRET_KEY_BASE` from the environment. Where the scanner points at the literal `secret_key_base` values in `config/dev.exs` and `config/test.exs`, the agent records them as development-only in the config rather than silencing the check.

Findings in code get the same treatment, one at a time. Where a template renders Markdown through `raw` and the content was produced by the app's own renderer, the agent adds a `# sobelow_skip` comment naming the check on that line with a short reason; where a finding is a real problem, it fixes it in its own commit; and anything it is unsure about is listed in the pull request as a question rather than skipped. The result is a `.sobelow-conf` with `exit: "low"` so any new finding fails the run, `skip: true` so the inline skips are honoured, and an `ignore` list that is as short as the code allows.

Then it wires the scan into CI the way the repository already does security work. A `sobelow` job in `codeql.yml` installs Elixir and OTP with `erlef/setup-beam` at the same pinned version the other jobs use, restores the same dependency cache key, runs `mix sobelow --config --format sarif` and uploads the file with the CodeQL `upload-sarif` action under the existing `security-events: write` permission, so findings appear on the Security tab and on pull requests as annotations. The agent runs `mix sobelow --config` until it exits clean, then `mix format --check-formatted` and `mix compile --warnings-as-errors` so the new dependency does not break the format job.

## Start it with one API call

`POST https://api.phoenix.so/o/{org}/sessions`

```json
{
  "agent": "claude",
  "input": "Add Sobelow security scanning to this Phoenix application. Read mix.exs, .github/workflows/main.yml, .github/workflows/codeql.yml, lib/*_web/router.ex, lib/*_web/endpoint.ex and the config files first. Add :sobelow to deps for dev and test with runtime: false, run mix deps.get and mix sobelow --verbose, and triage every finding against the code: fix real problems in their own commits, add a sobelow_skip comment with a one-line reason where the code is safe, and list anything you are unsure about in the pull request as a question. Write a .sobelow-conf with exit on low, skip enabled and the shortest ignore list the code allows, recording development-only secrets as such. Add a job to codeql.yml that installs Elixir and OTP with the same pinned erlef/setup-beam the other workflows use, restores the dependency cache, runs mix sobelow --config --format sarif and uploads the file with the CodeQL upload-sarif action. Run mix sobelow --config until it exits clean, then mix format --check-formatted and mix compile --warnings-as-errors. Open a pull request listing each finding and what you did with it.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
```

1. **Get an API key.** In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
2. **Send the request.** POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
3. **Read what came back.** Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

## What comes back

What comes back is a pull request on its own branch changing `mix.exs` and `mix.lock` for the dependency, adding `.sobelow-conf`, adding a job to `.github/workflows/codeql.yml`, and touching only the source files where a finding needed a fix or a `# sobelow_skip` comment, each in its own commit with the check name in the message. The description holds a table of findings with file, check, severity and outcome, and a short list of questions for the reviewer where the agent chose not to decide alone.

The transcript at `GET /o/{org}/sessions/{id}/messages` keeps the first `mix sobelow --verbose` run with every finding, the runs after each change, and the final `mix sobelow --config`, `mix format --check-formatted` and `mix compile --warnings-as-errors` output, so a reviewer can see the scanner go quiet rather than read that it did. The pull request's own checks then run the new SARIF job for the first time.

Before merging, a reviewer reads each skip comment and decides whether the reason holds, checks that `exit: "low"` is what the team wants on the first day, and confirms the SARIF upload appears on the Security tab. A follow-up on the same session can tighten the config, or add the scan to the pre-existing format job instead of a separate one.

## Run it on every Linear issue labelled security

Add an automation to the Project with this prompt and a `linear` trigger on `linear.issue.labeled`, filtered to the `security` label. When someone labels an issue such as "add a Phoenix security scan before the audit", the trigger places the issue after the prompt and the pull request links back to it. Only label changes made by a person in your Rush org fire the trigger, and the automation starts at most `max_runs_per_hour` runs in an hour.

Once the scan is in, keep it honest with a second automation: a `github` trigger on `pull_request.opened` and `pull_request.synchronize` with a prompt that runs `mix sobelow --config` on the pull request's branch and explains each new finding in context; the trigger clones that branch and adds the event after the prompt. From a script, either body goes to `POST /o/{org}/sessions` with the repository in `resources` and a `request_id` for a safe retry.

## Why run it on Rush?

The hard part of adding a security scanner is the first run, when it reports everything it has ever been taught to worry about and someone has to read each finding against the code. The agent does that reading in its own sandbox with the dependencies compiled and the scanner installed, and ships the decisions as a config with reasons and a table, so the team inherits a clean baseline instead of a wall of warnings to argue about.

Every scanner run and its output stay in the run's transcript, so a reviewer can check that a finding marked safe was read rather than suppressed. The compile and format checks run exactly as CI runs them, and the laptop that handed off the task is free while the dependency tree compiles.

## Frequently asked questions

### Will it suppress findings to get a clean run?

No. Each `# sobelow_skip` comment names the check and carries a one-line reason on the line it covers, fixes go in their own commits, and anything the agent cannot justify is listed in the pull request as a question. The `ignore` list in `.sobelow-conf` is kept as short as the code allows and each entry says why.

### Why put the job in codeql.yml rather than the main CI workflow?

Because that workflow already has the `security-events: write` permission and the SARIF upload pattern, so findings land on the repository's Security tab and as pull request annotations beside CodeQL's. The agent reuses the pinned `erlef/setup-beam` version and the dependency cache key from `main.yml` so the job compiles nothing twice.

### Does it flag the secret_key_base values in dev and test config?

Where the scanner does, the agent records them in `.sobelow-conf` as development-only values, with a note that `config/runtime.exs` reads the production secret from `HEXPM_SECRET_KEY_BASE`, rather than turning the secrets check off for the whole project.

### Does the app already pass the configuration checks?

The agent reads the router and config before running the scanner, and in this app the browser pipelines include `:protect_from_forgery` and `:put_secure_browser_headers` and production sets `force_ssl`, so those checks should pass. Whether they do is shown by the scanner output in the pull request, not assumed.

## More Elixir jobs

- [Explain the Architecture of an Elixir Codebase](https://getrush.ai/run/architecture-overview/elixir/)
- [Fix Flaky Tests in an Elixir App](https://getrush.ai/run/fix-flaky-tests/elixir/)
- [Fix "warning: Logger.warn/1 is deprecated. Use Logger.warning/2 instead" in an Elixir Library](https://getrush.ai/run/fix-logger-warn-is-deprecated/elixir/)

## More jobs like this: Turn a Linear label into a PR

- [Add Social Cards to an Eleventy Blog](https://getrush.ai/run/add-social-cards/eleventy/)
- [Add Social Cards to Sphinx Docs](https://getrush.ai/run/add-social-cards/sphinx/)
- [Add Social Cards to a VitePress Blog](https://getrush.ai/run/add-social-cards/vitepress/)

## Keep reading

- [How a Rush run works on your repository, from request to pull request](https://getrush.ai/cloud/)
- [Compare cloud agents, agent sandboxes and AI code review](https://getrush.ai/compare/)
- [Rush pricing: Free, Pro and Max](https://getrush.ai/pricing/)
