# Run a Security Audit on a Flask App

Yes. An agent reads your Flask app's auth, sessions, uploads and plugins, runs ruff's bandit rules and pip-audit, and reports each finding with file and line.

[Run this on Rush](https://cloud.getrush.ai/run/security-audit/flask/), in a Project that has your repository.

## What the agent does

The audit starts from the project's own tooling, not from a generic checklist. In a Flask app shaped like CTFd/CTFd, `pyproject.toml` pins Python to 3.11 with `requires-python = ">=3.11,<3.12"`, Flask at 2.1.3, Werkzeug at 2.2.3, SQLAlchemy at 1.4.54 and every other dependency at an exact version, and resolves them through `uv.lock`; `requirements.txt` is exported from that lock, and the lint workflow fails when the two drift. The ruff configuration already selects the `S` rule set, the flake8-bandit checks, with `S105` ignored and every `S` rule switched off under `tests/`. The agent runs `ruff check` with that configuration first, so its report begins from what the maintainers already accept, and then runs `pip-audit` against `requirements.txt` to list advisories against the pinned versions, Flask 2.1 and Werkzeug 2.2 included.

Then it reads the code paths that carry security weight. Authentication lives in `CTFd/utils/security/auth.py`, where `login_user` writes the user id, a CSRF nonce and an HMAC of the password hash into the session, and in `CTFd/utils/crypto/__init__.py`, where passwords are hashed with passlib's `bcrypt_sha256`. Sessions are server-side: `CTFd/utils/sessions/__init__.py` defines `CachingSessionInterface`, which stores session data in the cache under a signed id, and `CTFd/__init__.py` installs it on the app class next to a `SandboxedEnvironment` for Jinja, so theme templates render in a sandbox. The agent checks what each of these does with the `SECRET_KEY` from `CTFd/config.py`, which generates and writes a `.ctfd_secret_key` file when the key is unset, and whether `SESSION_COOKIE_HTTPONLY`, `SESSION_COOKIE_SAMESITE` and `PERMANENT_SESSION_LIFETIME` from `config.ini` reach the Flask config.

Request guards come next. `CTFd/utils/initialization/__init__.py` registers `before_request` handlers: one reads the `Authorization` header, splits it into type and token and resolves the token through `lookup_user_token`, with `401` on an unknown or expired token and `403` on a banned user or team; another enforces CSRF, skipping functions marked `_bypass_csrf`, theme assets on safe methods and any request carrying an `Authorization` header, then comparing the session nonce with the `CSRF-Token` header for JSON bodies or the `nonce` form field otherwise. The decorators in `CTFd/utils/decorators/__init__.py` (`authed_only`, `admins_only`, `require_verified_emails`, `ratelimit`) are the authorization layer, and the agent cross-references them against the 24 flask-restx namespaces registered in `CTFd/api/__init__.py`, which declares an `AccessToken` API key scheme in the `Authorization` header. Uploads go through `CTFd/utils/uploads/uploaders.py`, where `FilesystemUploader.upload` passes the filename through `secure_filename`, puts it under a random 16-byte hex directory and joins with `safe_join`, and an `S3Uploader` handles the other provider; plugins are imported with `importlib.import_module` from `CTFd/plugins/` unless `SAFE_MODE` is on.

Each finding is written with the file, the line, what an attacker could do and the smallest change that closes it, and each is ranked. Deployment is in scope too: the `Dockerfile` builds a venv in a build stage, runs the release stage as user 1001 and exposes port 8000, `docker-compose.yml` fronts the app with nginx and `conf/nginx/http.conf` sets `client_max_body_size 4G`, and `config.ini` has a `REVERSE_PROXY` setting that turns on Werkzeug's `ProxyFix`. The agent notes where a header such as `X-Forwarded-For` is trusted and what `TRUSTED_HOSTS`, backported into `CTFdFlask.create_url_adapter`, protects. It changes no code and pushes nothing.

## Start it with one API call

`POST https://api.prix.dev/o/{org}/sessions`

```json
{
  "agent": "claude",
  "input": "Audit this Flask application for security problems and write security-audit.md. Start from pyproject.toml, uv.lock and requirements.txt: record the Python and Flask versions, run ruff check with the project's own configuration (note which S rules it ignores) and run pip-audit against requirements.txt. Then read the code that handles authentication, sessions, CSRF, API tokens, password hashing, file uploads, HTML sanitization, plugin loading and the reverse-proxy settings, following CTFd/utils/security, CTFd/utils/sessions, CTFd/utils/initialization, CTFd/utils/decorators, CTFd/utils/uploads, CTFd/plugins and CTFd/config.py, plus the Dockerfile, docker-compose.yml and conf/nginx. For each finding give the file and line, what an attacker could do, how confident you are, and the smallest fix; rank them. Separate confirmed problems from things worth checking. List the dependency advisories with the pinned version and the first fixed version. Do not change code, do not push, do not open a pull request.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
```

1. **Get an API key.** In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
2. **Send the request.** POST the body in request.json to https://api.prix.dev/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
3. **Read what came back.** Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

## What comes back

What comes back is one Markdown report, `security-audit.md`, attached to the session as an artifact with a download link that stays valid for an hour and is refreshed on every session read. It opens with the versions it audited and the exact `ruff check` and `pip-audit` commands with their output, then lists findings in rank order, each with a path such as `CTFd/utils/initialization/__init__.py`, the line, the risk in a sentence, a confidence level and the fix. Dependency advisories are a table: package, pinned version, advisory id, first fixed version, whether the lock pins it transitively.

The transcript at `GET /o/{org}/sessions/{id}/messages` keeps every file the agent opened and every command it ran, so a reviewer can see that a claim about the CSRF handler came from reading the handler rather than from the Flask documentation. Nothing in the repository changes: the session read lists no pull request.

Before acting on the report, a reviewer checks each high-ranked finding against the code themselves, decides which dependency bumps the pinned Flask 2.1 line can take without a larger upgrade, and treats the `S` rules the project deliberately ignores as a policy to confirm with the maintainers rather than as findings.

## Run it on every release branch

Start one run per release from your own release script with this request body and a `request_id`, one UUID kept per tag, so a retried call returns the same run instead of starting a second audit. The result lands in the session as an artifact; read it from `GET /o/{org}/sessions/{id}` when `status` leaves `running`, or give the API key a `webhook_url` and handle the `run.completed` event, which carries `session_url` to fetch the report.

For a standing cadence, create an automation in the Project with a `schedule` trigger, for example `every: 1d` in your timezone, and this prompt. Each fire starts a fresh run on the current default branch with its own `max_runs_per_hour` cap, and the report from each run is kept with its session, so the audits can be diffed month to month.

## Why run it on Rush?

A CTFd audit needs a Python 3.11 environment with the exact lock installed, `ruff` and `pip-audit` on the path, and time to read through the API namespaces and the session code without interruption. On Rush that happens in a sandbox with its own clone, and the commands run there with their output captured, so nobody has to reproduce a `pip-audit` result on a laptop to trust it. Several audits, one per repository or per release branch, run at the same time in separate sandboxes.

The report carries its proof: each finding points at a file and line, and the transcript shows the read that produced it. That is what makes it reviewable by the person who owns `CTFd/utils/security`, and what lets the next audit start from the last one's list rather than from zero.

## Frequently asked questions

### Does it run bandit?

It runs the bandit rule set through ruff, because the project already selects the `S` rules in `pyproject.toml`. Running the project's own configuration keeps the report consistent with what CI enforces; a separate bandit install would duplicate those checks and disagree with the project's ignore list.

### Will it flag the pinned Flask 2.1 and Werkzeug 2.2?

It reports whatever `pip-audit` finds against `requirements.txt`, with the first fixed version beside each advisory. Whether to move off the 2.x line is a decision for the maintainers; the report says what each advisory affects so that decision can be made with the facts in front of it.

### Does it test the running application?

The main audit is a code read plus the two tools. The agent can start the app with `python serve.py` against SQLite to confirm a behaviour it is unsure about, such as the response headers set in `response_headers`, and it says in the report which claims it verified that way.

### Will it change anything in my repository?

No. The prompt forbids code changes, pushes and pull requests, and the report is the only output. Fixes are a separate run, started from the findings you choose to act on.

### Can it audit plugins and themes too?

Plugins under `CTFd/plugins/` are imported with `importlib` at startup, so the agent reads each plugin's `load` function and any routes it registers, and it notes which plugin assets are served through `register_plugin_assets_directory`. Themes render through the sandboxed Jinja environment, and the report says what that sandbox does and does not prevent.