What the agent does
The audit starts from the project's own tooling, not from a generic checklist. In a Flask app shaped like CTFd/CTFd, pyproject.toml pins Python to 3.11 with requires-python = ">=3.11,<3.12", Flask at 2.1.3, Werkzeug at 2.2.3, SQLAlchemy at 1.4.54 and every other dependency at an exact version, and resolves them through uv.lock; requirements.txt is exported from that lock, and the lint workflow fails when the two drift. The ruff configuration already selects the S rule set, the flake8-bandit checks, with S105 ignored and every S rule switched off under tests/. The agent runs ruff check with that configuration first, so its report begins from what the maintainers already accept, and then runs pip-audit against requirements.txt to list advisories against the pinned versions, Flask 2.1 and Werkzeug 2.2 included.
Then it reads the code paths that carry security weight. Authentication lives in CTFd/utils/security/auth.py, where login_user writes the user id, a CSRF nonce and an HMAC of the password hash into the session, and in CTFd/utils/crypto/__init__.py, where passwords are hashed with passlib's bcrypt_sha256. Sessions are server-side: CTFd/utils/sessions/__init__.py defines CachingSessionInterface, which stores session data in the cache under a signed id, and CTFd/__init__.py installs it on the app class next to a SandboxedEnvironment for Jinja, so theme templates render in a sandbox. The agent checks what each of these does with the SECRET_KEY from CTFd/config.py, which generates and writes a .ctfd_secret_key file when the key is unset, and whether SESSION_COOKIE_HTTPONLY, SESSION_COOKIE_SAMESITE and PERMANENT_SESSION_LIFETIME from config.ini reach the Flask config.
Request guards come next. CTFd/utils/initialization/__init__.py registers before_request handlers: one reads the Authorization header, splits it into type and token and resolves the token through lookup_user_token, with 401 on an unknown or expired token and 403 on a banned user or team; another enforces CSRF, skipping functions marked _bypass_csrf, theme assets on safe methods and any request carrying an Authorization header, then comparing the session nonce with the CSRF-Token header for JSON bodies or the nonce form field otherwise. The decorators in CTFd/utils/decorators/__init__.py (authed_only, admins_only, require_verified_emails, ratelimit) are the authorization layer, and the agent cross-references them against the 24 flask-restx namespaces registered in CTFd/api/__init__.py, which declares an AccessToken API key scheme in the Authorization header. Uploads go through CTFd/utils/uploads/uploaders.py, where FilesystemUploader.upload passes the filename through secure_filename, puts it under a random 16-byte hex directory and joins with safe_join, and an S3Uploader handles the other provider; plugins are imported with importlib.import_module from CTFd/plugins/ unless SAFE_MODE is on.
Each finding is written with the file, the line, what an attacker could do and the smallest change that closes it, and each is ranked. Deployment is in scope too: the Dockerfile builds a venv in a build stage, runs the release stage as user 1001 and exposes port 8000, docker-compose.yml fronts the app with nginx and conf/nginx/http.conf sets client_max_body_size 4G, and config.ini has a REVERSE_PROXY setting that turns on Werkzeug's ProxyFix. The agent notes where a header such as X-Forwarded-For is trusted and what TRUSTED_HOSTS, backported into CTFdFlask.create_url_adapter, protects. It changes no code and pushes nothing.
Start it with one API call
POST https://api.prix.dev/o/{org}/sessions with this body:
{
"agent": "claude",
"input": "Audit this Flask application for security problems and write security-audit.md. Start from pyproject.toml, uv.lock and requirements.txt: record the Python and Flask versions, run ruff check with the project's own configuration (note which S rules it ignores) and run pip-audit against requirements.txt. Then read the code that handles authentication, sessions, CSRF, API tokens, password hashing, file uploads, HTML sanitization, plugin loading and the reverse-proxy settings, following CTFd/utils/security, CTFd/utils/sessions, CTFd/utils/initialization, CTFd/utils/decorators, CTFd/utils/uploads, CTFd/plugins and CTFd/config.py, plus the Dockerfile, docker-compose.yml and conf/nginx. For each finding give the file and line, what an attacker could do, how confident you are, and the smallest fix; rank them. Separate confirmed problems from things worth checking. List the dependency advisories with the pinned version and the first fixed version. Do not change code, do not push, do not open a pull request.",
"resources": [
{
"url": "https://github.com/your-org/your-app"
}
]
}- Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
- Send the request. POST the body in request.json to https://api.prix.dev/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
- Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.
What comes back
What comes back is one Markdown report, security-audit.md, attached to the session as an artifact with a download link that stays valid for an hour and is refreshed on every session read. It opens with the versions it audited and the exact ruff check and pip-audit commands with their output, then lists findings in rank order, each with a path such as CTFd/utils/initialization/__init__.py, the line, the risk in a sentence, a confidence level and the fix. Dependency advisories are a table: package, pinned version, advisory id, first fixed version, whether the lock pins it transitively.
The transcript at GET /o/{org}/sessions/{id}/messages keeps every file the agent opened and every command it ran, so a reviewer can see that a claim about the CSRF handler came from reading the handler rather than from the Flask documentation. Nothing in the repository changes: the session read lists no pull request.
Before acting on the report, a reviewer checks each high-ranked finding against the code themselves, decides which dependency bumps the pinned Flask 2.1 line can take without a larger upgrade, and treats the S rules the project deliberately ignores as a policy to confirm with the maintainers rather than as findings.
Run it on every release branch
Start one run per release from your own release script with this request body and a request_id, one UUID kept per tag, so a retried call returns the same run instead of starting a second audit. The result lands in the session as an artifact; read it from GET /o/{org}/sessions/{id} when status leaves running, or give the API key a webhook_url and handle the run.completed event, which carries session_url to fetch the report.
For a standing cadence, create an automation in the Project with a schedule trigger, for example every: 1d in your timezone, and this prompt. Each fire starts a fresh run on the current default branch with its own max_runs_per_hour cap, and the report from each run is kept with its session, so the audits can be diffed month to month.
The use case this job follows, with its automation and trigger: Produce a report or file, no repo.
Why run it on Rush?
A CTFd audit needs a Python 3.11 environment with the exact lock installed, ruff and pip-audit on the path, and time to read through the API namespaces and the session code without interruption. On Rush that happens in a sandbox with its own clone, and the commands run there with their output captured, so nobody has to reproduce a pip-audit result on a laptop to trust it. Several audits, one per repository or per release branch, run at the same time in separate sandboxes.
The report carries its proof: each finding points at a file and line, and the transcript shows the read that produced it. That is what makes it reviewable by the person who owns CTFd/utils/security, and what lets the next audit start from the last one's list rather than from zero.
Frequently asked questions
Does it run bandit?
It runs the bandit rule set through ruff, because the project already selects the S rules in pyproject.toml. Running the project's own configuration keeps the report consistent with what CI enforces; a separate bandit install would duplicate those checks and disagree with the project's ignore list.
Will it flag the pinned Flask 2.1 and Werkzeug 2.2?
It reports whatever pip-audit finds against requirements.txt, with the first fixed version beside each advisory. Whether to move off the 2.x line is a decision for the maintainers; the report says what each advisory affects so that decision can be made with the facts in front of it.
Does it test the running application?
The main audit is a code read plus the two tools. The agent can start the app with python serve.py against SQLite to confirm a behaviour it is unsure about, such as the response headers set in response_headers, and it says in the report which claims it verified that way.
Will it change anything in my repository?
No. The prompt forbids code changes, pushes and pull requests, and the report is the only output. Fixes are a separate run, started from the findings you choose to act on.
Can it audit plugins and themes too?
Plugins under CTFd/plugins/ are imported with importlib at startup, so the agent reads each plugin's load function and any routes it registers, and it notes which plugin assets are served through register_plugin_assets_directory. Themes render through the sandboxed Jinja environment, and the report says what that sandbox does and does not prevent.