What the agent does
The agent builds the app and maps its surface before it judges anything. In a repository laid out like feathersjs/feathers-chat it runs npm install and npm test in feathers-chat-ts, then reads src/app.ts for the transports and middleware order, src/services/index.ts for the services, and each service's *.shared.ts for the methods exposed to clients. For every exposed method it writes down which hooks run: here users.create has no authenticate('jwt') while the other four user methods and all five message methods do. That table is the first section of the report, because in Feathers a method missing from a hook list is the usual way a rule goes missing.
Then it reads the schemas and resolvers as the access rules they are, and tests each suspicion against the running app. users.schema.ts types email and avatar as plain Type.String() although src/validators.ts registers an email format, and public/client.js writes user.email and user.avatar into innerHTML while escaping only the message text, so the agent registers a user whose email holds markup and loads the page in the sandbox's browser to see whether it runs. Both patch schemas are Type.Partial of the full schema with nothing narrowing them, so it sends a patch that sets userId on a message and githubId on a user and records the response. messages.ts logs params.user with console.log on every call, so it reads the server's output for a password hash. Any signed-in user can find every user's email, and src/channels.ts publishes every event to every authenticated connection; the report says so and asks whether that is intended for your app as it is for a public chat.
Configuration comes next. config/default.json holds a value for authentication.secret in the repository, with FEATHERS_SECRET mapped over it in config/custom-environment-variables.json; the JWT options name the audience https://yourdomain.com, a one-day lifetime and HS256; the GitHub OAuth key and secret are placeholders. src/app.ts calls cors() with no options for REST while Socket.io is limited to the origins list. The agent reports each with its line and what to check in production, and never copies a secret's value into the report.
Last come the dependencies and the pipeline. It runs npm audit in feathers-chat-ts, react-chat and quick-start separately, since each has its own package.json, and reads the results against what the app really loads: the React client pins react-scripts 5.0.1 and a pre-release range of the Feathers Socket.io client, and public/index.html pulls the Feathers client from a CDN by version range with no integrity attribute. In .github/workflows it notes actions/checkout@v2 and actions/setup-node@v1, a monthly update job on Node 15.x that pushes a branch and opens a pull request with a stored token, and no permissions block. Every finding gets a severity, the evidence that confirmed it, and a fix; what a tool flagged but the agent could not reproduce is listed as unconfirmed.
Start it with one API call
POST https://api.phoenix.so/o/{org}/sessions with this body:
{
"agent": "claude",
"input": "Audit this Feathers 5 application and write security-audit.md. Change no file in the repository. The server is in feathers-chat-ts: run npm install and npm test there, then start it on the test port. Read src/app.ts, src/authentication.ts, src/channels.ts, src/validators.ts, every file under src/services and src/hooks, the config directory, the migrations, public/index.html, public/client.js and .github/workflows. Section 1: a table of every service method exposed to clients and the hooks that run for it. Section 2: findings from the schemas, resolvers and client, each confirmed against the running app where possible: register a user whose email contains markup and load the page in the browser, patch userId on your own message and githubId on your own user, read the server output for anything logged from params.user, and check what another signed-in user can read. Section 3: configuration, including the committed authentication secret, the JWT options, CORS for REST and Socket.io, and OAuth settings; never paste a secret value. Section 4: npm audit for feathers-chat-ts, react-chat and quick-start, and the scripts loaded from CDNs. Section 5: the GitHub workflows. For each finding give the file and line, what you ran or read to confirm it, a severity and a fix, and mark tool findings you could not reproduce as unconfirmed. Finish with git status to show the tree is clean.",
"resources": [
{
"url": "https://github.com/your-org/your-app"
}
]
}- Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
- Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
- Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.
What comes back
security-audit.md comes back as an artifact on the session, with a name, a mime_type and a url that downloads it without a key until url_expires_at; reading the session again gives a fresh link. The repository gets no branch and no commit, and the transcript ends with a clean git status.
The report has the five sections the prompt names. The method-and-hooks table comes first so a reader can see the app's whole surface on one screen. Each finding after it cites a file and line and the request or command that confirmed it, with the response the agent received, so a claim such as markup in an email reaching the page is either shown happening or listed as not reproduced. Secrets are referred to by file and key, never by value.
A reviewer works through it by opening the cited lines, starting with anything confirmed against the running app. Some entries are questions more than defects: whether every signed-in user should see every email, whether one channel for all events suits your app. The fixes are suggestions and none is applied; a follow-up message on the same session turns an accepted finding into a pull request with a test.
Run it on every release
From your release script, POST this body to POST /o/{org}/sessions with the repository in resources and a request_id that is a UUID v5 of the release tag, so a rerun of the pipeline returns the audit already started and does not begin another. Give the key a webhook_url and Rush posts run.completed when the turn finishes; the report is a file on the session, so the handler reads data.session_url with the key and downloads the artifact from there.
For a regular check between releases, add an automation to the Project in the console with the same prompt, a schedule trigger and Read only switched on. Its runs can read the repository but cannot push, which suits an audit, and each appears in the Project's run list with its report.
The use case this job follows, with its automation and trigger: Produce a report or file, no repo.
Why run it on Rush?
Most of what goes wrong in a Feathers app is behaviour: a hook that does not run for one method, a schema that accepts a field it should not, a client that trusts a string. Reading finds the suspects; only a request proves them. The agent has its own sandbox with a computer and a browser, so it installs the app, starts it, signs up as two users and tries each thing it suspects, including loading the chat page to see what a stored value does there. Each command, request and response is kept with the run.
That is the difference between this report and a scanner's list. Findings arrive already sorted into confirmed and unconfirmed, with the evidence attached, and the code is untouched, so the audit can run against a release branch without anyone reviewing a diff. Auditing several services is several runs at once, each isolated from the others.
Frequently asked questions
Does the audit change my code or open a pull request?
No. The prompt forbids changes, and the report is the only output. If you run it from an automation with Read only on, the run could not push even if it tried. Fixes happen in a follow-up you start.
Will the report contain my JWT secret or other credentials?
No. The agent names the file and the key, such as authentication.secret in config/default.json, and says whether an environment variable overrides it. The prompt tells it never to paste a value. Read the report once yourself before sharing it widely, as you would any audit.
How does it confirm a cross-site scripting finding?
By doing it in the sandbox. It registers a user through the running app with markup in a field the client renders, opens the page in the sandbox's browser as another user and records what the page does. Nothing leaves the sandbox, and the test user exists only in the sandbox's database file.
Is npm audit not enough on its own?
It covers one section of five. npm audit knows about published advisories in your dependency tree; it cannot see that a service method lacks a hook or that a patch schema accepts userId. The agent runs it per package directory and then says which advisories touch code the app actually runs.
Can it audit only what changed since the last release?
Yes. Name the two tags in the prompt and ask for the services, hooks, schemas and configuration that differ between them. The method-and-hooks table is still produced in full, since a change in one hook list can alter the meaning of another.