What the agent does
The agent checks out the pull request in its own sandbox and learns what the repository already enforces, so the review adds to CI instead of repeating it. In a plugin built from obsidianmd/obsidian-sample-plugin, package.json defines build as tsc -noEmit -skipLibCheck && node esbuild.config.mjs production and lint as eslint .; eslint.config.mts spreads obsidianmd.configs.recommended from eslint-plugin-obsidianmd over a type-aware parser setup; tsconfig.json has strict and noUncheckedIndexedAccess on and includes only src/**/*.ts; and .github/workflows/lint.yml runs npm ci, the build and the lint on Node 20, 22 and 24 for every push and pull request. esbuild.config.mjs bundles src/main.ts to a CommonJS main.js and marks obsidian, electron, the CodeMirror and Lezer packages and every Node built-in as external. manifest.json carries id, version, minAppVersion and isDesktopOnly: false, and versions.json maps each plugin version to the app version it needs.
It runs the same three commands on the base commit and on the head and reports the difference: new type errors, lint findings that the head introduces with their rule names, and the size of the production main.js before and after, which is the quickest sign that a dependency was bundled in. The lint rules are worth quoting by name because they encode the plugin guidelines: obsidianmd/no-unsupported-api flags an API newer than the manifest's minAppVersion, obsidianmd/no-nodejs-modules flags a Node built-in import that is not guarded by Platform.isDesktop, obsidianmd/prefer-active-doc and obsidianmd/prefer-window-timers cover popout windows, obsidianmd/regex-lookbehind covers iOS, and obsidianmd/commands/no-default-hotkeys and obsidianmd/ui/sentence-case cover commands and interface text. A pull request that silences one with a disable comment gets that line cited.
Then it reads the diff for what no command reports. Because Node built-ins and electron are external, an unguarded fs import builds without error and only fails on a phone, so with isDesktopOnly false the agent traces each such import to its guard. It checks that every listener, DOM event and interval goes through registerEvent, registerDomEvent or registerInterval, as src/main.ts does, so disabling the plugin cleans up. It checks that a new field on the settings interface in src/settings.ts has a default in DEFAULT_SETTINGS, since loadSettings merges saved data over the defaults with a shallow Object.assign and a nested object would lose its defaults. It checks that existing command ids are unchanged, that the manifest id is untouched, that any network call is disclosed in the README as the repository's AGENTS.md asks, and that main.js is not committed.
Version changes get their own pass. version-bump.mjs writes npm_package_version into manifest.json and adds that version to versions.json with the current minAppVersion, and .github/workflows/release.yml publishes a draft release for any pushed tag with main.js, manifest.json and styles.css. So when a pull request touches a version, the agent confirms the three files agree and that a raised minAppVersion matches the newest API the diff uses. It posts one review with a verdict, findings by file and line, and the command output, and it states that the plugin was built and linted, not loaded in Obsidian.
Start it with one API call
POST https://api.phoenix.so/o/{org}/sessions with this body:
{
"agent": "claude",
"input": "Review the pull request this run was started for, on an Obsidian community plugin. Read the diff and every file it touches, plus package.json, eslint.config.mts, tsconfig.json, esbuild.config.mjs, manifest.json, versions.json, version-bump.mjs, AGENTS.md and the workflows under .github/workflows. In your sandbox run npm ci, npm run build and npm run lint on the base commit and on the head, and report new type errors, lint findings the head introduces with their rule names, any eslint-disable comment the diff adds, and the size of the production main.js before and after. Read the diff for what those commands miss: Node or Electron imports that are not guarded while isDesktopOnly is false, listeners and intervals not registered through the plugin's register helpers, settings fields without a default, renamed command ids, a changed manifest id, undisclosed network requests, and committed build output. If a version changed, check that package.json, manifest.json and versions.json agree and that minAppVersion covers the newest API used. Post one review with a verdict, findings citing file and line, and the command output. Push nothing, and say that the plugin was not loaded in Obsidian.",
"resources": [
{
"url": "https://github.com/your-org/your-app"
}
]
}- Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
- Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
- Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.
What comes back
The result is a single review on the pull request. It opens with a verdict, then lists findings, each tied to a file and line and marked as blocking or a suggestion: the lint rule or compiler message where a command found it, the reasoning where the agent did. Below that sit the build and lint results for base and head, the main.js size on both, and a short table of package.json, manifest.json and versions.json values when a version moved.
The branch is untouched, and the full command logs stay in the run's record. A maintainer reads the blocking findings, decides which suggestions to pass on to the contributor, and still loads the build in a test vault before merging, because nothing in the review replaces seeing the plugin run on desktop and on a phone.
Run it on every pull request opened
Create an automation in the Project that holds the plugin's repository, with this prompt and a github trigger on pull_request.opened and pull_request.synchronize. Turn read_only on so its runs comment and never push. The trigger puts the pull request event after the prompt, which is how the run knows which one to check out, and it fires only for work by an owner, member or collaborator; a pull request from a fork never fires it.
To drive it from your own server instead, receive GitHub's pull_request webhook, put the pull request URL in resources so the clone is that branch, and send this body to POST /o/{org}/sessions with a request_id that is a UUID v5 of the delivery id, so a redelivered event returns the run already started.
The use case this job follows, with its automation and trigger: Review every pull request.
Why run it on Rush?
Most plugin maintainers review alone, and the mistakes that reach users are the ones that pass CI: an import that only breaks on mobile, a listener that survives a reload, a version file left behind. The agent runs the repository's own build and lint on both sides of the diff in its sandbox and keeps every line of output, so the review cites a rule or an error message where one exists and marks the rest as reasoning.
It also scales with attention, not with time. Reviews for several open pull requests run side by side, each in its own sandbox with its own node_modules, and a contributor who pushes a fix gets a fresh review of the new head without the maintainer checking out a branch.
Frequently asked questions
CI already runs the build and lint. What does the review add?
CI reports pass or fail on the head. The review compares base and head, names the rules the change newly violates, cites disable comments the diff adds, and reads for problems no rule covers, such as a settings field with no default or a renamed command id. It also measures the bundle on both sides.
Can the agent test the plugin inside Obsidian?
The review does not claim to. It builds main.js and reasons from the code and the type definitions, and it says so in the review. Loading the build in a vault on desktop and mobile stays with the maintainer.
Does it work on pull requests from forks?
A github trigger does not fire for pull requests from forks, so the automation covers branches pushed to the repository itself. For an outside contribution, a maintainer can push the contributor's commits to a branch in the repository and open the pull request from there, which starts the review like any other.
Will it flag the sample plugin's own placeholder code?
The lint plugin ships rules for that, obsidianmd/sample-names and obsidianmd/no-sample-code, and the review reports them only when the diff introduces the finding. Findings that already exist on the base commit are counted once in the summary and left out of the line comments.
More Obsidian plugin jobs
The same job on other stacks
- Review Pull Requests That Change Passport.js Login Code (Passport.js)
- Review Pull Requests on a Payload CMS App (Payload CMS)
- Review Pull Requests Automatically in a Phoenix App (Phoenix)