What the agent does
The agent starts from the pull request diff, not a general instruction to increase coverage. In an application like hexpm/hexpm, lib/hexpm_web/controllers/api/package_controller.ex handles package listing and reads, while lib/hexpm_web/controllers/api/release_controller.ex handles release creation and publishing. It maps changed branches to the corresponding ExUnit files before adding anything. The task is to find behavior introduced or changed by this particular PR that existing assertions do not protect.
It reads test/hexpm_web/controllers/api/package_controller_test.exs and release_controller_test.exs for the established request and database setup. The package tests use HexpmWeb.ConnCase, insert fixtures and assert JSON responses. Release tests submit a tarball with create_tar, inspect the persisted Package and AuditLog records, and drain the registry queue when checking storage effects. The agent extends those real flows instead of replacing the controller or repository layer with a stub that cannot detect a broken publish path.
Environment preparation follows mix.exs and config/test.exs. The project declares Elixir ~> 1.20, and the mix test alias creates the database, loads its structure and applies migrations before invoking the tests. The test database uses Ecto.Adapters.SQL.Sandbox; configured storage buckets use Hexpm.Store.Memory. The agent runs against that disposable test environment and reports its boundaries. A controller test using the configured memory store is not evidence of a successful publish to the public package registry.
The agent adds focused assertions for the actual diff: for example, a changed authorization branch may need a refusal case, while a publishing change may need a database or audit assertion beyond the response code. It runs the affected ExUnit files first and then mix test, preserving application code. If a test exposes a defect in the proposed behavior, it reports the failure rather than editing the implementation or weakening the expectation. Passing additions are committed to the authorized PR branch with a summary of what each covers.
Start it with one API call
POST https://api.prix.dev/o/{org}/sessions with this body:
{
"agent": "codex",
"input": "For the pull request identified in this task, read the diff in the package publish and API controllers and compare it with the existing ExUnit coverage. Add only missing tests for new or changed behavior, beside the existing controller tests. Exercise the real request and database paths using the repository's test setup; do not mock our application modules or change application code. Run the affected test files and then mix test. Push verified test additions to this pull request's branch and comment with the behavior each test covers and the exact checks run. If an implementation defect or environment failure blocks verification, report it without claiming the tests pass.",
"resources": [
{
"url": "https://github.com/your-org/your-app"
}
]
}- Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
- Send the request. POST the body in request.json to https://api.prix.dev/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
- Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.
What comes back
The existing pull request receives test commits rather than a competing implementation PR. The additions identify the changed behavior in their names and assert observable responses or persisted effects. The summary connects each new case to a line of changed behavior, names the focused files run and records the full-suite result separately. Any failure caused by application code remains visible for the PR author.
A reviewer checks that the new tests would detect the regression they claim to cover and that the diff contains no production edits. For release behavior, they look beyond a successful response to the relevant ownership, persistence or audit effect. The Rush session read provides the final answer, and its transcript provides the tool output behind the check summary. Neither substitutes for required repository CI checks on the final branch.
Run it on every opened Phoenix pull request
Your GitHub webhook handler can turn pull_request.opened into a sessions API call after validating the event and confirming the repository and branch are allowed. Replace the generic resource URL with your PR's GitHub URL; Rush accepts an open PR URL and clones its branch. Include that PR's identity in the prompt, authorize same-branch test commits explicitly, and avoid dispatching another run for the test commits the agent itself adds.
The use case this job follows, with its automation and trigger: Write the missing tests.
Why run it on Rush?
Package publishing tests combine an Elixir toolchain, database transactions, tarball fixtures and queued side effects. Rush runs that investigation in its own sandbox, leaving the developer's active database and checkout alone. The agent can inspect the failing request and rerun a narrow ExUnit file while the PR author continues other work.
An API-driven run also attaches the test-writing work to the incoming change while its intent is still visible. Separate pull requests can be examined in separate sessions within your concurrency allowance. The retained transcript makes it possible to distinguish a focused controller run from a complete mix test run instead of receiving an unexplained statement that coverage improved.
Frequently asked questions
Will this change the application to make its tests pass?
No, the prompt limits edits to tests and necessary test support. If the changed implementation violates the expected behavior, the agent reports that failure to the PR author. A production fix is a separate decision rather than an unannounced part of test generation.
Does it publish a package to test publishing?
The inspected release tests exercise local application requests, a test database and configured test storage. They do not require publishing to the public registry. The agent should preserve that boundary and describe exactly which local side effects its assertions verify.
How does it know which tests are missing?
It compares the PR's changed branches with existing test setup and assertions. The package and release controllers already have substantial tests, so a missing spec file is not the criterion. Each addition needs a behavior from the diff that the existing suite does not already check.
Can it update an existing PR instead of opening another?
Yes, pass the accessible open PR as the resource and explicitly request test commits on its branch. The Rush GitHub App must cover the repository. Your webhook handler should reject untrusted or unauthorized write targets before starting a run that can push changes.