# Write Unit Tests for an Elixir Project

Yes. An agent runs mix test --cover on the Elixir project, picks the modules the report leaves thin, writes ExUnit tests with the project's own cases and opens a PR.

[Run this on Rush](https://cloud.getrush.ai/run/write-unit-tests/elixir/), in a Project that has your repository.

## What the agent does

The agent starts by finding out where the suite is thin, which in a project like hexpm/hexpm is a real question: `test/` mirrors `lib/` closely, with `DataCase`, `ConnCase` and a `ConcurrencyCase` in `test/support/`, a factory, StreamData properties and Mox mocks for billing, geolocation, HTTP, the CDN, the store and SSO. `mix.exs` requires Elixir ~> 1.20, compiles `test/support` only in the test env, and defines a `test` alias that runs `ecto.create --quiet`, `ecto.load --skip-if-loaded` and `ecto.migrate` before `test`. There is no `test_coverage` configuration, so the agent runs `mix test --cover --export-coverage default` followed by `mix test.coverage`, which prints coverage per module, and sorts that list rather than guessing from file names.

It then compares the per-module numbers with the modules that have no test file of their own. In this repository those include `Hexpm.VersionRequirement` in `lib/hexpm/ecto/version_requirement.ex`, an Ecto type whose `cast`, `load`, `dump` and `encode` clauses are reached only through schemas; `Hexpm.Accounts.TFA`, whose `generate_secret`, `time_based_token` and `token_valid?` are exercised through controller tests in `test/hexpm_web/controllers/tfa_auth_controller_test.exs`; `Hexpm.BlockAddress`, whose `blocked?`, `allowed?` and `try_reload` are covered through the attack plug test; `Hexpm.Accounts.ReservedUsername.by_name`; and `Hexpm.Emails.OutboxReconciler`, an Oban worker whose `perform` discards, purges and requeues outbox entries and is touched only by `outbox_worker_test.exs`. Indirect coverage leaves branches out, and the coverage listing shows which.

The agent writes direct tests for the five modules with the lowest numbers that hold logic. For `VersionRequirement` it is a plain `ExUnit.Case` with `async: true`: each `cast` clause, a bad string, `load` and `dump` round-trips and `embed_as`. For `TFA` it checks that `token_valid?` accepts the token `time_based_token` produces from a `generate_secret` and rejects a neighbouring one. For `BlockAddress` it uses `DataCase`, inserts entries through the factory, calls `reload` and asserts `blocked?` and `allowed?` on an IP inside and outside a mask. For `OutboxReconciler` it uses `DataCase` with Oban in `testing: :manual` from `config/test.exs`, inserts outbox rows in each state the worker handles, runs `perform/1` and asserts what was discarded, purged and requeued. Mocks come from `test/support/mocks.ex` with `Mox` expectations, never by stubbing the module under test.

It runs the suite the way `.github/workflows/main.yml` does: Elixir 1.20.4 on OTP 29.1 against a `postgres:18` service with `MIX_ENV=test`, `mix deps.get`, `mix compile` and `mix test`, plus `mix format --check-formatted` from the format job and `mix compile --warnings-as-errors`, since a new test module with an unused alias would fail that job. `config/test.exs` already points the repo at `hexpm_test` with the SQL sandbox, uses in-memory stores for every bucket and the Swoosh test mailer, so nothing external is needed. It records `mix test.coverage` before and after and puts both module listings in the pull request.

## Start it with one API call

`POST https://api.prix.dev/o/{org}/sessions`

```json
{
  "agent": "claude",
  "input": "Run the test suite of this Elixir project with coverage: mix test --cover --export-coverage default, then mix test.coverage, against a Postgres the way .github/workflows/main.yml does with MIX_ENV=test. Cross the per-module listing with the modules under lib/ that have no test file of their own, and pick the five least-covered that hold real logic. Write ExUnit tests for them using the project's own cases from test/support (DataCase, ConnCase), the factory and the Mox mocks in test/support/mocks.ex; test behaviour through public functions, not implementation details, and never stub the module under test. Do not change production code unless a test reveals a bug; if it does, fix it in a separate commit and say so. Run mix test, mix format --check-formatted and mix compile --warnings-as-errors, and open a pull request with the per-module coverage before and after.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
```

1. **Get an API key.** In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
2. **Send the request.** POST the body in request.json to https://api.prix.dev/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
3. **Read what came back.** Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

## What comes back

A pull request on its own branch that adds test files beside the modules it chose, such as `test/hexpm/ecto/version_requirement_test.exs`, `test/hexpm/accounts/tfa_test.exs`, `test/hexpm/block_address_test.exs` and `test/hexpm/emails/outbox_reconciler_test.exs`, and changes nothing under `lib/` unless a test found a bug. The description pastes the `mix test.coverage` lines for those modules before and after, says why each was chosen over others with lower numbers, and shows `mix test`, `mix format --check-formatted` and the warnings-as-errors compile passing.

If a test uncovered a defect, such as a `cast` clause accepting a requirement string that `Version.parse_requirement` rejects, the fix is its own commit and the description explains it. The session transcript keeps every `mix` command with its output, including both coverage listings, and `GET /o/{org}/sessions/{id}/messages` returns it.

A reviewer checks that tests marked `async: true` touch no database, that Oban tests use the manual testing mode rather than starting queues, and that Mox expectations are set with `expect` for calls the code must make and `stub` only for ones it may make.

## Run it on every nightly

Create an automation in the Project with a `schedule` trigger, a `cron` such as `0 2 * * *` with your `timezone`, and a prompt asking for tests on the least-covered module changed in the last day. Each night opens one pull request on its own branch with its coverage listing, so the suite follows the code instead of catching up once a quarter.

Set a webhook URL on the API key and Rush POSTs `run.completed` when the turn finishes, with the `session_url` and the result, so a handler can post the pull request link and the module numbers where the team will read them in the morning.

## Why run it on Rush?

In a large Phoenix application the untested code is not obvious: most modules are reached by some test, and only a per-module coverage listing shows which branches no test asserts. The agent produces that listing in its own sandbox, with the Postgres version and Elixir and OTP pair the workflow pins, and writes the tests without a single compile on your machine.

The pull request carries the before-and-after listings, the format and warnings checks and a transcript of every command, so the delta is a fact to check rather than a claim to trust. Several Elixir services can each get a run at once, and a person still merges each one.

## Frequently asked questions

### The suite already has hundreds of test files. What is left?

Modules covered only through controllers or plugs: the `VersionRequirement` Ecto type, `TFA`, `BlockAddress`, `ReservedUsername` and the `OutboxReconciler` worker have no test file of their own in this repository, and `mix test.coverage` shows the branches those indirect tests miss. The agent chooses from that list, not from file names.

### Does it need Postgres, S3 and the other services?

Only Postgres. `config/test.exs` replaces every bucket with `Hexpm.Store.Memory`, uses the Swoosh test mailer, puts Oban in manual testing mode and routes billing, geo, HTTP and CDN through Mox mocks, which is what lets the suite run in CI with a single `postgres:18` service.

### Will it use Mox to stub the module it is testing?

No. Mocks are for the behaviours in `test/support/mocks.ex` that the module calls, with `expect` for required calls. Stubbing the module under test would test nothing, and the prompt forbids it.

### Which Elixir and OTP does it run?

Elixir 1.20.4 on OTP 29.1, the pair in `main.yml`, so a test that passes in the sandbox passes in the `Test Hexpm` job. `mix.exs` requires Elixir ~> 1.20, and the agent says in the pull request which versions it used.