All jobs

Write Tests for Oban Workers in a Phoenix App

Yes. An agent finds the Oban workers with no test, writes ExUnit tests with perform_job and assert_enqueued against a real Postgres, and opens a PR.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

The agent starts by counting. In a Phoenix app like thechangelog/changelog.com, lib/changelog/oban_workers holds 17 workers and test/changelog/oban_workers holds tests for two of them, EpisodeStatsProcessor and FeedStatsProcessor. A few more are reached indirectly: test/changelog/notifier_test.exs drains the :email queue with Oban.drain_queue and so runs MailDeliverer for some mailers. The agent lists every worker, its queue and options from use Oban.Worker, what it calls outside the app, and whether any test reaches it, and picks the order: workers with branches and side effects first, cron-only wrappers last.

It follows the patterns the repository already uses. config/test.exs sets config :changelog, Oban, testing: :manual, so jobs are only inserted, never run, unless a test runs them. The existing worker tests use Changelog.SchemaCase and use Oban.Testing, repo: Changelog.Repo, build records with the ExMachina factory in lib/changelog/schema/factory.ex (insert(:podcast), insert(:feed), insert(:news_item_comment)) and replace outside modules with setup_with_mocks from the Mock library. Mail uses Swoosh.Adapters.Test and the helpers in test/support/email_case.ex. The agent adds no new test library.

Then it writes one file per worker. For MailDeliverer it calls perform_job(MailDeliverer, %{"mailer" => "sign_in", "person" => person.id}) and asserts the email was sent, and adds a test that an unknown mailer name fails, since String.to_existing_atom/1 raises on it. For FeedBouncer it inserts a plusplus feed owned by someone who is no longer a member, runs the job and asserts the feed is switched off and a FeedUpdater job is enqueued for it with assert_enqueued. FeedUpdater.queue/1 gets a test per podcast clause, asserting the exact slugs enqueued for news or friends. CommentNotifier.schedule/1 gets a test that the job is scheduled five minutes out. OvercastPinger and SocialPoster mock Changelog.PodPing.overcast/1 and the Bluesky, Mastodon and Zulip modules and assert which are called.

Some behaviour the agent finds is worth a test that documents it rather than one that changes it. OvercastPinger.perform/1 matches a 200 and a 429 only, so any other status raises a CaseClauseError and Oban retries the job; SocialPoster returns :ok even when a network call returns {:error, _}. The agent pins both with tests and lists them in the pull request. To run the suite it installs the Erlang and Elixir versions mise.toml pins, installs PostgreSQL with the postgres user and password config/test.exs expects, and runs mix test, whose alias in mix.exs creates and migrates changelog_test first.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "claude",
  "input": "Write ExUnit tests for the Oban workers in lib/changelog/oban_workers that have none. Read config/test.exs, mix.exs, test/support, lib/changelog/schema/factory.ex, the two existing worker tests and test/changelog/notifier_test.exs first and follow their patterns: Oban.Testing with perform_job and assert_enqueued, the ExMachina factory, Mock for modules that call outside services, Swoosh test assertions for mail. One test file per worker. Where a worker's current behaviour looks wrong, pin it with a test and list it in the pull request instead of changing it. Run mix test and open a pull request with the output.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

What comes back is a pull request that adds test files under test/changelog/oban_workers/, one per worker it covers, and changes nothing under lib/. The description has a table of all 17 workers: covered before, covered now, or left out with the reason, such as AudioUpdater, whose real work is downloading and tagging an mp3 from storage, which the agent covers only for an episode with no audio file.

It includes the mix test output from the sandbox and a short list of behaviour the tests pin: the unmatched status in OvercastPinger, the ignored error tuples in SocialPoster and MailDeliverer, and the max_attempts: 1 on MailDeliverer that means a failed send is never retried. Each is phrased as a question for the maintainers, with the test that would change if they decide to fix it.

A reviewer checks that the mocks replace only modules that call outside services (Bluesky, Zulip, Stripe, Overcast, Campaign Monitor) and never the app's own Repo or schema modules, and that the tests do not depend on order, since SchemaCase runs them in a shared sandbox when they are not async.

Run it on every pull request labelled needs-tests

Once the gap is closed, the agent is for the next worker someone adds without a test. An automation in the Project with a github trigger on pull_request.labeled and a label such as needs-tests adds the pull request after the prompt, so the prompt says to read the workers the pull request that follows adds or changes, write their tests in the same style and push them to its branch.

From your own code, the same request is one POST /o/{org}/sessions with the repository in resources, and the key's webhook URL receives run.completed with the session_url when the turn ends.

The use case this job follows, with its automation and trigger: Write the missing tests.

Why run it on Rush?

Oban tests need a real Postgres, because Oban stores jobs in the oban_jobs table and assert_enqueued reads it, plus the Erlang and Elixir versions the project pins. Setting that up on a laptop for a test-writing session is often the reason the tests never get written. The sandbox installs both, runs mix test against the real database, and keeps every command and its output in the run's record.

Fifteen workers with no test file of their own is a backlog no one picks up in an afternoon, and it is reading-heavy: each worker touches a different part of the schema and a different outside service. One run works through them in the repository's own style and returns a single pull request whose description says exactly what each test asserts and which behaviour looked wrong.

Frequently asked questions

What does perform_job check that calling perform directly does not?

perform_job/3 builds a real Oban.Job, round-trips the args through JSON the way Oban stores them, and checks the worker's options before calling perform/1. A test that passes atom keys or a struct where Oban would store strings fails there, as it would in production.

Will the tests send real email or post to Bluesky?

No. Mail goes to Swoosh.Adapters.Test, set in config/test.exs, and every module that calls an outside service is replaced with Mock inside the test. No credentials are needed to run the suite.

Why does testing: :manual matter?

With :manual, inserting a job does not run it, so a test can assert what was enqueued and then choose to run it with perform_job or Oban.drain_queue. Without it, jobs would run in the background during tests and their effects would be hard to assert.

Does it test the cron schedule in config/prod.exs?

It tests each cron worker's perform/1, not the crontab entries, which are configuration Oban's Cron plugin reads at startup. The pull request lists the schedule next to the workers so a reviewer can see which ones run every minute.

More Oban jobs

More jobs like this: Write the missing tests

Keep reading