All jobs

Write pytest Tests for a discord.py Bot

Yes. The agent tests your bot's parsers, checks, caches and migration logic with pytest and fakes, with no bot token and no connection to Discord.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

Most of a Discord bot that can be tested never touches Discord, and the agent starts by finding that part. In a bot laid out like Rapptz/RoboDanny there is no test directory, no pytest configuration and no workflow. requirements.txt lists discord.py[speed]>=2.3.1, asyncpg, parsedatetime, python-dateutil, lru_dict, click and a few more with no lock file, and pyproject.toml configures only black, isort and pyright. The cogs under cogs/ talk to Discord and PostgreSQL. The modules under cogs/utils/ mostly do not: time parsing, text formatting, fuzzy matching, a cache, a queue and a JSON-backed config store. launcher.py holds a Migrations class that reads V<n>__<description>.sql files. Those are where tests pay first.

It then solves the import problem, because that is what stops most people. bot.py and launcher.py both run import config, and config.py is in .gitignore; the README gives a template for it. Importing either module in a test fails until a config module exists. The agent adds a tests/conftest.py that puts a stand-in module with the README's fields into sys.modules before anything imports it, and writes no real config.py. Modules such as cogs/utils/time.py import discord at the top but need no client, so they import as they are once the requirements are installed.

The tests pin behaviour the code has today. For cogs/utils/formats.py: plural renders 1 vote and 3 votes, takes an explicit plural after a |, and drops the number when the format spec ends in !; human_join uses or unless told otherwise; TabularData.render centres each cell. For cogs/utils/time.py: ShortTime('2h30m') adds to the now it is given, a two-digit year count such as 10y is refused because the pattern takes a single digit, a Discord timestamp is accepted, FutureTime raises BadArgument for a past time, and human_timedelta splits weeks out of days and keeps three units unless accuracy says otherwise. For launcher.py: the revision file pattern, is_next_revision_taken, and create_revision writing its header into a directory under tmp_path.

Async code and the code nearest Discord get fakes, not a gateway. Config calls asyncio.get_running_loop() in its constructor and writes beside the working directory, so its tests are async and run inside tmp_path. CancellableQueue, the cache decorator and the permission predicates in cogs/utils/checks.py are driven with small stand-in objects for the context, author and channel. The agent adds pytest and pytest-asyncio to a new requirements-dev.txt and leaves requirements.txt alone. Where a test shows behaviour that looks unintended, it keeps the test and reports it. Two examples it would find here: ExpiringCache.get returns a value after its time to live has passed, while in and indexing both evict it first; and Migrations.create_revision, called before any revisions.json exists, writes the SQL file and then raises a TypeError while saving, because the default database_uri is discord.py's MISSING sentinel and JSON cannot encode it.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "claude",
  "input": "Write pytest tests for this discord.py bot that pass with no bot token, no connection to Discord and no database. Read requirements.txt, pyproject.toml, README.md, .gitignore, bot.py, launcher.py and every module under cogs/utils first, and skim the cogs for how they use those utilities. Create a virtual environment, install requirements.txt, and add pytest and pytest-asyncio in a new requirements-dev.txt; do not edit requirements.txt. bot.py and launcher.py import a config module that is not in the repository: add a tests/conftest.py that registers a stand-in config module with the fields the README lists before those imports, and do not create config.py. Write tests under tests/ for: the formatting helpers (plural, human_join, TabularData, format_dt); the time parsers and human_timedelta, always passing an explicit now or source so no test depends on the clock; the fuzzy matching functions; ExpiringCache and the cache decorator including invalidate and invalidate_containing; CancellableQueue; Config with tmp_path as the working directory; the permission predicates in cogs/utils/checks.py with fake context objects, including the owner bypass; the prefix callable in bot.py for a direct message and a guild; and the Migrations class in launcher.py using tmp_path for the revisions directory and a fake connection for upgrade. Test what the code does today. If a test shows behaviour that looks wrong, keep the test, mark it clearly and list it in the pull request; do not change any file outside tests/ and requirements-dev.txt. Do not start the bot. Run pytest, then black --check and isort --check on tests/ with the settings in pyproject.toml. Open a pull request listing each test module and what it covers, the behaviours you flagged, any warnings pytest printed, and the output of each command.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

A pull request that adds a tests/ directory and requirements-dev.txt and changes nothing else. The description lists each test module with the functions it covers and pastes the output of pytest, black --check and isort --check. Because no test reads a token or opens a socket, the suite runs the same way on a contributor's machine and in whatever CI the project adds later.

A second list holds what the tests pinned that a maintainer may want to change, each with the test that demonstrates it: a cache lookup that can return an expired entry, a duration pattern that refuses a two-digit year count, a new revision that cannot be saved before the database has been initialised, and any deprecation warning pytest printed, such as the one newer Python versions raise for datetime.datetime.utcnow(), which launcher.py calls when it stamps a new revision. The agent reports these and fixes none of them.

A reviewer reads the fakes with the most care. A stand-in context that returns the wrong shape for permissions_for makes a permission test pass for the wrong reason, so the description states which attributes each fake provides and where the real code reads them. They also decide what comes next: the cogs themselves, which need PostgreSQL and a fake Discord client, are named as untested and are a larger, separate piece of work.

Run it on every merged pull request

An automation in the Project that holds the repository, with a github trigger on pull_request.closed.merged, starts one run each time a pull request merges. Its prompt asks for tests of whatever that pull request added or changed under cogs/utils/ and the other modules that import cleanly, in the style of the existing tests/. The trigger puts the merged pull request after the prompt and fills in nothing, and it fires only for a pull request an owner, member or collaborator wrote, so a merged fork starts no run. The run opens a pull request of its own with the tests, and when the merge touched only cogs that need Discord or a database, it says so and opens nothing.

For a team that runs several bots, a script on your server sends this body to POST /o/{org}/sessions once per repository with a request_id that is a UUID v5 of the repository's full name, so a retry starts nothing new. A webhook_url on the key receives run.completed with the session_url of each run that key started.

The use case this job follows, with its automation and trigger: Write the missing tests.

Why run it on Rush?

Getting a first test to import is the hard step in a bot repository, and it is all trial and error: a missing config module, a dependency that builds from a git URL, an event loop that has to be running before an object can be constructed. The agent works through that in its own sandbox, with a fresh virtual environment and nothing of yours installed in it, and the pull request arrives once the whole suite passes there.

Nothing in the run needs a secret. The tests call no Discord endpoint, the prompt forbids starting the bot, and no token is given to the session. The commands and their output are kept in the transcript at GET /o/{org}/sessions/{id}/messages, so a maintainer can confirm the suite passed in an environment that had no way to reach a real server.

Frequently asked questions

Does it need my bot token?

No. The tests cover code that runs without a gateway connection, and code near Discord is exercised with stand-in objects. The agent never starts the bot, and the stand-in config module it registers in tests/conftest.py holds placeholder values only.

Can it test the commands in my cogs, not only the utilities?

It can test the logic a command delegates to: converters, checks, formatters and anything else that takes plain values. A command that queries PostgreSQL and replies in a channel needs a database and a fake client. The agent names those as the next step and leaves them out of this pull request so the first suite stays small and reliable.

Which async test plugin does it use?

The repository has none, so the agent adds pytest-asyncio in requirements-dev.txt and configures it in the test setup. In a project that already uses another plugin, such as anyio, it reads the existing tests and follows them.

What if a test reveals a bug?

The test stays, written to show what the code does today, and the behaviour is listed in the pull request with the line it comes from. The agent does not change code outside tests/, because whether an odd behaviour is a bug or something users rely on is the maintainer's call.

More discord.py jobs

The same job on other stacks

Keep reading