What the agent does
The agent reads what exists before it adds anything. In miguelgrinberg/microblog there is one test module, tests.py, with a UserModelCase of four unittest tests for password hashing, avatars, following and the followed-posts query, and a TestConfig that sets SQLALCHEMY_DATABASE_URI to sqlite:// and ELASTICSEARCH_URL to None. requirements.txt pins Flask 3.0.0, Flask-SQLAlchemy 3.1.1, SQLAlchemy 2.0.23, Flask-Login, Flask-WTF, Flask-HTTPAuth and Flask-Babel, and no pytest. app/__init__.py is an application factory that registers four blueprints: auth at /auth, main, api at /api and errors, and it also creates a Redis connection and an RQ queue from REDIS_URL, which the tests must not touch.
It creates tests/ with a conftest.py that builds the app from create_app(TestConfig) with TESTING, an in-memory SQLite URI, WTF_CSRF_ENABLED = False so forms can be posted, and ELASTICSEARCH_URL = None so SearchableMixin skips indexing; a db fixture that calls create_all and drop_all around each test; a client fixture from app.test_client(); and helpers that register a user and log in through the real /auth/login form rather than by poking the session. The four existing tests move into tests/test_models.py as plain functions with the same assertions, and tests.py is removed in the same commit so there is one way to run the suite.
Then it writes the blueprint tests. For auth, tests/test_auth.py posts RegistrationForm fields to /auth/register, checks that a duplicate username is refused by the form validator, logs in with remember_me, logs out, and requests a password reset, which app/auth/email.py sends through Flask-Mail, so the test records the outbox instead of connecting to a mail server. For main, tests/test_posts.py posts the PostForm to /index, checks the post on /explore, and covers follow and unfollow, including the You cannot follow yourself! flash and the redirect for an unknown username. For the API, tests/test_api.py creates a user with POST /api/users, gets a token from POST /api/tokens with basic auth, reads /api/users/<id>, /followers and /following with the bearer token, checks the 403 when PUT /api/users/<id> targets another user, and revokes the token with DELETE /api/tokens.
The suite runs with python -m pytest --cov=app --cov-report=term-missing after adding pytest and pytest-cov to a new requirements-dev.txt, and the agent records coverage once with only the moved model tests and once with the full suite. It keeps app/tasks.py, app/translate.py and app/search.py out of scope, since they need RQ, the Microsoft Translator key and Elasticsearch, and says so in the pull request rather than mocking them into a passing number.
Start it with one API call
POST https://api.prix.dev/o/{org}/sessions with this body:
{
"agent": "claude",
"input": "Write pytest tests for the auth, main and api blueprints of this Flask app. Create tests/ with a conftest.py that builds the app through create_app with a TestConfig using an in-memory SQLite database, CSRF disabled and ELASTICSEARCH_URL set to None, plus client and database fixtures and a helper that logs in through /auth/login. Move the four unittest tests from tests.py into tests/test_models.py and delete tests.py. Cover registration, login, logout and the password reset request; posting, explore, follow and unfollow including the self-follow flash and the unknown-user redirect; and the token API: POST /api/users, POST and DELETE /api/tokens, the users, followers and following endpoints and the 403 on updating another user. Use the real test database, not mocks, record the outbox for mail instead of sending it, and leave the RQ, translation and Elasticsearch modules out of scope. Add pytest and pytest-cov to requirements-dev.txt, run python -m pytest --cov=app --cov-report=term-missing, and open a pull request with the coverage before and after.",
"resources": [
{
"url": "https://github.com/your-org/your-app"
}
]
}- Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
- Send the request. POST the body in request.json to https://api.prix.dev/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
- Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.
What comes back
A pull request on its own branch that adds tests/conftest.py, tests/test_models.py, tests/test_auth.py, tests/test_posts.py and tests/test_api.py, a requirements-dev.txt with pytest and pytest-cov, and removes tests.py. The description shows the --cov-report=term-missing table before and after, names the lines in app/auth/routes.py, app/main/routes.py and app/api/users.py the new tests reach, and lists the three modules left out and why.
The session keeps every pytest run with its output and the files the agent read, so GET /o/{org}/sessions/{id}/messages shows how each fixture was arrived at. Production code is unchanged unless a test found a bug, in which case the description names it and the fix is in its own commit on the same branch.
A reviewer checks that the login helper goes through the real form and not login_user directly, that no test depends on another's rows surviving the drop_all, and that the API tests use the token flow in app/api/auth.py rather than a forged header.
Run it on every Sunday night
Create an automation in the Project with a schedule trigger, a cron such as 0 2 * * 0 with your timezone, and a prompt that asks for tests on the least-covered blueprint changed that week. Each run opens one pull request on its own branch, so Monday starts with a suite to review rather than a coverage number to feel bad about.
Set a webhook URL on the API key and Rush POSTs run.completed when the turn finishes, with the session_url and the result, so a small handler can post the pull request link and the coverage line into your team channel.
The use case this job follows, with its automation and trigger: Write the missing tests.
Why run it on Rush?
Moving a tutorial-grade app from one unittest file to a pytest suite is the kind of work that waits forever on a laptop: it needs a clean database, a way to post forms without CSRF, an outbox for mail and a decision about Redis before the first assertion. The agent makes those choices in its own sandbox, runs the suite there with coverage, and leaves your checkout untouched.
What comes back is a pull request with the suite's output and a transcript of every run, so the coverage delta is something you can check rather than take on faith. Several Flask services can each get the same treatment in parallel, and a person still merges each one.
Frequently asked questions
The app creates a Redis connection at startup. Do the tests need Redis?
No. Redis.from_url in create_app does not connect until a command is sent, and only launch_task in app/models.py and the export_posts route enqueue work. The agent leaves those routes out of scope and says so, rather than mocking RQ to inflate the number.
How do the tests send mail?
Flask-Mail's record_messages captures the outbox when TESTING is on, so the password reset test asserts that one message went to the right address with a token in its body, without a mail server.
Will it keep the old tests.py?
No. Its four tests move to tests/test_models.py unchanged in meaning, and the file is deleted in the same pull request, so python -m pytest is the one command to run. If you want both for a while, say so in the prompt.
Does it test the Elasticsearch search page?
Not against Elasticsearch. With ELASTICSEARCH_URL set to None, query_index in app/search.py returns no ids, so /search?q=... renders an empty results page and an empty query redirects to explore; the tests cover both paths. Real indexing needs an Elasticsearch service and is left out with a note in the description.