All jobs

Add Response Caching to a Litestar App

Yes. An agent adds Litestar's response cache on a Redis store to the read endpoints of your app, clears it on writes, tests it with a memory store, and opens a PR.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

The agent reads how the app is assembled before it caches anything, because a cache in front of the wrong handler serves one user's data to another. In an app laid out like litestar-org/litestar-fullstack, uv.lock resolves Litestar to 2.19.0 and redis to 7.1.0, and pyproject.toml already depends on litestar-saq[hiredis], whose SaqSettings reads SAQ_REDIS_URL with a default of redis://localhost:16379/0. app/server/core.py is an InitPluginProtocol plugin whose on_app_init sets the OpenAPI config, runs auth.on_app_init, assigns app_config.cors_config = config.cors, and extends the plugin list with structlog, Granian, Advanced Alchemy, Vite, SAQ, problem details, OAuth2, email and the domain plugin. app/config.py also builds a CompressionConfig(backend="gzip") as config.compression, but nothing in core.py assigns it to app_config.compression_config, which the agent notes in the pull request without changing. The domains are accounts, admin, system, tags and teams. SystemController serves GET /health, which runs select 1 on the database each time and answers 500 when it fails, and GET /api/config/oauth, which returns whether Google and GitHub sign-in are enabled from settings. TagController at /api/tags lists tags with limit-offset pagination of 20, a search on name and sorting, and reads one by {tag_id:uuid}; creating one requires a superuser. Integration tests under src/py/tests/integration use Litestar's AsyncTestClient with a seeded database, and ci.yaml runs them with uv run pytest src/py/tests --cov=src/py/app.

The cache is configured where the other configs live. The agent adds a CacheSettings dataclass to app/lib/settings.py with REDIS_URL from CACHE_REDIS_URL, defaulting to the SAQ URL so a single Redis serves both in development, a DEFAULT_EXPIRATION in seconds, and an ENABLED switch; app/config.py builds a ResponseCacheConfig from it with store="response_cache" and a cache_response_filter that caches only status codes from 200 to 299, since the documentation's default key builder and filter are what the app gets otherwise. In core.py, on_app_init registers the store under that name: RedisStore.with_client(url=settings.cache.REDIS_URL) normally, and MemoryStore() when the app runs under tests, so the test suite needs no Redis. The Redis store is given a namespace, which is what lets a later delete_all clear only the cache's keys.

Handlers opt in one at a time. get_oauth_config gets cache=True, because its answer is a function of environment variables that do not change while the process runs, and the default expiration still bounds it. list_tags and get_tag get cache=60: tags are shared across users rather than per user, the default key is the path plus sorted query parameters so a different search, pageSize or currentPage is a different entry, and a minute is the longest a stale list is tolerated. GET /health is left uncached on purpose, since a cached 200 would hide a database that has just gone away. Nothing under accounts, admin or teams is cached, because their responses depend on who is asking. On create_tag, update_tag and delete_tag the agent adds a call that deletes everything in the response cache store's namespace, so a tag edit is visible on the next list request rather than up to a minute later.

Tests go in src/py/tests/integration/tags beside the existing ones: a request to /api/tags twice asserts the second answer is identical and the service method ran once, a create followed by a list asserts the new tag appears, and a request to /health after the database fixture is torn down asserts a 500 rather than a cached 200. The agent runs make lint, which covers ruff, mypy, pyright, slotscheck, biome and codespell, and make test, and pastes their tails into the pull request.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "claude",
  "input": "Add response caching to this Litestar app. Read pyproject.toml, uv.lock, app/lib/settings.py, app/config.py, app/server/core.py, every controller under app/domain and the integration tests first. Add a CacheSettings dataclass with a Redis URL that defaults to the existing SAQ Redis URL, a default expiration in seconds and an enabled switch, build a ResponseCacheConfig in app/config.py that uses a store named response_cache and caches only 2xx responses, and register that store in on_app_init as a RedisStore with a namespace, or a MemoryStore when running under tests. Add cache=True to the OAuth config endpoint and cache=60 to the tag list and tag read handlers, leave the health endpoint and every handler whose response depends on the current user uncached, and on tag create, update and delete clear the response cache namespace. Write integration tests for a repeated list, a create followed by a list, and an uncached health check, run make lint and make test, and open a pull request. Note any config that is built but never applied without changing it.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

The pull request touches app/lib/settings.py, app/config.py, app/server/core.py, app/domain/system/controllers.py and app/domain/tags/controllers.py, adds the three tests, and adds CACHE_REDIS_URL to .env.local.example beside the SAQ URL. Its description lists each cached handler with its expiration and why that one is safe to cache, the handlers deliberately left out, and the tails of make lint and make test from the sandbox.

The existing ci.yaml runs the suite on the pull request with the memory store, so the reviewer sees the new tests pass on a GitHub runner without a Redis service. A note in the description says the CompressionConfig built in app/config.py is never assigned to the app, which is a separate one-line change if the team wants gzip responses.

Before merging, a reviewer decides whether a shared Redis with SAQ is right for production or the cache should get its own database number, whether 60 seconds is the right bound for the tag list, and whether the tag write handlers should clear only the list keys rather than the whole namespace once the key builder is customised.

Run it on every pull request that adds a GET handler

Once the cache exists, every new read endpoint is a decision. An automation on the Project with a github trigger on pull_request.opened and the prompt "if this pull request adds a GET handler, say whether its response depends on the current user, and if not, whether it should take a cache value and which write handlers would need to clear it" reviews each one. Automation runs are read-only, so the comment is all it posts, and only work by an owner, member or collaborator fires it.

A team with several Litestar services sends this body once per repository from its own server, each a POST /o/{org}/sessions with that repository in resources and a request_id that is a UUID v5 of the repository's full name, so a retried call returns the run already started. Each run reads its own controllers: a service with no Redis gets the memory store and a note on what that means across workers, and one whose list endpoints are per user gets no cache on them at all.

The use case this job follows, with its automation and trigger: Hand off a long task.

Why run it on Rush?

Litestar's response cache is a few lines to turn on and easy to turn on wrongly: a handler that varies by user, a health check that keeps answering 200, a list that never notices a write. The agent reads every controller and the auth setup first, caches only the handlers whose answers are the same for everyone, and clears the store where the data changes, in its own sandbox with the repository's own lint and test targets.

The lint and test output stay with the run and are pasted into the pull request, so the review is of code that passed ruff, mypy, pyright and the integration suite rather than of a config that looks right. Several services get the same treatment at once, each with its own list of safe handlers.

Frequently asked questions

Which endpoints get cached?

The OAuth config endpoint and the tag list and tag read. Their answers do not depend on who is asking. Health, accounts, admin and teams are left alone, and the pull request lists each one with the reason.

Do the tests need Redis?

No. Under tests the store registered as response_cache is a MemoryStore, so make test and the existing ci.yaml run as they do today. Redis is used when the app starts normally, at the URL in CACHE_REDIS_URL.

What happens when a tag is created?

The create, update and delete handlers clear the response cache's namespace before returning, so the next GET /api/tags is computed fresh. Between writes, a list entry lives at most 60 seconds.

Why not cache the health check?

Because its job is to notice a database that stopped answering. It runs select 1 on every call and answers 500 when that fails; a cached 200 would keep a load balancer sending traffic to a broken instance.

Will this change the compression setting?

No. The agent noticed that a gzip CompressionConfig is built in app/config.py but never assigned to the app and says so in the description, since turning it on is a separate decision with its own effect on every response.

More Litestar jobs

The same job on other stacks

Keep reading