All jobs

Add Smoke Tests to a Docker Compose Stack

Yes. An agent adds a health check to every Compose service and a smoke test that brings the stack up with --wait and calls each endpoint, then opens a PR.

Run this on RushIn a Project that has your repository.

What the agent does

The agent reads the Compose file and every Dockerfile it builds before changing anything, because a health check has to use a command the image actually contains. In docker/awesome-compose's react-express-mongodb, compose.yaml defines three services: frontend built from frontend/Dockerfile on node:lts-buster and published on host port 3000, backend built on node:lts-buster-slim with only expose: 3000, and mongo on mongo:4.2.0 with a named volume. None has a healthcheck, and depends_on uses the short list form, so Compose starts backend as soon as the mongo container exists, not when it accepts connections.

Then it reads the application code to learn what healthy means for each service. backend/server.js only calls app.listen(3000) after db.connect emits ready, and backend/db/index.js retries the Mongo connection every 2 seconds, so a backend that answers on port 3000 has already reached Mongo. backend/routes/index.js mounts GET /api (list todos) and POST /api/todos. frontend/package.json runs react-scripts start with "proxy": "http://backend:3000", so a request to localhost:3000/api from the host reaches the backend through the dev server.

With that it writes the health checks. The slim backend image has no curl or wget, so its check is node -e with an http.get to localhost:3000/api that exits non-zero on anything but a 200. Mongo 4.2 ships the legacy mongo shell rather than mongosh, so its check is mongo --quiet --eval "db.adminCommand('ping')". The frontend's check requests / from the dev server. It switches depends_on to the long form with condition: service_healthy, so backend waits for mongo and frontend for backend, and sets start_period long enough for npm ci-built images and the first CRA compile.

Last it writes smoke-test.sh: docker compose up -d --build --wait --wait-timeout 180, then a POST /api/todos with a unique text, a GET /api with Accept: application/json that must contain it, and a GET / that must return the React shell, each with curl --fail. On failure it prints docker compose ps and the last lines of docker compose logs, and on exit it runs docker compose down -v. It runs the script in its sandbox until it passes from a clean start twice, then adds a workflow that runs it.

Start it with one API call

POST https://api.prix.dev/o/{org}/sessions with this body:

{
  "agent": "codex",
  "input": "For the react-express-mongodb stack, add health checks to every service in compose.yaml using commands each image actually contains (read every Dockerfile first: the backend image is slim and has no curl), switch depends_on to condition: service_healthy, and write a smoke-test.sh that runs docker compose up -d --build --wait, creates a todo with POST /api/todos, reads it back from GET /api through the frontend on port 3000, checks / serves the React app, prints compose logs on failure and always runs docker compose down -v. Add a GitHub Actions workflow that runs the script on pull requests touching that directory. Run it in the sandbox from a clean start until it passes and open a pull request.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.prix.dev/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

The pull request changes react-express-mongodb/compose.yaml to add a healthcheck block to frontend, backend and mongo and the long-form depends_on, adds react-express-mongodb/smoke-test.sh (executable, set -euo pipefail), and adds .github/workflows/compose-smoke.yml with a paths filter on that directory, running the script on ubuntu-latest, which has Docker and the Compose plugin installed. Nothing in the backend or frontend source changes.

Its description shows each health check with the reason for its command (the slim image, the Mongo 4.2 shell), the timing values chosen and why, and the output of the final smoke run. The new workflow runs on the pull request itself, so the check that proves the stack comes up is green before anyone merges. The session's record keeps every docker compose command the agent ran in the sandbox, including the failed starts that set start_period.

A reviewer checks two things the agent flags but does not change on its own: restart: always on every service, which can hide a crash loop from --wait until the timeout, and the floating node:lts-buster base images on Debian 10. Both are listed as suggestions with the change they would need.

Run it on every Compose file changed

Once the smoke test exists, the workflow covers every pull request that touches the stack. To keep the health checks themselves current, a GitHub trigger on pull_request.opened with a prompt that asks to update healthcheck and smoke-test.sh for any service the pull request adds to compose.yaml gives each new service a check in the same pull request that introduces it.

For a repository like awesome-compose with dozens of stacks, send one request per stack directory, each naming its directory in input. The runs go in parallel up to the key's concurrency limit, and each opens its own pull request, so one stack's slow image build does not hold up the rest.

The use case this job follows, with its automation and trigger: Write the missing tests.

Why run it on Rush?

A Compose smoke test only means something if it starts from nothing: no cached images, no leftover volume with yesterday's data, no container already holding port 3000. On a laptop that means tearing down whatever you had running. A Rush sandbox starts clean every time, so when the agent says the stack came up twice from a cold start, that is literally what happened, and your own Docker state is untouched.

The evidence is the commands and their output. The session's record has each docker compose up --wait, each health check result and each curl, so a reviewer can see why start_period has the value it has. The new workflow repeats the same script on GitHub, which is the check that keeps the stack honest after this pull request merges.

Frequently asked questions

Why not use curl in every health check?

Because many images do not have it, and installing it just for a health check grows the image. In this stack the backend is built on node:lts-buster-slim, which has Node but no curl or wget, so the check uses node -e. The agent picks each command from what that image's Dockerfile and base image provide.

What does --wait actually wait for?

docker compose up --wait returns once every service is running and, for services with a health check, healthy; it fails if one exits or turns unhealthy. Without health checks it only waits for containers to start, which is why the checks come first and the smoke test second.

Does the sandbox run real containers?

The agent runs docker compose in its sandbox and reports the real output. If the sandbox cannot start containers for a stack, for example because an image needs a privileged mode, the agent says so in its answer and relies on the new workflow's run on GitHub, rather than claiming a pass it did not see.

Will it change my application code?

No. It edits compose.yaml, adds the smoke script and the workflow. If the app has no endpoint that proves it is healthy, it uses an existing read route, as here with GET /api, and suggests a dedicated health route in the pull request instead of adding one.