# Which Feature Flags Changed or Were Left On This Week? A Restaurant-Ordering Software Company's Monday Flag Review on Railway

A restaurant-ordering software company's agent reads every Railway feature flag each Monday and lists what changed, what is half rolled out, and what is unowned.

[Run this on Rush](https://cloud.getrush.ai/run/review-feature-flags-each-week/railway/). Opens the console with this prompt ready to send. Connect Railway there first.

## What the agent does

This is for a company that sells online ordering and delivery software to independent restaurants, hosted on Railway, and that uses Railway's feature flags to ship carefully: a new checkout behind a percentage rollout, a kill switch for the SMS order alerts, a number flag that sets how many orders a kitchen tablet pulls at once. Railway's docs say a flag can turn off an endpoint, enable maintenance mode or change a rate limit without a deploy, which is exactly why flags drift: a switch flipped during a Friday incident never shows up in a code review, and a rollout parked at 25% stays there for a month. Every Monday the agent writes the engineering lead a review of every flag.

Someone with viewer access to the projects connects Railway once with `POST /me/connections {"type": "mcp", "name": "railway", "url": "https://mcp.railway.com", "auth": {"mode": "oauth"}}` and picks, on Railway's consent page, the workspace and the projects that hold the flags. Railway's feature flag docs list two tools a viewer can call, `list-feature-flags` and `get-feature-flag`, and two that need an admin, `set-feature-flag` and `delete-feature-flag`. The review uses only the first two, and the prompt forbids the others.

On a run the agent calls `list-projects` and, for each project the prompt names, `list-feature-flags`, then `get-feature-flag` for each flag to read its type (Railway's four are `bool`, `string`, `number` and `json`), its default value and its targeting rules. A rule is a condition and the value to serve; Railway's docs describe attribute rules, such as `plan` equals `enterprise`, and percentage rollouts hashed on a key. The company's backend sends last week's snapshot in the prompt, so the agent can compare: a default that changed, a rule added or removed, a rollout whose percentage moved, a flag created or deleted since last Monday.

It writes `flags-review.md` with four sections. Changed since last week, with old and new values side by side. Kill switches and maintenance flags currently not at their normal value, which the company's flag register in the prompt names. Rollouts still partial, with their percentage and the attribute they hash on. Flags missing from the register, which have no owner anyone wrote down. It also returns `flags.json`, this week's snapshot, which the backend stores for next Monday's comparison. It sets, deletes and redeploys nothing.

## Start it with one API call

`POST https://api.phoenix.so/o/{org}/sessions`

```json
{
  "agent": "claude",
  "input": "Review the feature flags in our Railway projects ordering-api and kitchen-display for the week to 2026-11-16. Use the railway connection. Call list-projects to find them; put any project list-projects does not return at the top of the review as not read. Then call list-feature-flags for each and get-feature-flag for every flag, recording its type, default value and targeting rules. Last week's snapshot is the JSON at the end of this message; compare against it. Our flag register (name: owner, normal value): checkout-v2: payments team, rollout to 100% planned by 2026-11-30; sms-order-alerts: platform, true; maintenance-mode: platform, false; tablet-batch-size: kitchen team, 20. Write flags-review.md with sections: Changed since last week (old and new side by side), Switches not at their normal value, Rollouts still partial (percentage and the attribute hashed on), Flags not in the register. Also write flags.json: every flag with project, name, type, default and rules. Never call set-feature-flag, delete-feature-flag, redeploy, accept-deploy or railway-agent.\n\nLast week: {\"ordering-api\": [{\"name\": \"checkout-v2\", \"type\": \"bool\", \"default\": false, \"rules\": [\"plan == enterprise && bucket(key) < 0.25 -> true\"]}, {\"name\": \"maintenance-mode\", \"type\": \"bool\", \"default\": false, \"rules\": []}], \"kitchen-display\": [{\"name\": \"tablet-batch-size\", \"type\": \"number\", \"default\": 20, \"rules\": []}]}",
  "connections": [
    "railway"
  ]
}
```

1. **Get an API key.** In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
2. **Send the request.** POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
3. **Read what came back.** Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

## What comes back

The session returns `flags-review.md` and `flags.json` as artifacts. The engineering lead reads the review at the Monday stand-up: a maintenance flag still on from Friday gets turned off by an admin in Railway, a rollout that has not moved in two weeks gets a decision, and a flag nobody owns gets an owner or a delete. The backend stores `flags.json` for next week.

What the lead checks: that a change the review lists was intended (Railway's flags change without a deploy, so this review may be the first place anyone sees it), the register itself, which is only as current as the last person who edited it, and any project the Railway sign-in could not see, which the review cannot report on. Changing a flag stays an admin's action in Railway.

## Run it on every Monday morning

The company's backend sends `POST /o/{org}/sessions` at 8:00 every Monday with the body above, last week's stored snapshot appended to `input`, and a `request_id` that is a UUID v5 of the date, so a retried job starts no second run. The key's webhook delivers `run.completed` to the backend, which fetches `flags.json` from the session's artifacts and stores it.

After an incident, the on-call engineer can start the same request at once with the snapshot from before the incident, to see every flag that was flipped while it ran.

## Why run it on Rush?

Flags are the one change in this company's system that needs no deploy, so nothing else records them. The review keeps a weekly record in Rush, one session per Monday, with every Railway call and what it returned, and the Railway sign-in stays in Rush's connection, limited to the projects picked at consent, instead of an account token in a cron job.

Each run has its own computer and a record of what its model calls cost. Railway's docs name the flag tools that need an admin, the prompt forbids them by name, and the transcript shows they were never called, which matters for a review whose readers will act on it.

## Frequently asked questions

### Why not read the flags with Railway's SDK in a script?

Railway's docs say the SDK reads a flag's value for a given context and keeps flags in memory for fast reads; it answers what one user gets, not what changed across every flag this week. The review needs every flag's default and rules side by side with last week's, written in words the stand-up can act on.

### Can the agent turn off a flag that was left on?

It could only if the account behind the sign-in is a project admin, since Railway's docs mark `set-feature-flag` and `delete-feature-flag` as admin tools. Connecting Railway with a viewer account keeps the agent to reading, and the prompt forbids both tools either way.

### Does the review know who changed a flag?

No. Railway's flag docs do not describe a change history through the MCP tools, so the review compares this week's flags with last week's snapshot. It says what changed and leaves who changed it to the team.

### What if a project is not in the Railway sign-in?

Railway's OAuth consent lets the person choose projects, and the agent sees only those. A project named in the prompt that `list-projects` does not return is listed at the top of the review as not read.

## More Railway jobs

- [Which Railway Projects Are Still Running for Clients We No Longer Serve? A Web Agency's Monthly Hosting Check](https://getrush.ai/run/find-hosting-left-running-for-past-clients/railway/)

## More jobs like this: Produce a report or file, no repo

- [Did Our Public API Examples Break Overnight? An API Company's Postman Monitor Review](https://getrush.ai/run/review-overnight-api-monitor-failures/postman/)
- [Audit Row-Level Security in a Supabase Project](https://getrush.ai/run/row-level-security-audit/supabase/)
- [Schedule a Week of Social Posts in Buffer](https://getrush.ai/run/schedule-a-week-of-posts/buffer/)
- [Run a Security Audit on an Angular App](https://getrush.ai/run/security-audit/angular/)
- [Run a Security Audit on an ASP.NET Core App](https://getrush.ai/run/security-audit/aspnet-core/)

## Keep reading

- [Connect Railway to Rush once, then name it on any run](https://getrush.ai/integrations/railway/)
- [Start your first run from the developer API quickstart](https://getrush.ai/docs/quickstart/)
- [Rush pricing: Free, Pro and Max](https://getrush.ai/pricing/)
