What the agent does
This is for a SaaS company that sells an API to other businesses and sends its OpenTelemetry traces to Honeycomb, with each request tagged by the customer's account id. Before every enterprise customer's quarterly business review, the customer-success manager needs one page on how the API performed for that customer: how fast, how often it failed, which endpoints they lean on. Today that means asking an engineer to build the queries. The agent builds them instead. Someone connects Honeycomb once with POST /me/connections {"type": "mcp", "name": "honeycomb", "url": "https://mcp.honeycomb.io/mcp", "auth": {"mode": "oauth"}} and grants read-only access to all resources on Honeycomb's consent page. The team needs Honeycomb Intelligence turned on for the MCP server to answer.
The run calls get_workspace_context for the team's environments and datasets, then find_columns to find the account id, route, HTTP status and duration columns by what they mean, so the prompt does not have to know the exact attribute names. With run_query, every query filtered to this customer's account id, it gets the weekly request count, the weekly P50, P95 and P99 duration, the weekly share of requests answered with a 5xx status, and the ten routes this customer called most with each route's P95 and error share. When the dataset is sampled, Honeycomb returns sampling metadata with the results, and the agent says so in the page.
It calls get_slos to list the company's SLOs and their status for the quarter, and states plainly that they cover every customer, not this one. For the week with the highest P95 it runs run_bubbleup on that week's slowest requests, which tells it what sets them apart from the rest: one route, one region, one large request size. That becomes a sentence a customer-success manager can repeat without guessing.
It writes review.md for a reader who is not an engineer: a three-sentence summary, the weekly table, the routes table, and the slowest week explained. It creates no board, trigger or SLO; those Honeycomb tools need write access, which this connection was never given.
Start it with one API call
POST https://api.phoenix.so/o/{org}/sessions with this body:
{
"agent": "claude",
"input": "Prepare the API performance page for customer account <account id> for 2026-07-01 to 2026-09-30 UTC, from the production environment. Call get_workspace_context, then find_columns to find the columns for customer account id, route, HTTP status and duration. With run_query, every query filtered to that account id: weekly request count; weekly P50, P95 and P99 duration; weekly share of requests with status 500 or above; the 10 routes with the most requests, each with its P95 and error share. Note any sampling metadata in the results. Call get_slos and list each SLO's status for the quarter, saying they cover all customers. For the week with the highest P95, run run_bubbleup on that week's slowest requests and name what sets them apart. Write review.md for a customer-success manager: a three-sentence summary, the weekly table, the routes table, and the slowest week explained in plain words. If no column holds the account id, stop and say so. Do not create or update any board, trigger, SLO or recipient.",
"connections": [
"honeycomb"
]
}- Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
- Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
- Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.
What comes back
review.md comes back as an artifact on the session, and the result is the three-sentence summary. The customer-success manager reads it as a draft for one slide or one page of the review deck, not as something to forward unread.
Before it reaches the customer, two people look at it. The manager checks that a bad week matches what the customer remembers; a spike in P95 during a week the customer sent several times their normal traffic reads very differently from one during an outage. An engineer checks the slowest-week explanation against what the team knows about that week, and removes any internal route or service name the company does not show customers.
The transcript lists every query the agent ran in Honeycomb's own query terms, so the engineer can open any of them in Honeycomb and see the same chart.
Run it on every business review on the calendar
The review dates live in the company's CRM, which Rush does not connect to here. A small job on the company's side reads the reviews coming up in the next week and sends this request for each one, with the account id and the quarter in input and a request_id that is a UUID v5 of the account id and the review date, so a rerun of the job starts no second run.
Each finished page arrives on the key's webhook as run.completed. The handler matches execution_id to the account, downloads review.md from the session's artifact link, and attaches it to the review in the CRM for the manager.
The use case this job follows, with its automation and trigger: Produce a report or file, no repo.
Why run it on Rush?
The queries are the same for every customer; only the account id changes. An engineer building them by hand for each review spends time that adds nothing, and two engineers build them two slightly different ways. The agent runs the same set every time, explains the result for a non-engineer, and hands the manager a page with the caveats already written. Honeycomb's sign-in stays in Rush's connection with read-only access, not a team API key in a script.
Reviews bunch up at quarter end. Each one is its own run on its own computer, many start at once, and runs past the key's concurrent limit wait queued and start in order. Every run keeps a record of the queries it made and what its model calls cost, so the company can see what a review page costs in model calls.
Frequently asked questions
What if our telemetry does not carry a customer account id?
Then there is nothing to filter on, and find_columns finds no such column. The prompt tells the agent to stop and say so. Adding the account id as an attribute on every request span is a change for the engineering team, and every later review benefits from it.
Will many reviews at once hit Honeycomb's limits?
Honeycomb limits run_query to 100 calls a minute, and a rate-limited response says when to retry. A team starting many reviews together can start them in smaller batches from the job that sends the requests.
Can the agent send the page to the customer?
No. It writes a draft, and the customer-success manager decides what the customer sees and sends it. Performance numbers given to a customer are a commitment, so a person stands behind them.
Our Honeycomb team is in the EU region. Which server do we use?
Honeycomb's EU server is https://mcp.eu1.honeycomb.io/mcp. Connect it as an MCP server with that URL and name that connection on the request instead of honeycomb.