The demo took twelve minutes. The agent received a task, formed a plan, called four tools, and returned a structured report with citations. Everyone in the room was convinced. The VP of Engineering said they wanted it in production before end of quarter.
That quarter came. Three months of integration work later, the agent was handling roughly one in five requests correctly. The other four timed out, hallucinated a field the downstream system expected, or returned output the review team had to flag before it could proceed. Six months after the demo, the pilot was quietly canceled.
Nobody blamed the model. The model was fine. The demo had been real.
What the demo couldn't show is the number that decides whether an agent works in production: the cost of failure. No demo can show it.
In a demo, that number is zero. There are no downstream consequences. There is no review process that can block the output. There is no customer whose invoice gets corrupted, no analyst whose report goes out with a wrong figure, no engineer whose merged PR introduced a regression. The agent runs free. It looks effortless because it is; nothing it does carries a cost.
Production restores the cost. And with it comes the need to manage it — which requires the other variable the demo skipped: review.
The equation and the gap
The prior post introduced the equation that governs this: V = min(P, R) × Q − (1 − Q) × F. Production rate, review rate, quality, cost of failure. Four variables. The four that determine whether an agent-augmented system creates or destroys value.
A demo is a special case: Demo = f(P, Q). With F set to zero and R effectively unlimited (or simply absent), the two variables that constrain every production deployment disappear. What remains is best-case performance, evaluated under conditions that will never exist once the work is real.
Production restores both: Production = f(P, Q, R, F). R becomes finite and must scale with throughput. F becomes real and must be managed rather than ignored.
The gap between those two cases is not a model problem. Every model that looked compelling in a demo is still compelling. The model has not changed. What changed is the context — and context, specifically the cost of failure and the presence or absence of review, is entirely the responsibility of the harness, not the model.
What the numbers say
The gap between demo and production is not anecdotal. A March 2026 survey of 650 enterprise technology leaders found that 78% were running at least one agent pilot — but only 14% had scaled any of them to org-wide use.
That 64-point attrition rate is consistent across multiple research organizations. 2026 enterprise adoption data from Forrester, IDC, and LangChain converge around the same figure: roughly 88% of agent pilots never reach production. Not because the agents failed the demo. Because the teams couldn't bridge from demo conditions to production conditions.
Gartner added a harder number in June 2025: over 40% of agentic AI projects will be canceled outright by end of 2027. The reasons Gartner named were not capability failures. They named escalating costs, unclear business value, and inadequate risk controls — each a symptom of a harness gap, not an intelligence gap.
The same report coined the term "agent washing." Gartner estimates that of the thousands of vendors claiming agentic AI, roughly 130 are genuinely building it. The rest are calling a chatbot in a loop an agent. Their demos look identical to the real thing because demos don't show F.
Why it is not the model
The dominant post-mortem for a failed agent pilot is some version of: the model wasn't good enough yet. This framing is almost never accurate, and it matters because it leads teams to wait for better models rather than build better harnesses.
The Gartner findings name three actual causes: cost, value clarity, and risk controls. The March 2026 survey points to similar patterns. In published analyses of canceled pilots, the recurring causes are:
Integration with existing systems. The agent completes its task, but the task requires writing to a database, calling an API, or reading from a file system that wasn't designed for automated access. These integrations reveal scope that was invisible in the demo. They take months, and during that time the organization is paying for a model it can't deploy.
Unclear ownership. When something goes wrong, nobody knows whether to retrain the agent, fix the integration, rewrite the prompt, or escalate to the model vendor. The organization has no word for who owns an agent that misbehaves. A human employee who fails a task has a manager. An agent without a harness has none.
Data quality. The agent was tested on clean examples. Production sends it the edge cases, the malformed records, the missing fields. The model often handles these gracefully; the harness doesn't know what to do with the output, so the output disappears or goes through unreviewed.
None of these are model problems. A model that handles malformed inputs more gracefully does not solve the problem of a harness that can't route the output to a human reviewer when the confidence is low. A smarter model does not explain to the organization who owns the agent. A better model does not write the database integration. These are infrastructure problems and governance problems, and the intelligence gap between a good demo and a working production system is almost entirely in them.
Reliability vs Recoverability
The natural response to a failed agent pilot is to demand higher reliability. The agent should not have failed. The model should have handled the edge case. The prompt should have been more specific.
This response optimizes for the wrong metric.
Mean time between failures is the reliability metric. A high MTBF is good, but it is achieved through model improvements, prompt engineering, and data cleaning — all of which hit diminishing returns at the capability levels available now. A production agent that never fails is not a realistic near-term target.
Mean time to recovery is the recoverability metric. A low MTTR is the achievable target. It is achieved through checkpointing, review gates, structured output schemas, and rollback primitives — all of which are engineering choices, not model choices. Unlike MTBF, MTTR is addressable today.
This is not a distinction between pessimism and optimism. It is a distinction between two engineering strategies, and one of them is available now.
When an agent fails and the harness has checkpointed its state, logged the reasoning trace, surfaced the failure to a reviewer, and provided a rollback path, the failure cost is bounded and manageable. When an agent fails into a void — no record of what happened, no path back — the failure cost is catastrophic. One produces an iteration. The other produces a canceled pilot.
The teams that ship agents are not the teams with better models. They are the teams that made failure cheap.
The 12-minute coding pipeline from Agents Are Eating the Org Chart demonstrated this directly. A review agent found a bug the original agent missed — a bug that wasn't even in the original bug report. A third agent fixed it. Every step produced an auditable artifact. That is not a story about a smarter model. It is a story about a harness that made independent review cheap enough to include in every single run, so a failure at any step could be caught before it shipped.
The artifact viewer in Rush exposes exactly this structure: the full session state, the reasoning trace at each step, and a replay path from any checkpoint. A reviewer can inspect what the agent did, why, and what output it produced — and either approve, correct, or re-dispatch.
An honest caveat
The F/R frame does not account for every way a pilot fails. Integration complexity, data quality, and unclear ownership each impose real costs that exist even when the harness is well-built. Naming R and F as the gap is accurate; claiming they are the only gap would not be honest.
There is a separate caveat worth naming about Rush's own data.
The Rush harness, as of July 2026, records a 91.1% session success rate across all runs in the past week. Median session duration is 2.7 minutes — short enough to checkpoint frequently. Week-over-week retention is 40% among active users. These numbers are directionally encouraging.
The 91.1% is early-stage data from a small user base. It is an existence proof that the harness approach can hold; it does not prove it generalizes to every domain, every agent, or every kind of work. Own that.
What it does demonstrate is that F = 0 is not a fixed condition of agent deployment. With checkpointing, structured output validation, review gates, and rollback primitives, the effective cost of failure in a production system can be brought closer to the cost of failure in a demo — not by making failures impossible, but by making them cheap to recover from. Mean time to recovery, not mean time between failures, is the metric to manage.
The Siri trap describes an adjacent failure mode: routing all work through a single agent that tries to do everything. That architecture fails for the same underlying reason — there is no review loop, no intermediate state, and no way to recover a failure in step three without restarting from step one. The single-chatbot metaphor, like the demo condition, hides F by pretending failures don't compound.
Build the harness before the model
The sequence that produces the 88% attrition rate looks like this: run the demo, ship the model into production, discover the harness problems, patch them under pressure. The patching is reactive, slow, and expensive. By the time it is clear what the harness needs, the pilot has already embarrassed someone.
The sequence that survives is the reverse: build the harness first, then deploy the model into it.
A harness that makes failure cheap needs four properties:
- State that persists. Every agent run produces a checkpoint. A failed run resumes from the checkpoint, not from blank.
- Reasoning that is legible. The agent's full trace — what it read, what it concluded, what it called — is logged and attached to the output. A reviewer can audit any step without re-running the session.
- Output that can be blocked. Before agent output reaches a downstream system, a review gate holds it. The gate can be automated (schema validation, confidence thresholds) or human; either is better than no gate.
- A rollback path. If a bad output gets through, the system can undo it. The cost of failure is bounded by the checkpoint interval, not by the severity of the incident.
None of these are model capabilities. A model cannot checkpoint its own state. It cannot surface its reasoning trace to a reviewer. It cannot enforce a review gate on its own output. It cannot roll back a database write. These are platform responsibilities — and they are precisely what every demo omits, because in a demo none of them are necessary.
The equation says this directly. V = min(P, R) × Q − (1 − Q) × F. When F is zero and R is unconstrained, V is whatever Q delivers. When F is real and R must scale, V is determined by whichever of P and R is smaller, minus however much the failures cost. The model contribution is Q. The harness contribution is R and F. The demo only shows Q.
The 78% of organizations running a pilot already have Q. What they are discovering, one canceled project at a time, is that Q alone does not produce V.
The equation V = min(P, R) × Q − (1 − Q) × F was introduced in Agents Are Eating the Org Chart. Statistical anchors: Gartner press release, June 25, 2025 (40% cancelation forecast, "agent washing" finding); March 2026 enterprise technology survey (n=650, 78%/14% figures); 2026 enterprise adoption convergence across Forrester, IDC, LangChain. Rush production telemetry for the week ending July 13, 2026 (91.1% success rate, 2.7-minute median session, 40% WoW retention — early-stage data, small sample). Related: The Intelligence Gap; The Siri Trap.
