Back to blog
Engineering

Our Writing Agent Made 858 Commits. Then It Died and Nobody Noticed.

For 45 days an unattended agent committed every half hour, and the log looked like work. Then its configuration vanished and the scheduler ran empty for 64 days without an alarm. The checks that would have caught it: output, and whether the goal still matters.

An empty concrete room with one rectangle of daylight on the floor, and nobody in it.

On October 8 I restored the home directory of an agent gateway on a spare Mac and found that our writing agent had been dead for 64 days. The gateway process had been up the whole time. It had no agents and no jobs, and nothing told us.

Before it died, the same agent had made 858 commits in 45 days. I want to put those two numbers next to each other, because together they are the lesson. An unattended loop with no check that its goal still matters produces a commit log that looks like productivity. When ours lost its configuration, the scheduler kept running with nothing in it for two months and no alarm fired. A better prompt would have fixed neither problem. Two checks would have caught both: a liveness check on output, and a goal check on purpose.

858 commits that looked like work

The agent ran on OpenClaw, the open-source agent gateway, on a schedule: every 30 minutes, from 7am to 11pm Pacific. Each run read its notes, picked a page on one of our small marketing sites, made an SEO edit, and committed it.

From June 18 to August 1, 2026 it made 858 commits, and 856 of them landed between 07:00 and 23:59 Pacific (measured from our own git history on October 12, 2026). The average commit touched 1.2 files and added 13.5 lines. Most were a heading here, an internal link there, a keyword moved up a paragraph. On a busy day it committed 36 times.

Commits per day from our writing agent, June 18 to October 12, 2026Bars of 20 to 36 commits a day from June 18 to August 1, then nothing. A gold band marks August 5 to October 8, the 64 days the gateway ran with no jobs.64 daysrunning, no jobs,no alarm102030Jun 18Aug 1Sep 1Oct 8858 commits in 45 days, then nonePer day, Pacific. Our git history, measured Oct 12, 2026
Every bar is a day of commits from the writing agent. The gold band is the 64 days the gateway ran with no agents and no jobs.

If you had glanced at the repository in July you would have seen a contributor who started at seven every morning, worked past eleven at night, and shipped all day. The graph is a heartbeat. It is also the trap: a steady stream of small, plausible commits reads as progress, and nobody stops to ask what the commits are for.

The agent's own daily log for August 4 answers that question. The commits had already stopped on August 1. The runs that day re-read the same files, found no work assigned to them, and made another tiny edit anyway. The loop had outlived its queue. It kept producing because producing was the only thing the schedule asked of it.

The day the configuration disappeared

On August 5 the gateway's home directory was switched to a nearly empty install, and nobody noticed what the switch took with it. The gateway started, found no agents and no scheduled jobs, and carried on serving nothing.

Timeline of our writing agent, June 18 to October 9, 2026Nine dated events. The stretch from August 5 to October 8, when the gateway ran with no agents and no jobs, is marked in gold.Jun 18The loop starts: a run every 30 minutes,7am to 11pm PacificAug 1Commit 858, the last oneAug 3The blog's last post goes upAug 4Runs re-read the same files, find noassigned work, edit a line anywayAug 5The gateway's home is switched to anearly empty installAug 23A missing messaging-token error landsin a log nobody readsSep 10, 15The sites it edited are deleted asoff-roadmapOct 8Restored, 64 days laterOct 9The one blog job left writes a goodpost; a build failure keeps it unshipped
What happened, in order. Gold marks the stretch when the gateway ran empty.

Three things should have tipped us off, and none did:

  • The blog's last post, An Agent You Can't Find Is an Agent You Don't Have, went up on August 3. Nothing followed it. A missing post looks exactly like a quiet week.
  • On August 23 a missing messaging-token error appeared in the gateway's log. A log line is an alarm only if something reads it, and nothing did.
  • In September we deleted the directories of the sites those 858 commits had edited, in two commits on September 10 and 15 that removed 107,609 lines of site code (measured from our own git history on October 12, 2026), because those sites were off our roadmap. We deleted the agent's entire field of work, and nobody asked where the agent was.

That last one stings the most. We were careful in that cleanup about what each line of code was for, and never asked the same question of an agent that was, as far as anyone knew, still running.

Two failures, one missing question

Looking back, the busy loop and the silence are the same failure seen from two sides. In both, the only signal anyone could see was that the machinery existed. In July the machinery was visibly busy, so it looked healthy. From August to October it was quietly running, so it looked fine. At no point did anything ask whether the output arrived or whether it mattered.

Site reliability engineers worked this out long ago. Google's SRE book splits monitoring into "what's broken, and why," and describes black-box monitoring as "symptom-oriented," catching the problems that are active right now (Site Reliability Engineering, "Monitoring Distributed Systems"). For a scheduled job, the symptom is missing output. Cron monitors such as Healthchecks.io are built on that idea: the service "keeps silent as long as pings arrive on time" and "raises an alert as soon as a ping does not arrive on time," a dead man's switch for scheduled work (Healthchecks.io docs).

We knew this for servers. We did not apply it to an agent, because the agent felt like a person on the team who would mention it if something went wrong. A schedule with an empty job list has nothing to mention.

Four checks and whether each would have caught the two failuresA process check stays quiet in both periods. An output check fires on August 2, the first day without a commit. A goal check fires on August 4, when the runs found no assigned work.Busy loopJun 18 to Aug 1SilenceAug 2 to Oct 8Is the process up?quietquietAny errors in the log?quietone lineAug 23unreadDid new output land today?quietfires Aug 2Is there assigned workthat still matters?never askedfires Aug 4Each check applied to our own timeline.Only the last two would have told us anything.
The checks we had stayed quiet the whole time. An output check would have fired on August 2, the first day without a commit; a goal check would have fired on August 4.

Two checks and one durability rule

Alert on missing output. Process liveness told us nothing in either failure. The check that matters is "did a new post, commit or report land in the window it should have," measured where the output lives. A process that is up and producing nothing is down. For the blog, that is one check on the newest post date.

Re-check that the goal still matters. An unattended loop needs a question it asks before doing any work: is there assigned work, and does the thing it serves still exist? When the answer is no, the run should stop and say so, and that report is itself output a person sees. Our agent spent its last productive days polishing sites we were about to delete.

Keep the queue in the repository. The agent's sense of what to do next lived in its notes inside the gateway's home directory, so when that directory changed, the plan vanished with it. A queue in the repository survives a reinstall, shows up in review, and is visible to the person who decides it is finished. I made a version of this argument about memory in After the Org Chart; the queue is the part of memory that has to outlive the process.

None of these is a model problem. A smarter model would have written better SEO edits for sites we deleted, and it would have been equally dead after August 5. It is the same gap I wrote about in Every Agent Demo Hides the Same Number: what decides the outcome lives in the harness around the model.

The revive, and an honest postscript

After the restore, one blog job was left. Its first run wrote a good post, "The Interrupt Is the Interface," dated October 9. A build failure kept it from shipping. So the very first output of the revived loop proved the point again: the job ran, the agent did good work, and the result did not arrive. The check that would have caught it is the first one above.

Where Rush fits

This is the behavior we hold Rush to, and it is concrete. A Rush run that changes code ends with one PR carrying its checks and the full trajectory of what the agent did. A run that is missing its proof ends as needs_review. A member who connects Slack gets run finished, failed and needs-you messages in a direct message. Those signals cover runs that happen. Automations run a Project's saved prompt on a schedule, a webhook, or a GitHub or Linear event, and each fire is a recorded run. And a run that never starts sends nothing, so the missing-output check from the list above is still yours to set, and it is the first one I would set.

What I keep coming back to is how normal both failures looked from the outside. Thirty commits a day looked like diligence. Sixty-four quiet days looked like a lull. The only thing that tells them apart is a check that someone wrote down before either happened.

Read Next