Everyday Apparatus
IntelligencearXiv3 min read1 month ago

The AI That Tries the Side Door before Calling for Help

When an AI assistant hits a wall, how it recovers turns out to matter more than how smart it is — and the answer is embarrassingly human.

A read of Beyond Global Replanning: Hierarchical Recovery for Cross-Device Agent Systems · arXiv:2606.20487

Orchestrator

The top-level coordinator that assigns subtasks across devices and steps in when something goes wrong — the 'strategic command' in the article's analogy.

API / CLI / GUI

Three ways to operate software: a programming interface, a text-based command line, and a visual graphical interface — H-RePlan treats these as interchangeable doors on each device.

Hierarchical replanning

A two-level recovery structure: fix the failure locally first, escalate to the orchestrator for cross-device replanning only if local options are exhausted.

Fault injection

Deliberately introducing failures into a test environment to measure how robustly a system recovers — the method behind the paper's HeraBench benchmark.

What it’s not claiming · The paper does not claim that hierarchical recovery eliminates the need for any global replanning at all—global coordination is still required whenever all local strategies on a device have truly failed.

We have been promised assistants that live inside our phones and laptops and do the dull work for us. They are built to range across our devices at once: take a task on your laptop, move a file, then pick it up on your phone to attach it to an email. The excitement assumes the hard part is getting one to work at all. It isn't. The hard part is what happens when it fails. And it will fail. Anything that has to act in the messy real world eventually meets a door that won't open. The real question was never whether these systems stumble. It's what they do in the half-second after.

Until recently they had two options, both bad. The first was to try the same thing again, harder, like a person rattling a locked handle. The second was to kick the problem upstairs to the coordinator juggling the whole job across all those devices. Picture someone who tries the front door once, finds it locked, and immediately phones their manager. Polite, and ruinous. In one test, assistants that did everything by clicking through the on-screen interface — the most flexible route — burned more than ten million tokens, the currency these models spend as they think, for every task they finished cleanly. The thrifty alternative stuck to the official channel, the sanctioned set of commands the software hands out. It was cheaper and useless: zero tasks completed perfectly, because some jobs simply cannot be done that way. That channel exposes only what the software's designers chose to expose, and ordinary operations on the machine's own files fall outside it, invisible.

Here is the strange part, the thing that makes the work worth reading. The better answer turns out to be both cheaper and more reliable at once, which is rarely how improvements arrive. You usually buy dependability with money. This time the same move bought both.

And the move sounds too obvious to write a paper about. Before you escalate, try another way on the same device. Think of a seasoned technician who reaches a stuck panel and, instead of radioing headquarters, tries a second tool from her own bag, then a third, a blunter one when the polished tool can't reach. Only when her whole kit has failed does she call for help. The rookie radios the instant the first tool slips. The escalation still exists. It just becomes the last resort instead of the first reflex.

Letting the assistants fend for themselves locally took perfect completions from roughly one in seven up to more than one in three. The cost of each clean success fell from ten and a half million tokens to under two million, about five times cheaper. Both numbers moved the right way for the same reason: trying the side door first means you rarely kick down the front one, and kicking it down was where all the money went.

These breakages were staged, failures injected on purpose, on a handful of computers and phones, in conditions the researchers controlled. Real systems break in stranger, uglier ways than anyone thinks to script. And the trick only helps where more than one path exists; hand the assistant a single door and you are back where you started. The work offers a principle, not a cure: exhaust what is in reach before you cry for help. Whether that holds against the genuine chaos of the world is still unwritten.

Where this sits

Open question

Will the improvements shown by H‑RePlan’s device‑local strategy switching continue to hold when agents are used on a wide variety of uncontrolled devices and encounter unexpected, real‑world failure modes that differ from the synthetic faults in HeraBench?

Next readThe AI That Reads Your Assumptions at the Last Moment