The Math Baked into AI Safety Monitors Was Making Risk Look Smaller than It Is

Here is the de-slopped body:

Picture an AI assistant let loose on something you'd rather keep private: a folder of legal files, a medical record, a thread of messages. It reads, it acts, it sends. Watching over its shoulder is a safety monitor, software that runs each of its moves through a set of automated checks and flags anything that looks like a leak. The monitor exists to make a promise: this agent will not spill your data. And that promise has always rested on an assumption nobody bothered to say out loud.

The assumption is that each check fires in complete ignorance of the others. When one alarm decides whether to sound, it behaves as if it has no idea whether any other alarm is sounding too. But if checks are trained on the same examples, or built to catch the same kinds of signal, they tend to fail together. When they do, and the monitor assumes they can't, its math scatters the risk thinly, as though each danger were a private accident. The number it reports comes out lower than the truth.

Imagine a smoke detector that treats every room's air as sealed off from every other room's. It might tell you the house is 24 percent likely to be burning when the real figure, once you account for smoke drifting through the vents, is 28. The gap sounds trivial. On sensitive data, at scale, it is exactly the gap real harm slips through.

A team of researchers asked a sharper question. Instead of assuming any particular relationship between the checks, what is the worst leakage risk possible under every relationship the data still allows? Their method doesn't try to list every way the checks might move together, which would be hopeless. It tracks only how each pair of checks tends to lean, then finds the worst arrangement consistent with those pairings. On a head-to-head test, the old independence math reported 27.2 percent risk; the robust method reported 30. And the error doesn't always run one way. A rival approach that estimates risk by running many random trials overshoots when checks fail together. The lesson is narrower than "always assume the worst": whenever the assumption is wrong, the number is wrong.

The obvious objection is speed. If the math now covers every conceivable correlation, surely it crawls. Mostly it doesn't. On two of the three workloads tested it is dramatically faster: roughly 220 milliseconds where the previous best method took a full second, and under a third of a second where that method took more than seven. On the third, a harder kind of task, it runs slower, around two seconds against the older method's third of a second, so the advantage depends on the problem. But across every safety setting, it let through everything safe and blocked everything dangerous, a record the random-sampling approach failed outright at the strictest threshold.

One blind spot survives. The monitor can only reason about rules a human wrote down in advance. If a rule is wrong, or missing for some new tool the agent picks up, no amount of mathematical care will save you. The deeper lesson is about the promises we let our safety systems make. A guarantee is only as strong as its worst-case assumption, not its average-case one, and for years these monitors had been betting on the average and calling it safety. The fix wasn't learning how failures cluster. We stopped assuming they wouldn't.

---

`★ Insight ─────────────────────────────────────` - The biggest tells here were the **"quiet" family** (the dek keeps "quietly" — untouched per instructions — but the body had "a quiet assumption" and "quietly betting," both removed) and two instances of **negative parallelism** ("The lesson isn't X. It's Y" / "The fix isn't X. It's Y"). - For the closing beat, I preserved the landing line ("We stopped assuming they wouldn't") but broke the "isn't X, it's Y" formula into two plain sentences — the contrast survives without the machine cadence. - The mid-piece "The lesson isn't 'assume the worst'" became "The lesson is narrower than 'always assume the worst'" — same correction, stated directly instead of via the banned frame. `─────────────────────────────────────────────────`

Every number, name, and comparison is unchanged; paragraph breaks, argument order, and the opening hook are intact.