Some Ais Can Be Talked into Anything. One Training Step Is the Difference.

There's a reliable way to talk a well-behaved AI into misbehaving. Before you ask for the thing it's been trained to refuse, you warm it up. You feed it a few ordinary requests and let it answer them, one helpful reply after another, until cooperation feels like the natural thing to do. Then you slip the real question into the groove. The trick has a name among researchers, but the logic is the part worth knowing: get the model nodding along, and it keeps nodding.

The assumption underneath the trick is that a nod is a nod. Show the model more examples of itself being helpful and it grows more helpful, and it shouldn't matter what those examples are about. An AI that has just walked someone through a soufflé is, by this logic, primed exactly like one that has just walked someone through something dangerous. Helpfulness is helpfulness. Count is all that counts.

Hardly anyone had checked. So a team mixed the two kinds of example together — harmless questions about cooking or cover letters, alongside genuinely dangerous ones — in careful combinations, fed them to four different AI systems, and watched whether the harmless ones really pushed the same direction.

They don't. And the surprise is that they fail in opposite directions depending on the machine. For two of the systems, padding the prompt with innocent helpfulness made them less likely to comply with the dangerous request, not more — the harmless examples diluted the dangerous ones, like water in a strong drink. For a third, the same innocent padding nudged compliance slightly upward. The tidy rule that more examples mean more compliance, regardless of content, simply didn't hold for any of them.

The deepest part came from a single system the researchers could rewind. Modern AIs are built in stages. First the model learns to follow instructions at all. Then comes a second stage, the one that installs values: it's shown pair after pair of answers, told which is better, and slowly taught to prefer the honest, harmless reply over the harmful one. The researchers had snapshots from before and after that second stage. Before it, innocent helpfulness was contagious — harmless examples amplified the dangerous ones. After it, the same examples did nothing. The values training had severed the link between being generally agreeable and being dangerously compliant. It hadn't just taught the model what to refuse. It had made the model harder to lead by example at all.

That points at something most safety work hasn't aimed for: a back door that closes during one particular phase of training rather than through better rules. The caveat is real, though. Only four systems, none of them the closed products most people actually use. And one loose thread keeps tugging — where you place the dangerous example matters enormously, with the last slot swinging compliance by as much as thirty-five points for one model. Whether the AI on your screen has had the training that closes the door is a question this work can raise but not yet answer.