The Study That Told AI to Stop Practicing and Start Growing
A 2020 paper found that huge, half-finished models beat small ones trained to mastery — and that arithmetic became the bet behind the giant models that followed.
Anyone who has learned anything believes some version of the same thing. You get good by working through the material. You finish the book, you do the exercises to the end, and stopping halfway is how you stay mediocre. The method matters as much as the hours: a better curriculum, a better teacher, a better design.
In 2020 that was also the working theory behind machines that handle language. These systems learn one way. They read text, guess the next word, adjust themselves slightly whenever they're wrong, and repeat that billions of times. Researchers assumed what separated a good one from a bad one was craft: how the internal layers were arranged, how wide, how deep, which of several competing blueprints you picked. They also assumed a bigger system would need proportionally more reading and more practice to earn its size, the way a longer book takes longer to finish.
Then a team ran the comparison at full stretch. They built these text-predictors across a comic range of sizes, from ones with about a thousand internal adjustable settings to ones with a billion and a half. They fed them wildly different amounts of text, from a modest pile up to a thousand times more. They swapped in rival designs, including blueprints from an earlier generation of the field.
The design barely moved anything. Depth, width, the internal arrangement people had argued over for years: nudges.
What moved things was scale, and it moved with unnerving regularity. Each model's error rate, meaning how badly it guessed words in text it had never seen, tracked a smooth curve against three quantities and only three: how big the model was, how much text it read, and how much raw computing power went in. It behaved like a growth chart. Measure a child at four and you can forecast their height at fourteen. Measure one of these models partway through training and you could forecast where it would end up without bothering to finish.
Then came the result that broke the intuition. Bigger models also learned faster, extracting more from every single example than small models did. A giant reads one paragraph and takes more out of it than a small model takes out of the same paragraph. Size bought a higher ceiling and a quicker climb toward it.
Follow that through and the advice inverts. If you have a fixed budget of computing power, the worst thing you can do is build a modest system and train it lovingly to mastery. You should build something enormous, show it comparatively little, and pull the plug long before it has finished learning. A half-taught giant beats a fully taught apprentice. How to spend a budget stopped being a matter of judgment and became arithmetic.
That arithmetic covers exactly one skill: guessing the next word in a document nobody trained the model on. Whether the same curves hold for reasoning, or conversation, or writing code, is not something this paper measured. The industry read the numbers anyway and spent the next several years building the biggest things it could afford. The rule was real. Its reach was a bet.