Model Training, Explained Without the Hand-Waving
It's guess, check, adjust — a few billion times.
Training is the part of machine learning that sounds like sorcery and is, underneath, one stubbornly simple loop repeated an absurd number of times. Strip away the mystique and it's three moves: guess, check how wrong you were, adjust slightly. Do that a few billion times and you get something that behaves like intelligence.
The loop
Here's the whole thing. You show the model an example — an image, a sentence, whatever — and it makes a prediction. At the start the prediction is garbage, because the model's internal numbers (its weights) are essentially random. Then you compare its guess to the right answer and measure how far off it was. That measure is the loss. Finally, you nudge every weight a tiny amount in the direction that would have made the loss a little smaller.
That's it. Guess, measure the error, adjust to shrink it. Repeat across millions of examples, many times over. The model is never told the rules. It's corrected, relentlessly, until being right becomes the path of least resistance.
Why it needs so much data and compute
Each individual adjustment is deliberately tiny, because big jumps overshoot and the whole thing goes unstable. So learning anything real takes an enormous number of small corrections, which means an enormous number of examples and an enormous amount of arithmetic. That's why training the large models costs what it costs: you're running that guess-check-adjust loop across billions of examples and billions of weights, and the arithmetic piles up into numbers with a lot of zeros.
It's also why data quality matters more than newcomers expect. The model learns from whatever you correct it against. Feed it biased, wrong, or sloppy examples and it will faithfully learn to reproduce biased, wrong, sloppy answers — with total confidence. The loop has no concept of "good." It only knows "closer to the answers I was given."
The trap: memorising versus learning
There's one failure mode worth understanding, because it explains a lot. A model can get very good at the exact examples it trained on and still be useless on anything new — it memorised the answers instead of learning the pattern. That's overfitting. The fix is to hold back some data the model never trains on and check its performance there. Do well on examples it has never seen, and it learned something general. Ace the training set but flunk the held-out set, and it just memorised.
That distinction — did it learn the pattern or memorise the answers — is the whole game. It's also why "it works on our test cases" is the start of evaluation, not the end of it.
What actually changed recently
The loop is decades old. What changed is scale, and a bet: that if you make the model bigger, feed it more data, and run the loop longer, capabilities keep improving — sometimes in ways nobody predicted. That bet mostly paid off, which is why the last few years happened. But the mechanism under the headline models is the same humble loop a first ML course teaches in week two.
Training isn't the machine thinking. It's the machine being wrong, getting corrected, and being slightly less wrong — a staggering number of times, until wrong becomes rare enough to be useful.
