What an LLM Actually Is, Once You Strip the Magic Out
It's a next-token predictor. That sentence explains more than it has any right to.
Ask most explainers what a large language model is and you'll get a metaphor about a very well-read assistant. It's not wrong, but it hides the one fact that actually explains the behaviour: an LLM is a next-token predictor. Given some text, it estimates the most likely next word-piece, appends it, and repeats. That's the whole engine.
I resisted this framing at first. It sounds far too simple to explain something that writes working code and passes exams. It turns out to explain almost everything, including the parts people find most surprising.
Why "just predicting the next word" is more than it sounds
Here's the leap that took me a while to accept. To predict the next word well across the entire internet, a model is forced to absorb a staggering amount about the structure of what it's predicting. To finish "the capital of France is," it needs a fact. To continue a proof, it needs something that behaves like reasoning. To complete a function, it has to have internalised the shape of working code. None of that was programmed in. It fell out of the pressure to predict well.
That's the genuinely counterintuitive result of the last few years: a simple objective, applied at enormous scale, produces capabilities nobody explicitly built. "Understanding" is probably the wrong word for what's going on inside. But "just autocomplete" undersells it badly.
Why it lies to you, explained by the same fact
The next-token framing also explains the failure everyone complains about. A model doesn't have a fact store it looks things up in. It generates the most plausible continuation. Most of the time the most plausible continuation is also true, because true things are well-represented in the training text. But when they aren't — an obscure detail, a citation, your internal API — it produces something that looks right with exactly the same confidence, because plausibility is all it was ever optimising for.
This is why "hallucination" is a slightly misleading word. It implies a malfunction. It isn't. It's the normal operation of the system, showing up in a case where plausible and true happened to come apart. Once that clicks, you stop being surprised and start designing for it.
The two dials worth actually understanding
Two concepts do most of the practical work. The context window is how much text the model can consider at once — everything you send plus everything it generates has to fit inside it, and when you run out, the earliest content falls off the back. Tokens are the unit it works in: not words, but word-pieces, which is why it bills by the token, sometimes mangles unusual spellings, and struggles with tasks like counting letters that require seeing below the token.
Neither is exotic. But most of the confusion I see from people using these models — why did it forget the start of the document, why is this so expensive, why can't it spell — comes down to not holding these two dials in mind.
The engineer's stance
Knowing the mechanism doesn't make the model less useful. It makes you calibrated. You stop expecting a database and start treating it as what it is: a fluent, broadly knowledgeable, occasionally confabulating text engine that's extraordinary at some things and quietly unreliable at others. You verify what matters. You hand it the context instead of trusting its memory. You design for the failure mode rather than being ambushed by it.
The magic-assistant framing sets you up to be disappointed. The next-token framing sets you up to build something that works. It's less romantic, and far more useful.
