Embeddings: How Machines Turn Meaning Into Numbers
Almost everything modern AI does with language starts with one simple trick.
Embeddings are the least glamorous idea in modern AI and quietly one of the most important. Vector databases, semantic search, RAG, recommendations — they all sit on top of the same trick. And the trick, once you strip the jargon off it, is almost embarrassingly simple: turn a thing into a list of numbers, arranged so that similar things end up close together.
The one idea
Take a word, a sentence, an image, a product — and represent it as a point in a high-dimensional space. A vector, which is just a long list of numbers. The whole game is that the arrangement is meaningful: things that are alike land near each other, things that aren't land far apart. "King" sits near "queen." A support ticket about a refund sits near other refund tickets, even when they share no exact words.
That's it. Meaning becomes geometry. And once you can measure the distance between two points, you can ask "what's similar to this?" — a question that turns out to underpin a startling amount of what AI actually does.
Why this is such a big deal
Before embeddings, computers compared text by matching characters. "Car" and "automobile" were as different to a machine as "car" and "banana" — no shared letters, no match. Keyword search lived and died on you typing the exact word the document happened to use.
Embeddings break that. Because "car" and "automobile" land near each other in the space, a search can find one when you asked for the other. The machine isn't matching strings anymore; it's matching meaning. That single shift is what makes semantic search, recommendation engines, and retrieval for LLMs possible at all.
Where the numbers come from
You don't hand-assign these values — that would be hopeless. A model learns them, by being trained on enormous amounts of text (or images, or whatever) and nudged, over and over, to place things that appear in similar contexts near each other. The famous result is that the geometry ends up capturing relationships nobody programmed in: the direction from "man" to "woman" is roughly the same as "king" to "queen." The structure of meaning falls out of the training — the same way it does for language models, because it's the same kind of process.
What this looks like in practice
If you build anything AI-flavoured, embeddings are probably already underneath it. Semantic search over your docs: embed the documents, embed the query, return the nearest ones. Recommendations: embed what a user liked, find nearby items. RAG: the retrieval step is an embedding lookup. A vector database is, more or less, a database built to do one thing quickly — given a vector, find the closest others among millions.
The practical catch worth holding onto: embeddings capture similarity, not truth, and not exact matches. Two sentences that sit close together aren't necessarily both correct, and an embedding will happily miss an exact identifier — a product code, an error number — precisely because it's reaching for meaning rather than characters. That's exactly why serious retrieval systems pair embeddings with old-fashioned keyword search: each covers the other's blind spot.
The takeaway
Embeddings are the layer where messy human stuff — words, images, preferences — becomes something a computer can do maths on. It isn't a flashy idea. It's plumbing. But almost everything impressive sitting on top of modern AI is, underneath, some clever use of "turn it into a vector and measure the distance." Understand that one move and a surprising amount of the field stops looking like magic and starts looking like geometry.
