Vector Databases: What They're For, and When You Actually Need One
A specialized tool that got sold as a general one. Here's the honest version.
Vector databases had a moment. For about a year they were the answer to every AI infrastructure question, the mandatory box in every architecture diagram. Some of that was deserved. A lot of it was hype looking for a home. Here's what they actually do, and when you actually need one.
What a vector database is for
A regular database is brilliant at exact lookups. Give it an ID, a name, a date range, and it finds the matching rows fast. What it's bad at is the question modern AI keeps asking: "find me the things most similar to this," where similar means close in meaning, not an exact match.
That's the whole job of a vector database. You've turned your data into embeddings — vectors where nearby means similar. A vector database stores millions of those vectors and, given a new one, finds the nearest neighbours fast. That's it. It's a database with one specialty: nearest-neighbour search over high-dimensional vectors, at scale.
Why it isn't trivial
You might reasonably ask why this needs a special database. Can't you just compare the query vector to every stored vector and keep the closest? Yes — and at a few thousand vectors, you should; a loop and some NumPy is genuinely fine. The problem is scale. Comparing against every vector — an exact search — gets painfully slow at millions of items.
So vector databases use approximate nearest-neighbour algorithms: clever indexes that don't check every vector, only the ones likely to be close. You trade a sliver of accuracy — you might occasionally miss the true closest match — for an enormous speedup. That trade-off is the actual engineering, and it's why "just use Postgres" is sometimes right and sometimes not.
The part the hype skipped
Here's what the 2023 diagrams didn't tell you: you very often don't need a dedicated vector database. Postgres with the pgvector extension handles vector search perfectly well up to a surprisingly large scale, and it comes with the enormous advantage of being the database you already run, back up, and understand. Your vectors live next to your actual data, in one system, with one set of operational headaches instead of two.
The case for a dedicated vector database — Pinecone, Weaviate, Qdrant, and the rest — gets strong when you're genuinely at scale, tens of millions of vectors and up, or when nearest-neighbour search is the core of your product rather than a feature bolted onto the side. Below that, adding one is often just a second database to operate for benefits you can't yet measure.
How to actually decide
The honest decision tree is short. Are you doing similarity search at all? If not, you don't need this. If you are, are you at a scale where pgvector in your existing Postgres starts to struggle? If not, use that — fewer moving parts wins almost every time. If you are, or if vector search is the product, then a purpose-built vector database earns its keep.
That's a less thrilling story than "vector databases are the future of data storage," which was the framing going around. They're not the future of data storage. They're a good tool for one specific and increasingly common problem — and like every good tool, the real skill is knowing when you don't need it yet.
