Vector database

A vector database stores embeddings and answers one kind of question very quickly: which entries sit closest in meaning to this one? That makes it the search engine behind most AI applications working with your own data.

How does a vector database work?

A conventional database looks for exact values or strings. A vector database looks for proximity in a space with hundreds of dimensions. The query is not a word but a vector.

At its core is an index that shortcuts this nearest-neighbour search. Comparing every entry individually would be far too slow across millions of passages, so these indexes work with approximations. They return the best matches almost always, and occasionally the second best instead of the best. For search that is a good trade. For accounting it would not be.

What matters in practice is combining proximity with filters. Almost every real application needs both: the most similar document, but only from this department, only in this language, only for this permission group.

Why does a vector database matter?

The moment a language model is meant to work with your content, something has to find the right excerpts. That is the job. In a RAG setup the quality of that retrieval determines the quality of the answers far more than the choice of language model does.

Its second role is control. Because retrieval decides which documents the model sees at all, it is the natural place for access rights. Filter there and you do not have to trust the model to keep confidential material to itself.

Do you need a dedicated database for this?

Often not. Postgres can store and search vectors with an extension, as can several search systems you may already run. For a few tens of thousands of passages that is entirely sufficient, and it spares you another system in the backend.

A specialised vector database earns its place with very large corpora, high query load, or when you want hybrid search and result reranking supplied out of the box.

The question to settle first: where are those vectors allowed to live? They hold reconstructable fragments of your documents. A hosted service in another jurisdiction is therefore the same decision as outsourcing the documents themselves, and it belongs to AI governance.

What actually matters when choosing one

Not the vendors' throughput figures. Four other points decide how it runs.

Filtering applied together with the nearest-neighbour search. If a system finds neighbours first and filters afterwards, narrow filters make the relevant matches disappear. Both have to run together, or retrieval performs worst in exactly the cases where permissions apply.

Hybrid search. Semantic search finds paraphrases; keyword search finds part numbers and proper nouns. Most applications need both in practice, because people type both.

Updating in production. Documents change. A system where individual passages can be replaced without rebuilding the index saves work indefinitely.

And a way back. The corpus has to be recomputable, because switching embedding models requires exactly that.

Check those four and you often end up with the simplest option: the database you are already running.