Vector Database
A database optimized for storing and searching high-dimensional embedding vectors using similarity search, the storage layer of most RAG systems.
What Is a Vector Database
A vector database is a specialized data store for high-dimensional numeric vectors (embeddings) and the algorithms that find the nearest neighbors to a query vector in sub-linear time. It’s the storage and retrieval layer underneath nearly every production RAG system. Use one when your application needs to “find documents similar to this query” at scale beyond what a SQL LIKE or full-text search can handle.
How It Works
- Indexing — vectors are organized into structures like HNSW (Hierarchical Navigable Small World) graphs or IVF (Inverted File Index) that enable approximate nearest neighbor (ANN) search in O(log n) rather than O(n)
- Similarity metric — cosine similarity, dot product, or Euclidean distance, picked to match how the embedding model was trained
- Hybrid search — production systems usually combine vector search with keyword search (BM25) and re-ranking, because pure vector search misses exact matches
- Filtering — most vector DBs support metadata filters (e.g.,
tenant_id = X) that narrow the search space before ANN runs
Why It Matters
Without a vector database, RAG doesn’t scale. Brute-force similarity search across a million documents on every query is fine for a demo and unusable for a product. Picking the right vector store (and tuning the index) is often the difference between RAG that responds in 50ms and RAG that responds in 5 seconds.
Examples
- Managed services — Pinecone, Weaviate Cloud, Qdrant Cloud, Vertex AI Vector Search
- Self-hosted open source — Weaviate, Qdrant, Milvus, Chroma
- Postgres extensions — pgvector (the most common “good enough” starting point)
- Search engines with vector support — Elasticsearch, Vespa, OpenSearch