Before 2023 most engineers had never heard of vector databases. By the end of 2023 they were standard infrastructure in any AI application stack. The rise was rapid and driven entirely by the adoption of language models and the retrieval pattern they required.
A vector database stores and searches vectors. A vector is a list of numbers, typically hundreds or thousands of dimensions, that represents a piece of data: a sentence, an image, a product, a user preference. The useful property of vectors produced by modern embedding models is that semantically similar things are close together in vector space.
Traditional databases search by exact matching or range queries. You look for rows where name equals 'Alice' or where date is greater than yesterday. Vector databases search by similarity. You provide a query vector and the database finds the stored vectors that are most similar to it, typically using cosine similarity or dot product.
This semantic similarity search is what powers RAG. When a user asks "how do I reset my password", you convert that query to a vector and find the stored document vectors closest to it. The closest documents are about account access and password management, not about pricing or product features. The embedding model has learned that "reset password" is semantically similar to "forgotten credentials" and "account access", even without those exact words appearing in the query.
The reason dedicated vector databases emerged is that vector similarity search is computationally expensive and requires specialised indexing. Searching for the nearest neighbour in a space with 1536 dimensions across millions of vectors requires approximate nearest neighbour algorithms (HNSW, IVF) that general-purpose databases were not designed for. A PostgreSQL extension called pgvector emerged for teams that did not want a separate database, but purpose-built vector databases had better performance at scale.
Pinecone was the early market leader, cloud-only and managed. Weaviate was open source and more opinionated about how you used it. Chroma was the choice for local development due to its simplicity. Qdrant combined performance with a permissive open source licence.
The practical advice for most teams in 2023 was: start with pgvector if you already use PostgreSQL and your scale is moderate. Move to a dedicated vector database if you hit performance limitations or need features that pgvector does not provide. Do not adopt a new database technology because it is fashionable; adopt it because you have measured a need for it.