Vector Database

Full definition

Vector databases use approximate nearest neighbor (ANN) algorithms such as HNSW, IVF, or ScaNN to find the most similar vectors to a query in milliseconds, even across billions of items. Examples include Pinecone, Weaviate, Qdrant, Milvus, pgvector (Postgres extension), and the vector capabilities now bundled into Elastic, MongoDB, and Snowflake. The choice between a dedicated vector DB and a vector-enabled OLTP database is often more about ops than capability.

Why it matters

Vector databases are the new system-of-record for unstructured enterprise content. Every document, email, call transcript, and image becomes a searchable asset once embedded and stored. For executives this is a category-creating data-platform decision, not a tactical infrastructure choice.

Example

A bank embeds 12 million internal documents (policies, manuals, procedures) into a vector database. Employees ask natural-language questions in a chat interface; the system retrieves the top 5 chunks and the LLM answers with citations — replacing what was a 10-million-dollar legacy enterprise-search investment.

Next steps

• Get a free workflow map →We map one of your workflows and give you a written verdict: software, a process change or AI. No obligation.
• AI consulting cost guide →Hourly rates and project budgets by firm tier, from independents to the Big Four.

Related terms

• Retrieval-Augmented Generation (RAG)
  Retrieval-Augmented Generation (RAG) is a technique where a language model retrieves relevant documents from an external knowledge base before generating a response, which reduces hallucination and enables citation of sources.
• Embedding
  An embedding is a dense numerical vector — typically 384 to 4096 dimensions — that represents the semantic meaning of a piece of text, image, audio, or other content, so that semantically similar items end up near each other in vector space.
• Large Language Model (LLM)
  A Large Language Model (LLM) is a deep neural network — almost always a transformer — trained on hundreds of billions to trillions of words to predict the next token, and to generate, summarize, translate, or reason over text.
• Inference
  Inference is the process of running a trained AI model on new, unseen inputs to produce predictions, classifications, or generated content — the part that runs every time a user interacts with the system.
• MLOps
  MLOps is the discipline of operating machine-learning and AI systems reliably in production — covering data pipelines, model training, deployment, monitoring, drift detection, governance, and incident response — analogous to DevOps for traditional software.

More deployment & operations terms

• Token / Tokenization
  A token is the basic unit a language model reads or writes — usually a sub-word fragment (about 4 characters of English text) — and the unit by which API pricing, context-window limits, and inference cost are all measured.
• Inference Cost
  Inference cost is the dollar cost of running a trained AI model in production — per request, per user, or per business outcome — and the operating expense that determines whether an AI feature has positive unit economics at scale.