Semantic Search

Full definition

Both query and corpus are embedded with the same model; the system returns the top-k documents by cosine or dot-product similarity. Semantic search handles paraphrase, synonymy, and multi-language matching that keyword search (BM25) misses, but can fail on rare entities, exact strings, and out-of-domain jargon. Hybrid search combines both. Microsoft, Google, and Bing have all integrated semantic ranking into core web search since 2019.

Why it matters

Semantic search is the retrieval layer in 90%+ of RAG systems. It is also a major cost driver — every query runs an embedding inference plus an ANN lookup. Caching frequent queries can cut costs 50%+ in production.

Example

An HR portal indexes 12,000 policy documents; a query "can I work from another country in summer" surfaces the relevant remote-work and tax policies even though none of those exact words appear in the documents.

Related terms

• Embedding Model
  An embedding model is a neural network that maps text, images, or other inputs into fixed-dimensional vectors where semantic similarity corresponds to geometric closeness (cosine or dot-product distance).
• Hybrid Search
  Hybrid search combines lexical retrieval (BM25 or sparse vectors like SPLADE) with dense vector retrieval and fuses the results, capturing both exact-match precision and semantic recall.
• Vector Database
  A vector database is a database optimized for storing and searching high-dimensional vectors (embeddings) by similarity rather than by exact match — the storage layer of every RAG and semantic-search system.
• Retrieval-Augmented Generation (RAG)
  Retrieval-Augmented Generation (RAG) is a technique where a language model retrieves relevant documents from an external knowledge base before generating a response, reducing hallucination by 30-60% and enabling citation of sources.