Embedding Model

Full definition

Modern text embedding models are typically encoder-only transformers fine-tuned with contrastive learning on (anchor, positive, hard-negative) triplets. Leaders on the MTEB benchmark include OpenAI text-embedding-3-large, Cohere Embed v3, Voyage AI voyage-3, BGE, E5, and Nomic Embed. Dimensionality ranges from 384 to 4096; higher dimensions cost more storage and compute but may improve retrieval. Domain fine-tuning often beats a stronger general model.

Why it matters

Embedding-model choice is the single biggest lever on RAG quality before re-ranking. Buyers should run a held-out evaluation on their own domain — public benchmarks (MTEB) are weakly correlated with enterprise-document retrieval performance.

Example

A SaaS company tests 6 embedding models on 500 internal queries; voyage-3 wins on recall@10 by 8 points over text-embedding-3-large and reduces re-ranker workload by 40%.

Related terms

• Embedding
  An embedding is a dense numerical vector — typically 384 to 4096 dimensions — that represents the semantic meaning of a piece of text, image, audio, or other content, so that semantically similar items end up near each other in vector space.
• Masked Language Model
  A masked language model (MLM) is a model trained to predict tokens that have been hidden in the input, learning bidirectional context rather than left-to-right next-token prediction.
• Semantic Search
  Semantic search is information retrieval that ranks documents by meaning rather than exact-keyword overlap, using vector embeddings of query and documents to measure similarity in a learned semantic space.
• Vector Database
  A vector database is a database optimized for storing and searching high-dimensional vectors (embeddings) by similarity rather than by exact match — the storage layer of every RAG and semantic-search system.