Dense Passage Retrieval (DPR)

Full definition

Karpukhin et al. (Facebook, 2020) showed that a learned dual encoder, given enough training pairs and hard negatives, beats BM25 on open-domain QA — establishing the modern retrieval blueprint. DPR is the architectural ancestor of every production embedding model in vector search (E5, BGE, voyage, OpenAI text-embedding-3). Hard-negative mining (in-batch negatives plus BM25-mined hard negatives) is what makes the technique work in practice.

Why it matters

DPR established that retrieval quality is mostly a training-data problem, not an architecture problem. Enterprises that fine-tune their embedding models on domain-specific (query, passage) pairs typically see 10-20 NDCG points over off-the-shelf — the highest-ROI retrieval improvement after re-ranking.

Example

A pharma RAG system fine-tunes an embedding model on 30k (clinical-question, regulator-approved-passage) pairs mined from prior submissions; recall@10 rises from 71% to 89% on a held-out test set.

Related terms

• Embedding Model
  An embedding model is a neural network that maps text, images, or other inputs into fixed-dimensional vectors where semantic similarity corresponds to geometric closeness (cosine or dot-product distance).
• Semantic Search
  Semantic search is information retrieval that ranks documents by meaning rather than exact-keyword overlap, using vector embeddings of query and documents to measure similarity in a learned semantic space.
• Re-ranker
  A re-ranker is a second-stage retrieval model that re-scores a candidate set (typically top 50-100) from a fast first-stage retriever using a more expensive cross-encoder that jointly attends to query and document.
• Masked Language Model
  A masked language model (MLM) is a model trained to predict tokens that have been hidden in the input, learning bidirectional context rather than left-to-right next-token prediction.