Re-Ranker

Full definition

Bi-encoders (used in vector search) embed query and document separately — fast but loses interaction. Cross-encoders concatenate query and document and run them through a transformer together — expensive but markedly more accurate: Nogueira and Cho's BERT re-ranker beat the previous MS MARCO state of the art by 27% (relative) in MRR@10. The standard pipeline retrieves top-100 with a bi-encoder, then re-ranks to top-5 with a cross-encoder. Production options: Cohere Rerank, Voyage Rerank, BGE-Reranker, Jina Reranker, Mixedbread.

Why it matters

Adding a re-ranker is the single highest-ROI quality improvement most enterprise RAG systems can make: Databricks reports an average 15-point gain in retrieval accuracy (recall@10) on its enterprise benchmarks, at roughly $0.001–0.01 per query in incremental cost. It should be the first optimization after a baseline RAG ships.

Example

A bank's policy-search system retrieves top-50 documents in 60ms with pgvector, then re-ranks with Cohere Rerank in 200ms; final-answer accuracy on a 200-question eval rises from 67% to 84%.

Next steps

• Get a free workflow map →We map one of your workflows and give you a written verdict: software, a process change or AI. No obligation.
• AI consulting cost guide →Hourly rates and project budgets by firm tier, from independents to the Big Four.

Related terms

• Hybrid Search
  Hybrid search combines lexical retrieval (BM25 or sparse vectors like SPLADE) with dense vector retrieval and fuses the results, capturing both exact-match precision and semantic recall.
• Semantic Search
  Semantic search is information retrieval that ranks documents by meaning rather than exact-keyword overlap, using vector embeddings of query and documents to measure similarity in a learned semantic space.
• Retrieval-Augmented Generation (RAG)
  Retrieval-Augmented Generation (RAG) is a technique where a language model retrieves relevant documents from an external knowledge base before generating a response, which reduces hallucination and enables citation of sources.
• Embedding Model
  An embedding model is a neural network that maps text, images, or other inputs into fixed-dimensional vectors where semantic similarity corresponds to geometric closeness (cosine or dot-product distance).

More deployment & operations terms

• Chunking Strategy
  A chunking strategy is the rule set used to split source documents into retrievable units before embedding, which directly determines retrieval recall and answer faithfulness in a RAG system.
• Agent Memory
  Agent memory is the mechanism by which an AI agent stores and retrieves information across turns or sessions, beyond the fixed context window of the underlying language model.
• Planner-Executor
  Planner-executor is an agent pattern that separates a high-level planning model — which produces a multi-step plan up front — from a tool-calling executor that runs each step, decoupling strategic reasoning from tactical action.