Re-ranker

Full definition

Bi-encoders (used in vector search) embed query and document separately — fast but loses interaction. Cross-encoders concatenate query and document and run them through a transformer together — expensive but ~20-30 NDCG points more accurate. The standard pipeline retrieves top-100 with a bi-encoder, then re-ranks to top-5 with a cross-encoder. Production options: Cohere Rerank, Voyage Rerank, BGE-Reranker, Jina Reranker, Mixedbread.

Why it matters

Adding a re-ranker is the single highest-ROI quality improvement most enterprise RAG systems can make — typically a 10-25% lift in answer accuracy at $0.001-0.01 per query incremental cost. It should be the first optimization after a baseline RAG ships.

Example

A bank's policy-search system retrieves top-50 documents in 60ms with pgvector, then re-ranks with Cohere Rerank in 200ms; final-answer accuracy on a 200-question eval rises from 67% to 84%.

Related terms

• Hybrid Search
  Hybrid search combines lexical retrieval (BM25 or sparse vectors like SPLADE) with dense vector retrieval and fuses the results, capturing both exact-match precision and semantic recall.
• Semantic Search
  Semantic search is information retrieval that ranks documents by meaning rather than exact-keyword overlap, using vector embeddings of query and documents to measure similarity in a learned semantic space.
• Retrieval-Augmented Generation (RAG)
  Retrieval-Augmented Generation (RAG) is a technique where a language model retrieves relevant documents from an external knowledge base before generating a response, reducing hallucination by 30-60% and enabling citation of sources.
• Embedding Model
  An embedding model is a neural network that maps text, images, or other inputs into fixed-dimensional vectors where semantic similarity corresponds to geometric closeness (cosine or dot-product distance).