What Is Reranking in RAG?
A second pass in which a stronger model re-scores the top search results against the question, so the passages that actually answer it rise to the top.
First-stage search must be fast across every chunk, so it uses cheap signals: embedding similarity and keyword overlap. Both measure whether a passage is about the same subject as the question, not whether it answers it. Reranking takes the top few dozen candidates and scores each one again with a model that reads the question and the passage together.
The usual reranker is a cross-encoder: a model that takes the question and one passage as a single input and returns a relevance score. Because it sees both at once, it can tell a page that explains how to rotate an API key from one that only mentions API keys. It is too slow to run over a whole corpus, so it runs second, on a short list.
Reranker scores are also the best basis for deciding whether to answer at all. That makes the reranker the natural place for a relevance threshold, and for abstention when nothing clears it.
How Kelu uses this
After hybrid search and reciprocal rank fusion, Kelu sends the top candidates to a hosted cross-encoder reranker, chosen and tuned by Kelu rather than per workspace. Nothing is dropped for low relevance before this step. The reranker's scores set the coverage verdict and a precision floor; if the reranker is unavailable, an LLM takes over the scoring.
Related terms
Hybrid Search
A retrieval strategy that combines semantic (vector) search with keyword (full-text) search and merges the results, capturing both meaning and exact matches.
Reciprocal Rank Fusion (RRF)
A way to merge ranked result lists into one by scoring each item on its position in each list, not its raw score, so different search methods can be combined.
Abstention
When an AI assistant declines to answer because its sources don't cover the question, and says so plainly instead of writing a confident guess.
Semantic Search
Search that ranks results by meaning similarity between the query and documents, using embeddings, rather than by matching keywords.
RAG (Retrieval-Augmented Generation)
An AI architecture that retrieves relevant documents from a trusted knowledge source and passes them to a language model, so answers are grounded in real content instead of the model's training data.
From definition to deployment
Connect a knowledge source and see cited AI answers on your own content in minutes — free, no credit card required.
Free to start · No credit card required