Blog

Hybrid Search: Dense, Sparse, Fusion and Reranking

Written by

Dense vectors find meaning and miss exact words. Sparse vectors do the opposite. Running both and combining the results is the single biggest retrieval-quality improvement most systems can make — here is how, with Qdrant’s Query API.

Part 2 contains the most useful failure in this series. Asked keeping data safe when the machine loses power, a collection of correct embeddings returned a sentence about CPU register allocation, ranking it above the write-ahead log is what makes a crash survivable.

Recall was perfect. The index did exactly what it was asked. The answer was still wrong.

This part is the standard fix, and it is the highest-leverage thing in the series after getting your chunks right.

Try this first

Three queries against a documentation corpus:

  1. how do I stop my app crashing on startup
  2. ERR_MODULE_NOT_FOUND
  3. invoice INV-2026-00417

Predict which of the three a dense embedding search will handle well, and why the other two are hard for it.

The reason is the same for both, and it’s the reason this part exists.

What each kind of search is bad at

Dense embeddings map text to a point in a few hundred dimensions, trained so that similar meanings land close together. They handle paraphrase, synonyms, and questions phrased nothing like the answer. That’s query 1, and it’s genuinely hard for anything else.

What they cannot do is represent a token they have never meaningfully seen. ERR_MODULE_NOT_FOUND and INV-2026-00417 get tokenised into fragments and embedded into something, but that something reflects the shape of the string rather than its identity. Every invoice number embeds to roughly the same place. The model has no way to know that one of them is the one you typed.

Sparse vectors are the opposite. They represent text as which terms occur and how distinctive each is, over a vocabulary-sized space where almost every entry is zero. That makes INV-2026-00417 trivially findable — it’s a rare term, so it carries enormous weight — and makes query 1 nearly hopeless, because none of those words need appear in the answer.

So the two approaches fail on opposite things. Which is the argument for running both.

Storing both

From Part 8: one point, one dense vector, one sparse vector.

client.create_collection(
    "docs",
    vectors_config={"dense": models.VectorParams(size=384, distance=models.Distance.COSINE)},
    sparse_vectors_config={"lex": models.SparseVectorParams()},
)

client.upsert("docs", points=[
    models.PointStruct(
        id=1,
        vector={
            "dense": dense_vec,
            "lex": models.SparseVector(indices=[3, 17, 9042], values=[0.7, 0.4, 1.2]),
        },
        payload={"text": text},
    ),
])

The sparse vector’s indices are term ids and values are their weights. Where those come from is your choice: a classical BM25-style weighting computed from your corpus, or a learned sparse model such as SPLADE, which predicts term weights — including terms that aren’t literally in the text — using a neural model. Both produce the same shape and Qdrant doesn’t care which you used.

Combining them: the Query API

The naive approach is to run two searches and merge the results in your application. That works, and Qdrant gives you a better way: prefetch, which runs sub-queries server-side and feeds their results into a final step.

from qdrant_client import models

hits = client.query_points(
    "docs",
    prefetch=[
        models.Prefetch(query=dense_vec, using="dense", limit=50),
        models.Prefetch(
            query=models.SparseVector(indices=q_indices, values=q_values),
            using="lex", limit=50,
        ),
    ],
    query=models.FusionQuery(fusion=models.Fusion.RRF),
    limit=10,
).points

Each Prefetch is a real search producing its own ranked list. The outer query says how to combine them. One round trip, and the fusion happens next to the data.

The fusion problem, and why RRF wins

Here is the difficulty. The dense search returns cosine similarities — say 0.81, 0.77, 0.74. The sparse search returns BM25-style scores — say 18.4, 12.1, 9.7.

These numbers are not comparable. They’re on different scales, with different distributions, and the scales shift per query. Adding them means the sparse score dominates everything; normalising them per query means a query where everything scored badly gets its best result promoted to 1.0.

Qdrant offers two answers.

Fusion.RRF — Reciprocal Rank Fusion

Throw the scores away. Use only the rank.

Each document gets 1 / (k + rank) from each list it appears in, summed across lists, with a small constant k damping the top. A document ranked 1st and 4th beats one ranked 2nd and 20th; a document appearing in both lists beats one appearing in either alone.

Because it only uses ordering, RRF is immune to the scale problem entirely. It needs no tuning, no normalisation, and no knowledge of what the scores mean. It is the right default, and for most systems it is also the right permanent answer.

The cost is that it discards magnitude. A document the dense search found overwhelmingly relevant is treated identically to one it found marginally relevant, as long as they ranked the same.

Fusion.DBSF — Distribution-Based Score Fusion

Keep the scores, but normalise each list using its own distribution before combining, so the two scales become comparable.

This keeps the magnitude information RRF throws away, and in exchange it depends on the score distributions being well-behaved. When they are, it can beat RRF. When they aren’t — a query where one arm returns nothing good, so its normalised scores are all high anyway — it can do worse.

Start with RRF. Move to DBSF only if you have measured that it helps on your data, using Part 4’s methodology adapted to relevance rather than recall.

Two rankings, one answer the write-ahead log survives a crash register allocation and spills checkpointing and fsync WAL replay after restart disk caches and write barriers

Schematic. The dense and sparse scores are illustrative stand-ins chosen to show the scale mismatch, which is a property of the two score types. The RRF column is not: it is computed from the ranks shown by the same formula the text gives, with k = 60, so the figure cannot drift away from the prose.

Reranking: the third stage

Fusion combines two ranked lists. Reranking replaces the ranking altogether, using a model that is far better and far too slow to run over your whole corpus.

The distinction that matters: a dense retriever embeds the query and the document separately, and compares two summaries. It never sees them together. A cross-encoder reranker takes the query and one document as a single input and scores the pair directly, which is enormously more accurate — it can tell that a document mentions your error code in the context of causing it rather than fixing it.

The price is that it can’t be precomputed. There’s no index; you run the model per candidate, at query time.

So the shape is a funnel:

  1. Retrieve a few hundred candidates cheaply, using hybrid search.
  2. Rerank those few hundred with a cross-encoder.
  3. Return the top handful.

Retrieval’s job stops being “get the answer first” and becomes “get the answer somewhere in the candidate set” — which is a much easier job, and is why recall@100 is often the metric worth optimising rather than recall@10.

The reranker is a separate model, run in your application or a service beside it. Qdrant’s part is getting you good candidates.

Explain it like I’m ten

Two librarians.

The first has read everything and understands what books are about. Ask for “something about a boy who discovers he’s magic” and she’ll find it without you naming it. Ask for “the book with ISBN 978-0747532699” and she’ll shrug — that’s not what she pays attention to.

The second has memorised the index. Give her any exact word or number and she’ll tell you every page it appears on. Ask her about a boy who discovers he’s magic and she’ll find books containing those exact words, which is not the same question.

You want both. Ask each, then combine their lists — and the fair way is to compare their rankings rather than their confidence scores, because one is confident about meaning and the other about words, and those confidences aren’t in the same units.

Then, once you have twenty candidates, hand them to the head librarian, who actually reads each one next to your question and picks the best. She’s far too slow to read the whole library — that’s why she only gets twenty.

Where the analogy breaks: the two librarians aren’t reading at all. The first compares compressed summaries computed before your question existed; the second counts term matches. The head librarian is the only one who ever looks at your question and a document together, which is exactly why she’s better and why she can’t be precomputed.

The precise version

Let D be the dense ranking and S the sparse ranking for a query.

RRF scores document d as ΣL ∈ {D,S} 1/(k + rankL(d)), summed over the lists containing d, with k a small constant (60 is the value from the original formulation) that flattens the top of each list so rank 1 doesn’t dominate rank 2. It is invariant to any monotone transformation of either score, which is exactly the property that makes it robust: it cannot be broken by a rescaling.

DBSF normalises each list using its own score distribution — a standardisation against that list’s mean and spread — then sums. It preserves relative magnitude within a list, at the cost of assuming the distribution is informative. Both assumptions fail on a query where one retriever has nothing good to offer: standardising a list of uniformly poor scores produces high normalised values for its least-bad entries.

Cross-encoder reranking replaces the retrieval score with f(query, document) computed jointly, so it models interactions that a bi-encoder’s independent embeddings cannot represent. Cost is one model evaluation per candidate, so it is linear in candidate count with a large constant — hence a funnel, and hence the shift from optimising recall@10 to recall@100.

Trade-offs

Hybrid against dense alone. Much better on exact terms, identifiers and rare tokens, at the cost of a second index to build and maintain and a more complex query.

RRF against DBSF. RRF needs no tuning and cannot be broken by score scales. DBSF keeps magnitude information and can do better when distributions are well-behaved, and worse when they aren’t.

Fusion against reranking. Fusion is cheap and happens in the database. Reranking is expensive, happens outside it, and is usually a larger quality win. They compose — fusion picks the candidates, reranking orders them.

Prefetch limits. Larger sub-query limits give fusion and reranking more to work with, and cost more per query. This is the real tuning knob in a hybrid system.

A learned sparse model against BM25. SPLADE-style models handle vocabulary mismatch better than BM25 and need a model at index time and query time. BM25 weights are computed from corpus statistics and need no model at all.

Common mistakes

Adding raw scores from the two arms. Cosine similarities and BM25 scores aren’t on the same scale. Whichever has the larger numbers wins every time, and you’ve built an expensive version of single-arm search.

Per-query min-max normalisation. On a query where everything scored badly, the best of a bad list gets normalised to 1.0 and promoted above genuinely good results from the other arm.

Using dense search for identifiers. Order numbers, SKUs, error codes and hashes are what sparse vectors are for. No amount of ef fixes it.

Reranking too few candidates. If retrieval returns 10 and you rerank 10, the reranker can only reorder what was already there. The point is to give it a wide candidate set.

Reranking too many. It’s linear per candidate with a big constant. Several hundred is usually the practical ceiling for an interactive system.

Measuring hybrid search with recall@k against dense ground truth. The whole point is to return things dense search didn’t. Judged against dense’s own neighbours, hybrid looks worse while being better. You need relevance judgements, not nearest neighbours.

Skipping fusion and doing it in the application. It works, and it costs a round trip per arm and moves ranking logic away from the data.

Interview questions

1. Why does dense retrieval struggle with an order number?

Answer Because an embedding encodes meaning, and an identifier has no meaning to encode. The model tokenises it into fragments and produces a vector reflecting the string’s shape, so all similarly-shaped identifiers land in roughly the same region — there’s nothing distinguishing *this* one. Sparse retrieval handles it trivially: a rare term carries high weight, so an exact match dominates. **Follow-up:** would a bigger embedding model fix it? No. It’s a property of representing text by meaning, not of model capacity.

2. Why can’t you just add the dense and sparse scores together?

Answer They’re on incomparable scales with different distributions, and the scales move per query. Cosine sits in a bounded range; BM25-style scores are unbounded and depend on term rarity and document length. Adding them lets one arm dominate by units alone. Normalising per query has its own failure: a query where one arm found nothing good still gets its least-bad results scaled up to the top of the range. **Follow-up:** so what does RRF do instead? Uses only rank, which is invariant to any monotone rescaling of either score.

3. Explain RRF.

Answer Reciprocal Rank Fusion. Each document scores `1 / (k + rank)` from every list it appears in, summed, with a small constant `k` damping the top of each list. It ignores scores entirely, so it can’t be broken by scale differences, needs no tuning, and handles arms of wildly different score distributions. A document appearing in both lists beats one appearing in either, which is the behaviour you want from a hybrid system. The cost is discarding magnitude: an overwhelming match and a marginal one at the same rank contribute identically. **Follow-up:** when would you use DBSF instead? When you’ve measured that magnitude matters on your data and the score distributions are well-behaved.

4. What’s the difference between fusion and reranking?

Answer Fusion combines ranked lists you already have, using rank or normalised scores. It’s cheap, happens in the database, and adds no model. Reranking replaces the ordering using a model that scores the query and each document *together* — a cross-encoder — which is far more accurate than comparing two independently computed embeddings, and far too slow to run over a corpus. They compose: fusion assembles the candidates, reranking orders them. **Follow-up:** why can’t a cross-encoder be indexed? Its score depends on the query and the document jointly, so there’s nothing to precompute before the query arrives.

5. How many candidates should you rerank?

Answer Enough that the right answer is very likely in the set, few enough to afford. Several hundred is the usual interactive ceiling, since cost is linear per candidate with a large constant. The consequence worth stating: with reranking in the pipeline, retrieval should be optimised for recall@100 or recall@200 rather than recall@10. Retrieval’s job is to not miss the answer, not to rank it first. **Follow-up:** how would you choose the number? Measure end-to-end quality against candidate count and find where it stops improving — the same knee argument as Part 4.

6. Where do sparse vector weights come from?

Answer Either classical term weighting computed from corpus statistics — BM25-style, based on term frequency, document length and how rare the term is — or a learned sparse model such as SPLADE, which predicts weights with a neural network and can assign weight to terms not literally present, handling some vocabulary mismatch. Qdrant stores either: it takes indices and values and doesn’t care how you produced them. **Follow-up:** which would you start with? BM25-style weights, because they need no model and no inference at query time. Move to a learned model if measurement justifies it.

7. How do you evaluate whether hybrid search helped?

Answer Not with recall against dense ground truth — hybrid’s whole purpose is returning documents dense search didn’t, so that measurement penalises it for working. You need relevance judgements: a query set with known-good answers, and a metric over the ranked list such as nDCG or MRR. Build the query set from the failures you’re trying to fix — identifiers, rare terms, phrasings that missed — as well as ordinary queries, so you can see whether you improved the hard cases without hurting the easy ones. **Follow-up:** what’s the cheapest useful version? Twenty real queries where you know the right answer, checked by hand before and after. It won’t be statistically strong and it will catch the big regressions.

8. What does prefetch do that two separate queries don’t?

Answer It runs the sub-queries server-side and fuses their results in one round trip, next to the data, instead of shipping two ranked lists to your application to merge. Given [Part 1’s](/qdrant-why-vector-search/) finding that the round trip dominates client-observed latency, halving the number of round trips is a real saving. It also keeps ranking logic in one place. **Follow-up:** can prefetches nest? Yes — a prefetch can have its own prefetch, which is how you build staged retrieval such as a broad first pass narrowed by a second before fusion.

Sources

  • Qdrant documentation, https://qdrant.tech/documentation/concepts/hybrid-queries/ — the Query API, prefetch, and the available fusion methods.
  • models.Fusion on Qdrant 1.19.1 offers exactly RRF and DBSF; the prefetch plus FusionQuery(fusion=Fusion.RRF) call shown here was executed against 1.19.1 and returned fused results.
  • Cormack, G. V., Clarke, C. L. A. and Buettcher, S., Reciprocal Rank Fusion Outperforms Condorcet and Individual Rank Learning Methods, SIGIR 2009 — the origin of RRF and of the constant k = 60.
  • Robertson, S. and Zaragoza, H., The Probabilistic Relevance Framework: BM25 and Beyond, 2009 — the term weighting behind classical sparse vectors.
  • Formal, T., Piwowarski, B. and Clinchant, S., SPLADE: Sparse Lexical and Expansion Model for First Stage Ranking, SIGIR 2021 — learned sparse representations.

What to remember

Dense retrieval finds meaning and misses exact words. Sparse retrieval finds exact words and misses meaning. Store both on the same point and run both.

Combine them with rank, not score. RRF ignores the scores entirely, which is precisely why it works without tuning — the two arms’ numbers were never comparable. DBSF keeps magnitude and needs your distributions to cooperate.

Then, if quality still matters more than latency, put a cross-encoder in front of the results and let retrieval’s job become “don’t miss it” rather than “rank it first”.

Two retrievers that fail on different things are worth more than one retriever that is better at everything. Combine them by rank, because their confidences were never in the same units.

How useful was this post?

Click on a heart to rate it!

Average rating 0 / 5. Vote count: 0

No votes so far! Be the first to rate this post.