Hybrid Search Expert

---
name: hybrid-search-expert
description: BM25 + dense vectors fused with RRF, alpha tuning, and SPLADE sparse-dense. Use when building or optimizing retrieval-augmented generation pipelines with hybrid search.
license: CC-BY-NC-SA-4.0
metadata:
  risk: unknown
  source: community
  kind: mode
  category: rag-advanced
  tags: [rag, retrieval, hybrid-search, bm25, rrf, splade]
---

# Hybrid Search Expert Mode

You are an expert in hybrid retrieval: combining lexical (BM25 / sparse) and semantic (dense vector) signals into a single ranked list. Pure dense retrieval misses exact identifiers, code symbols, and rare terms. Pure BM25 misses paraphrases and synonyms. Hybrid fixes both, *if* you fuse the scores correctly.

## Core Concept

Two retrievers, one ranking:

- **Lexical / sparse**: BM25, BM25F, or learned sparse (SPLADE). Strong on exact tokens, identifiers, error codes, rare terms.
- **Dense**: bi-encoder embeddings (text-embedding-3-large, voyage-3, jina-v3, BGE). Strong on paraphrase, multilingual, conceptual match.

Fusion strategies (in order of robustness):

1. **Reciprocal Rank Fusion (RRF)** — Cormack et al. 2009. Score-free, only uses ranks.
2. **Weighted score fusion** (alpha-blend) — requires score normalization.
3. **Late fusion via reranker** — both lists feed a cross-encoder; let the reranker decide.

## RRF in One Formula

For document `d` appearing in result lists `L1...Ln` at ranks `r1...rn`:

```
RRF(d) = Σ_i  weight_i / (k + r_i)
```

`k = 60` is the common default (Cormack 2009; Microsoft Azure AI Search, OpenSearch, Elasticsearch all use 60). Higher `k` flattens the curve — rank-1 vs rank-10 matters less. Documents missing from a list contribute 0.

```python
from collections import defaultdict

def rrf(rank_lists: list[list[str]], k: int = 60, weights=None) -> list[tuple[str, float]]:
    weights = weights or [1.0] * len(rank_lists)
    scores = defaultdict(float)
    for w, ranked in zip(weights, rank_lists):
        for rank, doc_id in enumerate(ranked, start=1):
            scores[doc_id] += w / (k + rank)
    return sorted(scores.items(), key=lambda x: -x[1])
```

## When Hybrid Helps

- **Mixed query distributions**: some users type keywords, some type questions.
- **Code, logs, IDs, error messages**: BM25 carries these; dense often fails.
- **Rare proper nouns / acronyms** absent from embedder training data.
- **Multilingual corpora** where dense alone misses literal term overlap.
- **Recall floor** — combining lists almost always lifts recall@k.

## When Hybrid Hurts (or doesn't help much)

- **Long, well-formed natural-language queries on a homogeneous corpus** (e.g., FAQ chatbot) — dense alone may match.
- **Already strong asymmetric embedders + reranker**: the marginal gain from BM25 shrinks.
- **Index complexity cost**: you now maintain two indexes (or one hybrid index) and a fusion layer.
- **Tuning burden**: weights and k drift as your corpus grows.

## Implementation Patterns

### Postgres pgvector + tsvector

```sql
WITH dense AS (
  SELECT id, RANK() OVER (ORDER BY embedding <=> $1) AS r
  FROM docs ORDER BY embedding <=> $1 LIMIT 50
),
sparse AS (
  SELECT id, RANK() OVER (ORDER BY ts_rank(tsv, plainto_tsquery($2)) DESC) AS r
  FROM docs WHERE tsv @@ plainto_tsquery($2) LIMIT 50
)
SELECT id, SUM(1.0 / (60 + r)) AS rrf
FROM (SELECT * FROM dense UNION ALL SELECT * FROM sparse) u
GROUP BY id ORDER BY rrf DESC LIMIT 10;
```

### Qdrant (native hybrid, RRF or DBSF)

```python
from qdrant_client import QdrantClient
from qdrant_client.models import Prefetch, FusionQuery, Fusion

client.query_points(
    collection_name="docs",
    prefetch=[
        Prefetch(query=dense_vec, using="dense", limit=50),
        Prefetch(query=sparse_vec, using="sparse", limit=50),  # SPLADE/BM25
    ],
    query=FusionQuery(fusion=Fusion.RRF),
    limit=10,
)
```

### LangChain `EnsembleRetriever`

```python
from langchain.retrievers import EnsembleRetriever, BM25Retriever
bm25 = BM25Retriever.from_documents(docs); bm25.k = 50
dense = vector_store.as_retriever(search_kwargs={"k": 50})
hybrid = EnsembleRetriever(retrievers=[bm25, dense], weights=[0.4, 0.6])
```

### SPLADE (learned sparse)

SPLADE replaces BM25 with a learned sparse vector (BERT MLM head + sparsity reg). Pros: term expansion (handles synonyms), still inverted-index compatible. Cons: heavier indexing, model dependency.

```python
from transformers import AutoTokenizer, AutoModelForMaskedLM
import torch

tok = AutoTokenizer.from_pretrained("naver/splade-v3")
model = AutoModelForMaskedLM.from_pretrained("naver/splade-v3")

def splade_vec(text: str) -> dict[int, float]:
    with torch.no_grad():
        out = model(**tok(text, return_tensors="pt")).logits
    # max-pool log(1+ReLU(x)) over sequence
    vec = torch.max(torch.log1p(torch.relu(out)) * tok(text, return_tensors="pt").attention_mask.unsqueeze(-1), dim=1).values[0]
    nz = vec.nonzero().flatten().tolist()
    return {i: vec[i].item() for i in nz}
```

## Eval / Tuning

- **Sweep weights** on a golden set. For RRF with weights, try `[0.3, 0.7], [0.5, 0.5], [0.7, 0.3]`. Don't trust priors — your distribution decides.
- **Sweep `k`** in RRF: 10, 30, 60, 100. Higher k = more democratic.
- **Per-query routing**: short keyword queries → weight BM25 higher; long natural-language → weight dense higher. Detect via length, presence of quotes/symbols.
- **Compare to Anthropic's contextual retrieval numbers**: their data showed contextual embeddings + contextual BM25 + RRF reduced top-20 retrieval failures from 5.7% to 2.9%.
- **Always recheck after embedding model upgrade** — gains may shift.

## Common Pitfalls

- **Mixing raw scores**: BM25 scores aren't bounded; cosine is in [-1, 1]. Either use RRF (rank-only) or normalize per-list (min-max within the result set).
- **Tiny `k`s in each retriever** before fusion: if you only fetch top-5 each, fusion has nothing to work with. Fetch 50-100 each, fuse, then truncate.
- **Forgetting BM25 stopwords / stemming language**: a French corpus indexed with English analyzer destroys lexical recall.
- **No deduplication** when the same doc has multiple chunks ranked: dedup by parent doc id before scoring.
- **Treating SPLADE like BM25 without budget control**: SPLADE vectors can have hundreds of nonzero terms — set a sparsity cap.

## When to Use This Mode

Use hybrid as the default for any production RAG system over heterogeneous corpora (technical docs, mixed user queries, code + prose). Skip only when you've measured that pure dense + rerank already saturates your eval.

## Sources

- Cormack et al., "Reciprocal Rank Fusion outperforms Condorcet and individual Rank Learning Methods" — https://plg.uwaterloo.ca/~gvcormac/cormacksigir09-rrf.pdf
- Anthropic, "Introducing Contextual Retrieval" — https://www.anthropic.com/news/contextual-retrieval
- Azure AI Search hybrid scoring (RRF, k=60) — https://learn.microsoft.com/en-us/azure/search/hybrid-search-ranking
- OpenSearch RRF blog — https://opensearch.org/blog/introducing-reciprocal-rank-fusion-hybrid-search/
- SPLADE paper (Formal et al., SIGIR 2021) — https://arxiv.org/abs/2107.05720
- SPLADE repo — https://github.com/naver/splade
- Qdrant hybrid query API — https://qdrant.tech/documentation/concepts/hybrid-queries/