Hybrid Search: Combining BM25 and Vector Search

Vector search excels at meaning; BM25 excels at exact terms. A user searching "error 503" wants the doc containing "503", not the semantically closest paragraph. A user searching "how do I fix my connection" wants meaning, not keyword matches. Hybrid search runs both and fuses the results, and it consistently beats either approach alone on real-world retrieval quality.

Why Hybrid Beats Either Alone

Scenario	BM25	Vector
Exact identifiers (error codes, SKUs)	Strong	Weak
Rare technical terms	Strong	Weak
Synonyms and paraphrases	Weak	Strong
Cross-language queries	Weak	Strong
Typos and misspellings	Weak	Strong

The failure modes are complementary: BM25 misses paraphrases, vectors miss exact tokens. Fusing them covers both.

Reciprocal Rank Fusion (RRF)

The standard fusion method is RRF: combine rankings by reciprocal rank, which is robust to score scale differences (BM25 scores and cosine similarities are not comparable).

RRF(d) = Σ 1 / (k + rank_i(d))

def rrf_fuse(result_sets: list[list[str]], k: int = 60) -> list[str]:
    """Fuse ranked document lists from multiple retrievers."""
    scores: dict[str, float] = {}
    for results in result_sets:
        for rank, doc_id in enumerate(results, start=1):
            scores[doc_id] = scores.get(doc_id, 0.0) + 1.0 / (k + rank)

    return sorted(scores, key=scores.get, reverse=True)

The constant k (commonly 60) dampens the influence of high ranks. RRF needs no score normalization, which is its main advantage: you can fuse a BM25 index and a vector index with zero calibration.

Hybrid Search with Elasticsearch

Elasticsearch supports both BM25 and vector search natively. Define an index with a dense vector field alongside the text:

{
  "mappings": {
    "properties": {
      "title": { "type": "text" },
      "body": { "type": "text" },
      "embedding": {
        "type": "dense_vector",
        "dims": 1536,
        "index": true,
        "similarity": "cosine"
      }
    }
  }
}

Query both in one request and fuse with RRF:

from elasticsearch import Elasticsearch

es = Elasticsearch("http://localhost:9200")

def hybrid_search(query: str, query_embedding: list[float], top_k: int = 10):
    bm25_hits = es.search(index="docs", query={"match": {"body": query}}, size=top_k)
    vector_hits = es.search(
        index="docs",
        knn={"field": "embedding", "query_vector": query_embedding, "k": top_k},
        size=top_k,
    )

    bm25_ids = [h["_id"] for h in bm25_hits["hits"]["hits"]]
    vector_ids = [h["_id"] for h in vector_hits["hits"]["hits"]]
    fused = rrf_fuse([bm25_ids, vector_ids])

    # Map fused ids back to documents
    docs = {h["_id"]: h["_source"] for h in bm25_hits["hits"]["hits"]}
    docs.update({h["_id"]: h["_source"] for h in vector_hits["hits"]["hits"]})
    return [docs[doc_id] for doc_id in fused[:top_k]]

Hybrid Search with Weaviate

Weaviate has hybrid search built in, with configurable fusion:

import weaviate

client = weaviate.connect_to_local()

def hybrid_search(query: str, top_k: int = 10):
    response = client.collections.get("Docs").query.hybrid(
        query=query,          # BM25 part
        vector=embed(query),  # vector part
        limit=top_k,
        fusion_type="relativeScoreFusion",  # or "rankedFusion" (RRF)
        alpha=0.5,  # 0 = pure BM25, 1 = pure vector
    )
    return [obj.properties for obj in response.objects]

The alpha parameter weights the two signals. Start at 0.5 and tune on your labeled set.

Weighted Score Fusion (Alternative to RRF)

If both retrievers produce comparable scores (e.g. both normalized to 0-1), weighted linear fusion can outperform RRF:

def weighted_fusion(
    bm25_results: dict[str, float],
    vector_results: dict[str, float],
    alpha: float = 0.5,
) -> list[str]:
    scores: dict[str, float] = {}
    for doc_id, score in bm25_results.items():
        scores[doc_id] = scores.get(doc_id, 0.0) + (1 - alpha) * score
    for doc_id, score in vector_results.items():
        scores[doc_id] = scores.get(doc_id, 0.0) + alpha * score
    return sorted(scores, key=scores.get, reverse=True)

This requires score normalization (min-max or z-score per retriever), which RRF avoids. Use RRF for simplicity, weighted fusion when you have calibration data.

Tuning the Blend

The optimal blend depends on your corpus:

• Code and identifiers: lean BM25 (alpha 0.3-0.4).
• Natural language docs: lean vector (alpha 0.6-0.7).
• Mixed corpora: start at 0.5 and measure.

Tune against your labeled query set with recall@k and MRR, the same metrics as pure vector search:

def tune_alpha(queries, alphas=[0.3, 0.4, 0.5, 0.6, 0.7]):
    best = None
    for alpha in alphas:
        mrr_score = mean(
            mrr(hybrid_search(q, alpha=alpha), relevant[q]) for q in queries
        )
        if best is None or mrr_score > best[1]:
            best = (alpha, mrr_score)
    return best

Query-Time Strategy

Not every query needs both retrievers. A cheap classifier can route: queries with identifiers, codes, or rare tokens go BM25-heavy; natural language goes vector-heavy. This saves latency on the majority of queries.

def route_query(query: str) -> float:
    """Return alpha: 0 = pure BM25, 1 = pure vector."""
    if re.search(r"\b[A-Z]{2,}-\d+\b|\b\d{3,}\b", query):  # codes/IDs
        return 0.2
    if len(query.split()) <= 2:  # short keyword queries
        return 0.3
    return 0.6

Implementation Checklist

• Run BM25 and vector search in parallel
• Fuse with RRF (no calibration needed) or weighted fusion (with calibration)
• Use alpha 0.5 as the starting blend
• Tune alpha against recall@k and MRR on a labeled set
• Route identifier-heavy queries toward BM25
• Keep both indexes in sync on every document change
• Monitor per-retriever hit rates to detect drift

---

MatterAI builds frontier AI infrastructure for engineering teams — from inference-optimized models to autonomous coding agents and agentic code reviews.

Explore what we're building:

• Orbital IDE — Autonomous AI coding agent with background agents and deep codebase memory
• AI Code Reviews — Agentic pre-commit reviews across GitHub, GitLab, and Bitbucket
• Axon Models — Frontier-grade reasoning models at 70% lower inference cost

Get started free - https://app.matterai.so

• Read the docs
• View pricing

---

Follow us on X · LinkedIn · GitHub