Hybrid search

Two ways into the same idea

Before You Start

The Core Idea

You have two ways to search, and each one misses where the other hits. In this lesson you build each by hand, fuse them, then add a final, more precise stage.

1Keyword search: BM25

BM25 gives each document points for the words it shares with the question. A rare word is worth more than a common one, and repeats help, but less each time.

import math
from collections import Counter

docs = [
    "يحق للموظف إجازة سنوية مدتها واحد وعشرون يوم عمل في السنة.",
    "تقدم طلبات الإجازة السنوية قبل أسبوعين على الأقل من موعدها.",
    "الإجازة المرضية تحتاج تقريرا طبيا إذا زادت عن يومين متتاليين.",
    "يوافق المدير المباشر على الإجازة المرضية والسنوية.",
    "يمكن ترحيل عشرة أيام من الإجازة السنوية إلى السنة التالية فقط.",
    "تصرف بدلات السفر خلال عشرة أيام من تقديم الفواتير.",
    "يجب حجز تذاكر السفر عبر بوابة الحجز الداخلية.",
    "ساعات العمل من الثامنة صباحا حتى الرابعة عصرا.",
    "العمل عن بعد مسموح يومين في الأسبوع بموافقة المدير.",
    "رمز الخطأ E-1043 في بوابة الحجز يعني انتهاء الجلسة.",
]

def words(text):
    return [w.strip(".،؟") for w in text.split()]

tokenized = [words(d) for d in docs]
avg_len = sum(map(len, tokenized)) / len(tokenized)
df = Counter(w for doc in tokenized for w in set(doc))

def bm25(query, k1=1.5, b=0.75):
    scores = []
    for doc in tokenized:
        tf = Counter(doc)
        s = 0.0
        for w in words(query):
            if w not in tf:
                continue
            idf = math.log(1 + (len(docs) - df[w] + 0.5) / (df[w] + 0.5))
            s += idf * tf[w] * (k1 + 1) / (tf[w] + k1 * (1 - b + b * len(doc) / avg_len))
        scores.append(s)
    return scores

for query in ["ماذا يعني E-1043؟", "متى تصرف بدلات السفر؟"]:
    s = bm25(query)
    best = max(range(len(docs)), key=s.__getitem__)
    print(f"{query}\n   {s[best]:.2f}  [{best}] {docs[best]}")

E-1043 appears in only one document, so its weight (idf) is high, and BM25 found it immediately. This is what keyword search does well: codes, numbers, names and exact terms. But ask "how many days do I rest each year?" and it finds nothing, because no word is shared with "annual leave".

2Fusing by rank: RRF

Now you have two ranked lists for the same question, and you want one. Look at the picture: both lists go in, and out comes one list where what they agree on rises.

BM25 scores and vector similarities are on different scales and cannot be added directly. RRF (Reciprocal Rank Fusion) ignores the scores and uses only ranks: each document gets 1 / (60 + its rank) from each list.

def rrf(rankings, k=60):
    scores = {}
    for ranking in rankings:
        for rank, doc in enumerate(ranking, start=1):
            scores[doc] = scores.get(doc, 0) + 1 / (k + rank)
    return sorted(scores, key=scores.get, reverse=True)

keyword = [9, 6, 5, 2]      # BM25 found the exact error code first
vector = [6, 5, 9, 7]       # the vector search put it third
print("keyword:", keyword)
print("vector: ", vector)
print("fused:  ", rrf([keyword, vector]))

Document 6 came first because it is high in both lists (second and first), and 9 second for being first in one and third in the other. RRF rewards agreement: what both systems agree on rises. The constant 60 stops a first place in one list from dominating everything.

3Why both together

We ran all three methods on the same ten questions. Look at the picture: each method alone got 5 of 10 right at the top, and the hybrid got 6.

Ten golden questions, three about exact codes and the rest paraphrased. The vector stand-in here is words compressed to six "meaning" dimensions.

import math
from collections import Counter
import numpy as np
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.decomposition import TruncatedSVD

docs = [
    "يحق للموظف إجازة سنوية مدتها واحد وعشرون يوم عمل في السنة.",
    "تقدم طلبات الإجازة السنوية قبل أسبوعين على الأقل من موعدها.",
    "الإجازة المرضية تحتاج تقريرا طبيا إذا زادت عن يومين متتاليين.",
    "يوافق المدير المباشر على الإجازة المرضية والسنوية.",
    "يمكن ترحيل عشرة أيام من الإجازة السنوية إلى السنة التالية فقط.",
    "تصرف بدلات السفر خلال عشرة أيام من تقديم الفواتير.",
    "يجب حجز تذاكر السفر عبر بوابة الحجز الداخلية.",
    "ساعات العمل من الثامنة صباحا حتى الرابعة عصرا.",
    "العمل عن بعد مسموح يومين في الأسبوع بموافقة المدير.",
    "رمز الخطأ E-1043 في بوابة الحجز يعني انتهاء الجلسة.",
    "رمز الخطأ E-2210 في بوابة الحجز يعني أن البطاقة مرفوضة.",
    "يستخدم النموذج HR-22 لطلب شهادة راتب من الموارد البشرية.",
]

def words(text):
    return [w.strip(".،؟") for w in text.split()]

tokenized = [words(d) for d in docs]
avg_len = sum(map(len, tokenized)) / len(tokenized)
df = Counter(w for d in tokenized for w in set(d))

def bm25_rank(query, k1=1.5, b=0.75):
    scores = []
    for d in tokenized:
        tf = Counter(d)
        scores.append(sum(math.log(1 + (len(docs) - df[w] + 0.5) / (df[w] + 0.5))
                          * tf[w] * (k1 + 1) / (tf[w] + k1 * (1 - b + b * len(d) / avg_len))
                          for w in words(query) if w in tf))
    return [i for i in np.argsort(scores)[::-1] if scores[i] > 0]

# a stand-in for an embedding model: word TF-IDF compressed to 6 "meaning" dimensions
tfidf = TfidfVectorizer(analyzer="word", token_pattern=r"[^\s.،؟]+")
svd = TruncatedSVD(n_components=6, random_state=0)
vecs = svd.fit_transform(tfidf.fit_transform(docs))
vecs /= np.linalg.norm(vecs, axis=1, keepdims=True)

def vector_rank(query):
    q = svd.transform(tfidf.transform([query]))[0]
    return list(np.argsort(-(vecs @ (q / (np.linalg.norm(q) + 1e-9)))))

def rrf(rankings, k=60):
    s = {}
    for r in rankings:
        for rank, d in enumerate(r, 1):
            s[d] = s.get(d, 0) + 1 / (k + rank)
    return sorted(s, key=s.get, reverse=True)

golden = [("ماذا يعني E-1043؟", 9), ("ما معنى E-2210؟", 10), ("أين أجد HR-22؟", 11), ("كم يوما أرتاح كل عام؟", 0),
          ("متى أطلب الإجازة؟", 1), ("من يعتمد إجازتي المرضية؟", 3),
          ("هل أستطيع العمل من البيت؟", 8), ("متى يبدأ الدوام؟", 7),
          ("كيف أحجز تذكرة طيران؟", 6), ("متى أستلم مال الفواتير؟", 5)]

for name, fn in [("keyword (BM25)", bm25_rank), ("vector", vector_rank),
                 ("hybrid (RRF)", lambda q: rrf([bm25_rank(q), vector_rank(q)]))]:
    top1 = sum(int(target in fn(q)[:1]) for q, target in golden)
    top3 = sum(int(target in fn(q)[:3]) for q, target in golden)
    print(f"{name:15} top-1: {top1}/{len(golden)}   top-3: {top3}/{len(golden)}")
print("vector search for 'E-1043' ranks first:", docs[vector_rank("ماذا يعني E-1043؟")[0]])

Each method alone got 5 of 10 in first place, but on different questions, so hybrid got 6. And look at the last line: vector search for E-1043 returned the E-2210 document, because the two codes sit in very similar sentences. This is exactly the mistake a real embedding model makes with order numbers and product codes, and BM25 never makes it.

The numbers are small because the set is ten questions. In your project, run the same comparison on fifty golden questions and adopt hybrid only if it wins, which it does in most published cases.

The two methods got different questions right: keywords find exact codes like E-1043, and vectors understand rephrasing. Fusing takes the best of both.

4Reranking at the end

A reranking model reads the question and the document together, so it judges relevance far better than comparing two vectors. But it is slow, so give it only a few.

Each method fetches 50 candidates quickly, RRF fuses them, the reranker reads only the top 20, and 3 to 5 go into the context.

How It Works, Step by Step: how an RRF ranking is computed

Worked example

BM25 is the algorithm that has powered keyword search engines for decades. Every word shared by the question and the document adds points, rare words add more than common ones, repetition helps with diminishing returns, and long documents are penalised slightly.

Example: Hybrid inside Postgres

WITH kw AS (
  SELECT id, row_number() OVER (ORDER BY ts_rank(tsv, query) DESC) AS r
  FROM chunks, plainto_tsquery('simple', $1) query
  WHERE tsv @@ query
  LIMIT 50
), vec AS (
  SELECT id, row_number() OVER (ORDER BY embedding <=> $2) AS r
  FROM chunks
  ORDER BY embedding <=> $2
  LIMIT 50
)
SELECT id, sum(1.0 / (60 + r)) AS score
FROM (SELECT * FROM kw UNION ALL SELECT * FROM vec) both_lists
GROUP BY id
ORDER BY score DESC
LIMIT 20;

$1 is the question text for full-text search and $2 its vector. The simple configuration is used because Postgres's built-in language configurations do not cover Arabic, so you normalise the text yourself before storing tsv (next lesson). The top 20 from here go to the reranker.

Try it without code

Myth vs Reality

Try it yourself

Quick review cards

Check Your Understanding