Hybrid search runs keyword search (BM25 over an inverted index) and vector search side by side and merges their ranked results into one list. Interviewers probe it because "just use embeddings" is the naive RAG answer, and the mature answer is that each method has a blind spot the other covers: keyword search nails exact tokens (error codes, SKUs, names) but misses paraphrases, while vector search captures meaning but can blur or drop exact identifiers. For a Backend / AI engineer the real questions are how to combine two incomparable score scales, where the combination runs (application or database), and what it costs in latency and complexity.
Query type Keyword (BM25) Vector (embeddings)
---------------------------------- -------------------- ---------------------
"ERR_4021" (exact identifier) strong weak (rare token, no
stable meaning)
"my requests are sluggish" weak (needs the strong (maps to
(paraphrase, corpus says "slow") same word) "slow", "latency")
Product name / person / SKU strong unreliable
Long natural-language question moderate strong
Explainability high (matched terms) low (a distance)
Index inverted index ANN graph / clusters
Handles typos / synonyms no (without extras) often yes-- SQLite FTS5 (real run, see "Try It Yourself" for the full script)
-- The user types "sluggish"; the corpus only contains "slow", "stall", "hang".
query rows_matched
-------- ------------
sluggish 0
slow 1Keyword search is a literal token match, so a word that does not appear in the document simply cannot retrieve it, however close the meaning. Embeddings place "sluggish" near "slow" in vector space, so a semantic query still finds it; but the same embedding smooths away exactly the detail that made an identifier useful, and two different error codes can look nearly identical to the model. Neither is "better": they fail on different queries, which is the whole argument for running both.
score(D, Q) = sum over query terms t of:
IDF(t) * f(t,D) * (k1 + 1)
------------------------------------------
f(t,D) + k1 * (1 - b + b * |D| / avgdl)
f(t,D) = how many times t occurs in document D
IDF(t) = rarer term across the corpus -> bigger weight
k1 = term-frequency saturation (commonly ~1.2): the 10th repeat adds little
b = length normalisation (commonly ~0.75): long documents are discounted
|D| / avgdl = this document's length relative to the corpus average-- SQLite FTS5, real output (bm25() returns lower = better, so negative values)
id title bm25_score
-- -------------------- ----------
2 Timeout tuning guide -1.479
1 Error code reference -1.099Three ideas make BM25 better than counting matches. Rare terms matter more (IDF), so "ERR_4021" outweighs "the". Repeating a word helps with diminishing returns (saturation), so keyword stuffing does not win. Longer documents are penalised, because a long page matches more words by chance. Note that BM25 needs corpus statistics (document frequencies, average length), and that PostgreSQL's built-in full-text ranking (ts_rank, ts_rank_cd) is not BM25 and does not use corpus-wide IDF; true BM25 in Postgres generally comes from an extension, while engines such as Elasticsearch/OpenSearch and SQLite FTS5 provide it directly.
BM25 list (raw score) Vector list (cosine similarity)
1. doc_err4021 14.2 1. doc_timeout_guide 0.83
2. doc_timeout_guide 9.8 2. doc_retry 0.81
3. doc_retry 6.1 3. doc_network 0.79
4. doc_faq 2.3 4. doc_err4021 0.55
BM25 scores are unbounded and corpus-dependent; cosine is in [-1, 1].
Adding them means BM25 dominates. RRF ignores the scores and uses only RANK:
RRF(d) = sum over lists of 1 / (k + rank_in_list(d)) k is commonly 60# Python 3, real output (script in "Try It Yourself")
RRF (k=60):
doc_timeout_guide 0.03252
doc_err4021 0.03202
doc_retry 0.03200
doc_network 0.01587
doc_faq 0.01562
naive raw-score sum:
doc_err4021 14.75
doc_timeout_guide 10.63
doc_retry 6.91
doc_faq 2.30
doc_network 0.79
min-max blend 0.5/0.5:
doc_timeout_guide 0.815
doc_retry 0.624
doc_err4021 0.500
doc_network 0.429
doc_faq 0.000The three strategies disagree, which is the point. The raw sum is essentially BM25 with a small nudge, so the purely semantic doc_network lands last. RRF rewards documents that appear high in both lists (doc_timeout_guide is 2nd and 1st), keeps the identifier document near the top, and does not care that the two scales differ. Min-max normalisation with a weighted blend gives you a tuning knob (the 0.5/0.5 weights) but is sensitive to outliers in each list (one huge BM25 score squashes the rest) and the weights need evaluation data. RRF's strength is that it needs no calibration; its weakness is that it discards score magnitude, so a very confident keyword hit is treated the same as a marginal one at the same rank.
user query
|
+--------+---------+
v v
full-text query embed query
(tsvector + GIN) (vector + HNSW)
| |
top 50 by keyword top 50 by distance
+--------+---------+
v
fuse (RRF, k = 60) <- database or application
v
optional reranker <- cross-encoder on top ~20-50, adds latency
v
top k to the LLM-- PostgreSQL + pgvector (labelled: not run locally, no output shown)
ALTER TABLE chunks ADD COLUMN tsv tsvector
GENERATED ALWAYS AS (to_tsvector('english', content)) STORED;
CREATE INDEX chunks_tsv_gin ON chunks USING gin (tsv);
WITH kw AS ( -- $1 = query text
SELECT id, ROW_NUMBER() OVER (ORDER BY ts_rank_cd(tsv, q) DESC) AS rnk
FROM chunks, websearch_to_tsquery('english', $1) AS q
WHERE tsv @@ q
ORDER BY ts_rank_cd(tsv, q) DESC
LIMIT 50
), vec AS ( -- $2 = query embedding
SELECT id, ROW_NUMBER() OVER (ORDER BY embedding <=> $2) AS rnk
FROM chunks
ORDER BY embedding <=> $2
LIMIT 50
)
SELECT COALESCE(kw.id, vec.id) AS id,
COALESCE(1.0 / (60 + kw.rnk), 0) + COALESCE(1.0 / (60 + vec.rnk), 0) AS rrf
FROM kw FULL OUTER JOIN vec ON kw.id = vec.id
ORDER BY rrf DESC
LIMIT 10;Each retriever runs on its own index (GIN for the text side, HNSW or IVFFlat for the vector side), returns a bounded candidate list, and the FULL OUTER JOIN keeps documents that only one side found; a missing rank simply contributes zero. Retrieving 50 per side and fusing to 10 is a recall-versus-cost choice: fewer candidates is faster but a good document ranked 51st by one method is lost. Doing the fusion in SQL keeps one round trip and lets you apply tenant and permission filters in both CTEs; doing it in the application allows different engines per side (say Elasticsearch plus a vector DB) at the price of two network calls and consistency between two indexes. Because ts_rank_cd is not BM25, teams that need real BM25 in Postgres generally add an extension, or use a search engine for the keyword side.
k1) and length normalisation (b); its scores are unbounded and corpus-dependent, so they cannot be added to cosine similarities directly.1 / (k + rank) across lists (commonly k = 60); it needs no score calibration, but ignores score magnitude.ts_rank / ts_rank_cd are not BM25; real BM25 in Postgres generally needs an extension, while Elasticsearch/OpenSearch and SQLite FTS5 provide it built in.1 / (k + rank) from every list it appears in, and the contributions are summed; documents found by both methods rise to the top.-- hybrid.sql (SQLite with FTS5 compiled in, which most builds include)
-- run: sqlite3 :memory: < hybrid.sql
CREATE VIRTUAL TABLE docs USING fts5(title, body);
INSERT INTO docs(rowid, title, body) VALUES
(1, 'Error code reference', 'ERR_4021 means the upstream gateway closed the connection early'),
(2, 'Timeout tuning guide', 'Raise client timeout and add retries when requests hang or stall'),
(3, 'Retry and backoff', 'Use exponential backoff so retries do not overload a struggling service'),
(4, 'Network basics', 'Packets can be delayed or dropped between hosts causing slow responses'),
(5, 'Billing FAQ', 'How invoices are generated and when payment is due');
-- keyword side: FTS5 bm25() (lower = better; more negative = more relevant)
.headers on
.mode column
SELECT rowid AS id, title, round(bm25(docs), 3) AS bm25_score
FROM docs WHERE docs MATCH 'ERR_4021 OR timeout' ORDER BY bm25(docs);
-- Stand-in for the vector side: a hand-written ranking (NOT computed from real embeddings).
CREATE TABLE vec_rank(id INTEGER, rnk INTEGER);
INSERT INTO vec_rank VALUES (2,1),(3,2),(4,3),(1,4);
-- Reciprocal Rank Fusion in plain SQL, k = 60
WITH kw AS (
SELECT rowid AS id, ROW_NUMBER() OVER (ORDER BY bm25(docs)) AS rnk
FROM docs WHERE docs MATCH 'ERR_4021 OR timeout'
), fused AS (
SELECT id, 1.0/(60+rnk) AS s FROM kw
UNION ALL
SELECT id, 1.0/(60+rnk) FROM vec_rank
)
SELECT id, round(SUM(s), 5) AS rrf_score, COUNT(*) AS lists_present_in
FROM fused GROUP BY id ORDER BY rrf_score DESC;
-- paraphrase check: the user types "sluggish", the corpus says "slow" / "stall" / "hang"
SELECT 'sluggish' AS query, count(*) AS rows_matched FROM docs WHERE docs MATCH 'sluggish';
SELECT 'slow' AS query, count(*) AS rows_matched FROM docs WHERE docs MATCH 'slow';-- Real output of the script above
id title bm25_score
-- -------------------- ----------
2 Timeout tuning guide -1.479
1 Error code reference -1.099
id rrf_score lists_present_in
-- --------- ----------------
2 0.03279 2
1 0.03175 2
3 0.01613 1
4 0.01587 1
query rows_matched
-------- ------------
sluggish 0
query rows_matched
----- ------------
slow 1The vector side in that script is a hand-written ranking table standing in for a real embedding search (it is labelled as such in the code); the keyword side and the RRF arithmetic are real. The Python version below shows the fusion strategies compared; its output is in the previous section.
# rrf.py - run: python3 rrf.py
from collections import defaultdict
# Two ranked lists for the query "ERR_4021 timeout" (best first).
# Their raw scores live on different scales, which is the whole problem.
bm25 = [("doc_err4021", 14.2), ("doc_timeout_guide", 9.8), ("doc_retry", 6.1), ("doc_faq", 2.3)]
vector = [("doc_timeout_guide", 0.83), ("doc_retry", 0.81), ("doc_network", 0.79), ("doc_err4021", 0.55)]
def rrf(lists, k=60):
score = defaultdict(float)
for lst in lists:
for rank, (doc, _) in enumerate(lst, start=1):
score[doc] += 1.0 / (k + rank)
return sorted(score.items(), key=lambda kv: -kv[1])
print("RRF (k=60):")
for doc, s in rrf([bm25, vector]):
print(f" {doc:20} {s:.5f}")
# Naive alternative: add raw scores. BM25's larger scale swamps the cosine similarity.
raw = defaultdict(float)
for lst in (bm25, vector):
for doc, s in lst: raw[doc] += s
print("naive raw-score sum:")
for doc, s in sorted(raw.items(), key=lambda kv: -kv[1]):
print(f" {doc:20} {s:.2f}")
# Weighted alternative: min-max normalise each list then blend 0.5/0.5
def mm(lst):
lo, hi = min(s for _, s in lst), max(s for _, s in lst)
return {d: (s - lo) / (hi - lo) for d, s in lst}
a, b = mm(bm25), mm(vector)
blend = {d: 0.5 * a.get(d, 0) + 0.5 * b.get(d, 0) for d in set(a) | set(b)}
print("min-max blend 0.5/0.5:")
for d, s in sorted(blend.items(), key=lambda kv: -kv[1]):
print(f" {d:20} {s:.3f}")-- PostgreSQL + pgvector: run the hybrid CTE from the previous section on your own
-- chunks table. No output is shown because it was not executed locally. Things to try:
-- 1. Query an exact identifier from your data, compare vector-only, keyword-only, fused.
-- 2. Change the per-side LIMIT (50) to 10 and see which relevant rows disappear.
-- 3. Replace the RRF constant 60 with 10 and watch how top-rank documents gain weight.Q: Vector search vs keyword search: what is the difference and when does each win?
A: Keyword search matches literal tokens via an inverted index and scores with BM25 (rare terms, term frequency and length), so it wins on exact identifiers, names and jargon and is easy to explain. Vector search compares embeddings in a learned space, so it wins on paraphrase and natural-language questions ("sluggish" finds "slow") but can blur exact strings and costs an embedding call plus an ANN index. The trade-off is precision on exact terms versus recall on meaning; most real corpora contain both query types.
Q: What is hybrid search?
A: It runs a keyword retriever and a vector retriever on the same query, then merges their ranked results, commonly with Reciprocal Rank Fusion, optionally followed by a reranker. Because BM25 and cosine scores are on different scales, rank-based fusion avoids calibrating them. It generally improves recall over either method alone, but you pay for two indexes, two queries per request and more moving parts to evaluate and tune.
Q: Why not just add the BM25 score and the cosine similarity?
A: BM25 scores are unbounded and depend on corpus statistics and query length, while cosine similarity is bounded, so the sum is dominated by whichever has the larger scale (in the demo the raw sum pushed the semantic-only document to last). Options are RRF, which uses only ranks, or normalising each list (min-max, z-score) and blending with weights. Normalised blending is tunable but sensitive to outliers and needs labelled queries to set weights; RRF is robust but throws away magnitude.
Follow-up: Why is the RRF constant k often 60, and what does changing it do?
A: The k in 1 / (k + rank) controls how steeply the contribution drops with rank: a small k makes rank 1 far more valuable than rank 5, a large k flattens the difference and lets agreement between lists matter more. The value 60 is a common default from the original RRF paper and works reasonably across datasets, but it is a tunable, not a law, so validate it on your own evaluation set.
Follow-up: Is PostgreSQL ts_rank the same as BM25?
A: No. ts_rank and ts_rank_cd score using term frequency, position and optional length normalisation of a single document, but they do not use corpus-wide inverse document frequency, so a rare term is not weighted above a common one the way BM25 does. If ranking quality on the keyword side matters, use a BM25-capable extension or a search engine; if the corpus is small and the keyword side only supplies identifier matches, ts_rank_cd is often acceptable, but measure it.
Follow-up: Hybrid search made latency worse and quality only slightly better. What would you look at?
A: Run the two retrievers in parallel rather than sequentially, reduce per-side candidate counts, and drop the reranker or shrink its input if it dominates latency. Then check with an evaluation set whether the query mix actually contains identifier-style queries; if almost all queries are natural language, vector-only plus a good embedding model may be enough, and the extra index is not worth its operational cost. The trade-off is a measurable quality gain per query type versus permanent complexity, so decide from data.
In a RAG or enterprise-search design, hybrid retrieval is usually the recall stage of a multi-stage pipeline: cheap parallel retrievers produce a few dozen candidates, RRF fuses them, an optional cross-encoder reranks the top handful, and only then does the LLM see context. Keeping both indexes in PostgreSQL gives transactional consistency between the text, its embedding and metadata such as tenant and document version, and lets the same WHERE clause enforce permissions on both sides; separating into a search engine plus a vector store scales each independently but introduces dual-write or change-data-capture pipelines and the possibility that the two indexes briefly disagree. Design the evaluation loop alongside the retrieval: a labelled query set that includes identifier-style, paraphrase and multi-hop questions, with metrics reported per query type, so you can justify (or remove) each stage. Filters and access control must be applied inside both retrievers, never after fusion, otherwise the fused top-k can silently drop authorised results or leak unauthorised ones into the reranker.
The decision is to pay for a second retrieval path and a fusion step in exchange for robustness across query types. The cost is more infrastructure (two indexes to build, update and keep consistent), higher tail latency (the slower of two retrievers plus fusion and any reranker), and more parameters to tune (per-side candidate count, RRF constant or blend weights, reranker cut-off), each of which needs evaluation data. It is the wrong choice when queries are uniformly semantic and a good embedding model already meets the quality bar, when the corpus is small enough that a single method plus a reranker is simpler, or when latency budgets are tight and the gain is small; conversely, a corpus full of codes, names and jargon with keyword-only search is missing the paraphrase side, and pure vector search will miss its exact matches.