The retrieval step of RAG usually asks the vector index for the top-k chunks most similar to the question. That sounds right and often isn't: documents repeat themselves, chunking with overlap creates near-copies, and the k "best" chunks frequently say the same thing from the same section. Maximal Marginal Relevance (MMR) fixes this by choosing chunks one at a time, rewarding relevance to the query but penalising similarity to what has already been chosen. The result is a context window that covers more ground with the same token budget. It is a small change (one parameter in most vector-store libraries) with a real effect on answer quality for broad questions, which is why interviewers like it as a follow-up to "how does your retriever work?". Related pages: chunking and embeddings, vector databases, ANN search and hybrid search.
Plain similarity search ranks every candidate by one number, its similarity to the query, and keeps the top k. MMR ranks by marginal value: how relevant a chunk is, minus how much it repeats something already selected. The first pick is always the most relevant chunk; after that, a slightly less relevant chunk that adds a new angle can beat a highly relevant chunk that is a near-duplicate of pick number one.
In simple words: if you ask five friends for restaurant advice and four of them just read you the same review, you got one opinion, not five. MMR makes sure each new voice adds something.
An employee asks the internal assistant "what's our data privacy policy?". The policy document was chunked with overlap, so the four most similar chunks are four slices of section 2 ("data we collect"). The model gets four copies of the same paragraph and answers only about collection, never mentioning cookies, consent or the incident-reporting process, even though those chunks were in the index and only slightly less similar. With MMR the four chunks become: the best policy section, the cookie notice, the consent requirements and the incident report, and the answer covers all four.
MMR picks, one at a time, the candidate d (not yet selected) that maximises
score(d) = λ · sim(d, query) − (1 − λ) · max over s in Selected of sim(d, s)
\________________/ \_______________________________________/
relevance redundancy with what we already picked
λ = 1.0 → pure relevance: identical to plain top-k similarity search
λ = 0.0 → pure diversity: ignores the query after the first pick (useless alone)
λ ≈ 0.5–0.7 → the usual range; LangChain's default lambda_mult is 0.5Two details matter in practice. First, MMR runs over a candidate pool, not the whole index: fetch the top fetch_k (for example 20–50) by similarity, then pick k of them with MMR. That keeps it cheap and stops it from reaching for irrelevant chunks just because they are different. Second, redundancy is measured between chunk embeddings, so MMR needs the vectors of the candidates, not just their scores.
Using the numbers from the program below (λ = 0.7). Step 1 has nothing selected yet, so the most relevant chunk wins: policy §2.4 (0.875). Step 2 re-scores every remaining candidate against that pick:
candidate relevance sim to picked 0.7·rel − 0.3·sim
policy §2.2 (data we collect) 0.854 0.998 0.298
policy §2.1 (data we collect) 0.841 0.997 0.290
policy §2.3 (data we collect) 0.841 0.997 0.289
cookie & tracking notice 0.657 0.406 0.338 ← picked
GDPR consent requirements 0.686 0.486 0.334
2023 breach incident report 0.630 0.397 0.322The policy sections are more relevant but almost identical to the chunk already chosen, so their redundancy penalty sinks them. Steps 3 and 4 repeat the same calculation, where "sim to picked" is the maximum similarity to any chunk selected so far, and pick GDPR consent and then the breach report. Each step is O(candidates × selected), so for k = 5 over a pool of 50 it is a few hundred cosine similarities: negligible next to the LLM call.
Technique Fixes Costs
------------------- -------------------------------------- --------------------------------
MMR redundancy among the top-k may drop a 2nd genuinely-needed chunk
Cross-encoder rerank precision: is this chunk really about one model call per candidate
the question? (reads query + chunk) (tens of ms per batch)
Hybrid (BM25+vector) recall: exact terms, IDs, error codes a second index to maintain
Dedup at ingestion exact / near-duplicate documents misses overlap created by chunkingThey stack rather than compete. A strong pipeline often looks like: hybrid search for recall → rerank for precision → MMR (or a simple near-duplicate filter) on the reranked list for diversity → top k into the prompt. If you only add one thing to a naive retriever, measure first: when answers miss facets, MMR helps; when answers use the wrong passages, reranking helps; when exact identifiers are missed, hybrid search helps.
fetch_k larger than k), never the whole index.fetch_k candidates by similarity (with their vectors).A standard-library program with tiny hand-made "embeddings" over four topic axes, so every number can be checked by hand. It compares plain top-4 search with MMR at three values of λ.
# MMR vs plain similarity search, standard library only.
# Vectors are tiny hand-made "embeddings" over 4 topic axes so the numbers are easy to follow:
# [privacy-policy, cookies/tracking, breaches/incidents, consent/law]
import math
def cos(a, b):
dot = sum(x * y for x, y in zip(a, b))
return dot / (math.sqrt(sum(x * x for x in a)) * math.sqrt(sum(y * y for y in b)))
query = [1.0, 0.5, 0.5, 0.5] # "data privacy": mostly policy, a bit of everything
chunks = {
"policy §2.1 (data we collect)": [1.0, 0.10, 0.05, 0.10],
"policy §2.2 (data we collect II)":[0.98, 0.12, 0.05, 0.12],
"policy §2.3 (data we collect III)":[0.97, 0.08, 0.06, 0.10],
"policy §2.4 (retention)": [0.95, 0.10, 0.10, 0.15],
"cookie & tracking notice": [0.30, 1.00, 0.05, 0.20],
"2023 breach incident report": [0.30, 0.05, 1.00, 0.10],
"GDPR consent requirements": [0.35, 0.15, 0.10, 1.00],
}
rel = {name: cos(query, v) for name, v in chunks.items()}
def top_k(k):
return sorted(rel, key=rel.get, reverse=True)[:k]
def mmr(k, lam):
selected, candidates = [], list(chunks)
while candidates and len(selected) < k:
def score(c):
redundancy = max((cos(chunks[c], chunks[s]) for s in selected), default=0.0)
return lam * rel[c] - (1 - lam) * redundancy
best = max(candidates, key=score)
selected.append(best); candidates.remove(best)
return selected
def show(title, names):
print(title)
for i, n in enumerate(names, 1):
print(f" {i}. {n:<36} relevance {rel[n]:.3f}")
pairs = [(a, b) for i, a in enumerate(names) for b in names[i + 1:]]
avg = sum(cos(chunks[a], chunks[b]) for a, b in pairs) / len(pairs)
print(f" average similarity between picked chunks: {avg:.3f}\n")
show("Similarity search, top 4:", top_k(4))
for lam in (0.95, 0.7, 0.5):
show(f"MMR, k = 4, lambda = {lam}:", mmr(4, lam))Output:
Similarity search, top 4:
1. policy §2.4 (retention) relevance 0.875
2. policy §2.2 (data we collect II) relevance 0.854
3. policy §2.1 (data we collect) relevance 0.841
4. policy §2.3 (data we collect III) relevance 0.841
average similarity between picked chunks: 0.998
MMR, k = 4, lambda = 0.95:
1. policy §2.4 (retention) relevance 0.875
2. policy §2.2 (data we collect II) relevance 0.854
3. policy §2.1 (data we collect) relevance 0.841
4. policy §2.3 (data we collect III) relevance 0.841
average similarity between picked chunks: 0.998
MMR, k = 4, lambda = 0.7:
1. policy §2.4 (retention) relevance 0.875
2. cookie & tracking notice relevance 0.657
3. GDPR consent requirements relevance 0.686
4. 2023 breach incident report relevance 0.630
average similarity between picked chunks: 0.359
MMR, k = 4, lambda = 0.5:
1. policy §2.4 (retention) relevance 0.875
2. cookie & tracking notice relevance 0.657
3. 2023 breach incident report relevance 0.630
4. GDPR consent requirements relevance 0.686
average similarity between picked chunks: 0.359Reading the output: plain top-4 returns four slices of the same policy section (their average similarity to each other is 0.998). At λ = 0.95 MMR behaves exactly like plain search, since relevance dominates. At λ = 0.7 and 0.5 it keeps the best policy chunk and then brings in cookies, consent and the breach report, dropping average overlap to 0.359 while each chunk is still clearly on-topic (relevance 0.63–0.69). The order of the last two picks changes with λ: a lower λ weighs "different from what I have" more heavily. With real embeddings the same pattern holds, but you would judge the λ setting by answer quality on a labelled question set.
In LangChain the same thing is one call on any vector store:
retriever = vectorstore.as_retriever(
search_type="mmr",
search_kwargs={"k": 5, "fetch_k": 30, "lambda_mult": 0.6},
)Q: What problem does MMR solve in RAG?
A: Redundant retrieval. Top-k similarity often returns several near-duplicate chunks (overlapping chunks, repeated sections), wasting the context budget and hiding other relevant facets. MMR picks chunks that are relevant to the query but dissimilar to the chunks already chosen, so the same k tokens of context cover more information.
Q: Walk me through the MMR formula.
A: Greedily add the candidate maximising λ·sim(chunk, query) − (1 − λ)·max sim(chunk, already-selected). The first term is relevance, the second penalises redundancy with anything already picked. λ = 1 reduces to plain top-k; lower λ favours diversity.
Q: How do you choose λ, k and fetch_k?
A: Empirically. Build a labelled set of questions with the facts a good answer needs, then compare answer completeness and faithfulness (and recall of required chunks) across a few λ values. fetch_k should be several times k so MMR has real alternatives, but small enough that everything in the pool is still relevant.
Q: When would MMR hurt?
A: Narrow factual questions where two very similar chunks are both needed (for example a table split across two chunks), and corpora that are already deduplicated. There MMR can drop the second supporting chunk in favour of something only loosely related. Keep λ high or skip MMR for those query types.
Q: MMR or a cross-encoder reranker?
A: They solve different problems: a reranker improves precision (is this chunk truly about the question), MMR improves diversity (don't repeat yourself). A common pipeline is retrieve → rerank → MMR or near-duplicate filtering → prompt.
Follow-up: Is MMR expensive at query time?
A: No. It works on a small candidate pool, costing O(k × fetch_k) similarity computations, microseconds to a millisecond next to embedding the query and calling the LLM. The only requirement is that the vector store returns candidate embeddings.
In a RAG service, MMR is a retrieval-layer setting, so treat it like any ranking change: make it configurable per use case (on for broad "summarise what we know" assistants, off or high-λ for precise FAQ lookups), log the chosen chunks and their similarity to each other, and evaluate changes offline on a labelled question set before rolling out. It pairs naturally with ingestion-time deduplication (hash or near-duplicate detection) so the index itself carries less repetition; see the end-to-end support chatbot design.
MMR trades a little top-k relevance for coverage: the context contains more distinct information per token, which helps broad questions and repetitive corpora, but it can crowd out a second strongly relevant chunk that a narrow question needed. It adds one tunable (λ) and a candidate pool size that must be chosen with evaluation data, and it needs embeddings for the candidates. It does not replace better chunking, hybrid search or reranking; it is the last polish on an already-good candidate list.