Where: SYDLE, AI team
Stack: Elasticsearch, JavaScript
Scope: Mine, on a direction my tech lead set

Vector search could not match a name

Embeddings are good at knowing that a string is a person's name. They are bad at knowing which one.

What was reported

A customer told us that some searches were coming back wrong. Not slow and not empty: wrong. The pattern was queries with little semantic content and a need for exact correspondence, and looking up a person by name was the clearest case.

Retrieval was kNN over embeddings only. That is the query shape it handles worst, because the model encoded "a personal name" rather than the identity of the person. The right document and a wrong one sit almost on top of each other.

A lexical index has the opposite strength. It does not know what a name is, and does not need to.

Combining two rankings

My tech lead proposed hybrid retrieval: run both and combine. The part worth writing down is the word "combine", because the obvious way is a trap.

Why not add the scores

Normalise both score sets and take a weighted sum. It works on the day you build it. BM25 scores depend on term statistics across the corpus, so the same document scores differently as the corpus grows, and the weight you fitted stops being right. Nothing throws when that happens. Results just get worse.

What I used

Reciprocal Rank Fusion. A document's contribution from a ranking depends on its position in that ranking and nothing else.

score(d) = Σ  1 / (k + rank(d, r))
           r

k = 60, the conventional constant
r = each ranking being fused, here BM25 and kNN

Absolute scores are discarded. There is no weight to fit, so there is nothing to go stale.

The case the fusion exists for: fifth in one ranking, first in the other, first once fused.

The detail that makes it work

Fusing two rankings only helps if both are deep enough to disagree. Ask each strategy for exactly the ten results you intend to show, and a document ranked fiftieth by BM25 and first by kNN never reaches the fusion at all. You get hybrid search that returns what plain search returned.

So the candidate window is much wider than the result set: ten times the requested size, floor of fifty, ceiling of a thousand. The ceiling is there because the fusion pass is linear in candidates.

Edge cases in the code

Queries with no text. An image or video search has nothing for BM25, so the lexical pass is skipped and the fusion runs over one ranking, which returns that ranking's own order. Designed for, not discovered.
Ties. Two documents can land on the same fused score. The tiebreak is the sum of their original positions, so the one that did better across both wins instead of whichever came out of the map first.
Debugging. Every hit carries its BM25 rank, kNN rank and both raw scores next to the fused score, so "why did this rank here" is answered by the response rather than by a re-run.

Not measured

There is no measured quality delta. What I have is that the reported problem stopped being reported. That is evidence and it is not a number, and I would rather write that than produce one.

The window constants were not swept. Ten times, floor fifty, ceiling a thousand came from a sensible default and worked. I did not test whether five times would have done the same. The rank constant of sixty is the conventional value, not a tuned one.

Both gaps are the same gap: there is no offline evaluation set for retrieval. Building one is the next thing I would do here, and the first thing I would ask about on a team with retrieval already in production.

What carries over

When you combine two signals, the tempting move is to put them on a common scale, and the durable move is to use only what is already comparable between them. Rank is comparable. Score is not. The version with fewer parameters to fit is the one still working when nobody remembers what the weight was fitted to.