Adaptive per-query fusion, on by default#556
Closed
tobocop2 wants to merge 2 commits into
Closed
Conversation
The retrieval benchmark showed no single lexical_fusion_weight wins every corpus: SciFact wants a strong BM25 arm, NFCorpus a weak one, FiQA none. Rather than fix one weight, adaptive_fusion (off by default) scales the BM25 arm per query by how peaked the vector ranking is -- a top hit standing clear of the field downweights lexical toward zero, a flat ranking keeps it. adaptive_fusion_ margin sets the vector-similarity margin at which the lexical arm is fully silenced. lexical_fusion_weight becomes the ceiling the rule scales down from, so the default path is unchanged. (cherry picked from commit a0c2a96)
The retrieval benchmark validated adaptive per-query fusion against the fixed lexical weight on BEIR SciFact, NFCorpus, and FiQA: at margin 0.15 it beats the fixed w=1.0 default on all three, keeps the significant NFCorpus and SciFact wins, and cuts the FiQA regression from -0.056 to -0.018 (the biggest gain on the one corpus a fixed lexical arm hurt most). Default adaptive_fusion=true, adaptive_fusion_margin=0.15; set adaptive_fusion=false to pin the fixed weight.
tobocop2
marked this pull request as draft
July 18, 2026 04:55
Owner
Author
|
Superseded by the consolidated retrieval PR #557 — review and merge there. Converted to draft to keep it off the review queue; the branch and history stay intact for reference. |
Owner
Author
|
Superseded by the consolidated retrieval PR #557, which contains this change. Closing; the branch stays intact for reference. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The retrieval benchmark showed no single lexical fusion weight wins every corpus: on BEIR, SciFact wants a strong BM25 arm, NFCorpus a weak one, and FiQA none at all. A fixed
lexical_fusion_weightis therefore always wrong somewhere, and the shipped equal-weight default significantly regressed FiQA (financial Q&A, where lexical term-matching mostly adds noise) by 0.056 nDCG@10.Solution
Scale the BM25 arm per query by how confident the vector arm is: a peaked dense ranking (a top hit standing clear of the field) downweights the lexical arm toward zero, a flat one keeps it.
lexical_fusion_weightbecomes the ceiling the rule scales down from, andadaptive_fusion_marginsets the vector-similarity margin at which the lexical arm is fully silenced.On the same three datasets, adaptive fusion at margin 0.15 beats the fixed w=1.0 default on all three, keeps the significant NFCorpus (+0.0085, p=0.033) and SciFact (+0.0205, p=0.009) wins, and cuts FiQA’s regression to -0.018 (from -0.056 fixed, and better than the best hand-picked fixed weight). It is the strongest single retrieval policy the study found, so it ships on by default (
adaptive_fusion=true,margin=0.15); setadaptive_fusion=falseto pin the fixed weight. Full numbers and reproducible run files are inevals/benchmark/RESULTS.md.Stacks on #549 (the
lexical_fusion_weightknob it builds on).