Skip to content

Drop fuzzy matching from the multi-word title AND query - #2251

Open
Nayor wants to merge 1 commit into
masterfrom
fix/search-multiword-fuzzy-false-positive
Open

Nayor wants to merge 1 commit into
masterfrom
fix/search-multiword-fuzzy-false-positive

Conversation

@Nayor

@Nayor Nayor commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Follow-up to Fix multi-word title search ignoring word order and hyphens #2247. The multi-word branch of get_text_query_on_title() added fuzziness='AUTO' on top of the already-stemmed contentheavy fields. Stemming already shortens words, so two unrelated short stems can end up just one edit apart (e.g. "Gerbier" and "Gerri" both stem towards gerbi/geri, one edit apart) - fuzziness on top of that made the query match titles sharing no real word with the query, sometimes outranking the actual match: "Arêtes de Gerbier" was surfacing "Pic de Gerri : arête Noris - Gerri" ahead of "Gerbier : Traversée des arêtes".
  • Confirmed via _explain on a synthetic index built with the real analyzer settings: the query analyzes to ['aret', 'gerbi'], and "Pic de Gerri..." only matched because its stemmed "geri" token was within the allowed 1-edit fuzzy distance of "gerbi".
  • Removing fuzziness from that clause fixes the false positive without reducing recall for the cases Fix multi-word title search ignoring word order and hyphens #2247 was meant to fix (word order, hyphenated compounds) - typo tolerance was never functional on that branch before Fix multi-word title search ignoring word order and hyphens #2247 either (the old phrase query silently ignored its fuzziness parameter).

Test plan

  • Added a regression fixture/test (test_search_multiword_any_order) with a decoy title that reproduces the false positive; confirmed it fails without this fix and passes with it.
  • Full pytest c2corg_api/tests/search/ c2corg_api/tests/views/test_search.py (39 passed), rerun twice for stability.
  • flake8 clean.

The multi-word branch of get_text_query_on_title() added fuzziness='AUTO'
on top of the already-stemmed contentheavy fields. Stemming already
shortens words, so two unrelated short stems can end up just one edit
apart (e.g. "Gerbier" and "Gerri" both stem towards "gerbi"/"geri", one
edit apart) - fuzziness on top of that made the query match titles that
share no real word with the query, sometimes outranking the actual match
(e.g. "Arêtes de Gerbier" surfacing "Pic de Gerri : arête Noris - Gerri"
ahead of "Gerbier : Traversée des arêtes").

The single-word branch keeps its fuzziness (it isn't stacked on stemming,
since that field uses an ngram/prefix analyzer instead), and typo
tolerance wasn't part of what the multi-word AND rewrite was meant to add
in the first place - dropping it here doesn't reduce recall for the
originally reported issues (word order, hyphenated compounds), only the
false positives.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@codacy-production

codacy-production Bot commented Sep 16, 2026

Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics 0 complexity · 0 duplication

Metric Results
Complexity 0
Duplication 0

View in Codacy

🟢 Coverage 100.00% diff coverage · +0.00% coverage variation

Metric Results
Coverage variation +0.00% coverage variation
Diff coverage 100.00% diff coverage

View coverage diff in Codacy

Coverage variation details
Coverable lines Covered lines Coverage
Common ancestor commit (dd1ce78) 22269 20594 92.48%
Head commit (df6d23d) 22272 (+3) 20597 (+3) 92.48% (+0.00%)

Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch: <coverage of head commit> - <coverage of common ancestor commit>

Diff coverage details
Coverable lines Covered lines Diff coverage
Pull request (#2251) 3 3 100.00%

Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: <covered lines added or modified>/<coverable lines added or modified> * 100%

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

@Nayor
Nayor requested a review from eddy-geek September 16, 2026 14:56
@Nayor Nayor self-assigned this Sep 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant