Skip to content

Add spell check feature for queries that give few results - #152

Open
wandernauta wants to merge 6 commits into
berthubert:mainfrom
wandernauta:suggest
Open

Add spell check feature for queries that give few results#152
wandernauta wants to merge 6 commits into
berthubert:mainfrom
wandernauta:suggest

Conversation

@wandernauta

@wandernauta wandernauta commented Aug 23, 2026

Copy link
Copy Markdown
Contributor

This adds a "Did you mean..." ("Bedoelt u mogelijk...") suggestion feature that tries to correct misspelled words in search queries, using the search index as a dictionary. For example, correcting "digitaliserinsstrategie" to "digitaliseringsstrategie":

2026-08-23 13 58 47 localhost 3abb0d4d5c21

This uses the yhirose cpp-fstlib library to build an in-memory finite-state transducer over a "lexicon table" which is filled with the terms of the FTS5 full-text index at tkindex time.

Arbitrarily...

  1. We accept a word (or name or ...) as correctly spelled if it occurs in 20 or more documents.
  2. We regard a query as potentially needing correction if it gives fewer than 20 results.
  3. We try to correct at most 2 spelling errors per word.
  4. Instead of giving the suggestion that is closest in spelling, we give the suggestion that is most common in the index. The term "digitaliseringstrategie" is closest in spelling to the example above, but less common, so that's not the one returned.

All of these can be tweaked without having to re-run tkindex, except the first.

An in-memory FST is also a good way to serve as-you-type query suggestions ("kat" -> "kathmann", "raj" -> "rajkowski") but this is not yet included here, and would need a little bit more JavaScript to hook up with the autoComplete.min.js library.

This PR also contains the changes from #149.

Comment thread suggest.cc Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants