feat(core): guard trigram sub-3-codepoint search terms; hit-centered snippets

[Cyb3rN8]

Fixes #76. Follow-up to #9/#14 (configurable tokenizer).

Root cause and reproduction

Under trigram, a term shorter than three code points emits no tokens. Alone it can only match nothing; next to longer terms it becomes an empty phrase constraining nothing. Real-index measurement: MATCH 'quick ok' → 799 rows, LIKE ground truth 442 — 357 silent false positives. Two-code-point CJK words (修复) hit the same wall, i.e. exactly the audience the trigram option serves.

Change

search() detects the live tokenizer from sqlite_master (correct across migrations, no config dependency) and, for plain token-and-whitespace queries only:

• terms ≥3 code points MATCH as before; shorter terms ride the same statement as AND m.text LIKE ? substring conditions, filtering the full MATCH set before LIMIT (rank preserved) — reported per-hit as degraded: 'short-token-post-filter'. An earlier draft post-filtered an overselected window in JS and collapsed to zero recall on the real index (top bm25 hits for the long term lacked the short one); the SQL push-down is recall-exact and is pinned by a buried-hit regression test.
• all-short queries fall back to a LIKE scan (degraded: 'like-scan', rank: null, recency-ordered) instead of returning zero rows;
• explicit FTS5 operator/syntax queries bypass the guard; the existing punctuation fallback (catch → per-token quoting) routes through the same guard.

On the default unicode61 index the guard never engages; behavior is byte-for-byte unchanged (pinned by test).

Every hit also gains snippet — a window centered on the earliest literal term occurrence (long transcript messages routinely match far past the head; head-truncating previews never show the match). Omitted when no term appears literally.

Existing-assertion change (own section per the contributing guide)

contract-helper-shapes.test.mjs exactKeys for the search() hit gains 'snippet' — extended, not loosened: the fixture query guarantees a literal term hit, so the key is deterministically present. api-reference.md documents both new optional fields in the same commit (the doc-sync guard enforces this).

Verification

• New tests/search-short-token-guard.test.mjs (8 tests): mixed-term filtering, all-short LIKE fallback, two-code-point CJK + short ASCII, punctuation-fallback routing, raw-syntax bypass, unicode61 untouched, buried-hit recall regression, hit-centered snippet.
• Full suite on this branch head: 473/473 pass, typecheck clean, lint 0 errors (4 pre-existing warnings in unrelated app tests, present on main).
• Real-index spot check after deploy: search('quick ok') returns only rows containing both terms with numeric rank; search('修复 ok') returns like-scan hits instead of zero; snippets center on the match.

🤖 Generated with Claude Code