Skip to content

feat(core): opt-in OR fallback for zero-hit semantic searches #143

Description

@kongtou20070406

Problem

search(text) correctly preserves raw SQLite FTS5 semantics, so an ordinary
multi-token string is an implicit AND query. That is useful for precise lookup,
but a semantic-history request can return no context when even one conversational
token is absent.

On a frozen 50-item development split of the official small Mem2ActBench release
(commit b00726940b5abbe9bd324bdd7a2cb272f5c62a29), the current query retrieved
labelled evidence for 0/50 tasks. A safe, quoted OR query over at most 12 tokens
retrieved labelled evidence for 13/50 at 17.06 ms p95. A blind Luna Max reader
then improved end-to-end exact tool calls from 0/50 to 9/50 and argument F1 from
0.0313 to 0.3047 (9 paired wins, 0 harms; one-sided exact McNemar p=0.001953).
Oracle evidence reached 22/50, so this is useful but not ceiling-level recall.

Minimal proposal

Add an opt-in fallback: 'or' search option. It runs the existing raw/safe
primary query first and only retries with the existing safe tokenization joined
by OR when the primary result is empty.

  • Default and explicit FTS5 behavior remain unchanged.
  • Existing source/project/session/time/meta/visibility filters and rank/limit
    remain identical on both attempts.
  • The skill may opt in for semantic-history recall, but not for exact scopes or
    sentinels where an empty result is meaningful.

Acceptance evidence

  • Unit regression: default multi-token AND remains unchanged; non-empty primary
    results never broaden; the opt-in zero-hit path broadens; visibility/meta and
    structural filters still apply.
  • Enabled/disabled retrieval and downstream-reader ablation.
  • Frozen private-history regression, including an unanswerable abstention guard.
  • Query latency remains within the existing gate.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions