Harden @ottabase/ottasearch query parsing and remove partial reindex limits

[Copilot]

This PR addresses two valid issues in the new in-house search stack: fragile FTS query handling for malformed user input, and incomplete indexing caused by a hard 200-row reindex cap.
It also tightens helper coverage for query normalization edge cases.

• Problem summary

  • FTS MATCH accepted raw user input, which could produce malformed queries (***, unbalanced symbols) and unstable search behavior.
  • Reindex processed only the first page of records (limit: 200), leaving larger models partially indexed.

• Changes

  • Safe FTS query normalization
    • Added normalizeFtsQuery(query) in packages/ottasearch/src/search.ts.
    • Strips unsafe FTS tokens, preserves valid unicode alnum tokens, bounds term count, and emits prefix terms (term*) joined with OR.
    • Exported via packages/ottasearch/src/index.ts.
    • worker/routes/ottasearch.ts now uses normalized query and returns empty result sets when no valid terms remain.
  • Full-dataset reindexing
    • Replaced single-shot ModelClass.all({ limit: 200 }) with paged reads (limit + offset) until exhaustion.
    • Reindex now covers all rows for enabled models rather than only the first page.
  • Focused helper tests
    • Extended packages/ottasearch/src/__tests__/search.test.ts with normalization edge cases:
      • special-char-only input
      • whitespace-only input
      • unicode terms
      • term-limit truncation behavior

• Illustrative snippet

// Before: raw query directly into FTS MATCH
const query = url.searchParams.get('q')?.trim() ?? '';

// After: normalized + bounded FTS expression
const query = url.searchParams.get('q')?.trim() ?? '';
const ftsQuery = normalizeFtsQuery(query);
if (!ftsQuery) return jsonResponse({ results: [] });

const ftsResult = await env.OBCF_D1
  .prepare(`... WHERE ${OTTASEARCH_FTS_TABLE} MATCH ? ...`)
  .bind(ftsQuery, limit)
  .all();

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.