feat(core): index failed tool results for retrieval - #15
Open
Cyb3rN8 wants to merge 1 commit into
Open
Conversation
search() covers messages.text only, so a past failure is findable just when the assistant happened to quote the error. Measured against a 380k-message index, ~37% of failed tool calls leave no trace in messages.text at all -- the model fixes the error rather than restating it -- which makes "how did I fix this last time" miss exactly the cases it is asked about most. tool_errors_fts indexes is_error rows only: 0.6% of tool_results by size, ~6MB of inverted index on a 950MB database. Successful tool output stays unindexed -- it is 99% of the volume, and a LIKE scoped to a session_id is already ~1ms once retrieval has produced one. Refreshed wholesale in finalize rather than maintained by triggers: tool_results is written with INSERT OR REPLACE, and REPLACE fires DELETE triggers only when recursive_triggers is on, so a trigger pair would leave superseded error text searchable forever. A full refresh of a few thousand rows costs ~0.5s. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Cyb3rN8
pushed a commit
to Cyb3rN8/obelisk
that referenced
this pull request
Aug 19, 2026
…es up with the search guard Companion to the two upstream branches merged above, covering the local-only surface (tommy0103#15's tool_errors_fts and the trigram-pinned skill doc): - persist writes tool_results with ON CONFLICT DO UPDATE instead of INSERT OR REPLACE: the rowid stays stable and the UPDATE trigger fires, which is what makes trigger maintenance of tool_errors_fts sound (REPLACE fired no DELETE trigger — the reason this table was refreshed wholesale every finalize). - schema.sql grows rowid-aligned ai/ad/au triggers on tool_results (is_error rows only); hash pin updated in provider-schema-stability.test.mjs. - rebuildToolErrorsFts is now rowid-aligned and runs once per index under the __tool_errors_fts_synced__ marker (same one-time-heal pattern as __messages_fts_synced__), removing the remaining ~5.5 s fixed cost of every incremental build. The force path relies on the triggers (its DELETE FROM tool_results cascades through AD), fixing the pre-existing gap where a force rebuild left tool_errors_fts stale until the next incremental finalize. - SKILL.md: the short-term guidance now describes the automatic guard (degraded: short-token-post-filter / like-scan) and the hit-centered snippet field instead of prescribing manual sql() LIKE fallbacks. - tests/tool-errors-triggers.local.test.mjs pins every persist write shape against the triggers and the refresh's rowid alignment. Incremental build fixed cost measured on this machine's 1.8 GB index: ~30.2 s (messages_fts rebuild) + ~5.5 s (tool_errors_fts refresh) → both now one-time heals. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
search()coversmessages.textand nothing else, so a past failure is findable only when the assistant happened to quote the error — and it usually doesn't, because the model fixes an error rather than restating it. That makes "how did I fix this last time" miss precisely the cases it gets asked about most.Measurement
Against a real index — 380k messages, 127k tool results, 2.5 GB of transcripts — I sampled 400 error identifiers (
ENOENT,TypeError,error TS2339, …) taken fromis_error=1rows and asked whethermessages_ftscould locate the session they occurred in:messages_ftsmessages.textentirelyInputValidationError(96 rows) andJSONDecodeError(52 rows) are typical: zero hits inmessages_fts, because they are runtime class names that only ever appeared in tool output.For contrast, the same probe over all tool output — not just failures — is far less alarming (48.8% locatable), and over file names it is 74%. Failures are the sharp edge, which is why this PR indexes only them.
What this does
tool_errors_ftsindexesis_errorrows only — 0.6% oftool_resultsby size (1.3 MB here), about 6 MB of inverted index on a 950 MB database.Successful tool output stays unindexed on purpose. It is ~99% of the volume (
Read98.9 MB,Bash90.7 MB on my index), and full-text indexing it would add roughly 1.1 GB. It doesn't earn that: once retrieval has produced asession_id, aLIKEscoped to it is ~1 ms viaidx_tr_session, and even an unscoped scan of all 209 MB is 1.7–3.2 s — fine as a deliberate last resort.Refreshed wholesale in
finalize, not maintained by triggers.tool_resultsis written withINSERT OR REPLACE, and REPLACE fires DELETE triggers only whenrecursive_triggersis on (off by default) — a trigger pair would leave superseded error text searchable forever, and re-indexing the same tool call would silently accumulate duplicates. A full refresh of a few thousand rows costs ~0.5 s warm (4.7 s on a cold page cache), against a 113 s full build. Both index paths do it:packages/core/src/indexer.tsviarebuildToolErrorsFts(), and the app'srebuildFts().Every write path stays correct without bookkeeping, since the refresh is unconditional: force build (
DELETE FROM tool_resultsthen re-index), incremental build, and per-session retraction all converge.Verification
tests/tool-errors-fts.test.mjs, 7 tests — including the two that motivate the no-trigger decision: re-indexing the sametool_use_idleaves no stale text searchable, and a result that stops being an error drops out. Also idempotence,is_errorfiltering, session pivot, and the force-build path.Full suite /
lint/typecheck/build:core/build:cli: no regression — failing tests on my machine are identical with and without this change (22 files, allbetter-sqlite3, which cannot build on Node 26). I could not exercise the app'srebuildFts()at runtime for that reason, only typecheck it — that call site deserves a second look.Notes
references/schema.md,references/query-patterns.md); without it agents won't know the table exists and will keep missing these.tool_errors_ftsshould joinFTS_TABLESso a configured tokenizer applies to it as well. Happy to rebase in whichever order you prefer.unicode61is a reasonable default even for CJK users.🤖 Generated with Claude Code