searchBlocks: hybrid Math.max(...docs) hard-crashes at ~160K docs; docFeatures 8MB cap re-tokenizes large corpora on every call

[ranxianglei]

来源: Tyan66666/billion-context-dsh#133 分析 search_context 长会话劣化（190 块 / 13.3M 遮蔽 token 会话，单次搜索 ~20.8 分钟）时发现

Two independent kernel-layer costs in the search path, both measured on acp-kernel 0.0.29 (the billion-context-dsh host session from #133 + local reproduction at the reported scale).

1. hybridAlgorithm.score hard-crashes at large doc counts (P1)

src/search/algorithms/hybrid.ts normalizes the two channel scores with an argument spread:

const maxBm = Math.max(...bm.map((r) => r.score), 1e-9)
const maxFz = Math.max(...fz.map((r) => r.score), 1e-9)

The spread throws RangeError: Maximum call stack size exceeded once the doc count exceeds the JS argument limit (~65K–125K depending on the V8 build).

Reproduced locally with 160,170 docs (46MB text, 190 blocks): the call crashes at hybrid.ts before returning any result — the search tool THROWS instead of degrading. The #133 session (19.7K docs) was below the limit, but any session that compresses ~30K+ shadowed messages breaks search entirely.

Suggested fix: compute the max with a loop/reduce (O(n), no argument limit). Two lines.

2. docFeatures 8MB cap re-tokenizes large corpora on every call

DEFAULT_CAP_CHARS = 8 * 1024 * 1024, keyed by full doc text, evicted in insertion order (not LRU). A search corpus larger than the cap re-tokenizes (corpus − cap) on EVERY call:

• Measured (40.4MB corpus, 20,140 docs, CJK-mixed): 17.9s per call, identical across cold/warm calls — the 8MB window slides and nothing is reused.
• The kernel's own doc-cache docstring estimates ~0.3s/MB for the segmenter pass.
• setDocCacheCap() exists and the host now uses it (128MB → 9.0s cold + 0.14–0.16s warm), but the cap design is a multi-session-server tradeoff: steady-state feature heap is ~4–6.3× cached source chars (measured), so a single-session host gets full-corpus caching only at ~800MB+ heap for a 70MB corpus.

Suggested directions (design call for the kernel owner):

• key by doc ref with per-session / per-doc-set cache lifetime, so a single-session host can cache the whole corpus without a global char cap;
• or LRU eviction so repeated calls reuse the most-touched docs.

Note: the hybrid double pass (BM25 + fuzzy both scan all docs, then applyRoleWeight/buildResults each build a full Map) is fine once features are warm (~140ms at 20K docs) — no action needed unless profiling shows otherwise.

Data

• fix: size-aware emergency truncation (billion-context-pi#12) #133 live session (host 0.1.2-rc.1, plugin v0.2.19): 8 paired tool calls — 6 blocks → 1.8s … 190 blocks / 13.3M shadowed tokens → 1249s; 19.7K docs / ~50MB corpus per call.
• Local reproduction (this report): N=20,861 events / B=190 / 40.4MB corpus: searchBlocks 17.9–18.0s ×3 under the 8MB cap; 9.0s cold + 0.14–0.16s warm under a 128MB cap. 160,170-doc corpus: crash (item 1).

[ranxianglei]

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 进展同步：

• P1（hybrid Math.max(...scores) 参数 spread 崩溃） 已修复：PR (fix) hybrid scoring: compute the score max in a loop — the Math.max(...) spread crashed at 130K+ docs #228（分支 2026-09-09_hybrid-max-arg-limit，commit 9b5f022）——maxScore(results, floor) 循环求 max 替代两处 spread（语义不变），新增 130,000 文档回归测试（旧代码上验证失败：RangeError: Maximum call stack size exceeded；修复后 1.1s 通过且正确排序）。内核全套 625/625 通过。
• P2（8MB docFeatures 缓存上限） 留作设计决策，待你拍板（per-session ref-keyed 缓存 vs LRU 逐出 vs 可调上限的默认值）。下游 billion-context-dsh 已用公开的 setDocCacheCap 将上限调到 128MB 作为临时缓解（PR (fix) search_context: cache the block ledger and search corpus per log snapshot — 20-minute searches become sub-second Tyan66666/billion-context-dsh#134）。

[ranxianglei]

🤖 Powered by ework · qwen3.8-27b

Both items reproduced locally on the current tree before changing anything, then fixed. PR: #229

Verification (pre-change):

• Item 1: hybridAlgorithm.score with 160,170 docs → RangeError: Maximum call stack size exceeded — the search tool throws, returns nothing. Confirmed the V8 argument-spread limit is the cause.
• Item 2: 12.0MB corpus / 27,741 docs under the 8MB cap → cold 7242ms, warm 7524ms (warm/cold = 104%), cache pinned at 8.0MB. Confirms (corpus − cap) is re-tokenized on every call.

Item 1 fix (src/search/algorithms/hybrid.ts): the channel-score max was Math.max(...bm.map(r => r.score), 1e-9) — an argument spread that trips V8's limit past ~65K–125K docs. Now a reduce (O(n), no arg limit, no extra allocation). Audited the whole codebase: this was the only argument-spread hazard.

Item 2 fix (src/search/doc-cache.ts) — design call, I went with LRU over per-scope:

• Eviction is now LRU (a hit re-inserts at the tail), so the docs a host keeps re-querying are the ones retained. Strictly better than FIFO, no interface change.
• setDocCacheCap(≥corpus) / Infinity is now documented + tested as the opt-in for a single-session host to cache the whole corpus once → later calls become O(docs × query-terms). This is what the host needs: set the cap to ≥ corpus size (or Infinity) and the per-call re-tokenization goes away without a per-scope redesign.

Why not per-scope (key by doc ref): it would require threading a scope through the stateless public SearchAlgorithm.score(docs, query) contract — a breaking change for custom algorithms, or a fragile non-async-safe module-level "active scope". Too much surface for a multi-session-server use case that doesn't exist yet; the single-session host is already served by the cap knob. If you'd rather have per-scope as the primary design, say so and I'll scope it as a follow-up — but my recommendation stands on the cap knob + LRU.

Left the hybrid double pass (BM25 + fuzzy both scan all docs; applyRoleWeight/buildResults each build a full Map) as-is — fine once features are warm (~140ms at 20K docs), per your note.

Tests: hybrid no-crash at 160K docs; LRU keeps a re-accessed doc that FIFO would evict (object-identity); Infinity cap caches the whole corpus. Full suite 583 pass, typecheck clean, build green.