From c41d829842587ef75bd3443cf1efc3a86905eef0 Mon Sep 17 00:00:00 2001 From: Azeez Ishaqui <95100110+AzeezIsh@users.noreply.github.com> Date: Fri, 2 Oct 2026 08:28:08 -0400 Subject: [PATCH] prefill: refuse the mid-chunk tail capture on aux-carrying models Qwen3.8-Flash-Next carries per-sequence aux state (PLE n-gram history, QSA indexer keys), which is position-exact only at a completed pass end. The mid-chunk capture is therefore aux-less, and the restore gate declines aux-less slots. Its tb-bs sibling is also indexed under the same prefix key as prefill_b_save_checkpoint's tail-split checkpoint, which does carry aux, so registering the sibling displaced and freed the only anchor the next turn could restore. Every warm turn recomputed the whole prompt: winbox #149 replay, 2026-10-02, logs 'Prefix cache hit: N tokens ... but no SSM snapshot' on every turn and TTFT within 5-10% of no-cache. Refusing the plan leaves the tail-split checkpoint at tb-bs as the anchor for both match points, the same path non-atlas_scale builds (GB10, #69) already take. Co-Authored-By: Claude Opus 5.5 --- .../trait_impl/prefill_b/midchunk_capture.rs | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/crates/spark-model/src/model/trait_impl/prefill_b/midchunk_capture.rs b/crates/spark-model/src/model/trait_impl/prefill_b/midchunk_capture.rs index f03d893b0..044ea098f 100644 --- a/crates/spark-model/src/model/trait_impl/prefill_b/midchunk_capture.rs +++ b/crates/spark-model/src/model/trait_impl/prefill_b/midchunk_capture.rs @@ -153,6 +153,23 @@ impl TransformerModel { if !cfg!(atlas_scale) { return None; } + // Aux-carrying models (PLE n-gram history, QSA indexer keys: + // Qwen3.8-Flash-Next) cannot use this capture at all. Their lexical + // state is position-exact only at a completed pass end, so a mid-pass + // capture is necessarily aux-less, and the restore gate declines + // aux-less slots (prefix_lookup.rs / prefill_a.rs). Worse, the + // `tb - bs` sibling is indexed under the SAME prefix key as + // `prefill_b_save_checkpoint`'s tail-split checkpoint, which DOES + // carry aux, so registering the sibling displaced and freed the one + // anchor the next turn could restore. Every warm turn then recomputed + // the whole prompt (winbox #149 replay, 2026-10-02: "Prefix cache hit: + // 20288 tokens ... but no SSM snapshot" on every turn, TTFT within + // 5-10% of no-cache). With the plan refused, the tail-split checkpoint + // at `tb - bs` is the anchor for both match points (`tb`, `tb - bs`), + // the same path non-`atlas_scale` builds (GB10, #69) already take. + if self.requires_aux_state() { + return None; + } // Reuse gate: capture costs a per-prefill kernel split + D2D copy and // is only ever consumable by a LATER request of the SAME session (the // snapshot lookup is session-gated). A session seen for the FIRST