Skip to content

fix: count pooled blocks from the restored keys, not the donor table - #650

Open
goodbadwolf wants to merge 1 commit into
youssofal:mainfrom
goodbadwolf:fix/qsa-pooled-frontier
Open

goodbadwolf wants to merge 1 commit into
youssofal:mainfrom
goodbadwolf:fix/qsa-pooled-frontier

Conversation

@goodbadwolf

Copy link
Copy Markdown

Summary

A conversation restored from the SSD tier below its end loses the user's request on Flash-Next. The sub-prefix decoder cuts the keys and values to the restore point, but the indexer's raw and pooled tensors are stored whole, so they arrive at the donor entry's full length, and QSACache.state took its count of valid pooled blocks from that length. The indexer then found every block of the new tokens already pooled, computed none, and scored attention against the donor conversation's block keys, so the model answered from facts that are not in the prompt (#635). A RAM restore does not show this because it loads the full state and trims it, which clamps the count.

Changes

  • A sub-prefix restore recomputes the pooled blocks of the new tokens, as a RAM restore already does.

Verification

Three new tests in tests/test_qsa_state_restore_frontier.py: the prefix-restore case fails on v2.12.2 and passes on this branch, and the full-state round trip and the missing-table case pass on both.

On the real stored entry that failed (58,073 tokens, restore point 2,048), the old setter gave pooled_len=14518 with the keys at 2,048 rows, and the next 4,096-token chunk recomputed 0 pooled blocks where 1,024 were needed. With this change pooled_len=512, the value a RAM restore and trim gives.

End to end on Youssofal/Qwen3.8-Flash-Next-MTPLX-Optimized-Speed at 34ae535f, MacBook Pro M5 Max with 128 GB on macOS 27.0.1, 2026-10-20: a Claude Code request at temperature 0 with thinking on, restored from the SSD tier at 8,192 tokens from an entry holding a different conversation, with the RAM bank cleared before each run. Built from this branch, 5 of 5 runs found the task, at a first-depth draft acceptance of 0.89. On v2.12.2, 2 of 2 runs answered that there was no task, at 0.64, and the same request with x-mtplx-cache-mode: bypass found it at 0.85.

The 20 existing test files that mention QSACache or pooled_len give the same results before and after: one failure in both, tests/test_one_copy_reserve_resize.py::test_the_resize_allocates_its_bill_and_no_more. The three files named in CONTRIBUTING.md show the same 14 failures on v2.12.2 and on this branch in my environment, which runs them through a custom launcher. python -m build and scripts/fresh_venv_smoke.sh were not run.

Not examined: the cross-session entry sharing described in the second comment on #635, and cache families other than QSA.

Benchmark Evidence

This does not change runtime performance. A restore that previously skipped the pooled-block work now does it for the new tokens only.

A conversation restored from the SSD tier below its end kept the donor's whole pooled indexer table, and the cache counted its valid blocks from that table's length. The indexer then computed no blocks for the new tokens and scored attention against another conversation's keys, so the model lost the request (youssofal#635).
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-20T08:06:00.711065Z 2f41437 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant