Repository navigation
fix: guard QSA cache frontiers during resize and extension - #657
Open
transcendence-x wants to merge 1 commit into
Open
transcendence-x wants to merge 1 commit into
transcendence-x wants to merge 1 commit into
Conversation
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Flash-Next turns can fail before the first generated token with
QSA pooled hold 19843 rows; resizing to 8192 would cut themwhen reused cache metadata still describes a longer KV frontier. DSH Desktop surfaces this engine error asPI_AI_ERRORand ends the turn. Normalize pooled validity at the resize and extension consumers so newly completed blocks are recomputed, allow capacity planning to decline unsafe in-place resizes, and keep execution-stage would-cut refusals on eager verification without truncating live rows. Other exceptions still propagate.Related: #592 and #605. Reproduction report: #592 (comment). This complements #650's state-setter fix; it does not change that setter or claim to cover every source of stale state.
Verification
Base:
9882703f3105363ddc37eca9f97aa09a1d387112(main). The three affected source files match the app's original 2.12.2 runtime byte-for-byte.python -m pytest tests/test_qsa_cache_resize_frontier.py tests/test_one_copy_reserve_resize.py tests/test_one_copy_admission.py tests/test_one_copy_conversation.py tests/test_qwen4_exp_qsa_cache.py: 96 passed.python -m pytest tests/test_no_mlx_imports.py tests/test_public_cli.py tests/test_runtime_kpis.py, gives 343 passed / 14 failed, both here and on a clean archive of the unmodified base. The failing test IDs are identical: 8test_one_shot_expect_python_validates_the_final_programcases, 3test_one_shot_takes_its_prompt_from_a_pipecases,test_one_shot_prefers_an_explicit_prompt_over_the_pipe,test_ask_reasoning_effort_reaches_the_prompt_encoder, andtest_terminal_quickstart_max_uses_verified_max_session. The failures involve synthetic/mocked runtime imports; they are not reported as passing.python -m build: passed (sdist and wheel).scripts/fresh_venv_smoke.sh: passed no-MLX CLI survival checks.git diff --check, repository hygiene scan and attribution scan: passed.Benchmark Evidence
Correctness smoke coverage, not a throughput benchmark. The same functional patch was loaded by starting MTPLX.app from its UI, using
Youssofal/Qwen3.8-Flash-Next-MTPLX-Bare-Speedon an Apple M5 Max / 128 GiB / macOS 26.5.1, app Python 3.14.5, MLX 0.32.2, mlx-lm 0.31.3, on October 23, 2026 (Asia/Shanghai). Turbo, native MTP depth 3,ar_batch/agent, 512-token prefill chunks, SSD session cache on, context 262144, smart fan mode; requests used temperature 0 and thinking off.Six OpenAI-compatible SSE requests all completed with
CACHE_OK: 656 -> 68656 -> 68661 -> 31256 -> 15956 -> 660 input tokens. The continuation reused 68096 cached tokens; shortening the history exercised an actualpromotion=resizedto 32768 rows. No engine errors were recorded during this sequence. These synthetic requests verify recovery paths and cache reuse; the original private failing prompt was not replayed. No model weights, prompts, credentials or machine-specific paths are included in the change.