Skip to content

fix: guard QSA cache frontiers during resize and extension - #657

Open
transcendence-x wants to merge 1 commit into
youssofal:mainfrom
transcendence-x:fix/qsa-cache-resize-frontier
Open

transcendence-x wants to merge 1 commit into
youssofal:mainfrom
transcendence-x:fix/qsa-cache-resize-frontier

Conversation

@transcendence-x

Copy link
Copy Markdown

Summary

Flash-Next turns can fail before the first generated token with QSA pooled hold 19843 rows; resizing to 8192 would cut them when reused cache metadata still describes a longer KV frontier. DSH Desktop surfaces this engine error as PI_AI_ERROR and ends the turn. Normalize pooled validity at the resize and extension consumers so newly completed blocks are recomputed, allow capacity planning to decline unsafe in-place resizes, and keep execution-stage would-cut refusals on eager verification without truncating live rows. Other exceptions still propagate.

Related: #592 and #605. Reproduction report: #592 (comment). This complements #650's state-setter fix; it does not change that setter or claim to cover every source of stale state.

Verification

Base: 9882703f3105363ddc37eca9f97aa09a1d387112 (main). The three affected source files match the app's original 2.12.2 runtime byte-for-byte.

  • New deterministic regression file: 8 failures / 2 passes on unmodified main; 10 passes with this change. Covers the recorded pooled counts, active-prefix preservation, numerical pooled-key and fp32-mirror recomputation, other compression ratios, declined resize estimates and the eager execution fallback.
  • python -m pytest tests/test_qsa_cache_resize_frontier.py tests/test_one_copy_reserve_resize.py tests/test_one_copy_admission.py tests/test_one_copy_conversation.py tests/test_qwen4_exp_qsa_cache.py: 96 passed.
  • Installed development/server extras into a separate Python 3.14.3 venv. The required CONTRIBUTING command, python -m pytest tests/test_no_mlx_imports.py tests/test_public_cli.py tests/test_runtime_kpis.py, gives 343 passed / 14 failed, both here and on a clean archive of the unmodified base. The failing test IDs are identical: 8 test_one_shot_expect_python_validates_the_final_program cases, 3 test_one_shot_takes_its_prompt_from_a_pipe cases, test_one_shot_prefers_an_explicit_prompt_over_the_pipe, test_ask_reasoning_effort_reaches_the_prompt_encoder, and test_terminal_quickstart_max_uses_verified_max_session. The failures involve synthetic/mocked runtime imports; they are not reported as passing.
  • python -m build: passed (sdist and wheel).
  • scripts/fresh_venv_smoke.sh: passed no-MLX CLI survival checks.
  • New test-file Ruff checks, Python compilation, git diff --check, repository hygiene scan and attribution scan: passed.

Benchmark Evidence

Correctness smoke coverage, not a throughput benchmark. The same functional patch was loaded by starting MTPLX.app from its UI, using Youssofal/Qwen3.8-Flash-Next-MTPLX-Bare-Speed on an Apple M5 Max / 128 GiB / macOS 26.5.1, app Python 3.14.5, MLX 0.32.2, mlx-lm 0.31.3, on October 23, 2026 (Asia/Shanghai). Turbo, native MTP depth 3, ar_batch / agent, 512-token prefill chunks, SSD session cache on, context 262144, smart fan mode; requests used temperature 0 and thinking off.

Six OpenAI-compatible SSE requests all completed with CACHE_OK: 656 -> 68656 -> 68661 -> 31256 -> 15956 -> 660 input tokens. The continuation reused 68096 cached tokens; shortening the history exercised an actual promotion=resized to 32768 rows. No engine errors were recorded during this sequence. These synthetic requests verify recovery paths and cache reuse; the original private failing prompt was not replayed. No model weights, prompts, credentials or machine-specific paths are included in the change.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant