[marzukia]

Problem

The MTP model (Qwen3.5-122B-A10B-oQ4-mtp) bloats memory over a long conversation until it OOMs: ~88GB pinned at only ~40K tokens (~16GB above the ~72GB baseline), ratcheting per turn and never returning. Pre-MTP the same box reaches 192K context clean, so the retention is entirely MTP-path machinery.

Root cause (two retainers, both measured live)

1. Leaked MTP generator state. _install_mtp_vendored's per-uid _state only cleaned up on uid REUSE, but mlx-lm allocates a fresh uid per request, so every finished request left a suspended mtp_generate_step generator whose frame pins that turn's full cache set (12 attention KVCache, 36 GatedDeltaNet ArraysCache SSM states, rollback tape, MTP-head KV over the whole context).

2. In-memory prefix cache retention (the dominant one). --enable-prefix-cache defaults ON and was active regardless of the launcher flag (dropping --enable-prefix-cache just falls back to the default). Scheduler._cleanup_finished stores every finished request's full extracted cache into PrefixCacheManager, which evicts by ENTRY COUNT (100) only, not bytes; --cache-memory-mb was accepted but never enforced (MemoryAwarePrefixCache was never instantiated). On this hybrid model each entry is a full bf16 cache set, so the disk-restored, dequantized bf16 KV was retained turn over turn.

Latent no-op bug: cache.state = None on an mlx-lm KVCache hits the state setter (self.keys, self.values = v), which raises TypeError on None, swallowed by a bare except. So the existing cleanup (_clear_gb_cache, _cleanup_uid, and the prior _state reap) freed nothing on the 12 attention KV layers. A companion _clear_gb_cache NameError was dead code that never fired because uid reuse never happens.

Fix

• Free MTP speculative state at request completion. _release_kv_cache_fields() nulls the real per-type storage (KVCache keys/values/offset, ArraysCache .cache slots, rollback_state tapes) with an identity skip-set protecting caches still owned by the live batch. _cleanup_uid_storage() reaches the suspended generator frame (gi_frame.f_locals) to release model_cache/mtp_cache before close(). _purge_stale_states() sweeps _state from _mtp_step and _cleanup_finished. Fixes the NameError. Mid-stream handoff to plain decode frees only generator-owned MTP-head KV + tapes; the backbone stays live.
• PrefixCacheManager now tracks per-entry resident bytes and evicts LRU-first past max_bytes, wired from cache_memory_mb (the previously dead flag is now enforced).
• Skip the in-memory prefix-cache store for non-trimmable hybrid entries when the disk-checkpoint tier is on: the disk tier persists the same entry quantized and serves hits via kv-restore, so the resident bf16 copy bought zero hits on this workload. Pure-attention models and disk-tier-off deployments are unchanged.
• Env-gated diagnostics (/debug/mx_arrays, /debug/mx_referrers) return 404 unless QMLX_DEBUG_GC_SCAN=1 (off by default).

Files: vllm_mlx/scheduler.py, vllm_mlx/prefix_cache.py, vllm_mlx/routes/health.py (+530/-73).

Verification (on start-qmlx.sh verbatim: force-spec-decode, MTP K=2, disk-kv-restore)

• BEFORE (baseline): an 8-turn growing conversation ratchets +1.98GB idle (72.16 -> 74.14GB, never returns); SSM (1,64,128,128) arrays climb 72 -> 396, bf16 KV (1,2,N,256) 48 -> 248, monotonic.
• AFTER: +0.000GB every turn, 0 retained SSM/KV arrays after each, idle back to 72.00GB. Verified at 22K, 39K and 56K prompt sizes with real disk-restore hits, flat each time.
• MTP intact: accept ratio 0.70 (291/416) over the run, ~30 tok/s decode unchanged, replies coherent throughout.

Caveat: PFlash compressed the synthetic test prompts, so the largest restore-hit exercised under the fix was moderate-size. The retention mechanism removed is size-independent, so it is expected to hold at 192K, but that is not directly proven. Adversarial review recommended before merge.