probe(mac-bridge): vllm-mlx-niah — vLLM-MLX continuous batching on gemma-4 is broken (shared_kv)

[FluffyAIcode]

What

Adds a Mac-bridge preset vllm-mlx-niah + harness to empirically answer: is vLLM-MLX both parallel AND recall-preserving on our gemma-4 config? (the Apple-Silicon analog of the CUDA KIE-v2 question — Kakeya Attention on a borrowed runtime). Runs on the SAME local MLX gemma verifier used to reproduce the MLX B>1,L=1 batched-decode bug.

Finding (measured on M4 via the bridge, vllm-mlx 0.3.0, gemma-4-26B-A4B-it-mlx-4bit)

Answer: NO — not on gemma-4 with vllm-mlx 0.3.0.

phase	result
Simple mode (no continuous batching), N=1	model generates (26 tokens; gemma-4 reasoning output) → single-stream serving works
Continuous batching, N=8	0 real tokens/session, recall 0 — every request errors

Root cause (server traceback): vllm-mlx's own gemma-4 batching adapter crashes —

mlx_vlm/models/gemma4/language.py:320 ... self.self_attn(..., shared_kv=shared_kv, offset=offset)
TypeError: patch_gemma4_attention_for_batching.<locals>._patched_call() got an unexpected keyword argument 'shared_kv'
→ Request failed during preprocessing → finish_reason=error

So vLLM-MLX serves gemma-4 single-stream, but its continuous-batching patch for gemma-4 does not handle gemma-4's shared_kv attention → parallel decode on gemma-4 is unusable in 0.3.0. (Notably, gemma-4's shared-KV/hybrid attention under batching is the same hard spot as the original MLX B>1,L=1 bug — here it surfaces as an integration TypeError rather than silent recall corruption.)

vllm_mlx_gemma4_parallel_probe_m4.log

Changes

• scripts/research/vllm_mlx_niah_bench.py — stdlib-only two-phase A/B harness (simple vs continuous-batching), unique-needle NIAH, per-session recall + aggregate decode tok/s, always emits a verdict JSON.
• inference_engine/bridge/manifest.py — vllm-mlx-niah preset (pip install vllm-mlx, then the bench); bounded params; model from runner env.
• tests/inference_engine/bridge/test_manifest.py — allowlist + build_commands coverage.

Testing

• ✅ pytest tests/inference_engine/bridge/test_manifest.py — 28 passed, 100% coverage on manifest.py.
• ✅ End-to-end on the Mac M4 via the git-bus bridge (Actions run 27614317585, success).

_{To show artifacts inline, enable in settings.}