docs: MLX v0.4-mac multi-tenant is SERIAL-ONLY (batched parallel is CUDA-only)

Record the decision across README, ADR 0014, and the PR-A3c report: on
Apple-Silicon MLX, per-session binding is recall-preserving but sessions
are served serially. Batched/parallel B>1 cohort decode is CUDA-only,
blocked on MLX by the upstream B>1,L=1 quantized-decode kernel bug that
persists on the latest published mlx 0.31.2 / mlx-lm 0.31.3 and is not
Python-patchable. The 8.04-8.45x parallel throughput numbers are CUDA.

Co-authored-by: FluffyAIcode <FluffyAIcode@users.noreply.github.com>

Author: cursoragent

README.md

@@ -98,7 +98,7 @@ patterns — see [`docs/quickstart.md`](docs/quickstart.md).
 | **Restored verifier** | Gemma-4 26B-A4B (AR) + DFlash dLLM proposer + trained **f_θ**; **S5** keeps 5 full-attention layers exact, restores sliding layers → bounded resident KV, recall preserved. | `inference_engine.v04` (CUDA), `inference_engine.backends.mlx` (Apple Silicon) |
 | `SinkWindowVerifier` | Lightweight path: Qwen3 (0.6B / 1.7B / 4-bit MLX), sink+window K/V trim (ADR 0001 / 0002). | `kv_cache_proposer.verifier` (CPU), `inference_engine.backends.mlx.verifier` |
 | **Per-session binding** | `PerSessionVerifierRegistry` + coordinator resolver: each session owns isolated KV (shared weights) — true multi-tenant serving (PR-A3c, `--multi-tenant`). | `inference_engine.session.verifier_registry` |
-| **Batched scheduler** | Fuses a cohort's decode steps into one batched forward — **8.45× served throughput** at 8 sessions, recall 1.0. | `inference_engine.session.batch_scheduler` |
+| **Batched scheduler** | Fuses a cohort's decode steps into one batched forward — **8.45× served throughput** at 8 sessions, recall 1.0. **CUDA-only**: on Apple-Silicon MLX, `v0.4-mac` multi-tenant is **serial-only** (batched `B>1` decode is unsupported — upstream MLX `B>1, L=1` quantized-kernel bug, [ADR 0014](docs/adr/0014-agent-connection-capacity-and-cross-host-topology-tests.md)). | `inference_engine.session.batch_scheduler` |
 | `AppendTokens` / `Generation` coordinators | Drive prefill / incremental forward / greedy decode; route per-session (multi-tenant) or single. | `inference_engine.session.{coordinator,generator}` |
 | Python / TypeScript SDKs | `kakeya.Client` / `Session` (sync gRPC); `@kakeya/runtime` (Node 20+). | [`sdks/`](sdks/) |
 | HTTP shim (deprecated) | OpenAI-compatible `/v1/chat/completions`; `Deprecation` + `Sunset` headers. | `inference_engine.server.app` |
@@ -289,6 +289,13 @@ owns isolated KV (shared weights) — making serving truly multi-tenant, and a
 at 8 sessions with **per-session recall 1.0** (see the multi-tenant results
 below / [ADR 0014 §3.4–3.7](docs/adr/0014-agent-connection-capacity-and-cross-host-topology-tests.md)
 and the [detailed report](docs/reports/pr-a3c-multitenant-serving-test-report.md)).
+**Platform scope:** the batched/parallel cohort path is **CUDA-only**. On
+Apple-Silicon **MLX, `v0.4-mac` multi-tenant is serial-only** — per-session
+binding still gives isolated, recall-preserving sessions, but they are served
+**one at a time**; batched `B>1` decode is blocked by an upstream MLX
+quantized-kernel bug (`B>1, L=1` → per-session recall collapses to 0.125, while
+serialized stays 1.0; confirmed on the latest published `mlx 0.31.2 / mlx-lm
+0.31.3` — [ADR 0014 §3.4](docs/adr/0014-agent-connection-capacity-and-cross-host-topology-tests.md)).
 Pushing the connection sweep further (preset `agent-capacity-stress`, the
 open-file-descriptor limit `RLIMIT_NOFILE` raised to 100k / hard unlimited on
 the Mac — each connection uses one descriptor) shows the true ceilings: **the
@@ -584,9 +591,9 @@ scripts/
 | Milestone | Status | Description |
 | --- | --- | --- |
 | Session-bound gRPC runtime | ✅ shipped | Long-running gRPC `RuntimeService`, Python + TS SDKs, bounded memory + prefill (4-h Mac M4 evidence), Mac M4 self-hosted integration gate |
-| **v0.4 for Mac (`v0.4-mac`)** | ✅ shipped | MLX restored Gemma-4 26B engine: bounded KV (~90% saved), recall 1.0, ≈AR-parity spec-decode |
+| **v0.4 for Mac (`v0.4-mac`)** | ✅ shipped | MLX restored Gemma-4 26B engine: bounded KV (~90% saved), recall 1.0, ≈AR-parity spec-decode. Multi-tenant is **serial-only** (no batched `B>1` decode — upstream MLX kernel bug, [ADR 0014](docs/adr/0014-agent-connection-capacity-and-cross-host-topology-tests.md)) |
 | **v0.4 for CUDA (`v0.4-cuda`)** | ✅ shipped | Restored Gemma-4 26B engine on NVIDIA: fused DFlash spec-decode **1.79–2.06× AR**, 44–87× KV saving, recall 1.0 |
-| **v0.4 multi-tenant (PR-A3c)** | ✅ shipped | Per-session binding (isolated KV, shared weights) + batched scheduler — **8.45× served throughput**, per-session recall 1.0 |
+| **v0.4 multi-tenant (PR-A3c)** | ✅ shipped | Per-session binding (isolated KV, shared weights) on both platforms. **CUDA**: batched scheduler → **8.45× served throughput**, per-session recall 1.0. **MLX (Mac): serial-only** (sessions served one at a time; batched parallel decode unsupported upstream) |
 | Async continuous batching | designing | Dynamic mid-flight arrival + ragged-length cohorts under the async gRPC `Generate` handlers (current batcher is fixed-cohort) |
 | Deployment polish | queued | PyPI + npm publishing, GHCR Docker image, `kakeya prewarm` CLI, `kakeya chat` REPL |
 | Cross-request KV reuse | designing | Sessions survive across requests on gRPC; turns intra-session drift into 0 ms inter-request drift |

docs/adr/0014-agent-connection-capacity-and-cross-host-topology-tests.md

@@ -371,6 +371,18 @@ remains: **serialized**, or the `L≥2` padding probe (recall-safe but 0.67×).
 Evidence: `results/research/k3_mac_bridge_mlx_upstream_batch_probe.json` +
 `.mac-bridge/logs/mlx-upgrade-{0,1,2}.log`.
 
+**DECISION — MLX `v0.4-mac` multi-tenant is SERIAL-ONLY.** Given the upstream
+bug is present on the latest published mlx/mlx-lm and is not Python-patchable,
+the shipped Mac multi-tenant path is **per-session binding served serially**
+(one session decoded at a time): isolated, recall-preserving (1.0) sessions on
+shared weights, with bounded resident KV. **Batched/parallel cohort decode
+(`B>1`) is NOT supported on MLX** and remains a **CUDA-only** capability (§3.5 /
+§3.7: 8.04–8.45× near-linear, recall 1.0). Re-evaluate the Mac batched path
+only if (a) a future mlx release fixes the `B>1, L=1` quantized-decode kernel,
+or (b) a from-source mlx `main` build / upstream patch lands; the `L≥2` padding
+workaround stays available as a recall-safe (but sub-parity, 0.67×) escape
+hatch in the meantime.
+
 ## 4. Case 2 — cross-host proposer/verifier (FEASIBILITY VERDICT)
 
 ### 4.1 Verdict: the requested topology is not implementable today, and is architecturally bounded out

docs/reports/pr-a3c-multitenant-serving-test-report.md

@@ -100,11 +100,20 @@ PR-A3c delivers, recall-preserving (recall 1.0 throughout), the three multi-tena
 properties together:
 
 - **Bounded memory** — ~4.2× more concurrent agents per GB (§3.1).
-- **Parallel throughput** — 8.04× engine-level (§3.2) / **8.45× through the
-  served per-session adapters via the batched scheduler** (§3.4).
+- **Parallel throughput (CUDA)** — 8.04× engine-level (§3.2) / **8.45× through
+  the served per-session adapters via the batched scheduler** (§3.4). This is
+  **CUDA-only**; see the platform note below.
 - **Correct isolation** — true multi-tenant serving end-to-end through gRPC,
   per-session recall 1.0 (§3.3).
 
+**Platform scope — MLX (`v0.4-mac`) multi-tenant is serial-only.** Per-session
+binding (isolated KV, shared weights, recall 1.0) holds on both platforms, but
+the **batched/parallel cohort path is CUDA-only**. On Apple-Silicon MLX,
+batched `B>1` decode collapses per-session recall to 0.125 (vs serialized 1.0)
+because of an upstream MLX `B>1, L=1` quantized-decode kernel bug that persists
+on the latest published `mlx 0.31.2 / mlx-lm 0.31.3` and is not Python-patchable
+(ADR 0014 §3.4). Mac therefore serves multi-tenant sessions **serially**.
+
 ## 5. Remaining work (productization)
 
 - **Async continuous batching transport**: wire the batched scheduler under the
@@ -113,9 +122,11 @@ properties together:
   **dynamic mid-flight arrival + ragged-length** cohorts (this report's
   scheduler is a fixed synchronized cohort, the dominant burst case).
 - **Batched fused spec-decode** (DFlash is batch-1 today).
-- **Mac served path**: the served MLX gemma verifier can't load gemma-4's nested
-  config (`MLXSinkWindowVerifier`) — a v0.4 item; CUDA is the recall-preserving
-  served path today.
+- **Mac served path**: multi-tenant on MLX is **serial-only by decision** (ADR
+  0014 §3.4) — batched `B>1` decode is blocked by an upstream MLX kernel bug, so
+  Mac serves sessions one at a time (recall-preserving). CUDA is the batched
+  parallel path today. Revisit only if a future mlx release / source build fixes
+  the `B>1, L=1` quantized-decode kernel.
 
 ## 6. Evidence index