Skip to content

Commit c12bf40

Browse files
docs(adr-0014): mlx/mlx-lm upgrade re-test — B>1,L=1 bug persists on PyPI-latest
pip install --upgrade was a no-op (Mac already at mlx 0.31.2 / mlx_lm 0.31.3, newest on PyPI). Native L=1 batched probe on that latest build: batched recall 0.125 vs serialized 1.0, upstream_l1_batch_bug_fixed=false. tok0 (from prefill) matches all 8 rows; divergence only in L=1 decode steps — the B>1,L=1 signature. Upgrade can't fix it; only a from-source mlx git-main build or upstream patch would. Co-authored-by: FluffyAIcode <FluffyAIcode@users.noreply.github.com>
1 parent 9a11dd9 commit c12bf40

2 files changed

Lines changed: 49 additions & 0 deletions

File tree

docs/adr/0014-agent-connection-capacity-and-cross-host-topology-tests.md

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -344,6 +344,33 @@ padding tax exceeds the batching gain at this scale (**0.67×**), so it is a
344344
or a much larger cohort / cheaper verify. Evidence:
345345
`results/research/k3_mac_bridge_mlx_batched_pad_decode.json`.
346346

347+
**mlx/mlx-lm upgrade re-test — bug PERSISTS on the latest published release.**
348+
We attempted `pip install --upgrade mlx mlx-lm` on the Mac runner (preset
349+
`mlx-upgrade`): it was a **no-op** — the runner was already at the newest
350+
versions on PyPI (`mlx=0.31.2`, `mlx_lm=0.31.3`, `mlx-metal=0.31.2`; no newer
351+
stable or pre-release exists on the index). A self-contained probe (preset
352+
`mlx-upstream-batch-probe`, zero `inference_engine` imports, native
353+
`model.make_cache()`, plain `L=1` batched decode) then re-ran the parallel
354+
test on that latest build:
355+
356+
| metric | batched (native `L=1`) | serialized (truth) |
357+
| --- | --- | --- |
358+
| per-session recall | **0.125**| 1.0 |
359+
| per-row tok0 vs serialized | **all 8 match** (tok0 is from prefill) ||
360+
| aggregate decode tok/s | 29.6 (recall void) | 21.7 |
361+
| `upstream_l1_batch_bug_fixed` | **false** ||
362+
363+
The first decoded token (computed from the `L>1` prefill logits) matches on
364+
**all 8 rows**, and the divergence appears only in the subsequent `L=1` decode
365+
steps (rows 1–7 fail) — exactly the `B>1, L=1` signature. **Conclusion:** the
366+
latest PyPI mlx/mlx-lm still ships the bug; a pip upgrade cannot fix it because
367+
nothing newer is published. The only further "upgrade" is a from-source
368+
`mlx` git-`main` build (compiles Metal kernels; invasive on the pinned
369+
runner env) or an upstream patch/issue. Recall-safe Mac parallelism therefore
370+
remains: **serialized**, or the `L≥2` padding probe (recall-safe but 0.67×).
371+
Evidence: `results/research/k3_mac_bridge_mlx_upstream_batch_probe.json` +
372+
`.mac-bridge/logs/mlx-upgrade-{0,1,2}.log`.
373+
347374
## 4. Case 2 — cross-host proposer/verifier (FEASIBILITY VERDICT)
348375

349376
### 4.1 Verdict: the requested topology is not implementable today, and is architecturally bounded out
Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
{
2+
"kind": "mlx_upstream_batch_probe",
3+
"config": {
4+
"sessions": 8,
5+
"modal_prompt_len": 1149,
6+
"max_new_tokens": 24,
7+
"verifier_path": "/Users/fluffy314/kakeya-models/gemma-4-26B-A4B-it-mlx-4bit",
8+
"mlx_version": "0.31.2",
9+
"mlx_lm_version": "0.31.3",
10+
"decode": "native L=1 batched (model.make_cache())"
11+
},
12+
"serialized": {
13+
"aggregate_tps": 21.691,
14+
"recall": 1.0
15+
},
16+
"batched": {
17+
"aggregate_tps": 29.613,
18+
"recall": 0.125
19+
},
20+
"batched_speedup_vs_serialized": 1.37,
21+
"upstream_l1_batch_bug_fixed": false
22+
}

0 commit comments

Comments
 (0)