docs(vision): mlx0330nv campaign — nvfp4 fleet on 0.33.0, and the 12b/31b think-on split

[glennneuber]

The first vsuite campaign on the v0.33.0 fold's build (0.33.0-maxusai-21cfe88e, llama.cpp b10488 / MLX 27fec909), scoped by operator direction to nvfp4 tags only (no GGUF, mxfp8, or bf16). Two files: the campaign doc with both summarizer tables pasted verbatim, and the learnings-log entry for the gemma4:12b think-on runaway.

Findings

• The fleet is fully functional think-off: all five nvfp4 models (gemma4 12b/26b/31b, qwen3.8:27b, qwen3.6:35b-a3b) converge at the 16384 start rung with scene grounding 6/6 across the board (IoU 0.954–0.999, qwen3.8 best at 0.999 with the only 5/5 name_bbox), perfect serial + invoice extraction. Throughput tracks architecture: 26b / 35b-a3b at 75–90 tok/s (488–581 req/h), dense 31b at 13 tok/s (80 req/h).
• Think-on splits the gemma4 family. 31b converges at the first rung (27/27 arms, max eval 3,539 of 8,192, quality identical to think-off at 2.4× cost). 12b runs away on the bbox arms under the sanctioned on-policy sampling (card:gemma4+temp0): 25/27 arms capped at 8,192; 20/27 still capped at num_predict 57,344; descoped mid-131072 after ~9.5 h wall for 7 scores. The learnings-log entry is deliberately scope-fenced against the 2026-08-13 finding (which blamed off-policy sampling, on a different build and models) — this is a new observation, not a refutation.
• The seven 12b arms that did finish are the anchored/pinned ones — consistent with the 2026-08-19 qwen3.6 finding that the anchor bounds reasoning.
• The 12b think-on cell stays open per ADR 0012 conv 9: its scores file holds 7 finished + 20 capped rows (highest rung 65536) and must not be summarized as results.

Provenance

One host (10.8.0.3, native serve on :11436 via serve-apple-mlx.sh, cold server per cell), one build, campaign tag mlx0330nv1, power mode stamped per cell (the 12b think-off cell predates the powermode 0→2 shift; noted in Limits). Scores untracked on-host per convention; the doc and log entry are the durable record.

🤖 Generated with Claude Code