Repository navigation
Conversation
The recipe's measurements covered only the MLX affine checkpoint. Add the ModelOpt one, measured with beec1aa built unpatched for sm_121: decode and concurrency from bench_openai and bench_concurrent, cold first tokens from bench_cache with exact token arrays, the served exactness checks, the seams at fixed histories, and what the 128 GB machine admits.
|
The
|
|
Thanks @akol1 for carrying @balancedscorpion's measurements with their authorship. The supplied #952 write-up supports the code rates, cold prompts, memory figures and equality checks; I couldn't match the concurrent-chat rates to a receipt there, so please attach that receipt or omit that row. We'll add: “These are @balancedscorpion's measurements on Posted by Ash's AI Assistant Synthena |
…ncy row The ashhart#952 write-up records the concurrent aggregate for code at 1 / 4 / 8 streams, not for chat, so the chat row is dropped rather than carried without a receipt. The section now says whose measurements these are, which build they ran, and that 1.1.2 has not remeasured them.
|
Row dropped and the provenance sentence added at The #952 write-up's concurrency receipts are The section now reads: "These are @balancedscorpion's measurements on Documentation only, so no test receipt; |
What this changes
Adds the ModelOpt
nvidia/Qwen3.8-Flash-Next-FP4-Localcheckpoint todocs/recipes/flash-next-cuda.md. The recipe'smeasurements covered only the MLX affine checkpoint; this is one new "### The ModelOpt checkpoint on one GB10"
section with the decode and concurrency table, the cold first-token timings, the served exactness checks and the
memory seams.
The commit is unchanged from #952 (
b6fd0d92, authored by @balancedscorpion). The 1.1.0 merge53abe455took theearlier head
d71a2361, so this docs commit and the experts-rows test fix were left out. The test fix is going in asits own test-only PR so this stays one change. Documentation only: no source, tests, fixtures or generated files
change.
Review follow-up
The concurrent-chat row is dropped: the #952 write-up records the 1 / 4 / 8-stream aggregate for code only, so there is no receipt for the chat numbers. The section now dates the measurements as @balancedscorpion's, on
beec1aad6, the #952 branch build before 1.1.0, not remeasured on 1.1.2. Diff:c44892e5f..1613ff9d7, documentation only.Receipt
b6fd0d92on top of76aafcf27; documentation only. The measurements are the commit author's:GB10 (128 GB, driver 580.178, CUDA 13.3),
beec1aad6built unpatched for sm_121, checkpointnvidia/Qwen3.8-Flash-Next-FP4-Localatb9ce18a6,--prompt-cache-gib 0, with the recipe'sbench_openai,bench_concurrentandbench_cache. They are quoted as measured; nothing was generated here.author's receipts (each concurrent reply equals its solo run, drafted equals
"draft": false, cancelled/restartedequal fresh, cold
ttft_s, peak host and device memory).receipt").
git diff --checkis clean. The lean checker does not readdocs/.a new claim. Host and CUDA CI on the PR covers the tree.
Checklist
"draft": false, a resumed prompt equals a fresh one, and concurrent equals solo. (docs only)SUMMARYline is in the receipt. (no kernel or backend change)