[trickyfalcon]

Draft, opened early per #962 so the design can be steered before the kernels harden. The native Metal backend for nemotron_h_puzzle (Nemotron 3.5 Bolt 74B-A9B), on the current release (v1.1.0), which already parses the Puzzle config and serves it on CUDA; this adds the Metal side. It serves the published checkpoint end to end (bytebunker/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-MLX-4bit): on an M3 Max (128 GB) it loads in ~6 s and decodes at ~40 tok/s.

Receipts (M3 Max, applegpu_g15, the published MLX-4bit build)

Pinned checkpoint: config.json sha256 7279c7543826…, index 3b6189b91b55…, all nine shards hashed in the fidelity report (--hash-weights). Reference: MLX 0.32.3, mlx-lm 0.32.0. Engine: this branch.

Check	Result
Per-op bits vs MLX (check_mlx_ops.py --ops bolt_gemv,bolt_route,bolt_combine)	236/236 bit-identical: gate gemv at every expert count (240/304/368/432/496), both latent projections, the bf16 lm_head, group_expert_select at k 9, and the latent combine; MLX's own metallib kernel replayed at each shape agrees too
Per row	every row of a 2/9/16-row pass (64 for the combine) equals its own one-row MLX op, bit for bit
concurrent == solo (xpu_concurrent_receipt.py)	pass at 2 and 4 streams, 3 rounds each, greedy and seeded streams
resumed == fresh (xpu_resume_receipt.py, CONTRIBUTING.md, two turns)	pass: turn 2 resumed from 1,502 cached tokens has the same token_sha as on a server with --prompt-cache-gib 0, greedy and seeded
drafted == plain	n/a: the checkpoint ships no MTP weights (num_nextn_predict_layers: 0)
Lightning unchanged	test-golden 0 unexpected; and Lightning 30B-A3B MLX-4bit (d9d758fb) served by main and by this branch on the M3 Max: 32/32 requests the same token_sha (8 prompts × greedy/seeded × chat/raw)
zig build test, test-bolt-host, test-kernel-contract, tf-metal-tests, lean	pass; lean 0 new

Teacher-forced fidelity vs mlx-lm one token at a time (tools/zig/nemotron_fidelity.py, KL(mlx ‖ native) per position in nats). mlx-lm's own 32-chunk run against its serial run is given as a reference spread, not a bound:

fixture	tokens	top-1	logit RMS	KL mean / p99 / max	mlx chunk32 vs serial: RMS, KL mean	NLL native / mlx
fibonacci-raw	15	0.933	0.270	0.0040 / 0.0099 / 0.010	0.444, 0.0068	2.892 / 2.841
gpu-chat-no-think	35	0.943	0.189	0.0106 / 0.083 / 0.109	0.191, 0.0078	5.717 / 5.724
contribution-guide-512	512	0.918	0.401	0.0110 / 0.040 / 0.091	0.262, 0.0098	3.298 / 3.306
python-stdlib-2048	2048	0.986	0.557	0.0053 / 0.065 / 0.875	0.484, 0.0051	0.365 / 0.364

A correction to the earlier version of this table: native RMS is above mlx-lm's chunk spread on two fixtures (contribution-guide and python-stdlib), so "inside the floor" did not hold. Moving the gate, latent projections, head, route and combine onto MLX-exact kernels did not lower these numbers. The previous build and this one differ from each other by RMS 0.17–0.51, the same size as either differs from mlx-lm. The rest of the decode path on M1–M4 (the row kernels for the 4-bit projections, Mamba and attention) is not MLX-exact, and the model turns any one-ULP difference into this spread. The per-op table is the bit-level claim; the fidelity table bounds the whole-model effect.

What's in it

• Refusals before load: a puzzle layer whose top-k exceeds its own expert count; a latent projection that is missing, quantized, transposed or not bf16 (and experts stored over hidden); Bolt on a tensor-unit chip (M5), before any weight is read.
• Per-layer loader: each MoE layer's own expert count and width; routed experts in the 1024 latent through the runtime-dim row kernels; fc1/fc2_latent_proj and the unquantized bf16 lm_head read in place.
• The latent MoE on the shared ops (bolt.zig), in mlx_lm's NemotronHMoE order: gate gemv → tf_route_topk → fc1_latent_proj → routed experts → tf_moe_combine_rows → fc2_latent_proj → + shared → + residual. The gemvs, route and combine come from ops/ and the prefill set compiles them with MLX's options; the hand-written route, dense and combine kernels are gone.
• ops/gemv.metal gains MLX's bm4/bm8 bn1 gemv (tf_gemv_bf16_bm{4,8}_tm4) beside its bn8 one. tf_moe_combine now follows col_reduce_small's lane order past 8 picks (8 lanes, pick 8 onto pick 0); at 8 picks or fewer its sums are unchanged.
• Bolt-width instantiations of the generated Mamba tree conv/scan, gated group norm and add_norm kernels, hand-authored in the generator's style (the nemotron kernel generator isn't in the public repo).
• Tools: bolt_op_cases.py (the per-op cases), tf-nemotron-forward and nemotron_fidelity.py (teacher-forced logits, KL, pins).

Not yet / next

• M5 lane kernels: refused for now; Bolt variants of the lane kernels are a follow-up.
• Fused prefill (prefill_mixers) for the latent MoE; prompts run through the decode kernels 16 rows at a time.
• Streams don't batch yet (aggregate ~40 tok/s across 2–4 streams).

Deferred, with their gates kept

• Image input (RADIO): text backbone only here; the RADIO image receipts stay a shipping gate before Bolt is listed as served.
• The MTP draft head (no MTP weights in this checkpoint); drafted == plain applies once a checkpoint carries them.

[ashhart]

Thanks @trickyfalcon. The common lane host and latent combination order are now in place, and your M3 Max receipts help.

Please keep it as a draft while you reject top-k above any layer's expert count, add the latent-tensor negatives, and refuse unsupported tensor-unit chips before loading. The M5 path can reach unloaded row pipelines and Lightning-sized kernels. Reuse the shared GPU core for reusable operations and retain RADIO images as a shipping gate.

The fidelity table exceeds its quoted RMS floor on two fixtures; please correct that claim and attach pinned KL, per-operation bit/row, Lightning and session-equivalence receipts. Drafting is correctly marked inapplicable without MTP weights.

Posted by Ash's AI Assistant Synthena

[trickyfalcon]

Thanks. All of that is in, and it stays a draft.

• Top-k per layer: a puzzle layer whose top-k exceeds its own expert count is refused while the config is read. The old check compared against the largest layer only.
• Latent-tensor negatives: a missing, quantized, transposed or non-bf16 fc1/fc2_latent_proj is refused with its name, and experts stored over hidden are refused too. Each case has a test.
• Tensor-unit chips: Bolt is refused on M5 before any weight is read, so the Lightning-sized lane kernels and the unloaded row pipelines can't be reached.
• Shared GPU core: the hand-written route, dense and combine kernels are gone. Bolt's gate, latent projections and bf16 head use ops/gemv.metal, which now also has MLX's bm4/bm8 bn1 variant. Routing uses tf_route_topk, and the combine uses tf_moe_combine_rows. One fix came out of this: past 8 picks, col_reduce_small splits the picks over 8 lanes and adds pick 8 onto pick 0. The combine now does the same; at 8 picks or fewer its sums are unchanged.
• RADIO: kept as a shipping gate.

Receipts, on the M3 Max with the pinned checkpoint:

• Per op: 236/236 cases are bit-identical to MLX 0.32.3 at Bolt's shapes. Every row of a 1–16 row pass equals its own one-row MLX op, and MLX's metallib replay agrees.
• Sessions: concurrent == solo at 2 and 4 streams, greedy and seeded. Resumed == fresh, greedy and seeded: a turn resumed from 1,502 cached tokens matches the same turn on a server with no cache.
• Lightning: test-golden has 0 unexpected differences. Lightning 30B served from main and from this branch gives the same tokens on 32/32 requests.
• KL: pinned KL per position is now in the fidelity table.

I corrected the fidelity claim. Native RMS is above mlx-lm's chunk spread on two fixtures. Switching to the exact ops didn't move that, because the rest of the M1–M4 decode path isn't MLX-exact, and the model spreads one-ULP differences to that size. So the per-op table is the bit claim, and the fidelity table bounds the whole-model effect. Details are in the updated description.

[ashhart]

Thanks @trickyfalcon. The per-layer top-k bound, latent-tensor negatives, early tensor-unit refusal and shared-op reuse address our earlier source asks. Please also make Model.load release the Metal device and queue it creates before that refusal; its current error path releases neither. Keep this draft for RADIO images and the full numerical/session gate. Your M3 results are useful; please attach the updated pinned KL/fidelity table and raw operator, row-invariance, Lightning and session receipts. We'll retain the unchanged numerical bars and run the integrated M3 acceptance and config-only M5 refusal/cleanup cells before shipping this backend.

Posted by Ash's AI Assistant Synthena

[trickyfalcon]

Thanks. Model.load now releases the device and queue on every refusal (2667794): each gets an errdefer as soon as it's created. A new test counts the device's references before and after a refused load. It fails without the fix and passes with it. The Bolt-on-M5 refusal can't fire on an M3, so the test refuses on the time-step clamp instead. That check runs right after the tensor-unit check, inside the same cleanup. Your config-only M5 cell covers the refusal itself.

It stays a draft for RADIO and the full gate. The raw receipts are below, all from the M3 Max on the pinned checkpoint. The fix only touches the load error path, so these runs still stand.

Per operator and row invariance: check_mlx_ops.py --ops bolt_gemv,bolt_route,bolt_combine (236/236)

OK   gemv      L1.gate E=240 K=4096 N=240 rows=1 s0     gemv bm1 bn8 sn32 tm4 tn4           mlx     3.28 us  ours     3.35 us  (1.021x) 
OK   gemv      L1.gate E=240 K=4096 N=240 rows=1 s1     gemv bm1 bn8 sn32 tm4 tn4          
OK   gemv      L1.gate E=240 K=4096 N=240 rows=1 s2     gemv bm1 bn8 sn32 tm4 tn4          
OK   gemv      L1.gate E=240 K=4096 N=240 rows=2 s0     gemv bm1 bn8 sn32 tm4 tn4          
OK   gemv      L1.gate E=240 K=4096 N=240 rows=2 s1     gemv bm1 bn8 sn32 tm4 tn4          
OK   gemv      L1.gate E=240 K=4096 N=240 rows=2 s2     gemv bm1 bn8 sn32 tm4 tn4          
OK   gemv      L1.gate E=240 K=4096 N=240 rows=9 s0     gemv bm1 bn8 sn32 tm4 tn4          
OK   gemv      L1.gate E=240 K=4096 N=240 rows=9 s1     gemv bm1 bn8 sn32 tm4 tn4          
OK   gemv      L1.gate E=240 K=4096 N=240 rows=9 s2     gemv bm1 bn8 sn32 tm4 tn4          
OK   gemv      L1.gate E=240 K=4096 N=240 rows=16 s0    gemv bm1 bn8 sn32 tm4 tn4           ours    19.69 us 
OK   gemv      L1.gate E=240 K=4096 N=240 rows=16 s1    gemv bm1 bn8 sn32 tm4 tn4          
OK   gemv      L1.gate E=240 K=4096 N=240 rows=16 s2    gemv bm1 bn8 sn32 tm4 tn4          
OK   gemv      L8.gate E=304 K=4096 N=304 rows=1 s0     gemv bm4 bn1 sn32 tm4 tn4           mlx     7.51 us  ours     7.24 us  (0.963x) 
OK   gemv      L8.gate E=304 K=4096 N=304 rows=1 s1     gemv bm4 bn1 sn32 tm4 tn4          
OK   gemv      L8.gate E=304 K=4096 N=304 rows=1 s2     gemv bm4 bn1 sn32 tm4 tn4          
OK   gemv      L8.gate E=304 K=4096 N=304 rows=2 s0     gemv bm4 bn1 sn32 tm4 tn4          
OK   gemv      L8.gate E=304 K=4096 N=304 rows=2 s1     gemv bm4 bn1 sn32 tm4 tn4          
OK   gemv      L8.gate E=304 K=4096 N=304 rows=2 s2     gemv bm4 bn1 sn32 tm4 tn4          
OK   gemv      L8.gate E=304 K=4096 N=304 rows=9 s0     gemv bm4 bn1 sn32 tm4 tn4          
OK   gemv      L8.gate E=304 K=4096 N=304 rows=9 s1     gemv bm4 bn1 sn32 tm4 tn4          
OK   gemv      L8.gate E=304 K=4096 N=304 rows=9 s2     gemv bm4 bn1 sn32 tm4 tn4          
OK   gemv      L8.gate E=304 K=4096 N=304 rows=16 s0    gemv bm4 bn1 sn32 tm4 tn4           ours    21.25 us 
OK   gemv      L8.gate E=304 K=4096 N=304 rows=16 s1    gemv bm4 bn1 sn32 tm4 tn4          
OK   gemv      L8.gate E=304 K=4096 N=304 rows=16 s2    gemv bm4 bn1 sn32 tm4 tn4          
OK   gemv      L37.gate E=368 K=4096 N=368 rows=1 s0    gemv bm4 bn1 sn32 tm4 tn4           mlx     7.52 us  ours     7.20 us  (0.958x) 
OK   gemv      L37.gate E=368 K=4096 N=368 rows=1 s1    gemv bm4 bn1 sn32 tm4 tn4          
OK   gemv      L37.gate E=368 K=4096 N=368 rows=1 s2    gemv bm4 bn1 sn32 tm4 tn4          
OK   gemv      L37.gate E=368 K=4096 N=368 rows=2 s0    gemv bm4 bn1 sn32 tm4 tn4          
OK   gemv      L37.gate E=368 K=4096 N=368 rows=2 s1    gemv bm4 bn1 sn32 tm4 tn4          
OK   gemv      L37.gate E=368 K=4096 N=368 rows=2 s2    gemv bm4 bn1 sn32 tm4 tn4          
OK   gemv      L37.gate E=368 K=4096 N=368 rows=9 s0    gemv bm4 bn1 sn32 tm4 tn4          
OK   gemv      L37.gate E=368 K=4096 N=368 rows=9 s1    gemv bm4 bn1 sn32 tm4 tn4          
OK   gemv      L37.gate E=368 K=4096 N=368 rows=9 s2    gemv bm4 bn1 sn32 tm4 tn4          
OK   gemv      L37.gate E=368 K=4096 N=368 rows=16 s0   gemv bm4 bn1 sn32 tm4 tn4           ours    26.81 us 
OK   gemv      L37.gate E=368 K=4096 N=368 rows=16 s1   gemv bm4 bn1 sn32 tm4 tn4          
OK   gemv      L37.gate E=368 K=4096 N=368 rows=16 s2   gemv bm4 bn1 sn32 tm4 tn4          
OK   gemv      L21.gate E=432 K=4096 N=432 rows=1 s0    gemv bm4 bn1 sn32 tm4 tn4           mlx     7.55 us  ours     7.24 us  (0.960x) 
OK   gemv      L21.gate E=432 K=4096 N=432 rows=1 s1    gemv bm4 bn1 sn32 tm4 tn4          
OK   gemv      L21.gate E=432 K=4096 N=432 rows=1 s2    gemv bm4 bn1 sn32 tm4 tn4          
OK   gemv      L21.gate E=432 K=4096 N=432 rows=2 s0    gemv bm4 bn1 sn32 tm4 tn4          
OK   gemv      L21.gate E=432 K=4096 N=432 rows=2 s1    gemv bm4 bn1 sn32 tm4 tn4          
OK   gemv      L21.gate E=432 K=4096 N=432 rows=2 s2    gemv bm4 bn1 sn32 tm4 tn4          
OK   gemv      L21.gate E=432 K=4096 N=432 rows=9 s0    gemv bm4 bn1 sn32 tm4 tn4          
OK   gemv      L21.gate E=432 K=4096 N=432 rows=9 s1    gemv bm4 bn1 sn32 tm4 tn4          
OK   gemv      L21.gate E=432 K=4096 N=432 rows=9 s2    gemv bm4 bn1 sn32 tm4 tn4          
OK   gemv      L21.gate E=432 K=4096 N=432 rows=16 s0   gemv bm4 bn1 sn32 tm4 tn4           ours    30.66 us 
OK   gemv      L21.gate E=432 K=4096 N=432 rows=16 s1   gemv bm4 bn1 sn32 tm4 tn4          
OK   gemv      L21.gate E=432 K=4096 N=432 rows=16 s2   gemv bm4 bn1 sn32 tm4 tn4          
OK   gemv      L48.gate E=496 K=4096 N=496 rows=1 s0    gemv bm4 bn1 sn32 tm4 tn4           mlx     7.56 us  ours     7.29 us  (0.963x) 
OK   gemv      L48.gate E=496 K=4096 N=496 rows=1 s1    gemv bm4 bn1 sn32 tm4 tn4          
OK   gemv      L48.gate E=496 K=4096 N=496 rows=1 s2    gemv bm4 bn1 sn32 tm4 tn4          
OK   gemv      L48.gate E=496 K=4096 N=496 rows=2 s0    gemv bm4 bn1 sn32 tm4 tn4          
OK   gemv      L48.gate E=496 K=4096 N=496 rows=2 s1    gemv bm4 bn1 sn32 tm4 tn4          
OK   gemv      L48.gate E=496 K=4096 N=496 rows=2 s2    gemv bm4 bn1 sn32 tm4 tn4          
OK   gemv      L48.gate E=496 K=4096 N=496 rows=9 s0    gemv bm4 bn1 sn32 tm4 tn4          
OK   gemv      L48.gate E=496 K=4096 N=496 rows=9 s1    gemv bm4 bn1 sn32 tm4 tn4          
OK   gemv      L48.gate E=496 K=4096 N=496 rows=9 s2    gemv bm4 bn1 sn32 tm4 tn4          
OK   gemv      L48.gate E=496 K=4096 N=496 rows=16 s0   gemv bm4 bn1 sn32 tm4 tn4           ours    35.40 us 
OK   gemv      L48.gate E=496 K=4096 N=496 rows=16 s1   gemv bm4 bn1 sn32 tm4 tn4          
OK   gemv      L48.gate E=496 K=4096 N=496 rows=16 s2   gemv bm4 bn1 sn32 tm4 tn4          
OK   gemv      fc1_latent_proj K=4096 N=1024 rows=1 s0  gemv bm4 bn1 sn32 tm4 tn4           mlx    10.54 us  ours    10.76 us  (1.021x) 
OK   gemv      fc1_latent_proj K=4096 N=1024 rows=1 s1  gemv bm4 bn1 sn32 tm4 tn4          
OK   gemv      fc1_latent_proj K=4096 N=1024 rows=1 s2  gemv bm4 bn1 sn32 tm4 tn4          
OK   gemv      fc1_latent_proj K=4096 N=1024 rows=2 s0  gemv bm4 bn1 sn32 tm4 tn4          
OK   gemv      fc1_latent_proj K=4096 N=1024 rows=2 s1  gemv bm4 bn1 sn32 tm4 tn4          
OK   gemv      fc1_latent_proj K=4096 N=1024 rows=2 s2  gemv bm4 bn1 sn32 tm4 tn4          
OK   gemv      fc1_latent_proj K=4096 N=1024 rows=9 s0  gemv bm4 bn1 sn32 tm4 tn4          
OK   gemv      fc1_latent_proj K=4096 N=1024 rows=9 s1  gemv bm4 bn1 sn32 tm4 tn4          
OK   gemv      fc1_latent_proj K=4096 N=1024 rows=9 s2  gemv bm4 bn1 sn32 tm4 tn4          
OK   gemv      fc1_latent_proj K=4096 N=1024 rows=16 s0 gemv bm4 bn1 sn32 tm4 tn4           ours    69.70 us 
OK   gemv      fc1_latent_proj K=4096 N=1024 rows=16 s1 gemv bm4 bn1 sn32 tm4 tn4          
OK   gemv      fc1_latent_proj K=4096 N=1024 rows=16 s2 gemv bm4 bn1 sn32 tm4 tn4          
OK   gemv      fc2_latent_proj K=1024 N=4096 rows=1 s0  gemv bm8 bn1 sn32 tm4 tn4           mlx     6.70 us  ours     6.33 us  (0.946x) 
OK   gemv      fc2_latent_proj K=1024 N=4096 rows=1 s1  gemv bm8 bn1 sn32 tm4 tn4          
OK   gemv      fc2_latent_proj K=1024 N=4096 rows=1 s2  gemv bm8 bn1 sn32 tm4 tn4          
OK   gemv      fc2_latent_proj K=1024 N=4096 rows=2 s0  gemv bm8 bn1 sn32 tm4 tn4          
OK   gemv      fc2_latent_proj K=1024 N=4096 rows=2 s1  gemv bm8 bn1 sn32 tm4 tn4          
OK   gemv      fc2_latent_proj K=1024 N=4096 rows=2 s2  gemv bm8 bn1 sn32 tm4 tn4          
OK   gemv      fc2_latent_proj K=1024 N=4096 rows=9 s0  gemv bm8 bn1 sn32 tm4 tn4          
OK   gemv      fc2_latent_proj K=1024 N=4096 rows=9 s1  gemv bm8 bn1 sn32 tm4 tn4          
OK   gemv      fc2_latent_proj K=1024 N=4096 rows=9 s2  gemv bm8 bn1 sn32 tm4 tn4          
OK   gemv      fc2_latent_proj K=1024 N=4096 rows=16 s0 gemv bm8 bn1 sn32 tm4 tn4           ours    61.71 us 
OK   gemv      fc2_latent_proj K=1024 N=4096 rows=16 s1 gemv bm8 bn1 sn32 tm4 tn4          
OK   gemv      fc2_latent_proj K=1024 N=4096 rows=16 s2 gemv bm8 bn1 sn32 tm4 tn4          
OK   gemv      lm_head K=4096 N=131072 rows=1 s0        gemv bm8 bn1 sn32 tm4 tn4           mlx  2824.12 us  ours  2846.31 us  (1.008x) 
OK   gemv      lm_head K=4096 N=131072 rows=1 s1        gemv bm8 bn1 sn32 tm4 tn4          
OK   gemv      lm_head K=4096 N=131072 rows=1 s2        gemv bm8 bn1 sn32 tm4 tn4          
OK   gemv      lm_head K=4096 N=131072 rows=2 s0        gemv bm8 bn1 sn32 tm4 tn4          
OK   gemv      lm_head K=4096 N=131072 rows=2 s1        gemv bm8 bn1 sn32 tm4 tn4          
OK   gemv      lm_head K=4096 N=131072 rows=2 s2        gemv bm8 bn1 sn32 tm4 tn4          
OK   gemv      lm_head K=4096 N=131072 rows=9 s0        gemv bm8 bn1 sn32 tm4 tn4          
OK   gemv      lm_head K=4096 N=131072 rows=9 s1        gemv bm8 bn1 sn32 tm4 tn4          
OK   gemv      lm_head K=4096 N=131072 rows=9 s2        gemv bm8 bn1 sn32 tm4 tn4          
OK   gemv      lm_head K=4096 N=131072 rows=16 s0       gemv bm8 bn1 sn32 tm4 tn4           ours 10194.65 us 
OK   gemv      lm_head K=4096 N=131072 rows=16 s1       gemv bm8 bn1 sn32 tm4 tn4          
OK   gemv      lm_head K=4096 N=131072 rows=16 s2       gemv bm8 bn1 sn32 tm4 tn4          
OK   route     E=240 k=9 rows=1 p0                      group_expert_select (fusions, sort, take, sum)  ours    21.56 us 
OK   route     E=240 k=9 rows=1 p1                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=240 k=9 rows=1 p2                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=240 k=9 rows=1 p3                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=240 k=9 rows=1 p4                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=240 k=9 rows=1 p5                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=240 k=9 rows=2 p0                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=240 k=9 rows=2 p1                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=240 k=9 rows=2 p2                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=240 k=9 rows=2 p3                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=240 k=9 rows=2 p4                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=240 k=9 rows=2 p5                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=240 k=9 rows=9 p0                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=240 k=9 rows=9 p1                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=240 k=9 rows=9 p2                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=240 k=9 rows=9 p3                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=240 k=9 rows=9 p4                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=240 k=9 rows=9 p5                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=240 k=9 rows=16 p0                     group_expert_select (fusions, sort, take, sum) 
OK   route     E=240 k=9 rows=16 p1                     group_expert_select (fusions, sort, take, sum) 
OK   route     E=240 k=9 rows=16 p2                     group_expert_select (fusions, sort, take, sum) 
OK   route     E=240 k=9 rows=16 p3                     group_expert_select (fusions, sort, take, sum) 
OK   route     E=240 k=9 rows=16 p4                     group_expert_select (fusions, sort, take, sum) 
OK   route     E=240 k=9 rows=16 p5                     group_expert_select (fusions, sort, take, sum) 
OK   route     E=304 k=9 rows=1 p0                      group_expert_select (fusions, sort, take, sum)  ours    26.64 us 
OK   route     E=304 k=9 rows=1 p1                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=304 k=9 rows=1 p2                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=304 k=9 rows=1 p3                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=304 k=9 rows=1 p4                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=304 k=9 rows=1 p5                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=304 k=9 rows=2 p0                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=304 k=9 rows=2 p1                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=304 k=9 rows=2 p2                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=304 k=9 rows=2 p3                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=304 k=9 rows=2 p4                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=304 k=9 rows=2 p5                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=304 k=9 rows=9 p0                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=304 k=9 rows=9 p1                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=304 k=9 rows=9 p2                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=304 k=9 rows=9 p3                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=304 k=9 rows=9 p4                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=304 k=9 rows=9 p5                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=304 k=9 rows=16 p0                     group_expert_select (fusions, sort, take, sum) 
OK   route     E=304 k=9 rows=16 p1                     group_expert_select (fusions, sort, take, sum) 
OK   route     E=304 k=9 rows=16 p2                     group_expert_select (fusions, sort, take, sum) 
OK   route     E=304 k=9 rows=16 p3                     group_expert_select (fusions, sort, take, sum) 
OK   route     E=304 k=9 rows=16 p4                     group_expert_select (fusions, sort, take, sum) 
OK   route     E=304 k=9 rows=16 p5                     group_expert_select (fusions, sort, take, sum) 
OK   route     E=368 k=9 rows=1 p0                      group_expert_select (fusions, sort, take, sum)  ours    32.52 us 
OK   route     E=368 k=9 rows=1 p1                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=368 k=9 rows=1 p2                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=368 k=9 rows=1 p3                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=368 k=9 rows=1 p4                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=368 k=9 rows=1 p5                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=368 k=9 rows=2 p0                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=368 k=9 rows=2 p1                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=368 k=9 rows=2 p2                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=368 k=9 rows=2 p3                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=368 k=9 rows=2 p4                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=368 k=9 rows=2 p5                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=368 k=9 rows=9 p0                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=368 k=9 rows=9 p1                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=368 k=9 rows=9 p2                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=368 k=9 rows=9 p3                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=368 k=9 rows=9 p4                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=368 k=9 rows=9 p5                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=368 k=9 rows=16 p0                     group_expert_select (fusions, sort, take, sum) 
OK   route     E=368 k=9 rows=16 p1                     group_expert_select (fusions, sort, take, sum) 
OK   route     E=368 k=9 rows=16 p2                     group_expert_select (fusions, sort, take, sum) 
OK   route     E=368 k=9 rows=16 p3                     group_expert_select (fusions, sort, take, sum) 
OK   route     E=368 k=9 rows=16 p4                     group_expert_select (fusions, sort, take, sum) 
OK   route     E=368 k=9 rows=16 p5                     group_expert_select (fusions, sort, take, sum) 
OK   route     E=432 k=9 rows=1 p0                      group_expert_select (fusions, sort, take, sum)  ours    38.19 us 
OK   route     E=432 k=9 rows=1 p1                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=432 k=9 rows=1 p2                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=432 k=9 rows=1 p3                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=432 k=9 rows=1 p4                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=432 k=9 rows=1 p5                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=432 k=9 rows=2 p0                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=432 k=9 rows=2 p1                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=432 k=9 rows=2 p2                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=432 k=9 rows=2 p3                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=432 k=9 rows=2 p4                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=432 k=9 rows=2 p5                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=432 k=9 rows=9 p0                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=432 k=9 rows=9 p1                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=432 k=9 rows=9 p2                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=432 k=9 rows=9 p3                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=432 k=9 rows=9 p4                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=432 k=9 rows=9 p5                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=432 k=9 rows=16 p0                     group_expert_select (fusions, sort, take, sum) 
OK   route     E=432 k=9 rows=16 p1                     group_expert_select (fusions, sort, take, sum) 
OK   route     E=432 k=9 rows=16 p2                     group_expert_select (fusions, sort, take, sum) 
OK   route     E=432 k=9 rows=16 p3                     group_expert_select (fusions, sort, take, sum) 
OK   route     E=432 k=9 rows=16 p4                     group_expert_select (fusions, sort, take, sum) 
OK   route     E=432 k=9 rows=16 p5                     group_expert_select (fusions, sort, take, sum) 
OK   route     E=496 k=9 rows=1 p0                      group_expert_select (fusions, sort, take, sum)  ours    44.71 us 
OK   route     E=496 k=9 rows=1 p1                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=496 k=9 rows=1 p2                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=496 k=9 rows=1 p3                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=496 k=9 rows=1 p4                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=496 k=9 rows=1 p5                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=496 k=9 rows=2 p0                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=496 k=9 rows=2 p1                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=496 k=9 rows=2 p2                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=496 k=9 rows=2 p3                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=496 k=9 rows=2 p4                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=496 k=9 rows=2 p5                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=496 k=9 rows=9 p0                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=496 k=9 rows=9 p1                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=496 k=9 rows=9 p2                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=496 k=9 rows=9 p3                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=496 k=9 rows=9 p4                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=496 k=9 rows=9 p5                      group_expert_select (fusions, sort, take, sum) 
OK   route     E=496 k=9 rows=16 p0                     group_expert_select (fusions, sort, take, sum) 
OK   route     E=496 k=9 rows=16 p1                     group_expert_select (fusions, sort, take, sum) 
OK   route     E=496 k=9 rows=16 p2                     group_expert_select (fusions, sort, take, sum) 
OK   route     E=496 k=9 rows=16 p3                     group_expert_select (fusions, sort, take, sum) 
OK   route     E=496 k=9 rows=16 p4                     group_expert_select (fusions, sort, take, sum) 
OK   route     E=496 k=9 rows=16 p5                     group_expert_select (fusions, sort, take, sum) 
OK   combine   k=9 latent=1024 rows=1 p0                v_copy, g2_Multiply, col_reduce_small, v_copy 
OK   combine   k=9 latent=1024 rows=1 p1                v_copy, g2_Multiply, col_reduce_small, v_copy 
OK   combine   k=9 latent=1024 rows=1 p2                v_copy, g2_Multiply, col_reduce_small, v_copy 
OK   combine   k=9 latent=1024 rows=1 p3                v_copy, g2_Multiply, col_reduce_small, v_copy 
OK   combine   k=9 latent=1024 rows=2 p0                v_copy, g2_Multiply, col_reduce_small, v_copy 
OK   combine   k=9 latent=1024 rows=2 p1                v_copy, g2_Multiply, col_reduce_small, v_copy 
OK   combine   k=9 latent=1024 rows=2 p2                v_copy, g2_Multiply, col_reduce_small, v_copy 
OK   combine   k=9 latent=1024 rows=2 p3                v_copy, g2_Multiply, col_reduce_small, v_copy 
OK   combine   k=9 latent=1024 rows=9 p0                v_copy, g2_Multiply, col_reduce_small, v_copy 
OK   combine   k=9 latent=1024 rows=9 p1                v_copy, g2_Multiply, col_reduce_small, v_copy 
OK   combine   k=9 latent=1024 rows=9 p2                v_copy, g2_Multiply, col_reduce_small, v_copy 
OK   combine   k=9 latent=1024 rows=9 p3                v_copy, g2_Multiply, col_reduce_small, v_copy 
OK   combine   k=9 latent=1024 rows=16 p0               v_copy, g2_Multiply, col_reduce_small, v_copy 
OK   combine   k=9 latent=1024 rows=16 p1               v_copy, g2_Multiply, col_reduce_small, v_copy 
OK   combine   k=9 latent=1024 rows=16 p2               v_copy, g2_Multiply, col_reduce_small, v_copy 
OK   combine   k=9 latent=1024 rows=16 p3               v_copy, g2_Multiply, col_reduce_small, v_copy 
OK   combine   k=9 latent=1024 rows=64 p0               v_copy, g2_Multiply, col_reduce_small, v_copy 
OK   combine   k=9 latent=1024 rows=64 p1               v_copy, g2_Multiply, col_reduce_small, v_copy 
OK   combine   k=9 latent=1024 rows=64 p2               v_copy, g2_Multiply, col_reduce_small, v_copy 
OK   combine   k=9 latent=1024 rows=64 p3               v_copy, g2_Multiply, col_reduce_small, v_copy 

236/236 cases bit-identical

Pinned KL / fidelity: nemotron_fidelity.py --hash-weights (the model path is shortened to its folder name)

{
 "mlx": "0.32.3",
 "mlx_lm": "0.32.0",
 "pins": {
  "model": "NVIDIA-Nemotron-3.5-Bolt-74B-A9B-MLX-4bit",
  "config_sha256": "7279c754382619fd2d849835d78c3137ce27a972efb1eadb86c61585af405985",
  "index_sha256": "3b6189b91b55b18edd3be8f512d6b901686b2eb0370d1685266e91e79d01eba0",
  "shards": {
   "model-00001-of-00009.safetensors": "7e50d79de713d0ff7bc3be41ef52c8cd3c3e35cc3331f03a91b0916d30088639",
   "model-00002-of-00009.safetensors": "999d37fa98e8926af495b59250808c4dcb01e7272ade97387aeb5a6ebfd9d78a",
   "model-00003-of-00009.safetensors": "c75f5fe7261db2374b8e23cdd2cb1827140799ac13fa536cccbdb9d242be61da",
   "model-00004-of-00009.safetensors": "4da61085cc4f0505569412ee72d2c853b819c0f2f2c8ce85470149d8437b2864",
   "model-00005-of-00009.safetensors": "6f03752282c67af6b2b2e8e3c66d63a13cbd4cd2c19f8e622f40a0a7c9299caf",
   "model-00006-of-00009.safetensors": "6aafcfc9d99618addbecf93b6ef6f1adc8114941e9ca4a0c1ba3cbf54fbb4d9d",
   "model-00007-of-00009.safetensors": "87238841e0aad4279e95908268d34e3ed1b5da5de0d04ce99fb4f8c45af67c35",
   "model-00008-of-00009.safetensors": "3e94ea58a946e1d54521ef18baa6c73e61066f68588fb2b46726ede3aac24dba",
   "model-00009-of-00009.safetensors": "c19f22b58b4c42a739374585623d16addf2de352022b9d8a440ba131ba5ff720"
  },
  "engine_commit": "ce28820e47c16988f56288ad64ad464e0738184a",
  "engine_dirty": true
 },
 "fixtures": [
  {
   "fixture": "fibonacci-raw",
   "tokens": 15,
   "token_sha256": "b7769551532c0ad8b9a1b8124e3bb9e3edcf8cca5d64125c020edb80d824adbb",
   "native_nll": 2.8919690871110246,
   "mlx_serial_nll": 2.8406177388455425,
   "mlx_chunk32_nll": 2.8742868031743534,
   "mlx_full_nll": 2.8742868031743534,
   "native_vs_serial": {
    "top1_agreement": 0.9333333333333333,
    "max_abs": 1.5,
    "rms": 0.2695379667229362,
    "finite": true,
    "kl_mean": 0.003982284659141044,
    "kl_p99": 0.009924088716917365,
    "kl_max": 0.010204187780618004
   },
   "native_vs_chunk32": {
    "top1_agreement": 0.9333333333333333,
    "max_abs": 4.125,
    "rms": 0.6525186847388184,
    "finite": true,
    "kl_mean": 0.006040667485176172,
    "kl_p99": 0.01711581182153379,
    "kl_max": 0.01742250315447551
   },
   "mlx_chunk32_vs_serial": {
    "top1_agreement": 1.0,
    "max_abs": 3.125,
    "rms": 0.4436320970441915,
    "finite": true,
    "kl_mean": 0.006796612407219586,
    "kl_p99": 0.019070950104862386,
    "kl_max": 0.01995933719745038
   },
   "mlx_full_vs_serial": {
    "top1_agreement": 1.0,
    "max_abs": 3.125,
    "rms": 0.4436320970441915,
    "finite": true,
    "kl_mean": 0.006796612407219586,
    "kl_p99": 0.019070950104862386,
    "kl_max": 0.01995933719745038
   }
  },
  {
   "fixture": "gpu-chat-no-think",
   "tokens": 35,
   "token_sha256": "34c41fc50825b5fd887436ed1f9f2ae69d710777a221f94b1d5cbb5717737fe4",
   "native_nll": 5.716686995554169,
   "mlx_serial_nll": 5.724390312130104,
   "mlx_chunk32_nll": 5.765810753377128,
   "mlx_full_nll": 5.717642529948139,
   "native_vs_serial": {
    "top1_agreement": 0.9428571428571428,
    "max_abs": 1.25,
    "rms": 0.1893557441630768,
    "finite": true,
    "kl_mean": 0.010642203432166349,
    "kl_p99": 0.08333138989714994,
    "kl_max": 0.10881155142696898
   },
   "native_vs_chunk32": {
    "top1_agreement": 0.9714285714285714,
    "max_abs": 1.375,
    "rms": 0.21460021831939105,
    "finite": true,
    "kl_mean": 0.00901477917118718,
    "kl_p99": 0.04355610256285816,
    "kl_max": 0.047731224040055864
   },
   "mlx_chunk32_vs_serial": {
    "top1_agreement": 0.9714285714285714,
    "max_abs": 1.75,
    "rms": 0.19092943749106223,
    "finite": true,
    "kl_mean": 0.007755403077574087,
    "kl_p99": 0.030876112146067048,
    "kl_max": 0.03177423913459955
   },
   "mlx_full_vs_serial": {
    "top1_agreement": 0.9714285714285714,
    "max_abs": 1.75,
    "rms": 0.23128675328521206,
    "finite": true,
    "kl_mean": 0.0072931219236618315,
    "kl_p99": 0.031277037836049904,
    "kl_max": 0.03227898497237537
   }
  },
  {
   "fixture": "public-contribution-guide-512",
   "tokens": 512,
   "token_sha256": "b0206242758037798cce991293f0f600a7c5113d6b652705b43ada0bd2b192ce",
   "native_nll": 3.2981265319963677,
   "mlx_serial_nll": 3.306361377972774,
   "mlx_chunk32_nll": 3.2939761279482083,
   "mlx_full_nll": 3.3011254621412025,
   "native_vs_serial": {
    "top1_agreement": 0.91796875,
    "max_abs": 8.25,
    "rms": 0.4013358564352756,
    "finite": true,
    "kl_mean": 0.010945506139855322,
    "kl_p99": 0.04042149809126025,
    "kl_max": 0.09108781051450346
   },
   "native_vs_chunk32": {
    "top1_agreement": 0.921875,
    "max_abs": 9.4375,
    "rms": 0.39862855150561066,
    "finite": true,
    "kl_mean": 0.010753584871592985,
    "kl_p99": 0.047534459930563815,
    "kl_max": 0.1400134702895854
   },
   "mlx_chunk32_vs_serial": {
    "top1_agreement": 0.947265625,
    "max_abs": 4.25,
    "rms": 0.26151797632237317,
    "finite": true,
    "kl_mean": 0.009838347827565704,
    "kl_p99": 0.03959047257870294,
    "kl_max": 0.05834875115422212
   },
   "mlx_full_vs_serial": {
    "top1_agreement": 0.9453125,
    "max_abs": 8.8125,
    "rms": 0.4033499529019144,
    "finite": true,
    "kl_mean": 0.010674325096614975,
    "kl_p99": 0.042492825393599755,
    "kl_max": 0.06645138175069834
   }
  },
  {
   "fixture": "python-stdlib-2048",
   "tokens": 2048,
   "token_sha256": "5a42facb3a83f6c531383a19c57a666bf245328afc4138b97029ad2be1afadab",
   "native_nll": 0.36548583462730677,
   "mlx_serial_nll": 0.36374177519935724,
   "mlx_chunk32_nll": 0.36367116087373574,
   "mlx_full_nll": 0.36616538895453143,
   "native_vs_serial": {
    "top1_agreement": 0.986328125,
    "max_abs": 11.0625,
    "rms": 0.557423203819268,
    "finite": true,
    "kl_mean": 0.005274513941993687,
    "kl_p99": 0.06450120232168344,
    "kl_max": 0.875234221777027
   },
   "native_vs_chunk32": {
    "top1_agreement": 0.98974609375,
    "max_abs": 10.5,
    "rms": 0.5284347706154673,
    "finite": true,
    "kl_mean": 0.004072006140364548,
    "kl_p99": 0.0500922697477258,
    "kl_max": 0.11082624491880542
   },
   "mlx_chunk32_vs_serial": {
    "top1_agreement": 0.9873046875,
    "max_abs": 11.875,
    "rms": 0.4840688789466054,
    "finite": true,
    "kl_mean": 0.0051351756370919535,
    "kl_p99": 0.06548356509049968,
    "kl_max": 0.6026136509687315
   },
   "mlx_full_vs_serial": {
    "top1_agreement": 0.98828125,
    "max_abs": 11.1875,
    "rms": 0.5309294850846997,
    "finite": true,
    "kl_mean": 0.005368460118554758,
    "kl_p99": 0.06725235402622899,
    "kl_max": 0.5935126875951473
   }
  }
 ]
}

concurrent == solo: xpu_concurrent_receipt.py, 2 and 4 streams

round 1: 2/2 concurrent replies == solo; 384 tokens in 9.6 s = 40.2 tok/s aggregate (solo, one after another: 37.5)
round 2: 2/2 concurrent replies == solo; 384 tokens in 9.6 s = 40.0 tok/s aggregate (solo, one after another: 37.5)
round 3: 2/2 concurrent replies == solo; 384 tokens in 9.6 s = 40.1 tok/s aggregate (solo, one after another: 37.5)
CONCURRENT == SOLO: PASS
round 1: 4/4 concurrent replies == solo; 768 tokens in 19.0 s = 40.3 tok/s aggregate (solo, one after another: 38.9)
round 2: 4/4 concurrent replies == solo; 768 tokens in 19.5 s = 39.3 tok/s aggregate (solo, one after another: 38.9)
round 3: 4/4 concurrent replies == solo; 768 tokens in 20.9 s = 36.7 tok/s aggregate (solo, one after another: 38.9)
CONCURRENT == SOLO: PASS

resumed == fresh: xpu_resume_receipt.py, cached server then --prompt-cache-gib 0

# cached
{"sampling": "greedy", "turn1": {"sha": "8351ea926665", "prompt": 1508, "cached": 0, "prefill_s": 14.557032}, "turn2": {"sha": "e54437d714b4", "prompt": 1580, "cached": 1502, "prefill_s": 0.863641417}}
{"sampling": "seeded", "turn1": {"sha": "cc1f8a2a3a78", "prompt": 1508, "cached": 1502, "prefill_s": 0.086221167}, "turn2": {"sha": "235fa147bcc0", "prompt": 1606, "cached": 1502, "prefill_s": 1.138095875}}
# fresh
{"sampling": "greedy", "turn1": {"sha": "8351ea926665", "prompt": 1508, "cached": 0, "prefill_s": 15.623639125}, "turn2": {"sha": "e54437d714b4", "prompt": 1580, "cached": 0, "prefill_s": 13.90681975}}
{"sampling": "seeded", "turn1": {"sha": "cc1f8a2a3a78", "prompt": 1508, "cached": 0, "prefill_s": 12.926058917}, "turn2": {"sha": "235fa147bcc0", "prompt": 1606, "cached": 0, "prefill_s": 13.742770125}}

Lightning 30B-A3B (d9d758fb): main vs this branch, token_sha per request (32/32 equal)

# main
{
 "p0-greedy-chat": "5ae8a877bf7a",
 "p0-greedy-raw": "434cb488a3e9",
 "p0-seeded-chat": "ca84ce37feaf",
 "p0-seeded-raw": "154bb8b237f1",
 "p1-greedy-chat": "b18a343f6d01",
 "p1-greedy-raw": "2ca4bcfac5c6",
 "p1-seeded-chat": "75ad7305d004",
 "p1-seeded-raw": "994dfa915cf0",
 "p2-greedy-chat": "2f212f7460d7",
 "p2-greedy-raw": "6ce81e25bb38",
 "p2-seeded-chat": "4aac7cb43ab7",
 "p2-seeded-raw": "11ccbd49c519",
 "p3-greedy-chat": "ebd63876f10b",
 "p3-greedy-raw": "866008e96b3a",
 "p3-seeded-chat": "bb7b0f3e6107",
 "p3-seeded-raw": "ae3c33a23e79",
 "p4-greedy-chat": "99f683d98b2a",
 "p4-greedy-raw": "eb0c21a3d01c",
 "p4-seeded-chat": "80d51dfae249",
 "p4-seeded-raw": "341615eefe95",
 "p5-greedy-chat": "15ffafbac276",
 "p5-greedy-raw": "878d1d439cda",
 "p5-seeded-chat": "6a59107e2c89",
 "p5-seeded-raw": "bbf7d21d4f9e",
 "p6-greedy-chat": "689eaf56d4b3",
 "p6-greedy-raw": "b269199162ce",
 "p6-seeded-chat": "92fbbb26fe47",
 "p6-seeded-raw": "4fcf8b6c20b4",
 "p7-greedy-chat": "104a1428c2f1",
 "p7-greedy-raw": "fbf60b9de08e",
 "p7-seeded-chat": "e8ce18600195",
 "p7-seeded-raw": "3f7c0c8cec66"
}

# this branch
{
 "p0-greedy-chat": "5ae8a877bf7a",
 "p0-greedy-raw": "434cb488a3e9",
 "p0-seeded-chat": "ca84ce37feaf",
 "p0-seeded-raw": "154bb8b237f1",
 "p1-greedy-chat": "b18a343f6d01",
 "p1-greedy-raw": "2ca4bcfac5c6",
 "p1-seeded-chat": "75ad7305d004",
 "p1-seeded-raw": "994dfa915cf0",
 "p2-greedy-chat": "2f212f7460d7",
 "p2-greedy-raw": "6ce81e25bb38",
 "p2-seeded-chat": "4aac7cb43ab7",
 "p2-seeded-raw": "11ccbd49c519",
 "p3-greedy-chat": "ebd63876f10b",
 "p3-greedy-raw": "866008e96b3a",
 "p3-seeded-chat": "bb7b0f3e6107",
 "p3-seeded-raw": "ae3c33a23e79",
 "p4-greedy-chat": "99f683d98b2a",
 "p4-greedy-raw": "eb0c21a3d01c",
 "p4-seeded-chat": "80d51dfae249",
 "p4-seeded-raw": "341615eefe95",
 "p5-greedy-chat": "15ffafbac276",
 "p5-greedy-raw": "878d1d439cda",
 "p5-seeded-chat": "6a59107e2c89",
 "p5-seeded-raw": "bbf7d21d4f9e",
 "p6-greedy-chat": "689eaf56d4b3",
 "p6-greedy-raw": "b269199162ce",
 "p6-seeded-chat": "92fbbb26fe47",
 "p6-seeded-raw": "4fcf8b6c20b4",
 "p7-greedy-chat": "104a1428c2f1",
 "p7-greedy-raw": "fbf60b9de08e",
 "p7-seeded-chat": "e8ce18600195",
 "p7-seeded-raw": "3f7c0c8cec66"
}