RC2: deterministic greedy output at any concurrency, faster MoE decode

[data-angel]

RC2: greedy output that doesn't depend on concurrency or the drafter, plus faster MoE decode.

Engine

Enntity/atlas sparkglm/atlas-20261009-rc2 @ f2b805e7 = sparkglm/atlas-20261009-final (b85befc6) plus three changes:

• Canonical verify (ATLAS_GLM_CANONICAL_VERIFY, on by default). Every op a verify row touches now uses one row-invariant kernel family, whatever the verify width (1–32 rows):

  • W4A16 projections (the tc8 sweep), the dense BF16 GEMVs, the router, the shared expert and the dense FFN;
  • plain decode, which now runs as a one-row verify.

  Before this, the DFlash verify width selected different kernels. Which width ran depended on the drafter and on what else was running, so the same request could get different greedy output at C1 and at C4.

• Persistent MoE decode kernels (ATLAS_GLM_MOE_DECODE_PERSIST, on in the profiles): one CTA per SM, prefetching the next expert tile. Output bits are identical to the current kernels.

• KDA record commits: one launch per sequence instead of one per layer. Output bits are identical.

Measured on the pair (production profile)

• Determinism: same request, greedy, four prompts. C4 output equals C1 output, and a C4 rerun matches; it also matches with the draft width pinned. 4/4 on each check. The current release managed 0/4.

• Prompt-logprob hash: 246cd1331b41, the same with the KV shard on and off.

• Disk prefix cache beside the shard: turns 2–4 come back byte-identical to the resident run after three restores, turn-4 logprobs included.

• Speed at C1 (12 prose + 12 code prompts, two alternating rounds per engine):

  	RC2	current release	change
  Prose	42.92 tok/s	41.17 tok/s	+4.3%
  Code	61.74 tok/s	61.84 tok/s	flat

• Step time (nsys): prose 59.9 vs 61.3 ms, code 82.7 vs 85.3 ms. C2/C4 are flat to slightly faster.

• Quality: 491/500 arithmetic and 24/24 two-hop 24K needles. Every engine configuration we ran scored 484–495; that spread comes from borderline items flipping between configurations, so there is no measurable difference.

• Four concurrent 204K sessions: see the fill check below.

Receipts: sparkglm-research experiments/2026-10-09-canonical-verify.

🤖 Generated with Claude Code