You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Zig CUDA family for full GLM-5.3 (glm_moe_dsa) at TP4: plan and questions #552
We run four DGX Sparks (GB10, sm_121) on a 100G switch and would like to help bring full GLM-5.3 to the Zig engine, the port described in #402 and #310. We want to agree on the shape and the gates before writing code.
What we found in the tree (v1.0.2 plus main at 78ee0f4):
The Zig GLM family reads GLM-5.3-Flash only and runs on Metal only.
The native CUDA registry holds Nemotron only.
We found no EXL3 reader or kernel under zig/src or zig/kernels. The full-model checkpoint that fits four Sparks is EXL3 at 2.75 bits per weight.
cuda/nccl.zig has no send/recv, all-to-all or reduce-scatter, and the one-rank test is the only use we found. Would a TF_NCCL_LIB override for hosts without a system NCCL be welcome?
Proposal: a new glm_moe_dsa CUDA family at TP4 (four ranks), using the design of the Python engine in #159 as the reference: whole-tile EXL3 slices, replicated MLA latent, every expert split by width, rank-order fp32 sums. Measured numbers from the Python engine on four Sparks are the performance bar. Work would land as small independent pull requests:
An EXL3 reader and dequant kernel, with a bit-exact test.
EXL3 linear and grouped-expert kernels with rank slices.
One layer against an oracle, then layer subsets, then the full model with lanes and prefill.
Each step is gated on drafted equal to no-draft, resume equal to fresh, and concurrent equal to solo, plus an FP64 reference for every projection class whose reduction order changes.
Which multi-rank path should CUDA use: NCCL, or the RDMA Channel through cluster/?
What exactness reference is required: only the internal gates above, or also bit equality with the Python engine's results?
Is a hand-ported .cu EXL3 path acceptable, or do you prefer captured Triton kernels as AOT cubins? (The MLA, indexer, router and norms are Triton in the Python engine; the BF16 projections need a reference too.)
Is bf16 KV the baseline?
May CI use a tiny synthetic checkpoint?
We have an independent four-Spark exactness run of #159 and can share the receipts if useful.
Thank you, @BHCC2025, for asking before writing code, and for the four-Spark offer. Your six questions are answered in #593, which now tracks all GLM work on native CUDA. Full GLM-5.3 at TP=4 is step 5 there, after the Flash family's shared pieces. The multi-rank PR in step 2 would be a good first claim if you'd like it. Closing in favour of #593.
Activity
ashhart commentedon Oct 10, 2026
Thank you, @BHCC2025, for asking before writing code, and for the four-Spark offer. Your six questions are answered in #593, which now tracks all GLM work on native CUDA. Full GLM-5.3 at TP=4 is step 5 there, after the Flash family's shared pieces. The multi-rank PR in step 2 would be a good first claim if you'd like it. Closing in favour of #593.
glm: one rank's engine, its lane backend and rank 0 leading the other…