You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Living Weights for GLM-5.3-Flash on NVIDIA GB10 (DGX Spark) CUDA — port request + test hardware offer #566
First — Living Weights looks like the most interesting serving-side work this year. The claims in the launch thread (weight updates gated behind /learn, shard growth with a size cap, persistence across unload and machine moves, cracked catastrophic forgetting) are exactly what personal-model serving needs.
We run GLM-5.3-Flash as our primary pair-served model on two NVIDIA DGX Spark (GB10, sm_121) boxes via TensorFold (stock upstream, currently the 0.6.x Python line at v0.6.6 in Docker — the operator-mandated primary lane since 2026-10-01). LW ships with Nemotron Lightning, you mentioned Qwen3.6-35B-A3B works but is gated for a few days. Questions:
GLM-5.3 on the native CUDA server: 1.0.x qualifies GLM-5.3 on two M5 Ultras (Metal). Is GLM-5.3 on GB10 CUDA on the LW/native roadmap? If it needs a port, what does the model gate look like (which measurements must pass)?
Living Weights on quantized checkpoints: our serving checkpoint is a 4bpw TR3/EXL3-class mixed-bit GLM quant. Your thread says 4-bit recall "seems really good" — does LW's shard-growth mechanism operate on quantized tensors in place, or does it need a BF16/native-format base? What happens to a mixed-bit checkpoint?
Speculative decoding interaction: our GLM lane serves with DFlash2+MTP drafts and verifies reply-equality against plain decoding. Does LW updating live weights break draft/verify equality guarantees, and is draft acceptance re-measured after learned updates?
Rollback: is learned state separable from base weights (drop the grown shard / revert to the frozen checkpoint)? Is there a per-/learn-call audit trail (what was written, where)?
Cost: what's the per-/learn update cost on CUDA GB10 for a ~30B-class MoE at 4bpw (wall time, memory), and does serving pause during an update?
Happy to give the GLM-on-GB10 native port a real test target: the pair is on my desk, we have py-spy/nccl/pytest harnesses and a round-trip verification lane already built. If a CUDA GLM branch exists behind any door, point me at it.
Thank you, @GraithSecurity, for the careful questions and for offering your GB10 pair. Here is where 1.0.4 stands.
GLM-5.3-Flash is not on the native CUDA server yet. Native CUDA serves Nemotron 3.5 Lightning only. A GLM-5.3-Flash port to the native CUDA engine is in progress, and its gate is the one every family passes: drafted equal to "draft": false, resumed equal to fresh, concurrent equal to solo, and prefill and decode measured against the current release. We will post here when a build is ready for your pair.
Living Weights learns into one model today, Nemotron 3.5 Lightning in MLX 4-bit, and only on Apple-silicon Macs. Each kept fact changes the output projection, written into the checkpoint's safetensors shards in bf16. It has not been built for EXL3 or other mixed-bit checkpoints.
A learned folder is an ordinary checkpoint to the server, so drafts are verified against the same weights they came from and the exactness contract holds on them. We have not published draft acceptance before and after learning.
There is no per-fact undo. The guide says to teach a copy and keep the original, and a reset is a fresh copy. --slide-graph FILE keeps the list of learned facts, and GET /v1/slide/graph returns it.
On a Mac a fact takes two to four minutes, and learning runs only when no reply is being generated, so the server keeps answering. NVIDIA GPUs cannot learn yet, so there is no GB10 cost to quote.
docs/living-weights.md has the details. We will keep this issue as the request for Living Weights on CUDA and on GLM.
Thank you, @GraithSecurity. GLM-5.3-Flash on native CUDA is now tracked in #593, and Living Weights on CUDA is coming through #591, for Nemotron on a GB10 first. Living Weights for GLM-5.3-Flash on CUDA is listed in #593 as a later piece, after both. Your GB10 pair offer is welcome there when a build needs a second gate. Closing in favour of #593.
Activity
ashhart commentedon Oct 10, 2026
Thank you, @GraithSecurity, for the careful questions and for offering your GB10 pair. Here is where 1.0.4 stands.
docs/living-weights.md has the details. We will keep this issue as the request for Living Weights on CUDA and on GLM.
ashhart commentedon Oct 10, 2026
Thank you, @GraithSecurity. GLM-5.3-Flash on native CUDA is now tracked in #593, and Living Weights on CUDA is coming through #591, for Nemotron on a GB10 first. Living Weights for GLM-5.3-Flash on CUDA is listed in #593 as a later piece, after both. Your GB10 pair offer is welcome there when a build needs a second gate. Closing in favour of #593.