Repository navigation
Conversation
Owner
|
Thanks @Bizuayeu for the mixed-format loader, rank-share checks and stored float32 norms. This is a 1.1.2 candidate; we'll carry it onto the repaired FP4-Local foundations ourselves, keeping prompt grouping, per-sequence heads and stepped admission, and adapting the shared projections to the new weight types. Please make format classification reject empty or oversized shapes with checked arithmetic before computing widths or block counts, with failing named-refusal tests, and document the stricter legacy float32 narrowing. Please also supply raw final-source receipts and add matched bf16/NVFP4 quality, resume/grammar/media and per-rank load and peak-memory checks; TP=3 loader checks don't establish serving. Posted by Ash's AI Assistant Synthena |
Contributor
Author
I'll take these on in this pull request: the checked shape arithmetic with failing named-refusal tests, the float32 narrowing documented, matched bf16/NVFP4 quality, resume/grammar/media and per-rank load and peak-memory checks, TP=3 serving, and raw receipts from the final source. For media I'll also read FP4-Local's vision tower as stored (MXFP8 attention, W4A16_NVFP4 MLPs and merger), since the CUDA tower reads bf16 only. The description lists these under "Follow-ups taken on", and its decode table now compares FP4-Local with NVFP4 on matched inputs: drafts off, and the same token ids for the cold prompts. |
Contributor
Author
Pushed here: the checked shape arithmetic with failing named-refusal tests ( Grammar and prompt scoring for GLM on CUDA come with #1016 and #1017 in 1.1.1, so I won't add them here. Once 1.1.1 is out, I'll rebase this onto it and take FP4-Local's MXFP8 Before that, the final-source receipts and TP=3 serving go in |
Bizuayeu
force-pushed
the
pr/glm-fp4local
branch
from
October 19, 2026 17:21
9dc81b4 to
80517fc
Compare
Contributor
Author
This branch is now on v1.1.1 ( No-GPU CI's steps pass on this head (lean: no new problem against v1.1.1). With fresh caches one upstream test, On two GB10s at TP=2, on the real checkpoints:
Next come the receipts on this head (TP=2 with the full-layer gate, TP=3, prompt logprobs against a BF16 reference, and On #593's vLLM comparison: your 10-18 comment on #1015 asked for matching A4 prefill and FP8 KV first. A4 prefill can be |
Contributor
Author
The receipts from this head are in
On 1.1.1 the prompt grouping, the per-sequence heads and the stepped admission stay as they are; FP4-Local's MXFP8 One thing found on the way: the kernel contract's limits report 1,533 and 877 problems here, the same counts on plain Ready for review. If you'd like it on 1.1.2 when that is out, say so and I will rebase. The vLLM question in my last |
Bizuayeu
marked this pull request as ready for review
October 20, 2026 02:07
This was referenced Oct 20, 2026
Contributor
Author
|
The description now starts with a summary for review and covers the following; the branch's head is
|
9 tasks done
Contributor
Author
|
The description now covers four more code commits and their checks; the branch's head is
|
Closed
9 tasks done
9 tasks done
…ON recipe, and decode csf_32x2_v1 expert scales nvidia/GLM-5.3-Flash-FP4-Local stores each module in the format its MIXED_PRECISION recipe names. cuda_formats reads a projection's format from its module's recipe (core/modelopt's per-module table) checked against its tensors' dtypes and shapes, and names a mismatch. The loader checks each projection's stored format before reading it as bf16 or NVFP4 and refuses another by name. The entry admits MIXED_PRECISION in the recipes this engine reads and refuses another by its module, so FP4-Local reaches the loader's named refusal instead of the entry's. Routed experts whose block scales are csf_32x2_v1 decode on the host (core/nvfp4_csf32) into the e4m3 scales the packers take, each checked against its recipe first.
…der-gate checks every rank's uploads on the host tf-cuda-test glm-loader-gate compares every rank's uploads at TP 1, 2 and 3 with the checkpoint cut again on the host, byte for byte, and names the packed ones it does not compare yet; TF_GATE_TAMPER flips one bit of the upload it names after each download, so a run shows the gate refusing that upload by name. The MTP layer's routed experts stored as W4A16_NVFP4 (FP4-Local's) are read as stored, without an input_scale; bf16 ones are still quantized on the GPU, each checked against its recipe, and the gate compares the stored scales. A routed expert's NVFP4 read and config.json's recipes move to cuda_formats, so cuda_weights.zig stays within the lean rule's 600 lines.
…n as stored on qlinear's fp8g lane qlinear's decodeF32 and promptF32 take block FP8 (fp8g) on the lane, whose FP32 instances ashhart#908 builds, so a tensor-parallel rank's fp8g partials stay unrounded. MLA's [q_a | kv_a], q_b and o and the shared expert are read as stored, bf16 or FP8_PB_WO packed on the host for the lane, a TP share starting inside a 128 block included (the scale pack from global rank-share origins). tileLinears tiles a projection's bf16 rows for glm_mm and leaves a block-FP8 one to the lane's own tiles, which read as bf16 would be overwritten in place. FP8_PB_WO's read, sizes and pack live in cuda_fp8pb.zig. glm-synth fp8pb writes layer 1's MLA and shared expert as FP8_PB_WO under a MIXED_PRECISION config, glm-fp8g-shares checks the loader's TP shares on the lane against FP64 of the cut, and glm-loader-gate compares the packed uploads too (NVFP4 dense words and scales, routed experts, block-FP8 codes and scale tiles) against the host packers fed the gate's own cut; only the GPU quantizer's are left out, by name.
…nse layers, KDA, lm_head and the MTP layer qlinear's decodeF32 and promptF32 take MXFP8 on the lane too, so a TP rank's MXFP8 partials stay unrounded. An MXFP8 projection is read, cut and packed for the lane on the host (cuda_mxfp8.zig). A dense layer (layers 0-2) runs [gate | up] stacked into one weight that writes [rows][2 width] straight into the SwiGLU's input, and down as fp32 partials. The lm_head runs on the lane and stays out of glm_mm's tiles and TF_GLM_HEAD_BITS; the draft head quantizes its rows widened to bf16, exactly or refused by name. KDA's q, k and v run as one [q | k | v] weight and f_a, g_a and b as a bf16 rest, each into its columns of the stack's rows; o runs as stored, bf16 or MXFP8. The MTP layer's attention, indexer and shared expert run on the lane, its kv_b widened to bf16 exactly or refused by name; weights.load's MTP head lives in cuda_mtp_load.zig. glm-synth local writes layer 0's KDA, dense MLP and lm_head and the MTP layer in MXFP8 under FP4-Local's per-module recipes, with scale bytes about linear()'s RMS for their k, so its random model drafts copied rows as the bf16 one does. glm-loader-gate packs every lane upload from its own cut, block FP8 or MXFP8 as stored, and takes TF_GATE_WORLDS, TF_GATE_RANKS and TF_GATE_LAYERS, so a real checkpoint's gate runs the widths that fit a GB10. 1.1.1 runs the head on prompt rows in three new places: prompt_logprobs scores a chunk's rows (cuda_prompt_logprobs.zig), a grammar redraws the first token from the prompt's last row (cuda_structured.zig), and a batched prompt pass finishes each segment's last row (cuda_shared.zig). Each called b16.matmul on the head, which FP4-Local stores in MXFP8. They now go through cuda_proj's matmul as Forward.finish does: a bf16 head as before, with the same glm_mm prompt policy, and an MXFP8 one on qlinear's lane, whose rows' bits do not depend on the row count, so a prompt row scores as the forward's last-row head does. projectionSet carries the lane. glm_prompt_logprobs_host's head pass now counts its launches by function: an MXFP8 head runs the lane's kernel once and no bmm.
…orm_4096_f32w and hc_finish_f32w with the bf16 forms' arithmetic FP4-Local stores post_attention_layernorm (layers 0-44) and the final norm in float32, with values bf16 does not hold. The loader keeps such a hidden-size norm float32 (narrowing a float32 to bf16 only when exact, else refusing it by name), and glue.Norm carries bf16 or f32. rmsnorm_4096 and hc_finish take the weight's type as a template parameter: the float32 forms read two 16-byte loads where bf16 reads one and compute bf16(w * bf16(rinv x)) and the 64-sums in the same order, so a float32 norm adds no launch and the bf16 forms compile to the same SASS. glm-synth local writes the float32 norms, the gate compares them as stored, and glm-ref checks that a float32 weight holding bf16 values gives the bf16 weight's bytes (hc pre's four outputs, the final norm's rows and 64-sums, 1, 5 and 70 rows). Both kernels are under the kernel contract, each output against a float64 reference of the kernel's formula, hc_finish_f32w's post and comb through Sinkhorn, with their launches recorded from nvcc 13.0.
… docs say which float32 tensors stay float32 The classifier computed 2 * cols, scale columns * 16 and (n + 127) / 128 unchecked and let a zero dimension through: an empty weight read as a zero-width format, and an oversized one overflowed (a panic with safety checks, a wrap into a scale that fits without them). of() now refuses a zero dimension as EmptyShape, and a width, element count or block count that overflows as ShapeOverflow (checked arithmetic), before any of them is used; csfLinear checks the same, and the loader's refusal names the module with the reason. The two new tests fail on the previous classifier (an empty bf16 weight read as bf16, an oversized NVFP4 one panicked) and pass now. 1.1.0 rounded a float32 tensor it uploaded as bf16 to the nearest bf16. The loader now keeps float32 where the kernels read float32 and refuses, by the tensor's name, one whose values bf16 does not hold, so a checkpoint that loaded rounded now stops at load. docs/quantization.md says which tensors stay float32 and that. The NVFP4 and FP4-Local checkpoints load unchanged; the EXL3/TR3 checkpoint's float32 tensors (the hyper-connections' bases and scales, the correction bias, A_log and dt_bias: the 295,518 float32 values of the Hub's safetensors metadata) are all of the kinds kept float32.
…ections as stored, on the engine's qlinear nvidia/GLM-5.3-Flash-FP4-Local stores the tower's attention qkv and proj in MXFP8 (48) and its MLPs and merger in W4A16_NVFP4 (76), and the tower read bf16 only (UnsupportedVisionDType). It now classifies each projection its MIXED_PRECISION recipe names with formats.of, refusing another format by name, packs it as qlinear reads it (mxfp8.packCodes and packScales, nvfp4.packWords and packScales with the global scale), and runs it through the engine's qlinear.Linear: the lane for MXFP8 and the prompt GEMM for NVFP4, fp32 or bf16 out. The bf16 projections and the attention's products stay on cuBLASLt, and a checkpoint without recipes loads its bf16 tower as before. glm_vision_ref.py reads the quantized projections as the float32 values their codes and scales hold, so tf-cuda-test glm-vision checks the tower against it.
…y_receipts.py reads, and a follower frees what it owns before it leaves With TF_CUDA_MEMORY_RECEIPTS=1 (ashhart#865) each rank prints load_begin before NCCL and its weights, load_end once every rank has loaded, and cleanup_begin and cleanup_end around freeing its buffers, so the wrapper records one complete lifecycle a rank. A follower used to leave without freeing anything; after the end step it now frees its link, image rows, engine and group as rank 0's close does, so its cleanup receipts measure that. A follower that fails still leaves with _exit(1).
…nd TP=3 The loader gate on every layer, the lanes, served exactness, media, structured output and tools, prompt logprobs against a BF16 reference, memory, the kernel contract and glm-ref, the no-GPU CI steps and speed against plain v1.1.1. A receipt-only commit: the tested implementation is 80517fc.
…ith no host number or local label A cc-defer named one of the test hosts, and five comments and four tests carried a label from a local plan that the tree does not define. The comments on the MLA and shared-expert projections now name the MTP layer's MXFP8 next to bf16 and block FP8, and the tile list's says the head joins it only when it is bf16. glm_vision_ref.py's dequant has a one-line docstring. Two test comments state their rule in the present tense. docs/quantization.md: 1.1.1, not 1.1.0, is the release that rounded such float32 values; FP4-Local did not load before this branch, so the rule refuses none of its float32 tensors rather than loading them unchanged; and the MXFP8 lm_head widened for the draft head's quantizer follows the same rule as kv_b. The receipt's README now matches its logs: TP=3's resumed chats are 6,594 and 6,595 tokens, the batched pass's first tokens were 0.28 to 0.53 ms apart with prefill_seconds within 43 us, structured output ran at TP=2, TP=3's golden runs are listed, the reproduction names each serve's extra flags and openai_aligned.py, the kernel counts say why two kernels add six, and ashhart#1067's plain 1.1.1 run is in its thread (smoke3-mm1 ran this branch). SHA256SUMS follows. Comments and docs only: every .zig and .cuh change stays within its line, so no compiled line moves.
…hat compares nothing fails Three ways the gate could report the wrong thing. TF_GATE_WORLDS and TF_GATE_RANKS that select no rank (ranks 2 with worlds 1 and 2) exited 0 with no PASS or FAIL line, and a rank that compared no upload printed PASS; now such a run fails, and so does such a rank. A TF_GATE_TAMPER run exited 1 whether the flipped upload was caught or something else failed; it now passes only when every flipped copy is caught and nothing else differs, as the kernel contract's controls pass when red as planted. A checkpoint cut the gate cannot read (a missing tensor, a cut outside it) stopped every remaining rank without naming the upload; it now fails that upload by name and the run goes on, as a plan that cannot be made already did. laneMade's block-FP8 branch freed the codes before duplicating the scales, so an allocation failure there freed the codes a second time through the errdefer; the scales are now duplicated first. No normal run reaches that path.
…exts name what the loader reads The GLM family declared only modelopt-nvfp4, so `capabilities --json` and the Hugging Face cache listing named no format for the FP4-Local checkpoint it now serves; it also lists modelopt-mixed, the name FlashNext gives ModelOpt's MIXED_PRECISION checkpoints. The refusal of a checkpoint that is not ModelOpt, or not a quantization the family reads, now names NVFP4 and MIXED_PRECISION. The image tower's refusal names MXFP8 and NVFP4, since formats.of reads an NVFP4 recipe's weights as it reads W4A16_NVFP4's. The float32 narrowing's log says the tensor's kernels read bf16, where it said that only the hidden norms keep float32 (docs/quantization.md lists the others the loader keeps). Strings only: no load, kernel or serving path changes.
…on its own binaries The video serves at TP=2: with LD_LIBRARY_PATH naming the server's lib, vl-fl and vl-nv pass the checks of tools/zig/video_served.py; without it, vd-fl, vd-nv and plain v1.1.1's vd-111 are refused (ashhart#1107). In head-d23862ab/, the checks of the three commits after the receipt: the six-layer loader gate at TP=2, the gate's controls against 80517fc, FP4-Local's golden ids, capabilities and the no-GPU CI. A receipt-only commit: the tested implementation is 80517fc.
… as upstream's 15 and a repeat And it says how a video request fails without LD_LIBRARY_PATH: HTTP 400 from /tokenize and 500 from chat completions, both naming the library (ashhart#1107). A receipt-only commit: the tested implementation is 80517fc.
…r or container Their comments named this contributor's planning documents, a records folder, the containers and an image; the receipt's redaction rules now replace those as they replace machine names, and redaction-report.json counts them. A receipt-only commit: the tested implementation is 80517fc.
…e cannot run, before its upload On a GPU without thread block clusters (before sm_90) the lane sums a block-FP8 or MXFP8 projection's K slices in memory, and the GLM forward holds no slice partials, so matmul refused such a projection only at the first forward, after the whole checkpoint had loaded. The loader now asks matmul's question of each FP8 lane weight's shape, with the lane it packs for (weights.Tools.lane), and refuses the projection by name before packing or uploading it. One row stands for every row count: under qmmf.fused_rows the plan does not depend on the rows, and over it no call reduces. matmul keeps its own check. The new test failed to build before the check existed; with the check made at the fused row count instead of one row, it fails. Not run: a GPU before sm_90. On GB10 every lane weight fits, so nothing new is refused there.
…not as a config without recipes
readRecipes turned every error into null, and a null makes the loader read each module as declared NVFP4, so an
allocation failure or a read error showed up as FP4-Local's modules refused for what they store ("declared NVFP4,
stored f8_e4m3"). A failed read of config.json and an allocation failure now return as themselves. Null stays for a
config the entry refuses (not ModelOpt, a recipe this engine does not read, JSON it cannot parse) or reads as EXL3: the
entry has read and admitted config.json before any load, so after it only an allocation can fail there. No caller
reads a checkpoint without config.json: glm-synth writes one and the loader gate reads it first.
The new test failed before (a missing config.json read as null). It fails each allocation in turn, since
checkAllAllocationFailures' counts vary with the testing allocator's resizes; a readRecipes that swallows the parse's
OutOfMemory again fails it.
…the loader's formats.of meets first csfLinear refuses an empty weight (EmptyShape) and a shape whose widths overflow (ShapeOverflow), and no test held either: the loader calls csfLinear only after formats.of has refused such shapes. The test reads two empty weights from a checkpoint, no rows and no inputs, and expects EmptyShape before any width or scale count. It passes before and after this commit; without the check it fails with MissingTensor. ShapeOverflow has no test because no checkpoint reaches it: a safetensors header holds a weight only when its byte range equals its shape's product, and its dimensions are JSON integers (at most 2^63 - 1), so twice the inputs and the code count stay within a 64-bit usize. The check stays as input validation.
…rnel given that weight as bf16 values rmsnorm_4096_f32w reads FP4-Local's norm weight as float32, and gate 3's bound on its rows allows one bf16 rounding on the way (mids 1: the kernel rounds the normed row to bf16 before it multiplies by the weight), so nothing showed that the gate tells the float32 weight from a bf16 one. The control runs the same kernel with the weight rounded to bf16 on upload, which is what a kernel reading a bf16 weight multiplies by, against the float32 reference, and must go red at gate 3. The kernel contract's SUMMARY counts one more control.
…and the platform named as the hosts' model In head-797dd2e1/, the checks of the four commits after 61e9c85: the kernel contract with its new control red as planted and the limits as before, glm-ref against plain v1.1.1, the six-layer loader gate at TP=2, FP4-Local's golden ids, capabilities and the no-GPU CI. The platform line names the hosts, MSI EdgeXpert (a DGX Spark-compatible GB10). A receipt-only commit: the tested implementation is 80517fc.
1943f5b copied the hosts' whole runs.log, which had also logged four serves of another measurement (a GPU clock cap on plain v1.1.1, `cap-*`) between the receipt's checks. They are not checks of this branch, so the receipt's runs.log drops their 20 lines and again holds one line a check of its own. A receipt-only commit: the tested implementation is 80517fc.
Bizuayeu
added a commit
to Bizuayeu/TensorFold
that referenced
this pull request
Oct 22, 2026
…older; shard edits keep plain names 1.1.2's shared index check (46ce6df) takes a shard only as a plain file name, so an index that maps tensors into a folder of the model folder is refused with BadIndex. nvidia/GLM-5.3-Flash-FP4-Local (ashhart#1050) is stored that way: its index at revision 76348ddb maps 214 of its 295,041 tensors to tensors/native-QB.safetensors, the layout ashhart#1022's cuda_memory test sizes. - weightMap takes a plain name, or plain names joined by '/' with none starting with '.': each part goes through plainName (no empty part, no '.' or '..', no drive, backslash or device name), as cli/pull_file.zig's snapshotPathOk checks a hub path, so a shard stays inside the model folder and out of its hidden folders. - shard_edit's bake, grow and overwrite, through which Living Weights, --slide and LoRA merges write shards, check with plainWeightMap and keep plain names: the link checks before --slide look at a shard's own entry only (shard_edit.linked at the folder's entries, Flash Next's and Qwen3.5's at each index path's last part), so a shard in a folder, or a linked folder, would pass them unseen. - --slide refuses such a checkpoint at startup, after its link checks (slide_adapter.refusal), rather than at its first save, where shard_edit's refusal would leave the save pending until a rollback. - RADIO's image shards compare the index's names with the shard paths past the model folder, not their basenames. - Tests: the part rules in checkpoint_index.zig. In the security tests, a checkpoint with its shard in a folder opens; overwrite, grow and bake refuse such an index before writing; an overwrite through a linked folder leaves the file outside unchanged; sub/model.safetensors leaves the list of names that escape the folder. The slide adapter's tests hold the startup refusal; RADIO finds its image shard in a folder.
8 of 9 tasks
ashhart
pushed a commit
that referenced
this pull request
Oct 22, 2026
…older; shard edits keep plain names 1.1.2's shared index check (46ce6df) takes a shard only as a plain file name, so an index that maps tensors into a folder of the model folder is refused with BadIndex. nvidia/GLM-5.3-Flash-FP4-Local (#1050) is stored that way: its index at revision 76348ddb maps 214 of its 295,041 tensors to tensors/native-QB.safetensors, the layout #1022's cuda_memory test sizes. - weightMap takes a plain name, or plain names joined by '/' with none starting with '.': each part goes through plainName (no empty part, no '.' or '..', no drive, backslash or device name), as cli/pull_file.zig's snapshotPathOk checks a hub path, so a shard stays inside the model folder and out of its hidden folders. - shard_edit's bake, grow and overwrite, through which Living Weights, --slide and LoRA merges write shards, check with plainWeightMap and keep plain names: the link checks before --slide look at a shard's own entry only (shard_edit.linked at the folder's entries, Flash Next's and Qwen3.5's at each index path's last part), so a shard in a folder, or a linked folder, would pass them unseen. - --slide refuses such a checkpoint at startup, after its link checks (slide_adapter.refusal), rather than at its first save, where shard_edit's refusal would leave the save pending until a rollback. - RADIO's image shards compare the index's names with the shard paths past the model folder, not their basenames. - Tests: the part rules in checkpoint_index.zig. In the security tests, a checkpoint with its shard in a folder opens; overwrite, grow and bake refuse such an index before writing; an overwrite through a linked folder leaves the file outside unchanged; sub/model.safetensors leaves the list of names that escape the folder. The slide adapter's tests hold the startup refusal; RADIO finds its image shard in a folder.
Bizuayeu
force-pushed
the
pr/glm-fp4local
branch
from
October 22, 2026 10:02
b960da8 to
c918b6c
Compare
Contributor
Author
|
Rebased onto v1.1.2 and force-pushed: the branch's head is
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary for review
This branch is now on v1.1.2 (
4ff78441) atc918b6ca: the 21 commits of the v1.1.1 line that ended atb960da83, in the same order and with the same subjects. "On v1.1.2" below has what the rebase changed and what was retaken on it; the rest of this description anddocs/receipts/are the v1.1.1 line's and name its commits, except where they name v1.1.2.nvidia/GLM-5.3-Flash-FP4-Localloads as stored and serves on CUDA at TP=2 and TP=3, on v1.1.1. The tested implementation, called this head below, is80517fc3(eight commits), with its receipt inc595d643. Seven code commits follow the receipt. Three change comments and docs, the loader gate's own checks, andcapabilities' format names and the refusal texts (checked ond23862ab;839a32f7adds those checks to the receipt with video serves taken on this head, and two more receipt-only commits reword its video row and replace local names in its tools). Four change what the loader does or what is tested: the loader refuses by name, before its upload, an FP8 projection that a GPU without clusters could not run;readRecipesreturns a failed read or allocation as an error, not as a config without recipes; a test holdscsfLinear's refusal of an empty weight; and the kernel contract has a control for the float32 norm weight. The last of the four,797dd2e1, on its own binaries passed the kernel contract's gates 1 to 4 with that control red as planted (gate 5's limit problems as on this head and plain v1.1.1), glm-ref, the six-layer gate, FP4-Local's golden ids and the no-GPU CI but for upstream'scuda_cost_identitytest, as on this head (a timing test failed once and passed on its rerun);1943f5b0adds those logs to the receipt (head-797dd2e1/), andb960da83keeps its runs.log to its own checks. TP=3 and the rest of the receipt were not retaken on797dd2e1. On this head, the served exactness checks (golden, concurrent, resume, prompts sent together) pass on both checkpoints at TP=2 and TP=3; media, video (withLD_LIBRARY_PATH, #1107) and structured output pass on both at TP=2, and media on FP4-Local at TP=3; the loader gate and the lanes checks ran on FP4-Local at both. At TP=2, NVFP4's ids on this head equal plain v1.1.1's. Code links go to80517fc3(the gate'sruntod23862ab, after12389501), upstream's to v1.1.1 (b10231b9).On v1.1.2
c918b6caisb960da83rebased withgit rebase --onto v1.1.2 v1.1.1. Six files met 1.1.2's changes and were merged by hand:cuda_kda_block.zigandcuda_tile.zig: 1.1.2 (GLM-5.3-Flash on CUDA at TP=3: the default glm_mm path refuses KDA's input projection, and the first request fails with BadShape #1067) takes KDA's input projections off glm_mm and leaves them untiled, in row-major order, when the stack's width is not a multiple of 8 (tiledInput). Here that rule covers the bf16 stack, or the bf16[f_a | g_a | b]of FP4-Local's split stack, withf_bandg_b. FP4-Local's MXFP8[q | k | v], like its other FP8 projections, stays on the lane; neither glm_mm nor the tiler takes it. 1.1.2's test "GLM TP3: the tiler keeps unsupported KDA input and view weights in row-major order" takes this branch's types and also runs a split stack.cuda_dsa_block.zigandcuda_forward.zig: 1.1.2's profiling spans stay, now around this branch'sprojection.matmulcalls ([q_a | kv_a],q_band the indexer'skwandqb), andpretakes this branch'sglue.Norm.contract_tests.ziganddocs/kernel-contract.md: 1.1.2's registrations and text, and this branch's.cuda_kda_tp3_test.zigthe layer names itsprojandoas bf16 (commit 4,3b72fe8don v1.1.2), andcuda_kv8_test.zigzeroes its MLA layer with azeroedthat takes a projection as bf16 at address 0, sincestd.mem.zeroescannot zero the union (commit 3,96a95e5b).46ce6dfd, 1.1.2 requires every shard name inweight_mapto be a plain file name (plainName), and FP4-Local's index namestensors/native-QB.safetensorsfor 214 tensors, soc918b6caalone stops loading withBadIndex(seen on rank 1). Checkpoints: an index may name a shard in a folder inside the model folder; shard edits keep plain names #1179 reads a shard in a folder inside the model folder; it is inmainasd3707dfb, not in the 1.1.2 tag. The GPU checks below ran onc918b6cawith Checkpoints: an index may name a shard in a folder inside the model folder; shard edits keep plain names #1179's098ab795, the same patch asd3707dfb(git patch-id), on top.--parallel 4 --context 40960):glm-loader-gateon the real FP4-Local, layers 0, 3, 4, 43 and 44 and the MTP layer, both ranks: each 115 uploads as the checkpoint cut and 72 as the host packers make them, 3 left out, 0 differ, the same lines as inhead-797dd2e1/andhead-d23862ab/."draft": false, and equal to80517fc3's.--visionon FP4-Local, with noLD_LIBRARY_PATH(1.1.2 has Linux: video is refused withlibtfvideo.soin the server'slibdirectory, wherecapabilitiesfinds it #1107's fix): the receipt'stools/video_http.pypasses 16 of 16, upstream's 15 checks and a repeat; the 12 clips' prompt ids equal the processor's.s5/), both checkpoints: the twoplp-*.jsonloutputs are byte for byte80517fc3's on v1.1.1, with the sha256 in the receipt'ss5/plp-sha256.txt, so the NLL, top-1 and KL figures below hold on v1.1.2.c918b6ca:zig build test3,471 of 3,487 pass, 15 skip and 1 fails, upstream'scuda_cost_identitytest ("stable reads reuse a drafter, while a changed payload at the same path invalidates it"), as on 1.1.2;test-kernel-contract,test-golden,test-server-security test-pull-security,fuzz-cortheonandzig buildpass; the served Flash Next unittest passes 8 of 8; lean finds 438 problems, as on 1.1.2, 0 new;zig fmt --checkis clean on the 37 changed or new Zig files.capabilities --json; the lanes checks; the kernel contract and glm-ref on the GPU; media; structured output; the served concurrent and resume checks, prompts sent beside a decode and the batched prompt pass; speed and memory; each commit's own build.c918b6camerges ontomainatbd264368without conflicts (git merge-tree), and no file is changed on both sides. This pull request's CI onc918b6ca:zig,zig-windows,desktop,fmtandmobilepass.What FP4-Local stores, and where this branch reads it
FP4-Local's
config.jsonandhf_quant_config.jsonsayquant_method: modeloptandquant_algo: MIXED_PRECISION, with a recipe for each of 494 modules: MXFP8 205, FP8_PB_WO 170, W4A16_NVFP4 77 and NVFP4 42.MIXED_PRECISIONwithquantized_layerscuda_formats.zig:232, fromcuda_engine.zig:102csf_32x2_v1scales, MXFP8 or FP8_PB_WO; a mismatch, an empty shape or overflowing arithmetic is refused by namecuda_formats.zig:34,:52,:81;cuda_weights.zig:122csf_32x2_v1scalescuda_formats.zig:86,:238input_scalecuda_formats.zig:107,:39q_a,kv_a,q_bando(11 layers); the shared experts (42 layers)cuda_fp8pb.zig:36,:51;cuda_formats.zig:189q,k,vando(34 layers); the dense MLP (layers 0-2);lm_head; the MTP layer's attention, indexer and shared expert[q | k | v]and the bf16[f_a | g_a | b]; a bf16 widening only when every value is exact in bf16cuda_mxfp8.zig:46,:79;cuda_formats.zig:206;cuda_kda_block.zig:10lm_headon prompt rowsForward.finishdoescuda_proj.zig:12;cuda_prompt_logprobs.zig:80,cuda_structured.zig:46,cuda_shared.zig:36post_attention_layernorm(layers 0-44) and the final normrmsnorm_4096_f32wandhc_finish_f32w, both under the kernel contractglm_glue_norm.cuh:151,glm_glue_hc.cuh:236;contract_glm_f32w.zigqkvandproj(48); its MLPs and merger (76)cuda_vision.zig:22cuda_native.zig:153,:202tf-cuda-test glm-loader-gate(every rank's uploads against slices cut and packed again on the host);glm-synth local(a two-layer checkpoint in FP4-Local's formats)glm_loader_gate.zig:513;glm_synth.zig:22Upstream items this branch ran into
cuda_mm.zig:102; the shares,cuda_shard.zig:83n % 8; the first request returns HTTP 500 and every rank stops.TF_GLM_MM=0works around it, and also turns off the batched prompt pass, which needs tiled glm_mm (cuda_lanes.zig:196)pending_132.zig:6-10,registry.zig:353-367kernel-contractreports 1,533 limit problems andkernel-contract-limits877; no launch is over a limittfvideo.zig:30-36,optional_libraries.zig:99-105libtfvideo.soby its name alone, so with the library in the server'slib, where the refusal and RUNTIME.md's video row place it andcapabilitiesfinds it, every video request fails naming the library (/tokenize400, chat completions 500);LD_LIBRARY_PATHnaming thatlibworks around itcuda_native.zig:210-216TF_GLM_PROMPT_STEPS=0on rank 0; by default fresh prompts fill in chunks between roundsNCCL_COMM_ID's subnet; TP=3's serves here ran with a local commit that hands rank 0's NCCL id to the others in a file (tp3/bootstrap.patchin the receipt)cuda_cost_identity.zig:41,:64On
b960da83, CI'szig-windowsstopped at compile errors in code this branch does not touch, as it did then on other pull requests (#1053, for one); itszig,desktop,fmtandmobilepassed. Onc918b6caall five pass.FP4-Local against NVFP4, same inputs
On this head, GB10s (sm_121), without
--vision; TP=2 two serves each, TP=3 one. Decode: the median of five 256-token greedy runs with"draft": falseon each ofbench_openai.py's two prompts (a range spans the serves and the prompts). A cold prompt: the first cold request's time to its first token in each serve, the same token ids for both checkpoints. Memory: each rank'smemory_receipts.pyreceipt from one serve.Conditions: every GB10's GPU clock capped at 2,200 MHz (
nvidia-smi -lgc 300,2200, a local setting; the telemetry, every 2 s, read 2,197 MHz at most), each container on the efficiency cores (0-4,10-14, the RUNBOOK's pinning from #1033), 90 s of rest between serves and 180 s after long prompts, and no heat bands (the RUNBOOK'sTF_HEAT_HIGHandTF_HEAT_LOW, which #988 measured on three GB10s, were not set); the GPUs peaked at 70-81 °C in the speed serves, under the 90 °C above which a guard would have stopped a run. TP=3 ran with a local bootstrap patch (the receipt'stp3/bootstrap.patch, #636) andTF_GLM_MM=0on every rank (#1067), which takes the bf16 linears off glm_mm. NVFP4 keeps all its attention projections, the shared experts andlm_headin bf16; FP4-Local stores most of them (KDA'sq,k,vando, MLA'sq_a,kv_a,q_bando, the shared experts andlm_head) in MXFP8 or FP8 on qlinear's lanes and keeps the rest (KDA'sband gate projections, MLA'skv_band the indexer) in bf16 as NVFP4 does. The TP=3 columns hold for that setting."draft": false(tok/s)Quality at TP=2, prompt logprobs on the same 144,285 positions against BF16 (mean NLL 1.00691): FP4-Local 1.01811 (+1.11%), NVFP4 1.02069 (+1.37%); top-1 equal to BF16's 0.9282 and 0.9268. Drafted decode is not compared across the checkpoints: each drafts against its own greedy text.
What this changes
The CUDA GLM-5.3-Flash family loads
nvidia/GLM-5.3-Flash-FP4-Local(#1015, #593 (4)) as stored, with bf16 activations into every layer, and serves it, checked here at TP=2 and TP=3 on v1.1.1 and in part at TP=2 on v1.1.2 ("On v1.1.2" above). It is on v1.1.2 (4ff78441), rebased from v1.1.1 (b10231b9), which has #1021 to #1024's host foundations, and uses #907 and #908 from 1.1.0.Beyond the summary's first table:
EmptyShape) and one whose arithmetic would overflow (ShapeOverflow). A checkpoint without recipes (nvidia/GLM-5.3-Flash-NVFP4) reads as before; a read of config.json that fails after the entry has admitted it, or an allocation that fails while the loader reads the recipes, is an error, never a config without recipes, whose modules would then read as declared NVFP4. A test now holdscsfLinear's own refusal of an empty weight (EmptyShape); in the loader the classifier refuses such a weight first.csf_32x2_v1scales need no kernel change. The MTP layer's bf16 routed experts are still quantized on the GPU.[q | k | v]and bf16[f_a | g_a | b]each run into their columns of the stack's rows. A bf16 widening of an MXFP8 tensor (the MTP layer'skv_b, andlm_headfor the draft head's quantizer) loads only when every value is exact in bf16, else it is refused by name.lm_head's K split depends on the shape alone, so a row's bits do not depend on how many rows run with it._f32wforms run the bf16 forms' arithmetic on a float32 weight, so a float32 norm adds no launch. On 1.1.0's tree the bf16 forms compiled to the same SASS as before (all 40glm_gluefunctions on sm_121, nvcc 13.0), andhc_finish_f32wis checked against a float64 reference through Sinkhorn.docs/quantization.md's "GLM-5.3-Flash's float32 tensors on CUDA" names the float32 tensors the loader keeps in float32 and says that any other, which 1.1.1 rounded to bf16, loads as bf16 only when bf16 holds every value and is otherwise refused by name. The rule refuses none of the NVFP4 and FP4-Local checkpoints' float32 tensors, and the EXL3/TR3 checkpoint's float32 tensors are all of the kinds kept in float32.tools/glm_vision_ref.pyreads such projections back to the float32 values their codes and scales hold.glm-loader-gatecompares at TP 1, 2 and 3 against a separately opened checkpoint (csf decoded by the gate itself; NVFP4, MXFP8 and block-FP8 tiles from the host packers).TF_GATE_LAYERS,TF_GATE_WORLDSandTF_GATE_RANKSnarrow a real checkpoint's run, and a run that gates no rank or compares no upload fails.TF_GATE_TAMPERflips one upload's bit, and the run passes only red as planted: every gated rank reports that upload, and nothing else.modelopt-mixed(FlashNext's name for ModelOpt's MIXED_PRECISION) besidemodelopt-nvfp4, and the refusal of another quantization names NVFP4 and MIXED_PRECISION.The commits, as on the v1.1.1 line (on v1.1.2 the same 21 in the same order,
7128ed36toc918b6ca):461d4473Each projection's stored format from itsMIXED_PRECISIONrecipe, andcsf_32x2_v1expert scales decoded on the host.26894027The MTP layer'sW4A16_NVFP4experts read as stored, andglm-loader-gate.251ef9eaMLA's projections and the shared expert stored FP8_PB_WO on qlinear's fp8g lane.0eac5947MXFP8 on the lane (the dense layers, KDA,lm_headand the MTP layer), and 1.1.1's prompt-row heads with the MXFP8lm_head.3c341baeFloat32 norm weights stay float32 (rmsnorm_4096_f32wandhc_finish_f32w).4c1a57edEmpty and overflowing shapes refused, anddocs/quantization.mdnames the float32 tensors kept in float32.9e1d20f7The image tower's MXFP8 andW4A16_NVFP4projections as stored.80517fc3Every rank's load and cleanup memory receipts, and a follower frees what it owns before it leaves.c595d643The receipt,docs/receipts/glm-fp4local-80517fc3/(receipt only; the tested implementation is80517fc3).6beac759Comments and docs say what the code and the receipt hold, with no host number or local label.12389501glm-loader-gate: a tamper run passes only red as planted, and a run that compares nothing fails.d23862abCapabilities listmodelopt-mixed; the family's refusal names NVFP4 and MIXED_PRECISION, the image tower's MXFP8 and NVFP4, and the float32 log says the tensor's kernels read bf16.839a32f7The video serves (on80517fc3) andhead-d23862ab/, the checks of 10 to 12, in the receipt (receipt only).ae5b93f2The receipt's video row counts upstream's 15 checks and a repeat, and says how a video request fails withoutLD_LIBRARY_PATH(receipt only).61e9c858The receipt's tools name no local plan, records folder or container (receipt only).1f9d88e4The loader refuses by name, before its upload, an FP8 projection this GPU's lane cannot run.af81cf33readRecipesreturns a failed read or allocation as itself, not as a config without recipes.fac8a7aaA test holdscsfLinear's refusal of an empty weight, which the loader'sformats.ofmeets first.797dd2e1A gate 3 control for the float32 norm weight: the same kernel given that weight as bf16 values.1943f5b0head-797dd2e1/, the checks of 16 to 19, in the receipt, and the receipt's platform line names the hosts' model (receipt only).b960da83The receipt's runs.log keeps only its own checks, without the 20 lines of another measurement's serves that1943f5b0copied in (receipt only).Receipt
docs/receipts/glm-fp4local-80517fc3/has the raw results and its README their tables; the rebase left it unchanged. Taken on80517fc3against plain v1.1.1 built the same way, on GB10s (sm_121) at TP=2 and TP=3, containers on the efficiency cores (0-4,10-14, as #1033's runbook), checkpointsnvidia/GLM-5.3-Flash-FP4-Local@76348ddbandnvidia/GLM-5.3-Flash-NVFP4@423acf37.Loader gate on every layer, the real FP4-Local: TP=2 ranks 0 and 1 and TP=3 ranks 0, 1 and 2, each 894 uploads as the checkpoint cut and 486 as the host packers make them, 0 differ (TP=3's FP8 shares start inside a 128 block: cuda/fp8: pack a rank's share of block-FP8 scales from inside a 128 block #1024's repacking, per rank).
Lanes on the real FP4-Local, TP=2 and TP=3:
glm-lanes-cover40/40,glm-lanes-concurrent38/38,glm-shared-check108 rows of 5 streams over 12 rounds equal to their lone twins byte for byte.Served exactness, both checkpoints, TP=2 and TP=3: three decode-check prompts drafted =
"draft": false;bench_concurrent.py --alone --serial12 cells, 0 unequal; a chat of 6,593 to 6,595 tokens resumed from its kept state, with"draft": falseand two at once, equals the cold one; four fresh prompts sent beside a decode each equal their alone run. At TP=2, NVFP4's ids on this head equal plain v1.1.1's on all three prompts.1.1.1's batched prompt pass, which GLM takes when rank 0 has
TF_GLM_PROMPT_STEPS=0(by default prompts fill in chunks between rounds): three fresh prompts shared one pass (first tokens within 0.53 ms of each other,prefill_secondswithin 43 µs), each equal to its alone run, on plain v1.1.1 and this head with NVFP4 and on this head with FP4-Local, whose MXFP8 head ran three segments incuda_shared.heads.Media with
--vision(FP4-Local and NVFP4 at TP=2, FP4-Local at TP=3): drafted ="draft": false, repeated, three at once = alone; a chat resumed past its image equals the cold one.Video with
--vision, both checkpoints at TP=2 (tp2/serve/vl-*): withLD_LIBRARY_PATHnaming the server'slib,tools/video_http.pypasses 16 of 16, upstream's 15tools/zig/video_served.pychecks (the 12 clips' prompt ids equalGlm5NextProcessor's, read from a filevideo_ref.pycomputed; a later turn resumes past its video, and another video under the same text resumes only the text before it; a video request answers; drafted ="draft": false; four at once = alone; a video beside an image; the eight refusals) and a repeat. Without it every video request fails naming the library (/tokenize400, chat completions 500), on plain v1.1.1 too (vd-*, Linux: video is refused withlibtfvideo.soin the server'slibdirectory, wherecapabilitiesfinds it #1107).Structured output and tools, GLM-5.3-Flash on CUDA: structured output (first_masked and held) #1017's and Required tool calls in GLM's markup: xgrammar's glm_xml style #1018's items, on both checkpoints at TP=2: 26/26 with drafts (schemas with thinking off and on, a schema plain already meets,
"draft": false, seeded sampling twice, nested, resume from cache, required, named, parallel, single, nested and empty tool calls with and without drafts, mixed streams); 22/22 on a--no-draftsserve, whose 22 shared greedy replies equal the drafted serve's, and a resumed reply equals a cold one. The grammar library islib/libtfgrammar.sofrom the v1.1.1 release archive.Prompt logprobs against BF16 on the same tokens (128 held-out sequences, 144,413 tokens scored at 144,285 positions; the BF16
lm_headon the BF16 model's final hidden): mean NLL BF16 1.00691, FP4-Local 1.01811 (+1.11%), NVFP4 1.02069 (+1.37%); top-1 equal to BF16's 0.9282 and 0.9268; KL over BF16's top 20 (an approximation) 0.0528 and 0.0537.Memory,
memory_receipts.pyon every rank: FP4-Local at TP=2 90.27 GiB at load end, 92.96 GiB peak, 0.00 after cleanup on each rank; at TP=3 63.01, 62.88 and 57.74 GiB at load end. Rank 2 killed at TP=3: ranks 0 and 1 left 7 s later.Speed, two serves each (decode medians; a cold prompt is the first cold request's time to its first token), alternating with plain v1.1.1 on the same GB10s at TP=2:
kernel contract (CUDA, GB10, nvcc 13.0.88):
SUMMARY 91 kernels: 91 pass gates 1-4, 0 fail, 0 known findings; 7 controls: 7 red as planted, 0 missed; 1533 limit problems. kernel-contract-limits on sm_80, 86, 89, 120 and 121: 877 problems. Plain v1.1.1 gives the same 1533 and 877 (85 kernels: the contract counts each output, and this branch's two kernels add six); both come from the 656 pending launches that have no 13.0 records (CUDA kernel contract on nvcc 13.0: 656 pending launches have no 13.0 records, sokernel-contractandkernel-contract-limitsfail on plain v1.1.1 #1105). This branch's two kernels,tf_glm_rmsnorm_4096_f32wandtf_glm_hc_finish_f32w, pass on all five (record current), and every gate.glm-ref: 12 pass, every RESULT line equal to plain v1.1.1's.
No-GPU CI's steps on an LF tree:
zig build test211/211 steps,test-kernel-contract65/65,test-golden,fuzz-cortheon11/11 and the served Flash Next unittest pass; lean 0 new problems (444 on v1.1.1);zig fmt --checkon the 35 changed Zig files andgit diff --checkclean. With fresh caches, 2981/2996 pass and one upstream test fails,cuda_cost_identity.zig:64(a same-size rewrite's mtime under WSL2's coarse timestamps); plain v1.1.1 fails it the same way. Each of the eight commits builds, and its tests pass but for that test andbenchmark_receipt_test's four, which fail only in agit archivecopy, as on plain v1.1.1's.This pull request's CI on
b960da83(and before on1943f5b0,61e9c858,c595d643and80517fc3):zig,desktop,fmtandmobilepassed.zig-windowsstopped at compile errors in code this branch does not touch (std's dynamic library,core/digest_key.zig,server/speech.zig, flashnext'sws2_32use), the same 52 locations on all five, as GLM CUDA: the waiting ranks sleep while rank 0 has no stream (#1033) #1053's run of the same job did, and RUNBOOK: a GPU clock cap for GB10s, with what it cost and lowered for GLM-5.3-Flash NVFP4 at TP=2 #1110's, which changes onlyRUNBOOK.md. Onc918b6ca, on v1.1.2, all five pass ("On v1.1.2").d23862ab, the last of commits 10 to 12 (head-d23862ab/): the six-layer gate on both TP=2 ranks, each 115 uploads as the checkpoint cut and 72 as the host packers make them, 0 differ;TF_GATE_TAMPER=head.codeson TP=2 rank 0 passes red as planted where80517fc3exits 1, and a run that selects no rank fails where80517fc3exits 0 (gate-controls/); FP4-Local's golden ids at TP=2 equal80517fc3's on all three prompts;capabilities --jsonlistsmodelopt-mixed; the no-GPU CI steps give80517fc3's counts with fresh caches (ci/).797dd2e1, the last of commits 16 to 19 (head-797dd2e1/): kernel contractSUMMARY 91 kernels: 91 pass gates 1-4, 0 fail, 0 known findings; 8 controls: 8 red as planted, 0 missed; 1533 limit problems, the new control red at gate 3 as planted; its limits as above; glm-ref 12 pass, every RESULT line equal to plain v1.1.1's; the six-layer gate on both TP=2 ranks, each 115 uploads as the checkpoint cut and 72 as the host packers make them, 0 differ, as ond23862ab; FP4-Local's golden ids at TP=2 equal80517fc3's on all three prompts;capabilities --jsonas ond23862ab; the no-GPU CI steps with fresh caches: 2983/2999 pass, upstream'scuda_cost_identity.zig:64fails as before, andcortheon's linear-time fuzz test, which this branch does not touch, failed once (toTexttook 2 ms on 16 KiB and 65 ms on 128 KiB) and passed on its rerun;test-kernel-contract65/65,test-golden,fuzz-cortheon11/11 and the served Flash Next unittest pass; lean 0 new;zig fmt --checkclean (ci/).At TP=3 the gate and the lanes ran this branch's own binaries; the serves added the bootstrap patch and
TF_GLM_MM=0, as the summary's conditions say.Earlier, on 1.1.0's tree at
166b01c5:tf-cuda-test glm-visionwithassets/tensorfold-hero.png(2,028 image tokens) againsttools/glm_vision_ref.py's float32 reference: FP4-Local's tower read as stored differs by 0.2271 of the features' rms, 0.920 times the torch tower's bf16 run (0.2469); the NVFP4 checkpoint's bf16 tower differs by 0.2364, 0.929 times its bf16 run (0.2544).Not run:
fp8_ds_mlaornvfp4_ds_mla(--kv-cache-dtype automaps tofp8_ds_mla), while TensorFold's GLM keeps it in BF16, so the KV cache cannot be matched; this pull request claims no engine comparison.1f9d88e4's refusal applies (every GPU here is a GB10).Follow-ups taken on in this pull request
From the review, all done:
4c1a57ed).4c1a57ed).9e1d20f7).lm_headon its prompt rows (0eac5947); now on 1.1.2 ("On v1.1.2").docs/receipts/: resume, media served with images, per-rank load and peak memory, TP=3 serving on three GB10s, the grammar checks GLM-5.3-Flash on CUDA: structured output (first_masked and held) #1017 and Required tool calls in GLM's markup: xgrammar's glm_xml style #1018 list, and prompt logprobs against a BF16 reference.Checklist
"draft": false, a resumed prompt equals a fresh one, and concurrent equals solo. (On v1.1.1, on the real FP4-Local at TP=2 and TP=3: the three lanes checks pass, the served checks have 0 unequal, and prompts sent together, batched (at TP=2) or not, equal their alone runs. On v1.1.2 at TP=2, FP4-Local's golden prompts drafted equal"draft": false, and its video checks, a resumed video turn and four at once equal to alone among them, pass.)80517fc3's.)SUMMARYline is in the receipt. (rmsnorm_4096_f32wandhc_finish_f32w, each output against a float64 reference, and a gate 3 control that givesrmsnorm_4096_f32wits weight as bf16 values; CUDA'sSUMMARYlines above for80517fc3and797dd2e1, each with the 1533 limit problems plain v1.1.1 also gives; no Metal kernel changes. On v1.1.2 the registrations sit beside 1.1.2's andtest-kernel-contractpasses; the GPU run was not retaken.)90695a85on 1.1.0 and passed after it;4c1a57edis the same change line for line. The MXFP8-head prompt-row test cannot build against 1.1.1'sb16.matmulhead. Before this branch,admitrefuses FP4-Local's config as not NVFP4, so the gate and the lanes checks cannot load it;TF_GATE_TAMPERpasses only red as planted (head-d23862ab/gate-controls/);1f9d88e4's test does not build without its check and fails when the check asks at the fused row count;af81cf33's fails before it (a missing config.json read as no recipes) and when the parse'sOutOfMemoryis swallowed again;fac8a7aa's passes before and after (it holds an existing refusal) and fails without that refusal;797dd2e1's control goes red as planted; the split-stack case this branch adds to 1.1.2's tiler test cannot build on 1.1.2, which has no split stack; the host tests need onlystdand the repo's fixtures, and the GPU checks aretf-cuda-testsubcommands.)redaction-report.jsoncounts them.)