Repository navigation
cuda: expose native MXFP8 projections with exact scale semantics - #907
Conversation
|
Thank you, @akol1, for adding MXFP8 through the shared lane matmul and covering every UE8M0 scale encoding. We checked f1f7f9a. The packer permutes the stored bytes, and We'll rebuild the fatbin with the runtime and run We'll fold small wiring fixes in at landing, then run the existing-format regressions and the family's drafted/plain, resumed/fresh and concurrent/solo token checks. Posted by Ash's AI Assistant Synthena |
b3d4713 to
a008b5c
Compare
|
The shared comparison fix from #933 is now preserved with Christian Schröder's original authorship, so non-finite normalized errors are rejected before the maximum-error reduction. Its CPU regressions call the real The current resource correction from #945 remains inherited. This update changes tests and their build registration only; CUDA arithmetic and launch declarations are unchanged. At The separate native macOS build, |
…them, 13.2's differences in its set The MXFP8 and FP32-output launches came in with the numbers their author's compiler gives. All-SM fatbins built by nvcc 13.0 on a GB10 differ in six of them on sm_120 and sm_121: the five 64-row fused-slice tiles and FP8G's 32-row cluster tile with FP32 output. registry.cuda now holds 13.0's values for those six, and 13.2's set holds the author's, which nvcc 13.2 compiles exactly on an sm_120 card. Every other new record already matched both compilers. Co-authored-by: akol1 <15747200+akol1@users.noreply.github.com>
|
Thank you, @akol1, for the losslessly packed MXFP8 path and stronger numerical fixtures. We merged your work for 1.1.0 in 5c6ba32, keeping non-finite rejection and launch registration. Our integration records the release compiler's resources and later nvcc differences in the compiler-aware catalog. Packing retains the checkpoint values and existing dispatch rules. NVIDIA GB10/RTX PRO 6000 projection checks passed; nvcc 13.0/13.2 release limits had zero stale records. Posted by Ash's AI Assistant Synthena |
…tive qlinear, with their contract and launch records
…13.0 compiles them, 13.2's differences in its set The MXFP8 and FP32-output launches came in with the numbers their author's compiler gives. All-SM fatbins built by nvcc 13.0 on a GB10 differ in six of them on sm_120 and sm_121: the five 64-row fused-slice tiles and FP8G's 32-row cluster tile with FP32 output. registry.cuda now holds 13.0's values for those six, and 13.2's set holds the author's, which nvcc 13.2 compiles exactly on an sm_120 card. Every other new record already matched both compilers. Co-authored-by: akol1 <15747200+akol1@users.noreply.github.com>
3 participants
Native qlinear can execute E4M3 projections with UE8M0 scale bytes per 32 inputs. Expose the MXFP8 kernel mode, add missing fatbin instantiations and lossless scale permutation, and route prompt/decode calls through the native lane matmul. Codes and scale bytes retain their stored precision. UE8M0 byte zero decodes to
2^-127and byte 255 to NaN, following the CUDA datatype contract. Rebuild the fatbins with the runtime.Shared numerical contracts cover non-finite inputs, repeated output and buffer guards, served/tile-edge shapes, and mirrored integer stress against independent FP64 products. Clustered slices and explicit reduction have separate products. All seven MXFP8 tile/cluster/fused launches and the BF16-output reduction helper have compiler-derived resource records.
Review follow-up:
The shared comparison fix from #933 is now preserved with Christian Schröder's original authorship, so non-finite normalized errors are rejected before the maximum-error reduction. Its CPU regressions call the real
Rig.closewith only synchronization and readback replaced, covering F32/BF16 NaNs in every position, both infinities and the[2, NaN, 0]failure. Debug and ReleaseFast mutation runs fail when the finite guard is removed.The current resource correction from #945 remains inherited. This update changes tests and their build registration only; CUDA arithmetic and launch declarations are unchanged. At
5a94b2f9f0522df354b3217da31ae5cdf9024d3fagainst mainb090e3b19014bfa1eb2b7f38ea3cc4d39abec9be, the macOS host/golden/contract suites pass 216/216 build steps with 3,016 tests passed and ten platform/library skips; golden results are 309/319 equal with zero unexpected differences. The current No GPU workflow commands, including the reader fuzz target, pass. CUDA-enabled Linux cross compilation passes 61/61 steps using explicitly fake compile-only image placeholders; those outputs were never executed or deployed. Whole-tree format and whitespace checks pass; lean has 517 inherited findings and zero new findings. Device results reported for the prior head remain explicitly bound to that earlier source; the strengthened fixture still requires a fresh device run.The separate native macOS build,
zig build native -Dcpu=apple_m1 -Dversion=1.0.0, also passes 45/45 build steps. The resulting executable was built only, not launched.Prior device evidence, explicitly historical:
At
839a5a49104d0e1ba7f1b02d5a82d0068a0a9fcf, the exact-source Linux build passed 104/104 steps with real NVCC 13.3.73 fatbins for SM 80/86/89/120/121. The complete five-architecture limits audit exited 0; actual SM 120 execution on RTX PRO 6000 Blackwell reported:Those kernels, runtime sources and launch records are byte-identical in this update, but the changed fixture has not been rerun on a device. SM 80/86/89/121 hardware remains unrun. The macOS host workflow includes its existing Metal tests; no Metal equivalent of these CUDA fixture results or full-model qualification is claimed. No full-model, collective or performance qualification is claimed. Current-head GitHub CI is green: desktop, mobile, Zig format, and No GPU host/contract/golden/fuzz/lean checks (verified 2026-10-16 21:30 UTC).