Repository navigation
Conversation
The reference modules take a comptime real width, so the FP32 source contract and the FP64 ideal are one equation set instantiated twice and cannot drift apart on their own. The new oracle-precision mode runs both on identical checkpoint bytes and tokens with the GPU out of the loop, keeps both logit vectors, and reports the NRMS between the widths, the per-token argmax, and the residency receipt of the same invocation. The window is capped at 16 IDs, one teacher step. No serving path, kernel or checkpoint format changes: this measures precision rather than gating it. Tests run: test-deepseek-v41-reference 29/29, test-deepseek-v41 79/79, test-deepseek-v41-weights 13/13, zig fmt clean. (cherry picked from commit 53b7900)
…racle words stay frozen The negative-FP64 branch fell through the NaN check's `>` test with a positive NaN's sign stripped, so a positive FP64 NaN narrowed to the finite BF16 word 0x3ffc while a negative one produced 0x7fc0. Test the FP64 class instead and return the canonical quiet NaN of the input's sign. The new tests cover every IEEE class of the narrowing (both infinities, both zeros, FP64 subnormals, the largest-finite overflow, and six NaN payload classes), the BF16 subnormal and overflow tie grid against the FP32 branch, and frozen FP32 outputs of projection, the hyperconnection stages, attention pooling and the Engram gather. The frozen words were captured on 53b7900, before the repair; the repair touches only the FP64 narrowing, so those FP32 words must not move. Gates: `zig build test-deepseek-v41-reference` 43/43 steps, 32/32 tests; `lean_check.py` 501 problems against the same 501 on the parent commit; `zig fmt --check zig` clean. (cherry picked from commit 9641836)
The split in 7665649 left the Engram stage with its behavioral Metal tests and its FP32 equations but no three-way harness. This drives the real hash, gather and gate kernels against the FP32 source contract and an FP64 widening of the same equations. The family module is private, so tensorfold.zig exports it for the reference tree; the tables, scales and family API are unchanged. (cherry picked from commit 87aeaac)
…l contract The gate test now carries the contract.Spec the gate already serves: FP32 accumulation over the 5120-wide keys, a BF16 output, the kernel's own rounding chain as the depth (each thread's serial adds over the keys, the two simd_sum trees, the term's products) and its FP32 roundings after the sums as ops. Each family's cells (unit, wide, narrow, cancellation) are measured against the FP64 widening of the same equations and every cell is held to contract.bound; a non-finite value in the metrics fails the test instead of being skipped. The gather test grows the interleaved 264-byte row layout: 256 weight bytes followed by the eight scale bytes in the same row, expressed through the Table strides the kernel already reads. Both layouts match the encoded FP32 contract on all 24,576 cells. tensorfold.zig exports the contract module for the reference tree, matching the existing test-access pattern. (cherry picked from commit 075940d)
The 1.1.2 family line keeps the resident Engram at families/deepseek_v41/metal/engram.zig, while ashhart#1051's export assumed the old line's families/deepseek_v41/engram.zig. The module body is unchanged apart from its relative hash import, so the operator test still exercises the same arithmetic.
… name Exporting core/contract/contract.zig from tensorfold.zig made that file reachable through two module roots, which Zig refuses: tf-kernel-contract and test-deepseek-v41-reference both failed to compile with "file exists in modules core and contract". The reference check now imports the contract module the way zig/tests/contract/metal_main.zig does, and the engine root keeps no contract import, as upstream main has none.
|
Found and fixed a build break in this branch: Two steps failed with the same message:
The fix follows what upstream already does.
Three files, +2/−3. Verified on M3 Ultra (80-core GPU, 512 GB unified memory), Zig 0.17.0, Xcode 27.0:
Stashing just these three files restores the Two checks I could not clear, both environment prerequisites rather than results of this change, and both present before it:
So the contract executable now builds and starts; the gates behind those two prerequisites are unrun here, not passing. |
What this is
Review artifact for carrying ashhart#1052 and ashhart#1051 together on
the 1.1.2 family line: both changes kept, both reference import sets retained.
pr-dsv41-bc-family-11@4f69fea10(=d1018b318plus 12 family commits)pr-dsv41-ef-combined-11@b800c7205Commits
08203429dthe FP32 and FP64 oracles run side by side (DeepSeek V4.1: the FP32 and FP64 oracles run side by side ashhart/TensorFold#1052)b25c4a8cethe FP64 narrowing keeps NaN, and the FP32 oracle words stay frozen (DeepSeek V4.1: the FP32 and FP64 oracles run side by side ashhart/TensorFold#1052)dd8ce6d72fp64 tests for the Engram hash, gather and gate (DeepSeek V4.1: fp64 reference tests for the Engram hash, gather and gate ashhart/TensorFold#1051)9b440f904hold the Engram gate to the registered kernel contract (DeepSeek V4.1: fp64 reference tests for the Engram hash, gather and gate ashhart/TensorFold#1051)b800c7205the Engram export names the relocated module (carry adaptation)Adaptations required by the 1.1.2 layout
deepseek_v41_engramnames the resident module atzig/src/families/deepseek_v41/metal/engram.zig. That module is byte-identical to thepre-carry
engram.zigexcept for its relative hash import (@import("../engram_hash.zig")),so the Engram operator test exercises the same code it did on the old line.
(
projection,math,attention_math,engram_math,qualify,precision) plusoperator_engram_fp64. The eightoperator_*_fp64files from the old stack's DeepSeek V4.1: fp64 reference tests for the attention and hyperconnection operators ashhart/TensorFold#826/DeepSeek V4.1: the lane backend — one stream per slot on the lane core ashhart/TensorFold#827 arenot on the 1.1.2 carry, so they are not imported; each adds one import line when carried.
docs/deepseek-v41.mdhunk from DeepSeek V4.1: the FP32 and FP64 oracles run side by side ashhart/TensorFold#1052 is dropped: that file exists neither onmainnor on this carry.
Verification
zig build test-deepseek-v41-reference --summary allzig fmt --check zigpython3 tools/zig/lean_check.pymainbaseline (0 new)math.zig); others 535, 501, 332 or lesszig build test --summary allzig build test-golden,test-kernel-contract,fuzz-cortheonzigCI job (run 39318219955), with the served-Flash-Next regressions and the lean check also greenzig-windowszig/src/core/digest_key.zig:34andzig/src/server/speech.zig:279are unchanged 1.1.2 base files, the same two errors every PR based on currentmainhits; upstream ashhart#1034 carries the fixdesktop,mobileandfmtalso pass on the PR. Full-suite note: the firstzig build testattempt reported a failure in the server'scortheon fuzz: every reader takes linear time on long inputsassertion (2 ms on 16 KiB versus65 ms on 128 KiB) while the build itself saturated every core with parallel compilation. That
test compares wall-clock scaling and is unrelated to these changes. Re-running the same
compiled test binary by itself passed all 506 tests, including the fuzz test, and the rerun of
the full plan above succeeded.
ENG64 receipts from the reference run (4 families; the assertion is the declared contract
bound, which holds in every case):
masked-chunk-and-history-carry: 480 cells, 0 Metal-vs-contract differencesfp8-row-grid: 24,576 cells, 0 differences;fp8-interleaved-264-byte-rows:24,576 cells, 0 differences
unit: 2/40,960 differences, NRMS 2.8335300764278995e-8, max |Δ| 3.814697265625e-6,worst bound ratio 0.9919194591194878
wide: 1/40,960, NRMS 1.120183e-7, max |Δ| 2.441406e-4, worst bound ratio0.9923596285716317
narrow: 2/40,960, NRMS 1.147014e-7, max |Δ| 1.192093e-7, worst bound ratio0.9911134443853961
cancellation: 0/40,960 differences, 0 bound failures0.9935598582076476
Bit-level Metal-versus-contract differences are reported per cell as the test has always done;
none exceeds the registered envelope.
Still open
GPU/model benchmark is claimed here.
#1051and#1052themselves are unchanged; this branch is the combined carry for review.zig-windowsstays red until Windows: main's host tests build again on Windows hosts ashhart/TensorFold#1034 lands; the failure is in base files and reproduces on the unchanged base branch.