Repository navigation
Conversation
645a96a to
2500121
Compare
35e8365 to
37ac814
Compare
akol1
changed the title
|
Scoped the title and the opening claim to what this suite actually asserts: the three-way harnesses cover the shared sparse-attention and hyperconnection operators, and the Engram gather/gate stage keeps its behavioral regressions plus the projection-side receipt, with a stage-level three-way harness called out as the known gap. No code change. |
|
Scoping note: the Engram gap this PR named is closed in #1051. The only non-test change is one line exporting |
|
Thanks @akol1 for #826, alongside @jayleaton's CUDA chain. 1.1.1 includes your shared contribution. Shared host references and row/admission checks strengthen the consolidated checkpoint and Engram contracts without weakening numerical bounds. These shared pieces land with the CUDA family; the complete DeepSeek Metal backend remains separate. No release binary serves DeepSeek checkpoints yet; #827 tracks the whole-model gate. Posted by Ash's AI Assistant Synthena |
|
Mirroring the status now on #825: the 12-commit family stack is rebased onto the 1.1.1 Fresh at that head, M3 Ultra (80-core GPU, 512 GB unified memory), macOS 26.6.2, Zig 0.17.0 from The gap this PR names is unchanged: the Engram gather/gate stage still has no three-way fp64 harness of its own. Its behavioral regressions and FP32 equations are in |
|
The review for the whole stack is on #825. For this PR: please fix the Debug panic at - Synthema, Ash's AI Assistant. |
|
Bug 2 fixed on this head ( The full write-up of all three findings is on #825. Bug 3 does not affect this head — this tree never had that producer. |
|
Head updated to carry bug 1 from the stack review.
All four now pass a value: the loop at Verified on this head: This is a fast-forward, no rebase and no history change. The PR remains |
… staging Config parsing, the Engram hash layout, table plans over the original shards' embed weight and scale, and the EXL3 resident weight staging behind paced reads. Host tests cover exact integer boundaries, table selection and the staged bytes. On a 512 GiB Mac the resident payload is about 385 GiB; freeBytes counts reclaimable speculative pages so the load admission sees real room. No kernels; the served engine comes with the lane backend.
… qualification runner The Engram hash/commit, sparse and compressed attention (CSA2), the hyperconnection mix, routed and shared experts, the cache and rotary glue as source Metal kernels, with zig/tests/deepseek_v41_reference as the independent CPU oracle: plain reductions in fp64, no native statistics, no injected coefficients. tf-dsv41-run drives admission, teacher-forced logits, chunk seams, same-input operator replays and profiling on the resident checkpoint. The retained whole-model gate stays failed at 4.0961% NRMS against the unchanged 1% limit; operator checks are evidence, not qualification.
…-Flash's MIT inference
…ps to it Rank-split packs (jayleaton's 2.8bpw) duplicate whole-load tensors in native.safetensors beside the per-layer shards; the index's weight map decides each name's owner file, so the plan no longer refuses the pack as duplicates.
The released checkpoints store the gate f16 and its bias f16; rank-split repacks store the weight bf16 and the bias f32. The dense kernel's weight dtype and the route kernel's bias dtype already decode all three, so the loader accepts the range and the encoders follow the stored tensor.
The gate kernel decodes its q/k weights by dtype, the Gate descriptor carries it, and the loader accepts the rank-split packs' bf16 projections beside the released f32 ones.
Jayleaton's 2.8bpw stores the compressor and indexer matrices as a bf16 weight beside the rotation scales, and prefers it over the trellis copy it also ships. The exl3 module gains the dense form: a plain product kernel over the rotated rows writing the sums region the rotate_out stage reads, so the stage composition and its bit boundaries stay the ones the reference defines. The loader rides the pack's dtypes for the router gate, bias, Engram gate weights and the indexer's weights projection; the reference takes the union and names the dense apply it still owes. The 2.8bpw plan passes: 406.6 GB, 44 shards.
The fp64 oracle's dense path mirrors the quantized one's boundaries: the input rotation keeps its f16 rounding, the product is an FP64 dot over the f16-rounded rotated row and the bf16 weight, and the output stages are the rotate_out contract. The reference errors until now named the missing half; the union is complete.
…ument, and the duplicate-tensor case stops logging at error level d58f281f grew Plan.add an owned-index argument and left the in-file tests calling the old shape; the duplicate-tensor path also logged at error level, which the test runner treats as a failure even where the test expects the error.
…indow kernel's brace
…nuated Ported from the mhc worktree: the hyperconnection mix's per-stage fp64 harness (operator_hc_fp64.zig) and the comptime-Real reference parameterization in math.zig. Verdicts and the 1-ULP attenuation receipts live in the fork.
…se-head boundaries Carries the in-progress edits: comptime Real-style math (wide/d64/b64, rmsOf/rmsOfValues, poolOf, sparseHeadOf + KeyTable) with the FP32 instances kept as the source contract. Two repairs to that draft: model.zig re-exported kv as a duplicate member (compile error), and rmsOfValues captured the unused row index in its f32/f64 loops.
…l-faithful key addressing
…lection agreement
… in the FP64 ideal, read positions through the view offset The written-but-unrun indexer harness reported a 0.37% NRMS metal-vs-ideal delta and a vacuous selection check. Both were reference-side: * the ideal skipped the pinned source's per-head einsum BF16 output and the score's BF16 storage, so it compared an unrounded equation against a quantized contract; * the selection check sliced scratch.positions() from the buffer start, so it re-read the FP32 score words as positions, everything fell outside the key range and kernel_only counted zero by construction. With both boundaries in the ideal, the Metal kernel is bit-identical to the FP64 ideal and to the FP32 source reference on all 4096 scores of every case; the selection sets agree with valid positions (invalid_positions 0). The remaining 0.16% is the documented BF16 quantum itself, reported as bf16_score_boundary.
…output quantum sparseHeadPre captures acc/denominator before the output BF16 rounding, so the FP32-versus-FP64 distance is measurable instead of hidden by the 2^-9 output grid. Across 16 window/compressed/sink/cancellation cases the stage's intrinsic level is 2.5e-8..3.2e-7 NRMS (sub-ULP on the pre-rounding value), and the Metal kernel stays bit-identical to the FP32 source contract on every cell — 0 of 8704 BF16 cells differ from the FP64 ideal. Receipts: receipts/sparse-stage.jsonl (metal/f32_reference BF16 deltas, f32_vs_ideal_pre_rounding ULPs, scale lines).
…d RoPE Each stage boundary now has its own adversarial case family, and every case reports the kernel against BOTH the FP64 ideal and the FP32 source-contract reference, so a kernel-only delta is distinguishable from a contract delta. * sinks: one live column pins the running maximum to a known score, so the sweep -104..90 walks FP32 exp() from underflow to overflow, plus per-head sink indexing. * tile denominators: equal-score rows for the pure sum of exponentials, ramped scores that push the online correction through the FP32 underflow limit, a descending set where no correction is ever applied, and a single-tile row. * output mix: window and compressed rows on independent scale grids, and a 128-row mirror shape that cancels out of the score and the accumulator alike. * RoPE: production shape (64 rotary columns, YaRN compressed spectrum), positions 1..16383, inverse rotations and multi-row prefills, comparing the kernel's resident factor table against both references. Also: sparseCaseFrom gains split window/compressed scale exponents, and each sparse case now prints metal_vs_f32_reference on the identical decoded rows. Receipts: receipts/sinks-stage.jsonl, tile-denominators-stage.jsonl, output-mix-stage.jsonl, rope-stage.jsonl. Both qualification suites green.
The stage-by-stage verdict is that every stage is bit-identical to the FP32
source contract, which leaves the possibility that what this operator contributes
to the whole-model gate is not arithmetic but a knife-edge: a pick that sits closer
to the boundary than the score's own rounding noise.
So the indexer case now also selects from the FP64 scores the ideal kept, and reports
how many of the 512 picks change, the boundary margin and that margin in score
quanta. Random score draws tie exactly at the boundary and flip 2 of 512; the tie
case, whose margin is 13 quanta, flips none. Selection is insensitive to arithmetic
noise except where it is a genuine tie.
Also records the per-stage verdict table in RESUME.md, and the sink sweep now walks
the FP32 exp() overflow edge explicitly (80/83/84/85/86) so that the one place the
kernel and the contract disagree is a named case rather than an outlier.
Receipts regenerated from a single run: receipts/raw/final-run.log plus
{pool,indexer,sparse,tiles,mix,rope,sinks}-stage.jsonl. Both suites green.
…-plan test calls math.zig's wide and b64 arrived twice through the two worktree lines; the shard-plan tests now pass the index-ownership argument the loader gained.
The reference state now probes every scored window: the contract's boundary margin (BF16-rounded FP32 scores), the FP64-ideal margin on the same inputs, and the top-k pick flips between them — the real-data support-flip measurement. The poolOf call takes the partials by address; the oracle path had never been compiled since the runner landed, which the probe's first build exposed.
… scratch, verbatim The margins mode prefills a long prompt, then runs a single diagnostic token with the observer hooking each indexer layer, copying the selection scratch (score tiles, heap, positions) out verbatim. The offline decode reads the score tile directly: at 2600 tokens the boundary of the 512-pick selection sits inside an exact tie of the BF16-quantized scores at every writable layer — the support-flip condition the synthetic harness predicted, now on real weights. The candidate-masked layers' tiles are unwritten; their boundaries need the candidate-region decode.
…ompensated FP32 pairs The pinned source contract narrows the accumulator once per 64-column tile with the in-tile sums in wide precision; the kernel kept a running FP32 accumulator and crossed a BF16 rounding boundary on real inputs at layer 21 (one cell of 32768, head 8 dim 130), which the projections then amplified into the whole-model delta. The tile now accumulates as a hi/lo FP32 pair — the same two-sum the dot products already use — and narrows once per tile, matching the contract. The layer-21 replay is bit-exact after this; the whole-model error drops from 4.096% to 3.05%.
The reference chain sums the dense-kept dot sequentially in FP64; the kernel's plain FP32 loop crossed BF16 rounding boundaries on real rows. The 2.8bpw whole-model error drops 5.163% to 5.005% with this alone; the remaining delta is the other wide-reduction sites (the EXL3 tile shape, the RMS trees, the Hadamard) enumerated in the reduction inventory.
…ests The generated nfkd_data.zig now comes from tools/zig/nfkd_tables.py, a peer of tools/zig/unicode_tables.py: it rebuilds the table from unicodedata 16.0.0 and zig fmt, and its --check fails on drift. The table is one file with a generated header, and lean_check.py's generator-output exemption covers it, as it already covers the Flash Next generator outputs. tf-dsv41-token-map's test blocks were never reachable because the build registered the program only as an executable; a host_tests flag in the build's engine programs table makes zig build test run them, so the normalizer chain and the PCG64 draws are gated from now on. The tests' Hangul case expected a recomposed syllable; NFKD decomposes it and the release's chain never recomposes, so the expectation and a comment say so.
…-column rule Folds the remaining comment groups in the V4.1 reference, family, backend, native and Metal sources into single lines of at most 120 columns, keeping the technical content. The lean budget counts a comment group as a multi-line comment, so this keeps the tree at its pinned baseline.
operator_csa2_fp64.zig (1331 lines) and operator_hc_fp64.zig (1095) sit above the 600-line file budget. Split by what the harness measures, each part with the harness helpers it uses: csa2 into the shared view/buffers, the code tables, the sparse stages, rope and the indexer; hc into the shared view/config, the split-and-recombine bit-identity case and the amplify stages. The reference root imports every part in dependency order, so the tests.deepseek_v41_reference. filter and the suite counts are unchanged. No tolerance, assertion or receipt text moved.
Folds the last comment groups in the reference harnesses and the dense EXL3 kernel into single lines of at most 120 columns, keeping the technical content. The lean budget counts a comment group as a multi-line comment, so this keeps the tree at its pinned baseline.
beginSpan gained a metadata argument, so the four call sites in the Metal counter consumer no longer compiled. zig build test does not build that consumer, so nothing caught it. The loop passes its index, matching the profiler convention, and the three lifecycle tests pass the field default.
f5ed82f to
709574d
Compare
|
Thank you, @akol1, for the FP64 reference work and the two compensated reductions. We're carrying it as part of the Metal integration described on #825. Two things need fixing on the way: the CSA2 checks print their comparisons but never assert (
|
Piece (c)'s test side of the #690 split, rebased onto the 1.1.0 release (
76aafcf27) and stacked on #825: each shared sparse-attention and hyperconnection Metal operator gets an fp64 reference test — the harness computes the same stage three ways (the Metal kernel, the FP32 source contract, an FP64 ideal) on identical decoded rows, and asserts the kernel-vs-contract column at rounding level as a regression gate.Covered: hyperconnection mixing (every sub-stage; the mix attenuates 1-ULP noise 0.18–0.76×), sparse/compressed attention stage by stage — compressor pool + RMS, indexer score and top-k selection, sparse QK + PV, tile denominators, output mix, RoPE (both spectra, positions to 16383), sinks. Verdict: no stage above rounding level; the two named residuals (an exact-cancellation tie-break and the sink exp-overflow edge at sink−max ≈ 83–86) are unreachable for real sinks.
The one family operator outside this coverage is the Engram gather/gate stage: its behavioral regressions live in
engram_test.zig, its FP32 equations inengram_math.zig, and its WKV projection already gets an fp64 receipt fromoperators; a three-way fp64 harness for that stage itself is the gap left open here.The margin measurement is part of it: the oracle records each indexed window's selection boundary — the contract's margin, the FP64-ideal margin, the pick flips — and the margins mode captures the selection scratch verbatim at depth. Measured at 2600 tokens: the 512-pick boundary sits inside an exact BF16 tie at every indexed layer (tie-runs 2–15 keys; 71–288 distinct scores per 512 picks). A FP32-vs-FP64 score difference (~1e-7 relative) is far below the BF16 quantum (~2^-8), so support flips at tied boundaries are live — folded into the sensitivity accounting, not a kernel defect; each flip's output effect is bounded by the tied-key weight × value spread.
The three-way harnesses are now split at their seams — one file per operator family (pool/RMS, index, sparse, pack, RoPE, HC split/amplify) plus the shared math and margins helpers — so no single test file sits near the lean checker's 600-line limit.
Suites at this head:
zig build test -Dcpu=apple_m1exit 0, 304/304 steps, the reference and margins suites included (the aggregate case tally moves with the build cache, so it is not quoted);zig build native -Dcpu=apple_m1: 54/54 steps;tools/zig/lean_new.py: 502 problems against 503 on the 1.1.0 base, 0 new;zig fmt --checkclean.M3-class hardware here. tpischke offered an M5 Ultra run on the same packs and depths and we took it; those receipts will follow on the issue and are not claimed here.
#827 stacks on this branch.
Status on the 1.1.2 line. This head is unchanged and still rebased onto
76aafcf27; it is not retargeted here. The family stack refresh lives on the fork branchpr-dsv41-bc-family-11at4f69fea10(12 commits ond1018b318, the full family change preserved), and the #1052 and #1051 reference work is combined on that base in akol1#3. No action requested on this PR.