Repository navigation
Conversation
|
Thanks @akol1, the hash comparison and the explicit subnormal handling are useful. We'll take this with #826 on the 1.1.2 line once it's corrected; DeepSeek serving still waits for #827. Three asks:
The gate totals 163,840 cells across its four families but prints 40,960, so please report each family separately. Could you attach the full ENG64 output, the five differing gate cells and your chip and macOS version? The module export fits the existing test-access pattern. Posted by Ash's AI Assistant Synthena |
0a5f51f to
075940d
Compare
|
Re-cut at 1 — the interleaved 264-byte gather case. The gather stage now runs against both layouts: the default planar rows (a 256-byte weight row, an 8-byte scale row) and one interleaved 264-byte row — 256 weight bytes at +0, the 8 scale bytes at +256 — expressed through the 2 — every gate cell is held to the contract bound. The gate declares the 3 — non-finite values fail. Full ENG64 output, all lines this head prints: The five cells where the Metal gate differs from the FP32 contract (each within its Environment: Apple M3 Ultra, 80-core GPU, 512 GB unified memory; macOS 26.6.2 (25G83); Zig 0.17.0. Two notes on the numbers:
Both exports are the ones the review called fine: CI on this head: Update (CI, 2026-10-19T03:20Z): reruns need maintainer rights, so the checks were refreshed with a close/reopen ( |
The 1.1.2 family line keeps the resident Engram at families/deepseek_v41/metal/engram.zig, while ashhart#1051's export assumed the old line's families/deepseek_v41/engram.zig. The module body is unchanged apart from its relative hash import, so the operator test still exercises the same arithmetic.
|
Combined carry with #1052 is up as a review artifact: akol1#3, based on It keeps the Engram operator test (hash, gather, gate) and the gate's contract wiring, with both reference import sets retained as you noted. One layout adaptation: the Engram export names the module where the 1.1.2 carry keeps it, Measured on the carry: 43/43 reference steps and 23/23 tests, 0 bound failures across the four gate families, |
|
The review for the whole stack is on #825. If the precision lane is the one to keep, this is its head. The two Engram checks are useful coverage: the gate is a bound pass with five BF16 words different from the contract, and both gather layouts are bit-equal. It inherits bugs 1 and 2 from there. - Synthema, Ash's AI Assistant. |
|
Bug 2 fixed on this head ( This head does not carry bug 3's producer, so no frozen baseline needed regenerating here. On the five BF16 words: I agree it is a bound pass rather than bit identity, and I would rather keep that visible than claim identity we don't have. The Engram receipt already reports Full write-up of all three findings is on #825. |
|
Head is now Bug 1 from the stack review is fixed here. The other commit is the sparse-attention contract registration ( This is a fast-forward on |
… staging Config parsing, the Engram hash layout, table plans over the original shards' embed weight and scale, and the EXL3 resident weight staging behind paced reads. Host tests cover exact integer boundaries, table selection and the staged bytes. On a 512 GiB Mac the resident payload is about 385 GiB; freeBytes counts reclaimable speculative pages so the load admission sees real room. No kernels; the served engine comes with the lane backend.
… qualification runner The Engram hash/commit, sparse and compressed attention (CSA2), the hyperconnection mix, routed and shared experts, the cache and rotary glue as source Metal kernels, with zig/tests/deepseek_v41_reference as the independent CPU oracle: plain reductions in fp64, no native statistics, no injected coefficients. tf-dsv41-run drives admission, teacher-forced logits, chunk seams, same-input operator replays and profiling on the resident checkpoint. The retained whole-model gate stays failed at 4.0961% NRMS against the unchanged 1% limit; operator checks are evidence, not qualification.
…-Flash's MIT inference
The released checkpoints store the gate f16 and its bias f16; rank-split repacks store the weight bf16 and the bias f32. The dense kernel's weight dtype and the route kernel's bias dtype already decode all three, so the loader accepts the range and the encoders follow the stored tensor.
The gate kernel decodes its q/k weights by dtype, the Gate descriptor carries it, and the loader accepts the rank-split packs' bf16 projections beside the released f32 ones.
Jayleaton's 2.8bpw stores the compressor and indexer matrices as a bf16 weight beside the rotation scales, and prefers it over the trellis copy it also ships. The exl3 module gains the dense form: a plain product kernel over the rotated rows writing the sums region the rotate_out stage reads, so the stage composition and its bit boundaries stay the ones the reference defines. The loader rides the pack's dtypes for the router gate, bias, Engram gate weights and the indexer's weights projection; the reference takes the union and names the dense apply it still owes. The 2.8bpw plan passes: 406.6 GB, 44 shards.
The fp64 oracle's dense path mirrors the quantized one's boundaries: the input rotation keeps its f16 rounding, the product is an FP64 dot over the f16-rounded rotated row and the bf16 weight, and the output stages are the rotate_out contract. The reference errors until now named the missing half; the union is complete.
…indow kernel's brace
…nuated Ported from the mhc worktree: the hyperconnection mix's per-stage fp64 harness (operator_hc_fp64.zig) and the comptime-Real reference parameterization in math.zig. Verdicts and the 1-ULP attenuation receipts live in the fork.
…se-head boundaries Carries the in-progress edits: comptime Real-style math (wide/d64/b64, rmsOf/rmsOfValues, poolOf, sparseHeadOf + KeyTable) with the FP32 instances kept as the source contract. Two repairs to that draft: model.zig re-exported kv as a duplicate member (compile error), and rmsOfValues captured the unused row index in its f32/f64 loops.
…l-faithful key addressing
…lection agreement
… in the FP64 ideal, read positions through the view offset The written-but-unrun indexer harness reported a 0.37% NRMS metal-vs-ideal delta and a vacuous selection check. Both were reference-side: * the ideal skipped the pinned source's per-head einsum BF16 output and the score's BF16 storage, so it compared an unrounded equation against a quantized contract; * the selection check sliced scratch.positions() from the buffer start, so it re-read the FP32 score words as positions, everything fell outside the key range and kernel_only counted zero by construction. With both boundaries in the ideal, the Metal kernel is bit-identical to the FP64 ideal and to the FP32 source reference on all 4096 scores of every case; the selection sets agree with valid positions (invalid_positions 0). The remaining 0.16% is the documented BF16 quantum itself, reported as bf16_score_boundary.
…output quantum sparseHeadPre captures acc/denominator before the output BF16 rounding, so the FP32-versus-FP64 distance is measurable instead of hidden by the 2^-9 output grid. Across 16 window/compressed/sink/cancellation cases the stage's intrinsic level is 2.5e-8..3.2e-7 NRMS (sub-ULP on the pre-rounding value), and the Metal kernel stays bit-identical to the FP32 source contract on every cell — 0 of 8704 BF16 cells differ from the FP64 ideal. Receipts: receipts/sparse-stage.jsonl (metal/f32_reference BF16 deltas, f32_vs_ideal_pre_rounding ULPs, scale lines).
…d RoPE Each stage boundary now has its own adversarial case family, and every case reports the kernel against BOTH the FP64 ideal and the FP32 source-contract reference, so a kernel-only delta is distinguishable from a contract delta. * sinks: one live column pins the running maximum to a known score, so the sweep -104..90 walks FP32 exp() from underflow to overflow, plus per-head sink indexing. * tile denominators: equal-score rows for the pure sum of exponentials, ramped scores that push the online correction through the FP32 underflow limit, a descending set where no correction is ever applied, and a single-tile row. * output mix: window and compressed rows on independent scale grids, and a 128-row mirror shape that cancels out of the score and the accumulator alike. * RoPE: production shape (64 rotary columns, YaRN compressed spectrum), positions 1..16383, inverse rotations and multi-row prefills, comparing the kernel's resident factor table against both references. Also: sparseCaseFrom gains split window/compressed scale exponents, and each sparse case now prints metal_vs_f32_reference on the identical decoded rows. Receipts: receipts/sinks-stage.jsonl, tile-denominators-stage.jsonl, output-mix-stage.jsonl, rope-stage.jsonl. Both qualification suites green.
The stage-by-stage verdict is that every stage is bit-identical to the FP32
source contract, which leaves the possibility that what this operator contributes
to the whole-model gate is not arithmetic but a knife-edge: a pick that sits closer
to the boundary than the score's own rounding noise.
So the indexer case now also selects from the FP64 scores the ideal kept, and reports
how many of the 512 picks change, the boundary margin and that margin in score
quanta. Random score draws tie exactly at the boundary and flip 2 of 512; the tie
case, whose margin is 13 quanta, flips none. Selection is insensitive to arithmetic
noise except where it is a genuine tie.
Also records the per-stage verdict table in RESUME.md, and the sink sweep now walks
the FP32 exp() overflow edge explicitly (80/83/84/85/86) so that the one place the
kernel and the contract disagree is a named case rather than an outlier.
Receipts regenerated from a single run: receipts/raw/final-run.log plus
{pool,indexer,sparse,tiles,mix,rope,sinks}-stage.jsonl. Both suites green.
…-plan test calls math.zig's wide and b64 arrived twice through the two worktree lines; the shard-plan tests now pass the index-ownership argument the loader gained.
The reference state now probes every scored window: the contract's boundary margin (BF16-rounded FP32 scores), the FP64-ideal margin on the same inputs, and the top-k pick flips between them — the real-data support-flip measurement. The poolOf call takes the partials by address; the oracle path had never been compiled since the runner landed, which the probe's first build exposed.
… scratch, verbatim The margins mode prefills a long prompt, then runs a single diagnostic token with the observer hooking each indexer layer, copying the selection scratch (score tiles, heap, positions) out verbatim. The offline decode reads the score tile directly: at 2600 tokens the boundary of the 512-pick selection sits inside an exact tie of the BF16-quantized scores at every writable layer — the support-flip condition the synthetic harness predicted, now on real weights. The candidate-masked layers' tiles are unwritten; their boundaries need the candidate-region decode.
…ompensated FP32 pairs The pinned source contract narrows the accumulator once per 64-column tile with the in-tile sums in wide precision; the kernel kept a running FP32 accumulator and crossed a BF16 rounding boundary on real inputs at layer 21 (one cell of 32768, head 8 dim 130), which the projections then amplified into the whole-model delta. The tile now accumulates as a hi/lo FP32 pair — the same two-sum the dot products already use — and narrows once per tile, matching the contract. The layer-21 replay is bit-exact after this; the whole-model error drops from 4.096% to 3.05%.
The reference chain sums the dense-kept dot sequentially in FP64; the kernel's plain FP32 loop crossed BF16 rounding boundaries on real rows. The 2.8bpw whole-model error drops 5.163% to 5.005% with this alone; the remaining delta is the other wide-reduction sites (the EXL3 tile shape, the RMS trees, the Hadamard) enumerated in the reduction inventory.
…ests The generated nfkd_data.zig now comes from tools/zig/nfkd_tables.py, a peer of tools/zig/unicode_tables.py: it rebuilds the table from unicodedata 16.0.0 and zig fmt, and its --check fails on drift. The table is one file with a generated header, and lean_check.py's generator-output exemption covers it, as it already covers the Flash Next generator outputs. tf-dsv41-token-map's test blocks were never reachable because the build registered the program only as an executable; a host_tests flag in the build's engine programs table makes zig build test run them, so the normalizer chain and the PCG64 draws are gated from now on. The tests' Hangul case expected a recomposed syllable; NFKD decomposes it and the release's chain never recomposes, so the expectation and a comment say so.
…-column rule Folds the remaining comment groups in the V4.1 reference, family, backend, native and Metal sources into single lines of at most 120 columns, keeping the technical content. The lean budget counts a comment group as a multi-line comment, so this keeps the tree at its pinned baseline.
operator_csa2_fp64.zig (1331 lines) and operator_hc_fp64.zig (1095) sit above the 600-line file budget. Split by what the harness measures, each part with the harness helpers it uses: csa2 into the shared view/buffers, the code tables, the sparse stages, rope and the indexer; hc into the shared view/config, the split-and-recombine bit-identity case and the amplify stages. The reference root imports every part in dependency order, so the tests.deepseek_v41_reference. filter and the suite counts are unchanged. No tolerance, assertion or receipt text moved.
Folds the last comment groups in the reference harnesses and the dense EXL3 kernel into single lines of at most 120 columns, keeping the technical content. The lean budget counts a comment group as a multi-line comment, so this keeps the tree at its pinned baseline.
The split in 7665649 left the Engram stage with its behavioral Metal tests and its FP32 equations but no three-way harness. This drives the real hash, gather and gate kernels against the FP32 source contract and an FP64 widening of the same equations. The family module is private, so tensorfold.zig exports it for the reference tree; the tables, scales and family API are unchanged.
…l contract The gate test now carries the contract.Spec the gate already serves: FP32 accumulation over the 5120-wide keys, a BF16 output, the kernel's own rounding chain as the depth (each thread's serial adds over the keys, the two simd_sum trees, the term's products) and its FP32 roundings after the sums as ops. Each family's cells (unit, wide, narrow, cancellation) are measured against the FP64 widening of the same equations and every cell is held to contract.bound; a non-finite value in the metrics fails the test instead of being skipped. The gather test grows the interleaved 264-byte row layout: 256 weight bytes followed by the eight scale bytes in the same row, expressed through the Table strides the kernel already reads. Both layouts match the encoded FP32 contract on all 24,576 cells. tensorfold.zig exports the contract module for the reference tree, matching the existing test-access pattern.
…el contract dsv41_sparse_attention is compiled and launched on the served path but had no contract registration, so every gate on it was unchecked. This adds its probe, its launch record, and the source resolution gate 5 needs. Gates 1-5 pass; the receipt's SUMMARY line is in the pull request. The probe covers one chunk of rows with no ring and no compressed keys, so every key a row sees is a row of the chunk buffer. Two properties of the real call are modeled rather than assumed: - the key buffer is the packed FP4 cache, not linear bf16: a value is an e4 code and each 32 values share a power-of-two block scale, decoded to bf16. A NaN cannot be stored in it, so gate 1 plants non-finite inputs in the query, which is plain bf16, and the stated rule says what then happens. - the denominator sums unrounded probabilities while only the product rounds. Gate 4's stress inputs make the width-long dot cancel, where a narrow accumulator loses the result: fp32 is exact, fp16 misses by inf, bf16 by 1x. The M1 and M2 thread limits (768 both) come from --offline. The contract also stops travelling through the tensorfold facade here: it is its own module everywhere else on main, and re-exporting it made the module reachable two ways, which Zig rejects as "file exists in modules 'core' and 'contract'". The engram reference now imports it directly, as the other probes do.
Recorder.beginSpan gained a metadata argument, but metal_counter_consumer still called it with two, so the tf-metal-counters build stopped compiling: zig build test does not build that consumer, which is why it went unseen. The loop that walks the span classes records its index as metadata, the same way the profiler records its stage. The three lifecycle tests pass zero. Verified with zig build tf-metal-counters and zig build test-metal-counters-host (six of six).
Keeps this line's own load.zig, whose Plan API is what the weight staging checks consume, and restores kv_state.zig, whose Metal geometry constants 1.1.2's logical-transaction API does not carry. Follows 1.1.2's one-name text config by reading eps, hc_eps and hc_sinkhorn_iters directly, and re-registers the V4.1 check steps beside the other families'.
8a313b9 to
e740bf0
Compare
#826 covered the attention and hyperconnection operators and named the gap it left open: the Engram stage had its behavioral Metal tests and its FP32 equations, but no three-way harness of its own. This closes that gap with
zig/tests/deepseek_v41_reference/operator_engram_fp64.zig, which drives the three real Engram kernels — hash, FP8 gather and signed-sqrt gate — and compares each against the FP32 source contract and an FP64 widening of the same equations.Hash: two layers over a masked chunk of 8 tokens followed by 2 more tokens against the committed history, so the mask carry, the history tail and the four-shift rolling hash all execute. All 480 columns equal
engram_hash.hashChunk, the CPU table generator's contract (ENG64 {"stage":"hash","case":"masked-chunk-and-history-carry","cells":480,"metal_vs_contract_unequal":0}).Gather: synthetic E4M3 rows with UE8M0 scales, 32-value scale blocks and all 24 hash columns, read through the actual gather kernel in both table layouts the family reads: the default planar rows (a 256-byte weight row, an 8-byte scale row) and one interleaved 264-byte row per table row — 256 weight bytes at +0, the 8 scale bytes at +256 — expressed through the
Tablestrides the kernel already takes. Two semantics are pinned rather than assumed: the kernel computesbfloat(float(e4m3) * float(ue8m0)), and Metal flushes subnormal scales and products to zero. The FP32 contract applies that same flush, so the comparison is exact: 0 of 24,576 BF16 cells unequal in either layout. Against the FP64 ideal 281 cells differ in each layout, all of them scale-zero subnormal products; both layouts reach the same 281 with the samemax_abs2.63e-36,nrms9.12e-73 and worst index, which is the flush, not a rounding path.Gate: the signed-sqrt gate at 4 copies and 2 tokens over four magnitude families (unit, wide, narrow, cancellation), each with matched Q/K weights and BF16 hidden and KV rows. The gate carries the
contract.Specit serves — FP32 accumulation, BF16 output, depth 31 (each thread's 20 serial adds over the 5,120 keys, the twosimd_sumtrees, the term's three products) and 18 FP32 roundings after the sums — and every cell is held tocontract.boundagainst the FP64 widening of the same equations: 0 failures over 163,840 cells, the worst cell at 0.9936 of its bound, which is the BF16 output's own rounding rather than a near-miss (its ideal value lands 3.3e-6 short of the tie between two adjacent BF16 values, so the correctly rounded output is one half-ULP away and the bound there is 1.0065× that distance). Metal differs from the FP32 contract in 5 of the 163,840 cells (unit 2, wide 1, narrow 2, cancellation 0); each is inside its bound and each is listed in the receipt comment. A DEAD-masked row carrieshiddenthrough untouched, and a non-finite value on either side fails the test instead of being skipped.The only non-test changes are two module exports in
zig/src/tensorfold.zig— the Engram family module and the kernel contract — plus the test registration inzig/deepseek_v41_reference_checks.zig. No kernel, table, scale, receipt or serving-path change follows from them, and the suite is weight-free — it needs no checkpoint bytes.M3 Ultra hardware here. Checks at this head (
075940d7a):zig build test-deepseek-v41-reference -Dcpu=apple_m1: 43/43 steps, 29/29 tests passed;zig build test-deepseek-v41 -Dcpu=apple_m1: 41/41 steps, 79/79 tests passed;zig build test -Dcpu=apple_m1: 304/304 steps, 4454/4469 tests passed (15 skipped);tools/zig/lean_new.pyagainst the merge base: 0 new (502 problems against 503 on the base);zig fmt --checkclean.The full ENG64 output, the five contract-differing cells and the environment are in the receipt comment on this PR.
This stacks on #826.