DeepSeek V4.1: fp64 reference tests for the attention and hyperconnection operators by akol1 · Pull Request #826 · ashhart/TensorFold

DeepSeek V4.1 Flash: configuration, Engram tables and resident weight staging

akol1

committed

d4d3a59

DeepSeek V4.1 Flash: Metal operators with fp64-dot references and the qualification runner

akol1

committed

3ef88cd

THIRD_PARTY_NOTICES: the native DS4.1 reference follows DeepSeek-V4.1-Flash's MIT inference

akol1

committed

d9b30d3

DeepSeek V4.1 load: a shard contributes only the tensors the index maps to it

akol1

committed

2e23371

DeepSeek V4.1 load: the router gate and bias ride the pack's dtypes

akol1

committed

c7dad8c

DeepSeek V4.1 load: the Engram gate weights ride f32 or bf16

akol1

committed

91b8a68

DeepSeek V4.1: the kept-dense EXL3 form, for rank-split repacks

akol1

committed

9ed1303

DeepSeek V4.1 reference: the kept-dense apply

akol1

committed

79856d3

DeepSeek V4.1 load: the shard-plan tests pass the index-ownership argument, and the duplicate-tensor case stops logging at error level

akol1

committed

af191ce

DeepSeek V4.1: the rebase residue — one counters binding, the state-window kernel's brace

akol1

committed

dae11d3

mHC fp64 ideal reference: the mix at rounding level, 1-ULP noise attenuated

akol1

committed

35723cb

CSA2 reference: FP64-ideal parameterization of the pool, RMS and sparse-head boundaries

akol1

committed

6a69841

csa2: FP64 stage harness, compressor pool + RMS, six adversarial input families

akol1

committed

e2dfb0c

csa2: FP64 stage harness over the sparse attention kernel, with kernel-faithful key addressing

akol1

committed

50f389a

csa2: indexer stage harness — FP32/FP64 score comparison and top-k selection agreement

akol1

committed

6154d85

csa2: indexer stage receipts — carry the BF16 einsum/score boundaries in the FP64 ideal, read positions through the view offset

akol1

committed

041d492

csa2: sparse stage reads its own FP32 rounding level before the BF16 output quantum

akol1

committed

13ca5c1

csa2: per-stage harnesses for sinks, tile denominators, output mix and RoPE

akol1

committed

fa1e5ca

csa2: measure how much score noise the top-k selection absorbs

akol1

committed

d3efdc9

The fp64 harness ports: dedupe the shared math helpers, fix the shard-plan test calls

akol1

committed

0a2ab87

The oracle records the indexer's selection margins while it teaches

akol1

committed

1cae50b

The indexer-margin capture at depth: one diagnostic token's selection scratch, verbatim

akol1

committed

0d77fac

the operator harnesses through zig fmt

akol1

committed

331e89b

DeepSeek V4.1: the sparse output accumulates each 64-column tile in compensated FP32 pairs

akol1

committed

a881b7f

The dense EXL3 apply accumulates its dot as a hi/lo FP32 pair

akol1

committed

10fd1c1

DeepSeek V4.1: the token map ships its table generator and its unit tests

akol1

committed

8ba0ee9

DeepSeek V4.1: the family's comments on one line each, within the 120-column rule

akol1

committed

5a3183f

DeepSeek V4.1 reference: the operator harnesses split at their seams

akol1

committed

e3edac5

DeepSeek V4.1: the reference and Metal comments on one line each

akol1

committed

28c93ef

DeepSeek V4.1 reference: the index tie fixture broadcasts row 0 into every key

akol1

committed

4cb1b17

Pass the span metadata the counter recorder now requires

akol1

committed

709574d

Navigation Menu

Repository navigation

DeepSeek V4.1: fp64 reference tests for the attention and hyperconnection operators - #826

DeepSeek V4.1: fp64 reference tests for the attention and hyperconnection operators#826
akol1 wants to merge 31 commits into
ashhart:mainfrom
akol1:pr-dsv41-c-tests

Commits on Oct 21, 2026

DeepSeek V4.1 Flash: configuration, Engram tables and resident weight staging

DeepSeek V4.1 Flash: Metal operators with fp64-dot references and the qualification runner

THIRD_PARTY_NOTICES: the native DS4.1 reference follows DeepSeek-V4.1-Flash's MIT inference

DeepSeek V4.1 load: a shard contributes only the tensors the index maps to it

DeepSeek V4.1 load: the router gate and bias ride the pack's dtypes

DeepSeek V4.1 load: the Engram gate weights ride f32 or bf16

DeepSeek V4.1: the kept-dense EXL3 form, for rank-split repacks

DeepSeek V4.1 reference: the kept-dense apply

DeepSeek V4.1 load: the shard-plan tests pass the index-ownership argument, and the duplicate-tensor case stops logging at error level

DeepSeek V4.1: the rebase residue — one counters binding, the state-window kernel's brace

mHC fp64 ideal reference: the mix at rounding level, 1-ULP noise attenuated

CSA2 reference: FP64-ideal parameterization of the pool, RMS and sparse-head boundaries

csa2: FP64 stage harness, compressor pool + RMS, six adversarial input families

csa2: FP64 stage harness over the sparse attention kernel, with kernel-faithful key addressing

csa2: indexer stage harness — FP32/FP64 score comparison and top-k selection agreement

csa2: indexer stage receipts — carry the BF16 einsum/score boundaries in the FP64 ideal, read positions through the view offset

csa2: sparse stage reads its own FP32 rounding level before the BF16 output quantum

csa2: per-stage harnesses for sinks, tile denominators, output mix and RoPE

csa2: measure how much score noise the top-k selection absorbs

The fp64 harness ports: dedupe the shared math helpers, fix the shard-plan test calls

The oracle records the indexer's selection margins while it teaches

The indexer-margin capture at depth: one diagnostic token's selection scratch, verbatim

the operator harnesses through zig fmt

DeepSeek V4.1: the sparse output accumulates each 64-column tile in compensated FP32 pairs

The dense EXL3 apply accumulates its dot as a hi/lo FP32 pair

DeepSeek V4.1: the token map ships its table generator and its unit tests

DeepSeek V4.1: the family's comments on one line each, within the 120-column rule

DeepSeek V4.1 reference: the operator harnesses split at their seams

DeepSeek V4.1: the reference and Metal comments on one line each

DeepSeek V4.1 reference: the index tie fixture broadcasts row 0 into every key

Pass the span metadata the counter recorder now requires