Skip to content

Decode BF16 weights and score merged MoE expert tensors - #144

Merged
WestWaters merged 1 commit into
WestWaters:mainfrom
SloptimistPrime:contrib/bf16-moe-probe
Oct 6, 2026
Merged

WestWaters merged 1 commit into
WestWaters:mainfrom
SloptimistPrime:contrib/bf16-moe-probe

Conversation

@SloptimistPrime

Copy link
Copy Markdown
Contributor

The imatrix probe cast stored BF16 bytes to floats instead of decoding the weights. Decode BF16 correctly and handle merged MoE tensors with separate input-channel importance for each expert.

Record MoE routers separately and require them to retain their source precision. Unknown tensor families, invalid shapes and non-finite costs still stop the probe.

22 tests pass using real GGUF files and direct cost calculations. On Qwen3-30B-A3B, 26 sampled matrices containing 66,060,288 values decode exactly against the original safetensors. A small HF-versus-GGUF prediction check passed preselected limits in all three domains.

Includes public dataset revisions, row indices and hashes for reproduction. Expanded calibration reduced partial-expert warnings from 36 to 28, but coverage is still incomplete. These scores estimate rounding error; they are not measured quality gains or a completed allocation-versus-uniform comparison.

Results and limits: notes/imatrix-probe-gb10.md.

I used AI assistance and reviewed and tested the changes.

@WestWaters
WestWaters merged commit 7c865e9 into WestWaters:main Oct 6, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants