Repository navigation
Proposal: experimental DeepSeek V4.1 Flash resident Metal backend (qualification blocked) #685
Description
Proposing an explicitly draft-only implementation and qualification harness for DeepSeek V4.1 Flash on resident Metal, including EXL3 projections, Engram tables, hyperconnections, sparse/compressed attention, serialized text serving, and independent operator/whole-model diagnostics.
The latest retained eight-token whole-model comparison fails the unchanged 1% normalized RMS error gate: NRMS 0.040960584768951355, maximum absolute logit error 1.5123639106750488, zero greedy disagreements. Agreement of greedy choices is not numerical qualification. Source-CUDA unary rounding and full-head sparse attention remain unresolved; the available CUDA device cannot launch the canonical 64-head kernel because its shared-memory requirement exceeds the device limit.
A draft PR will preserve this failed gate, expose unsupported features rather than silently falling back, distinguish historical receipts from current-branch verification, and include reproducible commands, pinned public checkpoint revisions, provenance, and third-party notices. No release-qualified model claim or ready-for-review transition is proposed. The implementation is being rebased and hardened against current main before publication.
Related independently reviewable shared contributions: #648, #649, #650, #680, #681, and #684. The draft will identify overlap; none is treated as already merged.
Activity
ashhart commentedon Oct 14, 2026
Hello @akol1,
Thank you for keeping the failed gate in the proposal, and for saying greedy agreement is not numerical qualification.
The native binary has no DeepSeek-V4 family. pull and info refuse those checkpoints (docs/recipes/deepseek-v4-flash.md:5). README lists DeepSeek-V4 as still under qualification (README.md:88). Metal refuses any other model_type with the Python 0.6.6 install line (zig/src/native/metal.zig:119). The Python line is frozen at v0.6.6, which is where the MLX recipe still lives.
An experimental Metal backend is on the maintainer's list. A draft PR that preserves the failed 1% NRMS gate, names unsupported features instead of falling back, and separates old receipts from the branch you just ran, is what helps. No release-qualified claim until that gate passes.
Thank you again.
ashhart commentedon Oct 16, 2026
Thank you, @akol1, for preserving the failed whole-model numerical check alongside the Metal proposal.
The implementation is now in #824 through #827, coordinated with the CUDA work in #847. That tracker also holds the whole-model gate question and the shared configuration, loader and state layout.
We're closing this duplicate proposal onto #847. The numerical qualification remains open there; greedy agreement alone does not settle it.
Posted by Ash's AI Assistant Synthena