Add DeepSeek-V2 model#379
Open
GoodOlClint wants to merge 2 commits into
Open
Conversation
Contributor
Author
|
End-to-end validation (real weights) — caught and fixed a load bug. Loading After the fix, forward-pass parity on a fixed token sequence
Byte-exact match (MLA + YaRN + group-routed MoE + quantized loading). |
Port of mlx-lm's deepseek_v2.py. Reuses the in-tree DeepSeek-V3 MLA machinery (q_a/q_b + kv_a_with_mqa/kv_b compression, qk nope/rope split, shared initializeRope YaRN path) and SwitchGLU MoE with shared experts. Differs from V3 in the router (DeepseekV2MoEGate): plain softmax scoring (no sigmoid + e_score_correction_bias), optional group_limited_greedy group masking by per-group max (vs V3's top-2-group-sum), and an unconditional routed_scaling_factor. Dense layers below first_k_dense_replace use the standard MLP. sanitize() stacks per-expert gate/up/down projections into switch_mlp. Registered as "deepseek_v2". Tests (DeepseekV2Tests, no model download): forward-pass logits shape, deterministic group-limited-greedy routing (winning group confines top-k), and sanitize() expert-stacking. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
DeepSeek-V2-Lite sets `q_lora_rank: null` — direct q_proj with no low-rank q compression — while full DeepSeek-V2 sets 1536. The config previously defaulted a null/absent value to 1536, which built the q_a/q_b LoRA path and failed to load V2-Lite checkpoints (q_proj weight present, q_a_proj/q_b_proj expected). Make qLoraRank `Int?`; nil ⇒ q_proj. Caught by end-to-end load of mlx-community/DeepSeek-V2-Lite-Chat-4bit-mlx. Adds a regression test (null → nil, 1536 → 1536) and switches the test config to the realistic null form. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
GoodOlClint
force-pushed
the
pr/deepseek-v2
branch
from
June 29, 2026 20:06
6c8efeb to
7d7e1bb
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Proposed changes
Adds the DeepSeek-V2 architecture (
model_type: "deepseek_v2") to MLXLLM — a port of mlx-lm'sdeepseek_v2.py. Covers DeepSeek-V2 / V2-Lite, which had no Swift implementation.It reuses the in-tree DeepSeek-V3 MLA machinery (
q_a/q_b+kv_a_proj_with_mqa/kv_blow-rank compression, qk nope/rope split, the sharedinitializeRopeYaRN path) andSwitchGLUMoE with shared experts.It differs from V3 only in the router (
DeepseekV2MoEGate):e_score_correction_bias),group_limited_greedygroup masking by each group's max score (vs V3's top-2-per-group sum),routed_scaling_factor.Dense layers below
first_k_dense_replaceuse the standard MLP.sanitize()stacks per-expert gate/up/down projections (weight/scales/biases) intoswitch_mlp.Tests (
DeepseekV2Tests, no model download): forward-pass logit shape; a deterministic group-limited-greedy routing test (synthetic gate weights → the winning group confines top-k); andsanitize()expert-stacking. Verified viaxcodebuild test ... -only-testing:MLXLMTests/DeepseekV2Tests→ 3 tests, 0 failures, TEST SUCCEEDED.Checklist
pre-commit run --all-filesto format my code / installed pre-commit prior to committing changes