Skip to content

Add DeepSeek-V2 model#379

Open
GoodOlClint wants to merge 2 commits into
ml-explore:mainfrom
GoodOlClint:pr/deepseek-v2
Open

Add DeepSeek-V2 model#379
GoodOlClint wants to merge 2 commits into
ml-explore:mainfrom
GoodOlClint:pr/deepseek-v2

Conversation

@GoodOlClint

Copy link
Copy Markdown
Contributor

Proposed changes

Adds the DeepSeek-V2 architecture (model_type: "deepseek_v2") to MLXLLM — a port of mlx-lm's deepseek_v2.py. Covers DeepSeek-V2 / V2-Lite, which had no Swift implementation.

It reuses the in-tree DeepSeek-V3 MLA machinery (q_a/q_b + kv_a_proj_with_mqa/kv_b low-rank compression, qk nope/rope split, the shared initializeRope YaRN path) and SwitchGLU MoE with shared experts.

It differs from V3 only in the router (DeepseekV2MoEGate):

  • plain softmax scoring (no sigmoid + e_score_correction_bias),
  • optional group_limited_greedy group masking by each group's max score (vs V3's top-2-per-group sum),
  • an unconditional routed_scaling_factor.

Dense layers below first_k_dense_replace use the standard MLP. sanitize() stacks per-expert gate/up/down projections (weight/scales/biases) into switch_mlp.

Tests (DeepseekV2Tests, no model download): forward-pass logit shape; a deterministic group-limited-greedy routing test (synthetic gate weights → the winning group confines top-k); and sanitize() expert-stacking. Verified via xcodebuild test ... -only-testing:MLXLMTests/DeepseekV2Tests → 3 tests, 0 failures, TEST SUCCEEDED.

Note: validation is structural/unit-level (shape + routing + weight-remap); end-to-end numerical parity against a downloaded DeepSeek-V2 checkpoint has not been run.

Checklist

  • I have read the CONTRIBUTING document
  • I have run pre-commit run --all-files to format my code / installed pre-commit prior to committing changes
  • I have added tests that prove my fix is effective or that my feature works
  • I have updated the necessary documentation (if needed)

@GoodOlClint

Copy link
Copy Markdown
Contributor Author

End-to-end validation (real weights) — caught and fixed a load bug.

Loading mlx-community/DeepSeek-V2-Lite-Chat-4bit-mlx initially failed: V2-Lite sets q_lora_rank: null (direct q_proj, no low-rank q compression), but the config defaulted a null/absent value to 1536, taking the q_a/q_b LoRA path and mismatching the checkpoint's q_proj. Fixed by making qLoraRank optional (nil ⇒ q_proj); added a regression test.

After the fix, forward-pass parity on a fixed token sequence [1..8] vs the Python mlx_lm reference:

argmax top5(last)
Python mlx_lm [185,1,4,5,6,826,4698,575] [575,2,185,4,746]
this PR (Swift) [185,1,4,5,6,826,4698,575] [575,2,185,4,746]

Byte-exact match (MLA + YaRN + group-routed MoE + quantized loading).

GoodOlClint and others added 2 commits June 29, 2026 15:05
Port of mlx-lm's deepseek_v2.py. Reuses the in-tree DeepSeek-V3 MLA
machinery (q_a/q_b + kv_a_with_mqa/kv_b compression, qk nope/rope split,
shared initializeRope YaRN path) and SwitchGLU MoE with shared experts.

Differs from V3 in the router (DeepseekV2MoEGate): plain softmax scoring
(no sigmoid + e_score_correction_bias), optional group_limited_greedy
group masking by per-group max (vs V3's top-2-group-sum), and an
unconditional routed_scaling_factor. Dense layers below first_k_dense_replace
use the standard MLP. sanitize() stacks per-expert gate/up/down projections
into switch_mlp. Registered as "deepseek_v2".

Tests (DeepseekV2Tests, no model download): forward-pass logits shape,
deterministic group-limited-greedy routing (winning group confines top-k),
and sanitize() expert-stacking.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
DeepSeek-V2-Lite sets `q_lora_rank: null` — direct q_proj with no low-rank
q compression — while full DeepSeek-V2 sets 1536. The config previously
defaulted a null/absent value to 1536, which built the q_a/q_b LoRA path and
failed to load V2-Lite checkpoints (q_proj weight present, q_a_proj/q_b_proj
expected). Make qLoraRank `Int?`; nil ⇒ q_proj. Caught by end-to-end load of
mlx-community/DeepSeek-V2-Lite-Chat-4bit-mlx.

Adds a regression test (null → nil, 1536 → 1536) and switches the test config
to the realistic null form.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant