Skip to content

Refine training and conversion, support Qwen3.5 MoE and Gemma 4#18

Merged
liang2kl merged 12 commits intomainfrom
dev/train
Apr 6, 2026
Merged

Refine training and conversion, support Qwen3.5 MoE and Gemma 4#18
liang2kl merged 12 commits intomainfrom
dev/train

Conversation

@liang2kl
Copy link
Copy Markdown
Collaborator

@liang2kl liang2kl commented Mar 16, 2026

  • Adds support for quantizing and saving Qwen3.5 MoE models / Gemma 4.
  • Fix changes in vllm 0.19.0 that break the per-shard Marlin conversion.

vLLM real quant will be implemented in a separate PR.

@liang2kl liang2kl marked this pull request as ready for review March 20, 2026 05:38
@liang2kl liang2kl changed the title Refine training and conversion, support Qwen3.5 MoE Refine training and conversion, support Qwen3.5 MoE (training only) Apr 5, 2026
@liang2kl liang2kl changed the title Refine training and conversion, support Qwen3.5 MoE (training only) Refine training and conversion, support Qwen3.5 MoE and Gemma 4 Apr 5, 2026
@liang2kl liang2kl requested a review from zhijian-liu April 5, 2026 14:50
@liang2kl
Copy link
Copy Markdown
Collaborator Author

liang2kl commented Apr 5, 2026

For now the MoE support is hard-coded for Qwen3.5. Will refactor this in the next PR.

Comment thread paroquant/optim/train.py Outdated
Comment thread paroquant/optim/qexperts.py
Copy link
Copy Markdown
Member

@zhijian-liu zhijian-liu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good overall. But we need to think about MoE integration complexity.

@zhijian-liu zhijian-liu self-requested a review April 6, 2026 02:33
@liang2kl liang2kl merged commit 22d9e9f into main Apr 6, 2026
@liang2kl liang2kl deleted the dev/train branch April 6, 2026 02:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants