VEQ: Modality-Adaptive Quantization for MoE Vision-Language Models

Guangshuo Qin, Zhiteng Li, Zheng Chen, Weihang Zhang, Linghe Kong and Yulun Zhang [arXiv] [supplementary material]

🔥🔥🔥 News

• 2026-01-31: This repo is released.

---

Abstract: Mixture-of-Experts(MoE) Vision-Language Models (VLMs) offer remarkable performance but incur prohibitive memory and computational costs, making compression essential. Post-Training Quantization (PTQ) is an effective training-free technique to address the massive memory and computation overhead. Existing quantization paradigms fall short as they are oblivious to two critical forms of heterogeneity: the inherent discrepancy between vision and language tokens, and the non-uniform contribution of different experts. To bridge this gap, we propose Visual Expert Quantization (VEQ), a dual-aware quantization framework designed to simultaneously accommodate cross-modal differences and heterogeneity between experts. Specifically, VEQ incorporates 1) Modality-expert-aware Quantization, which utilizes expert activation frequency to prioritize error minimization for pivotal experts, and 2) Modality-affinity-aware Quantization, which constructs an enhanced Hessian matrix by integrating token-expert affinity with modality information to guide the calibration process. Extensive experiments across diverse benchmarks verify that VEQ consistently outperforms state-of-the-art baselines. Specifically, under the W3A16 configuration, our method achieves significant average accuracy gains of 2.04% on Kimi-VL and 3.09% on Qwen3-VL compared to the previous SOTA quantization methods, demonstrating superior robustness across various multimodal tasks. Our code will be available at https://github.com/qsstcl/VEQ.

⚒️ TODO

• Complete this repository
• Release the code

🔗 Contents

• Results
• Citation
• Acknowledgements

🔎 Results

We achieve significant average accuracy gains of 2.04% on Kimi-VL and 3.09% on Qwen3-VL compared to the previous SOTA quantization methods

Zero-shot performance of Kimi-VL-Instruct under 3-bit weight quantization (W3A16).

Detailed results can be found in the paper.

 Quantitative Comparisons (click to expand)

• Performance comparison of various methods on Kimi-VL-Instruct.

  

• Performance comparison of various methods on Qwen3-VL-30B-A3B-Instruct.

  

 Some examples on downstream tasks

Citation

If you find the code helpful in your research or work, please cite the following paper.

@article{qin2026veq,
      title={VEQ: Modality-Adaptive Quantization for MoE Vision-Language Models}, 
      author={Guangshuo Qin and Zhiteng Li and Zheng Chen and Weihang Zhang and Linghe Kong and Yulun Zhang},
      journal={arXiv preprint arxiv:2602.01037},
      year={2026}
}

💡 Acknowledgements

This work is released under the Apache 2.0 license.