Support for GLM-4.7-FP8

[maodoudou168]

Hi, may I ask a few questions about plans on supporting GLM-4.7 models?
I have used the DFLASH training code in SpecForge to train a draft model for GLM-4.7-FP8, and the accuracy is higher than 0.9, but when running on sglang, the accept rate is only 1.x, I used this sglang PR sgl-project/sglang#16818, ard there any other implementations needed for sglang inference for models that are not Qwen? May I know your official plans on suporting GLM-4.7-FP8?
Thanks a lot!

[jianc99]

Thanks for the detailed question!

The SpecForge training code for DFlash is not official—it was inferred from our inference implementation. We’ve just released the paper https://arxiv.org/abs/2602.06036, which includes the complete training details. Our official training code is still being actively developed: we’re optimizing efficiency and adding support for features like multi-node training, TP+DP, etc. The goal is to provide a push-the-button training pipeline for efficiently training DFlash draft models, which should be ready soon.

For GLM-4.7-FP8: yes, it’s on our roadmap. We plan to first support GLM-4.7-Flash, and then extend support to GLM-4.7 / GLM-4.7-FP8.

[maodoudou168]

Thanks for the detailed question!

The SpecForge training code for DFlash is not official—it was inferred from our inference implementation. We’ve just released the paper https://arxiv.org/abs/2602.06036, which includes the complete training details. Our official training code is still being actively developed: we’re optimizing efficiency and adding support for features like multi-node training, TP+DP, etc. The goal is to provide a push-the-button training pipeline for efficiently training DFlash draft models, which should be ready soon.

For GLM-4.7-FP8: yes, it’s on our roadmap. We plan to first support GLM-4.7-Flash, and then extend support to GLM-4.7 / GLM-4.7-FP8.

Thanks for your reply, I've made some further attempts on the GLM-4.7-FP8 model, but it seems that the loss can only decrease to 3.x, and the accuracy 0.2x, the accept rate of blocksize=16 is around 5.x on sharegpt dataset test. Since GLM-4.7-FP8 is a large MoE model, do you have any suggestions if the draft model has not fully converged, or has already met the limits? Thanks a lot!