Hi, thank you for sharing the code!
Qwen2.5-Math-7B has a positional embedding size of 4096, but the SFT script scripts/train_sft.sh sets MODEL_MAX_LENGTH=8000.
Could you let me know if you extended the model's context size or made other modifications?
Hi, thank you for sharing the code!
Qwen2.5-Math-7B has a positional embedding size of
4096, but the SFT scriptscripts/train_sft.shsetsMODEL_MAX_LENGTH=8000.Could you let me know if you extended the model's context size or made other modifications?