Summary
This issue documents the end-to-end deployment and training workflow for running MiniMax M2.5 GRPO training with Vime on Ascend NPU (Atlas A3), intended as a reference for the community.
| Item |
Version |
Commit |
| Ascend HDK |
25.5.1.B030 |
N/A |
| CANN |
9.0.0 |
N/A |
| python |
3.12.13 |
N/A |
| torch_npu |
2.10.0.post2 |
N/A |
| triton-ascend |
3.2.1 |
N/A |
| vllm |
0.22.1+empty |
0decac0 |
| vllm-ascend |
0.22.1rc1 |
5f6faa0c |
| Megatron-Bridge |
N/A |
7f0fb345 |
| Megatron-LM |
0.16.0rc0 |
1dcf0dafa |
| MegatronAdaptor |
0.1.0 |
15582ad |
| TransformerEngineNPU |
2.13.0 |
d743c83 |
| flash-linear-attention-npu |
1.0.0 |
14c2c92 |
Usage
1. Setup
git clone -b ascend https://github.com/vllm-project/vime.git
cd vime
docker build -f docker/Dockerfile.npu -t vime-ascend:latest .
export IMAGE=vime-ascend:latest
docker run -d --name vime-npu -it --net=host --shm-size=1024g \
--privileged=true \
--cap-add=SYS_PTRACE \
--device=/dev/davinci_manager \
--device=/dev/hisi_hdc \
--device=/dev/devmm_svm \
-v /usr/local/Ascend/driver:/usr/local/Ascend/driver \
-v /usr/local/dcmi:/usr/local/dcmi \
-v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \
-v /usr/local/sbin:/usr/local/sbin \
-v /home:/home \
-v /mnt:/mnt \
-v /tmp:/tmp \
-v /data:/data \
-v /usr/share/zoneinfo/Asia/Shanghai:/etc/localtime \
$IMAGE
docker exec -it vime-npu bash
2. Download
# (1) Model
hf download PrimeIntellect/MiniMax-M2.5-bf16 --local-dir /path/to/MiniMax-M2.5-bf16
# (2) Data
hf download --repo-type dataset zhuzilin/dapo-math-17k --local-dir /path/to/dapo-math-17k
hf download --repo-type dataset zhuzilin/aime-2024 --local-dir /path/to/aime-2024
3. Trim
trim_safetensor_layers.py
python /path/to/trim_safetensor_layers.py \
--model_dir /path/to/MiniMax-M2.5-bf16 \
--out_dir /path/to/MiniMax-M2.5-bf16-2layer \
--num_layers 2 \
--write_skipped \
--asset-include "*.md" \
--asset-include "*.jinja" \
--asset-include "*.txt" \
--asset-include "*.py"
4. Training
cd /root/vime
bash scripts/run-minimax-m2-npu.sh
Summary
This issue documents the end-to-end deployment and training workflow for running MiniMax M2.5 GRPO training with Vime on Ascend NPU (Atlas A3), intended as a reference for the community.
25.5.1.B030N/A9.0.0N/A3.12.13N/A2.10.0.post2N/A3.2.1N/A0.22.1+empty0decac00.22.1rc15f6faa0cN/A7f0fb3450.16.0rc01dcf0dafa0.1.015582ad2.13.0d743c831.0.014c2c92Usage
1. Setup
2. Download
3. Trim
trim_safetensor_layers.py
4. Training
cd /root/vime bash scripts/run-minimax-m2-npu.sh