From d0c244291059b6f26dc9b1b1830c206095f32b82 Mon Sep 17 00:00:00 2001 From: ooooo <3164076421@qq.com> Date: Fri, 11 Sep 2026 22:13:55 +0800 Subject: [PATCH 1/5] docs(links): fix broken documentation links MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit # 📝 Documentation - Correct DeepEyes dataset and Megatron Bridge links in English and Chinese docs. - Fix same-language autoscaler links to elastic rollout and fully async training. - Point math OPD examples to the current script and align model and configuration descriptions. ## Validation - Run all pre-commit hooks successfully. - Verify changed relative link targets exist. - Skip multi-node GPU integration tests because only documentation changed. --- docs/en/examples/deepeyes.md | 2 +- docs/en/guide/autoscaler-k8s-keda.md | 6 +++--- docs/en/guide/customize-training.md | 2 +- docs/zh/examples/deepeyes.md | 2 +- docs/zh/guide/autoscaler-k8s-keda.md | 6 +++--- docs/zh/guide/customize-training.md | 2 +- examples/on_policy_distillation/README.md | 8 ++++---- 7 files changed, 14 insertions(+), 14 deletions(-) diff --git a/docs/en/examples/deepeyes.md b/docs/en/examples/deepeyes.md index 2c171e84b..856cfa22b 100644 --- a/docs/en/examples/deepeyes.md +++ b/docs/en/examples/deepeyes.md @@ -13,7 +13,7 @@ The DeepEyes example demonstrates **agentic multi-turn vision-language RL** usin ## Data Preparation -DeepEyes uses the [Visual-Agent/DeepEyes](https://huggingface.co/datasets/Visual-Agent/DeepEyes) dataset from HuggingFace. The dataset contains visual QA samples with images stored inline (HF Image format). +DeepEyes uses the [ChenShawn/DeepEyes-Datasets-47k](https://huggingface.co/datasets/ChenShawn/DeepEyes-Datasets-47k) dataset from HuggingFace. The dataset contains visual QA samples with images stored inline (HF Image format). ### Download the Dataset diff --git a/docs/en/guide/autoscaler-k8s-keda.md b/docs/en/guide/autoscaler-k8s-keda.md index 2ad5583b6..9a4b2c105 100644 --- a/docs/en/guide/autoscaler-k8s-keda.md +++ b/docs/en/guide/autoscaler-k8s-keda.md @@ -450,7 +450,7 @@ ______________________________________________________________________ - 引擎会使用初始模型权重运行 - 后续权重更新在 Actor 的 `update_weights_fully_async()` 完成后自动触发 -- 如果对权重一致性要求严格，参考 [弹性 Rollout 扩缩容文档](../zh/guide/elastic-rollout.md) 中的权重同步机制 +- 如果对权重一致性要求严格，参考 [弹性 Rollout 扩缩容文档](./elastic-rollout.md) 中的权重同步机制 ______________________________________________________________________ @@ -549,7 +549,7 @@ ______________________________________________________________________ ## 延伸阅读 -- [弹性 Rollout 扩缩容](../zh/guide/elastic-rollout.md) — Relax 弹性扩缩容完整文档 -- [全异步训练流水线](../zh/guide/fully-async-training.md) — 弹性扩缩容的基础运行模式 +- [弹性 Rollout 扩缩容](./elastic-rollout.md) — Relax 弹性扩缩容完整文档 +- [全异步训练流水线](./fully-async-training.md) — 弹性扩缩容的基础运行模式 - [KEDA 官方文档](https://keda.sh/docs/) — KEDA ScaledObject 配置参考 - [Prometheus Operator](https://prometheus-operator.dev/) — ServiceMonitor 配置参考 diff --git a/docs/en/guide/customize-training.md b/docs/en/guide/customize-training.md index 87733ade7..41650d7f0 100644 --- a/docs/en/guide/customize-training.md +++ b/docs/en/guide/customize-training.md @@ -18,7 +18,7 @@ hf download Qwen/Qwen3-VL-4B-Instruct --local-dir /root/Qwen3-VL-4B-Instruct ### Megatron Weights to HF Weights ::: tip No Manual Conversion Needed with Megatron Bridge -Relax uses [Megatron Bridge](../../../relax/backends/megatron/mbridge/) as the weight bridging layer for its training backend, automatically handling bidirectional HF ↔ Megatron weight conversion during training — **no manual conversion steps required**. Simply specify the following option in your launch script: +Relax uses [Megatron Bridge](https://github.com/NVIDIA-NeMo/Megatron-Bridge) as the weight bridging layer for its training backend, automatically handling bidirectional HF ↔ Megatron weight conversion during training — **no manual conversion steps required**. Simply specify the following option in your launch script: ::: ```bash diff --git a/docs/zh/examples/deepeyes.md b/docs/zh/examples/deepeyes.md index bc735b071..7d8cc7fe0 100644 --- a/docs/zh/examples/deepeyes.md +++ b/docs/zh/examples/deepeyes.md @@ -13,7 +13,7 @@ DeepEyes 示例演示了使用 Relax 进行 **Agent 式多轮视觉语言强化 ## 数据准备 -DeepEyes 使用 HuggingFace 上的 [Visual-Agent/DeepEyes](https://huggingface.co/datasets/Visual-Agent/DeepEyes) 数据集。数据集包含视觉问答样本，图像以 HF Image 格式内嵌存储。 +DeepEyes 使用 HuggingFace 上的 [ChenShawn/DeepEyes-Datasets-47k](https://huggingface.co/datasets/ChenShawn/DeepEyes-Datasets-47k) 数据集。数据集包含视觉问答样本，图像以 HF Image 格式内嵌存储。 ### 下载数据集 diff --git a/docs/zh/guide/autoscaler-k8s-keda.md b/docs/zh/guide/autoscaler-k8s-keda.md index 2ad5583b6..9a4b2c105 100644 --- a/docs/zh/guide/autoscaler-k8s-keda.md +++ b/docs/zh/guide/autoscaler-k8s-keda.md @@ -450,7 +450,7 @@ ______________________________________________________________________ - 引擎会使用初始模型权重运行 - 后续权重更新在 Actor 的 `update_weights_fully_async()` 完成后自动触发 -- 如果对权重一致性要求严格，参考 [弹性 Rollout 扩缩容文档](../zh/guide/elastic-rollout.md) 中的权重同步机制 +- 如果对权重一致性要求严格，参考 [弹性 Rollout 扩缩容文档](./elastic-rollout.md) 中的权重同步机制 ______________________________________________________________________ @@ -549,7 +549,7 @@ ______________________________________________________________________ ## 延伸阅读 -- [弹性 Rollout 扩缩容](../zh/guide/elastic-rollout.md) — Relax 弹性扩缩容完整文档 -- [全异步训练流水线](../zh/guide/fully-async-training.md) — 弹性扩缩容的基础运行模式 +- [弹性 Rollout 扩缩容](./elastic-rollout.md) — Relax 弹性扩缩容完整文档 +- [全异步训练流水线](./fully-async-training.md) — 弹性扩缩容的基础运行模式 - [KEDA 官方文档](https://keda.sh/docs/) — KEDA ScaledObject 配置参考 - [Prometheus Operator](https://prometheus-operator.dev/) — ServiceMonitor 配置参考 diff --git a/docs/zh/guide/customize-training.md b/docs/zh/guide/customize-training.md index d7dc0b19d..e839209bb 100644 --- a/docs/zh/guide/customize-training.md +++ b/docs/zh/guide/customize-training.md @@ -18,7 +18,7 @@ hf download Qwen/Qwen3-VL-4B-Instruct --local-dir /root/Qwen3-VL-4B-Instruct ### Megatron 权重转 HF 权重 ::: tip 使用 Megatron Bridge 无需手动转换 -Relax 默认使用 [Megatron Bridge](../../../relax/backends/megatron/mbridge/) 作为训练后端的权重桥接层，在训练过程中自动完成 HF ↔ Megatron 权重的双向转换，**无需任何手动转换步骤**。只需在启动脚本中指定以下选项即可： +Relax 默认使用 [Megatron Bridge](https://github.com/NVIDIA-NeMo/Megatron-Bridge) 作为训练后端的权重桥接层，在训练过程中自动完成 HF ↔ Megatron 权重的双向转换，**无需任何手动转换步骤**。只需在启动脚本中指定以下选项即可： ::: ```bash diff --git a/examples/on_policy_distillation/README.md b/examples/on_policy_distillation/README.md index 8ec2a0172..9d699e54d 100644 --- a/examples/on_policy_distillation/README.md +++ b/examples/on_policy_distillation/README.md @@ -66,9 +66,9 @@ SGLang per-position top-K log-prob transfer optimization (Qwen3-4B, simulated da ### Text OPD -| Script | Description | -| ------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------- | -| [`math_opd/run-opd-sampled-8xgpu-colocate.sh`](math_opd/run-opd-sampled-8xgpu-colocate.sh) | Math OPD, Qwen3-4B → Qwen3-4B-Non-Thinking-RL, `student_sampled` + adv mode, 8 GPU colocate | +| Script | Description | +| -------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- | +| [`math_opd/run-opd-qwen35-35B-A3B-8xgpu-colocate.sh`](math_opd/run-opd-qwen35-35B-A3B-8xgpu-colocate.sh) | Math OPD, student: Qwen3.5-35B-A3B; teacher: Qwen3.5-35B-A3B-GRPO-dapomath17k-400step; `student_sampled` + adv + `reverse_kl`, 8 GPU colocate | ### Multimodal OPD @@ -80,7 +80,7 @@ SGLang per-position top-K log-prob transfer optimization (Qwen3-4B, simulated da ## Common Combinations -Based on [`math_opd/run-opd-sampled-8xgpu-colocate.sh`](math_opd/run-opd-sampled-8xgpu-colocate.sh), modify key variables to switch modes: +Based on [`math_opd/run-opd-qwen35-35B-A3B-8xgpu-colocate.sh`](math_opd/run-opd-qwen35-35B-A3B-8xgpu-colocate.sh), edit the corresponding flags in `OPD_ARGS` to switch modes. The variables below are illustrative: map `OPD_KL_COEF`, `OPD_LOSS_COEF`, `OPD_TOKEN_SELECTION`, and `OPD_KL_TYPE` to `--opd-kl-coef`, `--opd-loss-coef`, `--opd-token-selection`, and `--opd-kl-type`, respectively. The script does not read these variables automatically: > **KL type note**: `student_sampled` supports only `reverse_kl` and `low_var_kl`; `student_topk`, `teacher_topk`, and `union` support `reverse_kl`, `forward_kl`, and `jsd` (set via `--opd-kl-type`; `jsd` can be tuned with `--opd-jsd-alpha`). From b3ded5d62708c9b8b402fa551303790c71995506 Mon Sep 17 00:00:00 2001 From: ooooo <3164076421@qq.com> Date: Fri, 11 Sep 2026 22:26:39 +0800 Subject: [PATCH 2/5] docs(links): fix section anchor references MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit # 📝 Documentation - Correct English and Chinese low-precision training links to match existing heading anchors. - Point the MOPD configuration reference to the OPD common combinations section. - Preserve heading text without adding explicit anchor IDs. ## Validation - Pass Lychee offline fragment checks: 8 links, 0 errors. - Pass all pre-commit hooks and staged diff checks. - Skip multi-node GPU integration tests because only documentation links changed. --- docs/en/examples/low-precision-training.md | 4 ++-- docs/zh/examples/low-precision-training.md | 6 +++--- examples/on_policy_distillation/mopd/README.md | 2 +- 3 files changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/en/examples/low-precision-training.md b/docs/en/examples/low-precision-training.md index 177dabbbd..4e3332384 100644 --- a/docs/en/examples/low-precision-training.md +++ b/docs/en/examples/low-precision-training.md @@ -185,7 +185,7 @@ Relax ships two reference recipes for Qwen3-30B-A3B (8-GPU colocate): **FP8 nati 1. A BF16 HF checkpoint (e.g. `Qwen3-30B-A3B`). 2. The Megatron patch at `docker/patch/megatron/20260506-85bced0ae.patch` applied (baked into the project Dockerfile). It provides both the FP8 overrides and the INT4 `_FakeInt4QuantizationSTE` that overrides `TEGroupedLinear._get_weight_tensors()`. -The FP8 recipe additionally needs a TransformerEngine build with FP8 blockwise scaling support. The INT4 recipe additionally needs the `fake_int4_quant_cuda` CUDA extension built — see [Build the int4_qat kernel](#build-the-int4-qat-kernel) below. +The FP8 recipe additionally needs a TransformerEngine build with FP8 blockwise scaling support. The INT4 recipe additionally needs the `fake_int4_quant_cuda` CUDA extension built — see [Build the int4_qat kernel](#build-the-int4_qat-kernel) below. ### FP8 Recipe @@ -331,5 +331,5 @@ Do not swap `--sglang-hf-checkpoint` to the BF16 cast for "consistency". SGLang' ::: ::: tip -This recipe assumes the W4A16 release was produced with **symmetric** quantization (matching the training-side STE). If you regenerate the W4A16 from BF16 via `convert_hf_to_int4.py`, always pass `--is-symmetric` — see the warning in the [Offline Quantization Tools](#convert_hf_to_int4-py) section above. +This recipe assumes the W4A16 release was produced with **symmetric** quantization (matching the training-side STE). If you regenerate the W4A16 from BF16 via `convert_hf_to_int4.py`, always pass `--is-symmetric` — see the warning in the [Offline Quantization Tools](#convert_hf_to_int4py) section above. ::: diff --git a/docs/zh/examples/low-precision-training.md b/docs/zh/examples/low-precision-training.md index efb9e3fa5..bd98119d1 100644 --- a/docs/zh/examples/low-precision-training.md +++ b/docs/zh/examples/low-precision-training.md @@ -114,7 +114,7 @@ FP8 训练工作流下通常 **不需要** 这个脚本 — bridge 模式（`--m ### `convert_hf_to_int4.py` -把 BF16 的 HF checkpoint 量化为 W4A16（compressed-tensors）。依赖 `fake_int4_quant_cuda` kernel，需先编译（见 [编译 int4_qat kernel](#编译-int4-qat-kernel)）。 +把 BF16 的 HF checkpoint 量化为 W4A16（compressed-tensors）。依赖 `fake_int4_quant_cuda` kernel，需先编译（见 [编译 int4_qat kernel](#编译-int4_qat-kernel)）。 ```bash python scripts/tools/convert_hf_to_int4.py \ @@ -185,7 +185,7 @@ Relax 在 Qwen3-30B-A3B（8 卡 colocate）上提供两条参考配方：**FP8 1. 一个 BF16 HF checkpoint（例如 `Qwen3-30B-A3B`）。 2. 应用 Megatron patch `docker/patch/megatron/20260506-85bced0ae.patch`（项目 Dockerfile 已自动应用）—— 该 patch 同时提供 FP8 配套的 override 与 INT4 假量化的 `_FakeInt4QuantizationSTE`（override 了 `TEGroupedLinear._get_weight_tensors()`）。 -FP8 配方额外需要一个支持 FP8 blockwise scaling 的 TransformerEngine 构建；INT4 配方额外需要编译 `fake_int4_quant_cuda` CUDA 扩展，见下文 [编译 int4_qat kernel](#编译-int4-qat-kernel)。 +FP8 配方额外需要一个支持 FP8 blockwise scaling 的 TransformerEngine 构建；INT4 配方额外需要编译 `fake_int4_quant_cuda` CUDA 扩展，见下文 [编译 int4_qat kernel](#编译-int4_qat-kernel)。 ### FP8 低精度训练 @@ -329,5 +329,5 @@ bash scripts/entrypoint/ray-job.sh scripts/training/multimodal/run-kimi-k2.6-256 ::: ::: tip -这个配方假设 W4A16 发布版是用**对称量化**生成的（与训练侧 STE 对齐）。如果你从 BF16 出发用 `convert_hf_to_int4.py` 重新生成 W4A16，必须带上 `--is-symmetric` —— 详见上文 [离线量化工具](#convert_hf_to_int4-py) 的 warning。 +这个配方假设 W4A16 发布版是用**对称量化**生成的（与训练侧 STE 对齐）。如果你从 BF16 出发用 `convert_hf_to_int4.py` 重新生成 W4A16，必须带上 `--is-symmetric` —— 详见上文 [离线量化工具](#convert_hf_to_int4py) 的 warning。 ::: diff --git a/examples/on_policy_distillation/mopd/README.md b/examples/on_policy_distillation/mopd/README.md index b00400688..57c9ccfed 100644 --- a/examples/on_policy_distillation/mopd/README.md +++ b/examples/on_policy_distillation/mopd/README.md @@ -148,4 +148,4 @@ rollout 阶段按每条样本的 `data_source` 找到对应教师，请求 token ## 参考 - [On-Policy Distillation - Relax Docs](../README.md) -- [OPD 参数说明](../README.md#关键参数说明) +- [OPD 参数配置示例](../README.md#common-combinations) From 27ef3f9c0f1251106bf7cf629ea505bc2265226e Mon Sep 17 00:00:00 2001 From: ooooo <3164076421@qq.com> Date: Fri, 11 Sep 2026 22:31:42 +0800 Subject: [PATCH 3/5] docs(opd): keep link fix wording unchanged MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit # 📝 Documentation - Remove the added flag-mapping explanation from Common Combinations. - Preserve the original wording and corrected script link. ## Validation - All pre-commit hooks and staged diff checks pass. --- examples/on_policy_distillation/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/examples/on_policy_distillation/README.md b/examples/on_policy_distillation/README.md index 9d699e54d..ef4052d56 100644 --- a/examples/on_policy_distillation/README.md +++ b/examples/on_policy_distillation/README.md @@ -80,7 +80,7 @@ SGLang per-position top-K log-prob transfer optimization (Qwen3-4B, simulated da ## Common Combinations -Based on [`math_opd/run-opd-qwen35-35B-A3B-8xgpu-colocate.sh`](math_opd/run-opd-qwen35-35B-A3B-8xgpu-colocate.sh), edit the corresponding flags in `OPD_ARGS` to switch modes. The variables below are illustrative: map `OPD_KL_COEF`, `OPD_LOSS_COEF`, `OPD_TOKEN_SELECTION`, and `OPD_KL_TYPE` to `--opd-kl-coef`, `--opd-loss-coef`, `--opd-token-selection`, and `--opd-kl-type`, respectively. The script does not read these variables automatically: +Based on [`math_opd/run-opd-qwen35-35B-A3B-8xgpu-colocate.sh`](math_opd/run-opd-qwen35-35B-A3B-8xgpu-colocate.sh), modify key variables to switch modes: > **KL type note**: `student_sampled` supports only `reverse_kl` and `low_var_kl`; `student_topk`, `teacher_topk`, and `union` support `reverse_kl`, `forward_kl`, and `jsd` (set via `--opd-kl-type`; `jsd` can be tuned with `--opd-jsd-alpha`). From 4e096c44b73d4904d09106f7fda55ecee75e41e8 Mon Sep 17 00:00:00 2001 From: ooooo <3164076421@qq.com> Date: Fri, 11 Sep 2026 22:32:50 +0800 Subject: [PATCH 4/5] docs(opd): remove extra KL detail from table MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit # 📝 Documentation - Restore the original student_sampled and adv mode description. - Keep the corrected script link and updated student/teacher model names. ## Validation - All pre-commit hooks and staged diff checks pass. --- examples/on_policy_distillation/README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/examples/on_policy_distillation/README.md b/examples/on_policy_distillation/README.md index ef4052d56..4c9c32f22 100644 --- a/examples/on_policy_distillation/README.md +++ b/examples/on_policy_distillation/README.md @@ -66,9 +66,9 @@ SGLang per-position top-K log-prob transfer optimization (Qwen3-4B, simulated da ### Text OPD -| Script | Description | -| -------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- | -| [`math_opd/run-opd-qwen35-35B-A3B-8xgpu-colocate.sh`](math_opd/run-opd-qwen35-35B-A3B-8xgpu-colocate.sh) | Math OPD, student: Qwen3.5-35B-A3B; teacher: Qwen3.5-35B-A3B-GRPO-dapomath17k-400step; `student_sampled` + adv + `reverse_kl`, 8 GPU colocate | +| Script | Description | +| -------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- | +| [`math_opd/run-opd-qwen35-35B-A3B-8xgpu-colocate.sh`](math_opd/run-opd-qwen35-35B-A3B-8xgpu-colocate.sh) | Math OPD, student: Qwen3.5-35B-A3B; teacher: Qwen3.5-35B-A3B-GRPO-dapomath17k-400step; `student_sampled` + adv mode, 8 GPU colocate | ### Multimodal OPD From 0b8c0a636374a897d8bea63a24f9cc02eb97ccca Mon Sep 17 00:00:00 2001 From: ooooo <3164076421@qq.com> Date: Fri, 11 Sep 2026 22:55:57 +0800 Subject: [PATCH 5/5] docs(links): use portable section anchors MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit # 📝 Documentation - Add explicit HTML anchors for HF-to-INT4 conversion and INT4 QAT kernel builds in both languages. - Point all six section references to shared IDs while preserving heading text. - Avoid differences between GitHub Markdown and VitePress heading slug generation. ## Validation - GitHub Markdown rendering retains both anchors on each page. - VitePress 1.6.4 rendering resolves all six references with unique IDs. - Lychee offline fragment checks pass: 8 links, 0 errors. - All pre-commit hooks and staged diff checks pass. --- docs/en/examples/low-precision-training.md | 10 +++++++--- docs/zh/examples/low-precision-training.md | 10 +++++++--- 2 files changed, 14 insertions(+), 6 deletions(-) diff --git a/docs/en/examples/low-precision-training.md b/docs/en/examples/low-precision-training.md index 4e3332384..af9675deb 100644 --- a/docs/en/examples/low-precision-training.md +++ b/docs/en/examples/low-precision-training.md @@ -112,9 +112,11 @@ Output: For the FP8 training workflow you usually do **not** need this script — bridge mode (`--megatron-to-hf-mode bridge`) reads the FP8 HF directly. This tool is for offline conversion when you need a BF16 HF as input to a different pipeline (e.g. as a `--ref-load` source for another recipe, or to feed `convert_hf_to_int4.py`). ::: + + ### `convert_hf_to_int4.py` -Quantize a BF16 HF checkpoint to W4A16 (compressed-tensors). Uses the `fake_int4_quant_cuda` kernel, which must be built first (see [Build the int4_qat kernel](#build-the-int4_qat-kernel)). +Quantize a BF16 HF checkpoint to W4A16 (compressed-tensors). Uses the `fake_int4_quant_cuda` kernel, which must be built first (see [Build the int4_qat kernel](#build-int4-qat-kernel)). ```bash python scripts/tools/convert_hf_to_int4.py \ @@ -185,7 +187,7 @@ Relax ships two reference recipes for Qwen3-30B-A3B (8-GPU colocate): **FP8 nati 1. A BF16 HF checkpoint (e.g. `Qwen3-30B-A3B`). 2. The Megatron patch at `docker/patch/megatron/20260506-85bced0ae.patch` applied (baked into the project Dockerfile). It provides both the FP8 overrides and the INT4 `_FakeInt4QuantizationSTE` that overrides `TEGroupedLinear._get_weight_tensors()`. -The FP8 recipe additionally needs a TransformerEngine build with FP8 blockwise scaling support. The INT4 recipe additionally needs the `fake_int4_quant_cuda` CUDA extension built — see [Build the int4_qat kernel](#build-the-int4_qat-kernel) below. +The FP8 recipe additionally needs a TransformerEngine build with FP8 blockwise scaling support. The INT4 recipe additionally needs the `fake_int4_quant_cuda` CUDA extension built — see [Build the int4_qat kernel](#build-int4-qat-kernel) below. ### FP8 Recipe @@ -217,6 +219,8 @@ The FP8 recipe additionally needs a TransformerEngine build with FP8 blockwise s ### INT4 fake-QAT Recipe + + #### Build the int4_qat kernel ```bash @@ -331,5 +335,5 @@ Do not swap `--sglang-hf-checkpoint` to the BF16 cast for "consistency". SGLang' ::: ::: tip -This recipe assumes the W4A16 release was produced with **symmetric** quantization (matching the training-side STE). If you regenerate the W4A16 from BF16 via `convert_hf_to_int4.py`, always pass `--is-symmetric` — see the warning in the [Offline Quantization Tools](#convert_hf_to_int4py) section above. +This recipe assumes the W4A16 release was produced with **symmetric** quantization (matching the training-side STE). If you regenerate the W4A16 from BF16 via `convert_hf_to_int4.py`, always pass `--is-symmetric` — see the warning in the [Offline Quantization Tools](#convert-hf-to-int4) section above. ::: diff --git a/docs/zh/examples/low-precision-training.md b/docs/zh/examples/low-precision-training.md index bd98119d1..be60d5a2c 100644 --- a/docs/zh/examples/low-precision-training.md +++ b/docs/zh/examples/low-precision-training.md @@ -112,9 +112,11 @@ python scripts/tools/convert_fp8_to_bf16.py \ FP8 训练工作流下通常 **不需要** 这个脚本 — bridge 模式（`--megatron-to-hf-mode bridge`）会直接读取 FP8 HF。此工具用于离线转换：当你需要把 FP8 checkpoint 转回 BF16 HF 作为其他流水线的输入时（例如作为另一份配方的 `--ref-load`，或喂给 `convert_hf_to_int4.py`）。 ::: + + ### `convert_hf_to_int4.py` -把 BF16 的 HF checkpoint 量化为 W4A16（compressed-tensors）。依赖 `fake_int4_quant_cuda` kernel，需先编译（见 [编译 int4_qat kernel](#编译-int4_qat-kernel)）。 +把 BF16 的 HF checkpoint 量化为 W4A16（compressed-tensors）。依赖 `fake_int4_quant_cuda` kernel，需先编译（见 [编译 int4_qat kernel](#build-int4-qat-kernel)）。 ```bash python scripts/tools/convert_hf_to_int4.py \ @@ -185,7 +187,7 @@ Relax 在 Qwen3-30B-A3B（8 卡 colocate）上提供两条参考配方：**FP8 1. 一个 BF16 HF checkpoint（例如 `Qwen3-30B-A3B`）。 2. 应用 Megatron patch `docker/patch/megatron/20260506-85bced0ae.patch`（项目 Dockerfile 已自动应用）—— 该 patch 同时提供 FP8 配套的 override 与 INT4 假量化的 `_FakeInt4QuantizationSTE`（override 了 `TEGroupedLinear._get_weight_tensors()`）。 -FP8 配方额外需要一个支持 FP8 blockwise scaling 的 TransformerEngine 构建；INT4 配方额外需要编译 `fake_int4_quant_cuda` CUDA 扩展，见下文 [编译 int4_qat kernel](#编译-int4_qat-kernel)。 +FP8 配方额外需要一个支持 FP8 blockwise scaling 的 TransformerEngine 构建；INT4 配方额外需要编译 `fake_int4_quant_cuda` CUDA 扩展，见下文 [编译 int4_qat kernel](#build-int4-qat-kernel)。 ### FP8 低精度训练 @@ -216,6 +218,8 @@ FP8 配方额外需要一个支持 FP8 blockwise scaling 的 TransformerEngine ### INT4 低精度训练 + + #### 编译 int4_qat kernel ```bash @@ -329,5 +333,5 @@ bash scripts/entrypoint/ray-job.sh scripts/training/multimodal/run-kimi-k2.6-256 ::: ::: tip -这个配方假设 W4A16 发布版是用**对称量化**生成的（与训练侧 STE 对齐）。如果你从 BF16 出发用 `convert_hf_to_int4.py` 重新生成 W4A16，必须带上 `--is-symmetric` —— 详见上文 [离线量化工具](#convert_hf_to_int4py) 的 warning。 +这个配方假设 W4A16 发布版是用**对称量化**生成的（与训练侧 STE 对齐）。如果你从 BF16 出发用 `convert_hf_to_int4.py` 重新生成 W4A16，必须带上 `--is-symmetric` —— 详见上文 [离线量化工具](#convert-hf-to-int4) 的 warning。 :::