diff --git a/docs/en/examples/deepeyes.md b/docs/en/examples/deepeyes.md index 2c171e84b..856cfa22b 100644 --- a/docs/en/examples/deepeyes.md +++ b/docs/en/examples/deepeyes.md @@ -13,7 +13,7 @@ The DeepEyes example demonstrates **agentic multi-turn vision-language RL** usin ## Data Preparation -DeepEyes uses the [Visual-Agent/DeepEyes](https://huggingface.co/datasets/Visual-Agent/DeepEyes) dataset from HuggingFace. The dataset contains visual QA samples with images stored inline (HF Image format). +DeepEyes uses the [ChenShawn/DeepEyes-Datasets-47k](https://huggingface.co/datasets/ChenShawn/DeepEyes-Datasets-47k) dataset from HuggingFace. The dataset contains visual QA samples with images stored inline (HF Image format). ### Download the Dataset diff --git a/docs/en/examples/low-precision-training.md b/docs/en/examples/low-precision-training.md index 177dabbbd..af9675deb 100644 --- a/docs/en/examples/low-precision-training.md +++ b/docs/en/examples/low-precision-training.md @@ -112,9 +112,11 @@ Output: For the FP8 training workflow you usually do **not** need this script — bridge mode (`--megatron-to-hf-mode bridge`) reads the FP8 HF directly. This tool is for offline conversion when you need a BF16 HF as input to a different pipeline (e.g. as a `--ref-load` source for another recipe, or to feed `convert_hf_to_int4.py`). ::: + + ### `convert_hf_to_int4.py` -Quantize a BF16 HF checkpoint to W4A16 (compressed-tensors). Uses the `fake_int4_quant_cuda` kernel, which must be built first (see [Build the int4_qat kernel](#build-the-int4_qat-kernel)). +Quantize a BF16 HF checkpoint to W4A16 (compressed-tensors). Uses the `fake_int4_quant_cuda` kernel, which must be built first (see [Build the int4_qat kernel](#build-int4-qat-kernel)). ```bash python scripts/tools/convert_hf_to_int4.py \ @@ -185,7 +187,7 @@ Relax ships two reference recipes for Qwen3-30B-A3B (8-GPU colocate): **FP8 nati 1. A BF16 HF checkpoint (e.g. `Qwen3-30B-A3B`). 2. The Megatron patch at `docker/patch/megatron/20260506-85bced0ae.patch` applied (baked into the project Dockerfile). It provides both the FP8 overrides and the INT4 `_FakeInt4QuantizationSTE` that overrides `TEGroupedLinear._get_weight_tensors()`. -The FP8 recipe additionally needs a TransformerEngine build with FP8 blockwise scaling support. The INT4 recipe additionally needs the `fake_int4_quant_cuda` CUDA extension built — see [Build the int4_qat kernel](#build-the-int4-qat-kernel) below. +The FP8 recipe additionally needs a TransformerEngine build with FP8 blockwise scaling support. The INT4 recipe additionally needs the `fake_int4_quant_cuda` CUDA extension built — see [Build the int4_qat kernel](#build-int4-qat-kernel) below. ### FP8 Recipe @@ -217,6 +219,8 @@ The FP8 recipe additionally needs a TransformerEngine build with FP8 blockwise s ### INT4 fake-QAT Recipe + + #### Build the int4_qat kernel ```bash @@ -331,5 +335,5 @@ Do not swap `--sglang-hf-checkpoint` to the BF16 cast for "consistency". SGLang' ::: ::: tip -This recipe assumes the W4A16 release was produced with **symmetric** quantization (matching the training-side STE). If you regenerate the W4A16 from BF16 via `convert_hf_to_int4.py`, always pass `--is-symmetric` — see the warning in the [Offline Quantization Tools](#convert_hf_to_int4-py) section above. +This recipe assumes the W4A16 release was produced with **symmetric** quantization (matching the training-side STE). If you regenerate the W4A16 from BF16 via `convert_hf_to_int4.py`, always pass `--is-symmetric` — see the warning in the [Offline Quantization Tools](#convert-hf-to-int4) section above. ::: diff --git a/docs/en/guide/autoscaler-k8s-keda.md b/docs/en/guide/autoscaler-k8s-keda.md index 2ad5583b6..9a4b2c105 100644 --- a/docs/en/guide/autoscaler-k8s-keda.md +++ b/docs/en/guide/autoscaler-k8s-keda.md @@ -450,7 +450,7 @@ ______________________________________________________________________ - 引擎会使用初始模型权重运行 - 后续权重更新在 Actor 的 `update_weights_fully_async()` 完成后自动触发 -- 如果对权重一致性要求严格，参考 [弹性 Rollout 扩缩容文档](../zh/guide/elastic-rollout.md) 中的权重同步机制 +- 如果对权重一致性要求严格，参考 [弹性 Rollout 扩缩容文档](./elastic-rollout.md) 中的权重同步机制 ______________________________________________________________________ @@ -549,7 +549,7 @@ ______________________________________________________________________ ## 延伸阅读 -- [弹性 Rollout 扩缩容](../zh/guide/elastic-rollout.md) — Relax 弹性扩缩容完整文档 -- [全异步训练流水线](../zh/guide/fully-async-training.md) — 弹性扩缩容的基础运行模式 +- [弹性 Rollout 扩缩容](./elastic-rollout.md) — Relax 弹性扩缩容完整文档 +- [全异步训练流水线](./fully-async-training.md) — 弹性扩缩容的基础运行模式 - [KEDA 官方文档](https://keda.sh/docs/) — KEDA ScaledObject 配置参考 - [Prometheus Operator](https://prometheus-operator.dev/) — ServiceMonitor 配置参考 diff --git a/docs/en/guide/customize-training.md b/docs/en/guide/customize-training.md index 87733ade7..41650d7f0 100644 --- a/docs/en/guide/customize-training.md +++ b/docs/en/guide/customize-training.md @@ -18,7 +18,7 @@ hf download Qwen/Qwen3-VL-4B-Instruct --local-dir /root/Qwen3-VL-4B-Instruct ### Megatron Weights to HF Weights ::: tip No Manual Conversion Needed with Megatron Bridge -Relax uses [Megatron Bridge](../../../relax/backends/megatron/mbridge/) as the weight bridging layer for its training backend, automatically handling bidirectional HF ↔ Megatron weight conversion during training — **no manual conversion steps required**. Simply specify the following option in your launch script: +Relax uses [Megatron Bridge](https://github.com/NVIDIA-NeMo/Megatron-Bridge) as the weight bridging layer for its training backend, automatically handling bidirectional HF ↔ Megatron weight conversion during training — **no manual conversion steps required**. Simply specify the following option in your launch script: ::: ```bash diff --git a/docs/zh/examples/deepeyes.md b/docs/zh/examples/deepeyes.md index bc735b071..7d8cc7fe0 100644 --- a/docs/zh/examples/deepeyes.md +++ b/docs/zh/examples/deepeyes.md @@ -13,7 +13,7 @@ DeepEyes 示例演示了使用 Relax 进行 **Agent 式多轮视觉语言强化 ## 数据准备 -DeepEyes 使用 HuggingFace 上的 [Visual-Agent/DeepEyes](https://huggingface.co/datasets/Visual-Agent/DeepEyes) 数据集。数据集包含视觉问答样本，图像以 HF Image 格式内嵌存储。 +DeepEyes 使用 HuggingFace 上的 [ChenShawn/DeepEyes-Datasets-47k](https://huggingface.co/datasets/ChenShawn/DeepEyes-Datasets-47k) 数据集。数据集包含视觉问答样本，图像以 HF Image 格式内嵌存储。 ### 下载数据集 diff --git a/docs/zh/examples/low-precision-training.md b/docs/zh/examples/low-precision-training.md index efb9e3fa5..be60d5a2c 100644 --- a/docs/zh/examples/low-precision-training.md +++ b/docs/zh/examples/low-precision-training.md @@ -112,9 +112,11 @@ python scripts/tools/convert_fp8_to_bf16.py \ FP8 训练工作流下通常 **不需要** 这个脚本 — bridge 模式（`--megatron-to-hf-mode bridge`）会直接读取 FP8 HF。此工具用于离线转换：当你需要把 FP8 checkpoint 转回 BF16 HF 作为其他流水线的输入时（例如作为另一份配方的 `--ref-load`，或喂给 `convert_hf_to_int4.py`）。 ::: + + ### `convert_hf_to_int4.py` -把 BF16 的 HF checkpoint 量化为 W4A16（compressed-tensors）。依赖 `fake_int4_quant_cuda` kernel，需先编译（见 [编译 int4_qat kernel](#编译-int4-qat-kernel)）。 +把 BF16 的 HF checkpoint 量化为 W4A16（compressed-tensors）。依赖 `fake_int4_quant_cuda` kernel，需先编译（见 [编译 int4_qat kernel](#build-int4-qat-kernel)）。 ```bash python scripts/tools/convert_hf_to_int4.py \ @@ -185,7 +187,7 @@ Relax 在 Qwen3-30B-A3B（8 卡 colocate）上提供两条参考配方：**FP8 1. 一个 BF16 HF checkpoint（例如 `Qwen3-30B-A3B`）。 2. 应用 Megatron patch `docker/patch/megatron/20260506-85bced0ae.patch`（项目 Dockerfile 已自动应用）—— 该 patch 同时提供 FP8 配套的 override 与 INT4 假量化的 `_FakeInt4QuantizationSTE`（override 了 `TEGroupedLinear._get_weight_tensors()`）。 -FP8 配方额外需要一个支持 FP8 blockwise scaling 的 TransformerEngine 构建；INT4 配方额外需要编译 `fake_int4_quant_cuda` CUDA 扩展，见下文 [编译 int4_qat kernel](#编译-int4-qat-kernel)。 +FP8 配方额外需要一个支持 FP8 blockwise scaling 的 TransformerEngine 构建；INT4 配方额外需要编译 `fake_int4_quant_cuda` CUDA 扩展，见下文 [编译 int4_qat kernel](#build-int4-qat-kernel)。 ### FP8 低精度训练 @@ -216,6 +218,8 @@ FP8 配方额外需要一个支持 FP8 blockwise scaling 的 TransformerEngine ### INT4 低精度训练 + + #### 编译 int4_qat kernel ```bash @@ -329,5 +333,5 @@ bash scripts/entrypoint/ray-job.sh scripts/training/multimodal/run-kimi-k2.6-256 ::: ::: tip -这个配方假设 W4A16 发布版是用**对称量化**生成的（与训练侧 STE 对齐）。如果你从 BF16 出发用 `convert_hf_to_int4.py` 重新生成 W4A16，必须带上 `--is-symmetric` —— 详见上文 [离线量化工具](#convert_hf_to_int4-py) 的 warning。 +这个配方假设 W4A16 发布版是用**对称量化**生成的（与训练侧 STE 对齐）。如果你从 BF16 出发用 `convert_hf_to_int4.py` 重新生成 W4A16，必须带上 `--is-symmetric` —— 详见上文 [离线量化工具](#convert-hf-to-int4) 的 warning。 ::: diff --git a/docs/zh/guide/autoscaler-k8s-keda.md b/docs/zh/guide/autoscaler-k8s-keda.md index 2ad5583b6..9a4b2c105 100644 --- a/docs/zh/guide/autoscaler-k8s-keda.md +++ b/docs/zh/guide/autoscaler-k8s-keda.md @@ -450,7 +450,7 @@ ______________________________________________________________________ - 引擎会使用初始模型权重运行 - 后续权重更新在 Actor 的 `update_weights_fully_async()` 完成后自动触发 -- 如果对权重一致性要求严格，参考 [弹性 Rollout 扩缩容文档](../zh/guide/elastic-rollout.md) 中的权重同步机制 +- 如果对权重一致性要求严格，参考 [弹性 Rollout 扩缩容文档](./elastic-rollout.md) 中的权重同步机制 ______________________________________________________________________ @@ -549,7 +549,7 @@ ______________________________________________________________________ ## 延伸阅读 -- [弹性 Rollout 扩缩容](../zh/guide/elastic-rollout.md) — Relax 弹性扩缩容完整文档 -- [全异步训练流水线](../zh/guide/fully-async-training.md) — 弹性扩缩容的基础运行模式 +- [弹性 Rollout 扩缩容](./elastic-rollout.md) — Relax 弹性扩缩容完整文档 +- [全异步训练流水线](./fully-async-training.md) — 弹性扩缩容的基础运行模式 - [KEDA 官方文档](https://keda.sh/docs/) — KEDA ScaledObject 配置参考 - [Prometheus Operator](https://prometheus-operator.dev/) — ServiceMonitor 配置参考 diff --git a/docs/zh/guide/customize-training.md b/docs/zh/guide/customize-training.md index d7dc0b19d..e839209bb 100644 --- a/docs/zh/guide/customize-training.md +++ b/docs/zh/guide/customize-training.md @@ -18,7 +18,7 @@ hf download Qwen/Qwen3-VL-4B-Instruct --local-dir /root/Qwen3-VL-4B-Instruct ### Megatron 权重转 HF 权重 ::: tip 使用 Megatron Bridge 无需手动转换 -Relax 默认使用 [Megatron Bridge](../../../relax/backends/megatron/mbridge/) 作为训练后端的权重桥接层，在训练过程中自动完成 HF ↔ Megatron 权重的双向转换，**无需任何手动转换步骤**。只需在启动脚本中指定以下选项即可： +Relax 默认使用 [Megatron Bridge](https://github.com/NVIDIA-NeMo/Megatron-Bridge) 作为训练后端的权重桥接层，在训练过程中自动完成 HF ↔ Megatron 权重的双向转换，**无需任何手动转换步骤**。只需在启动脚本中指定以下选项即可： ::: ```bash diff --git a/examples/on_policy_distillation/README.md b/examples/on_policy_distillation/README.md index 8ec2a0172..4c9c32f22 100644 --- a/examples/on_policy_distillation/README.md +++ b/examples/on_policy_distillation/README.md @@ -66,9 +66,9 @@ SGLang per-position top-K log-prob transfer optimization (Qwen3-4B, simulated da ### Text OPD -| Script | Description | -| ------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------- | -| [`math_opd/run-opd-sampled-8xgpu-colocate.sh`](math_opd/run-opd-sampled-8xgpu-colocate.sh) | Math OPD, Qwen3-4B → Qwen3-4B-Non-Thinking-RL, `student_sampled` + adv mode, 8 GPU colocate | +| Script | Description | +| -------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- | +| [`math_opd/run-opd-qwen35-35B-A3B-8xgpu-colocate.sh`](math_opd/run-opd-qwen35-35B-A3B-8xgpu-colocate.sh) | Math OPD, student: Qwen3.5-35B-A3B; teacher: Qwen3.5-35B-A3B-GRPO-dapomath17k-400step; `student_sampled` + adv mode, 8 GPU colocate | ### Multimodal OPD @@ -80,7 +80,7 @@ SGLang per-position top-K log-prob transfer optimization (Qwen3-4B, simulated da ## Common Combinations -Based on [`math_opd/run-opd-sampled-8xgpu-colocate.sh`](math_opd/run-opd-sampled-8xgpu-colocate.sh), modify key variables to switch modes: +Based on [`math_opd/run-opd-qwen35-35B-A3B-8xgpu-colocate.sh`](math_opd/run-opd-qwen35-35B-A3B-8xgpu-colocate.sh), modify key variables to switch modes: > **KL type note**: `student_sampled` supports only `reverse_kl` and `low_var_kl`; `student_topk`, `teacher_topk`, and `union` support `reverse_kl`, `forward_kl`, and `jsd` (set via `--opd-kl-type`; `jsd` can be tuned with `--opd-jsd-alpha`). diff --git a/examples/on_policy_distillation/mopd/README.md b/examples/on_policy_distillation/mopd/README.md index b00400688..57c9ccfed 100644 --- a/examples/on_policy_distillation/mopd/README.md +++ b/examples/on_policy_distillation/mopd/README.md @@ -148,4 +148,4 @@ rollout 阶段按每条样本的 `data_source` 找到对应教师，请求 token ## 参考 - [On-Policy Distillation - Relax Docs](../README.md) -- [OPD 参数说明](../README.md#关键参数说明) +- [OPD 参数配置示例](../README.md#common-combinations)