Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/en/examples/deepeyes.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ The DeepEyes example demonstrates **agentic multi-turn vision-language RL** usin

## Data Preparation

DeepEyes uses the [Visual-Agent/DeepEyes](https://huggingface.co/datasets/Visual-Agent/DeepEyes) dataset from HuggingFace. The dataset contains visual QA samples with images stored inline (HF Image format).
DeepEyes uses the [ChenShawn/DeepEyes-Datasets-47k](https://huggingface.co/datasets/ChenShawn/DeepEyes-Datasets-47k) dataset from HuggingFace. The dataset contains visual QA samples with images stored inline (HF Image format).

### Download the Dataset

Expand Down
10 changes: 7 additions & 3 deletions docs/en/examples/low-precision-training.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,9 +112,11 @@ Output:
For the FP8 training workflow you usually do **not** need this script — bridge mode (`--megatron-to-hf-mode bridge`) reads the FP8 HF directly. This tool is for offline conversion when you need a BF16 HF as input to a different pipeline (e.g. as a `--ref-load` source for another recipe, or to feed `convert_hf_to_int4.py`).
:::

<a id="convert-hf-to-int4"></a>

### `convert_hf_to_int4.py`

Quantize a BF16 HF checkpoint to W4A16 (compressed-tensors). Uses the `fake_int4_quant_cuda` kernel, which must be built first (see [Build the int4_qat kernel](#build-the-int4_qat-kernel)).
Quantize a BF16 HF checkpoint to W4A16 (compressed-tensors). Uses the `fake_int4_quant_cuda` kernel, which must be built first (see [Build the int4_qat kernel](#build-int4-qat-kernel)).

```bash
python scripts/tools/convert_hf_to_int4.py \
Expand Down Expand Up @@ -185,7 +187,7 @@ Relax ships two reference recipes for Qwen3-30B-A3B (8-GPU colocate): **FP8 nati
1. A BF16 HF checkpoint (e.g. `Qwen3-30B-A3B`).
2. The Megatron patch at `docker/patch/megatron/20260506-85bced0ae.patch` applied (baked into the project Dockerfile). It provides both the FP8 overrides and the INT4 `_FakeInt4QuantizationSTE` that overrides `TEGroupedLinear._get_weight_tensors()`.

The FP8 recipe additionally needs a TransformerEngine build with FP8 blockwise scaling support. The INT4 recipe additionally needs the `fake_int4_quant_cuda` CUDA extension built — see [Build the int4_qat kernel](#build-the-int4-qat-kernel) below.
The FP8 recipe additionally needs a TransformerEngine build with FP8 blockwise scaling support. The INT4 recipe additionally needs the `fake_int4_quant_cuda` CUDA extension built — see [Build the int4_qat kernel](#build-int4-qat-kernel) below.

### FP8 Recipe

Expand Down Expand Up @@ -217,6 +219,8 @@ The FP8 recipe additionally needs a TransformerEngine build with FP8 blockwise s

### INT4 fake-QAT Recipe

<a id="build-int4-qat-kernel"></a>

#### Build the int4_qat kernel

```bash
Expand Down Expand Up @@ -331,5 +335,5 @@ Do not swap `--sglang-hf-checkpoint` to the BF16 cast for "consistency". SGLang'
:::

::: tip
This recipe assumes the W4A16 release was produced with **symmetric** quantization (matching the training-side STE). If you regenerate the W4A16 from BF16 via `convert_hf_to_int4.py`, always pass `--is-symmetric` — see the warning in the [Offline Quantization Tools](#convert_hf_to_int4-py) section above.
This recipe assumes the W4A16 release was produced with **symmetric** quantization (matching the training-side STE). If you regenerate the W4A16 from BF16 via `convert_hf_to_int4.py`, always pass `--is-symmetric` — see the warning in the [Offline Quantization Tools](#convert-hf-to-int4) section above.
:::
6 changes: 3 additions & 3 deletions docs/en/guide/autoscaler-k8s-keda.md
Original file line number Diff line number Diff line change
Expand Up @@ -450,7 +450,7 @@ ______________________________________________________________________

- 引擎会使用初始模型权重运行
- 后续权重更新在 Actor 的 `update_weights_fully_async()` 完成后自动触发
- 如果对权重一致性要求严格,参考 [弹性 Rollout 扩缩容文档](../zh/guide/elastic-rollout.md) 中的权重同步机制
- 如果对权重一致性要求严格,参考 [弹性 Rollout 扩缩容文档](./elastic-rollout.md) 中的权重同步机制

______________________________________________________________________

Expand Down Expand Up @@ -549,7 +549,7 @@ ______________________________________________________________________

## 延伸阅读

- [弹性 Rollout 扩缩容](../zh/guide/elastic-rollout.md) — Relax 弹性扩缩容完整文档
- [全异步训练流水线](../zh/guide/fully-async-training.md) — 弹性扩缩容的基础运行模式
- [弹性 Rollout 扩缩容](./elastic-rollout.md) — Relax 弹性扩缩容完整文档
- [全异步训练流水线](./fully-async-training.md) — 弹性扩缩容的基础运行模式
- [KEDA 官方文档](https://keda.sh/docs/) — KEDA ScaledObject 配置参考
- [Prometheus Operator](https://prometheus-operator.dev/) — ServiceMonitor 配置参考
2 changes: 1 addition & 1 deletion docs/en/guide/customize-training.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ hf download Qwen/Qwen3-VL-4B-Instruct --local-dir /root/Qwen3-VL-4B-Instruct
### Megatron Weights to HF Weights

::: tip No Manual Conversion Needed with Megatron Bridge
Relax uses [Megatron Bridge](../../../relax/backends/megatron/mbridge/) as the weight bridging layer for its training backend, automatically handling bidirectional HF ↔ Megatron weight conversion during training — **no manual conversion steps required**. Simply specify the following option in your launch script:
Relax uses [Megatron Bridge](https://github.com/NVIDIA-NeMo/Megatron-Bridge) as the weight bridging layer for its training backend, automatically handling bidirectional HF ↔ Megatron weight conversion during training — **no manual conversion steps required**. Simply specify the following option in your launch script:
:::

```bash
Expand Down
2 changes: 1 addition & 1 deletion docs/zh/examples/deepeyes.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ DeepEyes 示例演示了使用 Relax 进行 **Agent 式多轮视觉语言强化

## 数据准备

DeepEyes 使用 HuggingFace 上的 [Visual-Agent/DeepEyes](https://huggingface.co/datasets/Visual-Agent/DeepEyes) 数据集。数据集包含视觉问答样本,图像以 HF Image 格式内嵌存储。
DeepEyes 使用 HuggingFace 上的 [ChenShawn/DeepEyes-Datasets-47k](https://huggingface.co/datasets/ChenShawn/DeepEyes-Datasets-47k) 数据集。数据集包含视觉问答样本,图像以 HF Image 格式内嵌存储。

### 下载数据集

Expand Down
10 changes: 7 additions & 3 deletions docs/zh/examples/low-precision-training.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,9 +112,11 @@ python scripts/tools/convert_fp8_to_bf16.py \
FP8 训练工作流下通常 **不需要** 这个脚本 — bridge 模式(`--megatron-to-hf-mode bridge`)会直接读取 FP8 HF。此工具用于离线转换:当你需要把 FP8 checkpoint 转回 BF16 HF 作为其他流水线的输入时(例如作为另一份配方的 `--ref-load`,或喂给 `convert_hf_to_int4.py`)。
:::

<a id="convert-hf-to-int4"></a>

### `convert_hf_to_int4.py`

把 BF16 的 HF checkpoint 量化为 W4A16(compressed-tensors)。依赖 `fake_int4_quant_cuda` kernel,需先编译(见 [编译 int4_qat kernel](#编译-int4-qat-kernel))。
把 BF16 的 HF checkpoint 量化为 W4A16(compressed-tensors)。依赖 `fake_int4_quant_cuda` kernel,需先编译(见 [编译 int4_qat kernel](#build-int4-qat-kernel))。

```bash
python scripts/tools/convert_hf_to_int4.py \
Expand Down Expand Up @@ -185,7 +187,7 @@ Relax 在 Qwen3-30B-A3B(8 卡 colocate)上提供两条参考配方:**FP8
1. 一个 BF16 HF checkpoint(例如 `Qwen3-30B-A3B`)。
2. 应用 Megatron patch `docker/patch/megatron/20260506-85bced0ae.patch`(项目 Dockerfile 已自动应用)—— 该 patch 同时提供 FP8 配套的 override 与 INT4 假量化的 `_FakeInt4QuantizationSTE`(override 了 `TEGroupedLinear._get_weight_tensors()`)。

FP8 配方额外需要一个支持 FP8 blockwise scaling 的 TransformerEngine 构建;INT4 配方额外需要编译 `fake_int4_quant_cuda` CUDA 扩展,见下文 [编译 int4_qat kernel](#编译-int4-qat-kernel)。
FP8 配方额外需要一个支持 FP8 blockwise scaling 的 TransformerEngine 构建;INT4 配方额外需要编译 `fake_int4_quant_cuda` CUDA 扩展,见下文 [编译 int4_qat kernel](#build-int4-qat-kernel)。

### FP8 低精度训练

Expand Down Expand Up @@ -216,6 +218,8 @@ FP8 配方额外需要一个支持 FP8 blockwise scaling 的 TransformerEngine

### INT4 低精度训练

<a id="build-int4-qat-kernel"></a>

#### 编译 int4_qat kernel

```bash
Expand Down Expand Up @@ -329,5 +333,5 @@ bash scripts/entrypoint/ray-job.sh scripts/training/multimodal/run-kimi-k2.6-256
:::

::: tip
这个配方假设 W4A16 发布版是用**对称量化**生成的(与训练侧 STE 对齐)。如果你从 BF16 出发用 `convert_hf_to_int4.py` 重新生成 W4A16,必须带上 `--is-symmetric` —— 详见上文 [离线量化工具](#convert_hf_to_int4-py) 的 warning。
这个配方假设 W4A16 发布版是用**对称量化**生成的(与训练侧 STE 对齐)。如果你从 BF16 出发用 `convert_hf_to_int4.py` 重新生成 W4A16,必须带上 `--is-symmetric` —— 详见上文 [离线量化工具](#convert-hf-to-int4) 的 warning。
:::
6 changes: 3 additions & 3 deletions docs/zh/guide/autoscaler-k8s-keda.md
Original file line number Diff line number Diff line change
Expand Up @@ -450,7 +450,7 @@ ______________________________________________________________________

- 引擎会使用初始模型权重运行
- 后续权重更新在 Actor 的 `update_weights_fully_async()` 完成后自动触发
- 如果对权重一致性要求严格,参考 [弹性 Rollout 扩缩容文档](../zh/guide/elastic-rollout.md) 中的权重同步机制
- 如果对权重一致性要求严格,参考 [弹性 Rollout 扩缩容文档](./elastic-rollout.md) 中的权重同步机制

______________________________________________________________________

Expand Down Expand Up @@ -549,7 +549,7 @@ ______________________________________________________________________

## 延伸阅读

- [弹性 Rollout 扩缩容](../zh/guide/elastic-rollout.md) — Relax 弹性扩缩容完整文档
- [全异步训练流水线](../zh/guide/fully-async-training.md) — 弹性扩缩容的基础运行模式
- [弹性 Rollout 扩缩容](./elastic-rollout.md) — Relax 弹性扩缩容完整文档
- [全异步训练流水线](./fully-async-training.md) — 弹性扩缩容的基础运行模式
- [KEDA 官方文档](https://keda.sh/docs/) — KEDA ScaledObject 配置参考
- [Prometheus Operator](https://prometheus-operator.dev/) — ServiceMonitor 配置参考
2 changes: 1 addition & 1 deletion docs/zh/guide/customize-training.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ hf download Qwen/Qwen3-VL-4B-Instruct --local-dir /root/Qwen3-VL-4B-Instruct
### Megatron 权重转 HF 权重

::: tip 使用 Megatron Bridge 无需手动转换
Relax 默认使用 [Megatron Bridge](../../../relax/backends/megatron/mbridge/) 作为训练后端的权重桥接层,在训练过程中自动完成 HF ↔ Megatron 权重的双向转换,**无需任何手动转换步骤**。只需在启动脚本中指定以下选项即可:
Relax 默认使用 [Megatron Bridge](https://github.com/NVIDIA-NeMo/Megatron-Bridge) 作为训练后端的权重桥接层,在训练过程中自动完成 HF ↔ Megatron 权重的双向转换,**无需任何手动转换步骤**。只需在启动脚本中指定以下选项即可:
:::

```bash
Expand Down
8 changes: 4 additions & 4 deletions examples/on_policy_distillation/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,9 +66,9 @@ SGLang per-position top-K log-prob transfer optimization (Qwen3-4B, simulated da

### Text OPD

| Script | Description |
| ------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------- |
| [`math_opd/run-opd-sampled-8xgpu-colocate.sh`](math_opd/run-opd-sampled-8xgpu-colocate.sh) | Math OPD, Qwen3-4B → Qwen3-4B-Non-Thinking-RL, `student_sampled` + adv mode, 8 GPU colocate |
| Script | Description |
| -------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- |
| [`math_opd/run-opd-qwen35-35B-A3B-8xgpu-colocate.sh`](math_opd/run-opd-qwen35-35B-A3B-8xgpu-colocate.sh) | Math OPD, student: Qwen3.5-35B-A3B; teacher: Qwen3.5-35B-A3B-GRPO-dapomath17k-400step; `student_sampled` + adv mode, 8 GPU colocate |

### Multimodal OPD

Expand All @@ -80,7 +80,7 @@ SGLang per-position top-K log-prob transfer optimization (Qwen3-4B, simulated da

## Common Combinations

Based on [`math_opd/run-opd-sampled-8xgpu-colocate.sh`](math_opd/run-opd-sampled-8xgpu-colocate.sh), modify key variables to switch modes:
Based on [`math_opd/run-opd-qwen35-35B-A3B-8xgpu-colocate.sh`](math_opd/run-opd-qwen35-35B-A3B-8xgpu-colocate.sh), modify key variables to switch modes:

> **KL type note**: `student_sampled` supports only `reverse_kl` and `low_var_kl`; `student_topk`, `teacher_topk`, and `union` support `reverse_kl`, `forward_kl`, and `jsd` (set via `--opd-kl-type`; `jsd` can be tuned with `--opd-jsd-alpha`).

Expand Down
2 changes: 1 addition & 1 deletion examples/on_policy_distillation/mopd/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -148,4 +148,4 @@ rollout 阶段按每条样本的 `data_source` 找到对应教师,请求 token
## 参考

- [On-Policy Distillation - Relax Docs](../README.md)
- [OPD 参数说明](../README.md#关键参数说明)
- [OPD 参数配置示例](../README.md#common-combinations)