Skip to content

docs(links): fix Relax repository URLs #310

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
SigureMo merged 1 commit into from
Sep 10, 2026
+6 −6
Merged

docs(links): fix Relax repository URLs #310

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions docs/en/examples/generative-reward-model.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -98,15 +98,15 @@ Under `--colocate`, how Rollout and GenRM cohabit the shared bundles has three f
└─────────────────────────────────────────────────┘
```

All three colocate sub-modes reclaim every GPU for the Actor during training. Rollout produces candidate responses and (for Split and Shared / Co-resident) sends each one over HTTP to GenRM inline. In Shared / Defer-swap the HTTP call is batched once per rollout step from a userland `custom_reward_post_process` function; see [`examples/generate_reward_model/README.md`](https://github.com/xhs-tech/Relax/blob/main/examples/generate_reward_model/README.md) for the split-vs-defer trade-off matrix.
All three colocate sub-modes reclaim every GPU for the Actor during training. Rollout produces candidate responses and (for Split and Shared / Co-resident) sends each one over HTTP to GenRM inline. In Shared / Defer-swap the HTTP call is batched once per rollout step from a userland `custom_reward_post_process` function; see [`examples/generate_reward_model/README.md`](https://github.com/redai-studio/Relax/blob/main/examples/generate_reward_model/README.md) for the split-vs-defer trade-off matrix.

## Scripts

| Script | Colocate sub-mode | Description |
| :---------------------------------------------- | :------------------------- | :----------------------------------------------------------------------------- |
| `run-qwen3-4B-8xgpu-colocated.sh` | Split (small GenRM) | Qwen3-4B policy + small GenRM on 8 GPU; disjoint bundles, inline reward |
| `run-qwen35-35B-A3B-16xgpu-genrm-397B-split.sh` | Split (large GenRM) | 35B-A3B policy + 397B FP8 GenRM on 16 GPU; 8+8 disjoint shards, inline reward |
| `run-qwen35-35B-A3B-16xgpu-genrm-397B-defer.sh` | Shared / Defer-swap | 35B-A3B policy + 397B FP8 GenRM on 16 GPU; shared bundles, two-phase sleep-wake swap, batched reward via [`post_process_genrm_swap.py`](https://github.com/xhs-tech/Relax/blob/main/examples/generate_reward_model/post_process_genrm_swap.py) |
| `run-qwen35-35B-A3B-16xgpu-genrm-397B-defer.sh` | Shared / Defer-swap | 35B-A3B policy + 397B FP8 GenRM on 16 GPU; shared bundles, two-phase sleep-wake swap, batched reward via [`post_process_genrm_swap.py`](https://github.com/redai-studio/Relax/blob/main/examples/generate_reward_model/post_process_genrm_swap.py) |
| `run-qwen3-4B-8xgpu-async.sh` | (Fully Async) | Independent GPU pools per role; rollout & training fully overlapped |

### Resource Layout
Expand Down Expand Up @@ -268,7 +268,7 @@ On `--colocate` with GenRM, the GPU layout picks Split vs Shared automatically:
| `rollout_num_gpus == genrm_num_gpus == actor_total` | **Shared** (same bundles) |
| Anything else | Rejected at startup with a clear error |

Within Shared, the default is **Co-resident** (both engines held via `mem_fraction_static` split). Adding `--rm-type dummy` + `--defer-reward-to-post-process` + `--custom-reward-post-process-path` switches it to **Defer-swap** — sequenced sleep-wake, one engine holds full memory at a time. See the [example README](https://github.com/xhs-tech/Relax/blob/main/examples/generate_reward_model/README.md) for when to prefer defer-swap.
Within Shared, the default is **Co-resident** (both engines held via `mem_fraction_static` split). Adding `--rm-type dummy` + `--defer-reward-to-post-process` + `--custom-reward-post-process-path` switches it to **Defer-swap** — sequenced sleep-wake, one engine holds full memory at a time. See the [example README](https://github.com/redai-studio/Relax/blob/main/examples/generate_reward_model/README.md) for when to prefer defer-swap.
:::

::: warning Set `mem_fraction_static` in Shared / Co-resident
Expand Down
6 changes: 3 additions & 3 deletions docs/zh/examples/generative-reward-model.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -98,15 +98,15 @@ Relax 中 GenRM 有两种顶层部署模式：
└─────────────────────────────────────────────────┘
```

三种 colocate 子模式在训练阶段都把全部 GPU 归还给 Actor。Split 与 Shared / Co-resident 走 inline reward：Rollout 每生成一个候选就通过 HTTP 单发给 GenRM。Shared / Defer-swap 把 HTTP 调用改成每轮 rollout 后由 userland `custom_reward_post_process` 一次性批量发出；split 与 defer-swap 的完整取舍见 [`examples/generate_reward_model/README.md`](https://github.com/xhs-tech/Relax/blob/main/examples/generate_reward_model/README.md)。
三种 colocate 子模式在训练阶段都把全部 GPU 归还给 Actor。Split 与 Shared / Co-resident 走 inline reward：Rollout 每生成一个候选就通过 HTTP 单发给 GenRM。Shared / Defer-swap 把 HTTP 调用改成每轮 rollout 后由 userland `custom_reward_post_process` 一次性批量发出；split 与 defer-swap 的完整取舍见 [`examples/generate_reward_model/README.md`](https://github.com/redai-studio/Relax/blob/main/examples/generate_reward_model/README.md)。

## 脚本

| 脚本 | Colocate 子模式 | 描述 |
| :---------------------------------------------- | :---------------------- | :-------------------------------------------------------------------------------- |
| `run-qwen3-4B-8xgpu-colocated.sh` | Split（小 GenRM） | Qwen3-4B policy + 小 GenRM 共 8 GPU；不相交 bundle，inline reward |
| `run-qwen35-35B-A3B-16xgpu-genrm-397B-split.sh` | Split（大 GenRM） | 35B-A3B policy + 397B FP8 GenRM 共 16 GPU；8+8 不相交分片，inline reward |
| `run-qwen35-35B-A3B-16xgpu-genrm-397B-defer.sh` | Shared / Defer-swap | 35B-A3B policy + 397B FP8 GenRM 共 16 GPU；共享 bundle，两阶段 sleep-wake 切换，批量 reward，实现见 [`post_process_genrm_swap.py`](https://github.com/xhs-tech/Relax/blob/main/examples/generate_reward_model/post_process_genrm_swap.py) |
| `run-qwen35-35B-A3B-16xgpu-genrm-397B-defer.sh` | Shared / Defer-swap | 35B-A3B policy + 397B FP8 GenRM 共 16 GPU；共享 bundle，两阶段 sleep-wake 切换，批量 reward，实现见 [`post_process_genrm_swap.py`](https://github.com/redai-studio/Relax/blob/main/examples/generate_reward_model/post_process_genrm_swap.py) |
| `run-qwen3-4B-8xgpu-async.sh` | （Fully Async） | 每个角色独占 GPU 池；rollout 与训练完全并行 |

### 资源分配
Expand Down Expand Up @@ -267,7 +267,7 @@ python3 relax/entrypoints/train.py \
| `rollout_num_gpus == genrm_num_gpus == actor_total` | **Shared**（同一批 bundle） |
| 其他 | 启动时报错拒绝 |

Shared 内部默认是 **Co-resident**（两个引擎按 `mem_fraction_static` 同时驻留）。再加上 `--rm-type dummy` + `--defer-reward-to-post-process` + `--custom-reward-post-process-path` 就切成 **Defer-swap**——sleep-wake 串行，每次只有一个引擎占显存。何时优先 defer-swap 见 [示例 README](https://github.com/xhs-tech/Relax/blob/main/examples/generate_reward_model/README.md)。
Shared 内部默认是 **Co-resident**（两个引擎按 `mem_fraction_static` 同时驻留）。再加上 `--rm-type dummy` + `--defer-reward-to-post-process` + `--custom-reward-post-process-path` 就切成 **Defer-swap**——sleep-wake 串行，每次只有一个引擎占显存。何时优先 defer-swap 见 [示例 README](https://github.com/redai-studio/Relax/blob/main/examples/generate_reward_model/README.md)。
:::

::: warning Shared / Co-resident 必须设置 `mem_fraction_static`
Expand Down