feat(opd): add optional advantage clipping

[BobbyZhouZijian]

Motivation

Sampled OPD uses the teacher/student log-probability difference directly as its gradient weight. Large differences can dominate an update. Add an optional fixed bound, following the clipping used by Reef's OpenClawRL OPD branch.

Changes

• Add opd-advantage-clip through the shared Slime distillation settings. A positive value bounds the sampled reverse-KL advantage symmetrically before importance-sampling and sample weights.
• Apply the same bound to score centering. Keep the reported divergence unclipped.
• Reject negative or non-finite bounds and unsupported divergence representations. Test flag propagation, signed gradient bounds, and centering with importance sampling, masks, and sample weights.

Documentation impact

Readers need to know how to enable clipping and how it changes training. Update recipes/opd/README.md and docs/user-guide/recipes/opd.rst with the default, supported representation, and a 1.0 example. Review both sections in context. See Verification for checks; the GPU training example was not executed.

Compatibility and operational impact

The new option defaults to 0 (disabled), preserving existing behavior. Other distillation families inherit the optional flag under their own prefix. Clipping requires sampled reverse KL with positive top-K and renormalized support. It changes the gradient, not the reported divergence. No persisted formats, wire rows, deployment topology, or dependencies change.

Verification

• Focused distillation and OPD/SDPO/SDFT tests plus the localhost regression check: 215 passed, including CPU distributed and analytic gradient checks.
• .venv/bin/python -m mypy: passed for 378 source files.
• pre-commit run --all-files: passed.
• npm run check:docs in docs/site: passed.
• NO_PROXY=localhost,127.0.0.1 no_proxy=localhost,127.0.0.1 GIT_CONFIG_GLOBAL=/dev/null GIT_CONFIG_SYSTEM=/dev/null .venv/bin/python -m pytest tests/ -n 4 --dist loadfile -q --durations=10: 6282 passed, 101 skipped, 3 warnings. Local HTTP tests require the localhost proxy bypass on this host.
• No GPU training or performance comparison was run.

AI assistance

Codex inspected the existing objectives, implemented the option, added tests and documentation, and ran checks. The contributor remains responsible for reviewing and understanding the change.

Checklist

• The change is focused and contains no unrelated cleanup.
• Changed code satisfies the style, code structure, and naming rules in AGENTS.md.
• Tests cover behavior changes, or this pull request does not change behavior.
• Public interface changes include contract tests, or no public interface changes are present.
• Affected documentation is updated and reviewed against the Documentation review checklist in AGENTS.md.
• Root README changes update both languages and README.i18n.yaml, or do not affect README content.
• An accepted RFC is linked, or this change does not require an RFC.
• Relevant pre-commit and test checks pass.
• Non-trivial AI assistance is disclosed above, or none was used.

Review and merge

Reviewers are requested automatically from CODEOWNERS. See the maintenance model for review and merge requirements.

---

^{Need help on this PR? Tag @codesmith-bot with what you need.}

[MikeZheng777]

looks good man