【Task.27】feat(algorithms): add GDPO multi-reward advantage estimator - #277

Open

Men1scus wants to merge 22 commits into

redai-studio:mainfrom

Men1scus:pr2/gdpo

Commits on Aug 24, 2026

refactor(algorithms): declare each algorithm once in a registry
Men1scus
committed
309cd81
feat(gdpo): add the GDPO multi-reward advantage estimator
Men1scus
committed
5401070
fix(gdpo): agree on the segmentation before whitening, reject float32 overflow
Men1scus
committed
905d0e0
feat(gdpo): make the upstream reward consumers component-aware
Men1scus
committed
e5404dc
fix(gdpo): correct the motivation, and the consumers that encoded it
Men1scus
committed
884aa1a
test(algorithms): make two assertions capable of failing
Men1scus
committed
afe13f5
fix(algorithms): unbreak the CPU tests, and stop two more specs from lying
Men1scus
committed
7453148
docs(gdpo): state the motivation as one formula instead of two half-truths
Men1scus
committed
6f21447
fix(gdpo): stop cancelling components from becoming a gradient
Men1scus
committed
c1399bf
fix(rewards): stop a crashing label and a signal-swallowing floor
Men1scus
committed
4be7743
docs(algorithms): name what actually aligns the whitening segments
Men1scus
committed
d26387c
fix(rewards): withdraw the noise guard, keep what measuring it taught
Men1scus
committed
a0d8118
fix(gdpo): stop editing the training signal to record a judgement about it
Men1scus
committed
8567e34
test(gdpo): exercise step 3 through the production Megatron entry point
Men1scus
committed
a3971ff
fix(gdpo): remove a criterion that cannot answer the question it was asked
Men1scus
committed
3217cf0
fix(gdpo): fail together across ranks, and stop overstating what was measured
Men1scus
committed
5241b11
test(gdpo): prove the cross-rank failure, and close three gaps the review left
Men1scus
committed
772a4b2
test(gdpo): run the review's asked-for chain over a real process group
Men1scus
committed
24975d9
fix(gdpo): catch the reward error the metrics path could actually hit
Men1scus
committed
65ed22e
perf(gdpo): take the per-batch device syncs out of step 3's guards
Men1scus
committed
a233d7e
test(gdpo): bound the gloo timeout so a stranded rank fails instead of stalling
Men1scus
committed
b92ce2d
test(gdpo): bind the loss module to a one-process mpu whether or not MCore is installed
Men1scus
committed
cb7d181

Navigation Menu

Uh oh!

【Task.27】feat(algorithms): add GDPO multi-reward advantage estimator - #277

【Task.27】feat(algorithms): add GDPO multi-reward advantage estimator#277
Men1scus wants to merge 22 commits into
redai-studio:mainfrom
Men1scus:pr2/gdpo

Commits on Aug 24, 2026

refactor(algorithms): declare each algorithm once in a registry

feat(gdpo): add the GDPO multi-reward advantage estimator

fix(gdpo): agree on the segmentation before whitening, reject float32 overflow

feat(gdpo): make the upstream reward consumers component-aware

fix(gdpo): correct the motivation, and the consumers that encoded it

test(algorithms): make two assertions capable of failing

fix(algorithms): unbreak the CPU tests, and stop two more specs from lying

docs(gdpo): state the motivation as one formula instead of two half-truths

fix(gdpo): stop cancelling components from becoming a gradient

fix(rewards): stop a crashing label and a signal-swallowing floor

docs(algorithms): name what actually aligns the whitening segments

fix(rewards): withdraw the noise guard, keep what measuring it taught

fix(gdpo): stop editing the training signal to record a judgement about it

test(gdpo): exercise step 3 through the production Megatron entry point

fix(gdpo): remove a criterion that cannot answer the question it was asked

fix(gdpo): fail together across ranks, and stop overstating what was measured

test(gdpo): prove the cross-rank failure, and close three gaps the review left

test(gdpo): run the review's asked-for chain over a real process group

fix(gdpo): catch the reward error the metrics path could actually hit

perf(gdpo): take the per-batch device syncs out of step 3's guards

test(gdpo): bound the gloo timeout so a stranded rank fails instead of stalling

test(gdpo): bind the loss module to a one-process mpu whether or not MCore is installed