【Task.31】feat(sft): add reward modeling and preference evaluation - #258

Open

A-Words wants to merge 31 commits into

redai-studio:mainfrom

A-Words:feat/task-31-reward-modeling

Commits on Aug 7, 2026

feat(sft): add pair-aware DPO training
A-Words
committed
8f99e09
fix(sft): enforce DPO reference integrity
A-Words
committed
46154cd
fix(sft): complete DPO delivery contract
A-Words
committed
f03aa5c

Commits on Aug 8, 2026

fix(sft): reject --ref-load for DPO objectives
A-Words
committed
2903493
refactor(sft): apply DPO P3 review cleanups
A-Words
committed
83016e0

Commits on Aug 9, 2026

fix(dpo): enforce RFC validation contracts
A-Words
committed
76352f2
fix(dpo): pin frozen reference snapshot
A-Words
committed
5b26ecb
fix(dpo): verify local reference metadata
A-Words
committed
cac9492
fix(dpo): verify reference file digests
A-Words
committed
1b9d595
fix(dpo): verify reference weight manifests
A-Words
committed
72d9f69
fix(dpo): avoid iterator device readbacks
A-Words
committed
6f21770

Commits on Aug 10, 2026

refactor(dpo): remove dead identity builder
A-Words
committed
0e8c334
feat(sft): add reward modeling and preference evaluation
A-Words
committed
c3c9902
fix(reward-model): enforce RFC contracts
A-Words
committed
6407000
fix(sft): unblock colocate baseline eval
A-Words
committed
62b8759
fix(sft): preserve preference eval identity
A-Words
committed
a5b43c5
fix(sft): align async evaluation steps
A-Words
committed
8942a54
fix(checkpoint): support Megatron formats
A-Words
committed
e0ffa32
fix(sft): preserve preference eval pair IDs
A-Words
committed
38e0f2a
fix(reward-model): reject HF export
A-Words
committed
2cc08d7
fix(reward-model): enable Gloo iterator control
A-Words
committed
2268036
fix(sft): enforce frozen eval probe size
A-Words
committed
b57a181

Commits on Aug 11, 2026

chore(task31): merge upstream main into DPO
A-Words
committed
4f149d4
chore(task31): merge updated DPO base into reward modeling
A-Words
committed
94c940a
test(dpo): guard Megatron-only checkpoint import
A-Words
committed
0e0b059
chore(task31): merge DPO CI fix into reward modeling
A-Words
committed
08e8b45
test(sft): skip Megatron integration without backend
A-Words
committed
633d386
chore(task31): merge SFT CI fix into reward modeling
A-Words
committed
38d531e

Commits on Aug 21, 2026

docs(dpo): link Task 31 evidence bundle
A-Words
committed
d1b80d1
chore: merge upstream main into DPO
A-Words
committed
3feb0fe
chore(task31): merge DPO into reward modeling
A-Words
committed
3b1af71