question about Equation (27)

[yihao-123]

<img width="2834" height="716" alt="Image" src="https://github.com/user-attachments/assets/ad5f58b1-b5b6-4193-96b9-ecac31856534" /
Why does $f(\mu)$ disappear in Equation 27? Is $f(\mu)$ the same in DPO and TIDPO?

[yiboliu2002]

The disappearance of $f(\mu)$ in Eq.(27) relies on the unbiasedness assumption stated in Theorem 2: both estimators $\Delta r_{\text{global}}$ (DPO) and $\Delta r_{\text{token}}$ (TI-DPO) are assumed to be unbiased estimators of the same true reward difference $\mu$, i.e., $\mathbb{E}[\Delta r_{\text{global}}] = \mathbb{E}[\Delta r_{\text{token}}] = \mu$.

$$L_{\text{DPO}} \approx f(\mu) + \frac{1}{2}\mathbb{E}[f''(\xi)]\sigma^2_{\text{DPO}}$$

$$L_{\text{TI-DPO}} \approx f(\mu) + \frac{1}{2}\mathbb{E}[f''(\xi)]\sigma^2_{\text{TI-DPO}}$$

Subtracting the two equations, f(μ) as the common term directly cancels out, resulting in Eq.(27) :

$$L_{\text{DPO}} - L_{\text{TI-DPO}} \approx \frac{1}{2}\mathbb{E}[f''(\xi)](\sigma^2_{\text{DPO}} - \sigma^2_{\text{TI-DPO}})$$

[XuLi666231]

Hello, have you implemented the corresponding code?How can I convert the .pt checkpoint file into a format compatible with lm-evaluation-harness?Thank you.