Hi! Thanks for the great paper!
I believe there’s a small bug in the max-rescale implementation that applies eta twice.
In trainer_base.py, lines 714–717:
elif mode.startswith('max-rescale-'):
sigma_max = torch.minimum(
eta_t.expand_as(alpha_t), (1 - alpha_s) / alpha_t)
sigma = torch.where(alpha_t > 0, sigma_max, 1) * eta
This computes:
sigma = eta * min(eta, (1 - alpha_s) / alpha_t)
but the paper specifies:
sigma = eta * min(1, (1 - alpha_s) / alpha_t)
Therefore, when the minimum selects eta, the implementation gives eta²
Hi! Thanks for the great paper!
I believe there’s a small bug in the
max-rescaleimplementation that appliesetatwice.In
trainer_base.py, lines 714–717:This computes:
but the paper specifies:
Therefore, when the minimum selects
eta, the implementation giveseta²