Scaling in time embedding in Lab Three

[qwdong]

In the end, we put cos and sin into a sequence:

Step 3: Concatenate and return

    return torch.cat([sin_embed, cos_embed], dim=-1) * math.sqrt(2) # b d

In the lecture notes, we have the following equation, is this wrong?