Cannot reproduce KNN performance for vanilla ViT-S training

[wangh09]

Hi! I followed the instructions in the readme.md to train a vanilla ViT-S model, and yield 68.4% KNN top1 acc which is 0.9% less than the claimed one. Is it because pytorch version matters or I should export the teacher backbone for evaluation? Thanks!

[wangh09]

I re-run vanilla training using PyTorch 1.7 (68.53%), PyTorch 1.13 (68.43%) PyTorch 2.3 (68.40%) without changing a line of code. Although the results show some correlation with PyTorch&CUDA versions but the difference should less than 0.2%. Is there any chance that there's a typo in the doc or the code for the 69.3% training is different with current main branch? Thanks!