Skip to content

Reproducibility issue: I can only reproduce 80.92(FixMatch +DWD-UT), not the reported result #1

@Troys666

Description

@Troys666

Hi, thanks for releasing the code.

I tried to reproduce the reported results in this repository, but I can
currently only reach 80.92 on my side, which is noticeably below the
number reported in the paper/README.

Could you clarify the exact reproducibility setup?

My questions are:

  1. What is the exact command sequence used for the final reported result?
  2. Which commit/version of the repository was used?
  3. Are there any missing files, scripts, or preprocessing steps that are not
    included in the public repo?
  4. Which checkpoint/assets from Zenodo are strictly required?
  5. What random seeds and number of runs were used for the reported number?
  6. Are there any training details not reflected in the README, such as batch
    size, effective learning rate, augmentation settings, thresholding, warmup, or
    evaluation protocol?

My current reproduction result:

  • Best result reproduced: 80.92
  • Repository: khtks/DWD
  • Dataset/task: cifar10/100

If needed, I can also share logs and config files. Right now it seems that the
public repository may be missing some details required to match the reported
result.

Thanks.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions