The baseline reproduction result is wrong

[lmllmllm]

OISST dataset:
DDPM test results for SST:
test/50ens_mems/avg/mse=1211.059
test/50ens_mems/avg/crps=24.399
test/50ens_mems/avg/ssr=2.479
The results of the two repeated experiments are almost the same
But for Dyffusion and MCVD, I can reproduce the results similar to those in the paper according to your code
I would like to ask how to reproduce the results of DDPM
Secondly, do you have the corresponding codes for the other two baselines Dropout and Perturbation in your project?
I am a beginner and hope to get your help.

[salvaRC]

Hi and thanks for your comments!

To better understand why this is happening, could you share the config that you're using (i.e. the hydra_config.yaml file)?
I must say that I didn't re-run the DDPM baseline after cleaning up the code, so the default hyperparameters might not be optimal. For example, using fewer diffusion steps (e.g. diffusion.timesteps=5) was important for some datasets. Did you do this?

For the dropout baseline, use the <dataset>_time_conditioned.yaml configs in configs/experiment/.
For the perturbation baseline, simply train a deterministic model (module.enable_inference_dropout=False) and run inference/tests setting module.prediction_inputs_noise=EPS for EPS>0.

`

[lmllmllm]

Thank you for your reply, this is my configuration
hydra_config-v2.zip