DSEBM stability?

[kongyanye]

Hey Golan, I was just using your implementations of DSEBM on my own datasets. One problem is that the results (roc_auc) are very unstable. Like in 3 runs, the roc_auc could be 0.31, 0.41, 0.76 where the variance is 0.19! That's too large and in the experiments shown in your paper, the results of DSEBM is quite good and variance the largest 0.069. Do you have any idea of that? Would the reason be my dataset?

Hello Wang,
You can try running our code to verify the low variance persists on the reported datasets. Make sure you are using the same hyperparameters.
The dataset you are using could very well be the reason for the large variance in results. You could also try using a simpler encoder base model, and different amounts of Gaussian noise of the input layer.

[kongyanye]

I am using batch_size 64 and epochs of 100. I'll try if more epochs would make it more stable. Thanks.

Try increasing the batch size as well

…

On Fri, 14 Dec 2018, 10:02 Hanling Wang ***@***.*** wrote: I am using batch_size 64 and epochs of 100. I'll try if more epochs would make it more stable. Thanks. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AJD-u2_k5gyV_UeD9evjZ3WWpmzhTg0bks5u41sfgaJpZM4ZRLSB> .