Hi have you try your crossentropy loss

[jiqizaisikao]

I really do not understand your code,i think that the crossentropy loss is like this:
loss=-p*(log(Q/P))+(-(1-P)log((1-Q)/(1-P)))
when P=Q the loss is zero
if loss=-P(log(Q/P))
when P=Q the loss is not at the minum,

[zhf459]

i'm not sure i get you correctly ,do you mean the term H(Ps,Pt)?. all I want to get is the kl_loss(Ps,Pt) = H(Ps,Pt) - H(Ps),so I think I try to get this from two parts: H(Ps)=ln(s)+2; and for H(Ps,Pt) term -- the cross entropy between teacher and student distributions, for a sampled x from student ,I can get Ps, and feed the x into teacher I get y_hat, so log_Pt = discretized_mix_logistic_loss(y_hat, x) (maybe the original y) if we specific the parameters reduce=False ,and finally H(Ps,Pt) = -sum(log_Pt*Ps)-H(Ps)(maybe wrong). kl loss is final loss.

[jiqizaisikao]

@zhf459 ok,i will try it again, have you got any result?

[zhf459]

no,I got inf loss.can we chat on QQ or wechat for detail ?

[jiqizaisikao]

QQ:3617 28654

[neverjoe]

shall we work together？qq: 29 50 424 15