Bug in Evaluation #69
Answered
by
zhoujingbo
wang975679801
asked this question in
BaiduKDDCup2022
|
We found that if we provide nan for prediction, the mse and mae will be zero. Therefore, if we set all prediction as nan, the online score will be 0. I think it should be fixed as soon as possible. |
Answered by
zhoujingbo
May 24, 2022
Replies: 3 comments 1 reply
|
Indeed, this is a bug, we will fix it ASAP, and update the online
evaluation accordingly.
Please note the announcement on the official website.
…On Mon, May 16, 2022 at 1:22 PM wang975679801 ***@***.***> wrote:
We found that if we provide nan for prediction, the mse and mae will be
zero. Therefore, if we set all prediction as nan, the online score will be
0. I think it should be fixed as soon as possible.
—
Reply to this email directly, view it on GitHub
<#69>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AEVHY346S3OVMKRKPKKB4GDVKHLRLANCNFSM5WAJBX3A>
.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
|
1 reply
|
After the online evaluation is updated, it is suggested to empty the Leaderboard and use the new evaluation for the results of subsequent rankings. |
|
The false high results will be updated.
…On Mon, May 16, 2022 at 5:06 PM light101010 ***@***.***> wrote:
After the online evaluation is updated, it is suggested to empty the
Leaderboard and use the new evaluation for the results of subsequent
rankings.
—
Reply to this email directly, view it on GitHub
<#69 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AEVHY36QXXRX3CWYVLHARXTVKIFW5ANCNFSM5WAJBX3A>
.
You are receiving this because you commented.Message ID:
***@***.***
com>
|
0 replies
|
The online tests have been updated to avoid the false high scores, please refer to the GitHub(https://github.com/PaddlePaddle/PaddleSpatial/tree/main/apps/wpf_baseline_gru/kddcup22-sdwpf-evaluation) updates for more details. |
0 replies
Answer selected by
zhoujingbo
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The online tests have been updated to avoid the false high scores, please refer to the GitHub(https://github.com/PaddlePaddle/PaddleSpatial/tree/main/apps/wpf_baseline_gru/kddcup22-sdwpf-evaluation) updates for more details.