Hi thanks much for sharing.
I used only pump data for training and testing because my computer is slow. After 10 epoch training, auc=77.4, but after 300 epoch, auc is only 80. It didn't improve much.
I used the default configuration; the only difference is:
- cuda: False because I don't have nvidia gpu on my computer.
- only pump development data for training and testing to save the time.
Below is my running log:
INFO:utils:STgram-MFN(m=0.7,s=30)
INFO:utils:Num classes: 4
INFO:utils:Epoch-0 loss:15.62786
INFO:utils:Epoch-1 loss:5.49491
INFO:utils:Epoch-2 loss:2.27624
INFO:utils:Epoch-3 loss:1.02420
INFO:utils:Epoch-4 loss:0.42262
INFO:utils:Epoch-5 loss:0.24706
INFO:utils:Epoch-6 loss:0.15121
INFO:utils:Epoch-7 loss:0.09555
INFO:utils:Epoch-8 loss:0.05835
INFO:utils:Epoch-9 loss:0.02076
INFO:utils:Epoch-10 loss:0.22511
INFO:utils:pump AUC: 77.330 pAUC: 68.433
INFO:utils:Total average: AUC: 77.330 pAUC: 68.433
INFO:utils:Best epoch now is: 10
INFO:utils:Epoch-11 loss:0.02632
INFO:utils:Epoch-12 loss:0.03936
INFO:utils:Epoch-13 loss:0.04951
INFO:utils:Epoch-14 loss:0.03274
INFO:utils:Epoch-15 loss:0.11516
INFO:utils:Epoch-16 loss:0.06697
INFO:utils:Epoch-17 loss:0.01377
INFO:utils:Epoch-18 loss:0.01665
INFO:utils:Epoch-19 loss:0.00439
INFO:utils:Epoch-20 loss:0.00849
INFO:utils:pump AUC: 77.614 pAUC: 68.913
INFO:utils:Total average: AUC: 77.614 pAUC: 68.913
INFO:utils:Best epoch now is: 20
INFO:utils:Epoch-21 loss:0.00962
INFO:utils:Epoch-22 loss:0.02048
INFO:utils:Epoch-23 loss:0.01020
INFO:utils:Epoch-24 loss:0.00729
INFO:utils:Epoch-25 loss:0.00181
INFO:utils:Epoch-26 loss:0.00666
INFO:utils:Epoch-27 loss:0.00310
INFO:utils:Epoch-28 loss:0.00191
INFO:utils:Epoch-29 loss:0.00291
INFO:utils:Epoch-30 loss:0.03659
INFO:utils:pump AUC: 77.212 pAUC: 69.282
INFO:utils:Total average: AUC: 77.212 pAUC: 69.282
INFO:utils:Epoch-31 loss:0.01610
INFO:utils:Epoch-32 loss:0.00538
INFO:utils:Epoch-33 loss:0.00195
INFO:utils:Epoch-34 loss:0.00357
INFO:utils:Epoch-35 loss:0.01668
INFO:utils:Epoch-36 loss:0.01750
INFO:utils:Epoch-37 loss:0.04897
INFO:utils:Epoch-38 loss:0.05807
INFO:utils:Epoch-39 loss:0.01372
INFO:utils:Epoch-40 loss:0.04973
INFO:utils:pump AUC: 79.376 pAUC: 69.706
INFO:utils:Total average: AUC: 79.376 pAUC: 69.706
INFO:utils:Best epoch now is: 40
INFO:utils:Epoch-41 loss:0.04640
INFO:utils:Epoch-42 loss:0.02116
INFO:utils:Epoch-43 loss:0.05126
INFO:utils:Epoch-44 loss:0.02208
INFO:utils:Epoch-45 loss:0.00957
INFO:utils:Epoch-46 loss:0.00393
INFO:utils:Epoch-47 loss:0.00573
INFO:utils:Epoch-48 loss:0.03235
INFO:utils:Epoch-49 loss:0.04297
INFO:utils:Epoch-50 loss:0.01801
INFO:utils:pump AUC: 78.068 pAUC: 68.456
INFO:utils:Total average: AUC: 78.068 pAUC: 68.456
INFO:utils:Epoch-51 loss:0.04664
INFO:utils:Epoch-52 loss:0.01839
INFO:utils:Epoch-53 loss:0.00978
INFO:utils:Epoch-54 loss:0.02462
INFO:utils:Epoch-55 loss:0.05797
INFO:utils:Epoch-56 loss:0.03427
INFO:utils:Epoch-57 loss:0.02459
INFO:utils:Epoch-58 loss:0.04864
INFO:utils:Epoch-59 loss:0.02287
INFO:utils:Epoch-60 loss:0.00751
INFO:utils:pump AUC: 80.249 pAUC: 69.529
INFO:utils:Total average: AUC: 80.249 pAUC: 69.529
INFO:utils:Best epoch now is: 60
INFO:utils:Epoch-61 loss:0.00234
INFO:utils:Epoch-62 loss:0.00751
INFO:utils:Epoch-63 loss:0.00424
INFO:utils:Epoch-64 loss:0.00331
INFO:utils:Epoch-65 loss:0.00085
INFO:utils:Epoch-66 loss:0.00085
INFO:utils:Epoch-67 loss:0.00059
INFO:utils:Epoch-68 loss:0.00075
INFO:utils:Epoch-69 loss:0.00041
INFO:utils:Epoch-70 loss:0.00661
INFO:utils:pump AUC: 79.056 pAUC: 69.453
INFO:utils:Total average: AUC: 79.056 pAUC: 69.453
INFO:utils:Epoch-71 loss:0.01471
INFO:utils:Epoch-72 loss:0.00731
INFO:utils:Epoch-73 loss:0.00163
INFO:utils:Epoch-74 loss:0.00108
INFO:utils:Epoch-75 loss:0.00060
INFO:utils:Epoch-76 loss:0.01550
INFO:utils:Epoch-77 loss:0.03675
INFO:utils:Epoch-78 loss:0.00171
INFO:utils:Epoch-79 loss:0.00540
INFO:utils:Epoch-80 loss:0.00728
INFO:utils:pump AUC: 79.831 pAUC: 69.474
INFO:utils:Total average: AUC: 79.831 pAUC: 69.474
INFO:utils:Epoch-81 loss:0.00314
INFO:utils:Epoch-82 loss:0.06445
INFO:utils:Epoch-83 loss:0.05244
INFO:utils:Epoch-84 loss:0.07519
INFO:utils:Epoch-85 loss:0.01948
INFO:utils:Epoch-86 loss:0.01926
INFO:utils:Epoch-87 loss:0.01146
INFO:utils:Epoch-88 loss:0.00292
INFO:utils:Epoch-89 loss:0.00136
INFO:utils:Epoch-90 loss:0.00319
INFO:utils:pump AUC: 81.369 pAUC: 70.625
INFO:utils:Total average: AUC: 81.369 pAUC: 70.625
INFO:utils:Best epoch now is: 90
INFO:utils:Epoch-91 loss:0.00154
INFO:utils:Epoch-92 loss:0.00026
INFO:utils:Epoch-93 loss:0.00037
INFO:utils:Epoch-94 loss:0.00024
INFO:utils:Epoch-95 loss:0.00164
INFO:utils:Epoch-96 loss:0.00990
INFO:utils:Epoch-97 loss:0.00660
INFO:utils:Epoch-98 loss:0.01275
INFO:utils:Epoch-99 loss:0.00239
INFO:utils:Epoch-100 loss:0.00765
INFO:utils:pump AUC: 79.496 pAUC: 70.133
INFO:utils:Total average: AUC: 79.496 pAUC: 70.133
INFO:utils:Epoch-101 loss:0.00611
INFO:utils:Epoch-102 loss:0.00919
INFO:utils:Epoch-103 loss:0.00256
INFO:utils:Epoch-104 loss:0.00625
INFO:utils:Epoch-105 loss:0.00929
INFO:utils:Epoch-106 loss:0.00103
INFO:utils:Epoch-107 loss:0.00573
INFO:utils:Epoch-108 loss:0.00176
INFO:utils:Epoch-109 loss:0.03701
INFO:utils:Epoch-110 loss:0.01105
INFO:utils:pump AUC: 80.832 pAUC: 72.228
INFO:utils:Total average: AUC: 80.832 pAUC: 72.228
INFO:utils:Best epoch now is: 110
INFO:utils:Epoch-111 loss:0.00666
INFO:utils:Epoch-112 loss:0.07560
INFO:utils:Epoch-113 loss:0.00872
INFO:utils:Epoch-114 loss:0.08129
INFO:utils:Epoch-115 loss:0.03537
INFO:utils:Epoch-116 loss:0.00442
INFO:utils:Epoch-117 loss:0.00419
INFO:utils:Epoch-118 loss:0.00349
INFO:utils:Epoch-119 loss:0.00451
INFO:utils:Epoch-120 loss:0.00120
INFO:utils:pump AUC: 80.552 pAUC: 71.465
INFO:utils:Total average: AUC: 80.552 pAUC: 71.465
INFO:utils:Epoch-121 loss:0.00081
INFO:utils:Epoch-122 loss:0.00021
INFO:utils:Epoch-123 loss:0.06567
INFO:utils:Epoch-124 loss:0.03232
INFO:utils:Epoch-125 loss:0.00895
INFO:utils:Epoch-126 loss:0.00104
INFO:utils:Epoch-127 loss:0.00104
INFO:utils:Epoch-128 loss:0.00286
INFO:utils:Epoch-129 loss:0.00105
INFO:utils:Epoch-130 loss:0.00167
INFO:utils:pump AUC: 81.118 pAUC: 71.880
INFO:utils:Total average: AUC: 81.118 pAUC: 71.880
INFO:utils:Epoch-131 loss:0.00171
INFO:utils:Epoch-132 loss:0.04268
INFO:utils:Epoch-133 loss:0.01298
INFO:utils:Epoch-134 loss:0.00152
INFO:utils:Epoch-135 loss:0.00383
INFO:utils:Epoch-136 loss:0.00073
INFO:utils:Epoch-137 loss:0.00301
INFO:utils:Epoch-138 loss:0.00149
INFO:utils:Epoch-139 loss:0.00164
INFO:utils:Epoch-140 loss:0.01688
INFO:utils:pump AUC: 81.230 pAUC: 71.215
INFO:utils:Total average: AUC: 81.230 pAUC: 71.215
INFO:utils:Epoch-141 loss:0.01115
INFO:utils:Epoch-142 loss:0.00056
INFO:utils:Epoch-143 loss:0.00052
INFO:utils:Epoch-144 loss:0.00179
INFO:utils:Epoch-145 loss:0.00132
INFO:utils:Epoch-146 loss:0.00205
INFO:utils:Epoch-147 loss:0.00046
INFO:utils:Epoch-148 loss:0.00021
INFO:utils:Epoch-149 loss:0.00027
INFO:utils:Epoch-150 loss:0.00031
INFO:utils:pump AUC: 79.892 pAUC: 70.934
INFO:utils:Total average: AUC: 79.892 pAUC: 70.934
INFO:utils:Epoch-151 loss:0.00021
INFO:utils:Epoch-152 loss:0.00050
INFO:utils:Epoch-153 loss:0.01459
INFO:utils:Epoch-154 loss:0.00207
INFO:utils:Epoch-155 loss:0.00315
INFO:utils:Epoch-156 loss:0.00057
INFO:utils:Epoch-157 loss:0.00123
INFO:utils:Epoch-158 loss:0.00016
INFO:utils:Epoch-159 loss:0.00045
INFO:utils:Epoch-160 loss:0.00257
INFO:utils:pump AUC: 78.213 pAUC: 70.947
INFO:utils:Total average: AUC: 78.213 pAUC: 70.947
INFO:utils:Epoch-161 loss:0.00012
INFO:utils:Epoch-162 loss:0.00043
INFO:utils:Epoch-163 loss:0.00037
INFO:utils:Epoch-164 loss:0.00013
INFO:utils:Epoch-165 loss:0.00037
INFO:utils:Epoch-166 loss:0.01061
INFO:utils:Epoch-167 loss:0.00241
INFO:utils:Epoch-168 loss:0.05023
INFO:utils:Epoch-169 loss:0.01701
INFO:utils:Epoch-170 loss:0.00905
INFO:utils:pump AUC: 79.311 pAUC: 69.908
INFO:utils:Total average: AUC: 79.311 pAUC: 69.908
INFO:utils:Epoch-171 loss:0.00200
INFO:utils:Epoch-172 loss:0.00148
INFO:utils:Epoch-173 loss:0.00180
INFO:utils:Epoch-174 loss:0.00058
INFO:utils:Epoch-175 loss:0.00024
INFO:utils:Epoch-176 loss:0.00260
INFO:utils:Epoch-177 loss:0.00103
INFO:utils:Epoch-178 loss:0.00033
INFO:utils:Epoch-179 loss:0.01628
INFO:utils:Epoch-180 loss:0.02650
INFO:utils:pump AUC: 79.154 pAUC: 71.912
INFO:utils:Total average: AUC: 79.154 pAUC: 71.912
INFO:utils:Epoch-181 loss:0.00550
INFO:utils:Epoch-182 loss:0.00031
INFO:utils:Epoch-183 loss:0.00046
INFO:utils:Epoch-184 loss:0.00111
INFO:utils:Epoch-185 loss:0.02384
INFO:utils:Epoch-186 loss:0.01787
INFO:utils:Epoch-187 loss:0.00869
INFO:utils:Epoch-188 loss:0.00182
INFO:utils:Epoch-189 loss:0.00076
INFO:utils:Epoch-190 loss:0.01137
INFO:utils:pump AUC: 77.839 pAUC: 70.191
INFO:utils:Total average: AUC: 77.839 pAUC: 70.191
INFO:utils:Epoch-191 loss:0.00184
INFO:utils:Epoch-192 loss:0.00317
INFO:utils:Epoch-193 loss:0.00235
INFO:utils:Epoch-194 loss:0.00383
INFO:utils:Epoch-195 loss:0.00027
INFO:utils:Epoch-196 loss:0.00071
INFO:utils:Epoch-197 loss:0.00337
INFO:utils:Epoch-198 loss:0.00067
INFO:utils:Epoch-199 loss:0.00153
INFO:utils:Epoch-200 loss:0.00497
INFO:utils:pump AUC: 78.721 pAUC: 70.822
INFO:utils:Total average: AUC: 78.721 pAUC: 70.822
INFO:utils:Epoch-201 loss:0.00016
INFO:utils:Epoch-202 loss:0.00015
INFO:utils:Epoch-203 loss:0.00055
INFO:utils:Epoch-204 loss:0.00017
INFO:utils:Epoch-205 loss:0.00026
INFO:utils:Epoch-206 loss:0.00016
INFO:utils:Epoch-207 loss:0.00745
INFO:utils:Epoch-208 loss:0.04068
INFO:utils:Epoch-209 loss:0.01181
INFO:utils:Epoch-210 loss:0.00331
INFO:utils:pump AUC: 78.783 pAUC: 69.872
INFO:utils:Total average: AUC: 78.783 pAUC: 69.872
INFO:utils:Epoch-211 loss:0.00159
INFO:utils:Epoch-212 loss:0.00038
INFO:utils:Epoch-213 loss:0.00082
INFO:utils:Epoch-214 loss:0.00008
INFO:utils:Epoch-215 loss:0.00220
INFO:utils:Epoch-216 loss:0.00018
INFO:utils:Epoch-217 loss:0.00071
INFO:utils:Epoch-218 loss:0.02302
INFO:utils:Epoch-219 loss:0.00687
INFO:utils:Epoch-220 loss:0.00253
INFO:utils:pump AUC: 78.196 pAUC: 68.756
INFO:utils:Total average: AUC: 78.196 pAUC: 68.756
INFO:utils:Epoch-221 loss:0.00040
INFO:utils:Epoch-222 loss:0.00014
INFO:utils:Epoch-223 loss:0.00014
INFO:utils:Epoch-224 loss:0.00717
INFO:utils:Epoch-225 loss:0.00243
INFO:utils:Epoch-226 loss:0.00283
INFO:utils:Epoch-227 loss:0.00008
INFO:utils:Epoch-228 loss:0.00331
INFO:utils:Epoch-229 loss:0.00862
INFO:utils:Epoch-230 loss:0.00034
INFO:utils:pump AUC: 77.544 pAUC: 68.950
INFO:utils:Total average: AUC: 77.544 pAUC: 68.950
INFO:utils:Epoch-231 loss:0.00064
INFO:utils:Epoch-232 loss:0.00053
INFO:utils:Epoch-233 loss:0.00047
INFO:utils:Epoch-234 loss:0.00013
INFO:utils:Epoch-235 loss:0.00009
INFO:utils:Epoch-236 loss:0.00005
INFO:utils:Epoch-237 loss:0.00011
INFO:utils:Epoch-238 loss:0.00018
INFO:utils:Epoch-239 loss:0.00055
INFO:utils:Epoch-240 loss:0.00027
INFO:utils:pump AUC: 78.123 pAUC: 69.986
INFO:utils:Total average: AUC: 78.123 pAUC: 69.986
INFO:utils:Epoch-241 loss:0.00027
INFO:utils:Epoch-242 loss:0.00004
INFO:utils:Epoch-243 loss:0.01369
INFO:utils:Epoch-244 loss:0.00032
INFO:utils:Epoch-245 loss:0.05951
INFO:utils:Epoch-246 loss:0.00076
INFO:utils:Epoch-247 loss:0.00250
INFO:utils:Epoch-248 loss:0.00583
INFO:utils:Epoch-249 loss:0.00109
INFO:utils:Epoch-250 loss:0.00031
INFO:utils:pump AUC: 78.008 pAUC: 70.605
INFO:utils:Total average: AUC: 78.008 pAUC: 70.605
INFO:utils:Epoch-251 loss:0.00117
INFO:utils:Epoch-252 loss:0.00027
INFO:utils:Epoch-253 loss:0.00110
INFO:utils:Epoch-254 loss:0.00057
INFO:utils:Epoch-255 loss:0.03768
INFO:utils:Epoch-256 loss:0.00013
INFO:utils:Epoch-257 loss:0.00107
INFO:utils:Epoch-258 loss:0.00080
INFO:utils:Epoch-259 loss:0.00088
INFO:utils:Epoch-260 loss:0.00439
INFO:utils:pump AUC: 78.127 pAUC: 70.133
INFO:utils:Total average: AUC: 78.127 pAUC: 70.133
INFO:utils:Epoch-261 loss:0.00373
INFO:utils:Epoch-262 loss:0.00296
INFO:utils:Epoch-263 loss:0.04232
INFO:utils:Epoch-264 loss:0.00568
INFO:utils:Epoch-265 loss:0.06025
INFO:utils:Epoch-266 loss:0.00239
INFO:utils:Epoch-267 loss:0.00316
INFO:utils:Epoch-268 loss:0.00151
INFO:utils:Epoch-269 loss:0.03039
INFO:utils:Epoch-270 loss:0.00027
INFO:utils:pump AUC: 77.749 pAUC: 69.830
INFO:utils:Total average: AUC: 77.749 pAUC: 69.830
INFO:utils:Epoch-271 loss:0.09543
INFO:utils:Epoch-272 loss:0.00667
INFO:utils:Epoch-273 loss:0.00298
INFO:utils:Epoch-274 loss:0.00026
INFO:utils:Epoch-275 loss:0.00044
INFO:utils:Epoch-276 loss:0.00048
INFO:utils:Epoch-277 loss:0.01614
INFO:utils:Epoch-278 loss:0.00026
INFO:utils:Epoch-279 loss:0.00017
INFO:utils:Epoch-280 loss:0.00052
INFO:utils:pump AUC: 77.364 pAUC: 70.308
INFO:utils:Total average: AUC: 77.364 pAUC: 70.308
INFO:utils:Epoch-281 loss:0.00921
INFO:utils:Epoch-282 loss:0.03543
INFO:utils:Epoch-283 loss:0.00023
INFO:utils:Epoch-284 loss:0.00018
INFO:utils:Epoch-285 loss:0.00020
INFO:utils:Epoch-286 loss:0.00020
INFO:utils:Epoch-287 loss:0.00014
INFO:utils:Epoch-288 loss:0.00009
INFO:utils:Epoch-289 loss:0.01257
INFO:utils:Epoch-290 loss:0.00082
INFO:utils:pump AUC: 79.472 pAUC: 70.557
INFO:utils:Total average: AUC: 79.472 pAUC: 70.557
INFO:utils:Epoch-291 loss:0.00029
INFO:utils:Epoch-292 loss:0.00044
INFO:utils:Epoch-293 loss:0.00251
INFO:utils:Epoch-294 loss:0.07989
INFO:utils:Epoch-295 loss:0.00031
INFO:utils:Epoch-296 loss:0.00026
INFO:utils:Epoch-297 loss:0.02646
INFO:utils:Epoch-298 loss:0.00091
INFO:utils:Epoch-299 loss:0.00053
INFO:utils:Epoch-300 loss:0.03057
INFO:utils:pump AUC: 79.784 pAUC: 71.109
INFO:utils:Total average: AUC: 79.784 pAUC: 71.109
INFO:utils:pump AUC: 80.832 pAUC: 72.228
INFO:utils:Total average: AUC: 80.832 pAUC: 72.228
Hi thanks much for sharing.
I used only pump data for training and testing because my computer is slow. After 10 epoch training, auc=77.4, but after 300 epoch, auc is only 80. It didn't improve much.
I used the default configuration; the only difference is:
Below is my running log: