How to quickly obtain test metrics？

[renliao]

Hi, thank you for your outstanding work, I have re-train model again, when I try to test my model, I found it is very time-consuming to make a test like python main.py --yaml_path configs/hsergb_test.yaml I guess` this may be due to saving prediction results during testing, which takes a long time. If I only want to obtain test metrics, how should I modify the configuration file? thank you.

[XiangZ-0]

Hi, thanks for your feedback. I think a quick way is to comment the image-saving codes here and here. Maybe later on I can add a config parameter about this function :)

[renliao]

Thank you so much for your patient guidance, I have understand it.

[renliao]

Hello, I have commented the image-saving codes(three calls im.write() in model_interface.py), but I found it takes a long time about more than three and a half hours on a titan GPU, I using your configuration file hsergb_test.yaml without making any changes. Another issue is that under the setting of batch_size=1, the memory usage is as high as 16824MB. These two questions confuses me a lot, can you give me some advice, thank you!

[XiangZ-0]

Hi, I guess the inference speed might be limited by stacking events (I was using np.add.at function back then, not sure if there is a better way to do event stacking these days). The inference speed of the network should be fast if I remember it right. To make the evaluation faster, one possible solution might be stacking those events (i.e., generating EGER) in advance (related function is here), so we can directly load EGER during evaluation.

For the memory, I guess that might be because the input events are pretty similar to a 16-frame video clip (because we slice the events into several bins and then stack them as input). This might be the cause of high GPU memory. One possible way might be training a new model with a reduced number of event bins (like the config here), but I guess this might also degrade the performance a bit.

[renliao]

Thank you, I will try your suggestion!