ERROR: Warp Cashed. Results saved in ./tmp/

[samadhiiiii]

I try to train this model.But every time in the testing phase of epoch2/149 report errors。

Epoch gpu_mem loss1 loss2 loss3 total img_size
2/149 20.4G 0.1235 0.1084 0.1039 2.514 128: 100%|██████████| 391/391 [07:31<00:00, 1.03s/it]
Images loss1 loss2 loss3 total PSNR SSIM: 5%|▌ | 1/20 [00:03<01:11, 3.76s/it]ERROR: Warp Cashed. Results saved in ./tmp/

Process finished with exit code 123

If you know how to solve this problem. please tell me. Thank you.

[tgjjj]

This can happen when the warped mask is all zeros. Please check the "./tmp" directory to find out what went wrong.

[samadhiiiii]

I checked ./tmp/，there is no problems found.
Is there a problem with my dataset？
Or what other reasons may cause the warped mask is all zeros?

[tgjjj]

There are multiple causes of this problem, including datasets and training settings. You can check the output of the network during training and testing to find more hints about this problem. Including:

• If the network predicts very large offset values or even NaNs, it can cause the image to move completely out of the screen.
• If the warped image is not a convex quadrilateral.
• If it always crashes on the same image.
  You can post some outputs here so that we may come up with more detailed advice.