train: 17, 600 / 1000] FPS: 252.8 (389.1) , Loss/total: 1.08691 , Loss/giou: 0.26345 , Loss/l1: 0.02519 , Loss/location: 0.43331 , Loss/task_class: 0.00042 , IoU: 0.75680
ERROR: Unexpected segmentation fault encountered in worker.
Training crashed at epoch 17
Traceback for the error!
Traceback (most recent call last):
File "/home/lls/sutrack/lib/train/../../lib/train/trainers/base_trainer.py", line 85, in train
self.train_epoch()
File "/home/lls/sutrack/lib/train/../../lib/train/trainers/ltr_trainer.py", line 111, in train_epoch
self.cycle_dataset(loader)
File "/home/lls/sutrack/lib/train/../../lib/train/trainers/ltr_trainer.py", line 87, in cycle_dataset
torch.nn.utils.clip_grad_norm_(self.actor.net.parameters(), self.settings.grad_clip_norm)
File "/home/lls/miniconda3/envs/sutrack/lib/python3.8/site-packages/torch/nn/utils/clip_grad.py", line 55, in clip_grad_norm_
p.grad.detach().mul_(clip_coef_clamped.to(p.grad.device))
File "/home/lls/miniconda3/envs/sutrack/lib/python3.8/site-packages/torch/utils/data/_utils/signal_handling.py", line 66, in handler
_error_if_any_worker_fails()
RuntimeError: DataLoader worker (pid 346942) is killed by signal: Segmentation fault.
Restarting training from last epoch ...
Finished training!
请问有遇到这个过数据加载的问题吗 如何解决
train: 17, 600 / 1000] FPS: 252.8 (389.1) , Loss/total: 1.08691 , Loss/giou: 0.26345 , Loss/l1: 0.02519 , Loss/location: 0.43331 , Loss/task_class: 0.00042 , IoU: 0.75680
ERROR: Unexpected segmentation fault encountered in worker.
Training crashed at epoch 17
Traceback for the error!
Traceback (most recent call last):
File "/home/lls/sutrack/lib/train/../../lib/train/trainers/base_trainer.py", line 85, in train
self.train_epoch()
File "/home/lls/sutrack/lib/train/../../lib/train/trainers/ltr_trainer.py", line 111, in train_epoch
self.cycle_dataset(loader)
File "/home/lls/sutrack/lib/train/../../lib/train/trainers/ltr_trainer.py", line 87, in cycle_dataset
torch.nn.utils.clip_grad_norm_(self.actor.net.parameters(), self.settings.grad_clip_norm)
File "/home/lls/miniconda3/envs/sutrack/lib/python3.8/site-packages/torch/nn/utils/clip_grad.py", line 55, in clip_grad_norm_
p.grad.detach().mul_(clip_coef_clamped.to(p.grad.device))
File "/home/lls/miniconda3/envs/sutrack/lib/python3.8/site-packages/torch/utils/data/_utils/signal_handling.py", line 66, in handler
_error_if_any_worker_fails()
RuntimeError: DataLoader worker (pid 346942) is killed by signal: Segmentation fault.
Restarting training from last epoch ...
Finished training!
请问有遇到这个过数据加载的问题吗 如何解决