Skip to content

Resume training: AttributeError: 'NoneType' object has no attribute 'step' #30

@lburkert

Description

@lburkert

Hello,

I tried to train the model with intensity values but it always slows down after two hours and crashes around epoch 32-37. I now tried to resume training with the latest checkpoint but I get the following error:

Size of train_dataset = 3000
Size of test_dataset = 604
Size of val_dataset = 322
Batch size = 2
[2026-05-06 23:39:54,719][torch_points3d.datasets.base_dataset][INFO] - Available stage selection datasets: ['test', 'val']
[2026-05-06 23:39:54,719][torch_points3d.datasets.base_dataset][INFO] - The models will be selected using the metrics on following dataset: val
[2026-05-06 23:39:56,850][torch_points3d.trainer][INFO] - EPOCH 32 / 150
0%| | 0/1500 [00:12<?, ?it/s]
Traceback (most recent call last):
File "train.py", line 17, in main
trainer.train()
File "/home/lukas/ForAINet/PointCloudSegmentation/torch_points3d/trainer.py", line 157, in train
self._train_epoch(epoch)
File "/home/lukas/ForAINet/PointCloudSegmentation/torch_points3d/trainer.py", line 212, in _train_epoch
self._model.optimize_parameters2(epoch, i, self._dataset.batch_size)
File "/home/lukas/ForAINet/PointCloudSegmentation/torch_points3d/models/base_model.py", line 274, in optimize_parameters2
self._grad_scale.step(self._optimizer) # update parameters
AttributeError: 'NoneType' object has no attribute 'step'

Is there something else except the checkpoint_dir path that should be changed?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions