Finetuning model

[Srivatsa1648]

bash scripts/DINO_train.sh /home/slam/Downloads/REU_Srivatsa/radar_img --pretrain_model_path /home/slam/Downloads/REU_Srivatsa/checkpoint0011_4scale.pth --finetune_ignore label_enc.weight class_embed
when i give the above command and modify the config file as per the class in my dataset i am facing the below error of torch size mismatch
RuntimeError: Error(s) in loading state_dict for DINO:
size mismatch for transformer.decoder.class_embed.0.weight: copying a param with shape torch.Size([91, 256]) from checkpoint, the shape in current model is torch.Size([11, 256]).
size mismatch for transformer.decoder.class_embed.0.bias: copying a param with shape torch.Size([91]) from checkpoint, the shape in current model is torch.Size([11]).
size mismatch for transformer.decoder.class_embed.1.weight: copying a param with shape torch.Size([91, 256]) from checkpoint, the shape in current model is torch.Size([11, 256]).
size mismatch for transformer.decoder.class_embed.1.bias: copying a param with shape torch.Size([91]) from checkpoint, the shape in current model is torch.Size([11]).
size mismatch for transformer.decoder.class_embed.2.weight: copying a param with shape torch.Size([91, 256]) from checkpoint, the shape in current model is torch.Size([11, 256]).
size mismatch for transformer.decoder.class_embed.2.bias: copying a param with shape torch.Size([91]) from checkpoint, the shape in current model is torch.Size([11]).
size mismatch for transformer.decoder.class_embed.3.weight: copying a param with shape torch.Size([91, 256]) from checkpoint, the shape in current model is torch.Size([11, 256]).
size mismatch for transformer.decoder.class_embed.3.bias: copying a param with shape torch.Size([91]) from checkpoint, the shape in current model is torch.Size([11]).
size mismatch for transformer.decoder.class_embed.4.weight: copying a param with shape torch.Size([91, 256]) from checkpoint, the shape in current model is torch.Size([11, 256]).
size mismatch for transformer.decoder.class_embed.4.bias: copying a param with shape torch.Size([91]) from checkpoint, the shape in current model is torch.Size([11]).
size mismatch for transformer.decoder.class_embed.5.weight: copying a param with shape torch.Size([91, 256]) from checkpoint, the shape in current model is torch.Size([11, 256]).
size mismatch for transformer.decoder.class_embed.5.bias: copying a param with shape torch.Size([91]) from checkpoint, the shape in current model is torch.Size([11]).
size mismatch for transformer.enc_out_class_embed.weight: copying a param with shape torch.Size([91, 256]) from checkpoint, the shape in current model is torch.Size([11, 256]).
size mismatch for transformer.enc_out_class_embed.bias: copying a param with shape torch.Size([91]) from checkpoint, the shape in current model is torch.Size([11]).
size mismatch for label_enc.weight: copying a param with shape torch.Size([92, 256]) from checkpoint, the shape in current model is torch.Size([15, 256]).
size mismatch for class_embed.0.weight: copying a param with shape torch.Size([91, 256]) from checkpoint, the shape in current model is torch.Size([11, 256]).
size mismatch for class_embed.0.bias: copying a param with shape torch.Size([91]) from checkpoint, the shape in current model is torch.Size([11]).
size mismatch for class_embed.1.weight: copying a param with shape torch.Size([91, 256]) from checkpoint, the shape in current model is torch.Size([11, 256]).
size mismatch for class_embed.1.bias: copying a param with shape torch.Size([91]) from checkpoint, the shape in current model is torch.Size([11]).
size mismatch for class_embed.2.weight: copying a param with shape torch.Size([91, 256]) from checkpoint, the shape in current model is torch.Size([11, 256]).
size mismatch for class_embed.2.bias: copying a param with shape torch.Size([91]) from checkpoint, the shape in current model is torch.Size([11]).
size mismatch for class_embed.3.weight: copying a param with shape torch.Size([91, 256]) from checkpoint, the shape in current model is torch.Size([11, 256]).
size mismatch for class_embed.3.bias: copying a param with shape torch.Size([91]) from checkpoint, the shape in current model is torch.Size([11]).
size mismatch for class_embed.4.weight: copying a param with shape torch.Size([91, 256]) from checkpoint, the shape in current model is torch.Size([11, 256]).
size mismatch for class_embed.4.bias: copying a param with shape torch.Size([91]) from checkpoint, the shape in current model is torch.Size([11]).
size mismatch for class_embed.5.weight: copying a param with shape torch.Size([91, 256]) from checkpoint, the shape in current model is torch.Size([11, 256]).
size mismatch for class_embed.5.bias: copying a param with shape torch.Size([91]) from checkpoint, the shape in current model is torch.Size([11]).
and if i finetune without changing config file i am getting very least ap of 0.26 and i am training for dino_4scale model
pls reslove my error asap

[Abdul-Rehman-Astro]

@Srivatsa1648 I am also facing the issue , please tell me how did you resolved ?
@HaoZhang534 @SlongLiu @FengLi-ust @developer0hye I am also facing the same issue.
i have changed the num_classes = 2
and dn_labebook_size = 3
still !!!

"backbone.0.body.layer4.2.conv3.weight": 1048576
}
data_aug_params: {
"scales": [
480,
512,
544,
576,
608,
640,
672,
704,
736,
768,
800
],
"max_size": 1333,
"scales2_resize": [
400,
500,
600
],
"scales2_crop": [
384,
600
]
}
loading annotations into memory...
Done (t=0.08s)
creating index...
index created!
data_aug_params: {
"scales": [
480,
512,
544,
576,
608,
640,
672,
704,
736,
768,
800
],
"max_size": 1333,
"scales2_resize": [
400,
500,
600
],
"scales2_crop": [
384,
600
]
}
loading annotations into memory...
Done (t=0.00s)
creating index...
index created!
main.py:212: FutureWarning: You are using torch.load with weights_only=False (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for weights_only will be flipped to True. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via torch.serialization.add_safe_globals. We recommend you start setting weights_only=True for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
checkpoint = torch.load(args.resume, map_location='cpu')
Traceback (most recent call last):
File "main.py", line 389, in
main(args)
File "main.py", line 213, in main
model_without_ddp.load_state_dict(checkpoint['model'])
File "/home/abdul_rehman/scratch/anaconda3/envs/Proj1/lib/python3.8/site-packages/torch/nn/modules/module.py", line 2215, in load_state_dict
raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for DINO:
size mismatch for transformer.decoder.class_embed.0.weight: copying a param with shape torch.Size([91, 256]) from checkpoint, the shape in current model is torch.Size([2, 256]).
size mismatch for transformer.decoder.class_embed.0.bias: copying a param with shape torch.Size([91]) from checkpoint, the shape in current model is torch.Size([2]).
size mismatch for transformer.decoder.class_embed.1.weight: copying a param with shape torch.Size([91, 256]) from checkpoint, the shape in current model is torch.Size([2, 256]).
size mismatch for transformer.decoder.class_embed.1.bias: copying a param with shape torch.Size([91]) from checkpoint, the shape in current model is torch.Size([2]).
size mismatch for transformer.decoder.class_embed.2.weight: copying a param with shape torch.Size([91, 256]) from checkpoint, the shape in current model is torch.Size([2, 256]).
size mismatch for transformer.decoder.class_embed.2.bias: copying a param with shape torch.Size([91]) from checkpoint, the shape in current model is torch.Size([2]).
size mismatch for transformer.decoder.class_embed.3.weight: copying a param with shape torch.Size([91, 256]) from checkpoint, the shape in current model is torch.Size([2, 256]).
size mismatch for transformer.decoder.class_embed.3.bias: copying a param with shape torch.Size([91]) from checkpoint, the shape in current model is torch.Size([2]).
size mismatch for transformer.decoder.class_embed.4.weight: copying a param with shape torch.Size([91, 256]) from checkpoint, the shape in current model is torch.Size([2, 256]).
size mismatch for transformer.decoder.class_embed.4.bias: copying a param with shape torch.Size([91]) from checkpoint, the shape in current model is torch.Size([2]).
size mismatch for transformer.decoder.class_embed.5.weight: copying a param with shape torch.Size([91, 256]) from checkpoint, the shape in current model is torch.Size([2, 256]).
size mismatch for transformer.decoder.class_embed.5.bias: copying a param with shape torch.Size([91]) from checkpoint, the shape in current model is torch.Size([2]).
size mismatch for transformer.enc_out_class_embed.weight: copying a param with shape torch.Size([91, 256]) from checkpoint, the shape in current model is torch.Size([2, 256]).
size mismatch for transformer.enc_out_class_embed.bias: copying a param with shape torch.Size([91]) from checkpoint, the shape in current model is torch.Size([2]).
size mismatch for label_enc.weight: copying a param with shape torch.Size([92, 256]) from checkpoint, the shape in current model is torch.Size([3, 256]).
size mismatch for class_embed.0.weight: copying a param with shape torch.Size([91, 256]) from checkpoint, the shape in current model is torch.Size([2, 256]).
size mismatch for class_embed.0.bias: copying a param with shape torch.Size([91]) from checkpoint, the shape in current model is torch.Size([2]).
size mismatch for class_embed.1.weight: copying a param with shape torch.Size([91, 256]) from checkpoint, the shape in current model is torch.Size([2, 256]).
size mismatch for class_embed.1.bias: copying a param with shape torch.Size([91]) from checkpoint, the shape in current model is torch.Size([2]).
size mismatch for class_embed.2.weight: copying a param with shape torch.Size([91, 256]) from checkpoint, the shape in current model is torch.Size([2, 256]).
size mismatch for class_embed.2.bias: copying a param with shape torch.Size([91]) from checkpoint, the shape in current model is torch.Size([2]).
size mismatch for class_embed.3.weight: copying a param with shape torch.Size([91, 256]) from checkpoint, the shape in current model is torch.Size([2, 256]).
size mismatch for class_embed.3.bias: copying a param with shape torch.Size([91]) from checkpoint, the shape in current model is torch.Size([2]).
size mismatch for class_embed.4.weight: copying a param with shape torch.Size([91, 256]) from checkpoint, the shape in current model is torch.Size([2, 256]).
size mismatch for class_embed.4.bias: copying a param with shape torch.Size([91]) from checkpoint, the shape in current model is torch.Size([2]).
size mismatch for class_embed.5.weight: copying a param with shape torch.Size([91, 256]) from checkpoint, the shape in current model is torch.Size([2, 256]).
size mismatch for class_embed.5.bias: copying a param with shape torch.Size([91]) from checkpoint, the shape in current model is torch.Size([2]).

[Abdul-Rehman-Astro]

The command that i used was:

bash scripts/DINO_train.sh /home/abdul_rehman/scratch/DINO_object_detection_model_original/Dataset/COCODIR --pretrain_model_path /home/abdul_rehman/scratch/DINO_object_detection_model_original/Pretrained_weights/36_epoch_setting/checkpoint0011_4scale.pth --finetune_ignore label_enc.weight

[CoolinYu]

I have met the same problem with you. I trained dino_4_scale model on HRSC2016 dataset and i have already changed num_classes and dn_labebook_size to 34, still it returned error as "size mismatch for label_enc.weight: copying a param with shape torch.Size([92, 256]) from checkpoint, the shape in current model is torch.Size([35, 256])."

If you have a solution, please let me know. I would be very grateful.