Test phase error (tuple index out of index)

[SooYoungYang]

Hi,

I tried running your code on my custom dataset. Training works well, but I keep encountering a "tuple index out of range" error during the test phase at the following line of code:
representative_embeddings = [text_encoder(ids.to(device))[0] for ids in data["caption_ids_concept_token"][top_idx]]

To ensure that I didn't make any mistakes in my data loading process, I ran the code with the CUB dataset you provided, but I encountered the same error during the test phase.

I noticed in the GitHub README that you suggest saving the models in ckpts/cub during training, but for testing, you load from the ckpts/cub983 path. Could you explain why this discrepancy exists?

Also, do you have any insights into why this error might be occurring?

Thank you

[callsys]

1、It seems that you are using the top-1 inference mode, which requires the classification result on your custom dataset (You have to provide it yourself). Maybe you can use the gtk inference mode, which requires no classification result and use the ground-truth class for localization.
2、ckpts/cub983 is our provided checkpoint path. If you have trained your own checkpoint, replace it with your own path!

[SooYoungYang]

Thank you for the quick response.

However, I am still encountering the same error.
I followed the instructions in the GitHub README and ran the following commands for training:
python main.py --function train_token --config configs/cub.yml --opt "{'train': {'save_path': 'ckpts/cub/'}}"
python main.py --function train_unet --config configs/cub_stage2.yml --opt "{'train': {'load_token_path': 'ckpts/cub/tokens/', 'save_path': 'ckpts/cub/'}}"
The training completed successfully, and the files were generated as shown in the attached image.

And i started testing with this command : python main.py --function test --config configs/cub_stage2.yml --opt "{'test' : {'load_token_path' : 'ckpts/cub/tokens/', 'load_unet_path' : 'ckpts/cub/unet_s239/', 'save_log_path' : 'ckpts/cub/log.txt'}}'

(with 'gtk' mode)

but still getting same problem.

In test code,
representative_embeddings = [text_encoder(ids.to(device))[0] for ids in data["caption_ids_concept_token"][-1]]
this code is problem in my case.

In train code, there is similar part to this code, but it was working well( since i successfully finished training.)
combine_embeddings = text_encoder(data["caption_ids_concept_token"])[0]

So, I looked into this issue thoroughly. In the training phase, the shape of data["caption_ids_concept_token"] that goes into text_encoder is [4, 77], where 4 is the training batch size. However, during testing, when ids.to(device) is passed into text_encoder, it is being processed 1 id at a time. As a result, its shape is [], which seems to be causing the problem.

[callsys]

I can't locate the problem at the moment. You can download another unmodified copy of the code and use the provided checkpoint to inference, and then compare the differences to find the problem.