Questions about whether to use additional datasets

[challenge-my]

Hi Kevin,

I am very interested in your great work at DCASE2023.Therefore,I have some questions . Did you use GPU for training in your TensorFlow code? I got an error during the training process. Have you encountered this before? How did you handle it? Thank you. Here is the error message and error code.

Thank you!
Olaf

[wilkinghoff]

Hi Olaf,

Thank you for your interest! Yes, I used GPUs for training.

It seems that something went wrong during training. Maybe the scale parameter of the loss function somehow blew up. Sometimes I also observed this. Does this error occur every time you are training?

If you want you can try to replace the loss with the AdaProj loss, which is numerically much more stable: https://github.com/wilkinghoff/AdaProj

Best,
Kevin

[challenge-my]

First of all, thank you for your advice. I will run the code in the link you sent on my server. I have been encountering the above problem for about 8 times. Secondly, can I ask two questions? In the ./eval_data directory, I put the additional dataset and the evaluation dataset together according to the machine type, and then put the development dataset in the ./dev_data directory. I want to confirm whether this is correct? Thank you

[challenge-my]

Hi Kevin,
It still doesn't work. I ran the code repeatedly for more than ten times, and it still reported the above error. Do you have any good suggestions? Thank you

[wilkinghoff]

First of all, thank you for your advice. I will run the code in the link you sent on my server. I have been encountering the above problem for about 8 times. Secondly, can I ask two questions? In the ./eval_data directory, I put the additional dataset and the evaluation dataset together according to the machine type, and then put the development dataset in the ./dev_data directory. I want to confirm whether this is correct? Thank you

Hi Olaf,

yes, I think so. You should have a structure like this:
/dev_data/machine_type1/train/file42
or
/eval_data/machine_type3/test/file66

Best,
Kevin

[wilkinghoff]

Hi Kevin, It still doesn't work. I ran the code repeatedly for more than ten times, and it still reported the above error. Do you have any good suggestions? Thank you

Hi Olaf,

have you deleted the stored network weights between runs?

To make sure that the scale parameter is really the issue, you can replace line 41 in subcluster_adacos.py with
"if training:" --> "if 0:"

Best,
Kevin

[challenge-my]

Hi Kevin,
After I deleted the stored model weights, I found that the numerical error above was solved, but there was a type error. I tried many times but couldn't solve it. Have you encountered this problem? Thank you

[challenge-my]

Hi Kevin,
I would like to thank you for sharing your code. I encountered some difficulties when trying to run it locally and was unable to get it to work properly. To better proceed with my work, I was wondering if you could send me the version that is running successfully on your machine, which would help me better understand and resolve the issues. My email address is lixiang1@stu.xmu.edu.cn. I would greatly appreciate your help.

Additionally, I will make sure to cite your work wherever I use your code, acknowledging your contribution.

Thank you again for your support！

Best regards,
Olaf

[wilkinghoff]

Hi Kevin, After I deleted the stored model weights, I found that the numerical error above was solved, but there was a type error. I tried many times but couldn't solve it. Have you encountered this problem? Thank you

Hi Olaf,

that's good! I sometimes noticed this error when storing/re-loading the extracted features to hard disk within the same run. You can either just re-run the script after the features are already stored or only store them in RAM to solve the issue.

Best,
Kevin

[wilkinghoff]

Hi Kevin, I would like to thank you for sharing your code. I encountered some difficulties when trying to run it locally and was unable to get it to work properly. To better proceed with my work, I was wondering if you could send me the version that is running successfully on your machine, which would help me better understand and resolve the issues. My email address is lixiang1@stu.xmu.edu.cn. I would greatly appreciate your help.

Additionally, I will make sure to cite your work wherever I use your code, acknowledging your contribution.

Thank you again for your support！

Best regards, Olaf

Hi Olaf,

Sorry, but I cannot provide you any other code as this is the version I have been running on my machine.

Best,
Kevin

[challenge-my]

Hi Kevin, After I deleted the stored model weights, I found that the numerical error above was solved, but there was a type error. I tried many times but couldn't solve it. Have you encountered this problem? Thank you

Hi Olaf,

that's good! I sometimes noticed this error when storing/re-loading the extracted features to hard disk within the same run. You can either just re-run the script after the features are already stored or only store them in RAM to solve the issue.

Best, Kevin

Thank you very much，Kevin. Best wishes

[challenge-my]

Hi Kevin, I would like to thank you for sharing your code. I encountered some difficulties when trying to run it locally and was unable to get it to work properly. To better proceed with my work, I was wondering if you could send me the version that is running successfully on your machine, which would help me better understand and resolve the issues. My email address is lixiang1@stu.xmu.edu.cn. I would greatly appreciate your help.
Additionally, I will make sure to cite your work wherever I use your code, acknowledging your contribution.
Thank you again for your support！
Best regards, Olaf

Hi Olaf,

Sorry, but I cannot provide you any other code as this is the version I have been running on my machine.

Best, Kevin

Thank you~

[challenge-my]

Hi，Kevin
First of all, thank you for your answer above. Secondly, I have fixed the error above, but I encountered a problem when I ran the code again. I would be very grateful if you could tell me what this folder ('./dcase2023_task2_evaluator-main/ground_truth_data/ground_truth_') is for and where the files inside it come from? How should I get these files? You can see the red part in the picture for the specific directory and code. Thank you

Best, Olaf

[wilkinghoff]

Hi Olaf,
Happy that I can help. You can find the files in this official repo: https://github.com/nttcslab/dcase2023_task2_evaluator
Best,
Kevin

[challenge-my]

Hi，Kevin
Thanks to your help, I finally ran the results, but my results on the evaluation set are very poor, basically only about 50%, far less than the accuracy in your paper, some results are even about 20% lower than yours. What do you think is the reason? Is it the parameters? For example, bachsize or not enough rounds of training? Thank you

Best，Olaf

[wilkinghoff]

Hi Olaf,

the results for the evaluation set are completely random and the reason is that the audio recordings and label files do not correspond to each other because they are not sorted correctly. This depends on your operating system. You can try some sorting of the file lists before loading (both for the files themselves and the corresponding labels).

Best,
Kevin