Adding emotion tag to VoiceCraft

[mukund0911]

I'm interested in contributing to VoiceCraft by adding emotion control functionality. My goal is to enable the model to generate audio with a specified emotion while cloning a voice from a reference audio. I have a few questions about implementing this feature:

1. I'm considering using the Emotional Speech Dataset (ESD) for training. Is this dataset suitable, or would you recommend alternatives?

2. Should the loss function be modified to account for the emotion tag?

3. The paper mentions that training VoiceCraft took around 240 hours. Would implementing this new feature require a similar training time, or could it be done more efficiently?

As someone new to open-source contributions, I'd appreciate any guidance on how to proceed with this feature addition. Thank you for your help!

[jasonppy]

1. the dataset sounds good
2. this is something worth exploring
3. would need exp to see. making it more efficient is a good research/engineering problem