Inquiry about Data Augmentation Strategy in Training Process

[YuZheng-DT]

Hi team,
I read in your paper that the data augmentation method used during training follows the approach described in the MagNetO paper. Could you please clarify whether you strictly adhered to the noise addition strategy as outlined in the original MagNetO paper?
Additionally, I would like to know if, during the second stage of training, other types of noise and compression artifacts were also incorporated similar to the method in MagNetO.
Thank you for your attention, and I look forward to your response!

[vanIvan]

Hello! Thank you for your interest in our work, augmentation strategy is used directly from wespeaker pipeline, here is example config:

dataset_args:
  # the sample number which will be traversed within one epoch, if the value equals to 0,
  # the utterance number in the dataset will be used as the sample_num_per_epoch.
  sample_num_per_epoch: 0
  shuffle: True
  shuffle_args:
    shuffle_size: 2500
  filter: True
  filter_args:
    min_num_frames: 100
    max_num_frames: 800
  resample_rate: 16000
  speed_perturb: True
  num_frms: 200
  aug_prob: 0.6 # prob to add reverb & noise aug per sample
  fbank_args:
    num_mel_bins: 72
    frame_shift: 10
    frame_length: 25
    dither: 1.0
  spec_aug: False
  spec_aug_args:
    num_t_mask: 1
    num_f_mask: 1
    max_t: 10
    max_f: 8
    prob: 0.6

[YuZheng-DT]

Hello! Thank you for your interest in our work, augmentation strategy is used directly from wespeaker pipeline, here is example config:

dataset_args:
  # the sample number which will be traversed within one epoch, if the value equals to 0,
  # the utterance number in the dataset will be used as the sample_num_per_epoch.
  sample_num_per_epoch: 0
  shuffle: True
  shuffle_args:
    shuffle_size: 2500
  filter: True
  filter_args:
    min_num_frames: 100
    max_num_frames: 800
  resample_rate: 16000
  speed_perturb: True
  num_frms: 200
  aug_prob: 0.6 # prob to add reverb & noise aug per sample
  fbank_args:
    num_mel_bins: 72
    frame_shift: 10
    frame_length: 25
    dither: 1.0
  spec_aug: False
  spec_aug_args:
    num_t_mask: 1
    num_f_mask: 1
    max_t: 10
    max_f: 8
    prob: 0.6

Thank for your replay！We have successfully reproduced the results of ReDimNet-B2, but encountered difficulties in reproducing B5 and B6, especially during the second training stage, where their performance improvements are not as significant as reported.
Upon revisiting your paper, we noticed that the noise augmentation method used differs from WeSpeaker’s default implementation. We suspect this might be the reason for the discrepancy.
Could you please clarify whether the results in your paper were obtained using the exact same noise augmentation strategy as implemented in WeSpeaker?
Thank you in advance for your help!

[vanIvan]

@YuZheng-DT to be honest, reported results for b5 and b6 are cherry-picked, we did use not the most fair way of selecting checkpoints: we were selecting best checkpoints for this large scales based on vox1-O metrics, cause we found late training divergence/overfitting and we were not able to fix them in previous work.

We are releasing right now ReDimNet v2: https://github.com/PalabraAI/redimnet2 , we are trying to cope with the same problem for newer version of ReDimNet (late stage overfitting), we'll update when updated more stable training code and configs would be present for ReDimNet v2 b5,b6 scales.

So the main problem should be not in difference in augmentations strategy with original wespeaker's ones, but with late stage overfitting for larger models, especially attention 1d blocks - we almost pin pointed the cause of this over fitting and working on solving it.

[YuZheng-DT]

@YuZheng-DT to be honest, reported results for b5 and b6 are cherry-picked, we did use not the most fair way of selecting checkpoints: we were selecting best checkpoints for this large scales based on vox1-O metrics, cause we found late training divergence/overfitting and we were not able to fix them in previous work.

We are releasing right now ReDimNet v2: https://github.com/PalabraAI/redimnet2 , we are trying to cope with the same problem for newer version of ReDimNet (late stage overfitting), we'll update when updated more stable training code and configs would be present for ReDimNet v2 b5,b6 scales.

So the main problem should be not in difference in augmentations strategy with original wespeaker's ones, but with late stage overfitting for larger models, especially attention 1d blocks - we almost pin pointed the cause of this over fitting and working on solving it.

Thank you very much for your reply!
I noticed those parameters in the configuration file, which made me wonder whether the issue was related to the noise augmentation setup.
If it’s confirmed that the augmentation indeed follows the default WeSpeaker configuration, that clarifies my question.