I tested the b2_lmf model and found that the EER results I obtained are different from yours.

[zoyaqiao]

Hello, I tested the b2_lmf model and found that the EER results I obtained are different from yours. I saw in your paper that you tested using Voxceleb-clean. I conducted the test here, and the results are as follows:
O-clean: 0.4946
E-clean: 0.7592
H-clean: 1.4741

These results do not match the ones you provided.

[vanIvan]

hi @zoyaqiao, your code seems all right, except I'm not sure that during scoring using cosine similarity embeddings are normalized before. To save normalized embeddings you need to do:

xvec = F.normalize(xvec,p=2,dim=-1).cpu().numpy()

[zoyaqiao]

"Hello, I have already performed length normalization on the embeddings when calculating the EER, as I mentioned earlier. Could you let me know if there are any details I might have overlooked? Also, I would like to know the EER results for your PTN B2-Vox model on o-clean, e-clean, h-clean, o, e, and h."

[vanIvan]

I see, I've retested model, and I'm getting same results as you: O-clean: 0.4946. We need to figure out why me have this difference in metrics.

[zoyaqiao]

Hello, do you have records of the best results achieved by the b2-vox ptn model? Are these results publicly available for us to translate into English?

[vanIvan]

@zoyaqiao we retested models, and found out, that we've uploaded incorrect versions for b1 lm + b2 lm model sizes (probably ptn as well are not correct) - trained on AAM softmax instead of SphereFace2 loss, which gives better results, all other models (b0, b3, b4, b5, b6).

We'll try to recover correct models weights.

Answering:

do you have records of the best results achieved by the b2-vox ptn model? Are these results publicly available for us to translate into English?

No we don't, as it was not common practice to report pretrain results, I've tried to recover all evaluation information we had, and didn't found any reference of pretrains metrics. No, we didn't release this results anywhere.

[Aduomas]

@zoyaqiao we retested models, and found out, that we've uploaded incorrect versions for b1 lm + b2 lm model sizes (probably ptn as well are not correct) - trained on AAM softmax instead of SphereFace2 loss, which gives better results, all other models (b0, b3, b4, b5, b6).

We'll try to recover correct models weights.

Answering:

do you have records of the best results achieved by the b2-vox ptn model? Are these results publicly available for us to translate into English?

No we don't, as it was not common practice to report pretrain results, I've tried to recover all evaluation information we had, and didn't found any reference of pretrains metrics. No, we didn't release this results anywhere.

any update? did you upload updated models anywhere?

what about those using your models for a while, does this mean we have to recompute embeddings with the new models?

[vanIvan]

@Aduomas I'm sorry, there is no update.

any update? did you upload updated models anywhere?

we were not able to restore original weights for b2 model.

what about those using your models for a while, does this mean we have to recompute embeddings with the new models?

It depends your use case, we have released better quality multilingual models for b2 and b3 sizes, for general usage it's better to choose them:

model = torch.hub.load(
	"IDRnD/ReDimNet",
	"ReDimNet",
	model_name="S",
	train_type="ft_mix",
	dataset="vb2+vox2+cnc"
) # b2 like

model = torch.hub.load(
	"IDRnD/ReDimNet",
	"ReDimNet",
	model_name="M",
	train_type="ft_mix",
	dataset="vb2+vox2+cnc"
) # b3 like

Please note, that weights are distributed with licenses inherited from their train datasets, and for both datasets setups "vox2" and "vb2+vox2+cnc" the license is CC BY-NC-SA 4.0, that does not allow commercial use.