Quantitative metrics in the paper

[wzk1015]

Hi, I try to reproduce your experiment result of Training Set in Table 1. I use PCU, ISR, PRS from muspy and APS, PR, TUP, IOI from mgeval. I run these metrics on the whole Maestro-V1 dataset (2369 songs), but four of them are somewhat different from the paper, so I wonder where is wrong of my implementation.

Metric	Source	Name	Result in paper	Result I reproduced
PCU	muspy	muspy.n_pitch_classes_used	7.81	11.98
PRS	muspy	muspy.polyphony_rate (threshold=3)	0.40	0.16
PR	mgeval	pitch_range	67.98	73.60
TUP	mgeval	total_used_pitch	65.76	72.27

I wonder if I am using the correct methods in mgeval and muspy? Or did you preprocess the data in some way?

Also, are you planning to release the code for evaluation in Table 1?

Thanks for your help!

[li-car-fei]

Hi, I try to reproduce your experiment result of Training Set in Table 1. I use PCU, ISR, PRS from muspy and APS, PR, TUP, IOI from mgeval. I run these metrics on the whole Maestro-V1 dataset (2369 songs), but four of them are somewhat different from the paper, so I wonder where is wrong of my implementation.

Metric Source Name Result in paper Result I reproduced
PCU muspy muspy.n_pitch_classes_used 7.81 11.98
PRS muspy muspy.polyphony_rate (threshold=3) 0.40 0.16
PR mgeval pitch_range 67.98 73.60
TUP mgeval total_used_pitch 65.76 72.27
I wonder if I am using the correct methods in mgeval and muspy? Or did you preprocess the data in some way?

Also, are you planning to release the code for evaluation in Table 1?

Thanks for your help!

Please, have you solved this problem?

[wzk1015]

Hi, I try to reproduce your experiment result of Training Set in Table 1. I use PCU, ISR, PRS from muspy and APS, PR, TUP, IOI from mgeval. I run these metrics on the whole Maestro-V1 dataset (2369 songs), but four of them are somewhat different from the paper, so I wonder where is wrong of my implementation.
Metric Source Name Result in paper Result I reproduced
PCU muspy muspy.n_pitch_classes_used 7.81 11.98
PRS muspy muspy.polyphony_rate (threshold=3) 0.40 0.16
PR mgeval pitch_range 67.98 73.60
TUP mgeval total_used_pitch 65.76 72.27
I wonder if I am using the correct methods in mgeval and muspy? Or did you preprocess the data in some way?
Also, are you planning to release the code for evaluation in Table 1?
Thanks for your help!

Please, have you solved this problem?

Actually no. Stop working on this project after that:)

[li-car-fei]

Hi, I try to reproduce your experiment result of Training Set in Table 1. I use PCU, ISR, PRS from muspy and APS, PR, TUP, IOI from mgeval. I run these metrics on the whole Maestro-V1 dataset (2369 songs), but four of them are somewhat different from the paper, so I wonder where is wrong of my implementation.
Metric Source Name Result in paper Result I reproduced
PCU muspy muspy.n_pitch_classes_used 7.81 11.98
PRS muspy muspy.polyphony_rate (threshold=3) 0.40 0.16
PR mgeval pitch_range 67.98 73.60
TUP mgeval total_used_pitch 65.76 72.27
I wonder if I am using the correct methods in mgeval and muspy? Or did you preprocess the data in some way?
Also, are you planning to release the code for evaluation in Table 1?
Thanks for your help!

Please, have you solved this problem?

Actually no. Stop working on this project after that:)

I tested with muspy library on other datasets. muspy library metrics calculations should be accurate.