pretrained BERT checkpoint file

[zzingae]

Hello.

I am looking for "../BERT/checkpoint-1969000" checkpoint file, but I cannot find it.
Could you tell me where I can find it?

Thank you very much!

[MarcahuasiTraveler]

me too! did you manage to solve this?

[zzingae]

I just trained BERT on my side from scratch.

[MarcahuasiTraveler]

Hi, how many steps did you manage to train on BERT? and did you trained with the complete dataset, a fraction of it, or another one? Did you get generation results yet?

[zzingae]

Hi, I trained BERT for 187,000 steps with the complete dataset (maestro_magenta_s5_t3).

And evaluation result of BERT was loss = 0.9930851551848399, perplexity = tensor(2.6996) on test set.

But I paused experimenting transformer-gan (this repo) due to heavy computational cost. Instead, I am trying simpler models for music generation like 'Compound Word Transformer'.

[MarcahuasiTraveler]

Hi zzingae, did you find any simpler models for generating MIDI sequences with success and with lower computational cost?