Hi ! I enjoyed reading your research paper on the Bert-based lexical substitution.
Can you confirm this code is the implementation of the paper?
One thing I'm very curious about is whether the BERT generates (you used the term "propose") substitute candidates that are semantically close to the target word (this is very interesting), by using "embedding dropout", and evaluates those candidates.
Am I right? If so, do you feed a masked sentence or just an original sentence without masking? Would there a reason not to use BertForMaskedLM, but just the Bert model?
Thank you and have a great day !
Cheers.
Hi ! I enjoyed reading your research paper on the Bert-based lexical substitution.
Can you confirm this code is the implementation of the paper?
One thing I'm very curious about is whether the BERT generates (you used the term "propose") substitute candidates that are semantically close to the target word (this is very interesting), by using "embedding dropout", and evaluates those candidates.
Am I right? If so, do you feed a masked sentence or just an original sentence without masking? Would there a reason not to use BertForMaskedLM, but just the Bert model?
Thank you and have a great day !
Cheers.