Inquiry about Pretraining on Unlabeled Text for Experiments on Civil Comments Dataset and Congratulations on Article Acceptance

[ioslide]

I am wondering if, when conducting experiments on the Civil Comments dataset, it is necessary to follow the code provided by Wilds and first pretrain the model on unlabeled text before fine-tuning it on labeled data.