microsoft
/

deberta-v3-large

Model card Files Files and versions

DeBERTa commited on Oct 19, 2021

Commit

8d1abe6

·

1 Parent(s): 0b14b48

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -13,7 +13,7 @@ license: mit
 Please check the [official repository](https://github.com/microsoft/DeBERTa) for more details and updates.
-In DeBERTa V3 we replaced MLM objective with RTD(Replaced Token Detection) objective during pre-training, which significantly improves the model performance. Please check appendix A11 in our paper for more details.
 This is the DeBERTa V3 large model with 24 layers, 1024 hidden size. Total parameters is 418M while Embedding layer takes about 131M due to the usage of 128k vocabulary. It's trained with 160GB data.

 Please check the [official repository](https://github.com/microsoft/DeBERTa) for more details and updates.
+In DeBERTa V3 we replaced MLM objective with RTD(Replaced Token Detection) objective during pre-training, which significantly improves the model performance. Please check appendix A11 in our paper [DeBERTa](https://arxiv.org/abs/2006.03654) for more details.
 This is the DeBERTa V3 large model with 24 layers, 1024 hidden size. Total parameters is 418M while Embedding layer takes about 131M due to the usage of 128k vocabulary. It's trained with 160GB data.