Update README.md
Browse files
README.md
CHANGED
|
@@ -95,7 +95,7 @@ nor has it been tested on other downstream tasks.
|
|
| 95 |
Note that during pretraining, the --mlm-prob parameter has been set to 0.4. We have done some experiments with different mlm-probs (namely, 0.15, 0.3 and 0.4) and found that
|
| 96 |
0.4 yielded the best results, in accordance to [Wettig et al., 2023](https://aclanthology.org/2023.eacl-main.217.pdf).
|
| 97 |
|
| 98 |
-
Estimated time it has cost to train:
|
| 99 |
|
| 100 |
pretraining: 38 hours on 4 NVIDIA H200s
|
| 101 |
finetuning: 4 hours on 1 NVIDIA H200
|
|
|
|
| 95 |
Note that during pretraining, the --mlm-prob parameter has been set to 0.4. We have done some experiments with different mlm-probs (namely, 0.15, 0.3 and 0.4) and found that
|
| 96 |
0.4 yielded the best results, in accordance to [Wettig et al., 2023](https://aclanthology.org/2023.eacl-main.217.pdf).
|
| 97 |
|
| 98 |
+
**Estimated time it has cost to train:**
|
| 99 |
|
| 100 |
pretraining: 38 hours on 4 NVIDIA H200s
|
| 101 |
finetuning: 4 hours on 1 NVIDIA H200
|