DanielGallagherIRE/ModernBERT-O3-Baseline
0.2B • Updated • 62
ModernBERT with Obfuscated Pre-Training
ModernBERT pre-trained from scratch on various obfuscated representations of the same 10B tokens.
-
-
DanielGallagherIRE/ModernBERT-O3-PMI-2.5
0.2B • Updated • 5 -
DanielGallagherIRE/ModernBERT-O3-PMI-5
0.2B • Updated • 4 -
DanielGallagherIRE/ModernBERT-O3-PMI-7.5
0.2B • Updated • 11 -
DanielGallagherIRE/ModernBERT-O3-PMI-10
0.2B • Updated • 10 -
DanielGallagherIRE/ModernBERT-O3-PMI-12.5
0.2B • Updated • 11 -
DanielGallagherIRE/ModernBERT-O3-Nouns-DEF
0.2B • Updated • 13 -
DanielGallagherIRE/ModernBERT-O3-Nouns-DEL
0.2B • Updated • 9 -
DanielGallagherIRE/ModernBERT-O3-Nouns-POS
0.2B • Updated • 15 -
DanielGallagherIRE/ModernBERT-O3-Shuffling-SENT
0.2B • Updated • 10 -
DanielGallagherIRE/ModernBERT-O3-Shuffling-DOC
0.2B • Updated • 12