hafsteinn/icelandic-modernbert-ws-E-xl-s0

Fill-Mask • 1.0B • Updated about 4 hours ago

Note Best of the 27 encoders compared in the paper. 1B, warm-started from English Ettin (ettin-encoder-1b), vocabulary transplanted. Warm-start study, arm Ettin, seed 0; main = 80 tokens/param; branches b10, b20, b40.

hafsteinn/icelandic-modernbert-ws-S-xl-s0

Fill-Mask • 1.0B • Updated about 4 hours ago

Note 1B, trained from scratch. Warm-start study, arm Scratch, seed 0; main = 80 tokens/param.

hafsteinn/icelandic-modernbert-large-160tpp

Fill-Mask • 0.4B • Updated about 7 hours ago

Note large (378M), trained from scratch on Icelandic, 160 tokens/param (with CulturaX). Size ladder.

hafsteinn/icelandic-modernbert-large

Fill-Mask • 0.4B • Updated about 7 hours ago

Note large (378M), trained from scratch on Icelandic, 80 tokens/param. Size ladder.

hafsteinn/icelandic-modernbert-base-160tpp

Fill-Mask • 0.1B • Updated about 7 hours ago

Note base (136M), trained from scratch on Icelandic, 160 tokens/param (with CulturaX). Size ladder.

hafsteinn/icelandic-modernbert-base

Fill-Mask • 0.1B • Updated about 7 hours ago

Note base (136M), trained from scratch on Icelandic, 80 tokens/param. Size ladder.

hafsteinn/icelandic-modernbert-m-160tpp

Fill-Mask • 59.4M • Updated about 7 hours ago

Note m (59M), trained from scratch on Icelandic, 160 tokens/param (with CulturaX). Size ladder.

hafsteinn/icelandic-modernbert-m

Fill-Mask • 59.4M • Updated about 7 hours ago

Note m (59M), trained from scratch on Icelandic, 80 tokens/param. Size ladder.

hafsteinn/icelandic-modernbert-s-160tpp

Fill-Mask • 25.3M • Updated about 7 hours ago

Note s (25M), trained from scratch on Icelandic, 160 tokens/param (with CulturaX). Size ladder.

hafsteinn/icelandic-modernbert-s

Fill-Mask • 25.3M • Updated about 7 hours ago

Note s (25M), trained from scratch on Icelandic, 80 tokens/param. Size ladder.

hafsteinn/icelandic-modernbert-xs-160tpp

Fill-Mask • 12.4M • Updated about 7 hours ago

Note xs (12M), trained from scratch on Icelandic, 160 tokens/param (with CulturaX). Size ladder.

hafsteinn/icelandic-modernbert-xs

Fill-Mask • 12.4M • Updated about 7 hours ago

Note xs (12M), trained from scratch on Icelandic, 80 tokens/param. Size ladder.

hafsteinn/icelandic-modernbert-ws-E-large-s0

Fill-Mask • 0.4B • Updated about 4 hours ago

Note large (378M), warm-started from English Ettin (ettin-encoder-400m), vocabulary transplanted. Warm-start study, arm Ettin, seed 0; main = 80 tokens/param; branches b10, b20, b40.

hafsteinn/icelandic-modernbert-ws-E-base-s0

Fill-Mask • 0.1B • Updated about 4 hours ago

Note base (136M), warm-started from English Ettin (ettin-encoder-150m), vocabulary transplanted. Warm-start study, arm Ettin, seed 0; main = 80 tokens/param; branches b10, b20, b40.

hafsteinn/icelandic-modernbert-ws-E-m-s0

Fill-Mask • 59.4M • Updated about 4 hours ago

Note m (59M), warm-started from English Ettin (ettin-encoder-68m), vocabulary transplanted. Warm-start study, arm Ettin, seed 0; main = 80 tokens/param; branches b10, b20, b40.

hafsteinn/icelandic-modernbert-ws-E-m-s1

Fill-Mask • 59.4M • Updated about 4 hours ago

Note m (59M), warm-started from English Ettin (ettin-encoder-68m), vocabulary transplanted. Warm-start study, arm Ettin, seed 1; main = 80 tokens/param; branches b10, b20, b40.

hafsteinn/icelandic-modernbert-ws-E-s-s0

Fill-Mask • 25.3M • Updated about 4 hours ago

Note s (25M), warm-started from English Ettin (ettin-encoder-32m), vocabulary transplanted. Warm-start study, arm Ettin, seed 0; main = 80 tokens/param; branches b10, b20, b40.

hafsteinn/icelandic-modernbert-ws-E-s-s1

Fill-Mask • 25.3M • Updated about 4 hours ago

Note s (25M), warm-started from English Ettin (ettin-encoder-32m), vocabulary transplanted. Warm-start study, arm Ettin, seed 1; main = 80 tokens/param; branches b10, b20, b40.

hafsteinn/icelandic-modernbert-ws-E-xs-s0

Fill-Mask • 12.4M • Updated about 4 hours ago

Note xs (12M), warm-started from English Ettin (ettin-encoder-17m), vocabulary transplanted. Warm-start study, arm Ettin, seed 0; main = 80 tokens/param; branches b10, b20, b40.

hafsteinn/icelandic-modernbert-ws-E-xs-s1

Fill-Mask • 12.4M • Updated about 4 hours ago

Note xs (12M), warm-started from English Ettin (ettin-encoder-17m), vocabulary transplanted. Warm-start study, arm Ettin, seed 1; main = 80 tokens/param; branches b10, b20, b40.

hafsteinn/icelandic-modernbert-ws-Mswap-base-s0

Fill-Mask • 0.1B • Updated about 4 hours ago

Note base (136M), warm-started from mmBERT-base, vocabulary transplanted. Warm-start study, arm mmBERT-transplant, seed 0; main = 80 tokens/param; branches b10, b20, b40.

hafsteinn/icelandic-modernbert-ws-Mkeep-base-s0

Fill-Mask • 0.3B • Updated about 4 hours ago

Note base (136M), mmBERT-base continued with its own 256k vocabulary. Warm-start study, arm mmBERT-keep, seed 0; main = 80 tokens/param; branches b10, b20, b40.

hafsteinn/icelandic-modernbert-ws-Erand-m-s0

Fill-Mask • 59.4M • Updated about 4 hours ago

Note m (59M), Ettin's layers (ettin-encoder-68m) with new embeddings. Warm-start study, arm Ettin-body, seed 0; main = 80 tokens/param; branches b10, b20, b40.

hafsteinn/icelandic-modernbert-ws-Erand-s-s0

Fill-Mask • 25.3M • Updated about 4 hours ago

Note s (25M), Ettin's layers (ettin-encoder-32m) with new embeddings. Warm-start study, arm Ettin-body, seed 0; main = 80 tokens/param; branches b10, b20, b40.

hafsteinn/icelandic-modernbert-ws-Erand-xs-s0

Fill-Mask • 12.4M • Updated about 4 hours ago

Note xs (12M), Ettin's layers (ettin-encoder-17m) with new embeddings. Warm-start study, arm Ettin-body, seed 0; main = 80 tokens/param; branches b10, b20, b40.

hafsteinn/icelandic-modernbert-ws-S-base-s0

Fill-Mask • 0.1B • Updated about 4 hours ago

Note base (136M), trained from scratch. Warm-start study, arm Scratch, seed 0; main = 80 tokens/param; branches b10, b20, b40.

hafsteinn/icelandic-modernbert-ws-S-m-s0

Fill-Mask • 59.4M • Updated about 4 hours ago

Note m (59M), trained from scratch. Warm-start study, arm Scratch, seed 0; main = 80 tokens/param; branches b10, b20, b40.

hafsteinn/icelandic-modernbert-ws-S-m-s1

Fill-Mask • 59.4M • Updated about 4 hours ago

Note m (59M), trained from scratch. Warm-start study, arm Scratch, seed 1; main = 80 tokens/param; branches b10, b20, b40.

hafsteinn/icelandic-modernbert-ws-S-s-s0

Fill-Mask • 25.3M • Updated about 4 hours ago

Note s (25M), trained from scratch. Warm-start study, arm Scratch, seed 0; main = 80 tokens/param; branches b10, b20, b40.

hafsteinn/icelandic-modernbert-ws-S-s-s1

Fill-Mask • 25.3M • Updated about 4 hours ago

Note s (25M), trained from scratch. Warm-start study, arm Scratch, seed 1; main = 80 tokens/param; branches b10, b20, b40.

hafsteinn/icelandic-modernbert-ws-S-xs-s0

Fill-Mask • 12.4M • Updated about 4 hours ago

Note xs (12M), trained from scratch. Warm-start study, arm Scratch, seed 0; main = 80 tokens/param; branches b10, b20, b40.

hafsteinn/icelandic-modernbert-ws-S-xs-s1

Fill-Mask • 12.4M • Updated about 4 hours ago

Note xs (12M), trained from scratch. Warm-start study, arm Scratch, seed 1; main = 80 tokens/param; branches b10, b20, b40.