jvonrad/alignment-without-transfer
Updated
Alignment without Transfer: Tokenization Impacts Cross-lingual Representations during Multilingual Language Model Pretraining (paper forthcoming)
Note ~1B models trained from scratch under 4 tokenizers (2/5/20/419 languages): 164 checkpoints, tokenizers
Note Cached per-layer FLORES+ sentence embeddings for the MEXA alignment analysis
Note Data provenance: pool file lists, shard hashes, in-domain holdout text