---
language: [myv, mdf, kpv, koi, udm, sme, sms, smn, olo, krl, vep, liv, mhr, mrj, yrk, en, fi, ru]
license: mit
library_name: gensim
tags: [uralic, word-embeddings, fasttext, static-embeddings, fst, cross-lingual]
---

# MorphEmb: word embeddings for Uralic languages

Static word vectors (384 dimensions) for 15 Uralic languages, English, Finnish and Russian, all in one vector space.
The vector of a word is the average of its character n-gram vectors (n = 3–6) plus the average over its FST analyses
of the lemma vector and the average tag vector, so all inflected forms of a word share its lemma vector. The vectors
were distilled from the contextual vectors of [LexiBERT](https://huggingface.co/khalidmokha/lexibert) (layers 8–16),
and because the same encoder is used for every language, the languages share one space.

## Precomputed vectors

`vectors/<lang>.vec.gz` has vectors for the most frequent words of our corpora and the words that the FST can
analyse (fastText text format). For words that the FST
does not know, the analysis of the [unknown word model](https://huggingface.co/khalidmokha/lexibert-unknown-words) was
used when its probability was at least 0.5.

```python
from gensim.models import KeyedVectors

myv = KeyedVectors.load_word2vec_format("vectors/myv.vec.gz")
myv.most_similar("кудо")
```

## Vectors for any word form

`config.json`, `shared.safetensors` (character n-gram and tag vectors) and `lemmas/<lang>.{safetensors,txt}` (the
lemma vectors of one language) build the vector of any word that the FST can analyse. They are read
by the `load_morphemb` function of [MLTools](https://github.com/mikahama/MLTools) with the FSTs of UralicNLP, which
downloads only the files of the languages you ask for:

```python
# pip install git+https://github.com/mikahama/MLTools uralicNLP
from MLTools.LM import word_embeddings

model = word_embeddings.load_morphemb("khalidmokha/morphemb", languages=["myv", "fin"])
model.embed("кудосо", "myv")
model.similarity("кудо", "talo", "myv", "fin")
model.nearest_neighbors("кудо", "myv", n=5, target_lang="fin")
```

The tables are stored in float16; the vectors are the same as those of the trained model in float32 up to rounding.

| Language | Words in `vectors/` | Lemmas |
|---|---|---|
| Erzya (myv) | 262,778 | 58,317 |
| Moksha (mdf) | 184,682 | 54,874 |
| Komi-Zyrian (kpv) | 217,983 | 47,547 |
| Komi-Permyak (koi) | 226,284 | 42,569 |
| Udmurt (udm) | 305,257 | 51,780 |
| North Sámi (sme) | 283,508 | 62,755 |
| Skolt Sámi (sms) | 103,972 | 30,830 |
| Inari Sámi (smn) | 150,108 | 43,782 |
| Livvi (olo) | 220,881 | 45,074 |
| Karelian (krl) | 203,161 | 29,351 |
| Veps (vep) | 187,383 | 36,439 |
| Livonian (liv) | 17,902 | 7,716 |
| Meadow Mari (mhr) | 281,678 | 69,009 |
| Hill Mari (mrj) | 167,356 | 54,811 |
| Nenets (yrk) | 50,224 | 13,235 |
| English (eng) | 170,110 | 137,192 |
| Finnish (fin) | 200,000 | 175,526 |
| Russian (rus) | 200,000 | 174,212 |

## Results

In a BiLSTM parser with frozen word vectors (3 seeds), MorphEmb reaches 60.6 LAS on Erzya and 67.7 on North Sámi,
close to fastText vectors trained on the same corpora (60.7 and 68.3). When one parser is trained for all of the
Uralic treebanks, the average LAS is 54.1 with MorphEmb and 32.5 with fastText, as the fastText vectors of different
languages are not in the same space. As candidates for translation prediction in Ve'rdd, the 20 closest words of
MorphEmb raise Hits@1 from 19.2% to 23.6% on average.

## Citation

```bibtex
@inproceedings{alnajjar-etal-2026-grounding,
  title     = {Grounding Encoders and Word Embeddings in {FST}s and Dictionaries for {U}ralic Languages},
  author    = {Alnajjar, Khalid and H{\"a}m{\"a}l{\"a}inen, Mika and Rueter, Jack and Partanen, Niko},
  booktitle = {Proceedings of the International Workshop on Computational Linguistics for Uralic Languages (IWCLUL 2026)},
  year      = {2026}
}
```

Code: <https://github.com/mokha/lexibert> · All models and seeds: <https://doi.org/10.5281/zenodo.23545486>.