khalidmokha/lexibert-keyboard

LexiBERT keyboard autocorrection

Files for correcting words typed on a Finnish or Russian keyboard in 15 Uralic languages, as in the keyboard experiment of the paper: one word-frequency list per language and the weights of the ranking. The ranking uses the LexiBERT encoder and the FSTs of UralicNLP.

• lexicons/<lang>.tsv.gz: the words of our corpora with their frequencies (word, tab, count), most frequent first. They contain words only, no running text.
• config.json: the weights of the score.

The candidates of a typed word are the word itself, its versions with the letters that are missing from the keyboard restored (e.g. Komi-Zyrian о → ӧ), and the words within one edit, kept if they are in the lexicon or the FST accepts them (at most 24). Each candidate gets the score

0.1 * log p(candidate | preceding words) + 0.5 * log(1 + frequency) - 2.0 * edits

where the probability comes from LexiBERT, with the sub-words of the candidate masked one at a time. The weights were tuned on the UD training sentences.

Use

The implementation is in the demo of the code repository (demo/keyboard.py):

from huggingface_hub import snapshot_download
from transformers import AutoModelForMaskedLM, AutoTokenizer
from uralicNLP import uralicApi
from keyboard import Keyboard   # demo/keyboard.py

folder = snapshot_download("khalidmokha/lexibert-keyboard", allow_patterns=["config.json", "lexicons/myv.*"])
kb = Keyboard(folder, "myv", lambda w: uralicApi.analyze(w, "myv"))
tok = AutoTokenizer.from_pretrained("khalidmokha/lexibert")
model = AutoModelForMaskedLM.from_pretrained("khalidmokha/lexibert").eval()
best, candidates = kb.correct(tok, model, "Ютасть мазый шканзо", "лемзер")   # -> лёмзёр

Results

Accuracy on the UD test sentences of 10 languages, where one word at a time is corrupted (average over the languages):

System	Missing letters	Typos	Correct words kept	Unseen words
Frequency only	75.8	60.2	89.4	0.0
mmBERT-small + FST candidates	77.5	61.1	89.3	4.7
LexiBERT + FST candidates	78.8	66.7	92.7	13.1

Unseen words are corrupted words that never occur in our corpora, which only the FST candidates can correct.

Citation

@inproceedings{alnajjar-etal-2026-grounding,
  title     = {Grounding Encoders and Word Embeddings in {FST}s and Dictionaries for {U}ralic Languages},
  author    = {Alnajjar, Khalid and H{\"a}m{\"a}l{\"a}inen, Mika and Rueter, Jack and Partanen, Niko},
  booktitle = {Proceedings of the International Workshop on Computational Linguistics for Uralic Languages (IWCLUL 2026)},
  year      = {2026}
}

Code: https://github.com/mokha/lexibert