Samdei Type Mini

Samdei Type Mini converts romanized Khmer, as people type it in chat, into Khmer script. It is a 7.1M-parameter character-level encoder-decoder, small enough for a phone keyboard: 7 MiB as int8 ONNX.

Input Output
knhom jong tov phsar αžαŸ’αž‰αž»αŸ†αž…αž„αŸ‹αž‘αŸ…αž•αŸ’αžŸαžΆαžš
ot mean wifi te αž’αžαŸ‹αž˜αžΆαž“ wifi αž‘αŸ

Release article: Samdei Type: open models that turn romanized Khmer into Khmer script, with a demo of this model in a chat.

Quick start

With transformers alone (pip install transformers torch):

from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

repo = "samdei/samdei-type-mini"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForSeq2SeqLM.from_pretrained(repo)

def pieces(text: str, max_words: int = 8) -> list[str]:
    """At most max_words words per piece, with a break after punctuation, as in the paper."""
    out, cur = [], []
    for w in text.split():
        cur.append(w)
        if len(cur) == max_words or w[-1] in ",.;:?!αŸ”":
            out.append(" ".join(cur))
            cur = []
    if cur:
        out.append(" ".join(cur))
    return out or [text]

def convert(text: str) -> str:
    batch = tok(pieces(text), padding=True, truncation=True, return_tensors="pt")
    out = model.generate(**batch)  # beam search, k = 5 (generation_config.json)
    return " ".join(tok.batch_decode(out, skip_special_tokens=True))

print(convert("knhom jong tov phsar"))  # αžαŸ’αž‰αž»αŸ†αž…αž„αŸ‹αž‘αŸ…αž•αŸ’αžŸαžΆαžš

This is beam search alone (CER 0.072 on Samdei Type Bench). For the best quality, rerank with the word LM of this repo (0.058), using the samdei-type package:

pip install "samdei-type[deploy]"
from samdei_type.pipeline import Converter

conv = Converter("samdei/samdei-type-mini")   # int8 ONNX + word-LM rerank, as in the paper
print(conv.convert("knhom jong tov phsar"))   # αžαŸ’αž‰αž»αŸ†αž…αž„αŸ‹αž‘αŸ…αž•αŸ’αžŸαžΆαžš

Or on the command line: samdei-type "knhom jong tov phsar".

Files

config.json, generation_config.json, model.safetensors (28.4 MiB), tokenizer*.json   fp32 model (transformers)
onnx/encoder_kv.int8.onnx (5.0 MiB), onnx/decoder_step.int8.onnx (2.0 MiB), onnx/vocab.json   int8 ONNX
lm/wordfull_nowiki_p25.json.gz (1.2 MiB)   word bigram LM for reranking
rerank.json                                decoding and rerank settings

On a phone, the two ONNX graphs are all a runtime needs. encoder_kv runs once per input. decoder_step decodes one token with a KV cache, and its batch axis holds all beams. samdei_type/onnx_infer.py is a plain-numpy reference to port from. On one Apple M5 Pro CPU thread, beam 5 takes 4.9 ms per input (p50); with the rerank, a keystroke takes 5.7 ms. It has not been timed on a phone yet.

Results

Samdei Type Bench: 2,155 everyday messages and words with human-checked Khmer. The metrics are CER after normalization and exact match.

System CER Exact
Samdei Type Mini, reranked 0.058 24%
Samdei Type Mini, beam only (the transformers example above) 0.072 17%
Samdei Type Base, reranked 0.052 30%
Samdei Type Prime (not released) 0.027 61%
Akkhara-mini, 6-word pieces (0.159 on whole inputs) 0.066 23%
AkaraAlpha 0.281 2%
  • Prime is Base trained further on scenario-designed, LLM-drafted, human-audited data. Most bench items share that scenario design, so its lead is an upper bound.
  • Decode long messages in pieces, as above. On whole inputs, beam-only CER is 0.218.

Model and training

  • Architecture. Pre-LN Transformer encoder-decoder: 6 encoder and 2 decoder layers, d_model 256, 4 heads, FFN 1024. It has 215 input symbols (Latin letters, digits, punctuation) and 228 output symbols (Khmer characters, and copied characters).
  • transformers format. It loads as BlenderbotForConditionalGeneration because the architecture is the same; it has nothing else in common with BlenderBot. Both vocabularies share one id space (output symbols first), so transformers counts 7.43M parameters.
  • Training. Three stages on one laptop GPU (Apple M5 Pro), about 3 hours in all:
    1. pretraining on 6.5M pairs: real Khmer sentences spelled by a rule-based typing generator (samdei-type-mini-pretrained);
    2. fine-tuning on 1.16M chat-like pairs;
    3. adaptation to casual register (1.59M pairs, with replay).
  • Data. The exact training files are in samdei-type-pretraining-data and samdei-type-finetuning-data, with recipes.json. No evaluation data and no other romanization system's data was trained on. Akkhara-Preview, a public 110-row preview, was used only to pick stage 1–2 checkpoints; stage 3 used a closed development set.

Limitations

  • Every romanized input seen in training is synthetic (rules, mined spelling tables, an LLM). Real typing appears only in the evaluation.
  • Homophones that need more context than a word bigram: ដើរ / αžŠαŸ‚αžš (der), αžŠαžΉαž€ / αžŠαŸ‚αž€ / αžŠαŸαž€ (dek).
  • Loanwords typed in English (check for αž†αŸ‚αž€) stay English. A Latin comma is copied, and a missing final αŸ” is never added.
  • Rare dictionary words without context are hard, as is heavier abbreviation than in training.
  • The Khmer side is mostly web text, with its topics and biases.
  • Not for Khmer β†’ Latin, official romanization of names, or output nobody checks.

License

Apache-2.0 (LICENSE; keep NOTICE when redistributing).

  • Word LM: built without Wikipedia, from HPLT v2, FineWeb-2 (ODC-By), ALT (CC BY) and our LLM-written chat sentences.
  • Training data: CC BY-SA 4.0, with its sources in the dataset cards.
  • Tools: khmercut (Apache-2.0) and SIL khnormal (MIT).

Citation

@misc{samdei-type-2026,
  title  = {Samdei Type: Separating What Is Written from How It Is Spelled for Romanized Khmer Transliteration},
  author = {Mengsay Loem},
  year   = {2026},
  note   = {Technical report},
  url    = {https://github.com/Samdei-Labs/samdei-type/blob/main/paper/samdei-type.pdf}
}
Downloads last month
97
Safetensors
Model size
7.43M params
Tensor type
F32
Β·
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for samdei/samdei-type-mini

Finetuned
(1)
this model

Datasets used to train samdei/samdei-type-mini

Collection including samdei/samdei-type-mini

Evaluation results