Translation
Transformers
ONNX
Safetensors
Khmer
blenderbot
text2text-generation
transliteration
romanization
romanized-khmer
khmer
character-level
int8
keyboard
on-device
Eval Results (legacy)
Instructions to use samdei/samdei-type-mini with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use samdei/samdei-type-mini with Transformers:
# Use a pipeline as a high-level helper # Warning: Pipeline type "translation" is no longer supported in transformers v5. # You must load the model directly (see below) or downgrade to v4.x with: # pip install "transformers<5.0.0" from transformers import pipeline pipe = pipeline("translation", model="samdei/samdei-type-mini")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForSeq2SeqLM tokenizer = AutoTokenizer.from_pretrained("samdei/samdei-type-mini") model = AutoModelForSeq2SeqLM.from_pretrained("samdei/samdei-type-mini", device_map="auto") - Notebooks
- Google Colab
- Kaggle
|
Download README.md from samdei/samdei-type-mini: direct link, hf CLI and curl.
- Browser
- Download file 7.39 kB
-
https://huggingface.co/samdei/samdei-type-mini/resolve/main/README.md
- Command line
-
hf download hf://samdei/samdei-type-mini/README.md
-
curl -L -o README.md https://huggingface.co/samdei/samdei-type-mini/resolve/main/README.md
7.39 kB
| license: apache-2.0 | |
| language: | |
| - km | |
| library_name: transformers | |
| pipeline_tag: translation | |
| tags: | |
| - transliteration | |
| - romanization | |
| - romanized-khmer | |
| - khmer | |
| - character-level | |
| - onnx | |
| - int8 | |
| - safetensors | |
| - keyboard | |
| - on-device | |
| base_model: samdei/samdei-type-mini-pretrained | |
| base_model_relation: finetune | |
| datasets: | |
| - samdei/samdei-type-pretraining-data | |
| - samdei/samdei-type-finetuning-data | |
| metrics: | |
| - cer | |
| - exact_match | |
| model-index: | |
| - name: Samdei Type Mini | |
| results: | |
| - task: | |
| type: translation | |
| name: Romanized Khmer to Khmer script | |
| dataset: | |
| type: samdei/samdei-type-bench | |
| name: Samdei Type Bench | |
| split: test | |
| metrics: | |
| - type: cer | |
| value: 0.058 | |
| name: CER (beam 5, word-LM rerank, 8-word pieces) | |
| - type: exact_match | |
| value: 0.239 | |
| name: Exact match | |
| # Samdei Type Mini | |
| Samdei Type Mini converts romanized Khmer, as people type it in chat, into Khmer script. It is a 7.1M-parameter character-level encoder-decoder, small enough for a phone keyboard: 7 MiB as int8 ONNX. | |
| | Input | Output | | |
| |---|---| | |
| | `knhom jong tov phsar` | αααα»αα αααα ααααΆα | | |
| | `ot mean wifi te` | α’ααααΆα wifi αα | | |
| Release article: [Samdei Type: open models that turn romanized Khmer into Khmer script](https://huggingface.co/blog/mengsay/samdei-type), with a demo of this model in a chat. | |
| ## Quick start | |
| With transformers alone (`pip install transformers torch`): | |
| ```python | |
| from transformers import AutoModelForSeq2SeqLM, AutoTokenizer | |
| repo = "samdei/samdei-type-mini" | |
| tok = AutoTokenizer.from_pretrained(repo) | |
| model = AutoModelForSeq2SeqLM.from_pretrained(repo) | |
| def pieces(text: str, max_words: int = 8) -> list[str]: | |
| """At most max_words words per piece, with a break after punctuation, as in the paper.""" | |
| out, cur = [], [] | |
| for w in text.split(): | |
| cur.append(w) | |
| if len(cur) == max_words or w[-1] in ",.;:?!α": | |
| out.append(" ".join(cur)) | |
| cur = [] | |
| if cur: | |
| out.append(" ".join(cur)) | |
| return out or [text] | |
| def convert(text: str) -> str: | |
| batch = tok(pieces(text), padding=True, truncation=True, return_tensors="pt") | |
| out = model.generate(**batch) # beam search, k = 5 (generation_config.json) | |
| return " ".join(tok.batch_decode(out, skip_special_tokens=True)) | |
| print(convert("knhom jong tov phsar")) # αααα»αα αααα ααααΆα | |
| ``` | |
| This is beam search alone (CER 0.072 on Samdei Type Bench). For the best quality, rerank with the word LM of this repo (0.058), using the [`samdei-type`](https://github.com/Samdei-Labs/samdei-type) package: | |
| ```bash | |
| pip install "samdei-type[deploy]" | |
| ``` | |
| ```python | |
| from samdei_type.pipeline import Converter | |
| conv = Converter("samdei/samdei-type-mini") # int8 ONNX + word-LM rerank, as in the paper | |
| print(conv.convert("knhom jong tov phsar")) # αααα»αα αααα ααααΆα | |
| ``` | |
| Or on the command line: `samdei-type "knhom jong tov phsar"`. | |
| ## Files | |
| ``` | |
| config.json, generation_config.json, model.safetensors (28.4 MiB), tokenizer*.json fp32 model (transformers) | |
| onnx/encoder_kv.int8.onnx (5.0 MiB), onnx/decoder_step.int8.onnx (2.0 MiB), onnx/vocab.json int8 ONNX | |
| lm/wordfull_nowiki_p25.json.gz (1.2 MiB) word bigram LM for reranking | |
| rerank.json decoding and rerank settings | |
| ``` | |
| **On a phone,** the two ONNX graphs are all a runtime needs. `encoder_kv` runs once per input. `decoder_step` decodes one token with a KV cache, and its batch axis holds all beams. [`samdei_type/onnx_infer.py`](https://github.com/Samdei-Labs/samdei-type) is a plain-numpy reference to port from. On one Apple M5 Pro CPU thread, beam 5 takes 4.9 ms per input (p50); with the rerank, a keystroke takes 5.7 ms. It has not been timed on a phone yet. | |
| ## Results | |
| [Samdei Type Bench](https://huggingface.co/datasets/samdei/samdei-type-bench): 2,155 everyday messages and words with human-checked Khmer. The metrics are CER after normalization and exact match. | |
| | System | CER | Exact | | |
| |---|---|---| | |
| | **Samdei Type Mini**, reranked | 0.058 | 24% | | |
| | Samdei Type Mini, beam only (the transformers example above) | 0.072 | 17% | | |
| | [Samdei Type Base](https://huggingface.co/samdei/samdei-type-base), reranked | 0.052 | 30% | | |
| | Samdei Type Prime (not released) | 0.027 | 61% | | |
| | Akkhara-mini, 6-word pieces (0.159 on whole inputs) | 0.066 | 23% | | |
| | AkaraAlpha | 0.281 | 2% | | |
| - Prime is Base trained further on scenario-designed, LLM-drafted, human-audited data. Most bench items share that scenario design, so its lead is an upper bound. | |
| - Decode long messages in pieces, as above. On whole inputs, beam-only CER is 0.218. | |
| ## Model and training | |
| - **Architecture.** Pre-LN Transformer encoder-decoder: 6 encoder and 2 decoder layers, d_model 256, 4 heads, FFN 1024. It has 215 input symbols (Latin letters, digits, punctuation) and 228 output symbols (Khmer characters, and copied characters). | |
| - **transformers format.** It loads as `BlenderbotForConditionalGeneration` because the architecture is the same; it has nothing else in common with BlenderBot. Both vocabularies share one id space (output symbols first), so transformers counts 7.43M parameters. | |
| - **Training.** Three stages on one laptop GPU (Apple M5 Pro), about 3 hours in all: | |
| 1. pretraining on 6.5M pairs: real Khmer sentences spelled by a rule-based typing generator ([`samdei-type-mini-pretrained`](https://huggingface.co/samdei/samdei-type-mini-pretrained)); | |
| 2. fine-tuning on 1.16M chat-like pairs; | |
| 3. adaptation to casual register (1.59M pairs, with replay). | |
| - **Data.** The exact training files are in [`samdei-type-pretraining-data`](https://huggingface.co/datasets/samdei/samdei-type-pretraining-data) and [`samdei-type-finetuning-data`](https://huggingface.co/datasets/samdei/samdei-type-finetuning-data), with `recipes.json`. No evaluation data and no other romanization system's data was trained on. Akkhara-Preview, a public 110-row preview, was used only to pick stage 1β2 checkpoints; stage 3 used a closed development set. | |
| ## Limitations | |
| - Every romanized input seen in training is synthetic (rules, mined spelling tables, an LLM). Real typing appears only in the evaluation. | |
| - Homophones that need more context than a word bigram: ααΎα / ααα (`der`), ααΉα / ααα / ααα (`dek`). | |
| - Loanwords typed in English (`check` for ααα) stay English. A Latin comma is copied, and a missing final α is never added. | |
| - Rare dictionary words without context are hard, as is heavier abbreviation than in training. | |
| - The Khmer side is mostly web text, with its topics and biases. | |
| - Not for Khmer β Latin, official romanization of names, or output nobody checks. | |
| ## License | |
| Apache-2.0 (`LICENSE`; keep `NOTICE` when redistributing). | |
| - **Word LM:** built without Wikipedia, from HPLT v2, FineWeb-2 (ODC-By), ALT (CC BY) and our LLM-written chat sentences. | |
| - **Training data:** CC BY-SA 4.0, with its sources in the dataset cards. | |
| - **Tools:** `khmercut` (Apache-2.0) and SIL `khnormal` (MIT). | |
| ## Citation | |
| ```bibtex | |
| @misc{samdei-type-2026, | |
| title = {Samdei Type: Separating What Is Written from How It Is Spelled for Romanized Khmer Transliteration}, | |
| author = {Mengsay Loem}, | |
| year = {2026}, | |
| note = {Technical report}, | |
| url = {https://github.com/Samdei-Labs/samdei-type/blob/main/paper/samdei-type.pdf} | |
| } | |
| ``` | |