Translation
Transformers
ONNX
Safetensors
PyTorch
Khmer
blenderbot
text2text-generation
transliteration
romanization
romanized-khmer
khmer
character-level
int8
Eval Results (legacy)
Instructions to use samdei/samdei-type-base with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use samdei/samdei-type-base with Transformers:
# Use a pipeline as a high-level helper # Warning: Pipeline type "translation" is no longer supported in transformers v5. # You must load the model directly (see below) or downgrade to v4.x with: # pip install "transformers<5.0.0" from transformers import pipeline pipe = pipeline("translation", model="samdei/samdei-type-base")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForSeq2SeqLM tokenizer = AutoTokenizer.from_pretrained("samdei/samdei-type-base") model = AutoModelForSeq2SeqLM.from_pretrained("samdei/samdei-type-base", device_map="auto") - Notebooks
- Google Colab
- Kaggle
File size: 7,729 Bytes
7a7ad8f 0420834 a3cc27d 7a7ad8f 019ccf6 7a7ad8f a3cc27d 7a7ad8f a3cc27d 7a7ad8f 06a3e56 a3cc27d 7a7ad8f a3cc27d 0420834 a3cc27d 7a7ad8f a3cc27d fc9586e a3cc27d 7a7ad8f a3cc27d 7a7ad8f a3cc27d 7a7ad8f a3cc27d 7a7ad8f a3cc27d 7a7ad8f a3cc27d 7a7ad8f a3cc27d 7a7ad8f a3cc27d 7a7ad8f a3cc27d 7a7ad8f a3cc27d 7a7ad8f a3cc27d 7a7ad8f a3cc27d 7a7ad8f a3cc27d 7a7ad8f a3cc27d 7a7ad8f a3cc27d 7a7ad8f a3cc27d 7a7ad8f a3cc27d 7a7ad8f a3cc27d b9a2b3c 7a7ad8f | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 | ---
license: apache-2.0
language:
- km
library_name: transformers
pipeline_tag: translation
tags:
- transliteration
- romanization
- romanized-khmer
- khmer
- character-level
- pytorch
- onnx
- int8
- safetensors
base_model: samdei/samdei-type-base-pretrained
base_model_relation: finetune
datasets:
- samdei/samdei-type-pretraining-data
- samdei/samdei-type-finetuning-data
metrics:
- cer
- exact_match
model-index:
- name: Samdei Type Base
results:
- task:
type: translation
name: Romanized Khmer to Khmer script
dataset:
type: samdei/samdei-type-bench
name: Samdei Type Bench
split: test
metrics:
- type: cer
value: 0.052
name: CER (beam 5, char LM + word LM + OOV + channel rerank, 8-word pieces)
- type: exact_match
value: 0.303
name: Exact match
---
# Samdei Type Base
Samdei Type Base converts romanized Khmer, as people type it in chat, into Khmer script. It is an 18.2M-parameter character-level encoder-decoder, the most accurate open Samdei Type model, for desktop and server use. For a phone, use [Samdei Type Mini](https://huggingface.co/samdei/samdei-type-mini).
| Input | Output |
|---|---|
| `knhom jong tov phsar` | αααα»αα
αααα
ααααΆα |
| `ot mean wifi te` | α’ααααΆα wifi αα |
Release article: [Samdei Type: open models that turn romanized Khmer into Khmer script](https://huggingface.co/blog/mengsay/samdei-type).
## Quick start
With transformers alone (`pip install transformers torch`):
```python
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
repo = "samdei/samdei-type-base"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForSeq2SeqLM.from_pretrained(repo)
def pieces(text: str, max_words: int = 8) -> list[str]:
"""At most max_words words per piece, with a break after punctuation, as in the paper."""
out, cur = [], []
for w in text.split():
cur.append(w)
if len(cur) == max_words or w[-1] in ",.;:?!α":
out.append(" ".join(cur))
cur = []
if cur:
out.append(" ".join(cur))
return out or [text]
def convert(text: str) -> str:
batch = tok(pieces(text), padding=True, truncation=True, return_tensors="pt")
out = model.generate(**batch) # beam search, k = 5 (generation_config.json)
return " ".join(tok.batch_decode(out, skip_special_tokens=True))
print(convert("knhom jong tov phsar")) # αααα»αα
αααα
ααααΆα
```
This is beam search alone (CER 0.062 on Samdei Type Bench). For the best quality, rerank with the two LMs and the channel model of this repo (0.052), using the [`samdei-type`](https://github.com/Samdei-Labs/samdei-type) package:
```bash
pip install samdei-type
```
```python
from samdei_type.pipeline import Converter
conv = Converter("samdei/samdei-type-base") # PyTorch weights + full rerank, as in the paper
print(conv.convert("knhom jong tov phsar")) # αααα»αα
αααα
ααααΆα
```
Or on the command line: `samdei-type -m samdei/samdei-type-base "knhom jong tov phsar"`.
## Files
```
config.json, generation_config.json, model.safetensors (71.1 MiB), tokenizer*.json fp32 model (transformers)
onnx/encoder_kv.int8.onnx (11.3 MiB), onnx/decoder_step.int8.onnx (6.3 MiB), onnx/vocab.json int8 ONNX
channel/ (28.4 MiB) reverse model (Khmer -> romanized) for reranking, same format
lm/char5_nowiki.npz (8.9 MiB) character 5-gram LM
lm/wordfull_nowiki.json.gz (12.2 MiB) word bigram LM
rerank.json decoding and rerank settings
```
Reranking scores each beam hypothesis y for input x as `log P(y|x) + 0.3 log P_char(y) + 0.3 log P_word(y) β 1.0 #OOV(y) + 0.3 log P_ch(x|y)`. The channel term checks that y explains all of the input, so the LMs cannot swap a rare word for a frequent one. Nothing in this repo is a pickle.
## Results
[Samdei Type Bench](https://huggingface.co/datasets/samdei/samdei-type-bench): 2,155 everyday messages and words with human-checked Khmer. The metrics are CER after normalization and exact match.
| System | CER | Exact |
|---|---|---|
| **Samdei Type Base**, reranked | **0.052** | **30%** |
| Samdei Type Base, beam only (the transformers example above) | 0.062 | 24% |
| [Samdei Type Mini](https://huggingface.co/samdei/samdei-type-mini), reranked | 0.058 | 24% |
| Samdei Type Prime (not released) | 0.027 | 61% |
| Akkhara-mini, 6-word pieces (0.159 on whole inputs) | 0.066 | 23% |
| AkaraAlpha | 0.281 | 2% |
- Prime is Base trained further on scenario-designed, LLM-drafted, human-audited data. Most bench items share that scenario design, so its lead is an upper bound.
- Without punctuation, Base's CER is 0.046. Decode long messages in pieces, as above. On whole inputs, beam-only CER is 0.133.
- On one Apple M5 Pro CPU thread with int8 ONNX, beam 5 takes 9.3 ms per input (p50).
## Model and training
- **Architecture.** Pre-LN Transformer encoder-decoder: 6 encoder and 3 decoder layers, d_model 384, 6 heads, FFN 1536. It has 216 input and 228 output symbols. The channel model is Mini-sized (7.1M) and reads in the reverse direction.
- **transformers format.** It loads as `BlenderbotForConditionalGeneration` because the architecture is the same; it has nothing else in common with BlenderBot. Both vocabularies share one id space (output symbols first), so transformers counts 18.63M parameters.
- **Training.** Three stages on one laptop GPU (Apple M5 Pro), about 6 hours in all:
1. pretraining on 12.8M pairs: real Khmer sentences spelled by a rule-based typing generator ([`samdei-type-base-pretrained`](https://huggingface.co/samdei/samdei-type-base-pretrained));
2. fine-tuning on 1.16M chat-like pairs;
3. adaptation to casual register (1.59M pairs, with replay).
The rerank LMs are built from HPLT v2, FineWeb-2, ALT and our LLM-written chat sentences, without Wikipedia.
- **Data.** The exact training files are in [`samdei-type-pretraining-data`](https://huggingface.co/datasets/samdei/samdei-type-pretraining-data) and [`samdei-type-finetuning-data`](https://huggingface.co/datasets/samdei/samdei-type-finetuning-data), with `recipes.json`. No evaluation data and no other romanization system's data was trained on. Akkhara-Preview, a public 110-row preview, was used only to pick stage 1β2 checkpoints; stage 3 used a closed development set.
## Limitations
- Every romanized input seen in training is synthetic (rules, mined spelling tables, an LLM). Real typing appears only in the evaluation.
- Homophones that need more context than a word bigram: ααΎα / ααα (`der`), ααΉα / ααα / ααα (`dek`).
- Loanwords typed in English (`check` for ααα) stay English. A Latin comma is copied, and a missing final α is never added.
- Rare dictionary words without context are hard, many of them Pali-Sanskrit.
- The Khmer side is mostly web text, with its topics and biases.
- Not for Khmer β Latin, official romanization of names, or output nobody checks.
## License
Apache-2.0 (`LICENSE`; keep `NOTICE` when redistributing).
- **LMs:** built without Wikipedia, from HPLT v2, FineWeb-2 (ODC-By), ALT (CC BY) and our LLM-written chat sentences.
- **Training data:** CC BY-SA 4.0, with its sources in the dataset cards.
- **Tools:** `khmercut` (Apache-2.0) and SIL `khnormal` (MIT).
## Citation
```bibtex
@misc{samdei-type-2026,
title = {Samdei Type: Separating What Is Written from How It Is Spelled for Romanized Khmer Transliteration},
author = {Mengsay Loem},
year = {2026},
note = {Technical report},
url = {https://github.com/Samdei-Labs/samdei-type/blob/main/paper/samdei-type.pdf}
}
```
|