File size: 7,391 Bytes
73c7c04
 
 
 
0bfb5f4
c824a1a
73c7c04
 
 
 
 
 
 
 
0bfb5f4
73c7c04
 
3c22801
73c7c04
 
 
 
 
 
 
 
 
 
 
c824a1a
73c7c04
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
c824a1a
73c7c04
 
 
 
 
 
d0ad1f5
 
c824a1a
73c7c04
c824a1a
0bfb5f4
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
c824a1a
73c7c04
c824a1a
c6e8302
c824a1a
0bfb5f4
c824a1a
 
73c7c04
c6e8302
 
c824a1a
73c7c04
c824a1a
73c7c04
c824a1a
73c7c04
c824a1a
 
 
 
 
 
73c7c04
c824a1a
73c7c04
c824a1a
73c7c04
c824a1a
73c7c04
c824a1a
 
 
 
 
 
 
 
73c7c04
c824a1a
 
73c7c04
c824a1a
73c7c04
c824a1a
 
 
 
 
 
 
73c7c04
 
 
c824a1a
 
 
 
 
 
73c7c04
c824a1a
73c7c04
c824a1a
 
 
73c7c04
 
 
 
 
 
 
c824a1a
4e28d06
 
 
73c7c04
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
---
license: apache-2.0
language:
- km
library_name: transformers
pipeline_tag: translation
tags:
- transliteration
- romanization
- romanized-khmer
- khmer
- character-level
- onnx
- int8
- safetensors
- keyboard
- on-device
base_model: samdei/samdei-type-mini-pretrained
base_model_relation: finetune
datasets:
- samdei/samdei-type-pretraining-data
- samdei/samdei-type-finetuning-data
metrics:
- cer
- exact_match
model-index:
- name: Samdei Type Mini
  results:
  - task:
      type: translation
      name: Romanized Khmer to Khmer script
    dataset:
      type: samdei/samdei-type-bench
      name: Samdei Type Bench
      split: test
    metrics:
    - type: cer
      value: 0.058
      name: CER (beam 5, word-LM rerank, 8-word pieces)
    - type: exact_match
      value: 0.239
      name: Exact match
---

# Samdei Type Mini

Samdei Type Mini converts romanized Khmer, as people type it in chat, into Khmer script. It is a 7.1M-parameter character-level encoder-decoder, small enough for a phone keyboard: 7 MiB as int8 ONNX.

| Input | Output |
|---|---|
| `knhom jong tov phsar` | αžαŸ’αž‰αž»αŸ†αž…αž„αŸ‹αž‘αŸ…αž•αŸ’αžŸαžΆαžš |
| `ot mean wifi te` | αž’αžαŸ‹αž˜αžΆαž“ wifi αž‘αŸ |

Release article: [Samdei Type: open models that turn romanized Khmer into Khmer script](https://huggingface.co/blog/mengsay/samdei-type), with a demo of this model in a chat.

## Quick start

With transformers alone (`pip install transformers torch`):

```python
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

repo = "samdei/samdei-type-mini"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForSeq2SeqLM.from_pretrained(repo)

def pieces(text: str, max_words: int = 8) -> list[str]:
    """At most max_words words per piece, with a break after punctuation, as in the paper."""
    out, cur = [], []
    for w in text.split():
        cur.append(w)
        if len(cur) == max_words or w[-1] in ",.;:?!αŸ”":
            out.append(" ".join(cur))
            cur = []
    if cur:
        out.append(" ".join(cur))
    return out or [text]

def convert(text: str) -> str:
    batch = tok(pieces(text), padding=True, truncation=True, return_tensors="pt")
    out = model.generate(**batch)  # beam search, k = 5 (generation_config.json)
    return " ".join(tok.batch_decode(out, skip_special_tokens=True))

print(convert("knhom jong tov phsar"))  # αžαŸ’αž‰αž»αŸ†αž…αž„αŸ‹αž‘αŸ…αž•αŸ’αžŸαžΆαžš
```

This is beam search alone (CER 0.072 on Samdei Type Bench). For the best quality, rerank with the word LM of this repo (0.058), using the [`samdei-type`](https://github.com/Samdei-Labs/samdei-type) package:

```bash
pip install "samdei-type[deploy]"
```

```python
from samdei_type.pipeline import Converter

conv = Converter("samdei/samdei-type-mini")   # int8 ONNX + word-LM rerank, as in the paper
print(conv.convert("knhom jong tov phsar"))   # αžαŸ’αž‰αž»αŸ†αž…αž„αŸ‹αž‘αŸ…αž•αŸ’αžŸαžΆαžš
```

Or on the command line: `samdei-type "knhom jong tov phsar"`.

## Files

```
config.json, generation_config.json, model.safetensors (28.4 MiB), tokenizer*.json   fp32 model (transformers)
onnx/encoder_kv.int8.onnx (5.0 MiB), onnx/decoder_step.int8.onnx (2.0 MiB), onnx/vocab.json   int8 ONNX
lm/wordfull_nowiki_p25.json.gz (1.2 MiB)   word bigram LM for reranking
rerank.json                                decoding and rerank settings
```

**On a phone,** the two ONNX graphs are all a runtime needs. `encoder_kv` runs once per input. `decoder_step` decodes one token with a KV cache, and its batch axis holds all beams. [`samdei_type/onnx_infer.py`](https://github.com/Samdei-Labs/samdei-type) is a plain-numpy reference to port from. On one Apple M5 Pro CPU thread, beam 5 takes 4.9 ms per input (p50); with the rerank, a keystroke takes 5.7 ms. It has not been timed on a phone yet.

## Results

[Samdei Type Bench](https://huggingface.co/datasets/samdei/samdei-type-bench): 2,155 everyday messages and words with human-checked Khmer. The metrics are CER after normalization and exact match.

| System | CER | Exact |
|---|---|---|
| **Samdei Type Mini**, reranked | 0.058 | 24% |
| Samdei Type Mini, beam only (the transformers example above) | 0.072 | 17% |
| [Samdei Type Base](https://huggingface.co/samdei/samdei-type-base), reranked | 0.052 | 30% |
| Samdei Type Prime (not released) | 0.027 | 61% |
| Akkhara-mini, 6-word pieces (0.159 on whole inputs) | 0.066 | 23% |
| AkaraAlpha | 0.281 | 2% |

- Prime is Base trained further on scenario-designed, LLM-drafted, human-audited data. Most bench items share that scenario design, so its lead is an upper bound.
- Decode long messages in pieces, as above. On whole inputs, beam-only CER is 0.218.

## Model and training

- **Architecture.** Pre-LN Transformer encoder-decoder: 6 encoder and 2 decoder layers, d_model 256, 4 heads, FFN 1024. It has 215 input symbols (Latin letters, digits, punctuation) and 228 output symbols (Khmer characters, and copied characters).
- **transformers format.** It loads as `BlenderbotForConditionalGeneration` because the architecture is the same; it has nothing else in common with BlenderBot. Both vocabularies share one id space (output symbols first), so transformers counts 7.43M parameters.
- **Training.** Three stages on one laptop GPU (Apple M5 Pro), about 3 hours in all:
  1. pretraining on 6.5M pairs: real Khmer sentences spelled by a rule-based typing generator ([`samdei-type-mini-pretrained`](https://huggingface.co/samdei/samdei-type-mini-pretrained));
  2. fine-tuning on 1.16M chat-like pairs;
  3. adaptation to casual register (1.59M pairs, with replay).
- **Data.** The exact training files are in [`samdei-type-pretraining-data`](https://huggingface.co/datasets/samdei/samdei-type-pretraining-data) and [`samdei-type-finetuning-data`](https://huggingface.co/datasets/samdei/samdei-type-finetuning-data), with `recipes.json`. No evaluation data and no other romanization system's data was trained on. Akkhara-Preview, a public 110-row preview, was used only to pick stage 1–2 checkpoints; stage 3 used a closed development set.

## Limitations

- Every romanized input seen in training is synthetic (rules, mined spelling tables, an LLM). Real typing appears only in the evaluation.
- Homophones that need more context than a word bigram: ដើរ / αžŠαŸ‚αžš (`der`), αžŠαžΉαž€ / αžŠαŸ‚αž€ / αžŠαŸαž€ (`dek`).
- Loanwords typed in English (`check` for αž†αŸ‚αž€) stay English. A Latin comma is copied, and a missing final αŸ” is never added.
- Rare dictionary words without context are hard, as is heavier abbreviation than in training.
- The Khmer side is mostly web text, with its topics and biases.
- Not for Khmer β†’ Latin, official romanization of names, or output nobody checks.

## License

Apache-2.0 (`LICENSE`; keep `NOTICE` when redistributing).
- **Word LM:** built without Wikipedia, from HPLT v2, FineWeb-2 (ODC-By), ALT (CC BY) and our LLM-written chat sentences.
- **Training data:** CC BY-SA 4.0, with its sources in the dataset cards.
- **Tools:** `khmercut` (Apache-2.0) and SIL `khnormal` (MIT).

## Citation

```bibtex
@misc{samdei-type-2026,
  title  = {Samdei Type: Separating What Is Written from How It Is Spelled for Romanized Khmer Transliteration},
  author = {Mengsay Loem},
  year   = {2026},
  note   = {Technical report},
  url    = {https://github.com/Samdei-Labs/samdei-type/blob/main/paper/samdei-type.pdf}
}
```