Darayut commited on
Commit
d728d26
ยท
verified ยท
1 Parent(s): 601d161

Release AkaraAlpha

Browse files

Keras model, tokenizers (pickle and JSON), the Khmer dictionary, config and model card.

.gitattributes CHANGED
@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ assets/architecture.png filter=lfs diff=lfs merge=lfs -text
37
+ bi_gru_attention_khmer_transliterator.keras filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,182 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ language:
4
+ - km
5
+ library_name: keras
6
+ pipeline_tag: text2text-generation
7
+ tags:
8
+ - transliteration
9
+ - khmer
10
+ - romanization
11
+ - seq2seq
12
+ - gru
13
+ - attention
14
+ - keras
15
+ datasets:
16
+ - Chhunneng/khmer-text-transliteration
17
+ metrics:
18
+ - cer
19
+ - accuracy
20
+ model-index:
21
+ - name: AkaraAlpha
22
+ results:
23
+ - task:
24
+ type: text2text-generation
25
+ name: Romanized Khmer to Khmer transliteration
26
+ dataset:
27
+ type: Chhunneng/khmer-text-transliteration
28
+ name: Khmer Text Transliteration Dataset (20% held out)
29
+ metrics:
30
+ - type: cer
31
+ value: 15.07
32
+ name: CER (%), model alone
33
+ - type: accuracy
34
+ value: 73.6
35
+ name: Top-1 word accuracy (%), beam 5 with dictionary
36
+ ---
37
+
38
+ # AkaraAlpha
39
+
40
+ AkaraAlpha turns Khmer typed in the Latin alphabet back into Khmer script: `brodae` โ†’ แž”แŸ’แžšแžŠแŸ‚. It is a character-level encoder-decoder with **78,705 parameters**. A bidirectional GRU reads the romanized word, and a GRU with additive attention writes it in Khmer, one character at a time. A Khmer dictionary of 38,743 words then checks its suggestions.
41
+
42
+ - **Accuracy.** The model alone has a character error rate of 15.07% on held-out words, lower than every baseline trained on the same data, including a Transformer with three times as many parameters.
43
+ - **Dictionary.** Checking five beam candidates against the dictionary makes the first suggestion right 73.6% of the time, against 44.4% for the model's single best guess.
44
+ - **Speed.** The whole system takes 94 ms per word with five candidates.
45
+
46
+ The code is on [GitHub](https://github.com/netra-ai-lab/Romanized-Khmer-to-Khmer-Transliteration) and the package is [`netra-transliterate`](https://pypi.org/project/netra-transliterate/) on PyPI, both under the MIT licence.
47
+
48
+ ## Architecture
49
+
50
+ <p align="center"><img src="https://huggingface.co/Netra-Lab/AkaraAlpha/resolve/main/assets/architecture.png" width="100%" alt="AkaraAlpha turning brodae into Khmer: a bidirectional GRU encoder reading upwards and a GRU decoder with additive attention reading downwards"></p>
51
+
52
+ *AkaraAlpha turning `brodae` into แž”แŸ’แžšแžŠแŸ‚.* The encoder reads upwards: each letter becomes a 32-dimensional embedding, a bidirectional GRU reads the letters in both directions, and a projection brings each state to 64 dimensions, the keys and values for attention. The decoder reads downwards. At step 4, with แž” แŸ’ แžš written, its GRU output is the query for additive attention, and a softmax layer over 81 ids gives แžŠ with probability 0.997. The chosen character joins the input and the loop repeats until the end marker.
53
+
54
+ | | |
55
+ | --- | --- |
56
+ | Input | Romanized word, up to 25 characters: 26 Latin letters |
57
+ | Output | Khmer word, up to 24 characters: 77 Khmer characters |
58
+ | Character embeddings | 32 dimensions, separate for input and output |
59
+ | Encoder | Bidirectional GRU, 64 units each way, then a projection to 64 |
60
+ | Decoder | GRU, 64 units, starting from zeros |
61
+ | Attention | Additive (Keras `AdditiveAttention`, learned scale); keys and values are the projected encoder states |
62
+ | Output layer | Softmax over 81 ids, from the GRU output and the context |
63
+ | Parameters | 78,705 (encoder 46,784, decoder 31,921) |
64
+ | Dictionary | 38,743 Khmer words |
65
+
66
+ The decoder doesn't start from the encoder's final state, so everything it knows about the input reaches it through attention.
67
+
68
+ ## Results
69
+
70
+ Character error rate of the models alone on the 5,714 held-out words. Lower is better.
71
+
72
+ | Model | Parameters | CER (%) |
73
+ | --- | ---: | ---: |
74
+ | **AkaraAlpha** | 78,705 | **15.07** |
75
+ | Transformer | 248,337 | 17.78 |
76
+ | Attention BiLSTM | 96,753 | 18.26 |
77
+ | LSTM | 58,417 | 31.78 |
78
+ | GRU | 46,385 | 51.64 |
79
+ | RNN | 42,609 | 102.41 |
80
+
81
+ Word-level accuracy and latency of the full system, with the dictionary check. Top-1 is the share of words where the first suggestion is exactly right; Top-K the share where the right word is anywhere in the list.
82
+
83
+ | Decoding | Top-1 | Top-K | Latency per word |
84
+ | --- | ---: | ---: | ---: |
85
+ | Greedy | 44.44% | 44.44% | 21.92 ms |
86
+ | Beam, k = 3 | 70.06% | 78.72% | 66.03 ms |
87
+ | **Beam, k = 5 (recommended)** | **73.60%** | **83.41%** | **94.15 ms** |
88
+ | Beam, k = 7 | 74.87% | 85.60% | 122.10 ms |
89
+ | Beam, k = 10 | 75.85% | 87.37% | 167.21 ms |
90
+
91
+ ## Usage
92
+
93
+ ### With the package
94
+
95
+ The package includes beam search and the dictionary check:
96
+
97
+ ```bash
98
+ pip install netra-transliterate
99
+ ```
100
+
101
+ ```python
102
+ from khmer_transliterator import transliterate, transliterate_with_dict
103
+
104
+ transliterate("brodae") # 'แž”แŸ’แžšแžŠแŸ‚'
105
+ transliterate_with_dict("brodae", n=5) # dictionary-checked suggestions
106
+ ```
107
+
108
+ `netra-transliterate --serve` opens a typing interface in the browser that suggests Khmer words as you type romanized ones.
109
+
110
+ ### From this repository
111
+
112
+ The Keras model and a plain JSON copy of its tokenizers are enough for greedy decoding (TensorFlow 2.17 or later):
113
+
114
+ ```python
115
+ import json, re
116
+ import numpy as np
117
+ import keras
118
+ from huggingface_hub import snapshot_download
119
+
120
+ path = snapshot_download("Netra-Lab/AkaraAlpha")
121
+ model = keras.saving.load_model(f"{path}/bi_gru_attention_khmer_transliterator.keras")
122
+ tok = json.load(open(f"{path}/tokenizers.json", encoding="utf-8"))
123
+ latin, khmer = tok["latin"], tok["khmer"]
124
+ id_to_khmer = {i: c for c, i in khmer["word_index"].items()}
125
+ start, end = khmer["word_index"]["\t"], khmer["word_index"]["\n"]
126
+
127
+ def transliterate(word: str) -> str:
128
+ letters = re.sub("[^a-z]", "", word.lower())[: latin["max_len"]]
129
+ src = [latin["word_index"].get(c, latin["word_index"]["<unk>"]) for c in letters]
130
+ src = np.array([src + [0] * (latin["max_len"] - len(src))])
131
+ out = [start]
132
+ for _ in range(khmer["max_len"] + 1):
133
+ probs = model.predict([src, np.array([out])], verbose=0)[0, -1]
134
+ nxt = int(probs.argmax())
135
+ if nxt == end:
136
+ break
137
+ out.append(nxt)
138
+ return "".join(id_to_khmer[i] for i in out[1:])
139
+
140
+ print(transliterate("brodae")) # แž”แŸ’แžšแžŠแŸ‚
141
+ ```
142
+
143
+ ## Files
144
+
145
+ | File | What it is |
146
+ | --- | --- |
147
+ | `bi_gru_attention_khmer_transliterator.keras` | The trained model (Keras 3) |
148
+ | `tokenizers.json` | Character-to-id maps for both scripts, maximum lengths and special tokens |
149
+ | `gru_transliteration_assets.pkl` | The same tokenizers as pickled Keras `Tokenizer` objects, as the package loads them. Only unpickle files you trust |
150
+ | `khmer_dictionary.txt` | The Khmer word list used by the dictionary check, one word per line |
151
+ | `config.json` | Architecture and decoding settings |
152
+
153
+ ## Training
154
+
155
+ - **Data.** The [Khmer Text Transliteration Dataset](https://github.com/Chhunneng/khmer-text-transliteration) (Chhunneng, 2023): 28,569 single words in romanized form and in Khmer script. The last 20% (5,714 words) was held out; the model trained on the other 22,855.
156
+ - **Objective.** Teacher forcing with sparse categorical cross-entropy.
157
+ - **Optimiser.** Adam at 0.001, batch size 64, dropout 0.3 on the inputs and recurrent states of both GRUs.
158
+ - **Schedule.** 50 epochs. The learning rate was halved when the validation loss stopped improving for three epochs, which happened once, after epoch 42. The weights are from epoch 48, which had the lowest validation loss (0.1034).
159
+
160
+ ## Limitations
161
+
162
+ - **Words, not sentences.** The model transliterates one word at a time, so it can't use the surrounding sentence to choose between two words with the same romanized spelling.
163
+ - **No separate test set.** The held-out words were also used to adjust the learning rate and choose the epoch, so the scores may be slightly optimistic.
164
+ - **One way of romanizing.** All the training data comes from one dataset. People who spell Khmer differently in Latin letters may get worse results.
165
+ - **The dictionary has edges.** A name or new word that isn't in the dictionary may be "corrected" into a real word within two edits.
166
+ - **Latency depends on hardware.** The timings were measured on one machine; read them as the relative cost of each setting.
167
+
168
+ ## Citation
169
+
170
+ ```bibtex
171
+ @software{akaraalpha2026,
172
+ title = {AkaraAlpha: Romanized Khmer to Khmer Script Transliteration},
173
+ author = {Nhem, Darayut},
174
+ year = {2026},
175
+ publisher = {Netra Lab},
176
+ url = {https://huggingface.co/Netra-Lab/AkaraAlpha}
177
+ }
178
+ ```
179
+
180
+ ## Licence
181
+
182
+ MIT.
assets/architecture.png ADDED

Git LFS Details

  • SHA256: a36f01a162346aae10d849154cc06f2b33e7b863cbf762eb61fc0b0717fba600
  • Pointer size: 131 Bytes
  • Size of remote file: 196 kB
bi_gru_attention_khmer_transliterator.keras ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5f6b1af75679af1a3d56198a9035c4e70e973c99c354f0db750f5cfa13809680
3
+ size 1014442
config.json ADDED
@@ -0,0 +1,29 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "model_type": "bigru-additive-attention-seq2seq",
3
+ "framework": "keras",
4
+ "keras_file": "bi_gru_attention_khmer_transliterator.keras",
5
+ "embedding_dim": 32,
6
+ "encoder": {
7
+ "type": "bidirectional_gru",
8
+ "units_per_direction": 64
9
+ },
10
+ "encoder_projection": 64,
11
+ "decoder": {
12
+ "type": "gru",
13
+ "units": 64,
14
+ "initial_state": "zeros"
15
+ },
16
+ "attention": {
17
+ "type": "additive",
18
+ "use_scale": true
19
+ },
20
+ "latin_vocab_size": 28,
21
+ "khmer_vocab_size": 81,
22
+ "max_latin_len": 25,
23
+ "max_khmer_len": 24,
24
+ "parameters": 78705,
25
+ "dictionary_file": "khmer_dictionary.txt",
26
+ "dictionary_words": 38743,
27
+ "recommended_beam_width": 5,
28
+ "max_edit_distance": 2
29
+ }
gru_transliteration_assets.pkl ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a163c7264d1a68d3d78fc9777f0f82b9bb695796faf33f63b5c16bed2204b6d9
3
+ size 3538
khmer_dictionary.txt ADDED
The diff for this file is too large to render. See raw diff
 
tokenizers.json ADDED
@@ -0,0 +1,123 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "latin": {
3
+ "word_index": {
4
+ "<unk>": 1,
5
+ "a": 2,
6
+ "o": 3,
7
+ "h": 4,
8
+ "e": 5,
9
+ "n": 6,
10
+ "r": 7,
11
+ "k": 8,
12
+ "t": 9,
13
+ "u": 10,
14
+ "s": 11,
15
+ "m": 12,
16
+ "b": 13,
17
+ "i": 14,
18
+ "c": 15,
19
+ "p": 16,
20
+ "l": 17,
21
+ "g": 18,
22
+ "y": 19,
23
+ "v": 20,
24
+ "d": 21,
25
+ "j": 22,
26
+ "z": 23,
27
+ "f": 24,
28
+ "w": 25,
29
+ "x": 26,
30
+ "q": 27
31
+ },
32
+ "max_len": 25
33
+ },
34
+ "khmer": {
35
+ "word_index": {
36
+ "<unk>": 1,
37
+ "แŸ’": 2,
38
+ "แžถ": 3,
39
+ "แžš": 4,
40
+ "แž€": 5,
41
+ "แžŸ": 6,
42
+ "แž”": 7,
43
+ "แž": 8,
44
+ "แž“": 9,
45
+ "แž˜": 10,
46
+ "แž„": 11,
47
+ "แžท": 12,
48
+ "แž›": 13,
49
+ "แžข": 14,
50
+ "แž…": 15,
51
+ "แžป": 16,
52
+ "แžœ": 17,
53
+ "แž‘": 18,
54
+ "แž™": 19,
55
+ "แŸ‹": 20,
56
+ "แž–": 21,
57
+ "แŸ†": 22,
58
+ "แž ": 23,
59
+ "แŸ„": 24,
60
+ "แž‡": 25,
61
+ "แŸ": 26,
62
+ "แž‚": 27,
63
+ "แžธ": 28,
64
+ "แžŽ": 29,
65
+ "แž‰": 30,
66
+ "แžผ": 31,
67
+ "แž—": 32,
68
+ "แŸ‚": 33,
69
+ "แž’": 34,
70
+ "แžŠ": 35,
71
+ "แž": 36,
72
+ "แžพ": 37,
73
+ "แŸ‰": 38,
74
+ "แŸ‡": 39,
75
+ "แžน": 40,
76
+ "แžฝ": 41,
77
+ "แžก": 42,
78
+ "แž": 43,
79
+ "แž•": 44,
80
+ "แŸ€": 45,
81
+ "แŸ": 46,
82
+ "แŸˆ": 47,
83
+ "แž†": 48,
84
+ "แŸ": 49,
85
+ "แŸƒ": 50,
86
+ "แŸŠ": 51,
87
+ "แž‹": 52,
88
+ "แžบ": 53,
89
+ "แŸ—": 54,
90
+ "แžŒ": 55,
91
+ "แžง": 56,
92
+ "แŸ…": 57,
93
+ "แžƒ": 58,
94
+ "แžˆ": 59,
95
+ "แŸŒ": 60,
96
+ "แžฟ": 61,
97
+ "แžฅ": 62,
98
+ "แžฑ": 63,
99
+ "แžซ": 64,
100
+ "แžฏ": 65,
101
+ "แž": 66,
102
+ "แŸŽ": 67,
103
+ "แŸ": 68,
104
+ "แžช": 69,
105
+ "แžฉ": 70,
106
+ "แžฒ": 71,
107
+ "แžฆ": 72,
108
+ "แžฎ": 73,
109
+ "แžฐ": 74,
110
+ "แžณ": 75,
111
+ "แŸ”": 76,
112
+ "แžญ": 77,
113
+ "แžฌ": 78,
114
+ "\t": 79,
115
+ "\n": 80
116
+ },
117
+ "max_len": 24
118
+ },
119
+ "padding_id": 0,
120
+ "start_token": "\t",
121
+ "end_token": "\n",
122
+ "oov_token": "<unk>"
123
+ }