Sota-ASR-Khmer · Sota សោតៈ Khmer speech recognition (PyTorch)

Sota ("ear") turns Khmer speech into text. A small model trained from scratch (not fine-tuned): 26M parameters. This repo has the PyTorch weights; for running the model, use the ONNX repo.

Repo	What
Sota-ASR-Khmer (this repo)	PyTorch weights: model.safetensors (fp32, 25.89M parameters, 463 tensors) + config.json
Sota-ASR-Khmer-ONNX	To run it: ONNX int8 (≈33 MB) for onnxruntime, the browser and sherpa-onnx, with a Python helper
Sota-ASR-Demo	demo Space

Try it: in your browser (the audio never leaves your device) · website

Early test model. Best on clear Khmer speech. Phone calls, noisy audio and Khmer–English mixing have not been measured yet. The PyTorch model code is not public yet, so these weights are for reference and research; the ONNX files run without it.

Results

Character error rate (CER, lower is better) on 16 speakers never used in training: 12.0 hours, 6,513 clips. Every model ran on the same clips and was scored by the same rule (Khmer normalized, spaces and punctuation removed).

Model	Size	All (6,513)	nxkmer (5,508)	OpenSLR42 (1,005)
Sota (beam 5)	26M · 33 MB int8	3.40%	3.25%	5.20%
Sota (CTC greedy, fastest)	26M	3.62%	3.47%	5.54%
Qwen3-ASR-0.6B-Khmer	0.6B	13.20%	13.34%	11.41%
whisper-small-khmer-v2	≈244M	18.57%	19.72%	4.61% †

Read these numbers fairly:

• The test speakers are new to Sota, but the recordings come from the same sources and style as Sota's training data. That favours Sota. Each model tends to do best on audio like its own training data.
• † whisper-small-khmer-v2 was trained on OpenSLR42, so it may have heard those sentences before.
• Some differences are spelling style (for example ហ្នឹង vs នឹង); Sota learned the spelling of its own labels.
• Only 4 test clips contain English words, too few to score.
• The ONNX int8 files give the same accuracy as PyTorch (checked on 300 test clips: 3.19% vs 3.15% CER with beam search; greedy 3.24% vs 3.28%).

Speed: with the int8 ONNX model on a laptop CPU (4 threads), 1 minute of audio takes about 0.2 s (CTC greedy) or 1.5 s (beam 5).

Architecture

• Input: 16 kHz mono audio. The log-Mel front end (80 bins, 25 ms window, 10 ms hop) is inside the model, so the ONNX file takes raw audio.
• Subsampling: two stride-2 convolutions, 10 ms → 40 ms frames (1.8M parameters).
• Encoder: 12 Conformer blocks, d_model 256, 4 heads with rotary position embeddings, FFN 1024, conv kernel 31 (18.3M).
• Heads: a CTC head (0.9M) and a 3-layer attention decoder (4.9M). The decoder was pre-trained on Khmer text first.
• Tokens: 3,402 Khmer cluster tokens (whole written clusters, shared with the Chakkhu OCR model).

Decoding: a fast CTC greedy draft, then beam search with 5 beams scored by the decoder and CTC together (0.7 decoder + 0.3 CTC), with length guards that fall back to the CTC draft if the decoder runs away or stops early.

Training

Data	≈237 h real Khmer speech (NXKMER TTS 450, OpenSLR 42; see Citation) + ≈610 h synthetic speech from VoxPocket TTS; real speech always used in full, synthetic at most half of each epoch
Split	by speaker; cloned voices and repeated sentences removed from dev/test; dev 5.2 h (8 speakers), test 12.0 h (16 speakers), real speech only
Augmentation	SpecAugment, speed 0.9/1.0/1.1, background noise (MUSAN), room echo, phone-call simulation
Loss	0.3 · CTC + 0.7 · cross-entropy (label smoothing 0.1)
Schedule	decoder pre-trained on 315k Khmer sentences (10k steps), then 100k steps of speech training, AdamW, 25k warmup, bf16
Hardware	one RTX 3090 Ti, about 8 hours for the full run
Final weights	average of the last 5 checkpoints (80k–100k steps)

Files

File	What it is
model.safetensors	All weights, fp32: front end (log-Mel buffers are rebuilt in code), subsampling, 12 Conformer blocks, CTC head, 3-layer attention decoder
config.json	Model sizes (d_model 256, 4 heads, FFN 1024, 12 layers, kernel 31, 3 decoder layers, 3,402 tokens) and training-time SpecAugment settings
figures/	The diagrams on this page

Token list: tokens.json in Sota-ASR-Khmer-ONNX (0 pad, 1 BOS, 2 EOS, 3 unk, 4 CTC blank).

Limitations

• Early test model: tuned on clear read and spoken Khmer. Expect more errors on noisy audio, phone calls, fast casual speech and strong accents.
• English words inside Khmer speech are not measured yet and may be written as Khmer sounds.
• Output is Khmer text without spaces between words (Khmer is written that way), with little punctuation.

Citation

@misc{ing2026sota,
  author    = {Ing, Muyleang},
  title     = {Sota: A Small Khmer Speech Recognition Model Trained from Scratch},
  year      = {2026},
  publisher = {Hugging Face},
  url       = {https://huggingface.co/muyleanging/Sota-ASR-Khmer}
}

Data and voices Sota learned from

Real speech was used for training and for the dev/test sets. The other sets were used only as reference voices for VoxPocket to clone when making the synthetic speech; their recordings were not trained on directly. Please credit them too.

Set	By	Used as	License
NXKMER TTS 450	NTQAI	real speech (38 speakers): ≈212 h train, all dev, most test	CC BY 4.0
OpenSLR 42	Google	real speech (Sodimana et al., SLTU 2018)	CC BY-SA 4.0
VoxPocket + VoxCPM2	Ing Muyleang · OpenBMB	made the ≈610 h of synthetic speech	Apache-2.0
FLEURS	Google	reference voices	CC BY 4.0
Khmer Speech Corpus	seanghay	reference voices	CC BY 4.0
iAny Khmer Voice	sengtha / iAny community	reference voices	CC BY-SA 4.0
Cultural Khmer ASR Dataset	Digital Divide Data	reference voices	CC BY-SA 4.0
YODAS2	ESPnet	reference voices	CC BY 3.0
khm-asr-open	rinabuoy	reference voices	not stated
Khmer Speech of Phnom Penh Gendarmerie	seanghay	reference voices	research only

Compared against, not trained on: Qwen3-ASR-0.6B-Khmer and whisper-small-khmer-v2.

BibTeX for these sources

@misc{ntqai2026nxkmer,
  author    = {NTQAI},
  title     = {NXKMER TTS 450},
  year      = {2026},
  publisher = {Hugging Face},
  url       = {https://huggingface.co/datasets/NTQAI/nxkmer-tts-450}
}

@inproceedings{kjartansson-etal-tts-sltu2018,
  title     = {{A Step-by-Step Process for Building TTS Voices Using Open Source Data and Framework for Bangla, Javanese, Khmer, Nepali, Sinhala, and Sundanese}},
  author    = {Keshan Sodimana and Knot Pipatsrisawat and Linne Ha and Martin Jansche and Oddur Kjartansson and Pasindu De Silva and Supheakmungkol Sarin},
  booktitle = {Proc. The 6th Intl. Workshop on Spoken Language Technologies for Under-Resourced Languages (SLTU)},
  year      = {2018},
  address   = {Gurugram, India},
  pages     = {66--70},
  url       = {http://dx.doi.org/10.21437/SLTU.2018-14}
}

@misc{ing2026voxpocket,
  author    = {Ing, Muyleang},
  title     = {VoxPocket: A Small Khmer Text-to-Speech Model with Voice Cloning and Styles},
  year      = {2026},
  publisher = {Hugging Face},
  url       = {https://huggingface.co/muyleanging/VoxPocket-TTS-Khmer}
}

@article{voxcpm2025,
  title   = {VoxCPM: Tokenizer-Free TTS for Context-Aware Speech Generation and True-to-Life Voice Cloning},
  author  = {Zhou, Yixuan and Zeng, Guoyang and Liu, Xin and Li, Xiang and Yu, Renjie and Wang, Ziyang and Ye, Runchuan and Sun, Weiyue and Gui, Jiancheng and Li, Kehan and Wu, Zhiyong and Liu, Zhiyuan},
  journal = {arXiv preprint arXiv:2509.24650},
  year    = {2025}
}

@article{fleurs2022arxiv,
  title   = {FLEURS: Few-shot Learning Evaluation of Universal Representations of Speech},
  author  = {Conneau, Alexis and Ma, Min and Khanuja, Simran and Zhang, Yu and Axelrod, Vera and Dalmia, Siddharth and Riesa, Jason and Rivera, Clara and Bapna, Ankur},
  journal = {arXiv preprint arXiv:2205.12446},
  year    = {2022}
}

@inproceedings{li2023yodas,
  title     = {Yodas: Youtube-Oriented Dataset for Audio and Speech},
  author    = {Li, Xinjian and Takamichi, Shinnosuke and Saeki, Takaaki and Chen, William and Shiota, Sayaka and Watanabe, Shinji},
  booktitle = {2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)},
  pages     = {1--8},
  year      = {2023}
}

License

Preview release. See LICENSE: free to try and evaluate; the final license will be set once the license of the real training speech is confirmed.

Family

Akkhara អក្ខរៈ types · Chakkhu ចក្ខុ reads · Sota សោតៈ listens · VoxPocket speaks. By Ing Muyleang.