| --- |
| license: cc-by-nc-4.0 |
| language: |
| - km |
| - en |
| base_model: |
| - facebook/mms-1b-all |
| datasets: |
| - Digital-Divide-Data/khmer-speech-dataset |
| - nphearum/dataset-asr-tts |
| - nphearum/khmer-raw-text-3M |
| pipeline_tag: automatic-speech-recognition |
| library_name: transformers |
| tags: |
| - automatic-speech-recognition |
| - speech |
| - khmer |
| - code-switching |
| - wav2vec2 |
| - mms |
| - ctc |
| metrics: |
| - cer |
| - wer |
| model-index: |
| - name: PsarAI-ASR-1B |
| results: |
| - task: |
| type: automatic-speech-recognition |
| name: Automatic Speech Recognition |
| dataset: |
| type: seanghay/khmer_grkpp_speech |
| name: Khmer news speech (grkpp, unseen speakers, 1-30 s clips) |
| metrics: |
| - type: cer |
| value: 13.73 |
| name: CER (greedy) |
| - task: |
| type: automatic-speech-recognition |
| name: Automatic Speech Recognition |
| dataset: |
| type: nphearum/dataset-asr-tts |
| name: English (held-out synthetic speech) |
| metrics: |
| - type: wer |
| value: 1.39 |
| name: WER (greedy) |
| --- |
| |
| # PsarAI-ASR-1B |
|
|
| Speech recognition for **Khmer (ខ្មែរ)**, **English**, and **mixed Khmer–English** speech (wav2vec2, ~965M parameters, CTC). |
|
|
| - Starts from Meta's Khmer language adapter and private pretrain adapter layers and output head (~2.3M parameters). |
| - Adds a word-separator token, so English words are split by spaces (the original Khmer adapter has none). |
| - Everything is in one folder: no `target_lang` or separate adapter file is needed to load it. |
|
|
| ## Usage |
|
|
| ```python |
| import librosa |
| import torch |
| from transformers import AutoProcessor, Wav2Vec2ForCTC |
| |
| processor = AutoProcessor.from_pretrained("nphearum/PsarAI-ASR-1B") |
| model = Wav2Vec2ForCTC.from_pretrained("nphearum/PsarAI-ASR-1B", dtype=torch.float32).eval() |
| |
| audio, _ = librosa.load("speech.wav", sr=16_000, mono=True) # 16 kHz mono |
| inputs = processor(audio, sampling_rate=16_000, return_tensors="pt") |
| with torch.inference_mode(): |
| logits = model(**inputs).logits |
| print(processor.decode(logits.argmax(-1)[0])) |
| ``` |
|
|
| Weights are stored in float16 (1.9 GB); load them as float32 as above, especially on CPU. |
| Clips of up to ~30 s fit comfortably on a 24 GB GPU; split longer recordings into chunks. |
|
|
| ## Results |
|
|
| Character error rate (CER, letters only: spaces and punctuation ignored) and word error rate (WER), |
| lower is better. Before scoring, references and transcripts are normalized the same way: canonical |
| Khmer character order, one spelling per word (ឲ្យ/អោយ → ឱ្យ), numbers as spoken words (២០១៩ → ពីរពាន់ដប់ប្រាំបួន). |
|
|
| | Test set | Clips | What it is | Greedy | With Khmer LM decoder* | |
| |---|---:|---|---:|---:| |
| | grkpp (news) | 348 | [seanghay/khmer_grkpp_speech](https://huggingface.co/datasets/seanghay/khmer_grkpp_speech), new speakers and topics | 13.73% CER | 9.99% CER | |
| | grkpp, no digits | 125 | same, clips whose transcript has no digits | 11.68% CER | 8.95% CER | |
| | khmer10 | 10 | DDD speaker never used in training | 4.84% CER | 3.01% CER | |
| | DDD held-out | 150 | Digital-Divide-Data clips not used in training | 5.87% CER | 2.12% CER | |
| | Jarvis Khmer | 100 | held-out synthetic voice, Khmer | 1.27% CER | 0.17% CER | |
| | Jarvis English | 100 | held-out synthetic voice, English | 1.39% WER | 0.49% WER | |
| | Jarvis mixed | 100 | held-out synthetic voice, Khmer–English | 12.43% WER | 2.99% WER | |
|
|
| \* Greedy is what this model outputs on its own. The second column adds a separate decoder: CTC beam |
| search with a Khmer character 6-gram and a word trigram (built from DDD, Jarvis and |
| [nphearum/khmer-raw-text-3M](https://huggingface.co/datasets/nphearum/khmer-raw-text-3M) text, with every |
| test sentence removed), which is not included in this repository. |
|
|
| **Read the numbers with care:** |
| - **grkpp is the most realistic test**: real news speech, new speakers and topics, no overlap with training or LM text. |
| - **DDD held-out is optimistic**: DDD speakers read many of the same sentences, so held-out clips share text with training clips. |
| - **Jarvis is one synthetic voice**, also used for training, so those numbers are in-domain. English has **not** been tested on real English speakers. |
|
|
| ## Training |
|
|
| | | | |
| |---|---| |
| | Base | facebook/mms-1b-all, Khmer adapter (`khm`) | |
| | Trained | adapter layers + output head (~2.3M of ~965M parameters); base model frozen | |
| | Output vocabulary | the Khmer adapter's 152 characters + `\|` word separator (153) | |
| | Data | ~15 h of [Digital-Divide-Data/khmer-speech-dataset](https://huggingface.co/datasets/Digital-Divide-Data/khmer-speech-dataset) (real read speech, many speakers); [nphearum/dataset-asr-tts](https://huggingface.co/datasets/nphearum/dataset-asr-tts) synthetic "Jarvis" voice: 3,000 Khmer, 2,000 English and 800 mixed clips | |
| | Filtering | 1–15 s clips; Jarvis Khmer clips kept only if the base model already read them with CER ≤ 25% (the dataset card warns some audio does not match its text) | |
| | Held out | speaker `f-adt1-0001` (khmer10) and 300 DDD + 100 Jarvis clips per language | |
| | Steps | 4,000; AdamW, learning rate 1e-3 with warm-up and linear decay; bf16 autocast | |
| | Hardware | 1 × RTX 3090 | |
|
|
| ## Limitations |
|
|
| - **English**: trained and tested only on one synthetic voice; expect clearly weaker results on real English speakers and accents. Output is lowercase. |
| - **Numbers**: the vocabulary has the Khmer digits ០ ១ ២ ៣ ៥ ៧ but not ៤ ៦ ៨ ៩, so the model sometimes writes numbers in mixed scripts (e.g. `២០១9`). Convert digits to words in post-processing if you need consistent output. |
| - **Spaces**: Khmer is written without spaces between words; the model places spaces where its training text had them (between phrases), not between every word. |
| - **Missing characters**: a few rare Khmer characters are not in the vocabulary (e.g. ៛ ៕ ឰ ឝ ឞ). |
| - **Domain words and names** it has not heard (e.g. អាវុធហត្ថ before correction) are often misspelled; an LM decoder with domain text or hotwords helps. |
|
|
| ## License and credits |
|
|
| Released under **CC-BY-NC-4.0** (non-commercial), following the base model |
| [facebook/mms-1b-all](https://huggingface.co/facebook/mms-1b-all). |
|
|
| Training data: |
| - [Digital-Divide-Data/khmer-speech-dataset](https://huggingface.co/datasets/Digital-Divide-Data/khmer-speech-dataset) (CC-BY-SA-4.0) |
| - [nphearum/dataset-asr-tts](https://huggingface.co/datasets/nphearum/dataset-asr-tts) (CC-BY-NC-4.0; its synthetic voice imitates a |
| character voice whose rights belong to their owners). |
|
|
| Check that your use is permitted before using this model. |
|
|
| ```bibtex |
| @article{pratap2023mms, |
| title = {Scaling Speech Technology to 1,000+ Languages}, |
| author = {Pratap, Vineel and Tjandra, Andros and Shi, Bowen and others}, |
| journal = {arXiv preprint arXiv:2305.13516}, |
| year = {2023} |
| } |
| ``` |
