---
license: cc-by-nc-4.0
language:
- km
- en
base_model:
- facebook/mms-1b-all
datasets:
- Digital-Divide-Data/khmer-speech-dataset
- nphearum/dataset-asr-tts
- nphearum/khmer-raw-text-3M
pipeline_tag: automatic-speech-recognition
library_name: transformers
tags:
- automatic-speech-recognition
- speech
- khmer
- code-switching
- wav2vec2
- mms
- ctc
metrics:
- cer
- wer
model-index:
- name: PsarAI-ASR-1B
  results:
  - task:
      type: automatic-speech-recognition
      name: Automatic Speech Recognition
    dataset:
      type: seanghay/khmer_grkpp_speech
      name: Khmer news speech (grkpp, unseen speakers, 1-30 s clips)
    metrics:
    - type: cer
      value: 13.73
      name: CER (greedy)
  - task:
      type: automatic-speech-recognition
      name: Automatic Speech Recognition
    dataset:
      type: nphearum/dataset-asr-tts
      name: English (held-out synthetic speech)
    metrics:
    - type: wer
      value: 1.39
      name: WER (greedy)
---

# PsarAI-ASR-1B

Speech recognition for **Khmer (ខ្មែរ)**, **English**, and **mixed Khmer–English** speech (wav2vec2, ~965M parameters, CTC).

- Starts from Meta's Khmer language adapter and private pretrain adapter layers and output head (~2.3M parameters).
- Adds a word-separator token, so English words are split by spaces (the original Khmer adapter has none).
- Everything is in one folder: no `target_lang` or separate adapter file is needed to load it.

## Usage

```python
import librosa
import torch
from transformers import AutoProcessor, Wav2Vec2ForCTC

processor = AutoProcessor.from_pretrained("nphearum/PsarAI-ASR-1B")
model = Wav2Vec2ForCTC.from_pretrained("nphearum/PsarAI-ASR-1B", dtype=torch.float32).eval()

audio, _ = librosa.load("speech.wav", sr=16_000, mono=True)  # 16 kHz mono
inputs = processor(audio, sampling_rate=16_000, return_tensors="pt")
with torch.inference_mode():
    logits = model(**inputs).logits
print(processor.decode(logits.argmax(-1)[0]))
```

Weights are stored in float16 (1.9 GB); load them as float32 as above, especially on CPU.
Clips of up to ~30 s fit comfortably on a 24 GB GPU; split longer recordings into chunks.

## Results

Character error rate (CER, letters only: spaces and punctuation ignored) and word error rate (WER),
lower is better. Before scoring, references and transcripts are normalized the same way: canonical
Khmer character order, one spelling per word (ឲ្យ/អោយ → ឱ្យ), numbers as spoken words (២០១៩ → ពីរពាន់ដប់ប្រាំបួន).

| Test set | Clips | What it is | Greedy | With Khmer LM decoder* |
|---|---:|---|---:|---:|
| grkpp (news) | 348 | [seanghay/khmer_grkpp_speech](https://huggingface.co/datasets/seanghay/khmer_grkpp_speech), new speakers and topics | 13.73% CER | 9.99% CER |
| grkpp, no digits | 125 | same, clips whose transcript has no digits | 11.68% CER | 8.95% CER |
| khmer10 | 10 | DDD speaker never used in training | 4.84% CER | 3.01% CER |
| DDD held-out | 150 | Digital-Divide-Data clips not used in training | 5.87% CER | 2.12% CER |
| Jarvis Khmer | 100 | held-out synthetic voice, Khmer | 1.27% CER | 0.17% CER |
| Jarvis English | 100 | held-out synthetic voice, English | 1.39% WER | 0.49% WER |
| Jarvis mixed | 100 | held-out synthetic voice, Khmer–English | 12.43% WER | 2.99% WER |

\* Greedy is what this model outputs on its own. The second column adds a separate decoder: CTC beam
search with a Khmer character 6-gram and a word trigram (built from DDD, Jarvis and
[nphearum/khmer-raw-text-3M](https://huggingface.co/datasets/nphearum/khmer-raw-text-3M) text, with every
test sentence removed), which is not included in this repository.

**Read the numbers with care:**
- **grkpp is the most realistic test**: real news speech, new speakers and topics, no overlap with training or LM text.
- **DDD held-out is optimistic**: DDD speakers read many of the same sentences, so held-out clips share text with training clips.
- **Jarvis is one synthetic voice**, also used for training, so those numbers are in-domain. English has **not** been tested on real English speakers.

## Training

| | |
|---|---|
| Base | facebook/mms-1b-all, Khmer adapter (`khm`) |
| Trained | adapter layers + output head (~2.3M of ~965M parameters); base model frozen |
| Output vocabulary | the Khmer adapter's 152 characters + `\|` word separator (153) |
| Data | ~15 h of [Digital-Divide-Data/khmer-speech-dataset](https://huggingface.co/datasets/Digital-Divide-Data/khmer-speech-dataset) (real read speech, many speakers); [nphearum/dataset-asr-tts](https://huggingface.co/datasets/nphearum/dataset-asr-tts) synthetic "Jarvis" voice: 3,000 Khmer, 2,000 English and 800 mixed clips |
| Filtering | 1–15 s clips; Jarvis Khmer clips kept only if the base model already read them with CER ≤ 25% (the dataset card warns some audio does not match its text) |
| Held out | speaker `f-adt1-0001` (khmer10) and 300 DDD + 100 Jarvis clips per language |
| Steps | 4,000; AdamW, learning rate 1e-3 with warm-up and linear decay; bf16 autocast |
| Hardware | 1 × RTX 3090 |

## Limitations

- **English**: trained and tested only on one synthetic voice; expect clearly weaker results on real English speakers and accents. Output is lowercase.
- **Numbers**: the vocabulary has the Khmer digits ០ ១ ២ ៣ ៥ ៧ but not ៤ ៦ ៨ ៩, so the model sometimes writes numbers in mixed scripts (e.g. `២០១9`). Convert digits to words in post-processing if you need consistent output.
- **Spaces**: Khmer is written without spaces between words; the model places spaces where its training text had them (between phrases), not between every word.
- **Missing characters**: a few rare Khmer characters are not in the vocabulary (e.g. ៛ ៕ ឰ ឝ ឞ).
- **Domain words and names** it has not heard (e.g. អាវុធហត្ថ before correction) are often misspelled; an LM decoder with domain text or hotwords helps.

## License and credits

Released under **CC-BY-NC-4.0** (non-commercial), following the base model
[facebook/mms-1b-all](https://huggingface.co/facebook/mms-1b-all).

Training data:
- [Digital-Divide-Data/khmer-speech-dataset](https://huggingface.co/datasets/Digital-Divide-Data/khmer-speech-dataset) (CC-BY-SA-4.0)
- [nphearum/dataset-asr-tts](https://huggingface.co/datasets/nphearum/dataset-asr-tts) (CC-BY-NC-4.0; its synthetic voice imitates a
character voice whose rights belong to their owners). 

Check that your use is permitted before using this model.

```bibtex
@article{pratap2023mms,
  title   = {Scaling Speech Technology to 1,000+ Languages},
  author  = {Pratap, Vineel and Tjandra, Andros and Shi, Bowen and others},
  journal = {arXiv preprint arXiv:2305.13516},
  year    = {2023}
}
```