PsarAI-ASR-1B

Speech recognition for Khmer (ខ្មែរ), English, and mixed Khmer–English speech (wav2vec2, ~965M parameters, CTC).

• Starts from Meta's Khmer language adapter and private pretrain adapter layers and output head (~2.3M parameters).
• Adds a word-separator token, so English words are split by spaces (the original Khmer adapter has none).
• Everything is in one folder: no target_lang or separate adapter file is needed to load it.

Usage

import librosa
import torch
from transformers import AutoProcessor, Wav2Vec2ForCTC

processor = AutoProcessor.from_pretrained("nphearum/PsarAI-ASR-1B")
model = Wav2Vec2ForCTC.from_pretrained("nphearum/PsarAI-ASR-1B", dtype=torch.float32).eval()

audio, _ = librosa.load("speech.wav", sr=16_000, mono=True)  # 16 kHz mono
inputs = processor(audio, sampling_rate=16_000, return_tensors="pt")
with torch.inference_mode():
    logits = model(**inputs).logits
print(processor.decode(logits.argmax(-1)[0]))

Weights are stored in float16 (1.9 GB); load them as float32 as above, especially on CPU. Clips of up to ~30 s fit comfortably on a 24 GB GPU; split longer recordings into chunks.

Results

Character error rate (CER, letters only: spaces and punctuation ignored) and word error rate (WER), lower is better. Before scoring, references and transcripts are normalized the same way: canonical Khmer character order, one spelling per word (ឲ្យ/អោយ → ឱ្យ), numbers as spoken words (២០១៩ → ពីរពាន់ដប់ប្រាំបួន).

Test set	Clips	What it is	Greedy	With Khmer LM decoder*
grkpp (news)	348	seanghay/khmer_grkpp_speech, new speakers and topics	13.73% CER	9.99% CER
grkpp, no digits	125	same, clips whose transcript has no digits	11.68% CER	8.95% CER
khmer10	10	DDD speaker never used in training	4.84% CER	3.01% CER
DDD held-out	150	Digital-Divide-Data clips not used in training	5.87% CER	2.12% CER
Jarvis Khmer	100	held-out synthetic voice, Khmer	1.27% CER	0.17% CER
Jarvis English	100	held-out synthetic voice, English	1.39% WER	0.49% WER
Jarvis mixed	100	held-out synthetic voice, Khmer–English	12.43% WER	2.99% WER

* Greedy is what this model outputs on its own. The second column adds a separate decoder: CTC beam search with a Khmer character 6-gram and a word trigram (built from DDD, Jarvis and nphearum/khmer-raw-text-3M text, with every test sentence removed), which is not included in this repository.

Read the numbers with care:

• grkpp is the most realistic test: real news speech, new speakers and topics, no overlap with training or LM text.
• DDD held-out is optimistic: DDD speakers read many of the same sentences, so held-out clips share text with training clips.
• Jarvis is one synthetic voice, also used for training, so those numbers are in-domain. English has not been tested on real English speakers.

Training

Base	facebook/mms-1b-all, Khmer adapter (khm)
Trained	adapter layers + output head (~2.3M of ~965M parameters); base model frozen
Output vocabulary	the Khmer adapter's 152 characters + | word separator (153)
Data	~15 h of Digital-Divide-Data/khmer-speech-dataset (real read speech, many speakers); nphearum/dataset-asr-tts synthetic "Jarvis" voice: 3,000 Khmer, 2,000 English and 800 mixed clips
Filtering	1–15 s clips; Jarvis Khmer clips kept only if the base model already read them with CER ≤ 25% (the dataset card warns some audio does not match its text)
Held out	speaker f-adt1-0001 (khmer10) and 300 DDD + 100 Jarvis clips per language
Steps	4,000; AdamW, learning rate 1e-3 with warm-up and linear decay; bf16 autocast
Hardware	1 × RTX 3090

Limitations

• English: trained and tested only on one synthetic voice; expect clearly weaker results on real English speakers and accents. Output is lowercase.
• Numbers: the vocabulary has the Khmer digits ០ ១ ២ ៣ ៥ ៧ but not ៤ ៦ ៨ ៩, so the model sometimes writes numbers in mixed scripts (e.g. ២០១9). Convert digits to words in post-processing if you need consistent output.
• Spaces: Khmer is written without spaces between words; the model places spaces where its training text had them (between phrases), not between every word.
• Missing characters: a few rare Khmer characters are not in the vocabulary (e.g. ៛ ៕ ឰ ឝ ឞ).
• Domain words and names it has not heard (e.g. អាវុធហត្ថ before correction) are often misspelled; an LM decoder with domain text or hotwords helps.

License and credits

Released under CC-BY-NC-4.0 (non-commercial), following the base model facebook/mms-1b-all.

Training data:

• Digital-Divide-Data/khmer-speech-dataset (CC-BY-SA-4.0)
• nphearum/dataset-asr-tts (CC-BY-NC-4.0; its synthetic voice imitates a character voice whose rights belong to their owners).

Check that your use is permitted before using this model.

@article{pratap2023mms,
  title   = {Scaling Speech Technology to 1,000+ Languages},
  author  = {Pratap, Vineel and Tjandra, Andros and Shi, Bowen and others},
  journal = {arXiv preprint arXiv:2305.13516},
  year    = {2023}
}