Automatic Speech Recognition
Transformers
ONNX
Safetensors
Khmer
English
wav2vec2
speech
khmer
code-switching
mms
ctc
Eval Results (legacy)
Instructions to use phonsobon/PsarAI-ASR-1B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use phonsobon/PsarAI-ASR-1B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("automatic-speech-recognition", model="phonsobon/PsarAI-ASR-1B")# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForCTC processor = AutoProcessor.from_pretrained("phonsobon/PsarAI-ASR-1B") model = AutoModelForCTC.from_pretrained("phonsobon/PsarAI-ASR-1B", device_map="auto") - Notebooks
- Google Colab
- Kaggle
File size: 12,314 Bytes
1a24b5a 8948417 1a24b5a | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 | ---
license: cc-by-nc-4.0
language:
- km
- en
base_model: nphearum/PsarAI-ASR-1B
base_model_relation: quantized
datasets:
- Digital-Divide-Data/khmer-speech-dataset
- nphearum/dataset-asr-tts
- nphearum/khmer-raw-text-3M
pipeline_tag: automatic-speech-recognition
library_name: transformers
tags:
- automatic-speech-recognition
- speech
- khmer
- code-switching
- wav2vec2
- mms
- ctc
- onnx
metrics:
- cer
- wer
model-index:
- name: PsarAI-ASR-1B
results:
- task:
type: automatic-speech-recognition
name: Automatic Speech Recognition
dataset:
type: seanghay/khmer_grkpp_speech
name: Khmer news speech (grkpp, unseen speakers, 1-30 s clips)
metrics:
- type: cer
value: 13.73
name: CER (greedy)
- task:
type: automatic-speech-recognition
name: Automatic Speech Recognition
dataset:
type: nphearum/dataset-asr-tts
name: English (held-out synthetic speech)
metrics:
- type: wer
value: 1.39
name: WER (greedy)
---
# PsarAI-ASR-1B (PyTorch + ONNX)
Speech recognition for **Khmer (ខ្មែរ)**, **English**, and **mixed Khmer–English** speech
(wav2vec2, ~965M parameters, CTC).
This repository contains the original PyTorch weights of
[nphearum/PsarAI-ASR-1B](https://huggingface.co/nphearum/PsarAI-ASR-1B) plus **ONNX exports**
(fp32, fp16, int8) that run with `onnxruntime` only, without PyTorch.
## Files
| Path | Format | Size | Use it for |
|---|---|---:|---|
| `model.safetensors` (+ configs) | PyTorch, fp16 weights | 1.9 GB | 🤗 Transformers, GPU or CPU |
| `onnx/fp32/` | ONNX, fp32 | 3.9 GB | reference accuracy, CPU/GPU |
| `onnx/fp16/` | ONNX, fp16 (fp32 inputs/outputs) | 1.9 GB | GPU (CUDA / DirectML) |
| `onnx/int8/` | ONNX, int8 dynamic quantization | 1.0 GB | **CPU** (smallest, recommended for CPU) |
| `onnx/infer_onnx.py` | script | | command-line transcription with ONNX |
| `onnx/export_onnx.py` | script | | how the ONNX files were made |
| `results/` | benchmark | | CER / speed comparison of all variants |
Each `onnx/<variant>/` folder is self-contained (`model.onnx` + `model.onnx.data` + vocab/tokenizer files).
## Install
```bash
# PyTorch / Transformers
pip install torch transformers librosa soundfile
# ONNX only (no torch needed)
pip install onnxruntime librosa huggingface_hub # CPU
pip install onnxruntime-gpu librosa huggingface_hub # NVIDIA GPU
```
## Quick test — 🤗 Transformers
```python
import librosa
import torch
from transformers import AutoProcessor, Wav2Vec2ForCTC
REPO = "phonsobon/PsarAI-ASR-1B"
device = "cuda" if torch.cuda.is_available() else "cpu"
processor = AutoProcessor.from_pretrained(REPO)
model = Wav2Vec2ForCTC.from_pretrained(REPO, dtype=torch.float32).to(device).eval()
audio, _ = librosa.load("speech.wav", sr=16_000, mono=True) # any format librosa reads; 16 kHz mono
inputs = processor(audio, sampling_rate=16_000, return_tensors="pt").to(device)
with torch.inference_mode():
logits = model(**inputs).logits
print(processor.decode(logits.argmax(-1)[0]))
```
Weights are stored in float16; load them as float32 as above, especially on CPU.
Clips of up to ~30 s fit comfortably on a 24 GB GPU.
### Long audio (pipeline with chunking)
```python
from transformers import pipeline
asr = pipeline(
"automatic-speech-recognition",
model="phonsobon/PsarAI-ASR-1B",
device=0, # -1 for CPU
chunk_length_s=20, # split long recordings into 20 s windows
stride_length_s=4, # overlap between windows
)
print(asr("long_recording.wav")["text"])
```
## Quick test — ONNX Runtime (no PyTorch)
### Command line
```bash
# Download only the int8 model + script (~1 GB instead of ~9 GB)
hf download phonsobon/PsarAI-ASR-1B --include "onnx/int8/*" "onnx/infer_onnx.py" --local-dir PsarAI-ASR-1B
python PsarAI-ASR-1B/onnx/infer_onnx.py speech.wav # int8 on CPU
python PsarAI-ASR-1B/onnx/infer_onnx.py speech.wav --variant fp16 --device cuda # needs onnx/fp16/*
python PsarAI-ASR-1B/onnx/infer_onnx.py speech.wav --variant fp16 --device dml # Windows DirectML
```
(With older `huggingface_hub`, use `huggingface-cli download` instead of `hf download`.)
### Python
```python
import json
from pathlib import Path
import librosa
import numpy as np
import onnxruntime as ort
from huggingface_hub import snapshot_download
VARIANT = "int8" # "int8" for CPU, "fp16" for GPU, "fp32" for reference
local = snapshot_download("phonsobon/PsarAI-ASR-1B", allow_patterns=[f"onnx/{VARIANT}/*"])
model_dir = Path(local) / "onnx" / VARIANT
vocab = json.loads((model_dir / "vocab.json").read_text(encoding="utf-8"))
id_to_token = {i: t for t, i in vocab.items()}
skip_ids = {vocab[t] for t in ("<pad>", "<s>", "</s>", "<unk>") if t in vocab}
session = ort.InferenceSession(
str(model_dir / "model.onnx"),
providers=["CPUExecutionProvider"], # ["CUDAExecutionProvider", "CPUExecutionProvider"] for GPU
)
def transcribe(path: str) -> str:
audio, _ = librosa.load(path, sr=16_000, mono=True)
x = ((audio - audio.mean()) / np.sqrt(audio.var() + 1e-7)).astype(np.float32)[None, :]
mask = np.ones_like(x, dtype=np.int64)
(logits,) = session.run(["logits"], {"input_values": x, "attention_mask": mask})
# Greedy CTC decoding: merge repeats, drop blanks/special tokens, "|" = space.
out, prev = [], None
for i in logits[0].argmax(-1).tolist():
if i != prev and i not in skip_ids:
out.append(id_to_token[i])
prev = i
return " ".join("".join(out).replace("|", " ").split())
print(transcribe("speech.wav"))
```
## Test accuracy on a public Khmer test set
Computes CER on a few clips of [seanghay/khmer_grkpp_speech](https://huggingface.co/datasets/seanghay/khmer_grkpp_speech)
(real Khmer news speech, never used in training).
```bash
pip install "datasets[audio]" jiwer
```
```python
import re
import jiwer
import torch
from datasets import Audio, load_dataset
from transformers import AutoProcessor, Wav2Vec2ForCTC
REPO = "phonsobon/PsarAI-ASR-1B"
N_CLIPS = 20
device = "cuda" if torch.cuda.is_available() else "cpu"
processor = AutoProcessor.from_pretrained(REPO)
model = Wav2Vec2ForCTC.from_pretrained(REPO, dtype=torch.float32).to(device).eval()
ds = load_dataset("seanghay/khmer_grkpp_speech", split="train", streaming=True)
ds = ds.cast_column("audio", Audio(sampling_rate=16_000))
def norm(text: str) -> str:
# letters only: drop spaces, punctuation and zero-width characters
return re.sub(r"[\s។៖!?.,\"'()\-]", "", text.lower())
refs, hyps = [], []
for sample in ds:
audio = sample["audio"]["array"]
if not 1 <= len(audio) / 16_000 <= 30: # same 1–30 s range as the reported results
continue
inputs = processor(audio, sampling_rate=16_000, return_tensors="pt").to(device)
with torch.inference_mode():
pred = processor.decode(model(**inputs).logits.argmax(-1)[0])
refs.append(norm(sample["transcription"]))
hyps.append(norm(pred))
print(f"REF: {sample['transcription']}\nHYP: {pred}\n")
if len(refs) == N_CLIPS:
break
print(f"CER on {len(refs)} clips: {jiwer.cer(refs, hyps):.2%}")
```
This simple normalization does not convert digits to words, so expect a CER somewhat above
the 13.73% reported below (which uses the full normalization described in *Results*).
## Results
Character error rate (CER, letters only: spaces and punctuation ignored) and word error rate (WER),
lower is better. Before scoring, references and transcripts are normalized the same way: canonical
Khmer character order, one spelling per word (ឲ្យ/អោយ → ឱ្យ), numbers as spoken words (២០១៩ → ពីរពាន់ដប់ប្រាំបួន).
| Test set | Clips | What it is | Greedy | With Khmer LM decoder* |
|---|---:|---|---:|---:|
| grkpp (news) | 348 | [seanghay/khmer_grkpp_speech](https://huggingface.co/datasets/seanghay/khmer_grkpp_speech), new speakers and topics | 13.73% CER | 9.99% CER |
| grkpp, no digits | 125 | same, clips whose transcript has no digits | 11.68% CER | 8.95% CER |
| khmer10 | 10 | DDD speaker never used in training | 4.84% CER | 3.01% CER |
| DDD held-out | 150 | Digital-Divide-Data clips not used in training | 5.87% CER | 2.12% CER |
| Jarvis Khmer | 100 | held-out synthetic voice, Khmer | 1.27% CER | 0.17% CER |
| Jarvis English | 100 | held-out synthetic voice, English | 1.39% WER | 0.49% WER |
| Jarvis mixed | 100 | held-out synthetic voice, Khmer–English | 12.43% WER | 2.99% WER |
\* Greedy is what this model outputs on its own. The second column adds a separate decoder: CTC beam
search with a Khmer character 6-gram and a word trigram, which is not included in this repository.
### PyTorch vs ONNX variants
Benchmark on 136 clips (42 min: English, Khmer, and mixed Khmer–English voice-over).
RTF = processing time / audio duration (lower is faster). Full table: [`results/summary.md`](results/summary.md).
| Variant | Device | CER | RTF | Size |
|---|---|---:|---:|---:|
| PyTorch | CUDA | 6.28% | 0.009 | 1.93 GB |
| ONNX fp32 | CPU | 6.28% | 0.370 | 3.86 GB |
| ONNX fp16 | CPU | 6.28% | 0.732 | 1.93 GB |
| ONNX int8 | CPU | 6.34% | 0.545 | 1.02 GB |
- All ONNX variants match PyTorch accuracy (int8 loses < 0.1 pt CER).
- On **CPU**, use **int8** (smallest) or fp32 (fastest on CPU in this test). fp16 is meant for GPU; on CPU it is slow.
- On **GPU**, use PyTorch or ONNX fp16 with `CUDAExecutionProvider`.

## Training
| | |
|---|---|
| Base | facebook/mms-1b-all, Khmer adapter (`khm`) |
| Trained | adapter layers + output head (~2.3M of ~965M parameters); base model frozen |
| Output vocabulary | the Khmer adapter's 152 characters + `\|` word separator (153) |
| Data | ~15 h of [Digital-Divide-Data/khmer-speech-dataset](https://huggingface.co/datasets/Digital-Divide-Data/khmer-speech-dataset) (real read speech, many speakers); [nphearum/dataset-asr-tts](https://huggingface.co/datasets/nphearum/dataset-asr-tts) synthetic "Jarvis" voice: 3,000 Khmer, 2,000 English and 800 mixed clips |
| Steps | 4,000; AdamW, learning rate 1e-3 with warm-up and linear decay; bf16 autocast |
| Hardware | 1 × RTX 3090 |
See the original model card at [nphearum/PsarAI-ASR-1B](https://huggingface.co/nphearum/PsarAI-ASR-1B) for full details.
ONNX export: opset 17, dynamic batch and length axes, weights in an external `model.onnx.data` file
(the fp32 graph is above protobuf's 2 GB limit). int8 uses dynamic quantization of the transformer
MatMuls only; the convolutional feature encoder stays fp32. See [`onnx/export_onnx.py`](onnx/export_onnx.py).
## Limitations
- **English**: trained and tested only on one synthetic voice; expect clearly weaker results on real English speakers and accents. Output is lowercase.
- **Numbers**: the vocabulary has the Khmer digits ០ ១ ២ ៣ ៥ ៧ but not ៤ ៦ ៨ ៩, so the model sometimes writes numbers in mixed scripts (e.g. `២០១9`).
- **Spaces**: Khmer is written without spaces between words; the model places spaces where its training text had them (between phrases), not between every word.
- **Missing characters**: a few rare Khmer characters are not in the vocabulary (e.g. ៛ ៕ ឰ ឝ ឞ).
- **Domain words and names** it has not heard are often misspelled; an LM decoder with domain text or hotwords helps.
## License and credits
Released under **CC-BY-NC-4.0** (non-commercial), following the base model
[facebook/mms-1b-all](https://huggingface.co/facebook/mms-1b-all).
- Model: [nphearum/PsarAI-ASR-1B](https://huggingface.co/nphearum/PsarAI-ASR-1B)
- Training data: [Digital-Divide-Data/khmer-speech-dataset](https://huggingface.co/datasets/Digital-Divide-Data/khmer-speech-dataset) (CC-BY-SA-4.0),
[nphearum/dataset-asr-tts](https://huggingface.co/datasets/nphearum/dataset-asr-tts) (CC-BY-NC-4.0; its synthetic voice imitates a
character voice whose rights belong to their owners).
Check that your use is permitted before using this model.
```bibtex
@article{pratap2023mms,
title = {Scaling Speech Technology to 1,000+ Languages},
author = {Pratap, Vineel and Tjandra, Andros and Shi, Bowen and others},
journal = {arXiv preprint arXiv:2305.13516},
year = {2023}
}
```
|