Hugging Face's logo

phonsobon
/
PsarAI-ASR-1B

Automatic Speech Recognition
Transformers
ONNX
Safetensors
Khmer
English
wav2vec2
speech
khmer
code-switching
mms
ctc
Eval Results (legacy)
Model card Files
xet
 Community
PsarAI-ASR-1B / README.md
phonsobon's picture
phonsobon
Update README.md
8948417 verified
preview code
|
Raw History Blame Contribute Delete
12.3 kB
---
license: cc-by-nc-4.0
language:
- km
- en
base_model: nphearum/PsarAI-ASR-1B
base_model_relation: quantized
datasets:
- Digital-Divide-Data/khmer-speech-dataset
- nphearum/dataset-asr-tts
- nphearum/khmer-raw-text-3M
pipeline_tag: automatic-speech-recognition
library_name: transformers
tags:
- automatic-speech-recognition
- speech
- khmer
- code-switching
- wav2vec2
- mms
- ctc
- onnx
metrics:
- cer
- wer
model-index:
- name: PsarAI-ASR-1B
results:
- task:
type: automatic-speech-recognition
name: Automatic Speech Recognition
dataset:
type: seanghay/khmer_grkpp_speech
name: Khmer news speech (grkpp, unseen speakers, 1-30 s clips)
metrics:
- type: cer
value: 13.73
name: CER (greedy)
- task:
type: automatic-speech-recognition
name: Automatic Speech Recognition
dataset:
type: nphearum/dataset-asr-tts
name: English (held-out synthetic speech)
metrics:
- type: wer
value: 1.39
name: WER (greedy)
---
# PsarAI-ASR-1B (PyTorch + ONNX)
Speech recognition for **Khmer (ខ្មែរ)**, **English**, and **mixed Khmer–English** speech
(wav2vec2, ~965M parameters, CTC).
This repository contains the original PyTorch weights of
[nphearum/PsarAI-ASR-1B](https://huggingface.co/nphearum/PsarAI-ASR-1B) plus **ONNX exports**
(fp32, fp16, int8) that run with `onnxruntime` only, without PyTorch.
## Files
| Path | Format | Size | Use it for |
|---|---|---:|---|
| `model.safetensors` (+ configs) | PyTorch, fp16 weights | 1.9 GB | 🤗 Transformers, GPU or CPU |
| `onnx/fp32/` | ONNX, fp32 | 3.9 GB | reference accuracy, CPU/GPU |
| `onnx/fp16/` | ONNX, fp16 (fp32 inputs/outputs) | 1.9 GB | GPU (CUDA / DirectML) |
| `onnx/int8/` | ONNX, int8 dynamic quantization | 1.0 GB | **CPU** (smallest, recommended for CPU) |
| `onnx/infer_onnx.py` | script | | command-line transcription with ONNX |
| `onnx/export_onnx.py` | script | | how the ONNX files were made |
| `results/` | benchmark | | CER / speed comparison of all variants |
Each `onnx/<variant>/` folder is self-contained (`model.onnx` + `model.onnx.data` + vocab/tokenizer files).
## Install
```bash
# PyTorch / Transformers
pip install torch transformers librosa soundfile
# ONNX only (no torch needed)
pip install onnxruntime librosa huggingface_hub # CPU
pip install onnxruntime-gpu librosa huggingface_hub # NVIDIA GPU
```
## Quick test — 🤗 Transformers
```python
import librosa
import torch
from transformers import AutoProcessor, Wav2Vec2ForCTC
REPO = "phonsobon/PsarAI-ASR-1B"
device = "cuda" if torch.cuda.is_available() else "cpu"
processor = AutoProcessor.from_pretrained(REPO)
model = Wav2Vec2ForCTC.from_pretrained(REPO, dtype=torch.float32).to(device).eval()
audio, _ = librosa.load("speech.wav", sr=16_000, mono=True) # any format librosa reads; 16 kHz mono
inputs = processor(audio, sampling_rate=16_000, return_tensors="pt").to(device)
with torch.inference_mode():
logits = model(**inputs).logits
print(processor.decode(logits.argmax(-1)[0]))
```
Weights are stored in float16; load them as float32 as above, especially on CPU.
Clips of up to ~30 s fit comfortably on a 24 GB GPU.
### Long audio (pipeline with chunking)
```python
from transformers import pipeline
asr = pipeline(
"automatic-speech-recognition",
model="phonsobon/PsarAI-ASR-1B",
device=0, # -1 for CPU
chunk_length_s=20, # split long recordings into 20 s windows
stride_length_s=4, # overlap between windows
)
print(asr("long_recording.wav")["text"])
```
## Quick test — ONNX Runtime (no PyTorch)
### Command line
```bash
# Download only the int8 model + script (~1 GB instead of ~9 GB)
hf download phonsobon/PsarAI-ASR-1B --include "onnx/int8/*" "onnx/infer_onnx.py" --local-dir PsarAI-ASR-1B
python PsarAI-ASR-1B/onnx/infer_onnx.py speech.wav # int8 on CPU
python PsarAI-ASR-1B/onnx/infer_onnx.py speech.wav --variant fp16 --device cuda # needs onnx/fp16/*
python PsarAI-ASR-1B/onnx/infer_onnx.py speech.wav --variant fp16 --device dml # Windows DirectML
```
(With older `huggingface_hub`, use `huggingface-cli download` instead of `hf download`.)
### Python
```python
import json
from pathlib import Path
import librosa
import numpy as np
import onnxruntime as ort
from huggingface_hub import snapshot_download
VARIANT = "int8" # "int8" for CPU, "fp16" for GPU, "fp32" for reference
local = snapshot_download("phonsobon/PsarAI-ASR-1B", allow_patterns=[f"onnx/{VARIANT}/*"])
model_dir = Path(local) / "onnx" / VARIANT
vocab = json.loads((model_dir / "vocab.json").read_text(encoding="utf-8"))
id_to_token = {i: t for t, i in vocab.items()}
skip_ids = {vocab[t] for t in ("<pad>", "<s>", "</s>", "<unk>") if t in vocab}
session = ort.InferenceSession(
str(model_dir / "model.onnx"),
providers=["CPUExecutionProvider"], # ["CUDAExecutionProvider", "CPUExecutionProvider"] for GPU
)
def transcribe(path: str) -> str:
audio, _ = librosa.load(path, sr=16_000, mono=True)
x = ((audio - audio.mean()) / np.sqrt(audio.var() + 1e-7)).astype(np.float32)[None, :]
mask = np.ones_like(x, dtype=np.int64)
(logits,) = session.run(["logits"], {"input_values": x, "attention_mask": mask})
# Greedy CTC decoding: merge repeats, drop blanks/special tokens, "|" = space.
out, prev = [], None
for i in logits[0].argmax(-1).tolist():
if i != prev and i not in skip_ids:
out.append(id_to_token[i])
prev = i
return " ".join("".join(out).replace("|", " ").split())
print(transcribe("speech.wav"))
```
## Test accuracy on a public Khmer test set
Computes CER on a few clips of [seanghay/khmer_grkpp_speech](https://huggingface.co/datasets/seanghay/khmer_grkpp_speech)
(real Khmer news speech, never used in training).
```bash
pip install "datasets[audio]" jiwer
```
```python
import re
import jiwer
import torch
from datasets import Audio, load_dataset
from transformers import AutoProcessor, Wav2Vec2ForCTC
REPO = "phonsobon/PsarAI-ASR-1B"
N_CLIPS = 20
device = "cuda" if torch.cuda.is_available() else "cpu"
processor = AutoProcessor.from_pretrained(REPO)
model = Wav2Vec2ForCTC.from_pretrained(REPO, dtype=torch.float32).to(device).eval()
ds = load_dataset("seanghay/khmer_grkpp_speech", split="train", streaming=True)
ds = ds.cast_column("audio", Audio(sampling_rate=16_000))
def norm(text: str) -> str:
# letters only: drop spaces, punctuation and zero-width characters
return re.sub(r"[\s​‌‍។៖!?.,\"'()\-]", "", text.lower())
refs, hyps = [], []
for sample in ds:
audio = sample["audio"]["array"]
if not 1 <= len(audio) / 16_000 <= 30: # same 1–30 s range as the reported results
continue
inputs = processor(audio, sampling_rate=16_000, return_tensors="pt").to(device)
with torch.inference_mode():
pred = processor.decode(model(**inputs).logits.argmax(-1)[0])
refs.append(norm(sample["transcription"]))
hyps.append(norm(pred))
print(f"REF: {sample['transcription']}\nHYP: {pred}\n")
if len(refs) == N_CLIPS:
break
print(f"CER on {len(refs)} clips: {jiwer.cer(refs, hyps):.2%}")
```
This simple normalization does not convert digits to words, so expect a CER somewhat above
the 13.73% reported below (which uses the full normalization described in *Results*).
## Results
Character error rate (CER, letters only: spaces and punctuation ignored) and word error rate (WER),
lower is better. Before scoring, references and transcripts are normalized the same way: canonical
Khmer character order, one spelling per word (ឲ្យ/អោយ → ឱ្យ), numbers as spoken words (២០១៩ → ពីរពាន់ដប់ប្រាំបួន).
| Test set | Clips | What it is | Greedy | With Khmer LM decoder* |
|---|---:|---|---:|---:|
| grkpp (news) | 348 | [seanghay/khmer_grkpp_speech](https://huggingface.co/datasets/seanghay/khmer_grkpp_speech), new speakers and topics | 13.73% CER | 9.99% CER |
| grkpp, no digits | 125 | same, clips whose transcript has no digits | 11.68% CER | 8.95% CER |
| khmer10 | 10 | DDD speaker never used in training | 4.84% CER | 3.01% CER |
| DDD held-out | 150 | Digital-Divide-Data clips not used in training | 5.87% CER | 2.12% CER |
| Jarvis Khmer | 100 | held-out synthetic voice, Khmer | 1.27% CER | 0.17% CER |
| Jarvis English | 100 | held-out synthetic voice, English | 1.39% WER | 0.49% WER |
| Jarvis mixed | 100 | held-out synthetic voice, Khmer–English | 12.43% WER | 2.99% WER |
\* Greedy is what this model outputs on its own. The second column adds a separate decoder: CTC beam
search with a Khmer character 6-gram and a word trigram, which is not included in this repository.
### PyTorch vs ONNX variants
Benchmark on 136 clips (42 min: English, Khmer, and mixed Khmer–English voice-over).
RTF = processing time / audio duration (lower is faster). Full table: [`results/summary.md`](results/summary.md).
| Variant | Device | CER | RTF | Size |
|---|---|---:|---:|---:|
| PyTorch | CUDA | 6.28% | 0.009 | 1.93 GB |
| ONNX fp32 | CPU | 6.28% | 0.370 | 3.86 GB |
| ONNX fp16 | CPU | 6.28% | 0.732 | 1.93 GB |
| ONNX int8 | CPU | 6.34% | 0.545 | 1.02 GB |
- All ONNX variants match PyTorch accuracy (int8 loses < 0.1 pt CER).
- On **CPU**, use **int8** (smallest) or fp32 (fastest on CPU in this test). fp16 is meant for GPU; on CPU it is slow.
- On **GPU**, use PyTorch or ONNX fp16 with `CUDAExecutionProvider`.
![Benchmark dashboard](results/charts/dashboard.png)
## Training
| | |
|---|---|
| Base | facebook/mms-1b-all, Khmer adapter (`khm`) |
| Trained | adapter layers + output head (~2.3M of ~965M parameters); base model frozen |
| Output vocabulary | the Khmer adapter's 152 characters + `\|` word separator (153) |
| Data | ~15 h of [Digital-Divide-Data/khmer-speech-dataset](https://huggingface.co/datasets/Digital-Divide-Data/khmer-speech-dataset) (real read speech, many speakers); [nphearum/dataset-asr-tts](https://huggingface.co/datasets/nphearum/dataset-asr-tts) synthetic "Jarvis" voice: 3,000 Khmer, 2,000 English and 800 mixed clips |
| Steps | 4,000; AdamW, learning rate 1e-3 with warm-up and linear decay; bf16 autocast |
| Hardware | 1 × RTX 3090 |
See the original model card at [nphearum/PsarAI-ASR-1B](https://huggingface.co/nphearum/PsarAI-ASR-1B) for full details.
ONNX export: opset 17, dynamic batch and length axes, weights in an external `model.onnx.data` file
(the fp32 graph is above protobuf's 2 GB limit). int8 uses dynamic quantization of the transformer
MatMuls only; the convolutional feature encoder stays fp32. See [`onnx/export_onnx.py`](onnx/export_onnx.py).
## Limitations
- **English**: trained and tested only on one synthetic voice; expect clearly weaker results on real English speakers and accents. Output is lowercase.
- **Numbers**: the vocabulary has the Khmer digits ០ ១ ២ ៣ ៥ ៧ but not ៤ ៦ ៨ ៩, so the model sometimes writes numbers in mixed scripts (e.g. `២០១9`).
- **Spaces**: Khmer is written without spaces between words; the model places spaces where its training text had them (between phrases), not between every word.
- **Missing characters**: a few rare Khmer characters are not in the vocabulary (e.g. ៛ ៕ ឰ ឝ ឞ).
- **Domain words and names** it has not heard are often misspelled; an LM decoder with domain text or hotwords helps.
## License and credits
Released under **CC-BY-NC-4.0** (non-commercial), following the base model
[facebook/mms-1b-all](https://huggingface.co/facebook/mms-1b-all).
- Model: [nphearum/PsarAI-ASR-1B](https://huggingface.co/nphearum/PsarAI-ASR-1B)
- Training data: [Digital-Divide-Data/khmer-speech-dataset](https://huggingface.co/datasets/Digital-Divide-Data/khmer-speech-dataset) (CC-BY-SA-4.0),
[nphearum/dataset-asr-tts](https://huggingface.co/datasets/nphearum/dataset-asr-tts) (CC-BY-NC-4.0; its synthetic voice imitates a
character voice whose rights belong to their owners).
Check that your use is permitted before using this model.
```bibtex
@article{pratap2023mms,
title = {Scaling Speech Technology to 1,000+ Languages},
author = {Pratap, Vineel and Tjandra, Andros and Shi, Bowen and others},
journal = {arXiv preprint arXiv:2305.13516},
year = {2023}
}
```