---
license: cc-by-nc-4.0
language:
- km
- en
base_model: nphearum/PsarAI-ASR-1B
base_model_relation: quantized
datasets:
- Digital-Divide-Data/khmer-speech-dataset
- nphearum/dataset-asr-tts
- nphearum/khmer-raw-text-3M
pipeline_tag: automatic-speech-recognition
library_name: transformers
tags:
- automatic-speech-recognition
- speech
- khmer
- code-switching
- wav2vec2
- mms
- ctc
- onnx
metrics:
- cer
- wer
model-index:
- name: PsarAI-ASR-1B
  results:
  - task:
      type: automatic-speech-recognition
      name: Automatic Speech Recognition
    dataset:
      type: seanghay/khmer_grkpp_speech
      name: Khmer news speech (grkpp, unseen speakers, 1-30 s clips)
    metrics:
    - type: cer
      value: 13.73
      name: CER (greedy)
  - task:
      type: automatic-speech-recognition
      name: Automatic Speech Recognition
    dataset:
      type: nphearum/dataset-asr-tts
      name: English (held-out synthetic speech)
    metrics:
    - type: wer
      value: 1.39
      name: WER (greedy)
---

# PsarAI-ASR-1B (PyTorch + ONNX)

Speech recognition for **Khmer (ខ្មែរ)**, **English**, and **mixed Khmer–English** speech
(wav2vec2, ~965M parameters, CTC).

This repository contains the original PyTorch weights of
[nphearum/PsarAI-ASR-1B](https://huggingface.co/nphearum/PsarAI-ASR-1B) plus **ONNX exports**
(fp32, fp16, int8) that run with `onnxruntime` only, without PyTorch.

## Files

| Path | Format | Size | Use it for |
|---|---|---:|---|
| `model.safetensors` (+ configs) | PyTorch, fp16 weights | 1.9 GB | 🤗 Transformers, GPU or CPU |
| `onnx/fp32/` | ONNX, fp32 | 3.9 GB | reference accuracy, CPU/GPU |
| `onnx/fp16/` | ONNX, fp16 (fp32 inputs/outputs) | 1.9 GB | GPU (CUDA / DirectML) |
| `onnx/int8/` | ONNX, int8 dynamic quantization | 1.0 GB | **CPU** (smallest, recommended for CPU) |
| `onnx/infer_onnx.py` | script | | command-line transcription with ONNX |
| `onnx/export_onnx.py` | script | | how the ONNX files were made |
| `results/` | benchmark | | CER / speed comparison of all variants |

Each `onnx/<variant>/` folder is self-contained (`model.onnx` + `model.onnx.data` + vocab/tokenizer files).

## Install

```bash
# PyTorch / Transformers
pip install torch transformers librosa soundfile

# ONNX only (no torch needed)
pip install onnxruntime librosa huggingface_hub     # CPU
pip install onnxruntime-gpu librosa huggingface_hub # NVIDIA GPU
```

## Quick test — 🤗 Transformers

```python
import librosa
import torch
from transformers import AutoProcessor, Wav2Vec2ForCTC

REPO = "phonsobon/PsarAI-ASR-1B"
device = "cuda" if torch.cuda.is_available() else "cpu"

processor = AutoProcessor.from_pretrained(REPO)
model = Wav2Vec2ForCTC.from_pretrained(REPO, dtype=torch.float32).to(device).eval()

audio, _ = librosa.load("speech.wav", sr=16_000, mono=True)  # any format librosa reads; 16 kHz mono
inputs = processor(audio, sampling_rate=16_000, return_tensors="pt").to(device)
with torch.inference_mode():
    logits = model(**inputs).logits
print(processor.decode(logits.argmax(-1)[0]))
```

Weights are stored in float16; load them as float32 as above, especially on CPU.
Clips of up to ~30 s fit comfortably on a 24 GB GPU.

### Long audio (pipeline with chunking)

```python
from transformers import pipeline

asr = pipeline(
    "automatic-speech-recognition",
    model="phonsobon/PsarAI-ASR-1B",
    device=0,             # -1 for CPU
    chunk_length_s=20,    # split long recordings into 20 s windows
    stride_length_s=4,    # overlap between windows
)
print(asr("long_recording.wav")["text"])
```

## Quick test — ONNX Runtime (no PyTorch)

### Command line

```bash
# Download only the int8 model + script (~1 GB instead of ~9 GB)
hf download phonsobon/PsarAI-ASR-1B --include "onnx/int8/*" "onnx/infer_onnx.py" --local-dir PsarAI-ASR-1B

python PsarAI-ASR-1B/onnx/infer_onnx.py speech.wav                                # int8 on CPU
python PsarAI-ASR-1B/onnx/infer_onnx.py speech.wav --variant fp16 --device cuda   # needs onnx/fp16/*
python PsarAI-ASR-1B/onnx/infer_onnx.py speech.wav --variant fp16 --device dml    # Windows DirectML
```

(With older `huggingface_hub`, use `huggingface-cli download` instead of `hf download`.)

### Python

```python
import json
from pathlib import Path

import librosa
import numpy as np
import onnxruntime as ort
from huggingface_hub import snapshot_download

VARIANT = "int8"  # "int8" for CPU, "fp16" for GPU, "fp32" for reference
local = snapshot_download("phonsobon/PsarAI-ASR-1B", allow_patterns=[f"onnx/{VARIANT}/*"])
model_dir = Path(local) / "onnx" / VARIANT

vocab = json.loads((model_dir / "vocab.json").read_text(encoding="utf-8"))
id_to_token = {i: t for t, i in vocab.items()}
skip_ids = {vocab[t] for t in ("<pad>", "<s>", "</s>", "<unk>") if t in vocab}

session = ort.InferenceSession(
    str(model_dir / "model.onnx"),
    providers=["CPUExecutionProvider"],  # ["CUDAExecutionProvider", "CPUExecutionProvider"] for GPU
)

def transcribe(path: str) -> str:
    audio, _ = librosa.load(path, sr=16_000, mono=True)
    x = ((audio - audio.mean()) / np.sqrt(audio.var() + 1e-7)).astype(np.float32)[None, :]
    mask = np.ones_like(x, dtype=np.int64)
    (logits,) = session.run(["logits"], {"input_values": x, "attention_mask": mask})

    # Greedy CTC decoding: merge repeats, drop blanks/special tokens, "|" = space.
    out, prev = [], None
    for i in logits[0].argmax(-1).tolist():
        if i != prev and i not in skip_ids:
            out.append(id_to_token[i])
        prev = i
    return " ".join("".join(out).replace("|", " ").split())

print(transcribe("speech.wav"))
```

## Test accuracy on a public Khmer test set

Computes CER on a few clips of [seanghay/khmer_grkpp_speech](https://huggingface.co/datasets/seanghay/khmer_grkpp_speech)
(real Khmer news speech, never used in training).

```bash
pip install "datasets[audio]" jiwer
```

```python
import re

import jiwer
import torch
from datasets import Audio, load_dataset
from transformers import AutoProcessor, Wav2Vec2ForCTC

REPO = "phonsobon/PsarAI-ASR-1B"
N_CLIPS = 20
device = "cuda" if torch.cuda.is_available() else "cpu"

processor = AutoProcessor.from_pretrained(REPO)
model = Wav2Vec2ForCTC.from_pretrained(REPO, dtype=torch.float32).to(device).eval()

ds = load_dataset("seanghay/khmer_grkpp_speech", split="train", streaming=True)
ds = ds.cast_column("audio", Audio(sampling_rate=16_000))

def norm(text: str) -> str:
    # letters only: drop spaces, punctuation and zero-width characters
    return re.sub(r"[\s​‌‍។៖!?.,\"'()\-]", "", text.lower())

refs, hyps = [], []
for sample in ds:
    audio = sample["audio"]["array"]
    if not 1 <= len(audio) / 16_000 <= 30:  # same 1–30 s range as the reported results
        continue
    inputs = processor(audio, sampling_rate=16_000, return_tensors="pt").to(device)
    with torch.inference_mode():
        pred = processor.decode(model(**inputs).logits.argmax(-1)[0])
    refs.append(norm(sample["transcription"]))
    hyps.append(norm(pred))
    print(f"REF: {sample['transcription']}\nHYP: {pred}\n")
    if len(refs) == N_CLIPS:
        break

print(f"CER on {len(refs)} clips: {jiwer.cer(refs, hyps):.2%}")
```

This simple normalization does not convert digits to words, so expect a CER somewhat above
the 13.73% reported below (which uses the full normalization described in *Results*).

## Results

Character error rate (CER, letters only: spaces and punctuation ignored) and word error rate (WER),
lower is better. Before scoring, references and transcripts are normalized the same way: canonical
Khmer character order, one spelling per word (ឲ្យ/អោយ → ឱ្យ), numbers as spoken words (២០១៩ → ពីរពាន់ដប់ប្រាំបួន).

| Test set | Clips | What it is | Greedy | With Khmer LM decoder* |
|---|---:|---|---:|---:|
| grkpp (news) | 348 | [seanghay/khmer_grkpp_speech](https://huggingface.co/datasets/seanghay/khmer_grkpp_speech), new speakers and topics | 13.73% CER | 9.99% CER |
| grkpp, no digits | 125 | same, clips whose transcript has no digits | 11.68% CER | 8.95% CER |
| khmer10 | 10 | DDD speaker never used in training | 4.84% CER | 3.01% CER |
| DDD held-out | 150 | Digital-Divide-Data clips not used in training | 5.87% CER | 2.12% CER |
| Jarvis Khmer | 100 | held-out synthetic voice, Khmer | 1.27% CER | 0.17% CER |
| Jarvis English | 100 | held-out synthetic voice, English | 1.39% WER | 0.49% WER |
| Jarvis mixed | 100 | held-out synthetic voice, Khmer–English | 12.43% WER | 2.99% WER |

\* Greedy is what this model outputs on its own. The second column adds a separate decoder: CTC beam
search with a Khmer character 6-gram and a word trigram, which is not included in this repository.

### PyTorch vs ONNX variants

Benchmark on 136 clips (42 min: English, Khmer, and mixed Khmer–English voice-over).
RTF = processing time / audio duration (lower is faster). Full table: [`results/summary.md`](results/summary.md).

| Variant | Device | CER | RTF | Size |
|---|---|---:|---:|---:|
| PyTorch | CUDA | 6.28% | 0.009 | 1.93 GB |
| ONNX fp32 | CPU | 6.28% | 0.370 | 3.86 GB |
| ONNX fp16 | CPU | 6.28% | 0.732 | 1.93 GB |
| ONNX int8 | CPU | 6.34% | 0.545 | 1.02 GB |

- All ONNX variants match PyTorch accuracy (int8 loses < 0.1 pt CER).
- On **CPU**, use **int8** (smallest) or fp32 (fastest on CPU in this test). fp16 is meant for GPU; on CPU it is slow.
- On **GPU**, use PyTorch or ONNX fp16 with `CUDAExecutionProvider`.

![Benchmark dashboard](results/charts/dashboard.png)

## Training

| | |
|---|---|
| Base | facebook/mms-1b-all, Khmer adapter (`khm`) |
| Trained | adapter layers + output head (~2.3M of ~965M parameters); base model frozen |
| Output vocabulary | the Khmer adapter's 152 characters + `\|` word separator (153) |
| Data | ~15 h of [Digital-Divide-Data/khmer-speech-dataset](https://huggingface.co/datasets/Digital-Divide-Data/khmer-speech-dataset) (real read speech, many speakers); [nphearum/dataset-asr-tts](https://huggingface.co/datasets/nphearum/dataset-asr-tts) synthetic "Jarvis" voice: 3,000 Khmer, 2,000 English and 800 mixed clips |
| Steps | 4,000; AdamW, learning rate 1e-3 with warm-up and linear decay; bf16 autocast |
| Hardware | 1 × RTX 3090 |

See the original model card at [nphearum/PsarAI-ASR-1B](https://huggingface.co/nphearum/PsarAI-ASR-1B) for full details.

ONNX export: opset 17, dynamic batch and length axes, weights in an external `model.onnx.data` file
(the fp32 graph is above protobuf's 2 GB limit). int8 uses dynamic quantization of the transformer
MatMuls only; the convolutional feature encoder stays fp32. See [`onnx/export_onnx.py`](onnx/export_onnx.py).

## Limitations

- **English**: trained and tested only on one synthetic voice; expect clearly weaker results on real English speakers and accents. Output is lowercase.
- **Numbers**: the vocabulary has the Khmer digits ០ ១ ២ ៣ ៥ ៧ but not ៤ ៦ ៨ ៩, so the model sometimes writes numbers in mixed scripts (e.g. `២០១9`).
- **Spaces**: Khmer is written without spaces between words; the model places spaces where its training text had them (between phrases), not between every word.
- **Missing characters**: a few rare Khmer characters are not in the vocabulary (e.g. ៛ ៕ ឰ ឝ ឞ).
- **Domain words and names** it has not heard are often misspelled; an LM decoder with domain text or hotwords helps.

## License and credits

Released under **CC-BY-NC-4.0** (non-commercial), following the base model
[facebook/mms-1b-all](https://huggingface.co/facebook/mms-1b-all).

- Model: [nphearum/PsarAI-ASR-1B](https://huggingface.co/nphearum/PsarAI-ASR-1B)
- Training data: [Digital-Divide-Data/khmer-speech-dataset](https://huggingface.co/datasets/Digital-Divide-Data/khmer-speech-dataset) (CC-BY-SA-4.0),
  [nphearum/dataset-asr-tts](https://huggingface.co/datasets/nphearum/dataset-asr-tts) (CC-BY-NC-4.0; its synthetic voice imitates a
  character voice whose rights belong to their owners).

Check that your use is permitted before using this model.

```bibtex
@article{pratap2023mms,
  title   = {Scaling Speech Technology to 1,000+ Languages},
  author  = {Pratap, Vineel and Tjandra, Andros and Shi, Bowen and others},
  journal = {arXiv preprint arXiv:2305.13516},
  year    = {2023}
}
```