File size: 11,208 Bytes
4c94eeb 5c60154 4c94eeb 5c60154 4c94eeb 5c60154 6a060c4 5c60154 4b9d895 5c60154 e071365 5c60154 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 | ---
language:
- th
- en
- de
- fr
- es
- it
- nl
- pl
license: apache-2.0
library_name: cactus
datasets:
- google/fleurs
- n-order/Thai-dialect-corpus
- huangjianbyte2024/thai_Elderly_Speech_dataset
- typhoon-ai/thai-dialect-isan-dataset
base_model:
- Cactus-Compute/whistle
pipeline_tag: automatic-speech-recognition
---
# MaprangSTT
MaprangSTT is a Thai speech-to-Text model built on [Cactus-Compute/whistle](https://huggingface.co/Cactus-Compute/whistle) for **Standard Thai and Regional Thai Dialects** (Central, Isan / Northeastern, Northern / Khummuang, Korat, Southern / Pattani, and Elderly speech). It was trained on open-data datasets.
Trained on **1,022.78 hours (594,231 clips)** of curated Thai speech using quantization-aware LoRA (`CQ W4 STE + A8`) and an expanded Thai SentencePiece vocabulary, the final model compiles into a standalone **32.49 MB** `.cact` archive ready for fast on-device CPU/GPU/mobile inference.
- **GitHub**: [https://github.com/wannaphong/MaprangSTT](https://github.com/wannaphong/MaprangSTT)
- **In-Browser Web Demo (`cactus-needle`)**: [https://wannaphong.github.io/MaprangSTT/demo](https://wannaphong.github.io/MaprangSTT/demo)
## Model Summary
| Property | Value |
| :--- | :--- |
| **Base Architecture** | Cactus Whistle (8 encoder/decoder layers) |
| **Fine-Tuning Method** | Quantization-Aware LoRA (`rank=16`, `alpha=32`, 15 weight groups) + Trainable New Embedding Rows |
| **Precision / Quantization** | `W4A8` (4-bit weights with straight-through estimator, 8-bit activations & KV cache) |
| **Supported Languages** | `th` (Thai), plus base `en`, `de`, `fr`, `es`, `it`, `nl`, `pl` |
| **Vocabulary Expansion** | `+2,067` Thai tokens (`+10,958` SentencePiece pieces + `<\|th\|>` language tag) |
| **Standalone `.cact` Size** | **32.49 MB** (`maprang.cact`, 681 tensors) |
| **LoRA Adapter Size** | **29.0 MB** (`adapter.safetensors`) |
### Dataset Summary
| Dataset Source | Split / Text Column | Valid Clips | Duration (Hours) |
| :--- | :--- | ---: | ---: |
| **Mozilla Common Voice v27 (`th`)** | `validated` − (`dev` + `test`) / `sentence` | 126,690 | 149.41 h |
| **Google FLEURS (`th_th`)** | `train` / `transcription` | 2,174 | 6.84 h |
| **`n-order/Thai-dialect-corpus`** | `*_train` (all 4 subsets) / `sentence` | 437,563 | 816.74 h |
| **`huangjianbyte2024/thai_Elderly_Speech_dataset`** | `train` / `transcription` | 19,198 | 26.56 h |
| **`typhoon-ai/thai-dialect-isan-dataset`** | `train` / **`isan_spelling`** | **8,606** | **23.22 h** |
| **TOTAL** | **All 5 Datasets Combined** | **594,231** | **1,022.78 h** |
*(1,381 clips from `typhoon-ai/thai-dialect-isan-dataset` were filtered out because they were $>30\text{s}$ or contained non-Thai characters after number-to-Thai-words conversion.)*
## Preprocessing & Tokenizer Expansion
1. **Thai Text Normalization & Cleaning**:
- Normalized Thai character order and vowel sequences via `pythainlp.util.normalize` (including `ํา` $\rightarrow$ `ำ`).
- Applied community-verified transcript fixes from [`thai_commonvoice_dataset`](https://github.com/wannaphong/thai_commonvoice_dataset) (`db-lastest.db` and `list_file.tsv`).
2. **Number-to-Thai-Words Conversion**:
- Converted both Thai digits (`๐-๙`) and Arabic digits (`0-9`), comma-formatted numbers, and decimals into spoken Thai words using `pythainlp.util.num_to_thaiword` and `pythainlp.util.digit_to_text`.
- Filtered out any remaining clips containing non-Thai characters or exceeding 30 seconds.
3. **Word Tokenization**:
- Segmented all reference transcripts into space-delimited words using `pythainlp.tokenize.word_tokenize(..., engine="newmm")`.
4. **Thai National Corpus (TNC) Vocabulary Expansion (`tokens.txt`)**:
- Whistle's base SentencePiece model only covers Latin/European scripts (`en, de, fr, es, it, nl, pl`).
- We expanded the vocabulary with **2,067 tokens** consisting of all **67 base Thai characters** (`ก-ฮ`, vowels, and tone marks) plus the **top 2,000 most frequent Thai words** from the Thai National Corpus (`pythainlp.corpus.tnc`), adding `10,958` prefix-closed SentencePiece pieces.
## Training Progression & Evaluation
### Hyperparameters
- **Epochs**: 3 (`111,420` total steps, `37,140` steps/epoch)
- **Batch Size**: 16
- **Optimizer**: AdamW (`peak_lr=1e-3`, warmup `5,571` steps, cosine decay to `0.0`, global grad norm clip `1.0`)
- **Hardware**: NVIDIA GeForce RTX 4090 (24 GB VRAM)
### Epoch-by-Epoch Loss
| Epoch | Step | Train Loss | Validation Loss (`Common Voice dev`) |
| :---: | :---: | :---: | :---: |
| **Epoch 1 / 3** | `37,140 / 111,420` | `1.3467` | `0.5841` |
| **Epoch 2 / 3** | `74,280 / 111,420` | `0.2417` | `0.4239` |
| **Epoch 3 / 3** | `111,420 / 111,420` | **`0.1803`** | **`0.3867`** |
## Evaluation Results
All benchmarks below were evaluated on the exported `W4A8` `.cact` archive using `cactus.transcribe(..., language="th")`.
- **Character Error Rate (CER)**: Computed over Thai characters (whitespace removed).
- **Word Error Rate (WER)**: Computed over space-delimited tokens segmented via `pythainlp` (`newmm`).
### Google FLEURS (`th_th`) Test Set — Comparison with Popular Thai ASR Models (`1,021` Clips)
Evaluated on all **1,021 clips** of the [`google/fleurs`](https://huggingface.co/datasets/google/fleurs) (`th_th`) test set alongside popular open-source Thai ASR models via [`pythaiasr`](https://github.com/PyThaiNLP/pythaiasr) ([compare_fleurs.py](file:///workspace/whistle-thai/compare_fleurs.py)). All outputs were normalized with the same PyThaiNLP text normalizer, number-to-Thai-words converter, and `newmm` word tokenizer:
| Model | Architecture / Precision | Model Size | Google FLEURS (`th_th`) CER | Google FLEURS (`th_th`) WER | Size Ratio vs Ours |
| :---------------------------------------------------------------------------------------------------- | :------------------------------------ | ----------: | :-------------------------: | :-------------------------: | :----------------: |
| **`MaprangSTT` (Ours)** | **Cactus Whistle (`W4A8` `.cact`)** | **32.5 MB** | **`13.58%`** | **`31.05%`** | **1.0×** |
| [`openai/whisper-small`](https://huggingface.co/openai/whisper-small) | Whisper Small (`244M`, FP32) | 967.0 MB | `24.21%` | `75.74%` | 29.8× larger |
| [`airesearch/wav2vec2-large-xlsr-53-th`](https://huggingface.co/airesearch/wav2vec2-large-xlsr-53-th) | Wav2Vec2-Large-XLSR-53 (`315M`, FP32) | 1,260.0 MB | `9.56%` | `35.98%` | 38.8× larger |
| `typhoon_nemotron_asr` | Nemotron-3.5-ASR (`0.6B`, INT4 ONNX) | ~450.0 MB | `12.28%` | `27.52%` | 13.8× larger |
| [`biodatlab/whisper-small-th-combined`](https://huggingface.co/biodatlab/whisper-small-th-combined) | Whisper Small Thai (`244M`, FP16) | 563.0 MB | `9.32%` | `24.30%` | 17.3× larger |
| `typhoon_asr` | FastConformer RNN-T (ONNX) | 435.0 MB | `9.02%` | `20.11%` | 13.4× larger |
> **Key Takeaways**:
> - At only **32.5 MB**, **`MaprangSTT`** outperforms [`openai/whisper-small`](https://huggingface.co/openai/whisper-small) (`967 MB`) by **10.63% absolute CER** (`13.58%` vs `24.21%`) and **44.69% absolute WER** (`31.05%` vs `75.74%`).
> - It achieves a **4.93% lower WER** (`31.05%` vs `35.98%`) than [`airesearch/wav2vec2-large-xlsr-53-th`](https://huggingface.co/airesearch/wav2vec2-large-xlsr-53-th) while being **38.8× smaller** (`32.5 MB` vs `1.26 GB`).
> - It comes within **1.30% CER** of the `0.6B`-parameter `typhoon_nemotron_asr` (`13.58%` vs `12.28%`) at **1/14th the model size**.
## Usage
### 1. Install Cactus
```bash
pip install cactus-compute huggingface_hub cactus-needle
```
### 2. Transcribe Audio in Python
`cactus.transcribe()` expects a 16 kHz mono PCM `.wav` file:
```python
import os
os.environ.setdefault("NEEDLE_TELEMETRY", "0") # needle reports usage unless these are set
os.environ.setdefault("DO_NOT_TRACK", "1")
from huggingface_hub import hf_hub_download
from needle import Whistle
model = Whistle(hf_hub_download("pythainlp/MaprangSTT", "maprang.cact"))
print(model.transcribe("speech.wav", language="th")["text"])
```
If your input audio is `.mp3` or a different sample rate, convert it to 16 kHz mono `.wav` first (e.g., with `ffmpeg` or `soundfile` + `soxr`):
```python
import subprocess
import tempfile
import cactus
def transcribe_any_audio(audio_path: str, weights_path: str = "maprang.cact") -> str:
with tempfile.NamedTemporaryFile(suffix=".wav") as tmp_wav:
subprocess.run(
["ffmpeg", "-y", "-i", audio_path, "-ar", "16000", "-ac", "1", tmp_wav.name],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
check=True,
)
res = cactus.transcribe(tmp_wav.name, weights=weights_path, language="th")
return res["text"].strip()
```
### 3. Rebuild `.cact` Archive from the LoRA Adapter
If you want to build a platform-specific runnable bundle or slice decoder layers (`--layers 4` or `--layers 6`) from `adapter.safetensors`:
```bash
pip install "cactus-compute[train]"
# Build full 8-layer standalone .cact archive
cactus whistle build \
--lora adapter.safetensors \
--out model.cact
# Or build a smaller/faster 4-layer decoder archive
cactus whistle build \
--lora adapter.safetensors \
--layers 4 \
--out model-l.cact
```
## Repository Files
- `maprang.cact` (`32.49 MB`): Standalone quantized (`W4A8`) Whistle model archive ready for `cactus.transcribe()`.
- `adapter.safetensors` (`29.0 MB`): Trained LoRA weights (`rank=16`, `alpha=32`) and expanded Thai vocabulary embedding rows.
- `tokens.txt` (`2,067` lines): Base Thai characters and top 2,000 TNC words used to expand the SentencePiece tokenizer.
## Citation & Acknowledgements
- **Base Model & Framework**: [Cactus Compute — Finetuning Whistle](https://cactuscompute.com/blog/finetuning-whistle)
- **NLP Preprocessing & Corpus**: [PyThaiNLP](https://github.com/PyThaiNLP/pythainlp) & [thai_commonvoice_dataset](https://github.com/wannaphong/thai_commonvoice_dataset)
- **Speech Datasets**:
- [Mozilla Common Voice](https://commonvoice.mozilla.org/)
- [Google FLEURS (`google/fleurs`)](https://huggingface.co/datasets/google/fleurs)
- [`n-order/Thai-dialect-corpus`](https://huggingface.co/datasets/n-order/Thai-dialect-corpus)
- [`huangjianbyte2024/thai_Elderly_Speech_dataset`](https://huggingface.co/datasets/huangjianbyte2024/thai_Elderly_Speech_dataset)
- [`typhoon-ai/thai-dialect-isan-dataset`](https://huggingface.co/datasets/typhoon-ai/thai-dialect-isan-dataset) |