| --- |
| language: |
| - th |
| - en |
| - de |
| - fr |
| - es |
| - it |
| - nl |
| - pl |
| license: apache-2.0 |
| library_name: cactus |
| datasets: |
| - google/fleurs |
| - n-order/Thai-dialect-corpus |
| - huangjianbyte2024/thai_Elderly_Speech_dataset |
| - typhoon-ai/thai-dialect-isan-dataset |
| base_model: |
| - Cactus-Compute/whistle |
| pipeline_tag: automatic-speech-recognition |
| --- |
| |
| # MaprangSTT |
|
|
| MaprangSTT is a Thai speech-to-Text model built on [Cactus-Compute/whistle](https://huggingface.co/Cactus-Compute/whistle) for **Standard Thai and Regional Thai Dialects** (Central, Isan / Northeastern, Northern / Khummuang, Korat, Southern / Pattani, and Elderly speech). It was trained on open-data datasets. |
|
|
| Trained on **1,022.78 hours (594,231 clips)** of curated Thai speech using quantization-aware LoRA (`CQ W4 STE + A8`) and an expanded Thai SentencePiece vocabulary, the final model compiles into a standalone **32.49 MB** `.cact` archive ready for fast on-device CPU/GPU/mobile inference. |
|
|
| - **GitHub**: [https://github.com/wannaphong/MaprangSTT](https://github.com/wannaphong/MaprangSTT) |
| - **In-Browser Web Demo (`cactus-needle`)**: [https://wannaphong.github.io/MaprangSTT/demo](https://wannaphong.github.io/MaprangSTT/demo) |
|
|
| ## Model Summary |
|
|
| | Property | Value | |
| | :--- | :--- | |
| | **Base Architecture** | Cactus Whistle (8 encoder/decoder layers) | |
| | **Fine-Tuning Method** | Quantization-Aware LoRA (`rank=16`, `alpha=32`, 15 weight groups) + Trainable New Embedding Rows | |
| | **Precision / Quantization** | `W4A8` (4-bit weights with straight-through estimator, 8-bit activations & KV cache) | |
| | **Supported Languages** | `th` (Thai), plus base `en`, `de`, `fr`, `es`, `it`, `nl`, `pl` | |
| | **Vocabulary Expansion** | `+2,067` Thai tokens (`+10,958` SentencePiece pieces + `<\|th\|>` language tag) | |
| | **Standalone `.cact` Size** | **32.49 MB** (`maprang.cact`, 681 tensors) | |
| | **LoRA Adapter Size** | **29.0 MB** (`adapter.safetensors`) | |
|
|
| ### Dataset Summary |
|
|
| | Dataset Source | Split / Text Column | Valid Clips | Duration (Hours) | |
| | :--- | :--- | ---: | ---: | |
| | **Mozilla Common Voice v27 (`th`)** | `validated` − (`dev` + `test`) / `sentence` | 126,690 | 149.41 h | |
| | **Google FLEURS (`th_th`)** | `train` / `transcription` | 2,174 | 6.84 h | |
| | **`n-order/Thai-dialect-corpus`** | `*_train` (all 4 subsets) / `sentence` | 437,563 | 816.74 h | |
| | **`huangjianbyte2024/thai_Elderly_Speech_dataset`** | `train` / `transcription` | 19,198 | 26.56 h | |
| | **`typhoon-ai/thai-dialect-isan-dataset`** | `train` / **`isan_spelling`** | **8,606** | **23.22 h** | |
| | **TOTAL** | **All 5 Datasets Combined** | **594,231** | **1,022.78 h** | |
| |
| *(1,381 clips from `typhoon-ai/thai-dialect-isan-dataset` were filtered out because they were $>30\text{s}$ or contained non-Thai characters after number-to-Thai-words conversion.)* |
| |
| ## Preprocessing & Tokenizer Expansion |
| |
| 1. **Thai Text Normalization & Cleaning**: |
| - Normalized Thai character order and vowel sequences via `pythainlp.util.normalize` (including `ํา` $\rightarrow$ `ำ`). |
| - Applied community-verified transcript fixes from [`thai_commonvoice_dataset`](https://github.com/wannaphong/thai_commonvoice_dataset) (`db-lastest.db` and `list_file.tsv`). |
| 2. **Number-to-Thai-Words Conversion**: |
| - Converted both Thai digits (`๐-๙`) and Arabic digits (`0-9`), comma-formatted numbers, and decimals into spoken Thai words using `pythainlp.util.num_to_thaiword` and `pythainlp.util.digit_to_text`. |
| - Filtered out any remaining clips containing non-Thai characters or exceeding 30 seconds. |
| 3. **Word Tokenization**: |
| - Segmented all reference transcripts into space-delimited words using `pythainlp.tokenize.word_tokenize(..., engine="newmm")`. |
| 4. **Thai National Corpus (TNC) Vocabulary Expansion (`tokens.txt`)**: |
| - Whistle's base SentencePiece model only covers Latin/European scripts (`en, de, fr, es, it, nl, pl`). |
| - We expanded the vocabulary with **2,067 tokens** consisting of all **67 base Thai characters** (`ก-ฮ`, vowels, and tone marks) plus the **top 2,000 most frequent Thai words** from the Thai National Corpus (`pythainlp.corpus.tnc`), adding `10,958` prefix-closed SentencePiece pieces. |
| ## Training Progression & Evaluation |
| |
| ### Hyperparameters |
| - **Epochs**: 3 (`111,420` total steps, `37,140` steps/epoch) |
| - **Batch Size**: 16 |
| - **Optimizer**: AdamW (`peak_lr=1e-3`, warmup `5,571` steps, cosine decay to `0.0`, global grad norm clip `1.0`) |
| - **Hardware**: NVIDIA GeForce RTX 4090 (24 GB VRAM) |
| |
| ### Epoch-by-Epoch Loss |
| |
| | Epoch | Step | Train Loss | Validation Loss (`Common Voice dev`) | |
| | :---: | :---: | :---: | :---: | |
| | **Epoch 1 / 3** | `37,140 / 111,420` | `1.3467` | `0.5841` | |
| | **Epoch 2 / 3** | `74,280 / 111,420` | `0.2417` | `0.4239` | |
| | **Epoch 3 / 3** | `111,420 / 111,420` | **`0.1803`** | **`0.3867`** | |
| ## Evaluation Results |
|
|
| All benchmarks below were evaluated on the exported `W4A8` `.cact` archive using `cactus.transcribe(..., language="th")`. |
| - **Character Error Rate (CER)**: Computed over Thai characters (whitespace removed). |
| - **Word Error Rate (WER)**: Computed over space-delimited tokens segmented via `pythainlp` (`newmm`). |
|
|
| ### Google FLEURS (`th_th`) Test Set — Comparison with Popular Thai ASR Models (`1,021` Clips) |
| |
| Evaluated on all **1,021 clips** of the [`google/fleurs`](https://huggingface.co/datasets/google/fleurs) (`th_th`) test set alongside popular open-source Thai ASR models via [`pythaiasr`](https://github.com/PyThaiNLP/pythaiasr) ([compare_fleurs.py](file:///workspace/whistle-thai/compare_fleurs.py)). All outputs were normalized with the same PyThaiNLP text normalizer, number-to-Thai-words converter, and `newmm` word tokenizer: |
|
|
| | Model | Architecture / Precision | Model Size | Google FLEURS (`th_th`) CER | Google FLEURS (`th_th`) WER | Size Ratio vs Ours | |
| | :---------------------------------------------------------------------------------------------------- | :------------------------------------ | ----------: | :-------------------------: | :-------------------------: | :----------------: | |
| | **`MaprangSTT` (Ours)** | **Cactus Whistle (`W4A8` `.cact`)** | **32.5 MB** | **`13.58%`** | **`31.05%`** | **1.0×** | |
| | [`openai/whisper-small`](https://huggingface.co/openai/whisper-small) | Whisper Small (`244M`, FP32) | 967.0 MB | `24.21%` | `75.74%` | 29.8× larger | |
| | [`airesearch/wav2vec2-large-xlsr-53-th`](https://huggingface.co/airesearch/wav2vec2-large-xlsr-53-th) | Wav2Vec2-Large-XLSR-53 (`315M`, FP32) | 1,260.0 MB | `9.56%` | `35.98%` | 38.8× larger | |
| | `typhoon_nemotron_asr` | Nemotron-3.5-ASR (`0.6B`, INT4 ONNX) | ~450.0 MB | `12.28%` | `27.52%` | 13.8× larger | |
| | [`biodatlab/whisper-small-th-combined`](https://huggingface.co/biodatlab/whisper-small-th-combined) | Whisper Small Thai (`244M`, FP16) | 563.0 MB | `9.32%` | `24.30%` | 17.3× larger | |
| | `typhoon_asr` | FastConformer RNN-T (ONNX) | 435.0 MB | `9.02%` | `20.11%` | 13.4× larger | |
|
|
| > **Key Takeaways**: |
| > - At only **32.5 MB**, **`MaprangSTT`** outperforms [`openai/whisper-small`](https://huggingface.co/openai/whisper-small) (`967 MB`) by **10.63% absolute CER** (`13.58%` vs `24.21%`) and **44.69% absolute WER** (`31.05%` vs `75.74%`). |
| > - It achieves a **4.93% lower WER** (`31.05%` vs `35.98%`) than [`airesearch/wav2vec2-large-xlsr-53-th`](https://huggingface.co/airesearch/wav2vec2-large-xlsr-53-th) while being **38.8× smaller** (`32.5 MB` vs `1.26 GB`). |
| > - It comes within **1.30% CER** of the `0.6B`-parameter `typhoon_nemotron_asr` (`13.58%` vs `12.28%`) at **1/14th the model size**. |
| ## Usage |
|
|
| ### 1. Install Cactus |
|
|
| ```bash |
| pip install cactus-compute huggingface_hub cactus-needle |
| ``` |
|
|
| ### 2. Transcribe Audio in Python |
|
|
| `cactus.transcribe()` expects a 16 kHz mono PCM `.wav` file: |
|
|
| ```python |
| import os |
| os.environ.setdefault("NEEDLE_TELEMETRY", "0") # needle reports usage unless these are set |
| os.environ.setdefault("DO_NOT_TRACK", "1") |
| |
| from huggingface_hub import hf_hub_download |
| from needle import Whistle |
| |
| model = Whistle(hf_hub_download("pythainlp/MaprangSTT", "maprang.cact")) |
| print(model.transcribe("speech.wav", language="th")["text"]) |
| ``` |
|
|
| If your input audio is `.mp3` or a different sample rate, convert it to 16 kHz mono `.wav` first (e.g., with `ffmpeg` or `soundfile` + `soxr`): |
|
|
| ```python |
| import subprocess |
| import tempfile |
| import cactus |
| |
| def transcribe_any_audio(audio_path: str, weights_path: str = "maprang.cact") -> str: |
| with tempfile.NamedTemporaryFile(suffix=".wav") as tmp_wav: |
| subprocess.run( |
| ["ffmpeg", "-y", "-i", audio_path, "-ar", "16000", "-ac", "1", tmp_wav.name], |
| stdout=subprocess.DEVNULL, |
| stderr=subprocess.DEVNULL, |
| check=True, |
| ) |
| res = cactus.transcribe(tmp_wav.name, weights=weights_path, language="th") |
| return res["text"].strip() |
| ``` |
|
|
| ### 3. Rebuild `.cact` Archive from the LoRA Adapter |
|
|
| If you want to build a platform-specific runnable bundle or slice decoder layers (`--layers 4` or `--layers 6`) from `adapter.safetensors`: |
|
|
| ```bash |
| pip install "cactus-compute[train]" |
| |
| # Build full 8-layer standalone .cact archive |
| cactus whistle build \ |
| --lora adapter.safetensors \ |
| --out model.cact |
| |
| # Or build a smaller/faster 4-layer decoder archive |
| cactus whistle build \ |
| --lora adapter.safetensors \ |
| --layers 4 \ |
| --out model-l.cact |
| ``` |
|
|
| ## Repository Files |
|
|
| - `maprang.cact` (`32.49 MB`): Standalone quantized (`W4A8`) Whistle model archive ready for `cactus.transcribe()`. |
| - `adapter.safetensors` (`29.0 MB`): Trained LoRA weights (`rank=16`, `alpha=32`) and expanded Thai vocabulary embedding rows. |
| - `tokens.txt` (`2,067` lines): Base Thai characters and top 2,000 TNC words used to expand the SentencePiece tokenizer. |
|
|
| ## Citation & Acknowledgements |
|
|
| - **Base Model & Framework**: [Cactus Compute — Finetuning Whistle](https://cactuscompute.com/blog/finetuning-whistle) |
| - **NLP Preprocessing & Corpus**: [PyThaiNLP](https://github.com/PyThaiNLP/pythainlp) & [thai_commonvoice_dataset](https://github.com/wannaphong/thai_commonvoice_dataset) |
| - **Speech Datasets**: |
| - [Mozilla Common Voice](https://commonvoice.mozilla.org/) |
| - [Google FLEURS (`google/fleurs`)](https://huggingface.co/datasets/google/fleurs) |
| - [`n-order/Thai-dialect-corpus`](https://huggingface.co/datasets/n-order/Thai-dialect-corpus) |
| - [`huangjianbyte2024/thai_Elderly_Speech_dataset`](https://huggingface.co/datasets/huangjianbyte2024/thai_Elderly_Speech_dataset) |
| - [`typhoon-ai/thai-dialect-isan-dataset`](https://huggingface.co/datasets/typhoon-ai/thai-dialect-isan-dataset) |
