tiny-voca-arabic / README.md
oddadmix's picture
Update training row
e529f71 verified
|
Raw History Blame Contribute Delete
7.81 kB
---
language: [ar]
license: apache-2.0
library_name: onnxruntime
pipeline_tag: text-to-speech
tags:
- text-to-speech
- tts
- speech-synthesis
- arabic
- msa
- modern-standard-arabic
- tiny-voca
- onnx
- cpu
- small-model
- trained-from-scratch
datasets: [oddadmix/msa-duplex-assistant-tts]
---
# tiny-voca-arabic
A **4.5M-parameter** Modern Standard Arabic (فصحى) text-to-speech model, trained **from scratch** with the
[tiny-voca](https://huggingface.co/n4ze3m/tiny-voca) architecture and a new Arabic phonetic front end.
The waveform generator is included: text in, 24 kHz audio out, about 200× real time on two CPU threads.
> **Based on [tiny-voca](https://huggingface.co/n4ze3m/tiny-voca) by Muhammed Nazeem ([@n4ze3m](https://huggingface.co/n4ze3m)).**
> This model uses the original tiny-voca architecture and training code
> ([github.com/n4ze3m/tiny-voca](https://github.com/n4ze3m/tiny-voca), Apache-2.0). No tiny-voca weights were reused:
> it was trained from scratch on Arabic data. The changes are listed under [Changes from tiny-voca](#changes-from-tiny-voca).
[Demo](https://huggingface.co/spaces/oddadmix/tiny-voca-arabic) ·
[Dataset](https://huggingface.co/datasets/oddadmix/msa-duplex-assistant-tts)
| | |
|---|---|
| Parameters | 4,531,723 (18 MB PyTorch, 21 MB ONNX) |
| Voice | one synthetic MSA voice (`a000`) |
| Audio | 24 kHz mono |
| Training data | 45,737 utterances, 89 h |
| Training | 104k steps, 2.6 h on one GPU |
## Samples
| Text | Audio |
|---|---|
| مرحبًا بكم، هذا نموذج صغير لتحويل النص إلى كلام باللغة العربية الفصحى. | [sample-0.wav](samples/sample-0.wav) |
| تقع مدينة القاهرة على ضفاف نهر النيل، وهي من أكبر المدن في العالم العربي. | [sample-1.wav](samples/sample-1.wav) |
| هل يمكنك أن تخبرني عن حالة الطقس غدًا؟ أريد أن أخرج في نزهة مع أصدقائي. | [sample-2.wav](samples/sample-2.wav) |
| يبلغ عدد سكان المدينة 2,500,000 نسمة، ونسبة الشباب فيها 35%. | [sample-3.wav](samples/sample-3.wav) |
| العلم نور، والجهل ظلام. ومن جدّ وجد، ومن زرع حصد. | [sample-4.wav](samples/sample-4.wav) |
## How it works
1. **Numbers** are read as MSA words (`code/numbers.py`).
2. **Diacritization** with [CATT](https://github.com/abjadai/catt) (encoder-decoder, Apache-2.0).
3. **Phonemes** from a rule-based MSA converter (`code/arabic_frontend.py`), 49 symbols:
sun-letter assimilation, hamzat al-wasl elision and helping vowels, long vowels shortened before
wasl, shadda as a doubled consonant, tanween, tāʾ marbūṭa (`t` inside a phrase, dropped at a pause),
pausal forms at punctuation, emphatic backing of /a/ next to ص ض ط ظ ق (and after خ غ ر), dark lām in
الله, and stress from syllable weight.
4. **Acoustic model + synthesizer**: the tiny-voca network predicts durations, pauses, pitch, voicing and
energy, then a source-filter synthesizer makes the waveform. There is no separate neural vocoder.
**Case endings.** Speakers often drop mid-sentence iʿrāb. While aligning the training data, each
mid-phrase case ending was kept or removed per clip according to which version the audio supports
(a CTC likelihood comparison), so the model learned from what was actually said. About half were dropped.
## Evaluation
Whisper large-v3 transcripts of generated speech against the input text (diacritics and punctuation removed).
Held-out sets come from dialogues not seen in training; the new sentences appear in no split.
| Set | Clips | CER | WER | CER of the original recordings |
|---|---|---|---|---|
| Validation | 25 | 0.5% | 1.2% | 1.0% |
| Test | 25 | 1.8% | 2.4% | 2.1% |
| New sentences | 8 | 0.3% | 1.4% | — |
Most remaining errors are Whisper writing spoken numbers as digits. Generated durations are within 3% of
the recordings and pitch variation matches (3.6 semitones).
## Usage (ONNX Runtime)
```bash
pip install onnxruntime numpy soundfile tqdm
hf download oddadmix/tiny-voca-arabic --local-dir tiny-voca-arabic
```
The front end in `code/` uses the CATT diacritizer; the
[demo Space](https://huggingface.co/spaces/oddadmix/tiny-voca-arabic/tree/main) contains it ready to run
(`app.py`, `tiny_voca_ar/`, `catt_tashkeel/`). Minimal call:
```python
import numpy as np, onnxruntime as ort, soundfile as sf
from tiny_voca_ar.numbers import expand_numbers
from tiny_voca_ar.text import phonemize
session = ort.InferenceSession("tiny-voca-arabic/onnx/model.onnx", providers=["CPUExecutionProvider"])
phones = phonemize(expand_numbers("مرحبًا بكم في عالم الذكاء الاصطناعي."))
one = lambda v: np.array([v], dtype=np.float32)
audio, durations = session.run(None, {
"tokens": np.array([phones.ids], dtype=np.int64), "stress": np.array([phones.stress], dtype=np.int64),
"voice": np.array([0], dtype=np.int64), "speed": one(1.0), "pitch_shift": one(0.0), "pitch_range": one(1.0)})
sf.write("hello.wav", audio[0], 24000)
```
Inputs: `tokens`/`stress` int64 `[1, N]`, `voice` int64 `[1]`, `speed` (1 = as predicted, larger is
faster), `pitch_shift` (added to log F0; 0.058 ≈ one semitone), `pitch_range` (scales intonation). Read
long text one sentence at a time (at most 500 phonemes / 32 s per call) and join the audio.
The PyTorch checkpoint `tiny-voca-arabic.pt` loads with the tiny-voca code
(`tiny_voca.infer.load_model`) using the Arabic `text.py` / `arabic_frontend.py` from `code/`.
## Limitations
- One voice, Modern Standard Arabic only; dialectal text is read with MSA rules.
- Pronunciation follows the automatic diacritization. When CATT picks the wrong reading (for example
نَمَوذج instead of نُموذج), the model says it. Add tashkeel yourself to fix a word.
- Latin script, emoji and symbols other than numbers and % are skipped.
- The ص/ط/ض/ظ emphatics are well covered, but the dark lām only appears in الله.
## Training data
[oddadmix/msa-duplex-assistant-tts](https://huggingface.co/datasets/oddadmix/msa-duplex-assistant-tts)
(batches 0–8): single-speaker synthetic MSA speech with scripted assistant dialogue turns. Of 49,793 clips,
45,737 passed alignment quality checks. The voice is synthetic; please tell listeners when audio
they hear was generated.
## License
Apache-2.0.
## Changes from tiny-voca
The architecture is unchanged from tiny-voca 903f940: same network, synthesizer and losses, at the same size. The changes are:
- An Arabic front end that replaces the English phonemizer: CATT diacritization, MSA phoneme rules, a
49-symbol inventory and a number reader.
- One voice instead of two (F2/M2).
- Aligner label mapping for the Arabic emphatics, and per-clip selection of spoken case endings during annotation.
- Training on 89 h of Arabic speech with packed memory-mapped features, plus an Arabic evaluation that uses
Whisper large-v3 CER.
## Credits
- **Original model, architecture and training code:** [tiny-voca](https://huggingface.co/n4ze3m/tiny-voca)
by Muhammed Nazeem ([@n4ze3m](https://huggingface.co/n4ze3m)), Apache-2.0
([code](https://github.com/n4ze3m/tiny-voca)).
- **Diacritizer:** [CATT](https://github.com/abjadai/catt) by Abjad AI, Apache-2.0.
## Citation
If you use this model, please cite:
```bibtex
@software{oddadmix2026tinyvocaarabic,
author = {oddadmix},
title = {tiny-voca-arabic: a 4.5M-parameter Modern Standard Arabic text-to-speech model},
year = {2026},
url = {https://huggingface.co/oddadmix/tiny-voca-arabic},
license = {Apache-2.0}
}
```
Architecture and training code are from [tiny-voca](https://huggingface.co/n4ze3m/tiny-voca) by Muhammed Nazeem (see Credits).