Qaf ยท ู‚ุงู: Arabic text-to-speech for Fusha and Damascus Shami

Code on GitHub Base model: OmniVoice License: CC BY-NC 4.0

Most Arabic text-to-speech sounds like a newsreader, and almost none of it speaks the way people in Damascus actually talk. Qaf (ู‚ุงู, the letter Damascenes say as a hamza) reads:

  • Fusha, clear enough for maths lessons: numbers, fractions, decimals and algebra, read the way a Syrian maths teacher reads them.
  • Damascus Shami, written and spoken the Damascus way.

Listen

These are the reference clips the model copies the voice from.

Clip What it says
Fusha

ุงูู†ู’ุธูุฑู’ ู…ูŽุนููŠ ุฅูู„ูŽู‰ ู‡ูŽุฐูู‡ู ุงู„ู’ู…ูŽุณู’ุฃูŽู„ูŽุฉู ุฌูŽูŠูู‘ุฏู‹ุงุŒ ุซูู…ูŽู‘ ู†ูŽุญูู„ูู‘ู‡ูŽุง ู…ูŽุนู‹ุง ุฎูุทู’ูˆูŽุฉู‹ ุจูุฎูุทู’ูˆูŽุฉ.

Shami

ุฃูƒุชุฑ ุดูŠ ุจุญุจูˆ ุจุงู„ุดุงู… ู‡ูˆ ุงู„ู†ุงุณุŒ ูˆู„ุงุฏ ุงู„ุจู„ุฏ ุจูŠุถุญูƒูˆู„ูƒ ุญุชู‰ ู„ูˆ ู…ุง ุจูŠุนุฑููˆูƒ.

Quick start

The easiest way is the small app in the GitHub repo. Type a sentence, press play.

git clone https://github.com/M-Alkassem/qaf && cd qaf
pip install "omnivoice[lora] @ git+https://github.com/k2-fsa/OmniVoice" peft soundfile onnxruntime
pip install --no-deps catt-tashkeel==1.0.2
hf download M-Alkassem/qaf --local-dir voices
python voice/app.py

Then open http://localhost:8768. The first start downloads OmniVoice (about 3.3 GB). With an NVIDIA or Apple GPU a sentence takes seconds; on a plain CPU it takes minutes.

Use it in Python

import torch, soundfile as sf
from omnivoice import OmniVoice
from peft import PeftModel

model = OmniVoice.from_pretrained("k2-fsa/OmniVoice", device_map="cuda", dtype=torch.float16)
peft = PeftModel.from_pretrained(model, "voices/shami_adapter")
peft.to(torch.float16)
model.audio_tokenizer.float()  # the audio tokenizer must stay in float32
peft.eval()

shami = model.generate(text="ุดูˆ ุนู… ุชุนู…ู„ ู‡ู„ู‘ุฃุŸ", ref_audio="voices/shami.wav",
                       ref_text=open("voices/shami.txt").read().strip(), language="apc")[0]
with peft.disable_adapter():  # Fusha uses the base model
    fusha = model.generate(text="ุซูู…ูŽู‘ ู†ูุญูŽู„ูู‘ู„ู ุงู„ู’ุจูŽุณู’ุทูŽ.", ref_audio="voices/fusha.wav",
                           ref_text=open("voices/fusha.txt").read().strip())[0]
sf.write("shami.wav", shami.squeeze(), 24000)  # generate() returns NumPy arrays, 24 kHz

Loading the adapter prints a warning about missing adapter keys in audio_tokenizer. That is expected: those layers share names with the trained ones, start at zero and change nothing.

Tips

  • Fusha: give it fully voweled text. It reads exactly the vowels it is given (ุฃูŽู„ู’ูู‹ุง or ุฃูŽู„ู’ูู, by the grammar). The app adds tashkeel for you with CATT.
  • Shami: write words the way Damascenes say them, like ู‡ู„ู‘ุฃ and ุฃุฏู‘ูŠุด. voice/spelling.py in the repo does this for common words.

What's in this repo

File What it is
fusha.wav, fusha.txt A short Fusha reference clip from a maths lesson, and exactly what it says.
shami.wav, shami.txt The same in Damascus Shami.
shami_adapter/ A LoRA adapter for OmniVoice (about 100 MB) that teaches it Damascus Shami.

Fusha needs no adapter: it is the base OmniVoice model copying fusha.wav.

How the Shami adapter was trained

Base model k2-fsa/OmniVoice
Method OmniVoice's own LoRA recipe
Data Damascus Shami speech, written in the Damascus spelling (not published)
Language id apc (Levantine Arabic)
LoRA rank 16, alpha 32
Schedule learning rate 1e-4, 600 steps, cosine
Hardware 2 ร— T4, fp16
Checkpoint step 400, chosen by ear from steps 200, 400 and 600

Limits

  • Shami was judged by ear only: Whisper can't tell a Damascus accent from another one.
  • Fusha case endings depend on the voweling you give it.
  • On a CPU it is slow (minutes per sentence); use a GPU for anything long.

License and use

CC BY-NC 4.0, because OmniVoice's own weights are non-commercial. Free for learning, research and non-profit work, with credit.

This is a real person's voice. Please don't use it to impersonate anyone or to put words in someone's mouth, and say that the audio is AI-generated when you share it.

Downloads last month
-
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for M-Alkassem/qaf

Finetuned
Qwen/Qwen3-0.6B
Finetuned
k2-fsa/OmniVoice
Adapter
(58)
this model