Instructions to use M-Alkassem/qaf with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use M-Alkassem/qaf with PEFT:
Task type is invalid.
- Notebooks
- Google Colab
- Kaggle
Most Arabic text-to-speech sounds like a newsreader, and almost none of it speaks the way people in Damascus actually talk. Qaf (ูุงู, the letter Damascenes say as a hamza) reads:
- Fusha, clear enough for maths lessons: numbers, fractions, decimals and algebra, read the way a Syrian maths teacher reads them.
- Damascus Shami, written and spoken the Damascus way.
Listen
These are the reference clips the model copies the voice from.
| Clip | What it says | |
|---|---|---|
| Fusha | ุงูููุธูุฑู ู ูุนูู ุฅูููู ููุฐููู ุงููู ูุณูุฃูููุฉู ุฌููููุฏูุงุ ุซูู ูู ููุญููููููุง ู ูุนูุง ุฎูุทูููุฉู ุจูุฎูุทูููุฉ. |
|
| Shami | ุฃูุชุฑ ุดู ุจุญุจู ุจุงูุดุงู ูู ุงููุงุณุ ููุงุฏ ุงูุจูุฏ ุจูุถุญูููู ุญุชู ูู ู ุง ุจูุนุฑููู. |
Quick start
The easiest way is the small app in the GitHub repo. Type a sentence, press play.
git clone https://github.com/M-Alkassem/qaf && cd qaf
pip install "omnivoice[lora] @ git+https://github.com/k2-fsa/OmniVoice" peft soundfile onnxruntime
pip install --no-deps catt-tashkeel==1.0.2
hf download M-Alkassem/qaf --local-dir voices
python voice/app.py
Then open http://localhost:8768. The first start downloads OmniVoice (about 3.3 GB). With an NVIDIA or Apple GPU a sentence takes seconds; on a plain CPU it takes minutes.
Use it in Python
import torch, soundfile as sf
from omnivoice import OmniVoice
from peft import PeftModel
model = OmniVoice.from_pretrained("k2-fsa/OmniVoice", device_map="cuda", dtype=torch.float16)
peft = PeftModel.from_pretrained(model, "voices/shami_adapter")
peft.to(torch.float16)
model.audio_tokenizer.float() # the audio tokenizer must stay in float32
peft.eval()
shami = model.generate(text="ุดู ุนู
ุชุนู
ู ูููุฃุ", ref_audio="voices/shami.wav",
ref_text=open("voices/shami.txt").read().strip(), language="apc")[0]
with peft.disable_adapter(): # Fusha uses the base model
fusha = model.generate(text="ุซูู
ูู ููุญูููููู ุงููุจูุณูุทู.", ref_audio="voices/fusha.wav",
ref_text=open("voices/fusha.txt").read().strip())[0]
sf.write("shami.wav", shami.squeeze(), 24000) # generate() returns NumPy arrays, 24 kHz
Loading the adapter prints a warning about missing adapter keys in
audio_tokenizer. That is expected: those layers share names with the trained ones, start at zero and change nothing.
Tips
- Fusha: give it fully voweled text. It reads exactly the vowels it is given
(
ุฃูููููุงorุฃููููู, by the grammar). The app adds tashkeel for you with CATT. - Shami: write words the way Damascenes say them, like
ูููุฃandุฃุฏููุด.voice/spelling.pyin the repo does this for common words.
What's in this repo
| File | What it is |
|---|---|
fusha.wav, fusha.txt |
A short Fusha reference clip from a maths lesson, and exactly what it says. |
shami.wav, shami.txt |
The same in Damascus Shami. |
shami_adapter/ |
A LoRA adapter for OmniVoice (about 100 MB) that teaches it Damascus Shami. |
Fusha needs no adapter: it is the base OmniVoice model copying fusha.wav.
How the Shami adapter was trained
| Base model | k2-fsa/OmniVoice |
| Method | OmniVoice's own LoRA recipe |
| Data | Damascus Shami speech, written in the Damascus spelling (not published) |
| Language id | apc (Levantine Arabic) |
| LoRA | rank 16, alpha 32 |
| Schedule | learning rate 1e-4, 600 steps, cosine |
| Hardware | 2 ร T4, fp16 |
| Checkpoint | step 400, chosen by ear from steps 200, 400 and 600 |
Limits
- Shami was judged by ear only: Whisper can't tell a Damascus accent from another one.
- Fusha case endings depend on the voweling you give it.
- On a CPU it is slow (minutes per sentence); use a GPU for anything long.
License and use
CC BY-NC 4.0, because OmniVoice's own weights are non-commercial. Free for learning, research and non-profit work, with credit.
This is a real person's voice. Please don't use it to impersonate anyone or to put words in someone's mouth, and say that the audio is AI-generated when you share it.
- Downloads last month
- -