Tashkeel-v4-Saudi

This model is ahmedsamirtarjama/Tashkeel-v4, fine-tuned on about 367k diacritized segments of Saudi speech transcripts. It adds diacritics to Saudi dialect text in the style of the speech labels, which is what text-to-speech input needs, rather than formal Classical or Modern Standard Arabic endings.

Research use only. This is a derivative of Tashkeel-v4 and is distributed under the same Tashkeel Research-Only License. Commercial or business use is not permitted. See LICENSE.

Results

These results are on a held-out test set of 22,381 segments, split by episode, so no test episode appears in training. The metric is the error rate on explicitly marked letters that are not word-final. Lower is better.

Source	Shakkelha base	Shakkelha fine-tuned	Tashkeel-v4 base	This model
All	21.52%	9.75%	17.59%	6.98%
Eman-Fouda/youtube-001-part02-tashkeel	20.05%	9.47%	16.55%	6.90%
Rabe3/maha-host	26.78%	11.66%	22.50%	8.37%
Rabe3/nafas-host-podcast-001	21.01%	8.77%	17.12%	6.47%
Rabe3/sara-host-podcast-001	22.82%	10.64%	18.96%	7.32%
Rabe3/shakhsiya-host-podcast-001	23.16%	9.56%	19.16%	7.02%
Rabe3/socrates-host-youtube-005	18.74%	8.36%	13.28%	5.46%
Rabe3/swalif-host-podcast-001	22.74%	9.56%	18.03%	6.82%

Across all supervised letters, word-final included, the error falls from 28.14% for the base model to 5.51%. The fine-tuned Shakkelha model (2.7M parameters) was trained and scored on the same splits with the same evaluator.

The references are automatic annotations of speech transcripts, not human-verified pronunciations. These scores measure agreement with those labels.

Example

	Text
Input	أبغى وجبة ستريبس دجاج، ومعها بطاطس ومشروب بارد.
Tashkeel-v4 (base)	أَبْغَى وَجْبَةَ سَتْرَيبْسِ دَجَاجٍ، وَمَعَهَا بُطَاْطَسٌ وَمَشْرُوبٌ بَارِدٌ.
This model	أَبْغَى وَجْبَةْ سْتِرْيِبْسْ دْجَاجْ، وَمَعَهَا بَطَاطِسْ وَمَشْرُوبْ بَارِدْ.

Usage

import torch
from transformers import AutoModel, AutoTokenizer

model_id = "Rabe3/Tashkeel-v4-Saudi"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModel.from_pretrained(model_id, trust_remote_code=True)
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model.to(device).eval()

print(model.diacritize("ممكن تعطيني رقم الطلب عشان أشيك لك على حالته؟", tokenizer=tokenizer, device=device))
print(model.diacritize(["حياك الله، تفضل", "وش أقدر أساعدك فيه؟"], tokenizer=tokenizer, device=device, batch_size=64))

Pass in undiacritized text. The model is a character tagger: it never adds, removes or changes letters, digits or punctuation.

Speed

Sentences had a median of 10 words. Measurements use fp32 and diacritize().

Hardware	Setup	Throughput	Latency
2× Xeon Gold 6430 (64 cores)	1 request at a time, 8 threads	28 req/s	34 ms median
2× Xeon Gold 6430 (64 cores)	16 pinned processes × 4 threads	183 req/s	81 ms median, 136 ms p95
RTX 5880 Ada	batch 64	about 400 sentences/s	–

CPU output is identical to GPU output in fp32. bf16 autocast on CPU changed 4 of 152 outputs and was slower for single requests.

Training

• Data: Rabe3/saudi-tashkeel-splits, which has 366,878 train, 19,783 validation and 22,381 test segments. They come from seven Saudi speech sources and are split by episode with a fixed SHA-256 hash.
• Labels:
  • A word with no marks at all is treated as unknown and marginalised.
  • Unmarked letters inside a partly marked word count as "no mark".
  • Non-letters are labelled "no mark".
  • Mark combinations the model can't represent are treated as unknown.
• Loss: a CRF negative log-likelihood over every path consistent with the known labels, plus 0.5 × token cross-entropy on the emissions.
• Optimiser: AdamW, with learning rate 2e-5 for the MARBERTv2 encoder and 1e-4 for the BiLSTM, heads and CRF. 1,000 warmup steps, then linear decay. Batch 64, 3 epochs (17,199 steps), seed 42. The encoder ran in bf16 autocast.
• LSTM packing: the BiLSTM was packed during training and evaluation. With batch size 1, diacritize() gives the same results.
• Checkpoint selection: the best validation score, which came at the final step. Validation error went 16.89% → 7.78% → 7.21% → 7.01% → 6.82% → 6.80% → 6.76% at half-epoch steps. The test split was used once, after selection.
• Hardware: one RTX 5880 Ada, 27 minutes, peak memory 9.6 GB.

training_config.json, history.json and test_results.json hold the full run record.

Limitations

• Speech-label conventions: the model follows the conventions of its training labels. These include a sukun on the first letter of a word that starts with a consonant cluster (سْتِرْيِبْسْ، تْخَيَّل). Check that your TTS voice handles this well.
• Formal text: formal Arabic (MSA or Classical) is no longer the target. Use the base Tashkeel-v4 for that.
• Rare words: rare loanwords and brand names are still the main error source.
• No human-verified test: no human-verified Saudi evaluation set exists yet.

License and attribution

The Tashkeel Research-Only License applies, inherited from Tashkeel-v4 by Ahmed Samir Tarjama. You may use it for non-commercial research and education only. Publications must cite and link the base model.