KFUPM-JRCAI/tnqeet-models

tnqeet models: Arabic Rasm (Dot) Restoration

These are the pretrained models from the EMNLP 2026 paper Lossy Orthography and Linguistic Inference: Computational Approaches to Arabic Rasm Restoration by Maged S. Al-Shaibani and Irfan Ahmad.

tnqeet (تنقيط) is the task of putting the dots back on dotless Arabic text, which is known as the rasm. When the dots are removed, several letters share one shape: for example, ب ت ث all have the same dotless form. The right letter has to be worked out from context.

Dotless (rasm):  لساں الڡٮى سطر وسطر ڡواده
Restored:        لسان الفتى شطر وشطر فؤاده

The repository holds every trained model from the paper. They cover three families: character-level n-gram language models (KenLM), BiLSTM and Transformer sequence labelers trained from scratch, and fine-tuned CANINE encoders.

• 📄 Paper: https://aclanthology.org/2026.emnlp-main.2258/
• 💻 Code: https://github.com/MagedSaeed/tnqeet
• 📦 PyPI: https://pypi.org/project/tnqeet/
• 🔤 Tokenizer: KFUPM-JRCAI/tnqeet-tokenizer
• 🤗 Collection: tnqeet: models, tokenizer, training and testing datasets

Repository contents

Folder	Files	Description
transformer/	3L.ckpt, 6L.ckpt, 9L.ckpt, 12L.ckpt	Transformer encoder sequence labelers with 3–12 layers
lstm/	1L.ckpt … 6L.ckpt	BiLSTM sequence labelers with 1–6 layers
canine/	c.ckpt, s.ckpt	Fine-tuned CANINE-C and CANINE-S
ngrams/	order_2.binary … order_8.binary	Character-level KenLM n-gram models of order 2–8, decoded with beam search

Usage

Install the library:

pip install tnqeet
# CPU-only machines:
pip install "tnqeet[cpu]" --extra-index-url https://download.pytorch.org/whl/cpu

The library downloads the weights from this repository the first time you use them and caches them locally.

from tnqeet import remove_dots
from tnqeet.dotting_models.transformer.models import TransformerDottingModel

rasm = remove_dots("لسان الفتى شطر وشطر فؤاده")   # -> "لساں الڡٮى سطر وسطر ڡواده"

model = TransformerDottingModel.from_pretrained("12L").eval()  # default: "6L"
model.restore_dots(rasm)                                       # -> "لسان الفتى شطر وشطر فؤاده"

restore_dots takes a single string or a list of strings. The neural models are ordinary PyTorch modules, so you can move one to a GPU with .to("cuda").

Method	Class	Sizes (default in bold)
Transformer	tnqeet.dotting_models.transformer.models.TransformerDottingModel	3L, 6L, 9L, 12L
BiLSTM	tnqeet.dotting_models.lstm.models.LSTMDottingModel	1L–6L (4L)
CANINE	tnqeet.dotting_models.canine.models.CanineDottingModel	c, s
n-gram	tnqeet.dotting_models.ngrams.models.NgramDotter	order 2–8 (6)

n-gram models need KenLM built with MAX_ORDER ≥ 8:

MAX_ORDER=8 pip install "git+https://github.com/kpu/kenlm.git"

from tnqeet.dotting_models.ngrams.models import NgramDotter

dotter = NgramDotter.from_pretrained(order=6, beam_size=10)
dotter.restore_dots("لساں الڡٮى سطر وسطر ڡواده")

Training data

The models were trained on 3.59M samples (431.1M words) from eight sources: Arabic Wikipedia, Tashkeela, OSCAR, Ashaar (poetry), AOC, SANAD, IWSLT, and WASM. Together these cover encyclopedic, religious/classical, web, poetry, news, translation/speech, and dialectal social-media text. Only samples with at least 10 words were kept. The dotless input is produced from dotted text with the standard rasm mapping.

Datasets: tnqeet-training-datasets · tnqeet-testing-datasets

Model details

• Transformer: a PyTorch nn.TransformerEncoder with $d_{model}=512$, 8 heads, FFN dimension 2048, dropout 0.1, sinusoidal positions, and a linear output head. It was trained with AdamW (peak LR $3\times10^{-4}$, inverse-square-root schedule, 4000 warmup steps) in bf16.
• BiLSTM: 512-dimensional embeddings, hidden size 512, dropout 0.33. Trained with Adam (LR $10^{-3}$) on a linear-decay schedule.
• CANINE: fine-tuned from google/canine-c / google/canine-s with a token-classification head (AdamW, LR $3\times10^{-5}$, 10% warmup). Predictions are applied only at ambiguous rasm positions.
• n-grams: character-level KenLM models with modified Kneser–Ney smoothing. Spaces are represented as <SPACE> tokens, and singletons are pruned for orders ≥ 3. Decoding is a beam search over the dotted candidates for each rasm.
• Tokenizer: the BiLSTM and Transformer models share one character-level tokenizer, KFUPM-JRCAI/tnqeet-tokenizer. Its vocabulary has 2,891 entries: Arabic letters, punctuation, other characters seen in the training corpus, and special tokens such as [UNK], [PAD], [BOS], [EOS] and [SEP]. The tnqeet library loads it automatically. CANINE works directly on Unicode codepoints and does not use it.
• All neural models are character-level. The maximum sequence length is 1024 characters; longer inputs are split at word boundaries and joined back together after restoration.

Evaluation

The results below are on the test set: 5K samples (500 per domain), about 802K words, across 10 domains. Three of those domains do not appear in the training data: LLM-generated abstracts, Ar/En code-switching, and conversational Q&A. The metrics are word error rate (WER), character error rate (CER), and dotting error rate (DotER). DotER counts errors only on characters that need dots to be told apart. Inference cost is the mean time in milliseconds per input character.

Model	Checkpoint	WER (%)	CER (%)	DotER (%)	ms / char
Transformer
12 layers	transformer/12L.ckpt	6.97	1.33	1.55	0.0445
9 layers	transformer/9L.ckpt	7.01	1.36	1.58	0.0369
6 layers	transformer/6L.ckpt	7.08	1.39	1.62	0.0294
3 layers	transformer/3L.ckpt	8.35	1.64	1.91	0.0214
CANINE
CANINE-S	canine/s.ckpt	7.48	1.48	1.72	0.0965
CANINE-C	canine/c.ckpt	7.66	1.53	1.78	0.0839
BiLSTM
6 layers	lstm/6L.ckpt	8.43	1.59	1.84	0.2086
4 layers	lstm/4L.ckpt	8.49	1.62	1.87	0.1345
2 layers	lstm/2L.ckpt	9.21	1.77	2.06	0.0745
n-gram (KenLM)
8-gram (beam 60)	ngrams/order_8.binary	13.32	2.59	2.99	0.2239
6-gram (beam 50)	ngrams/order_6.binary	13.76	2.72	3.14	0.1848
4-gram (beam 30)	ngrams/order_4.binary	19.66	3.94	4.58	0.1070
3-gram (beam 15)	ngrams/order_3.binary	30.63	6.43	7.52	0.0540

For comparison, the best LLM in the study, Claude Sonnet 4 with 8-shot prompting, reaches 20.42 WER. That is about the level of the 4-gram baseline. The checkpoints not listed above (BiLSTM 1/3/5 layers, n-gram orders 2/5/7) were evaluated only on the validation set; the paper's appendix has those results.

Which model to use:

• Best accuracy: transformer/12L.
• Best accuracy for the cost: transformer/6L (the library default).
• No GPU, or only a few dependencies: the 6-gram or 8-gram models.

Citation

@inproceedings{al-shaibani-ahmad-2026-lossy,
    title = "Lossy Orthography and Linguistic Inference: Computational Approaches to {A}rabic Rasm Restoration",
    author = "Al-Shaibani, Maged S.  and
      Ahmad, Irfan",
    booktitle = "Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing",
    year = "2026",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2026.emnlp-main.2258/",
    pages = "51351--51371"
}

Acknowledgements

This work was supported by King Fahd University of Petroleum and Minerals (KFUPM) and the SDAIA–KFUPM Joint Research Center for Artificial Intelligence.