Naqta - Arabic Punctuation Restoration

Restores punctuation in Modern Standard Arabic. Two components in one repo:

component path task
point-mark model repo root 8 classes (XLM-RoBERTa-large)
span tagger span-tagger/ paired interjection dashes (BIO, XLM-RoBERTa-base)

Usage

from transformers import AutoModelForTokenClassification, AutoTokenizer

tok = AutoTokenizer.from_pretrained("MostafaMaroof/Naqta")
model = AutoModelForTokenClassification.from_pretrained("MostafaMaroof/Naqta")

span_tok = AutoTokenizer.from_pretrained("MostafaMaroof/Naqta", subfolder="span-tagger")
span_model = AutoModelForTokenClassification.from_pretrained(
    "MostafaMaroof/Naqta", subfolder="span-tagger")

Use sliding-window inference for long text. The model was fine-tuned on short windows; a single truncated pass leaves everything past roughly 230 words unpunctuated. See app.py in the Space for a reference implementation.

Evaluation

Token accuracy is the primary metric.

In-domain test set

Held-out split of the training corpus (books, Wikipedia, diacritised text), with the same sources and editorial conventions as training.

metric score
Token accuracy 0.9714
Macro F1 0.8960

External guideline benchmark

91 Modern Standard Arabic passages (about 27k tokens) annotated by an independent Arabic language centre in Saudi Arabia, following its own published punctuation guideline. The centre evaluated the current release and reported:

metric current release
Token accuracy 0.9833

Limitations

  • Exclamation and semicolon tend to be over-predicted: the model finds these marks but does not withhold them reliably.
  • Brackets and quotation marks are not produced. The span tagger covers them in training but their precision is too low to enable in production.
  • Expect lower accuracy on text following a punctuation standard different from the training sources.

Training

XLM-RoBERTa-large fine-tuned on about 1.3M paragraphs (classical and modern books, Wikipedia, news, QA corpora) with weighted focal loss, oversampling of rare marks, and a final adaptation stage on guideline-conforming data.

Downloads last month
264
Safetensors
Model size
0.6B params
Tensor type
F32
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for MostafaMaroof/Naqta

Finetunes
2 models

Space using MostafaMaroof/Naqta 1