Update MagharibiBERT model card

README.md

@@ -3,44 +3,73 @@ language:
 - ar
 library_name: transformers
 pipeline_tag: fill-mask
-base_model: UBC-NLP/MARBERT
+base_model:
+- UBC-NLP/MARBERT
 tags:
 - arabic
 - maghrebi-arabic
 - darija
 - arabizi
 - code-switching
+- algeria
+- morocco
+- tunisia
+- libya
 - bert
+- pytorch
 - masked-language-modeling
 ---
 
+<img src="https://i.imgur.com/cwgkvqe.png"
+alt="MagharibiBERT" width="35%" align="right"/>
+
 # MagharibiBERT
 
-**MagharibiBERT** is a regionally adapted BERT-based language model for Maghrebi Arabic.
+**MagharibiBERT** is a regionally adapted Transformer language model
+for **Maghrebi Arabic**.
+
+It is initialized from `UBC-NLP/MARBERT` and further adapted using
+Masked Language Modeling (MLM) on a large multi-source corpus of
+Maghrebi text.
+
+The model targets the linguistic and orthographic diversity of
+North African online communication, including Arabic-script dialect,
+Arabizi, spelling variation, and Arabic-French/English code-switching.
 
-It is designed for North African social-media text, including dialectal Arabic, Arabizi,
-non-standard spelling, and Arabic-French/English code-switching.
+MagharibiBERT is developed by **NumidiaNLP**.
 
-MagharibiBERT was developed by **NumidiaNLP**.
+---
 
 ## Model architecture
 
-- Parent model: `UBC-NLP/MARBERT`
-- Objective: Masked Language Modeling (MLM)
-- Parameters: 162,942,880
-- Hidden size: 768
-- Transformer layers: 12
-- Attention heads: 12
-- Vocabulary size: 100,000
-- Maximum sequence length: 512
+| Property | Value |
+|---|---:|
+| Architecture | BERT |
+| Parent model | `UBC-NLP/MARBERT` |
+| Objective | Masked Language Modeling |
+| Parameters | 162,942,880 |
+| Transformer layers | 12 |
+| Hidden size | 768 |
+| Attention heads | 12 |
+| Vocabulary size | 100,000 |
+| Maximum position embeddings | 512 |
+
+---
 
-## Continued pretraining
+## Regional continued pretraining
 
-MagharibiBERT was obtained through additional regional continued pretraining
-on a curated multi-source corpus covering Maghrebi Arabic varieties.
+MagharibiBERT underwent additional regional continued pretraining on
+a curated mixture of **17 corpora** covering Maghrebi Arabic and
+North African social-media language.
 
+The training mixture includes material associated with Algeria, Tunisia,
+Morocco, Libya, and cross-regional Maghrebi sources, with explicit
+coverage of Arabic script, Arabizi, and code-mixed text.
+
+### Training configuration
+
+- MLM sequence length: 128
 - Optimizer steps: 120,000
-- Training sequence length: 128
 - Effective batch size: 48
 - Learning rate: 2e-5
 - Warmup steps: 6,000
@@ -48,20 +77,111 @@ on a curated multi-source corpus covering Maghrebi Arabic varieties.
 - Precision: BF16
 - Random seed: 42
 
-## Usage
+---
+
+## How to use
+
+### Fill-mask pipeline
+
+```python
+from transformers import pipeline
+
+pipe = pipeline(
+    "fill-mask",
+    model="NumidiaNLP/MagharibiBERT"
+)
+
+results = pipe("واش راك [MASK]", top_k=5)
+
+for result in results:
+    print(result["token_str"], result["score"])
+```
+
+### Example output
+
+Output produced by the released MagharibiBERT checkpoint:
+
+```text
+{"token": "دير", "score": 0.277943}
+{"token": "تهدر", "score": 0.071477}
+{"token": "تدير", "score": 0.062739}
+{"token": "تقول", "score": 0.047855}
+{"token": "تحكي", "score": 0.038360}
+```
+
+> Scores depend on the complete masked context. The example above is
+> provided only to demonstrate model usage rather than as an evaluation result.
+
+### Load the model directly
+
+```python
+from transformers import AutoTokenizer, AutoModelForMaskedLM
+
+model_name = "NumidiaNLP/MagharibiBERT"
 
-Load the model with `NumidiaNLP/MagharibiBERT` using Hugging Face Transformers.
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+model = AutoModelForMaskedLM.from_pretrained(model_name)
+```
+
+---
 
 ## Intended use
 
-The model is intended for research on Maghrebi Arabic NLP, including Arabizi,
-code-switching, sentiment analysis, hate and offensive language detection,
-dialect identification, and related text-processing tasks.
+MagharibiBERT is intended primarily for research involving Maghrebi Arabic,
+including:
+
+- hate and offensive language detection,
+- sentiment analysis,
+- dialect identification,
+- Arabizi processing,
+- code-switched text processing,
+- social-media text classification,
+- representation learning for Maghrebi Arabic.
+
+---
+
+## Intrinsic evaluation
+
+On the fixed held-out MLM test set used during development, the released
+checkpoint obtained:
+
+| Metric | MagharibiBERT |
+|---|---:|
+| MLM loss | 2.8099 |
+| Perplexity | 16.6080 |
+| Masked-token accuracy | 52.42% |
+
+Downstream evaluation against MARBERT and other Arabic language models
+is being conducted separately.
+
+---
+
+## Limitations
+
+Maghrebi Arabic is highly heterogeneous across countries, regions,
+writing systems, platforms, and communities. Performance may therefore
+vary across domains and linguistic varieties.
+
+The model should not be interpreted as representing all North African
+languages or all Maghrebi speakers equally.
+
+---
 
 ## Citation
 
-A formal citation will be added following publication of the accompanying paper.
+The citation for the accompanying MagharibiBERT paper will be added
+following publication.
+
+MagharibiBERT is initialized from MARBERT. Users should also cite the
+original MARBERT work:
+
+> Abdul-Mageed, M., Elmadany, A., and Nagoudi, E. M. B. (2021).
+> ARBERT & MARBERT: Deep Bidirectional Transformers for Arabic.
+> ACL-IJCNLP 2021.
+
+---
 
 ## Organization
 
 **NumidiaNLP — Language Intelligence for North Africa**
+