Instructions to use algerian-nlp/BENKHLOUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sentence-transformers
How to use algerian-nlp/BENKHLOUF with sentence-transformers:
from sentence_transformers import SentenceTransformer model = SentenceTransformer("algerian-nlp/BENKHLOUF") sentences = [ "واش راك خويا لابس عليك؟", "wesh rak khoya cava chwiya?", "كيراك خويا كلش مليح؟", "السوق بعيد على الدار بزاف" ] embeddings = model.encode(sentences) similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [4, 4] - Notebooks
- Google Colab
- Kaggle
BENKHLOUF
BENKHLOUF (بن خلوف) is a 278.1M-parameter Matryoshka sentence embedding model for Algerian Darija (arq) across Arabic script, Latin Arabizi, and French code-switching, trained with Matryoshka Representation Learning (MRL) over Multiple Negatives Ranking Loss on 123,602 decontaminated sentence pairs.
On 1,000 held-out Algerian Darija validation pairs, BENKHLOUF achieves 0.853 mean positive cosine similarity at 768 dimensions and 0.897 at its 64-dimensional Matryoshka slice (a 12× reduction in vector storage), with 0.001 mean negative cosine (+0.852 contrastive margin) and 0.708 cross-script Arabizi-to-Arabic alignment.
| Task | Sentence Embeddings, Cross-Script Retrieval & Semantic Similarity |
| Architecture | 12L × 768 Mean-Pooled Bi-Encoder with Matryoshka Slices (768 → 64d) |
| Parameters | 278,075,904 (209.6M backbone + 68.4M expanded embeddings) |
| Input / Scripts | Arabic script (arq-Arab), Latin Arabizi (arq-Latn), FR/EN code-switch (max 128 tokens) |
| License (Weights) | Apache-2.0 |
| Author(s) | Abderahmane Ainouche, Algerian NLP Collective |
Quickstart
Install dependencies:
pip install sentence-transformers torch
Standard 768d Embeddings (sentence-transformers)
from sentence_transformers import SentenceTransformer
REPO = "algerian-nlp/BENKHLOUF"
model = SentenceTransformer(REPO)
sentences = [
"واش راك خويا لابس عليك؟", # Arabic script
"wesh rak khoya cava chwiya?", # Latin Arabizi
"السوق راه بعيد على الدار بزاف", # Unrelated negative
]
embeddings = model.encode(sentences, normalize_embeddings=True)
similarity = embeddings @ embeddings.T
print(similarity.shape)
# (3, 3)
print(f"Arabic script <-> Arabizi similarity: {similarity[0, 1]:.3f}")
# Arabic script <-> Arabizi similarity: 0.424
print(f"Arabic script <-> Unrelated similarity: {similarity[0, 2]:.3f}")
# Arabic script <-> Unrelated similarity: 0.034
Matryoshka 64d Truncation (12× Smaller Index)
import numpy as np
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("algerian-nlp/BENKHLOUF")
sentences = [
"واش راك خويا لابس عليك؟",
"wesh rak khoya cava chwiya?",
]
embeddings = model.encode(sentences)
embeddings_64d = embeddings[:, :64]
embeddings_64d = embeddings_64d / np.linalg.norm(embeddings_64d, axis=1, keepdims=True)
sim_64d = np.dot(embeddings_64d[0], embeddings_64d[1])
print(f"64d cosine similarity: {sim_64d:.3f}")
# 64d cosine similarity: 0.463
Direct transformers Usage
import torch
import torch.nn.functional as F
from transformers import AutoModel, AutoTokenizer
REPO = "algerian-nlp/BENKHLOUF"
tokenizer = AutoTokenizer.from_pretrained(REPO)
model = AutoModel.from_pretrained(REPO).eval()
def mean_pooling(model_output, attention_mask):
token_embeddings = model_output[0]
mask = attention_mask.unsqueeze(-1).expand(token_embeddings.size()).float()
return torch.sum(token_embeddings * mask, 1) / torch.clamp(mask.sum(1), min=1e-9)
texts = ["واش راك خويا", "wesh rak khoya"]
inputs = tokenizer(texts, padding=True, truncation=True, max_length=128, return_tensors="pt")
with torch.inference_mode():
outputs = model(**inputs)
embeddings = F.normalize(mean_pooling(outputs, inputs["attention_mask"]), p=2, dim=1)
print(embeddings.shape)
# torch.Size([2, 768])
Evaluation
Evaluated on 1,000 held-out Algerian Darija validation sentence pairs (2,000 sentences across distinct semantic clusters) after 3 epochs (2,898 global steps, batch size 64, single deterministic pass, seed 42).
Semantic Alignment & Contrastive Margin
| Evaluation Dimension | Dim | Measured Value | Target | Property |
|---|---|---|---|---|
| Positive pair cosine | 768 | 0.853 | > 0.800 | Paraphrase and parallel alignment |
| Matryoshka compressed slice | 64 | 0.897 | > 0.800 | 12× compressed vector search |
| Cross-script transfer | 768 | 0.708 | > 0.650 | Arabizi ↔ Arabic script alignment |
| Negative pair cosine | 768 | 0.001 | < 0.050 | Unrelated pair orthogonality |
| Contrastive margin | 768 | +0.852 | > +0.750 | Positive–negative separation |
| Embedding isotropy (mean cosine) | 768 | 0.013 | < 0.100 | Isotropic space (no collapse) |
Training & Generalization Trajectory
| Checkpoint | Step | Train Loss | Validation Loss | Mean Cosine (Isotropy) |
|---|---|---|---|---|
| Initial | 50 | 14.9100 | — | — |
| Epoch 1 | 966 | 0.0639 | 2.1770 | 0.0261 |
| Epoch 2 | 1,932 | 0.0226 | 2.0640 | 0.0168 |
| Epoch 3 (Final) | 2,898 | 0.0125 | 2.0330 | 0.0160 |
Note on anchor suites: Collective suites such as DziriEval and MADAR evaluate multiple-choice QA and dialect classification rather than sentence retrieval / STS; standardized public Algerian Darija retrieval benchmarks remain an open community gap.
Intended Use & Scope
Recommended uses:
- Dense retrieval, semantic search, and Retrieval-Augmented Generation (RAG) over Algerian Darija content.
- Cross-script matching between Arabizi queries and Arabic-script documents without transliteration.
- Semantic deduplication, FAQ matching, and customer-support routing.
Out-of-scope uses:
- Text generation or machine translation.
- Long-document encoding beyond 128 tokens.
- Languages or dialects outside Algerian Darija, or automated decisions about individuals.
Training pairs include informal social-media vernacular. The model has not been evaluated for bias, toxicity, or factuality.
Architecture
| Component | Specification |
|---|---|
| Parameters | 278,075,904 (209,635,584 backbone + 68,440,320 embeddings) |
| Backbone | 12 layers, hidden size 768, 12 attention heads, GELU FFN (dim 3,072), initialized from XLM-RoBERTa base |
| Tokenizer | 250,044 BPE tokens (expanded by 42 tokens over 250,002) |
| Pooling & Slices | Mean pooling; Matryoshka dimensions 768, 512, 256, 128, 64 |
| Context Length | 128 tokens (learned absolute positions) |
| Objective | MatryoshkaLoss wrapping MultipleNegativesRankingLoss |
| Release Precision | float32 (trained with fp16 mixed precision) |
Dialectal character expansion: 42 tokens were added to the BPE vocabulary to provide single-token entries for Algerian Darija characters missing from standard multilingual vocabularies, including ڨ (U+06A8, voiced velar stop /g/, occurring 1,112 times in training pairs), ڢ (U+06A2), and ڋ (U+068B), preventing <unk> fallback.
Training Data
Trained on 123,602 deduplicated, decontaminated Algerian Darija sentence pairs across 62,301 semantic clusters (with 1,000 held-out validation pairs). Paraphrases and translation variants are grouped into disjoint semantic clusters so mini-batches sample strictly across distinct clusters, preventing false negatives. Text normalization applies Unicode NFC and collapses redundant whitespace while preserving Arabizi phoneme digits (3, 7, 9), tatweel (ـ), diacritics, and French accented characters.
| Source | Domain / Type | License Tier | Train Pairs | Share |
|---|---|---|---|---|
| Kamel Touati et al. | Parallel English–Darija corpus | Permissive (CC-BY-4.0) | 113,804 | 92.07% |
| PADIC (Algerian component) | Dialectal Arabic parallel corpus | Academic research use | 9,798 | 7.93% |
| Total | Tri-script Algerian Darija pairs | Mixed | 123,602 | 100.00% |
Training Data License Composition
The model weights are released under Apache-2.0. That grant applies to weights and code only and does not relicense the underlying training text.
| Tier | Pairs | Share |
|---|---|---|
| Permissive (CC-BY-4.0) | 96,830 | 78.34% |
| Academic research use | 26,772 | 21.66% |
Training Details
| Parameter | Value |
|---|---|
| Objective | MatryoshkaLoss (768, 512, 256, 128, 64) + MultipleNegativesRankingLoss |
| Optimizer | AdamW (lr=2e-5, weight_decay=0.01, betas=(0.9, 0.999), eps=1e-8) |
| Batch Size | 64 pairs/step (128 sequences/step) |
| Schedule | Linear warmup (289 steps, 10%) + cosine decay to 0 |
| Steps / Epochs | 2,898 steps (3 epochs, 370,806 pair exposures) |
| Regularization | Dropout 0.10, gradient clipping 1.0 |
| Precision | fp16 mixed precision |
| Hardware & Duration | 2× NVIDIA Tesla T4 (16 GB), 2.78 h (9,996.8 s) |
| Seed | 42 |
Limitations
- 128-token context cap. Optimized for sentences, search queries, comments, and short messages rather than long documents.
- Colloquial Algerian domain focus. Trained specifically on Algerian Darija; formal MSA and non-Maghrebi Arabic varieties are better served by general Arabic encoders.
- Single training run. All reported metrics come from one seed-42 run; cross-seed variance is unmeasured.
- No safety evaluation. Bias, toxicity, and factuality have not been evaluated.
Repository Contents
| File | Size | Description / SHA-256 |
|---|---|---|
model.safetensors |
1.11 GB | Float32 weights (199 tensors) · SHA-256 0d718427ea4437db4477f0e0dc77c3ba9bd6fd40c86018e86e26b6bf1c94f129 |
config.json |
742 B | Model architecture configuration |
config_sentence_transformers.json |
284 B | SentenceTransformers metadata and similarity config |
modules.json |
229 B | Pipeline definition (Transformer + Pooling) |
1_Pooling/config.json |
90 B | Mean-pooling configuration |
tokenizer.json |
17.1 MB | Fast BPE tokenizer (250,044 tokens) · SHA-256 d45fd5f5313b236c9edabcdd81258f6ffb9f788bf4639f2afd6bc4672294785c |
sentencepiece.bpe.model |
5.07 MB | SentencePiece BPE model · SHA-256 cfc8146abe2a0488e9e2a0c56de7952f7c11ab059eca145a0a727afce0db2865 |
tokenizer_config.json |
382 B | Tokenizer settings |
special_tokens_map.json |
166 B | Special token definitions |
export_report.json |
1.7 KB | Verified file sizes, SHA-256 checksums, and parameter count |
Optimizer states and intermediate checkpoints are excluded. The published weights reload via SentenceTransformer("algerian-nlp/BENKHLOUF") with zero missing or unexpected keys.
The name
Sidi Lakhdar Ben Khlouf (c. 1479–c. 1585), from the Dahra region near Mostaganem, is celebrated as a foundational master of Algerian Malhun poetry. Writing in vernacular Algerian Arabic rather than classical court verse, his poetry preserved both popular memory and everyday speech across the 16th century. Naming a cross-script Darija embedding model after him honors that vernacular tradition: capturing shared meaning across written scripts.
The naming is homage and implies no endorsement.
Citation
@misc{algerian_nlp_benkhlouf_2026,
title = {BENKHLOUF: Matryoshka Sentence Embeddings for Algerian Darija},
author = {Ainouche, Abderahmane and Algerian NLP Collective},
year = {2026},
url = {https://huggingface.co/algerian-nlp/BENKHLOUF}
}
Upstream backbone:
@inproceedings{conneau-etal-2020-unsupervised,
title = {Unsupervised Cross-lingual Representation Learning at Scale},
author = {Conneau, Alexis and Khandelwal, Kartikay and Goyal, Naman and Chaudhary, Vishrav and Wenzek, Guillaume and Guzm{\'a}n, Francisco and Grave, Edouard and Ott, Myle and Zettlemoyer, Luke and Stoyanov, Veselin},
booktitle = {Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics},
year = {2020},
url = {https://aclanthology.org/2020.acl-main.747}
}
License
Weights and code are released under Apache-2.0. Refer to the Training Data License Composition section above regarding the underlying training text before redistributing derivatives.
- Downloads last month
- 165
Collection including algerian-nlp/BENKHLOUF
Evaluation results
- Positive Pair Cosine Similarity (768d, 3 epochs) on Algerian Darija STS Validation (1,000 pairs)self-reported0.853
- Matryoshka 64d Cosine Similarity (64d, 3 epochs) on Algerian Darija STS Validation (1,000 pairs)self-reported0.897
- Cross-Script Transfer Arabizi-to-Arabic (768d, 3 epochs) on Algerian Darija STS Validation (1,000 pairs)self-reported0.708
- Negative Pair Orthogonality (768d, 3 epochs) on Algerian Darija STS Validation (1,000 pairs)self-reported0.001