Arabic Sparse Embedding
Collection
3 items โข Updated
Arabic SPLADE โ Phase 3
Efficient symmetric SPLADE using DistilBERT multilingual for faster inference.
Architecture
Symmetric shared (MLMTransformer+SpladePooling, sequential)
Base model: distilbert-base-multilingual-cased
Training
- Dataset:
oddadmix/arabic-triplets-large(104K triplets, 92K unique passages) - Loss:
SpladeLoss(SparseMultipleNegativesRankingLoss, q_reg=5e-5, d_reg=3e-5) - Batch: 16 per GPU, grad accum 4
- Learning rate: 2e-5
- Epochs: 1
- AMP: fp16
- Sampler: NO_DUPLICATES
Evaluation on Arabic NanoBEIR (13 datasets)
| Metric | Score |
|---|---|
| NDCG@10 | 0.2528 |
| MRR@10 | 0.3052 |
For reference: BM25 scores 0.3824 NDCG@10, 0.4483 MRR@10 on the same benchmark.
Training Details
DistilBERT multilingual (6-layer, 119K vocab), ~2x faster than AraBERT
Hardware
- 2ร NVIDIA TITAN RTX (23.5 GB each)
- DDP via
torchrun
Usage
from sentence_transformers.sparse_encoder import SparseEncoder
model = SparseEncoder("Abdelkareem/arabic-splade-efficient")
embeddings = model.encode([
"ู
ุง ูู ุนุงุตู
ุฉ ู
ุตุฑุ",
"ุงููุงูุฑุฉ ูู ุนุงุตู
ุฉ ู
ุตุฑ ูุฃูุจุฑ ู
ุฏููุง.",
])
print(embeddings.shape)
# Decode top tokens
decoded = model.decode(embeddings, top_k=10)
for d in decoded:
print(d)
- Downloads last month
- 6
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐ Ask for provider support
from sentence_transformers import SparseEncoder model = SparseEncoder("Waqf-AI/arabic-splade-efficient") queries = ["Which planet is known as the Red Planet?"] documents = [ "Venus is often called Earth's twin because of its similar size and proximity.", "Mars, known for its reddish appearance, is often referred to as the Red Planet.", "Jupiter, the largest planet in our solar system, has a prominent red spot.", ] query_embeddings = model.encode_query(queries) document_embeddings = model.encode_document(documents) similarities = model.similarity(query_embeddings, document_embeddings) print(similarities)