haryads/kurdish-sentence-embeddings

Kurdish Sorani Sentence Embedding Model

A Kurdish Sorani sentence embedding model for Central Kurdish (ckb). It maps Kurdish sentences to 512-dimensional vectors for semantic search, retrieval (RAG), clustering and sentence similarity.

Unlike the multilingual models adapted for Kurdish by vocabulary trimming, this model was trained on Kurdish text: a RoBERTa base trained from scratch on a 1-million-sentence Kurdish corpus with a custom 32k Kurdish BPE tokenizer, then fine-tuned with contrastive learning (MultipleNegativesRankingLoss) on ~22k Kurdish sentence pairs with hard negatives.

If you searched for a "Kurdish sentence embedding model", a "Sorani semantic search model", or a "ckb text embedding model" — this is one, and it is trained on Kurdish rather than repurposed from a multilingual model.

Usage

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("haryads/kurdish-sentence-embeddings")

sentences = [
    "کەشوهەوای ئەمڕۆ زۆر باراناوییە",
    "ئەمڕۆ ئاسمان دەگرێت و باران دەبارێت",
    "من حەزم لە خواردنی شۆربای نیسکە",
]
emb = model.encode(sentences)
print(model.similarity(emb, emb))

Evaluation

Measured on an internal Kurdish benchmark: a retrieval task (match a sentence to its lemmatized form among 1,000 candidates) and a topic-separation task (800 triplets). Compared against paraphrase-multilingual-MiniLM-L12-v2 on identical data.

Model	Retrieval R@1	MRR	Topic accuracy
This model	0.978	0.984	0.931
paraphrase-multilingual-MiniLM-L12-v2	0.960	0.970	0.786
RoBERTa base, mean-pooled (no fine-tuning)	0.930	0.944	0.635

The model outperforms the multilingual baseline on Kurdish for both retrieval and topic separation.

Details

• Output dimension: 512
• Max sequence length: 128 tokens
• Pooling: mean, with L2 normalization
• Base: from-scratch Kurdish RoBERTa + 32k Kurdish BPE tokenizer
• Fine-tuning: MultipleNegativesRankingLoss on paraphrase, lemma and topic pairs with hard negatives
• Language: Central Kurdish / Sorani (ckb), Arabic script

Intended use and limits

Best for Kurdish semantic search, retrieval (RAG), clustering and similarity. Sorani only for now. Trained on news-heavy text, so very domain-specific or colloquial input may need in-domain fine-tuning. For best results, normalize input (Arabic vs. Kurdish ی/ک, ZWNJ, digits) before encoding.

License

Apache-2.0.