🧠 Hakim-small: Compact Persian Text Embedding Model

arXiv

Model Availability

Status: Proprietary / Closed-weight

Hakim-small is a proprietary Persian text embedding model developed at MCINext as part of the Hakim project.

The model weights are not publicly available. An experimental inference API was previously offered but has since been discontinued.

This repository is maintained for research documentation, including the model's training methodology and evaluation results. For further details, please refer to our technical report.

Overview

Hakim-small is a compact variant of Hakim designed to balance embedding quality and model size. With approximately 38 million parameters, compared with approximately 124 million in the main Hakim model, it was developed for Persian-language text representation tasks where computational resources are limited.

Its intended applications include semantic search, dense retrieval, retrieval-augmented generation (RAG), semantic textual similarity (STS), classification, and question answering. These are the model's research use cases, not currently offered public inference services.

In the FaMTEB evaluation reported in May 2025, Hakim-small achieved an average score of 70.45. This result reflects the comparison reported at that time, not a claim about current leaderboard rankings.

πŸ“Œ Model Highlights

  • πŸ” FaMTEB results: Average score of 70.45 in the reported evaluation across Persian NLP tasks.
  • 🧾 Instruction-tuned: Trained using task-oriented data for classification, STS, retrieval, question answering, and related tasks.
  • πŸ—£οΈ Chat-history-aware training: The Hakim-small model card reports training that incorporates conversational context.
  • βš™οΈ Compact architecture: Approximately 38M parameters, versus approximately 124M for the main Hakim model.

πŸ—οΈ Training Datasets

Hakim-small draws on the corpora and training approach developed for the Hakim project.

πŸ“š Pretraining

  • Corpesia: 11B tokens from 46 Persian websites spanning 21 domains (including news, health, religion, and technology).
  • hmBlogs: 6.8B tokens from approximately 20M Persian blog posts.
  • Queries: 8.5M anonymized search queries.

πŸ”„ Unsupervised Stage (Pairsia-unsup)

  • 5M Persian text pairs from sources such as document–title, FAQ, question–answer, and paper title–abstract pairs.
  • The project also used machine-translated datasets, including MS MARCO, SAMSum, and AdversarialQA.

🧠 Supervised Stage (Pairsia-sup)

  • 1.3M labeled text pairs with multiple negatives per query.
  • Instruction-based fine-tuning for tasks including classification, retrieval, STS, question answering, and natural language inference (NLI).

For additional details on the training approach and data preparation, see the technical report.

πŸ§ͺ Benchmark Results (FaMTEB, May 2025)

The following scores are from the evaluation reported in the Hakim technical report. They are not a continuously updated leaderboard.

Model Avg. Score Classification Clustering PairClass. Reranking Retrieval STS Summarization
Hakim 73.81 84.56 70.46 89.75 69.46 40.43 76.62 85.41
Hakim-small 70.45 80.19 66.31 87.41 67.30 38.05 75.53 78.40
Hakim-unsup 64.56 60.65 58.89 86.41 67.56 37.71 79.36 61.34
BGE-m3 65.29 58.75 57.73 85.21 74.56 43.38 76.35 61.07
Jina-embeddings-v3 64.53 59.93 59.15 83.71 61.26 43.51 78.65 65.50
multilingual-e5-large 64.40 59.86 57.19 84.42 74.34 42.98 75.38 56.61
GTE-multilingual-base 63.64 56.07 57.28 84.58 69.72 41.22 75.75 60.88
multilingual-e5-base 62.93 57.62 56.52 84.04 72.07 41.20 74.45 54.58
Tooka-SBERT 60.65 59.40 56.45 87.04 58.29 27.86 76.42 59.06

Related Models

  • Hakim β€” the main, approximately 124M-parameter model after supervised fine-tuning.
  • Hakim-unsup β€” the approximately 124M-parameter model from the unsupervised stage, before subsequent supervised fine-tuning.

Citation

If you refer to this model or the Hakim project in research, please cite the technical report:

@article{sarmadi2025hakim,
  title={Hakim: Farsi Text Embedding Model},
  author={Sarmadi, Mehran and Alikhani, Morteza and Zinvandi, Erfan and Pourbahman, Zahra},
  journal={arXiv preprint arXiv:2505.08435},
  year={2025}
}
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Paper for MCINext/Hakim-small