🧠 Hakim: Farsi Text Embedding Model

Model Availability

Status: Proprietary / Closed-weight

Hakim is a proprietary Persian text embedding model developed at MCINext and has been used in real-world applications.

The model weights are not publicly available. An experimental inference API was previously offered but has since been discontinued.

This repository is maintained for research documentation, including the model's training methodology and evaluation results. For further details, please refer to our technical report.

Overview

Hakim is a Persian text embedding model designed for applications such as semantic search, dense retrieval, retrieval-augmented generation (RAG), and instruction-based NLP tasks, including classification and question answering.

In the FaMTEB evaluation reported in May 2025, Hakim achieved the highest average score among the models compared (73.81), approximately 8.5 points higher than the next-highest score (65.29). These results describe the evaluation at the time of the report, not a claim about current rankings.

📌 Model Highlights

• 🔍 FaMTEB results: Ranked first by average score among the evaluated models as of May 31, 2025, across 63 Persian NLP datasets.
• 🧾 Instruction-tuned: Trained for tasks including classification, semantic textual similarity (STS), retrieval, question answering, and cross-task settings.
• 🗣️ Chat-history-aware training: Fine-tuned with data incorporating conversational context.
• ⚙️ Compact architecture: Approximately 124 million parameters.

🏗️ Training Datasets

📚 Pretraining

• Corpesia: 11B tokens from 46 Persian websites across 21 domains (e.g., news, health, religion, and technology).
• hmBlogs: 6.8B tokens from approximately 20M Persian blog posts.
• Queries: 8.5M anonymized search queries.

🔄 Unsupervised Stage (Pairsia-unsup)

• 5M high-quality Persian text pairs, including:
  • Document–title, FAQ, question–answer, and paper title–abstract pairs.
  • Machine-translated datasets (MS MARCO, SAMSum, AdversarialQA, etc.).

🧠 Supervised Stage (Pairsia-sup)

• 1.3M labeled pairs with 9 negatives per query.
• Instruction-based fine-tuning across classification, retrieval, STS, QA, and NLI.

For methodology and additional training details, see the technical report.

🧪 Benchmark Results (FaMTEB, May 2025)

The following scores are from the evaluation reported at the time of the technical report. They should not be interpreted as a continuously updated leaderboard.

Model	Avg. Score	Classification	Clustering	PairClass.	Reranking	Retrieval	STS	Summarization
Hakim	73.81	84.56	70.46	89.75	69.46	40.43	76.62	85.41
Hakim-small	70.45	80.19	66.31	87.41	67.30	38.05	75.53	78.40
Hakim-unsup	64.56	60.65	58.89	86.41	67.56	37.71	79.36	61.34
BGE-m3	65.29	58.75	57.73	85.21	74.56	43.38	76.35	61.07
Jina-embeddings-v3	64.53	59.93	59.15	83.71	61.26	43.51	78.65	65.50
multilingual-e5-large	64.40	59.86	57.19	84.42	74.34	42.98	75.38	56.61
GTE-multilingual-base	63.64	56.07	57.28	84.58	69.72	41.22	75.75	60.88
multilingual-e5-base	62.93	57.62	56.52	84.04	72.07	41.20	74.45	54.58
Tooka-SBERT	60.65	59.40	56.45	87.04	58.29	27.86	76.42	59.06

Task-Specific Input Formats

Hakim's documented input formats used task-specific Persian prefixes. The formats below are retained for technical reference only; they are not instructions for accessing an active API.

Task type	Intended use	Example of formatted Persian input
sentiment	Sentiment analysis	مسئله : دسته بندی , تحلیل احساس رضایت متن | متن : [متن شما]
classification	Topic or text classification	مسئله : دسته بندی , دسته بندی موضوعی متن | متن : [متن شما]
clustering	Text clustering	مسئله : دسته بندی , دسته بندی موضوعی متن | متن : [متن شما]
sts.sent1	STS, first sentence	مسئله : تشخیص ارتباط , آیا متن دوم شباهت معنایی با متن اول دارد ؟ | متن اول : [متن شما]
sts.sent2	STS, second sentence	مسئله : تشخیص ارتباط , آیا متن دوم شباهت معنایی با متن اول دارد ؟ | متن دوم : [متن شما]
retrieval.query	Retrieval query	مسئله : تشخیص ارتباط , آیا متن دوم به متن اول مرتبط است ؟ | متن اول : [متن شما]
retrieval.passage	Retrieval passage	مسئله : تشخیص ارتباط , آیا متن دوم به متن اول مرتبط است ؟ | متن دوم : [متن شما]
cross	Two-text semantic relationship	مسئله : دسته بندی با دو ورودی , نوع ارتباط معنایی متن دوم با متن اول چگونه است ؟ | متن اول : [متن ۱] | متن دوم : [متن ۲]

For STS, the first and second texts were formatted with sts.sent1 and sts.sent2, respectively. For retrieval, queries and documents were formatted with retrieval.query and retrieval.passage. The cross format combined two texts into a single task-specific input. These descriptions document the original input conventions, not a publicly available inference service.

Citation

If you use or refer to Hakim in your research, please cite the technical report:

@article{sarmadi2025hakim,
  title={Hakim: Farsi Text Embedding Model},
  author={Sarmadi, Mehran and Alikhani, Morteza and Zinvandi, Erfan and Pourbahman, Zahra},
  journal={arXiv preprint arXiv:2505.08435},
  year={2025}
}