| <details> |
| <summary>KinyaEmbed model card content (click to expand)</summary> |
|
|
|
|
| # KinyaEmbed: Contrastive Sentence Embeddings for Kinyarwanda |
|
|
| **Ireddi Rakshitha** (Software Engineer, Barclays) · **Devavarapu Yashwanth** (Software Engineer, Barclays) · **Pierre Ntakirutimana** (Research Associate, Carnegie Mellon University) |
|
|
| > **arXiv preprint · August 2026** · [Paper](https://arxiv.org/abs/XXXX.XXXXX) · [Code](https://github.com/TabuLM-Research/KinyaEmbed) |
|
|
| --- |
|
|
| ## Model Description |
|
|
| **KinyaEmbed** is the first dedicated sentence embedding model for Kinyarwanda (ISO 639-1: `rw`), a Bantu language spoken by 12 million people in Rwanda. Built on KinyaBERT-large and trained via a four-stage curriculum with MultipleNegativesRankingLoss (MNRL). |
|
|
| ### Training Stages |
|
|
| | Stage | Data | Scale (1/τ) | Checkpoints | |
| |---|---|---|---| |
| | 1 — Gazette Paraphrases | ~18,000 Official Gazette pairs | 30 / 35 / 40 | `sc30`, `sc35`, `sc40` | |
| | 2 — MNLI Triplets | 715 translated NLI triplets | 35 | `v12` | |
| | 3 — OPUS-100 Alignment | English–Kinyarwanda pairs | 20 | `step22A` | |
| | 4 — KinyaCOMET | 2,936 human-quality pairs (≥0.8) | 20 | `step23A` | |
|
|
| **Ensemble:** `all5+23A×2` — 7 checkpoints averaged, `step23A` double-weighted. |
|
|
| --- |
|
|
| ## Performance |
|
|
| | Model | SemRel2024-rw ρ | Wiki-RW-STS ρ | Clustering Silhouette | |
| |---|---|---|---| |
| | LaBSE | 0.4535 | 0.2197 | 0.1882 | |
| | mE5-large | 0.6039 | 0.5337 | 0.0794 | |
| | AfriE5-instruct | 0.6037 | 0.5391 | 0.1104 | |
| | OpenAI text-emb-3-large | 0.5175 | 0.5319 | 0.0846 | |
| | **KinyaEmbed (ours)** | **0.7298** | **0.6005** | **0.2146** | |
|
|
| **+20.9%** over mE5-large on SemRel2024-rw · **+8.6%** on Wiki-RW-STS · Best document clustering across all 7 models. |
|
|
| --- |
|
|
| ## Usage |
|
|
| ```python |
| from sentence_transformers import SentenceTransformer |
| import numpy as np |
| |
| # Ensemble (recommended) |
| checkpoints = [ |
| "TabuLM-Research/KinyaEmbed/sc30", |
| "TabuLM-Research/KinyaEmbed/sc35", |
| "TabuLM-Research/KinyaEmbed/sc40", |
| "TabuLM-Research/KinyaEmbed/v12", |
| "TabuLM-Research/KinyaEmbed/step22A", |
| "TabuLM-Research/KinyaEmbed/step23A", |
| "TabuLM-Research/KinyaEmbed/step23A", # double-weighted |
| ] |
| models = [SentenceTransformer(c) for c in checkpoints] |
| |
| def ensemble_encode(sentences): |
| embs = [m.encode(sentences, normalize_embeddings=True) for m in models] |
| avg = np.mean(embs, axis=0) |
| return avg / np.linalg.norm(avg, axis=1, keepdims=True) |
| |
| # Single checkpoint (lower latency) |
| model = SentenceTransformer("TabuLM-Research/KinyaEmbed/sc35") |
| embeddings = model.encode(["Kinyarwanda text here"], normalize_embeddings=True) |
| |
| --- |
| |
| Included Files |
| |
| ┌───────────────────────────┬─────────────────────────────────────────────────────┐ |
| │ File │ Description │ |
| ├───────────────────────────┼─────────────────────────────────────────────────────┤ |
| │ sc30/, sc35/, sc40/ │ Stage 1 checkpoints (gazette, 3 temperature scales) │ |
| ├───────────────────────────┼─────────────────────────────────────────────────────┤ |
| │ v12/ │ Stage 2 checkpoint (MNLI triplets) │ |
| ├───────────────────────────┼─────────────────────────────────────────────────────┤ |
| │ step22A/ │ Stage 3 checkpoint (OPUS-100) │ |
| ├───────────────────────────┼─────────────────────────────────────────────────────┤ |
| │ step23A/ │ Stage 4 checkpoint (KinyaCOMET) │ |
| ├───────────────────────────┼─────────────────────────────────────────────────────┤ |
| │ kinyacomet_filtered.jsonl │ 2,936 high-quality Kinyarwanda–English pairs │ |
| ├───────────────────────────┼─────────────────────────────────────────────────────┤ |
| │ wiki_rw_sts.jsonl │ Wiki-RW-STS benchmark (300 pairs, CC-BY-SA 4.0) │ |
| └───────────────────────────┴─────────────────────────────────────────────────────┘ |
| |
| --- |
| |
| Citation |
| |
| bibtex |
| @article{ireddi2026kinyaembed, |
| title = {{KinyaEmbed}: Contrastive Sentence Embeddings for {Kinyarwanda} |
| via Multi-Stage Curriculum Training}, |
| author = {Ireddi, Rakshitha and Devavarapu, Yashwanth and Ntakirutimana, Pierre}, |
| journal = {arXiv preprint arXiv:XXXX.XXXXX}, |
| year = {2026} |
| } |
| |
| License |
| |
| Code: MIT · KinyaCOMET filtered pairs: CC-BY 4.0 · Wiki-RW-STS: CC-BY-SA 4.0 |
| |
| </details> |
