Hugging Face's logo

TabuLM-Research
/
KinyaEmbed

Safetensors
Model card Files
xet
 Community
KinyaEmbed
File size: 5,692 Bytes
ce1b1c3
 
 
1facfa7
ce1b1c3
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1facfa7
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
<details>
<summary>KinyaEmbed model card content (click to expand)</summary>


# KinyaEmbed: Contrastive Sentence Embeddings for Kinyarwanda

**Ireddi Rakshitha** (Software Engineer, Barclays) · **Devavarapu Yashwanth** (Software Engineer, Barclays) · **Pierre Ntakirutimana** (Research Associate, Carnegie Mellon University)

> **arXiv preprint · August 2026** · [Paper](https://arxiv.org/abs/XXXX.XXXXX) · [Code](https://github.com/TabuLM-Research/KinyaEmbed)

---

## Model Description

**KinyaEmbed** is the first dedicated sentence embedding model for Kinyarwanda (ISO 639-1: `rw`), a Bantu language spoken by 12 million people in Rwanda. Built on KinyaBERT-large and trained via a four-stage curriculum with MultipleNegativesRankingLoss (MNRL).

### Training Stages

| Stage | Data | Scale (1/τ) | Checkpoints |
|---|---|---|---|
| 1 — Gazette Paraphrases | ~18,000 Official Gazette pairs | 30 / 35 / 40 | `sc30`, `sc35`, `sc40` |
| 2 — MNLI Triplets | 715 translated NLI triplets | 35 | `v12` |
| 3 — OPUS-100 Alignment | English–Kinyarwanda pairs | 20 | `step22A` |
| 4 — KinyaCOMET | 2,936 human-quality pairs (≥0.8) | 20 | `step23A` |

**Ensemble:** `all5+23A×2` — 7 checkpoints averaged, `step23A` double-weighted.

---

## Performance

| Model | SemRel2024-rw ρ | Wiki-RW-STS ρ | Clustering Silhouette |
|---|---|---|---|
| LaBSE | 0.4535 | 0.2197 | 0.1882 |
| mE5-large | 0.6039 | 0.5337 | 0.0794 |
| AfriE5-instruct | 0.6037 | 0.5391 | 0.1104 |
| OpenAI text-emb-3-large | 0.5175 | 0.5319 | 0.0846 |
| **KinyaEmbed (ours)** | **0.7298** | **0.6005** | **0.2146** |

**+20.9%** over mE5-large on SemRel2024-rw · **+8.6%** on Wiki-RW-STS · Best document clustering across all 7 models.

---

## Usage

```python
from sentence_transformers import SentenceTransformer
import numpy as np

# Ensemble (recommended)
checkpoints = [
    "TabuLM-Research/KinyaEmbed/sc30",
    "TabuLM-Research/KinyaEmbed/sc35",
    "TabuLM-Research/KinyaEmbed/sc40",
    "TabuLM-Research/KinyaEmbed/v12",
    "TabuLM-Research/KinyaEmbed/step22A",
    "TabuLM-Research/KinyaEmbed/step23A",
    "TabuLM-Research/KinyaEmbed/step23A",  # double-weighted
]
models = [SentenceTransformer(c) for c in checkpoints]

def ensemble_encode(sentences):
    embs = [m.encode(sentences, normalize_embeddings=True) for m in models]
    avg = np.mean(embs, axis=0)
    return avg / np.linalg.norm(avg, axis=1, keepdims=True)

# Single checkpoint (lower latency)
model = SentenceTransformer("TabuLM-Research/KinyaEmbed/sc35")
embeddings = model.encode(["Kinyarwanda text here"], normalize_embeddings=True)

---

Included Files

┌───────────────────────────┬─────────────────────────────────────────────────────┐
│           File            │                     Description                     │
├───────────────────────────┼─────────────────────────────────────────────────────┤
│ sc30/, sc35/, sc40/       │ Stage 1 checkpoints (gazette, 3 temperature scales) │
├───────────────────────────┼─────────────────────────────────────────────────────┤
│ v12/                      │ Stage 2 checkpoint (MNLI triplets)                  │
├───────────────────────────┼─────────────────────────────────────────────────────┤
│ step22A/                  │ Stage 3 checkpoint (OPUS-100)                       │
├───────────────────────────┼─────────────────────────────────────────────────────┤
│ step23A/                  │ Stage 4 checkpoint (KinyaCOMET)                     │
├───────────────────────────┼─────────────────────────────────────────────────────┤
│ kinyacomet_filtered.jsonl │ 2,936 high-quality Kinyarwanda–English pairs        │
├───────────────────────────┼─────────────────────────────────────────────────────┤
│ wiki_rw_sts.jsonl         │ Wiki-RW-STS benchmark (300 pairs, CC-BY-SA 4.0)     │
└───────────────────────────┴─────────────────────────────────────────────────────┘

---

Citation

bibtex
@article{ireddi2026kinyaembed,
  title   = {{KinyaEmbed}: Contrastive Sentence Embeddings for {Kinyarwanda}
             via Multi-Stage Curriculum Training},
  author  = {Ireddi, Rakshitha and Devavarapu, Yashwanth and Ntakirutimana, Pierre},
  journal = {arXiv preprint arXiv:XXXX.XXXXX},
  year    = {2026}
}

License

Code: MIT · KinyaCOMET filtered pairs: CC-BY 4.0 · Wiki-RW-STS: CC-BY-SA 4.0

</details>