kieran_base: direct Luganda ⇄ Runyankole translation (pretrained base)
kieran_base is a 89M-parameter encoder-decoder Transformer that translates Luganda → Runyankole and
Runyankole → Luganda directly, with one model and no English on either side. The direction is set by a prefix token on the
source (<2nyn> / <2lug>; the tag names the target language).
This is a pretrained base, not a finished translator. It was trained only on machine-translated pairs: both the Luganda side and the Runyankole side of every training pair are output of one NLLB-based model (see Training). It has not been fine-tuned on human-translated data. A fine-tuned version will be published separately as
kieran_finetuned. Full test-set scores have not been run yet (see Evaluation).
Quick start
pip install torch sentencepiece safetensors huggingface_hub
import sys
from huggingface_hub import snapshot_download
folder = snapshot_download("amDANIEL2024/kieran_base") # add token="hf_..." if the repo is still private
sys.path.insert(0, folder)
from kieran import Translator
t = Translator(folder) # uses the GPU if there is one
print(t.translate("Okuzimba bbugwe kwatandika mu Gatonnya 2000.", "lug2nyn"))
print(t.translate(["Okwombeka orugo kukatandika omu kwezi kw'okubanza 2000."], "nyn2lug"))
Directions are "lug2nyn" and "nyn2lug". translate takes a string or a list and accepts beam (default 4).
This is a custom PyTorch model, not a transformers model, so AutoModel will not load it.
(The two input sentences above are shortened from a sample row of the training data, which is itself machine output.)
Example outputs
Not yet added. Examples will be taken unedited from the SALT test set, mistakes included, once the evaluation has been run.
Model
| Architecture | pre-LN Transformer encoder-decoder, 6 + 6 layers, d_model 640, 8 heads, FFN 2560, GELU |
| Parameters | 89M (input embedding, decoder embedding and output projection are tied) |
| Tokenizer | shared SentencePiece unigram, 32,000 pieces, case preserved, digits split (spm.model); trained on Luganda and Runyankole text only |
| Context | 256 positions (sinusoidal); training pairs were limited to 128 pieces per side |
| Weights | model.safetensors (float32) |
Training
- Data: the Luganda–Runyankole pairs of tulati, used in both directions. In tulati every non-English column is a machine translation of one English column, made by Sunbird/translate-nllb-1.3b-salt; no human-written Luganda or Runyankole is in it. The English source column was not used for this model.
- Filtering (counts from this run): of 1,534,699 tulati rows, 58,115 (3.8%) were dropped because they were flagged empty, degenerate, looks-like-English or length-outlier in either language. Then 2,389 pairs with identical sides and 576 exact duplicates were removed, leaving 1,473,619 pairs. One more pair was removed because a sentence appears in the SALT dev or test sets, leaving 1,473,618; 2,000 of those were held out for validation. No further pair-level filter was used.
- Tokenizer: trained on tulati plus the text of the SALT train split (Luganda and Runyankole only). The model weights were trained on tulati only.
- Schedule: 5 epochs, 19,305 optimizer steps (about 25k target tokens per step), fp16 on one Kaggle T4, label smoothing 0.1, AdamW, linear warmup then inverse-sqrt decay (peak LR 3e-4).
- Source-data quality, for context. The tulati card reports the quality of the translator that made the data, English → each language, on SALT test (500 sentences, plain chrF): Luganda 63.17 (BLEU 30.40), Runyankole 54.17 (BLEU 15.56). No figure exists for Luganda ⇄ Runyankole. Those numbers describe the data generator, not this model.
Evaluation
Complete SALT dev/test scores (BLEU, chrF, chrF++): not run yet. SALT test has not been used for anything.
What has been measured so far, at the end of pretraining (step 19,305):
| Set | Direction | Loss | Quick chrF++ (greedy, first 128 sentences) |
|---|---|---|---|
| tulati validation (2,000 held-out machine-translated pairs) | lug→nyn | 1.280 | 59.37 |
| tulati validation | nyn→lug | 1.538 | 56.26 |
| SALT dev (human translations, 500 pairs) | lug→nyn | 3.073 | 44.67 |
| SALT dev | nyn→lug | 2.551 | 44.55 |
The quick chrF++ is a noisy check on a small subset with greedy decoding; it is not comparable to full-set scores. SALT dev was only observed here, not used to choose anything.
Copy baseline on the complete SALT dev set (output the source sentence unchanged), sacreBLEU 2.5.1:
| Direction | BLEU | chrF | chrF++ |
|---|---|---|---|
| lug→nyn | 0.71 | 27.08 | 22.87 |
| nyn→lug | 0.72 | 27.59 | 23.21 |
Note that SALT's Luganda and Runyankole sentences are independent human translations of the same English sentence, so a reference in one language is not a translation of the reference in the other.
Limitations
- Both sides of every training pair are machine output, so the model inherits that translator's errors and style. The translator was clearly weaker on Runyankole than on Luganda (see the figures above). Real human-written text (dialect, spelling, code-switching) may be handled worse than benchmark numbers suggest.
- Meaning can be dropped or changed, especially for numbers, names and rare words. Do not use it for medical, legal or other high-stakes content without human review.
- Sentence-level model: inputs longer than about 128 pieces are truncated or degrade.
- Outputs have not been reviewed by a Runyankole reader.
Files
model.safetensors weights · config.json architecture and training summary · spm.model tokenizer ·
kieran.py standalone inference code · README.md this card.
License
Released under the MIT License (as are tulati, the translation model that generated the training data, and SALT).
Author
Amanyire Daniel (amDANIEL2024) · domusdeiug@gmail.com · https://domusdeiug.com
- Downloads last month
- -