nami: English ⇄ Luganda translation (pretrained base)

nami is a 89M-parameter encoder-decoder Transformer that translates English β†’ Luganda and Luganda β†’ English with one model. The direction is set by a prefix token on the source (<2lug> / <2eng>).

This is a pretrained base, not a polished translator. It was trained only on machine-translated pairs (the Luganda side was generated by an NLLB-based model, see Training) and has not been fine-tuned on human-translated data. Expect fluent output with occasional dropped or changed meaning.

Quick start

pip install torch sentencepiece safetensors huggingface_hub
import sys
from huggingface_hub import snapshot_download

folder = snapshot_download("amDANIEL2024/nami")      # add token="hf_..." if the repo is still private
sys.path.insert(0, folder)
from nami import Translator

t = Translator(folder)                                # uses the GPU if there is one
print(t.translate("I want to go to town.", "eng2lug"))
print(t.translate(["Njagala kugenda mu kibuga."], "lug2eng"))

Directions are "eng2lug" and "lug2eng". translate takes a string or a list and accepts beam (default 4). This is a custom PyTorch model, not a transformers model, so AutoModel will not load it.

Example outputs (unedited, beam 4)

Source Output
I want to go to town. Njagala kugenda mu kibuga.
I have high blood pressure. Nnina puleesa.
What they did was very awful. Kye baakola kyali kibi nnyo.
Enkambi y'abanoonyiboobubudamu teyaliiwo. The refugee camp was not available.
Nina puleesa eya waggulu. I have high blood pressure.
Njagala kugenda mu kibuga ku wiikendi. I want to go to the city on weekends.

Note the second row: "high" was dropped. Check outputs before relying on them.

Model

Architecture pre-LN Transformer encoder-decoder, 6 + 6 layers, d_model 640, 8 heads, FFN 2560, GELU
Parameters 89M (input embedding, decoder embedding and output projection are tied)
Tokenizer shared SentencePiece unigram, 32,000 pieces, case preserved, digits split (spm.model)
Context 256 positions (sinusoidal); training pairs were limited to 128 pieces per side
Weights model.safetensors (float32)

Training

  • Data: the English-Luganda pairs of tulati, about 1.5M pairs after filtering (rows flagged empty / degenerate / looks-like-English / length-outlier, and duplicates, removed), used in both directions. The English side is the full pool of agentlans/high-quality-english-sentences; the Luganda side is machine-translated from it with Sunbird/translate-nllb-1.3b-salt. No human-translated Luganda was used to train the weights. Pairs overlapping the SALT dev/test sets were removed (none were found).
  • Tokenizer: trained on tulati plus the text of the SALT train split. The model weights themselves were trained on tulati only.
  • Schedule: 5 epochs, 19,825 optimizer steps (about 25k target tokens per step), fp16 on one Kaggle T4, label smoothing 0.1, AdamW, linear warmup then inverse-sqrt decay (peak LR 3e-4).

Evaluation

Scores on the complete human-translated SALT dev and test sets (beam 4, one reference per sentence, sacreBLEU). The model weights were not trained on SALT.

Split Direction Sentences BLEU chrF++
dev English β†’ Luganda 500 27.5 56.8
dev Luganda β†’ English 500 23.5 47.2
test English β†’ Luganda 500 24.8 55.0
test Luganda β†’ English 500 26.1 47.9

Comparison with the model that generated the training data

The Luganda training data was produced by Sunbird/translate-nllb-1.3b-salt, a 1.3B-parameter model fine-tuned on SALT. On English β†’ Luganda, nami (about 89M parameters, about 15Γ— smaller, trained only on that model's output) comes close to it:

English β†’ Luganda BLEU chrF
Sunbird/translate-nllb-1.3b-salt 30.4 63.2
nami (SALT test) 24.8 59.0
nami as a share of the teacher about 82% about 93%

Limitations

  • The model was trained only on machine-translated Luganda, so it inherits the translation model's errors and style. For Luganda β†’ English it learned from synthetic Luganda, so real human-written Luganda (dialect, spelling, code-switching) may be handled worse than the numbers on the English side suggest.
  • The English side is general web text from a "high-quality sentences" pool; other domains may be weaker.
  • Meaning can be dropped or changed, especially for numbers, names and rare words. Do not use it for medical, legal or other high-stakes content without human review.
  • Sentence-level model: inputs longer than about 128 pieces are truncated or degrade.

Files

model.safetensors weights config.json architecture and training summary spm.model tokenizer nami.py standalone inference code README.md this card

Downloads last month
16
Safetensors
Model size
89.4M params
Tensor type
F32
Β·
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Datasets used to train amDANIEL2024/nami_base