metadata
tags:
- translation
- machine-translation
- shilluk
- low-resource
Shilluk ↔ English translation (trained from scratch)
Two encoder-decoder Transformers trained from scratch (no pretrained weights) on a Shilluk–English Bible parallel corpus.
| Direction | File | Test-set scores |
|---|---|---|
| Shilluk → English | shilluk2en.pt |
BLEU 14.4 / chrF 34.0 / chrF++ 33.4 (test split, 1507 sentences, beam 5) |
| English → Shilluk | en2shilluk.pt |
BLEU 11.0 / chrF 32.2 / chrF++ 30.8 (test split, 1507 sentences, beam 5) |
- Shared SentencePiece tokenizer:
spm_shared.model - Architecture/decoding code:
src/model.py,src/decode.py,tokenizer/tokenizer_utils.py
Usage
import torch
from src.model import load_model_from_checkpoint
from src.decode import translate
from tokenizer.tokenizer_utils import SPTokenizer
tok = SPTokenizer("spm_shared.model")
model, meta = load_model_from_checkpoint("en2shilluk.pt", "cpu")
print(translate(model, tok, "In the beginning God created the heaven and the earth.", "cpu", beam_size=5))
(Put spm_shared.model at tokenizer/spm_shared.model or pass its path as above.)
Limitations
Trained on Bible text only: expect good results on Bible-style sentences and unreliable output on everyday or modern vocabulary. Scores are on held-out chapters of the same corpus, so they overestimate quality on other domains.