|
Download README.md from dayomtechnologies/shilluk_mt: direct link, hf CLI and curl.
- Browser
- Download file 1.44 kB
-
https://huggingface.co/dayomtechnologies/shilluk_mt/resolve/main/README.md
- Command line
-
hf download hf://dayomtechnologies/shilluk_mt/README.md
-
curl -L -o README.md https://huggingface.co/dayomtechnologies/shilluk_mt/resolve/main/README.md
1.44 kB
| tags: | |
| - translation | |
| - machine-translation | |
| - shilluk | |
| - low-resource | |
| # Shilluk ↔ English translation (trained from scratch) | |
| Two encoder-decoder Transformers trained from scratch (no pretrained weights) on a Shilluk–English Bible parallel corpus. | |
| | Direction | File | Test-set scores | | |
| |---|---|---| | |
| | Shilluk → English | `shilluk2en.pt` | BLEU 14.4 / chrF 34.0 / chrF++ 33.4 (test split, 1507 sentences, beam 5) | | |
| | English → Shilluk | `en2shilluk.pt` | BLEU 11.0 / chrF 32.2 / chrF++ 30.8 (test split, 1507 sentences, beam 5) | | |
| * Shared SentencePiece tokenizer: `spm_shared.model` | |
| * Architecture/decoding code: `src/model.py`, `src/decode.py`, `tokenizer/tokenizer_utils.py` | |
| ## Usage | |
| ```python | |
| import torch | |
| from src.model import load_model_from_checkpoint | |
| from src.decode import translate | |
| from tokenizer.tokenizer_utils import SPTokenizer | |
| tok = SPTokenizer("spm_shared.model") | |
| model, meta = load_model_from_checkpoint("en2shilluk.pt", "cpu") | |
| print(translate(model, tok, "In the beginning God created the heaven and the earth.", "cpu", beam_size=5)) | |
| ``` | |
| (Put `spm_shared.model` at `tokenizer/spm_shared.model` or pass its path as above.) | |
| ## Limitations | |
| Trained on Bible text only: expect good results on Bible-style sentences and unreliable output on everyday or modern vocabulary. Scores are on held-out chapters of the same corpus, so they overestimate quality on other domains. | |