Instructions to use Sunbird-experimental/nllb1.3b-67languages with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Sunbird-experimental/nllb1.3b-67languages with Transformers:
# Use a pipeline as a high-level helper # Warning: Pipeline type "translation" is no longer supported in transformers v5. # You must load the model directly (see below) or downgrade to v4.x with: # pip install "transformers<5.0.0" from transformers import pipeline pipe = pipeline("translation", model="Sunbird-experimental/nllb1.3b-67languages")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForSeq2SeqLM tokenizer = AutoTokenizer.from_pretrained("Sunbird-experimental/nllb1.3b-67languages") model = AutoModelForSeq2SeqLM.from_pretrained("Sunbird-experimental/nllb1.3b-67languages", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Sunflower Translate (NLLB-1.3B, salt-69)
Sunbird-experimental/nllb1.3b-67languages is
NLLB-200 1.3B fine-tuned by
Sunbird AI to translate between English and 74 other
languages. Most are spoken in Uganda; the rest are from elsewhere in Africa,
plus French. It translates in any direction between these languages, including
between two non-English languages.
It is a small, fast alternative to the Sunflower LLMs for translation only. On the salt-69 benchmark, it scores about 3 chrF points lower than Sunflower-Qwen3.5-27B on average (43.7 vs 46.5). It is level with or ahead of Qwen in about a third of the eng→xx directions, at a fraction of the size (1.3B vs 27B parameters). See Evaluation.
Additions to standard NLLB:
- 42 languages NLLB doesn't support, through repurposed language tokens (see Language codes).
- The source language is optional. The model was trained to translate with or without being told the source language. This is useful for code-switched text, or when the source language is unknown. Leaving it out costs only about 0.3 chrF on average.
Status: experimental. Quality varies a lot by language: from strong (Afrikaans, Zulu, Swahili, Luganda) to unusable (Ikposo, Lendu, Kalenjin). Read Limitations before using it for a particular language.
Quick start
pip install torch transformers sentencepiece sacremoses huggingface_hub
import json
import re
import torch
from huggingface_hub import hf_hub_download
from sacremoses import MosesPunctNormalizer
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
MODEL_ID = 'Sunbird-experimental/nllb1.3b-67languages'
device = 'cuda' if torch.cuda.is_available() else 'cpu'
tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
model = AutoModelForSeq2SeqLM.from_pretrained(
MODEL_ID, dtype=torch.bfloat16).to(device).eval()
# SALT language code (e.g. 'lug') -> NLLB language token (e.g. 'lug_Latn').
with open(hf_hub_download(MODEL_ID, 'nllb_language_codes.json')) as f:
LANG_CODES = json.load(f)
UNKNOWN_SOURCE = 'und' # Source language not specified
_punct = MosesPunctNormalizer(
pre_replace_unicode_punct=True, post_remove_control_chars=True,
norm_quote_commas=False, norm_numbers=False)
def normalize(text):
'''Applies the same normalization as in training: typographic quotes and
dashes -> ASCII, whitespace collapsed, control characters removed.'''
text = re.sub(r'\s+', ' ', text)
text = text.replace('̇', '').replace('̦', '')
return _punct.normalize(text)
@torch.no_grad()
def translate(texts, tgt_lang, src_lang=None, num_beams=4):
'''Translates a list of texts into tgt_lang (a SALT code such as 'lug').
src_lang is optional; leave it as None if unknown or code-switched.'''
src_token = LANG_CODES[src_lang or UNKNOWN_SOURCE]
inputs = tokenizer([normalize(t) for t in texts], return_tensors='pt',
padding=True, truncation=True, max_length=512).to(device)
# NLLB puts the source-language token first: set it explicitly.
inputs['input_ids'][:, 0] = tokenizer.convert_tokens_to_ids(src_token)
# Cap the output length to stop runaway repetition on weaker languages.
max_new_tokens = min(512, 2 * inputs['input_ids'].shape[1] + 10)
outputs = model.generate(
**inputs,
forced_bos_token_id=tokenizer.convert_tokens_to_ids(LANG_CODES[tgt_lang]),
num_beams=num_beams, max_new_tokens=max_new_tokens, max_length=None)
return tokenizer.batch_decode(outputs, skip_special_tokens=True)
print(translate(['How are you this morning?',
'Farmers should plant their crops before the rains begin.'],
tgt_lang='lug', src_lang='eng'))
# ['Oli otya enkya eno?',
# "Abalimi basaanidde okusimba ebirime byabwe ng'enkuba tennatandika."]
print(translate(['Abaana bagenda ku ssomero.'], tgt_lang='eng')) # no source language
# ['Children go to school.']
print(translate(['Abaana bagenda ku ssomero.'], tgt_lang='ach', src_lang='lug'))
# ['Lutino gicito i cukul.']
The model needs about 3 GB of GPU memory in bfloat16. It also runs on CPU, more slowly.
Why not tokenizer.src_lang / pipeline('translation')?
The standard NLLB recipe (tokenizer.src_lang = 'lug_Latn', or the
translation pipeline with src_lang/tgt_lang) works for languages with
a native NLLB code. The code above is more general:
- It uses the SALT codes from
nllb_language_codes.json. You don't need to know which NLLB token stands for each language. - It supports
src_lang=None(theundtoken), which the standard tokenizer doesn't know about. - It applies the input normalization the model was trained with.
Language codes
Use the SALT codes (mostly ISO 639-3) below for src_lang and tgt_lang.
nllb_language_codes.json in this repo maps each one to an NLLB language
token:
- Native codes: 33 languages (including English) use their own NLLB
token, e.g.
lug→lug_Latn,swa→swh_Latn,orm→gaz_Latn,mlg→plt_Latn. - Repurposed codes: 42 languages NLLB doesn't support reuse the token of
a European language that this model doesn't translate. For example, Acholi
ach→fur_Latn(Friulian), Runyankolenyn→lim_Latn, and Atesoteo→lmo_Latn. The NLLB language named in such a token is not a supported language. For example, asking forfur_Latngives Acholi, not Friulian. - Unknown source:
und→smo_Latn. Use it only on the source side, to translate without saying what the source language is. It is not a valid target.
The full mapping, with each language's scores, is in the
per-language results. The model has tokens for
five more languages that salt-69 doesn't cover: Thur (lth), Lulubo (lzm),
Kebu (ndp), Sebei (sbx) and So (soc). Their quality has not been
measured.
Inference tips
Normalize inputs. NLLB's vocabulary has no typographic quotes or dashes, so the model was trained on text normalized as in
normalize()above. Output uses ASCII punctuation ('"-); post-process it if you need typographic punctuation.Preserve line breaks. The tokenizer turns newlines into spaces. To keep the layout of multi-line text, split it into lines, translate the non-empty lines as one batch, and re-join:
def translate_multiline(text, tgt_lang, src_lang=None): lines = text.split('\n') nonempty = [l for l in lines if l.strip()] translated = iter(translate(nonempty, tgt_lang, src_lang)) if nonempty else iter([]) return '\n'.join(next(translated) if l.strip() else '' for l in lines)The model was also trained on whole multi-line dialogues, so translating a short paragraph in one go works too.
Input length. Training pairs were at most 512 tokens. Longer inputs are truncated, so split long documents into sentences or paragraphs first.
Source language: give it if you know it. It helps slightly (+0.3 chrF on average), mostly for xx→eng into English. Leave it out for code-switched or mixed-language text.
Mixed target languages in one batch.
forced_bos_token_idsets one target language pergeneratecall. To translate into several target languages in one batch, passdecoder_input_ids=[[2, tgt_token_id], ...](decoder start token, then each row's target-language token) instead.Decoding. The reported scores use beam search with 4 beams. Greedy decoding (
num_beams=1) is faster and only slightly worse. The length cap intranslate()matters for the weaker languages: without it, some outputs repeat a phrase until they reach the token limit.
Evaluation
All results are on the sunflower test split of salt-69: 100 sentences
per language, each translated eng→xx and xx→eng. Scores are chrF (×100),
averaged over sentences and then over directions. Before scoring, predictions
and references are normalized with Whisper's BasicTextNormalizer. Decoding
uses beam search (4 beams, up to 256 new tokens). None of the salt-69 texts
were used in training: training pairs containing any salt-69 sentence were
removed.
The comparison model, Sunflower-Qwen3.5-27B, was evaluated with the same metric code and test set.
| Model | Params | Avg chrF | eng→xx chrF | xx→eng chrF | Avg BLEU | Length ratio |
|---|---|---|---|---|---|---|
| This model, source language given | 1.3B | 43.7 | 40.0 | 47.4 | 19.0 | 1.27 |
| This model, source language not given | 1.3B | 43.4 | 40.0 | 46.7 | 18.8 | 1.27 |
| Sunflower-Qwen3.5-27B | 27B | 46.5 | 42.0 | 50.9 | 20.9 | 1.02 |
Notes:
- eng→xx is close. This model scores higher than Qwen in 22 of 69 eng→xx directions. These include Zulu, Igbo, Nigerian Pidgin, Sotho, Bambara, Kanuri and Dinka, and most of the larger Southern, Eastern and West African languages.
- xx→eng favours Qwen, which leads in 63 of 69 xx→eng directions. A 27B LLM writes more fluent English than a 1.3B translation model.
- Translating without the source language costs 0.3 chrF overall. eng→xx is unchanged, because the model recognizes English. xx→eng loses 0.7, most for Wolof, Kanuri, Yoruba, Bemba and Kikuyu (−2.3 to −3.2).
- Length ratio 1.27. On average, outputs are 27% longer than the references. Nearly all of this comes from eng→xx on the weakest languages, where the model sometimes repeats a phrase until it reaches the token limit (see Limitations). The length cap in the Quick start code limits this.
How many of the 69 languages reach a given average chrF (mean of eng→xx and xx→eng):
| Avg chrF | ≥ 50 | ≥ 40 | ≥ 30 | < 30 |
|---|---|---|---|---|
| Languages | 26 | 38 | 52 | 17 |
Per-language results
chrF ×100, source language given unless marked "no src". Bold marks the higher score of this model and Qwen in each direction. Sorted by this model's average.
| Code | Language | Token | eng→xx | xx→eng | eng→xx (no src) | xx→eng (no src) | Qwen eng→xx | Qwen xx→eng |
|---|---|---|---|---|---|---|---|---|
| afr | Afrikaans | afr_Latn |
80.0 | 82.7 | 80.2 | 83.2 | 82.2 | 85.1 |
| zul | Zulu | zul_Latn |
72.2 | 75.5 | 72.2 | 75.1 | 68.8 | 79.6 |
| xho | Xhosa | xho_Latn |
67.8 | 74.5 | 67.5 | 72.7 | 66.5 | 76.6 |
| nya | Chichewa | nya_Latn |
72.8 | 68.2 | 73.1 | 69.1 | 71.4 | 73.5 |
| fra | French | fra_Latn |
67.3 | 70.8 | 67.5 | 70.6 | 67.1 | 72.8 |
| swa | Swahili | swh_Latn |
67.1 | 70.6 | 67.2 | 68.9 | 66.2 | 72.6 |
| pcm | Nigerian Pidgin | deu_Latn |
63.8 | 72.6 | 63.6 | 72.1 | 58.0 | 78.4 |
| som | Somali | som_Latn |
63.5 | 72.5 | 63.3 | 70.5 | 61.3 | 77.1 |
| lin | Lingala | lin_Latn |
63.5 | 68.8 | 63.7 | 67.4 | 63.7 | 73.5 |
| sot | Sotho | sot_Latn |
64.6 | 67.3 | 64.3 | 67.3 | 59.7 | 71.0 |
| ibo | Igbo | ibo_Latn |
64.5 | 67.3 | 64.5 | 66.6 | 58.6 | 70.3 |
| mlg | Malagasy | plt_Latn |
62.0 | 65.5 | 61.7 | 64.9 | 60.1 | 71.5 |
| amh | Amharic | amh_Ethi |
53.3 | 70.8 | 53.2 | 70.5 | 49.5 | 77.9 |
| tsn | Tswana | tsn_Latn |
58.0 | 63.3 | 57.9 | 62.6 | 55.8 | 64.9 |
| kin | Kinyarwanda | kin_Latn |
61.3 | 57.9 | 61.5 | 57.6 | 59.1 | 62.1 |
| yor | Yoruba | yor_Latn |
52.2 | 66.2 | 51.8 | 63.5 | 50.3 | 71.3 |
| hau | Hausa | hau_Latn |
57.1 | 60.5 | 56.7 | 60.0 | 55.7 | 65.3 |
| lug | Luganda | lug_Latn |
54.7 | 57.2 | 54.6 | 56.4 | 57.5 | 63.8 |
| sna | Shona | sna_Latn |
51.7 | 60.0 | 51.4 | 59.1 | 52.6 | 63.7 |
| nbl | Ndebele | ita_Latn |
48.3 | 63.4 | 48.0 | 65.4 | 49.1 | 68.1 |
| ewe | Ewe | ewe_Latn |
51.9 | 58.1 | 51.7 | 57.3 | 51.0 | 59.0 |
| luo | Luo | luo_Latn |
52.8 | 54.0 | 52.8 | 53.5 | 50.0 | 57.9 |
| run | Kirundi | run_Latn |
49.2 | 56.9 | 49.0 | 56.1 | 50.8 | 65.9 |
| bam | Bambara | bam_Latn |
48.0 | 55.9 | 47.8 | 55.2 | 41.1 | 53.8 |
| nyo | Runyoro | dan_Latn |
49.0 | 53.9 | 48.9 | 54.0 | 50.9 | 61.2 |
| orm | Oromo | gaz_Latn |
47.5 | 53.1 | 47.5 | 51.6 | 46.1 | 58.9 |
| nyn | Runyankole | lim_Latn |
48.0 | 50.6 | 47.8 | 49.6 | 49.6 | 54.9 |
| cgg | Rukiga | fao_Latn |
45.5 | 51.0 | 45.2 | 50.4 | 46.4 | 54.3 |
| aka | Akan | aka_Latn |
41.8 | 54.5 | 42.0 | 54.6 | 42.7 | 58.6 |
| bem | Bemba | bem_Latn |
44.0 | 52.4 | 43.9 | 50.0 | 45.4 | 52.6 |
| ach | Acholi | fur_Latn |
47.2 | 49.1 | 47.5 | 48.6 | 51.2 | 52.7 |
| xog | Lusoga | ltg_Latn |
44.4 | 51.9 | 45.2 | 50.3 | 45.6 | 60.1 |
| kik | Kikuyu | kik_Latn |
40.1 | 53.5 | 39.4 | 51.2 | 40.7 | 64.4 |
| ttj | Rutooro | scn_Latn |
43.3 | 49.3 | 43.7 | 48.0 | 43.6 | 54.3 |
| ruc | Ruruuli | fin_Latn |
39.3 | 46.7 | 39.3 | 46.9 | 38.6 | 56.8 |
| laj | Lango | mlt_Latn |
37.9 | 46.9 | 37.9 | 46.7 | 39.8 | 50.1 |
| kdi | Kumam | gle_Latn |
37.1 | 44.0 | 37.3 | 44.2 | 38.4 | 45.9 |
| gwr | Lugwere | oci_Latn |
38.0 | 42.8 | 37.6 | 43.5 | 38.3 | 51.2 |
| teo | Ateso | lmo_Latn |
37.9 | 40.1 | 38.2 | 38.6 | 42.8 | 46.7 |
| kab | Kabyle | kab_Latn |
33.6 | 43.4 | 33.7 | 42.1 | 34.1 | 49.4 |
| myx | Lumasaba | srd_Latn |
35.9 | 39.6 | 35.8 | 39.6 | 39.6 | 44.0 |
| din | Dinka | dik_Latn |
28.7 | 45.5 | 28.9 | 44.8 | 20.1 | 26.1 |
| lgg | Lugbara | lij_Latn |
36.1 | 36.6 | 36.1 | 35.5 | 41.2 | 44.8 |
| wol | Wolof | wol_Latn |
32.3 | 40.2 | 31.8 | 37.0 | 32.8 | 48.9 |
| rub | Lugungu | est_Latn |
28.1 | 44.2 | 28.1 | 43.7 | 39.9 | 51.1 |
| nuj | Lunyole | nob_Latn |
35.0 | 36.5 | 35.0 | 36.9 | 36.9 | 43.1 |
| adh | Jopadhola | szl_Latn |
33.0 | 36.0 | 33.1 | 35.0 | 34.0 | 41.5 |
| tlj | Lubwisi | nno_Latn |
29.1 | 37.2 | 29.6 | 37.2 | 35.0 | 41.8 |
| koo | Rukonjo | epo_Latn |
32.2 | 33.4 | 32.5 | 32.3 | 36.4 | 37.1 |
| lsm | Samia | als_Latn |
32.1 | 33.2 | 31.8 | 34.5 | 34.8 | 39.1 |
| bfa | Bari | ast_Latn |
28.2 | 36.6 | 29.2 | 36.4 | 34.8 | 40.9 |
| alz | Alur | vec_Latn |
28.4 | 36.3 | 28.7 | 36.0 | 29.3 | 38.9 |
| dag | Dagbani | nld_Latn |
31.1 | 28.9 | 31.3 | 28.5 | 33.4 | 36.1 |
| keo | Kakwa | eus_Latn |
30.3 | 29.0 | 30.1 | 29.2 | 35.5 | 31.0 |
| kau | Kanuri | knc_Latn |
20.3 | 35.2 | 20.2 | 32.3 | 14.5 | 24.3 |
| ful | Fulani | fuv_Latn |
22.0 | 32.1 | 21.9 | 31.6 | 22.6 | 32.6 |
| kdj | Karamojong | cym_Latn |
25.2 | 28.6 | 25.1 | 27.2 | 32.7 | 32.9 |
| rwm | Kwamba | hun_Latn |
21.6 | 27.1 | 21.7 | 27.1 | 27.1 | 27.1 |
| mhi | Ma'di | slk_Latn |
16.9 | 28.1 | 16.9 | 25.9 | 29.4 | 28.5 |
| luy | Luhya | pol_Latn |
14.6 | 26.7 | 15.1 | 26.6 | 18.2 | 28.1 |
| kpz | Kupsabiny | ltz_Latn |
12.5 | 23.4 | 12.2 | 23.1 | 20.8 | 22.5 |
| pok | Pokot | lvs_Latn |
13.7 | 21.1 | 13.6 | 20.3 | 19.5 | 21.2 |
| luc | Aringa | hrv_Latn |
8.0 | 26.6 | 8.4 | 25.7 | 23.4 | 29.3 |
| dga | Dagaare | swe_Latn |
11.5 | 22.9 | 11.6 | 22.9 | 16.1 | 26.4 |
| ikx | Ik | gla_Latn |
10.6 | 22.9 | 10.9 | 22.2 | 21.5 | 22.1 |
| ber | Berber | tzm_Tfng |
0.1 | 31.8 | 0.0 | 30.2 | 24.6 | 38.6 |
| kln | Kalenjin | cat_Latn |
11.0 | 20.8 | 10.9 | 21.4 | 16.5 | 23.8 |
| led | Lendu | isl_Latn |
6.7 | 24.6 | 6.8 | 24.6 | 21.8 | 24.3 |
| kpo | Ikposo | glg_Latn |
3.1 | 18.6 | 3.1 | 18.6 | 4.3 | 19.2 |
Limitations
Problems seen in salt-69 scores and in manual spot checks (eng→xx, beam search):
- Weak languages. The ~17 languages with average chrF below 30 (Kakwa through Ikposo in the table above) should not be used for eng→xx without human review. xx→eng into English is usually more usable (chrF 20–35) than eng→xx.
- Output in the wrong language. For some repurposed codes, the model
sometimes falls back to the European language whose token was reused. In
spot checks, eng→Kalenjin (
cat_Latn) gave Catalan and eng→Ikposo (glg_Latn) gave Galician. Some languages also drift into a related, better-resourced language: eng→Ik and eng→Pokot gave text resembling Ateso. - Repetition. For several low-resource targets (e.g. Lendu, Dagaare,
Aringa, Kupsabiny, Kwamba), the decoder sometimes repeats a word or phrase
until it reaches the length limit. The length cap in
translate()limits the damage. Addingno_repeat_ngram_size=4togeneratealso helps, but it doesn't recover a correct translation. - Berber script. The Berber (
ber) training data is in Tifinagh script, so eng→ber output is Tifinagh. The salt-69 references are presumably in Latin script, which would explain the near-zero eng→ber score. Use the Kabyle (kab) code if you need Latin-script Berber output. - Domain. Training data is mostly general-domain and conversational sentences. Expect lower quality on specialised domains (legal, medical, technical) and on long documents.
- Punctuation. Output uses ASCII punctuation only (see Inference tips).
- Evaluation size. Each direction is scored on 100 sentences, so differences of 1–2 chrF points for a single language are within noise.
Training details
- Base model:
facebook/nllb-200-1.3B, fully fine-tuned (all parameters). - Data:
Sunbird/sunflower-pretrain-data(parallel_text): multi-way parallel text. Up to 6,000 pairs were sampled per eng→xx and per xx→eng direction. Non-English pairs were added, equal to 10% of the English-pivot pairs, with at most 2,000 per language pair.Sunbird/sunflower-posttrain-data(sft_translations): curated translation pairs, upsampled 2×. Multi-line dialogues were included both whole and split line by line.- Pairs were removed if they were empty, identical on both sides, duplicated, had a length ratio over 3:1, or contained any salt-69 test sentence.
- Source-language dropout: in 25% of training pairs, the source-language
token was replaced with the unknown-source token (
und/smo_Latn). This is what makes the source language optional at inference. - Hyperparameters: AdamW 8-bit (torchao) in pure bfloat16 with stochastic rounding. Learning rate 1e-4, linear schedule with 100 warmup steps, no weight decay, up to 2 epochs. Batches were token-budgeted (≤3,072 padded tokens per micro-batch, 12 micro-batches per step, about 28k tokens per optimizer step). Max 512 tokens per side. Seed 42.
- Model selection: evaluated every 500 steps with greedy-decoded chrF on
the
sft_translationsdev split. The best checkpoint was kept, with early stopping after 6 evaluations without improvement. salt-69 was used only for the final evaluation. - Hardware: one NVIDIA RTX 5090 (32 GB).
- Code:
notebooks/training/translation_nllb_training_69_languages.pyin the SunbirdAI/salt repository (prepare/train/evalcommands, run with--base_model facebook/nllb-200-1.3B).
License
This model is derived from NLLB-200, which is released under CC-BY-NC-4.0, and is therefore also for non-commercial use.
Citation
If you use this model, please cite Sunbird AI and the NLLB paper:
@article{nllb2022,
title = {No Language Left Behind: Scaling Human-Centered Machine Translation},
author = {{NLLB Team} and Costa-juss{\`a}, Marta R. and others},
journal = {arXiv preprint arXiv:2207.04672},
year = {2022}
}
- Downloads last month
- 25
Model tree for Sunbird-experimental/nllb1.3b-67languages
Base model
facebook/nllb-200-1.3B