Translation
Transformers
Safetensors
m2m_100
text2text-generation
nllb
african-languages
uganda
sunbird

Sunflower Translate (NLLB-1.3B, salt-69)

Sunbird-experimental/nllb1.3b-67languages is NLLB-200 1.3B fine-tuned by Sunbird AI to translate between English and 74 other languages. Most are spoken in Uganda; the rest are from elsewhere in Africa, plus French. It translates in any direction between these languages, including between two non-English languages.

It is a small, fast alternative to the Sunflower LLMs for translation only. On the salt-69 benchmark, it scores about 3 chrF points lower than Sunflower-Qwen3.5-27B on average (43.7 vs 46.5). It is level with or ahead of Qwen in about a third of the eng→xx directions, at a fraction of the size (1.3B vs 27B parameters). See Evaluation.

Additions to standard NLLB:

  • 42 languages NLLB doesn't support, through repurposed language tokens (see Language codes).
  • The source language is optional. The model was trained to translate with or without being told the source language. This is useful for code-switched text, or when the source language is unknown. Leaving it out costs only about 0.3 chrF on average.

Status: experimental. Quality varies a lot by language: from strong (Afrikaans, Zulu, Swahili, Luganda) to unusable (Ikposo, Lendu, Kalenjin). Read Limitations before using it for a particular language.

Quick start

pip install torch transformers sentencepiece sacremoses huggingface_hub
import json
import re

import torch
from huggingface_hub import hf_hub_download
from sacremoses import MosesPunctNormalizer
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

MODEL_ID = 'Sunbird-experimental/nllb1.3b-67languages'
device = 'cuda' if torch.cuda.is_available() else 'cpu'

tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
model = AutoModelForSeq2SeqLM.from_pretrained(
    MODEL_ID, dtype=torch.bfloat16).to(device).eval()

# SALT language code (e.g. 'lug') -> NLLB language token (e.g. 'lug_Latn').
with open(hf_hub_download(MODEL_ID, 'nllb_language_codes.json')) as f:
    LANG_CODES = json.load(f)
UNKNOWN_SOURCE = 'und'  # Source language not specified

_punct = MosesPunctNormalizer(
    pre_replace_unicode_punct=True, post_remove_control_chars=True,
    norm_quote_commas=False, norm_numbers=False)


def normalize(text):
    '''Applies the same normalization as in training: typographic quotes and
    dashes -> ASCII, whitespace collapsed, control characters removed.'''
    text = re.sub(r'\s+', ' ', text)
    text = text.replace('̇', '').replace('̦', '')
    return _punct.normalize(text)


@torch.no_grad()
def translate(texts, tgt_lang, src_lang=None, num_beams=4):
    '''Translates a list of texts into tgt_lang (a SALT code such as 'lug').
    src_lang is optional; leave it as None if unknown or code-switched.'''
    src_token = LANG_CODES[src_lang or UNKNOWN_SOURCE]
    inputs = tokenizer([normalize(t) for t in texts], return_tensors='pt',
                       padding=True, truncation=True, max_length=512).to(device)
    # NLLB puts the source-language token first: set it explicitly.
    inputs['input_ids'][:, 0] = tokenizer.convert_tokens_to_ids(src_token)
    # Cap the output length to stop runaway repetition on weaker languages.
    max_new_tokens = min(512, 2 * inputs['input_ids'].shape[1] + 10)
    outputs = model.generate(
        **inputs,
        forced_bos_token_id=tokenizer.convert_tokens_to_ids(LANG_CODES[tgt_lang]),
        num_beams=num_beams, max_new_tokens=max_new_tokens, max_length=None)
    return tokenizer.batch_decode(outputs, skip_special_tokens=True)


print(translate(['How are you this morning?',
                 'Farmers should plant their crops before the rains begin.'],
                tgt_lang='lug', src_lang='eng'))
# ['Oli otya enkya eno?',
#  "Abalimi basaanidde okusimba ebirime byabwe ng'enkuba tennatandika."]

print(translate(['Abaana bagenda ku ssomero.'], tgt_lang='eng'))  # no source language
# ['Children go to school.']

print(translate(['Abaana bagenda ku ssomero.'], tgt_lang='ach', src_lang='lug'))
# ['Lutino gicito i cukul.']

The model needs about 3 GB of GPU memory in bfloat16. It also runs on CPU, more slowly.

Why not tokenizer.src_lang / pipeline('translation')?

The standard NLLB recipe (tokenizer.src_lang = 'lug_Latn', or the translation pipeline with src_lang/tgt_lang) works for languages with a native NLLB code. The code above is more general:

  • It uses the SALT codes from nllb_language_codes.json. You don't need to know which NLLB token stands for each language.
  • It supports src_lang=None (the und token), which the standard tokenizer doesn't know about.
  • It applies the input normalization the model was trained with.

Language codes

Use the SALT codes (mostly ISO 639-3) below for src_lang and tgt_lang. nllb_language_codes.json in this repo maps each one to an NLLB language token:

  • Native codes: 33 languages (including English) use their own NLLB token, e.g. lug → lug_Latn, swa → swh_Latn, orm → gaz_Latn, mlg → plt_Latn.
  • Repurposed codes: 42 languages NLLB doesn't support reuse the token of a European language that this model doesn't translate. For example, Acholi ach → fur_Latn (Friulian), Runyankole nyn → lim_Latn, and Ateso teo → lmo_Latn. The NLLB language named in such a token is not a supported language. For example, asking for fur_Latn gives Acholi, not Friulian.
  • Unknown source: und → smo_Latn. Use it only on the source side, to translate without saying what the source language is. It is not a valid target.

The full mapping, with each language's scores, is in the per-language results. The model has tokens for five more languages that salt-69 doesn't cover: Thur (lth), Lulubo (lzm), Kebu (ndp), Sebei (sbx) and So (soc). Their quality has not been measured.

Inference tips

  • Normalize inputs. NLLB's vocabulary has no typographic quotes or dashes, so the model was trained on text normalized as in normalize() above. Output uses ASCII punctuation (' " -); post-process it if you need typographic punctuation.

  • Preserve line breaks. The tokenizer turns newlines into spaces. To keep the layout of multi-line text, split it into lines, translate the non-empty lines as one batch, and re-join:

    def translate_multiline(text, tgt_lang, src_lang=None):
        lines = text.split('\n')
        nonempty = [l for l in lines if l.strip()]
        translated = iter(translate(nonempty, tgt_lang, src_lang)) if nonempty else iter([])
        return '\n'.join(next(translated) if l.strip() else '' for l in lines)
    

    The model was also trained on whole multi-line dialogues, so translating a short paragraph in one go works too.

  • Input length. Training pairs were at most 512 tokens. Longer inputs are truncated, so split long documents into sentences or paragraphs first.

  • Source language: give it if you know it. It helps slightly (+0.3 chrF on average), mostly for xx→eng into English. Leave it out for code-switched or mixed-language text.

  • Mixed target languages in one batch. forced_bos_token_id sets one target language per generate call. To translate into several target languages in one batch, pass decoder_input_ids=[[2, tgt_token_id], ...] (decoder start token, then each row's target-language token) instead.

  • Decoding. The reported scores use beam search with 4 beams. Greedy decoding (num_beams=1) is faster and only slightly worse. The length cap in translate() matters for the weaker languages: without it, some outputs repeat a phrase until they reach the token limit.

Evaluation

All results are on the sunflower test split of salt-69: 100 sentences per language, each translated eng→xx and xx→eng. Scores are chrF (×100), averaged over sentences and then over directions. Before scoring, predictions and references are normalized with Whisper's BasicTextNormalizer. Decoding uses beam search (4 beams, up to 256 new tokens). None of the salt-69 texts were used in training: training pairs containing any salt-69 sentence were removed.

The comparison model, Sunflower-Qwen3.5-27B, was evaluated with the same metric code and test set.

Model Params Avg chrF eng→xx chrF xx→eng chrF Avg BLEU Length ratio
This model, source language given 1.3B 43.7 40.0 47.4 19.0 1.27
This model, source language not given 1.3B 43.4 40.0 46.7 18.8 1.27
Sunflower-Qwen3.5-27B 27B 46.5 42.0 50.9 20.9 1.02

Notes:

  • eng→xx is close. This model scores higher than Qwen in 22 of 69 eng→xx directions. These include Zulu, Igbo, Nigerian Pidgin, Sotho, Bambara, Kanuri and Dinka, and most of the larger Southern, Eastern and West African languages.
  • xx→eng favours Qwen, which leads in 63 of 69 xx→eng directions. A 27B LLM writes more fluent English than a 1.3B translation model.
  • Translating without the source language costs 0.3 chrF overall. eng→xx is unchanged, because the model recognizes English. xx→eng loses 0.7, most for Wolof, Kanuri, Yoruba, Bemba and Kikuyu (−2.3 to −3.2).
  • Length ratio 1.27. On average, outputs are 27% longer than the references. Nearly all of this comes from eng→xx on the weakest languages, where the model sometimes repeats a phrase until it reaches the token limit (see Limitations). The length cap in the Quick start code limits this.

How many of the 69 languages reach a given average chrF (mean of eng→xx and xx→eng):

Avg chrF ≥ 50 ≥ 40 ≥ 30 < 30
Languages 26 38 52 17

Per-language results

chrF ×100, source language given unless marked "no src". Bold marks the higher score of this model and Qwen in each direction. Sorted by this model's average.

Code Language Token eng→xx xx→eng eng→xx (no src) xx→eng (no src) Qwen eng→xx Qwen xx→eng
afr Afrikaans afr_Latn 80.0 82.7 80.2 83.2 82.2 85.1
zul Zulu zul_Latn 72.2 75.5 72.2 75.1 68.8 79.6
xho Xhosa xho_Latn 67.8 74.5 67.5 72.7 66.5 76.6
nya Chichewa nya_Latn 72.8 68.2 73.1 69.1 71.4 73.5
fra French fra_Latn 67.3 70.8 67.5 70.6 67.1 72.8
swa Swahili swh_Latn 67.1 70.6 67.2 68.9 66.2 72.6
pcm Nigerian Pidgin deu_Latn 63.8 72.6 63.6 72.1 58.0 78.4
som Somali som_Latn 63.5 72.5 63.3 70.5 61.3 77.1
lin Lingala lin_Latn 63.5 68.8 63.7 67.4 63.7 73.5
sot Sotho sot_Latn 64.6 67.3 64.3 67.3 59.7 71.0
ibo Igbo ibo_Latn 64.5 67.3 64.5 66.6 58.6 70.3
mlg Malagasy plt_Latn 62.0 65.5 61.7 64.9 60.1 71.5
amh Amharic amh_Ethi 53.3 70.8 53.2 70.5 49.5 77.9
tsn Tswana tsn_Latn 58.0 63.3 57.9 62.6 55.8 64.9
kin Kinyarwanda kin_Latn 61.3 57.9 61.5 57.6 59.1 62.1
yor Yoruba yor_Latn 52.2 66.2 51.8 63.5 50.3 71.3
hau Hausa hau_Latn 57.1 60.5 56.7 60.0 55.7 65.3
lug Luganda lug_Latn 54.7 57.2 54.6 56.4 57.5 63.8
sna Shona sna_Latn 51.7 60.0 51.4 59.1 52.6 63.7
nbl Ndebele ita_Latn 48.3 63.4 48.0 65.4 49.1 68.1
ewe Ewe ewe_Latn 51.9 58.1 51.7 57.3 51.0 59.0
luo Luo luo_Latn 52.8 54.0 52.8 53.5 50.0 57.9
run Kirundi run_Latn 49.2 56.9 49.0 56.1 50.8 65.9
bam Bambara bam_Latn 48.0 55.9 47.8 55.2 41.1 53.8
nyo Runyoro dan_Latn 49.0 53.9 48.9 54.0 50.9 61.2
orm Oromo gaz_Latn 47.5 53.1 47.5 51.6 46.1 58.9
nyn Runyankole lim_Latn 48.0 50.6 47.8 49.6 49.6 54.9
cgg Rukiga fao_Latn 45.5 51.0 45.2 50.4 46.4 54.3
aka Akan aka_Latn 41.8 54.5 42.0 54.6 42.7 58.6
bem Bemba bem_Latn 44.0 52.4 43.9 50.0 45.4 52.6
ach Acholi fur_Latn 47.2 49.1 47.5 48.6 51.2 52.7
xog Lusoga ltg_Latn 44.4 51.9 45.2 50.3 45.6 60.1
kik Kikuyu kik_Latn 40.1 53.5 39.4 51.2 40.7 64.4
ttj Rutooro scn_Latn 43.3 49.3 43.7 48.0 43.6 54.3
ruc Ruruuli fin_Latn 39.3 46.7 39.3 46.9 38.6 56.8
laj Lango mlt_Latn 37.9 46.9 37.9 46.7 39.8 50.1
kdi Kumam gle_Latn 37.1 44.0 37.3 44.2 38.4 45.9
gwr Lugwere oci_Latn 38.0 42.8 37.6 43.5 38.3 51.2
teo Ateso lmo_Latn 37.9 40.1 38.2 38.6 42.8 46.7
kab Kabyle kab_Latn 33.6 43.4 33.7 42.1 34.1 49.4
myx Lumasaba srd_Latn 35.9 39.6 35.8 39.6 39.6 44.0
din Dinka dik_Latn 28.7 45.5 28.9 44.8 20.1 26.1
lgg Lugbara lij_Latn 36.1 36.6 36.1 35.5 41.2 44.8
wol Wolof wol_Latn 32.3 40.2 31.8 37.0 32.8 48.9
rub Lugungu est_Latn 28.1 44.2 28.1 43.7 39.9 51.1
nuj Lunyole nob_Latn 35.0 36.5 35.0 36.9 36.9 43.1
adh Jopadhola szl_Latn 33.0 36.0 33.1 35.0 34.0 41.5
tlj Lubwisi nno_Latn 29.1 37.2 29.6 37.2 35.0 41.8
koo Rukonjo epo_Latn 32.2 33.4 32.5 32.3 36.4 37.1
lsm Samia als_Latn 32.1 33.2 31.8 34.5 34.8 39.1
bfa Bari ast_Latn 28.2 36.6 29.2 36.4 34.8 40.9
alz Alur vec_Latn 28.4 36.3 28.7 36.0 29.3 38.9
dag Dagbani nld_Latn 31.1 28.9 31.3 28.5 33.4 36.1
keo Kakwa eus_Latn 30.3 29.0 30.1 29.2 35.5 31.0
kau Kanuri knc_Latn 20.3 35.2 20.2 32.3 14.5 24.3
ful Fulani fuv_Latn 22.0 32.1 21.9 31.6 22.6 32.6
kdj Karamojong cym_Latn 25.2 28.6 25.1 27.2 32.7 32.9
rwm Kwamba hun_Latn 21.6 27.1 21.7 27.1 27.1 27.1
mhi Ma'di slk_Latn 16.9 28.1 16.9 25.9 29.4 28.5
luy Luhya pol_Latn 14.6 26.7 15.1 26.6 18.2 28.1
kpz Kupsabiny ltz_Latn 12.5 23.4 12.2 23.1 20.8 22.5
pok Pokot lvs_Latn 13.7 21.1 13.6 20.3 19.5 21.2
luc Aringa hrv_Latn 8.0 26.6 8.4 25.7 23.4 29.3
dga Dagaare swe_Latn 11.5 22.9 11.6 22.9 16.1 26.4
ikx Ik gla_Latn 10.6 22.9 10.9 22.2 21.5 22.1
ber Berber tzm_Tfng 0.1 31.8 0.0 30.2 24.6 38.6
kln Kalenjin cat_Latn 11.0 20.8 10.9 21.4 16.5 23.8
led Lendu isl_Latn 6.7 24.6 6.8 24.6 21.8 24.3
kpo Ikposo glg_Latn 3.1 18.6 3.1 18.6 4.3 19.2

Limitations

Problems seen in salt-69 scores and in manual spot checks (eng→xx, beam search):

  • Weak languages. The ~17 languages with average chrF below 30 (Kakwa through Ikposo in the table above) should not be used for eng→xx without human review. xx→eng into English is usually more usable (chrF 20–35) than eng→xx.
  • Output in the wrong language. For some repurposed codes, the model sometimes falls back to the European language whose token was reused. In spot checks, eng→Kalenjin (cat_Latn) gave Catalan and eng→Ikposo (glg_Latn) gave Galician. Some languages also drift into a related, better-resourced language: eng→Ik and eng→Pokot gave text resembling Ateso.
  • Repetition. For several low-resource targets (e.g. Lendu, Dagaare, Aringa, Kupsabiny, Kwamba), the decoder sometimes repeats a word or phrase until it reaches the length limit. The length cap in translate() limits the damage. Adding no_repeat_ngram_size=4 to generate also helps, but it doesn't recover a correct translation.
  • Berber script. The Berber (ber) training data is in Tifinagh script, so eng→ber output is Tifinagh. The salt-69 references are presumably in Latin script, which would explain the near-zero eng→ber score. Use the Kabyle (kab) code if you need Latin-script Berber output.
  • Domain. Training data is mostly general-domain and conversational sentences. Expect lower quality on specialised domains (legal, medical, technical) and on long documents.
  • Punctuation. Output uses ASCII punctuation only (see Inference tips).
  • Evaluation size. Each direction is scored on 100 sentences, so differences of 1–2 chrF points for a single language are within noise.

Training details

  • Base model: facebook/nllb-200-1.3B, fully fine-tuned (all parameters).
  • Data:
    • Sunbird/sunflower-pretrain-data (parallel_text): multi-way parallel text. Up to 6,000 pairs were sampled per eng→xx and per xx→eng direction. Non-English pairs were added, equal to 10% of the English-pivot pairs, with at most 2,000 per language pair.
    • Sunbird/sunflower-posttrain-data (sft_translations): curated translation pairs, upsampled 2×. Multi-line dialogues were included both whole and split line by line.
    • Pairs were removed if they were empty, identical on both sides, duplicated, had a length ratio over 3:1, or contained any salt-69 test sentence.
  • Source-language dropout: in 25% of training pairs, the source-language token was replaced with the unknown-source token (und/smo_Latn). This is what makes the source language optional at inference.
  • Hyperparameters: AdamW 8-bit (torchao) in pure bfloat16 with stochastic rounding. Learning rate 1e-4, linear schedule with 100 warmup steps, no weight decay, up to 2 epochs. Batches were token-budgeted (≤3,072 padded tokens per micro-batch, 12 micro-batches per step, about 28k tokens per optimizer step). Max 512 tokens per side. Seed 42.
  • Model selection: evaluated every 500 steps with greedy-decoded chrF on the sft_translations dev split. The best checkpoint was kept, with early stopping after 6 evaluations without improvement. salt-69 was used only for the final evaluation.
  • Hardware: one NVIDIA RTX 5090 (32 GB).
  • Code: notebooks/training/translation_nllb_training_69_languages.py in the SunbirdAI/salt repository (prepare / train / eval commands, run with --base_model facebook/nllb-200-1.3B).

License

This model is derived from NLLB-200, which is released under CC-BY-NC-4.0, and is therefore also for non-commercial use.

Citation

If you use this model, please cite Sunbird AI and the NLLB paper:

@article{nllb2022,
  title   = {No Language Left Behind: Scaling Human-Centered Machine Translation},
  author  = {{NLLB Team} and Costa-juss{\`a}, Marta R. and others},
  journal = {arXiv preprint arXiv:2207.04672},
  year    = {2022}
}
Downloads last month
25
Safetensors
Model size
1B params
Tensor type
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for Sunbird-experimental/nllb1.3b-67languages

Finetuned
(38)
this model

Paper for Sunbird-experimental/nllb1.3b-67languages