License: free for commercial use, with required attribution to the author's work. If you use this model, or text it generated, in a product, service or publication, credit "Translation models by Aletheia AI" and cite the accompanying paper (in preparation). This model is a derivative of Google Gemma 3, so the Gemma Terms of Use and Prohibited Use Policy (https://ai.google.dev/gemma/terms) also apply.

gemma-3-1b-science-sft

What this is. The science SFT stage (weight interpolation alpha 0.5) of the 1B science line of the SabiYarn / gemma_mt project, and the starting point of the science RL stage that produced gemma-3-1b-science-translate-best. It is gemma-3-1b-translate-best (general RL model, Gemma-3-1B-it architecture, about 1.0B parameters, fp32) fine-tuned on AfriScience-MT windows and whole papers (amh, hau, lug, nso, yor, zul) and then averaged with its start model: theta = 0.5 x start + 0.5 x science-tuned. English to African language only; 125 tags (20 without general SFT data, three of them with a small science top-up). Role: science / SFT / starting point for science RL and further fine-tuning. Release status: public.

For translation use BeardedMonster/gemma-3-1b-science-translate-best, which adds the science RL stage (whole papers finish 96% of the time against 89% here on dev). Pick this SFT model when you want the starting point for the science RL or for further fine-tuning. It was evaluated on the AfriScience-MT dev split and on a small general dev set only; FLORES+ devtest, NTREX, MAFAND and AfriScience-MT test were not measured for this model.

Headline (dev): AfriScience-MT sentence chrF macro of 6 languages 53.52 (start 51.07); whole-paper chrF 52.96 (start 49.21; 89% of papers finished); general dev chrF++ macro (18 languages x 40 sentences) 43.76 (start 43.90).

Quick start (read first)

One user turn, exactly as in training (two spaces after the colon):

Translate the text to {Language name}:  {English text}

Example: Translate the text to Yoruba: Good morning, how are you?

Rendered with the chat template (this is what the model sees):

<bos><start_of_turn>user
Translate the text to Yoruba:  Good morning, how are you?<end_of_turn>
<start_of_turn>model

The model answers with the language tag as ONE token, then the translation, then <end_of_turn> (id 106). Format only (illustrative, not a measured output):

<yor> {Yoruba translation}<end_of_turn>

The tag tokens (<yor>, <hau>, ...) are ordinary (non-special) vocabulary entries, so skip_special_tokens=True keeps them; remove the leading tag yourself (re.sub(r"^<[a-z]{3}>\s*", "", text)).

  • Use the language name from the table below in the prompt (Yoruba), never the ISO code. The tag in the answer is only an output marker.
  • Decoding: greedy (do_sample=False / temperature=0), max_new_tokens=256 for a sentence or short paragraph (the benchmarks below used 768). Optional in vLLM: frequency_penalty=0.1 as a guard against repetition loops on long inputs.
  • English to African language only. One sentence or a short paragraph per prompt; split long documents into sentences.

Whole papers use the same prompt (see the science card); on dev 11% of papers did not finish for this model (Amharic 0.68, Luganda 0.88), so check for a missing <end_of_turn>.

Code

Transformers

import re, torch
from transformers import AutoTokenizer, AutoModelForCausalLM   # transformers >= 4.56 (tested 4.57.6); older versions: torch_dtype=

repo = "BeardedMonster/gemma-3-1b-science-sft"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, dtype=torch.bfloat16).cuda().eval()

def translate(text, language, max_new_tokens=256):
    msgs = [{"role": "user", "content": f"Translate the text to {language}:  {text}"}]   # two spaces after the colon
    enc = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt", return_dict=True).to("cuda")
    with torch.no_grad():
        out = model.generate(**enc, max_new_tokens=max_new_tokens, do_sample=False)
    raw = tok.decode(out[0, enc["input_ids"].shape[1]:], skip_special_tokens=True)   # "<yor> ..."
    return re.sub(r"^<[a-z]{3}>\s*", "", raw.strip())

print(translate("Good morning, how are you?", "Yoruba"))

# one sentence into five languages
for language in ["Yoruba", "Hausa", "Igbo", "Swahili", "Zulu"]:
    print(language, "->", translate("The meeting starts at nine tomorrow.", language))

vLLM (tested with vLLM 0.32.0)

from vllm import LLM, SamplingParams

repo = "BeardedMonster/gemma-3-1b-science-sft"
llm = LLM(repo, dtype="bfloat16", max_model_len=4096)
tok = llm.get_tokenizer()

def build(language, text):
    msgs = [{"role": "user", "content": f"Translate the text to {language}:  {text}"}]
    return tok.apply_chat_template(msgs, add_generation_prompt=True, tokenize=False)

import re
sentence = "The meeting starts at nine tomorrow."
languages = ["Yoruba", "Hausa", "Igbo", "Swahili", "Zulu"]
params = SamplingParams(temperature=0, max_tokens=256, frequency_penalty=0.1)   # greedy; the penalty is optional
outs = llm.generate([build(l, sentence) for l in languages], params)             # one batch, five languages
for language, o in zip(languages, outs):
    print(language, "->", re.sub(r"^<[a-z]{3}>\s*", "", o.outputs[0].text.strip()))

OpenAI-compatible server: vllm serve BeardedMonster/gemma-3-1b-science-sft --max-model-len 4096, then send the same user message to /v1/chat/completions with temperature=0.

Languages

125 tags; the prompt uses the name in the first column. Numbers in the last two columns are of this model on small dev sets (see "How measured"), not FLORES+ devtest.

Name used in the prompt Tag Script SFT data chrF++ (general dev, 40 sentences) chrF (AfriScience-MT dev, 843 sentences)
Yoruba <yor> Latin yes 30.72 41.36
Hausa <hau> Latin yes 51.44 64.08
Igbo <ibo> Latin yes 40.94 n/a
Pidgin <pcm> Latin yes n/a n/a
Efik <efi> Latin yes n/a n/a
Twi <twi> Latin yes 38.33 n/a
Ewe <ewe> Latin yes 32.64 n/a
Fon <fon> Latin yes 20.94 n/a
Wolof <wol> Latin yes 34.13 n/a
Swahili <swh> Latin yes 58.02 n/a
Kinyarwanda <kin> Latin yes 44.38 n/a
Somali <som> Latin yes 45.39 n/a
Oromo <orm> Latin yes 34.67 n/a
Amharic <amh> Ethiopic (Ge'ez) yes n/a 42.40
Tigrinya <tir> Ethiopic (Ge'ez) yes n/a n/a
Luganda <lug> Latin yes n/a 47.31
Lingala <lin> Latin yes n/a n/a
Zulu <zul> Latin yes 55.59 63.88
Xhosa <xho> Latin yes 49.88 n/a
Afrikaans <afr> Latin yes 65.92 n/a
Shona <sna> Latin yes 41.00 n/a
Chichewa <nya> Latin yes 48.47 n/a
Sesotho <sot> Latin yes 46.06 n/a
Setswana <tsn> Latin yes 49.19 n/a
Malagasy <mlg> Latin yes n/a n/a
Ibibio <ibb> Latin science top-up only (small) n/a n/a
Urhobo <urh> Latin yes n/a n/a
Bemba <bem> Latin yes n/a n/a
Tumbuka <tum> Latin yes n/a n/a
Kirundi <run> Latin yes n/a n/a
Venda <ven> Latin yes n/a n/a
Tsonga <tso> Latin yes n/a n/a
Sepedi <nso> Latin yes n/a 62.08
Swati <ssw> Latin yes n/a n/a
South Ndebele <nbl> Latin yes n/a n/a
North Ndebele <nde> Latin yes n/a n/a
Sango <sag> Latin yes n/a n/a
Tiv <tiv> Latin yes n/a n/a
Umbundu <umb> Latin yes n/a n/a
Kimbundu <kmb> Latin yes n/a n/a
Mossi <mos> Latin yes n/a n/a
Kikuyu <kik> Latin yes n/a n/a
Luo <luo> Latin yes n/a n/a
Kamba <kam> Latin yes n/a n/a
Edo <bin> Latin none (cannot translate) n/a n/a
Dagbani <dag> Latin science top-up only (small) n/a n/a
Okpe <oke> Latin yes n/a n/a
Tachelhit <shi> Latin yes n/a n/a
Dyula <dyu> Latin yes n/a n/a
Bambara <bam> Latin yes n/a n/a
Kabiye <kbp> Latin yes n/a n/a
Ghomala <bbj> Latin none (cannot translate) n/a n/a
Kongo <kon> Latin yes n/a n/a
Kabuverdianu <kea> Latin yes n/a n/a
Fante <fat> Latin none (cannot translate) n/a n/a
Kanuri <knc> Latin yes n/a n/a
Ga <gaa> Latin none (cannot translate) n/a n/a
Kabyle <kab> Latin yes n/a n/a
Nuer <nus> Latin yes n/a n/a
Congo Swahili <swc> Latin yes n/a n/a
Luba-Kasai <lua> Latin yes n/a n/a
Kaonde <kqn> Latin yes n/a n/a
Lunda <lun> Latin yes n/a n/a
Isoko <iso> Latin yes n/a n/a
Lozi <loz> Latin yes n/a n/a
Wolaytta <wal> Latin yes n/a n/a
Acholi <ach> Latin yes n/a n/a
Tswa <tsc> Latin yes n/a n/a
Makhuwa <vmw> Latin yes n/a n/a
Kuanyama <kua> Latin yes n/a n/a
Sena <seh> Latin yes n/a n/a
Nyankore <nyn> Latin yes n/a n/a
Herero <her> Latin science top-up only (small) n/a n/a
Yao <yao> Latin yes n/a n/a
Sidamo <sid> Latin yes n/a n/a
Ndau <ndc> Latin yes n/a n/a
Mende <men> Latin none (cannot translate) n/a n/a
Krio <kri> Latin yes n/a n/a
Luyia <luy> Latin none (cannot translate) n/a n/a
Esan <ish> Latin none (cannot translate) n/a n/a
Duala <dua> Latin none (cannot translate) n/a n/a
Basaa <bas> Latin yes n/a n/a
Cameroon Pidgin <wes> Latin none (cannot translate) n/a n/a
Tamazight <tzm> Latin yes n/a n/a
Fulah <ful> Latin yes n/a n/a
Gun <guw> Latin yes n/a n/a
Luvale <lue> Latin yes n/a n/a
Tetela <tll> Latin yes n/a n/a
Zande <zne> Latin yes n/a n/a
Luba-Katanga <lub> Latin yes n/a n/a
Seychellois Creole <crs> Latin yes n/a n/a
Mauritian Creole <mfe> Latin yes n/a n/a
Ruund <rnd> Latin yes n/a n/a
Nyaneka <nyk> Latin yes n/a n/a
Kwangali <kwn> Latin yes n/a n/a
Ndonga <ndo> Latin yes n/a n/a
Zarma <dje> Latin yes n/a n/a
Adangme <ada> Latin yes n/a n/a
Egyptian Arabic <arz> Arabic yes n/a n/a
Songe <sop> Latin yes n/a n/a
Mambwe <mgr> Latin yes n/a n/a
Konzo <koo> Latin yes n/a n/a
Chokwe <cjk> Latin yes n/a n/a
Coptic <cop> Coptic yes n/a n/a
Lelemi <dop> Latin yes n/a n/a
Bulu <bum> Latin yes n/a n/a
Nyemba <nba> Latin yes n/a n/a
Nyungwe <nyu> Latin yes n/a n/a
Dinka <dik> Latin yes n/a n/a
Chopi <cce> Latin yes n/a n/a
Chuabo <chw> Latin yes n/a n/a
Tooro <ttj> Latin yes n/a n/a
Lamba <lam> Latin yes n/a n/a
Nupe <nup> Latin yes n/a n/a
Tamasheq <taq> Latin yes n/a n/a
Alur <alz> Latin yes n/a n/a
Tonga <toi> Latin yes n/a n/a
Berber <ber> Latin none (cannot translate) n/a n/a
Standard Moroccan Tamazight <zgh> Tifinagh none (cannot translate) n/a n/a
Afar <aar> Latin none (cannot translate) n/a n/a
Baoule <bci> Latin none (cannot translate) n/a n/a
Moroccan Arabic <ary> Arabic none (cannot translate) n/a n/a
Tunisian Arabic <aeb> Arabic none (cannot translate) n/a n/a
Teso <teo> Latin none (cannot translate) n/a n/a
Adhola <adh> Latin none (cannot translate) n/a n/a

Per-language performance of this model

Best / weakest on the AfriScience dev sentences: hau 64.1 and zul 63.9 best, yor 41.4 and amh 42.4 weakest. AfriScience-MT dev, greedy, plain chrF (this model = alpha 0.5; other alphas for reference, they are not released here):

Language (tag) AfriScience dev sentence chrF: start (a=0) this model (a=0.5) a=0.7 a=1.0 whole paper chrF: start this model a=0.7 a=1.0 whole papers finished (this model)
Amharic (amh) 40.93 42.40 42.28 41.58 30.35 27.82 28.80 21.68 0.68
Hausa (hau) 61.26 64.08 64.91 64.92 61.70 65.56 62.16 57.54 1.00
Luganda (lug) 44.81 47.31 49.03 49.67 47.49 48.25 47.33 35.52 0.88
Sepedi (nso) 57.23 62.08 63.30 63.79 56.30 62.80 66.06 64.59 0.92
Yoruba (yor) 39.91 41.36 42.04 41.69 37.40 45.35 41.81 36.17 0.92
Zulu (zul) 62.27 63.88 64.37 64.88 62.02 67.99 71.39 66.43 0.96
macro (6) 51.07 53.52 54.32 54.42 49.21 52.96 52.93 46.99 0.89

General dev (the project's clean general dev set: 18 languages x 40 sentences from held-out FLORES+ dev text, greedy, chrF++ word_order=2; macro over languages):

Language (tag) general dev chrF++: start (a=0) this model (a=0.5) diff a=1.0
Yoruba (yor) 29.34 30.72 +1.39 29.93
Hausa (hau) 50.38 51.44 +1.07 51.34
Igbo (ibo) 41.17 40.94 -0.23 40.86
Twi (twi) 38.14 38.33 +0.19 37.04
Ewe (ewe) 33.21 32.64 -0.57 30.39
Fon (fon) 21.73 20.94 -0.79 20.71
Wolof (wol) 34.80 34.13 -0.67 33.29
Swahili (swh) 59.31 58.02 -1.29 58.61
Kinyarwanda (kin) 44.81 44.38 -0.44 39.71
Somali (som) 45.39 45.39 +0.00 44.54
Oromo (orm) 35.64 34.67 -0.97 32.31
Zulu (zul) 55.92 55.59 -0.33 53.88
Xhosa (xho) 50.39 49.88 -0.51 50.04
Afrikaans (afr) 66.11 65.92 -0.19 65.91
Shona (sna) 42.12 41.00 -1.11 39.72
Chichewa (nya) 48.86 48.47 -0.38 47.23
Sesotho (sot) 45.27 46.06 +0.80 44.50
Setswana (tsn) 47.66 49.19 +1.54 47.88
macro (18) 43.90 43.76 -0.14 42.66

How measured. AfriScience-MT (Apache-2.0) dev: 843 sentences and 25 whole papers per language (sentences joined by single spaces, one prompt, max_new_tokens 7,168; sentence level max_new_tokens 512), greedy, sacreBLEU chrF (word_order=0) with the model's Translate the text to {language}: {english} prompt; run 2026-11-14 on one RTX PRO 6000. The general dev set has only 40 sentences per language, so per-language differences of a point or two are noise; the macro difference of 0.14 is about the size of the evaluation noise of such macros (standard deviation 0.09 to 0.10 in the project's 385M study). No contaminated-row marker exists for these dev sets. Source: reports/data/gemma1b_2026-11-14/eval/sci_sweep/alpha_{0.00,0.50,0.70,1.00}/stage_eval.json.

Selection rule (fixed beforehand): the largest alpha whose general dev macro stays within 0.5 chrF++ of the start. Alpha sweep (dev, macro): AfriScience sentence 51.07 / 53.52 / 54.32 / 54.42, whole paper 49.21 / 52.96 / 52.93 / 46.99 (finished 0.96 / 0.89 / 0.91 / 0.80), general dev 43.90 / 43.76 / 43.34 / 42.66 for alpha 0 / 0.5 / 0.7 / 1.0. Alpha 0.7 and 1.0 failed the rule (-0.56, -1.24).

Comparison with other systems

Not measured on a shared benchmark: published systems (fine-tuned NLLB, GPT-5.4, Gemma2-9B LoRA) are reported on the AfriScience-MT test split and NLLB / TranslatePsy / AfriqueLLM on FLORES+ devtest, NTREX and MAFAND, none of which was run for this SFT model. Within the family (dev, same split):

model AfriScience dev sentence chrF AfriScience dev whole-paper chrF (finished) general dev chrF++ (18 languages)
general RL model (start, a=0) 51.07 49.21 (0.96) 43.90
this model (SFT, a=0.5) 53.52 52.96 (0.89) 43.76
science RL final (gemma-3-1b-science-translate-best) 53.54 55.93 (0.96) not measured on this set

For comparison with NLLB and others on the science task and on the general benchmarks use the cards of gemma-3-1b-science-translate-best (test, 6 languages: 53.11 chrF; behind fine-tuned NLLB-1.3B and GPT-5.4 10-shot on the five languages other than Yoruba) and gemma-3-1b-translate-best. This SFT model is expected to sit between the general model and the science RL model; that was not verified on test.

Training summary

  1. Start: gemma-3-1b-translate-best (Gemma-3-1B-it, four general SFT shards, GRPO arm M1 best).
  2. Science SFT: 378 steps, AdamW 3e-6, 7,168-token packs (3,786 packs; whole papers train intact).
  3. Interpolation: alpha 0.5 between the start and the science-tuned weights (this repository).
  4. Next stage (not in this model): science RL, 128 steps.

Science SFT. AfriScience-MT train windows (Apache-2.0; single sentences, windows of 2 to 4 and 5 to 12 sentences, and whole papers, two passes; 30,761 windows, 6.65M tokens) packed into 7,168-token packs so that whole papers train intact (longest paper 6,616 tokens); 3.2M tokens of top-up pairs for Ibibio ibb (998 pairs, IbomMT), Herero her (4,397, FineTranslations, ODC-By) and Dagbani dag (717, OPUS Wikimedia, CC-BY-SA); 17.3M tokens of replay from SFT shard 5 (never trained on before, drawn from the general stage-B packs, so the general data caveats below apply to this replay). 378 steps of AdamW at 3e-6 starting from the general RL model (gemma-3-1b-translate-best, arm M1 best), then weight interpolation theta = (1 - alpha) theta_start + alpha theta_science with alpha = 0.5 (the largest alpha whose general dev macro stays within 0.5 chrF++ of the start: -0.14 / -0.56 / -1.24 for alpha 0.5 / 0.7 / 1.0).

Data disclosures

SFT data. The 1B SFT used shards 1 to 4 of the same decontaminated "stage B" packs as the 385M model (language-stratified, packed to 4,096 tokens; shards were checked for language homogeneity, total variation 0.2 to 0.3% against shard 1; the task-type mix per shard was not checked). The data statement of the 385M study applies in substance and its shares were measured for the 385M stage B, not for the four 1B shards: OPUS-NLLB mined bitext (43 languages, ODC-BY with special attribution; 13.3% of the executed 385M stage-B steps, 14.4% of pack tokens by another count), the NLLB-3.3B-derived Aya aligned part and NLLB-Seed Tamazight (17.2% of executed steps together with the mined data, 18.7% of tokens by the other count), Tumbuka and other machine-translated instruction sets with unstated systems (possibly NLLB; 22.8% of tokens with all of the above), MT560 (mostly Jehovah's Witnesses / Bible text), several further sources (listed in the disclosure file), religious text for many low-resource languages, and IbomMT. The FLORES+ dev "train half" (440 ids) is in the SFT on purpose; the held-out half, devtest, NTREX and MAFAND test text are not (benchmark sentences of 3 or more words were removed; short strings, numerals and names remain, see the clean-row counts).

Intended use and limitations

Use: the science SFT checkpoint, the starting point for science RL or further fine-tuning and for weight-interpolation experiments.

  • Science quality is measured for six languages on dev only; whole papers do not always finish (see above); no test-split, FLORES+ devtest, NTREX or MAFAND numbers.
  • English to African language only; 20 tags with no general SFT data (cannot translate); weak general languages as in the general card. Automatic metrics only; one seed; no human evaluation.
  • Machine translations of scientific text may alter numbers, units and terms.

Family map

Repository What it is When to use it
BeardedMonster/gemma-3-270m-translate-best General, about 270M parameters (Gemma-3-270M-it), 77 languages with text tags smallest and cheapest general model; CPU or edge use
BeardedMonster/gemma-3-385m-translate-best General, 385M parameters, 125 <iso> tags (FLORES+ 54-language chrF++ 31.66 on the project's paper protocol) middle size; weaker than the 1B model on the benchmarks below
BeardedMonster/gemma-3-1b-translate-best General, 1B, final RL model (arm M1 best) default choice for general English to African translation
BeardedMonster/gemma-3-1b-translate-sft-best General, 1B, SFT only (RL starting point) best SFT checkpoint, the starting point for further fine-tuning or RL
BeardedMonster/gemma-3-0.8b-translate-Q8 General 1B, Q8_0 GGUF (full vocabulary) llama.cpp on any GPU from a T4, CPU, Apple Silicon
BeardedMonster/gemma-3-270m-science-translate-best Science, about 270M science text on very small hardware
BeardedMonster/gemma-3-385m-science-translate-best Science, 385M science text, middle size
BeardedMonster/gemma-3-1b-science-translate-best Science, 1B, science SFT + science RL (final) scientific / academic text in amh hau lug nso yor zul and long inputs
BeardedMonster/gemma-3-1b-science-sft (this repo) Science, 1B, science SFT stage (interpolation alpha 0.5) before science RL science SFT checkpoint, the starting point for science RL or further fine-tuning
BeardedMonster/gemma-3-0.8b-science-translate-Q8 Science 1B, Q8_0 GGUF (full vocabulary) llama.cpp on any GPU from a T4, CPU, Apple Silicon
BeardedMonster/gemma-3-270m-science-translate-best-GGUF Science 270M, Q8_0 GGUF llama.cpp / CPU
BeardedMonster/gemma-3-1b-rl-translate-best Identical copy of gemma-3-1b-translate-best (byte-identical weights) either id loads the same model
BeardedMonster/gemma-3-1b-translate-rl Archive of the 1B RL arms (branches latest-m1, main, dpo-trial) kept for reproduction of the paper tables
AletheiaAING/gemma-3-270m-translate-rl, AletheiaAING/gemma-0.4B-translate-rl Archives of the 270M and 385M RL runs and ablations kept for reproduction of the paper tables

Provenance

All paths are relative to the project repository root (sabiyarn_MoE/). Numbers: reports/data/gemma1b_2026-11-14/eval/sci_sweep/alpha_{0.00,0.50,0.70,1.00}/stage_eval.json (per language, dev), reports/data/gemma1b_2026-11-14/eval/sci_final_afs/afriscience_eval.json (science RL final dev), reports/data/gemma1b_2026-11-14/clean_rerun/sci/runs/sci_sft/summary.json (378 steps), paper_385m/SECTION_1B_followup.md Section 17.4, paper_385m/DISCLOSURES.md, reports/data/gemma1b_2026-11-14/push_sci_sft.log (push of science_sft_chosen to this repository).

Citation and licence

Paper in preparation (placeholder, do not cite a venue yet):

@misc{sabiyarn_gemma_mt_2026,
  title  = {English-to-African-language translation with small Gemma-3 models: supervised fine-tuning and reinforcement learning},
  author = {{authors to be added}},
  year   = {2026},
  note   = {in preparation; BeardedMonster/gemma-3-1b-science-sft}
}

Licence. license: other (sabiyarn-commercial-use-with-attribution); the full text is in LICENSE.md. Commercial and non-commercial use, modification and redistribution are allowed, with required attribution: credit "Translation models by Aletheia AI" and cite the accompanying paper (in preparation). These weights are a derivative of Google Gemma 3, so the Gemma Terms of Use and Prohibited Use Policy also apply. The training data sources are listed under "Data disclosures".

Downloads last month
22
Safetensors
Model size
1.0B params
Tensor type
F32
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for BeardedMonster/gemma-3-1b-science-sft

Finetuned
(1)
this model
Finetunes
1 model