Luganda Gemma 1B IT

A fine-tuned version of Google Gemma 3 1B Instruct for English ↔ Luganda translation and Luganda conversational AI.

Highlights

  • BLEU 13.85 on English→Luganda translation, up from 0.06 on the base model
  • chrF++ 46.59 — strong character-level accuracy for morphologically rich Luganda
  • Trained with QLoRA (4-bit quantization + LoRA adapters) — runs on consumer GPUs
  • Only 52 MB adapter on top of the 1B base model

Results

Model BLEU chrF++
Gemma 3 1B base (no fine-tuning) 0.06 8.84
This model 13.85 46.59

Evaluated on 100 held-out examples from the test split described under Training details (SALT and Kimera corpus sentences), mixing English→Luganda and Luganda→English prompts (see eval_results.json). This is not a FLORES evaluation, so these scores are not directly comparable to our FLORES-evaluated Swahili models.

Usage

Quick start

from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel
import torch

base_model_id = "google/gemma-3-1b-it"
adapter_id = "AmplifiedAccess/Luganda-gemma-1b-it"

# Load with same quantization used during training
tokenizer = AutoTokenizer.from_pretrained(base_model_id)

model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    quantization_config=BitsAndBytesConfig(
        load_in_4bit=True,
        bnb_4bit_quant_type="nf4",
        bnb_4bit_compute_dtype=torch.bfloat16,
        bnb_4bit_use_double_quant=True,
    ),
    device_map={"": 0},
)

model = PeftModel.from_pretrained(model, adapter_id)
model.eval()

Translation (English → Luganda)

prompt = "Translate to Luganda:\nThe farmers need better seeds to improve their harvest."

messages = [{"role": "user", "content": prompt}]
inputs = tokenizer.apply_chat_template(
    messages, return_tensors="pt", return_dict=True, add_generation_prompt=True
).to(model.device)

with torch.no_grad():
    outputs = model.generate(**inputs, max_new_tokens=128, do_sample=False)

prompt_len = inputs["input_ids"].shape[1]
response = tokenizer.decode(outputs[0][prompt_len:], skip_special_tokens=True).strip()
print(response)

Supported prompt formats

The model was trained on varied prompt templates. All of these work:

English → Luganda:

  • Translate to Luganda:\n{text}
  • Convert this to Luganda:\n{text}
  • English to Luganda:\n{text}
  • How do you say this in Luganda?\n{text}

Luganda → English:

  • Translate to English:\n{text}
  • Convert this to English:\n{text}
  • Luganda to English:\n{text}
  • What does this mean in English?\n{text}

Conversational (respond in Luganda):

  • Respond in Luganda: {text}
  • Answer in Luganda:\n{text}
  • Yogera mu Luganda: {text}

Example translations

English Model output
Five prisoners were released yesterday. Abasibe bataano baateereddwa ku kkomera jjo.
The government take long to pay doctors in public hospitals Gavumenti etwala obudde bungi okusasula abasawo mu malwaliro ga gavumenti.
The professor is the presidential advisor. Omusomesa ye munnamateeka wa pulezidenti.
My father inspired me to become a surgeon. Taata wange yamukubiriza okufuuka omusawo.
How much money did they share amongst themselves? Baawaddeyo ssente mmeka?

Training details

Data

  • 94,542 training examples derived from 42,019 clean English-Luganda parallel sentence pairs:
  • Three task types: English→Luganda translation, Luganda→English translation, and Luganda conversation
  • Split: 90% train / 5% validation / 5% test

Configuration

Parameter Value
Base model google/gemma-3-1b-it
Method QLoRA (4-bit NF4 + LoRA)
LoRA rank 16
LoRA alpha 32
LoRA dropout 0.05
Target modules q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Trainable parameters 13.05M (2.0% of total)
Epochs 1
Effective batch size 16 (8 × 2 gradient accumulation)
Learning rate 2e-4 (cosine schedule)
Max sequence length 256
Precision bf16
Optimizer paged AdamW 8-bit
Training time ~8 hours on Tesla T4
Final training loss 1.55
Final validation loss 1.26

Training loss curve

Step Train loss Val loss
500 2.07 2.01
1000 1.74 1.73
2000 1.49 1.48
3000 1.38 1.35
4000 1.29 1.28
5000 1.24 1.26
5500 1.29 1.26

Important notes

  • Use 4-bit quantization for inference — the LoRA weights were trained against 4-bit base weights. Loading in 8-bit or full precision will produce degraded translations.
  • Use do_sample=False for deterministic, highest-quality translations.
  • Use the base model tokenizer (google/gemma-3-1b-it) for best results.
  • The model was trained on 1 epoch. Further training (3+ epochs) would likely improve scores further.

Training data and licenses

Dataset License
SALT (Sunbird AI) CC BY-SA 4.0
Kimera English–Luganda Parallel Corpus (Richard Kimera et al.) CC0 1.0

Kimera corpus. The Kimera English–Luganda Parallel Corpus is released under CC0 1.0 and builds on public data from Zenodo and Sunbird AI. Of its 41,070 pairs, 18,076 remained after removing duplicates. Reference: Kimera, R., Rim, D.N. and Choi, H., Building a Parallel Corpus and Training Translation Models Between Luganda and English, KIISE 49(11), 2022.

SALT attribution. SALT was created by Sunbird AI with the Makerere University AI Lab and KenCorpus (Maseno University). Reference: Akera et al., Machine Translation For African Languages: Community Creation Of Datasets And Models In Uganda, 3rd Workshop on African Natural Language Processing, 2022.

Share-alike data. SALT is licensed CC BY-SA 4.0. We treat the trained model weights as a separate work, not an adaptation of the SALT text, so SALT's share-alike condition does not set this model's license. We are seeking written confirmation of this reading from Sunbird AI and will update this section with their answer.

SALT excerpts in this repo. eval_results.json and the example translations on this card quote sentences from SALT. Those excerpts remain under CC BY-SA 4.0, not under this model's license.

Limitations

  • Trained on Sunbird SALT data, which covers agriculture, health, and society topics, and the Kimera corpus. Performance may be weaker on highly specialized domains (legal, medical, technical).
  • Luganda→English translation quality may lag behind English→Luganda since the base model already has strong English capabilities.
  • Short sentences (under 5 words) may produce less accurate translations.
  • The model may occasionally produce Luganda synonyms or dialectal variants that differ from a specific reference translation while still being semantically correct.

Intended use

  • English ↔ Luganda machine translation
  • Luganda conversational AI and chatbot applications
  • Educational tools for Luganda language learning
  • Research on low-resource African language NLP

Acknowledgments

Framework versions

  • PEFT 0.18.1
  • Transformers 4.x
  • TRL 0.x
  • PyTorch 2.x
  • bitsandbytes 0.45.x

License

This adapter is a Model Derivative of Gemma 3 1B IT and is distributed under the Gemma Terms of Use. Gemma is provided under and subject to the Gemma Terms of Use found at ai.google.dev/gemma/terms.

Use of this model must also comply with the Gemma Prohibited Use Policy.

Citation

If you use this model, please cite:

@misc{luganda-gemma-1b-2026,
  title={Luganda Gemma 1B IT: Fine-tuned Gemma 3 1B for English-Luganda Translation},
  author={Amplified Access},
  year={2026},
  publisher={HuggingFace},
  url={https://huggingface.co/AmplifiedAccess/Luganda-gemma-1b-it}
}
Downloads last month
42
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for AmplifiedAccess/Luganda-gemma-1b-it

Adapter
(229)
this model

Dataset used to train AmplifiedAccess/Luganda-gemma-1b-it

Evaluation results