Instructions to use Bennerdo/rieko with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Bennerdo/rieko with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("google/gemma-3-1b-pt")
model = PeftModel.from_pretrained(base_model, "Bennerdo/rieko")

Transformers

How to use Bennerdo/rieko with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Bennerdo/rieko")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("Bennerdo/rieko", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Bennerdo/rieko with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Bennerdo/rieko"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Bennerdo/rieko",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Bennerdo/rieko

SGLang

How to use Bennerdo/rieko with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Bennerdo/rieko" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Bennerdo/rieko",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Bennerdo/rieko" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Bennerdo/rieko",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Bennerdo/rieko with Docker Model Runner:
```
docker model run hf.co/Bennerdo/rieko
```

RiekoLM: Gemma 3 1B for Dholuo

RiekoLM ("Rieko") is a LoRA adapter for google/gemma-3-1b-pt. It makes the model a small assistant that speaks Dholuo (Luo of Kenya). It answers questions, translates between Dholuo, English and Swahili, and handles simple conversation in Dholuo.

Model Details

Model Description

The adapter was trained in two stages with QLoRA:

Continued pretraining (CPT) on Dholuo text, to improve the base model's knowledge of the language.
Supervised fine-tuning (SFT) on a rebalanced mix of instructions, chat and translation examples (SFT v2).

Developed by: Benard Otieno (Bennerdo)
Funded by: Benard Otieno
Model type: Small causal language model (LoRA adapter)
Language(s) (NLP): Dholuo (luo); English and Swahili for translation
License: Gemma Terms of Use (inherited from the base model)
Finetuned from model: google/gemma-3-1b-pt

Model Sources

Repository: https://huggingface.co/bennerdo/rieko
Base Model: google/gemma-3-1b-pt

Uses

Direct Use

Answering questions and having short conversations in Dholuo.
Translating between English and Dholuo, and between Swahili and Dholuo.
Drafting and rewriting simple Dholuo text.
Research on NLP for low-resource African languages.

Downstream Use

A starting point for further fine-tuning on Dholuo tasks such as education, health information or customer support.
A component in Dholuo chat or translation applications, with human review of the output.

Out-of-Scope Use

Medical, legal or financial advice, or other high-stakes decisions.
Treating the output as an authoritative reference for Dholuo grammar or spelling.
Factual questions where an error causes harm. A 1B model often invents facts.
Generating content that impersonates people, or deceptive or harmful content.

Bias, Risks, and Limitations

Small model: at 1B parameters, the model has limited reasoning ability and makes factual errors and hallucinations.
Synthetic data: part of the SFT data was generated by Gemini, and the identity and greeting examples were written from templates. A native Dholuo speaker (the author) reviewed this data and found almost all of it correct. A few unnatural phrasings or spelling errors may remain.
Domain skew: the translation data is weighted towards religious (Bible) and OPUS text. Caps reduce this skew but do not remove it.
Dialect: the training data favours Kenyan Nyanza Dholuo. The model is not tuned to other Luo varieties.
Code-switching: the model sometimes mixes in English or Swahili.

Recommendations

Have a native speaker review the output before using it anywhere it matters. Don't rely on the model for facts without checking them.

How to Get Started with the Model

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base, adapter = "google/gemma-3-1b-pt", "bennerdo/rieko"
tok = AutoTokenizer.from_pretrained(adapter)
model = AutoModelForCausalLM.from_pretrained(base, dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(model, adapter).eval()

msgs = [{"role": "user", "content": "An iluonga ni Akinyi, in iluongi ni ng'a?"}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(ids, max_new_tokens=200, do_sample=True, temperature=0.3, top_p=0.9, top_k=40,
                     repetition_penalty=1.1, no_repeat_ngram_size=4,
                     eos_token_id=[tok.eos_token_id, tok.convert_tokens_to_ids("<end_of_turn>")])
print(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True))

Training Details

Training Data

Stage 1 (CPT): 338,912 decontaminated training rows (~5,404,734 words) and 2,000 validation rows. Sources include:
- Parallel text extracts: OPUS parquet (134,108 rows kept), OPUS text (61,235), Bible (30,558), MAFAND (4,248 cpt + 113 text)
- Monolingual corpora: Speech transcripts (84,269), ThinkKenya news/web (19,421), CLEAR synthetic text (5,000)
- Cultural & grammatical texts: Wikipedia Luo (235), Lwanda Magere folklore (38), dictionary definitions (7)
- English replay buffer: Teacher-generated English monolingual text (1,680 rows) to limit catastrophic forgetting
Stage 2 (SFT):
- SFT v1 (Trained & Evaluated): 166,650 rows (OPUS parallel 80k, Swahili–Luo 44.8k, Bible 30k, MAFAND 8.5k, Gemini instructions 925, dictionary 984, grammar 266, Lwanda Magere 104, English replay 1,118).
- SFT v2 (Rebalanced mix): 52,398 rows with aggressive caps on translation sources (OPUS 20k, Bible 8k, Swahili–Luo 8k) and upsampled scarce conversational/cultural sources:
  - Gemini instructions: 925 unique pairs repeated ×5 (4,625 rows)
  - Identity & greeting templates: 68 unique pairs repeated ×5 (340 rows)
  - Multi-turn Gemini chat (gemini_chat_sft)
  - Grammar: 266 pairs repeated ×2 (532 rows)
  - Lwanda Magere: 104 pairs repeated ×3 (312 rows)
  - Dictionary: 984 rows
  - MAFAND-MT: 8,474 rows
  - English replay: 1,118 rows

Training Procedure

Preprocessing

Every source went through the same steps:

Decontamination: remove any row that matches a held-out evaluation item exactly or shares an 8-gram with one (checking against FLORES-200 dev/devtest, Belebele, MAFAND-test, and Swahili-Luo-test).
Global Deduplication on normalized MD5 hashes across all sources.
Alignment filter for translation pairs: drop pairs whose length ratio exceeds 2.5, and pairs where the source and target are identical.
Caps on high-volume translation sources to avoid religious-domain overfitting.
Upsampling of instruction, persona, grammar, and cultural folklore data.

Training Hyperparameters

Hardware & Frameworks: Single NVIDIA RTX 6000 Blackwell GPU (48 GB VRAM); PyTorch, Transformers, PEFT 0.21.2, TRL (SFTTrainer), bitsandbytes 0.45+.
Precision: bfloat16 mixed precision, 4-bit NormalFloat (NF4) base quantization with double quantization (QLoRA).
LoRA Configuration:
- Rank ($r$): 64
- Alpha ($\alpha$): 128 (scaling factor = 2.0)
- Dropout: 0.05
- Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj (all 18 layers)
- Trainable parameters: 52,183,040 / 1,052,068,992 (4.96%)
Stage 1 (CPT):
- Objective: Causal language modeling with EOS token appended between packed documents
- Max sequence length: 1,024 tokens
- Batch size: 16 per device (effective batch size: 16, gradient accumulation: 1)
- Peak learning rate: $1\times 10^{-4}$ with cosine decay schedule and 5% warmup
- Optimizer: paged_adamw_8bit
- Epochs: 2.0 (1,358 update steps)
- Runtime: 1h 01m 16s (3,675.92 seconds) at 5.91 samples/s (0.37 steps/s)
- Compute: $1.001\times 10^{17}$ FLOS
- Metrics: Train loss: 3.0122 (started at 6.607); Validation loss: 2.6729; Validation Perplexity: 14.4819
Stage 2 (SFT v1):
- Starting weights: Stage 1 CPT final adapter (outputs/gemma_dholuo_cpt/final_adapter)
- Objective: Instruction-tuning with TRL SFTTrainer and completion_only_loss=True (loss computed exclusively on model response tokens)
- Chat template: Gemma format (<start_of_turn>user\n...<end_of_turn>\n<start_of_turn>model\n...<end_of_turn>)
- Max sequence length: 1,024 tokens
- Batch size: 16 per device (effective batch size: 16, gradient accumulation: 1)
- Peak learning rate: $2\times 10^{-4}$ with cosine decay schedule and 5% warmup
- Optimizer: paged_adamw_8bit
- Epochs: 3.0 (31,248 update steps on 166,650 rows)
- Runtime: 3h 28m 17s (12,497.10 seconds) at 40.01 samples/s (2.50 steps/s)
- Compute: $3.547\times 10^{17}$ FLOS
- Metrics: Train loss: 1.4136 (final step loss: 0.8868, train token accuracy: 78.29%); Validation loss on FLORES dev: 2.281 (eval token accuracy: 57.84%, eval entropy: 1.368)

Evaluation

Testing Data, Factors & Metrics

Testing Data

FLORES-200 devtest, in both directions: English to Dholuo (eng_Latn → luo_Latn) and Dholuo to English.
Belebele, Dholuo reading comprehension (luo_Latn).

All of this data was held out from training (see Preprocessing).

Metrics

Translation: chrF++ and BLEU (sacrebleu). chrF++ is the main metric, because it handles Dholuo's rich morphology better than BLEU.
Belebele: multiple-choice accuracy, scored by picking the answer option the model rates most likely.

Results

FLORES-200 devtest (n = 1,012). Belebele chance level is 0.25.

Model	eng→luo chrF++ / BLEU	luo→eng chrF++ / BLEU	Belebele acc
gemma-3-1b-pt (base)	10.27 / 0.46	14.55 / 1.82	0.2267
RiekoLM (SFT v1)	34.52 / 9.01	36.13 / 12.72	0.2278

Translation improves sharply: chrF++ rises by about 24 points from English to Dholuo and about 22 points from Dholuo to English. Belebele accuracy stays at chance level, so the adapter does not yet improve reading comprehension.

The two models are prompted differently. The base model has no chat template, so it gets plain completion prompts. The adapter gets the Gemma chat format. Keep this in mind when comparing the rows.

Environmental Impact

Hardware Type: NVIDIA RTX 6000 (Blackwell architecture, 48 GB VRAM)
Hours used: ~4.85 hours (CPT: 1.02h, SFT: 3.47h, Eval & Chat Synthesis: ~0.35h)
Cloud Provider: molab (marimo reactive notebook platform)
Compute Region: US Cloud Instance
Total Compute FLOS: ~4.55e+17 FLOS (CPT: 1.00e+17 FLOS, SFT: 3.55e+17 FLOS)
Carbon Emitted: Estimated ~0.55 kg CO2eq (based on ~300W average GPU load over 4.85 hours ≈ 1.455 kWh at 0.38 kg CO2/kWh grid emission factor)

Technical Specifications

Model Architecture and Objective

Gemma 3 decoder-only transformer, 1B parameters, 18 layers, 256k vocabulary, with LoRA adapters (52.18M trainable parameters, 4.96% of total).
Objective: Causal language modeling during CPT. During SFT, cross-entropy loss is computed exclusively on assistant response tokens (completion_only_loss=True).

Compute Infrastructure

Hardware: 1× NVIDIA RTX 6000 Blackwell GPU (96 GB VRAM)
Software: Linux, PyTorch 2.6+ (CUDA 12.8), Transformers >= 4.50, PEFT 0.21.2, TRL (SFTTrainer), bitsandbytes >= 0.45, sacrebleu

Citation

@misc{otieno2026riekolm,
  title  = {RiekoLM: A Small Dholuo Language Model},
  author = {Otieno, Benard},
  year   = {2026},
  url    = {https://huggingface.co/bennerdo/rieko},
  note   = {LoRA adapter for google/gemma-3-1b-pt}
}

Model Card Authors

Benard Otieno (Bennerdo)

Model Card Contact

Hugging Face Profile: https://huggingface.co/bennerdo
Model Repository: https://huggingface.co/bennerdo/rieko

Framework versions

PEFT 0.21.2
Transformers >= 4.50.0
TRL
PyTorch (CUDA 12.8)

Downloads last month: 23

Model tree for Bennerdo/rieko

Base model

google/gemma-3-1b-pt

Adapter

(63)

this model

Bennerdo
/

rieko