| base_model: google/gemma-4-E4B-it | |
| library_name: peft | |
| license: gemma | |
| tags: | |
| - lora | |
| - peft | |
| - sinhala | |
| - mmlu | |
| # Team Sub4 - Sinhala MMLU Shared Task (ICTer 2026): LoRA adapter | |
| LoRA adapter (rank 32, alpha 64) for `google/gemma-4-E4B-it`, fine-tuned on Sinhala school-curriculum multiple-choice questions. | |
| It adapts the attention and MLP projections of the language model only (258 layers, 69.8M trainable parameters). | |
| - **Base model:** `google/gemma-4-E4B-it`, loaded text-only (audio and vision towers removed). | |
| - **Inference parameters:** 7.46B text backbone + 0.07B adapter = 7.53B (limit 8B). | |
| - **Use:** see `inference.py` in the team repository `SinhalaMMLU-SharedTask/team-Sub4`. One forward pass per question, a fixed 4-example prompt, the answer is the highest of the four digit logits ("1".."4"). | |
| - **Training:** see `training/` and `README.md` in the team repository. Training data: Sinhala Global MMLU, school past-paper questions, and questions generated from textbook passages with Gemini 3.8 Flash. | |
| - **Stage:** this adapter is the state after continuation stage 2 (stage 1: rows unseen by the first run; stage 2: all extracted and generated questions with shuffled answer positions). | |
| ```python | |
| from peft import PeftModel | |
| model = PeftModel.from_pretrained(base_model, "SinhalaMMLU-SharedTask/team-Sub4-gemma4-e4b-lora") | |
| ``` | |