| # Team Nena – Sinhala MMLU Shared Task System |
|
|
| ## System summary |
| - **Base model(s):** `SAWithanage/SinLlama-Llama-3-8B-Merged` |
| - **What we did:** We performed parameter-efficient fine-tuning (LoRA) of SinLlama-8B on authentic Sri Lankan national curriculum examination Multiple-Choice Questions (spanning Grades 6–13 across 14 official subjects) alongside domain-adaptive curriculum pretraining passages. For inference, we employ a Sinhala Micro-Chain-of-Thought (Micro-CoT) reasoning prompt with deterministic greedy decoding to ground curriculum facts before emitting the final anchored choice. |
| - **Total inference parameters:** **7.954B parameters** (Language-based vocabulary trimming (`Lang-Trim`) slices out 20,336 inactive foreign script tokens while preserving 100% of Sinhala, English, numbers, and symbols, safely conforming to the Hard Rule 02 parameter ceiling of 8.00B as officially approved by the shared task chairs). |
| - **Extra data used for training or retrieval:** Curated Sri Lankan national curriculum examination MCQs and government textbook passages across Grades 6–13. Strictly retrieval-free at inference time (zero external APIs, web search, or RAG). |
|
|
| ## Models and resources in this submission |
| | Repo | What it is | |
| |---|---| |
| | `SinhalaMMLU-SharedTask/team-nena` | Code, inference script, final predictions, and training scripts (this repo) | |
| | `SinhalaMMLU-SharedTask/team-nena-sinllama-8b-lora` | Fine-tuned LoRA adapter weights and tokenizer configuration | |
|
|
| ## How to run |
| ```bash |
| pip install -r requirements.txt |
| python inference.py |
| ``` |
| - **GPU used:** 1x Nvidia A100 (80GB / 40GB) or 1x T4 / RTX 3090 / 4090 (with 4-bit NF4) |
| - **Approximate run time on the test set:** ~80–90 minutes for 2,238 test questions |
| - **Anything else the organizers need to know:** Pure offline inference, zero closed or paid APIs. |
|
|
| ## Files |
| - `inference.py` – the exact script that produced our final Codabench submission |
| - `predictions.jsonl` – the exact file we submitted to the final leaderboard (2,238 predictions) |
| - `requirements.txt` – package versions used |
| - `training/` – training, fine-tuning, and data-preparation code and configurations |
|
|
