Sinhala
Files changed (1) hide show
  1. README.md +21 -5
README.md CHANGED
@@ -14,6 +14,10 @@ language:
14
 
15
  This work was carried out for the **ICTER Sinhala MMLU Shared Task 2026**, conducted by **UCSC and IIT, Sri Lanka**.
16
 
 
 
 
 
17
  ## Models and resources in this submission
18
 
19
  | Repository | What it is |
@@ -90,6 +94,18 @@ Because easy-level MCQs were underrepresented compared with medium- and hard-lev
90
  | Filtered dataset for the first three MCQ epochs | 46,373 |
91
  | Expanded, filtered dataset for the fourth MCQ epoch | 47,025 |
92
 
 
 
 
 
 
 
 
 
 
 
 
 
93
  ## Training methodology
94
 
95
  ### Stage 1: Sinhala syllabus adaptation using synthetic Q&A
@@ -273,11 +289,11 @@ We used **Google Colab with an NVIDIA A100 80 GB GPU** for training and inferenc
273
 
274
  | Evaluation metric | Result |
275
  |---|---|
276
- | Overall score | — |
277
- | Leaderboard rank | — |
278
- | Easy-level score | — |
279
- | Medium-level score | — |
280
- | Hard-level score | — |
281
 
282
  ## Team members
283
 
 
14
 
15
  This work was carried out for the **ICTER Sinhala MMLU Shared Task 2026**, conducted by **UCSC and IIT, Sri Lanka**.
16
 
17
+ ## System architecture
18
+
19
+ ![Team Cognilex Sinhala MMLU training and inference architecture](./architecture.jpeg)
20
+
21
  ## Models and resources in this submission
22
 
23
  | Repository | What it is |
 
94
  | Filtered dataset for the first three MCQ epochs | 46,373 |
95
  | Expanded, filtered dataset for the fourth MCQ epoch | 47,025 |
96
 
97
+ #### Subject distribution
98
+
99
+ Humanities (38.8%) and STEM (29.5%) account for the largest shares of the MCQ dataset.
100
+
101
+ ![MCQ dataset distribution by subject](./subject-distribution.png)
102
+
103
+ #### Difficulty distribution
104
+
105
+ Medium questions comprise 52.9% of the MCQ dataset, followed by hard (32.8%) and easy (14.3%) questions.
106
+
107
+ ![MCQ dataset distribution by difficulty](./difficulty-distribution.png)
108
+
109
  ## Training methodology
110
 
111
  ### Stage 1: Sinhala syllabus adaptation using synthetic Q&A
 
289
 
290
  | Evaluation metric | Result |
291
  |---|---|
292
+ | Overall score | 68.55 |
293
+ | Easy-level score | 61.21 |
294
+ | Medium-level score | 67.96 |
295
+ | Hard-level score | 76.49 |
296
+
297
 
298
  ## Team members
299