Note Cultural MCQs for 6 countries (QCRI). 1 of 180 removed. No answer column: Option A is always correct.

Note Commonsense cultural reasoning across 13 Arab countries (MBZUAI). 73 of 3,482 removed.

Note Arabic GSM8K math word problems. 13 of 1,319 removed from main_test; the other splits were not audited.

Note School and university exam questions in 40 subjects (MBZUAI). 765 of 14,455 test questions removed.

Note MMLU in 5 Arabic dialects plus MSA and English (MBZUAI). 935 of 15,675 dialect rows removed; MSA and English were not audited.

Note Arabic medical MCQs in 26 subject categories. ID filter to apply to QIMMA's public copy: 896 of 4,927 removed (18.2%, the highest rate in the collection).

Note Arabic MMLU-ProX: 14 subjects, up to 10 options per question. 68 of 11,759 removed.

Note Emirati dialect benchmark (TII). The source states no license, so this is an ID filter to apply to the public source: 15 of 1,173 removed.

Note HELM Arabic finance and legal QA (Stanford CRFM). 19 of 299 finance and 11 of 200 legal removed; content_generation was not audited.

Note Native Arabic STEM MCQs. The source states no license, so this is an ID filter to apply to the public source: 330 of 10,824 removed.

Note Fann or Flop Arabic poetry MCQs, 14 tasks. 230 of 6,984 removed.

Note Math questions from the Saudi General Aptitude Test (GAT). 199 of 7,017 removed.

Note IslamicEval 2025 Subtask 1B (dev): judging whether quoted Qur'an and Hadith spans are correct. 16 of 247 removed.

Note PalmX 2025 Islamic culture MCQs. The source is gated, so this is an ID filter to apply to your own copy: 37 of 985 removed.

Note QIAS 2025 MCQs on Islamic inheritance (task 1) and general Islamic knowledge (task 2). ID filter to apply to the public source: 113 of 1,700 removed.