REFERENCES.md

References & Data Provenance

Arabic version: REFERENCES_AR.md

This file documents what the project actually uses. Where the supplied materials do not record a fact (for example the upstream origin or licence of a data file), that is stated instead of guessed.

Project Specification

IslamicAIChallengePresentation.pdf: the project proposal for the challenge تحدي الذكاء الاصطناعي في خدمة المحتوى الإسلامي 2026 (12 slides, Arabic). It defines the workflow (detect, retrieve, verify with evidence, correct or refer to human review), the safety principles (source first, traceable evidence, abstain when unsure) and the expected deliverables. It also reports the previous IslamicEval 2025 research baselines (1A macro F1 0.9091 with CAMeLBERT-MSA; 1B accuracy 93.12%; 1C accuracy 70.39%). Those are historical figures, not results of this application.

IslamicEval 2025

• Official source: Mubarak, H. et al. (2025). IslamicEval 2025: The First Shared Task of Capturing LLMs Hallucination in Islamic Content. Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks, pp. 480–493, Suzhou, China. ACL Anthology ID 2025.arabicnlp-sharedtasks.67, DOI 10.18653/v1/2025.arabicnlp-sharedtasks.67. The task has two subtasks; this project uses only Subtask 1 (hallucination detection and correction of quoted ayahs and Hadith). Subtask 2 (question answering) is not used.
• Subtasks represented: 1A (quotation span detection), 1B (verification: Correct / Incorrect), 1C (correction of incorrect quotations; خطأ marks "no source").
• Files actually used: data/islamiceval_dev_subset.jsonl, 150 real development responses (50 each for 1A, 1B, 1C), with 194 / 247 / 179 annotated spans. It was taken from the project's unified dataset file (islamic_unified_dataset.jsonl), which the research notebook builds from the development files dev_SubtaskA/B/C (.xml + .tsv). Only the records with source = real_A, real_B, real_C were kept. The 1,500 synthetic records of the unified file are not in this evaluation subset: they are training data generated by the notebook from the Quran / Hadith corpora, not part of IslamicEval.
• How it is used: evaluate.py reads the subset for the prototype evaluation; the optional training script reads the unified file. The running application reads neither.
• Not included: the rest of the IslamicEval 2025 data (including training and test data).

Dataset Provenance

File	Content	What is verifiable
data/quran.json	6,236 ayahs, 114 surahs; fields surah_id, surah_name, ayah_id, ayah_text (with diacritics)	Supplied by the project team as quranic_verses.json. Used unchanged.
data/hadith.json	34,994 records; fields hadithID, BookID (1–6), title, hadithTxt, Matn (present in 31,811 records)	Supplied by the project team as six_hadith_books.json; only whitespace outside strings was removed (JSON re-serialised compactly), content unchanged.
data/islamiceval_dev_subset.jsonl	See above	Derived from the project's unified dataset file.
data/islamic_unified_dataset.jsonl	1,650 records: 150 real IslamicEval development responses + 1,500 synthetic ones generated by the research notebook	Supplied by the project team; used only by the optional research/train_detector.py.
index/quran.idx.gz, index/hadith.idx.gz	Pre-tokenised search indexes derived from data/quran.json and data/hadith.json by index_builder.py	Generated; rebuild with python index_builder.py.

The research notebook uses quranic_verses.json and six_hadith_books.json as the reference database of its Subtask 1C scoring routine (copied there from the organisers' scoring.py). The supplied materials do not record where these two files were originally obtained or under which licence. The team should record the upstream source and licence here before redistributing the repository publicly.

Research References

• Mubarak et al. (2025), cited above. This is the only paper the project relies on directly (task definition and subtask structure).

Models

• The application uses no machine-learning model and loads no model weights. Detection is rule-based plus a corpus scan; retrieval is BM25 over pre-built indexes; verification is word / character alignment.
• research/train_detector.py is optional research code (fine-tuning aubmindlab/bert-base-arabertv2 or CAMeLBERT-MSA for Subtask 1A). It is not used by the application and no trained weights exist in this repository.
• Mode B calls a third-party language model chosen by the user (Google Gemini generateContent, OpenAI chat completions, or the Hugging Face router) with the user's own key. Default model names (gemini-2.5-flash, gpt-4o-mini, Qwen/Qwen2.5-72B-Instruct) are editable placeholders; only the Gemini name was checked against the provider's documentation. Provider request formats were tested against a local mock server, not against the live services.
• The 1A baseline of 0.9091 in the proposal comes from earlier work with CAMeLBERT-MSA, not from this code.

Libraries

Library	Purpose	Source	Version	Licence
Python standard library (difflib, re, unicodedata, json, …)	Matching, normalisation, data loading	https://www.python.org	Developed and tested with 3.12.3	Python Software Foundation License
Gradio	Web interface (local / server runs only)	https://github.com/gradio-app/gradio	>=5.0,<6; not installed in the authoring environment (see below)	Apache-2.0 (as published by the project)
RapidFuzz	Optional faster Levenshtein similarity (pure-Python fallback exists)	https://github.com/rapidfuzz/RapidFuzz	>=3.0; not installed in the authoring environment	MIT (as published by the project)
NumPy	Character-level F1 in evaluate.py	https://numpy.org	2.4.4 (tested)	BSD-3-Clause (as published by the project)
Pyodide	Runs the Python code inside a browser Web Worker for the Static Space demo (loaded from the jsDelivr CDN)	https://pyodide.org	0.26.4	MPL-2.0 (as published by the project)

Licences above are those declared by the respective projects; confirm them at the source before relying on them.

Deployment

• GitHub for the source code.
• Hugging Face Static Space for the public demo: index.html starts a Web Worker that loads Pyodide 0.26.4 from the jsDelivr CDN and runs the project's Python modules in the visitor's browser (build_static_space.py assembles the folder). Gradio is used only for local runs, because Gradio and Docker Spaces require a paid Hugging Face plan.
• No deployment has been performed from this repository yet, so no public URL exists in the documentation.

Licensing & Attribution

• Source code: MIT (see LICENSE), applied to the code only.
• Data files in data/ are third-party resources and are not covered by that licence. Their licences are not recorded in the supplied materials (see Dataset Provenance).
• IslamicEval 2025 data belongs to the shared-task organisers; follow their terms and cite the overview paper.

Reproducibility

• Python: 3.10 or newer; developed and tested with 3.12.3.
• Requirements: requirements.txt. The core logic was tested with the Python standard library and NumPy only. The Gradio interface, RapidFuzz and Pyodide could not be installed or downloaded in the authoring environment (no network access). The rendering functions were tested natively, and the static page was exercised in a browser with a stubbed Pyodide; the real Gradio server and the real Pyodide runtime were not run there.
• Data: the three files in data/, as described above.
• Preprocessing: Arabic normalisation in retrieval.py (diacritic and tatweel removal, alef unification, punctuation stripping, and for matching ta marbuta / alef maqsura folding); the index is built in memory at start-up.
• Execution: python index_builder.py (only after changing data/); python app.py; python evaluate.py; python -m unittest discover -s tests; python build_static_space.py for the browser demo.

Competition and benchmark links (not verified automatically)

The following were provided with the task and are cited as references. Automated access to the GitHub repository was blocked (robots rules) and the organiser pages and PDF were not read while preparing this version, so no rule, deadline or file format is quoted from them here:

• IslamicEval 2025 task site: https://sites.google.com/view/islamiceval-2025/home
• Subtask 1 code repository: https://github.com/qcri/IslamicEval-2025-Subtask-1
• Challenge site and timeline: https://islamicaich.org/
• Hackathon rules (PDF): https://islamicaich.org/files/HackathonFile/da2OrbRMEsNrIQaoq8z6jEv26opKXdQmq1i0XfXo.pdf

Model references

• CAMeLBERT-MSA (CAMeL Lab, CAMeL-Lab/bert-base-arabic-camelbert-msa) is the base model for the optional fine-tuned token classifier described in the README. The browser page runs it as a simulation unless HF_MODEL_ID is configured.
• "Ask then Verify" uses the OpenAI Chat Completions API (gpt-4o-mini by default, set in web/config.js).