gemma2-9b-CS-AR

gemma-2-9b-it fine-tuned for Arabic commonsense reasoning (ArabicSense)

Paper GitHub Encoders Collection

A LoRA adapter for gemma-2-9b-it trained on all three ArabicSense tasks: deciding which of two Arabic statements does not make sense, choosing the reason it does not make sense, and writing that reason.

Part of ArabicSense, a benchmark for commonsense reasoning in Arabic, from the paper ArabicSense: A Benchmark for Evaluating Commonsense Reasoning in Arabic with Large Language Models (WACL-4 2025). Code: KamyarZeinalipour/ArabicSense; encoder experiments: EL-Amrany/Arabic-Commonsense-Reasoning.

Results on the ArabicSense test set

Task Metric Zero-shot Fine-tuned
A: validation Accuracy / F1 0.869 / 0.867 0.947 / 0.947
B: explanation selection Accuracy / F1 0.921 / 0.921 0.944 / 0.944
C: explanation generation BERTScore F1 0.656 0.759

Usage

import torch
from datasets import load_dataset
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("google/gemma-2-9b-it")
model = AutoModelForCausalLM.from_pretrained("google/gemma-2-9b-it", torch_dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(model, "Kamyar-zeinalipour/gemma2-9b-CS-AR").merge_and_unload()

# test prompts in this model's chat format (the gold answer follows the assistant marker)
row = load_dataset("Kamyar-zeinalipour/ArabicCommonsense_DB_Gemma", split="test")[0]["text"]

The complete script, which splits each row into prompt and answer and decodes as in the paper, is inference/generate.py (--model gemma2).

Training

LoRA r = 16, α = 32, dropout 0.1 on all attention and MLP projections, embed_tokens and lm_head; 1 epoch; AdamW, learning rate 1e-4, cosine decay; 8 sequences per GPU × 4 accumulation steps; up to 1,024 tokens; bf16, DeepSpeed, seed 100. Script: scripts/train_gemma2.sh.

ArabicSense models

Citation

@inproceedings{lamsiyah-etal-2025-arabicsense,
    title = "{A}rabic{S}ense: A Benchmark for Evaluating Commonsense Reasoning in {A}rabic with Large Language Models",
    author = "Lamsiyah, Salima  and
      Zeinalipour, Kamyar  and
      El amrany, Samir  and
      Brust, Matthias  and
      Maggini, Marco  and
      Bouvry, Pascal  and
      Schommer, Christoph",
    editor = "Ezzini, Saad  and
      Alami, Hamza  and
      Berrada, Ismail  and
      Benlahbib, Abdessamad  and
      El Mahdaouy, Abdelkader  and
      Lamsiyah, Salima  and
      Derrouz, Hatim  and
      Haddad Haddad, Amal  and
      Jarrar, Mustafa  and
      El-Haj, Mo  and
      Mitkov, Ruslan  and
      Rayson, Paul",
    booktitle = "Proceedings of the 4th Workshop on {A}rabic Corpus Linguistics ({WACL}-4)",
    month = jan,
    year = "2025",
    address = "Abu Dhabi, UAE",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2025.wacl-1.1/",
    pages = "1--11"
}

License

Released under the base model's license (gemma).

Downloads last month
28
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for Kamyar-zeinalipour/gemma2-9b-CS-AR

Adapter
(494)
this model

Datasets used to train Kamyar-zeinalipour/gemma2-9b-CS-AR

Collection including Kamyar-zeinalipour/gemma2-9b-CS-AR