Instructions to use Kamyar-zeinalipour/Mistral-7b-CS-AR with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use Kamyar-zeinalipour/Mistral-7b-CS-AR with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-Instruct-v0.3") model = PeftModel.from_pretrained(base_model, "Kamyar-zeinalipour/Mistral-7b-CS-AR") - Notebooks
- Google Colab
- Kaggle
A LoRA adapter for Mistral-7B-Instruct-v0.3 trained on all three ArabicSense tasks: deciding which of two Arabic statements does not make sense, choosing the reason it does not make sense, and writing that reason.
Part of ArabicSense, a benchmark for commonsense reasoning in Arabic, from the paper ArabicSense: A Benchmark for Evaluating Commonsense Reasoning in Arabic with Large Language Models (WACL-4 2025). Code: KamyarZeinalipour/ArabicSense; encoder experiments: EL-Amrany/Arabic-Commonsense-Reasoning.
Results on the ArabicSense test set
| Task | Metric | Zero-shot | Fine-tuned |
|---|---|---|---|
| A: validation | Accuracy / F1 | 0.523 / 0.601 | 0.948 / 0.948 |
| B: explanation selection | Accuracy / F1 | 0.805 / 0.805 | 0.934 / 0.934 |
| C: explanation generation | BERTScore F1 | 0.728 | 0.771 |
Usage
import torch
from datasets import load_dataset
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("mistralai/Mistral-7B-Instruct-v0.3")
model = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-Instruct-v0.3", torch_dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(model, "Kamyar-zeinalipour/Mistral-7b-CS-AR").merge_and_unload()
# test prompts in this model's chat format (the gold answer follows the assistant marker)
row = load_dataset("Kamyar-zeinalipour/ArabicCommonsense_DB_Mistral", split="test")[0]["text"]
The complete script, which splits each row into prompt and answer and decodes as in the paper, is
inference/generate.py (--model mistral).
Training
LoRA r = 16, α = 32, dropout 0.1 on all attention and MLP projections, embed_tokens and lm_head;
3 epochs; AdamW, learning rate 1e-4, cosine decay; 8 sequences per GPU × 4 accumulation steps;
up to 1,024 tokens; bf16, DeepSpeed, seed 100. Script: scripts/train_mistral.sh.
ArabicSense models
| Base model | Adapter | Training data |
|---|---|---|
| gemma-2-9b-it | gemma2-9b-CS-AR | ArabicCommonsense_DB_Gemma |
| Meta-Llama-3.1-8B-Instruct | Llama3-8B-CS-AR | ArabicCommonsense_DB_Llama3 |
| Mistral-7B-Instruct-v0.3 | Mistral-7b-CS-AR (this model) | ArabicCommonsense_DB_Mistral |
Citation
@inproceedings{lamsiyah-etal-2025-arabicsense,
title = "{A}rabic{S}ense: A Benchmark for Evaluating Commonsense Reasoning in {A}rabic with Large Language Models",
author = "Lamsiyah, Salima and
Zeinalipour, Kamyar and
El amrany, Samir and
Brust, Matthias and
Maggini, Marco and
Bouvry, Pascal and
Schommer, Christoph",
editor = "Ezzini, Saad and
Alami, Hamza and
Berrada, Ismail and
Benlahbib, Abdessamad and
El Mahdaouy, Abdelkader and
Lamsiyah, Salima and
Derrouz, Hatim and
Haddad Haddad, Amal and
Jarrar, Mustafa and
El-Haj, Mo and
Mitkov, Ruslan and
Rayson, Paul",
booktitle = "Proceedings of the 4th Workshop on {A}rabic Corpus Linguistics ({WACL}-4)",
month = jan,
year = "2025",
address = "Abu Dhabi, UAE",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2025.wacl-1.1/",
pages = "1--11"
}
License
Released under the base model's license (apache-2.0).
- Downloads last month
- 22
Model tree for Kamyar-zeinalipour/Mistral-7b-CS-AR
Base model
mistralai/Mistral-7B-v0.3