base_model: mistralai/Mistral-7B-Instruct-v0.2
library_name: peft
pipeline_tag: text-generation
license: apache-2.0
language:
- en
datasets:
- Kamyar-zeinalipour/Essay-Syntax-Instruct
tags:
- education
- writing-feedback
- syntax
- lora
- peft
Mistral7B-Syntax-Instruct
Mistral-7B-Instruct-v0.2 fine-tuned to give students feedback on essay syntax
Reads an English student essay and returns structured feedback on its syntax across eight categories β spelling, conjunctions and connectives, modifiers, prepositions, modal verbs, punctuation, articles and pronouns β with examples from the essay and suggestions for each. A LoRA adapter for Mistral-7B-Instruct-v0.2 trained on Essay-Syntax-Instruct.
From Advancing Student Writing Through Automated Syntax Feedback (AIEER 2024). Code: Essay-Syntax-Feedback.
Results
ROUGE against the reference feedback, base β fine-tuned:
| Model | ROUGE-1 | ROUGE-2 | ROUGE-L |
|---|---|---|---|
| Llama-2-7b-chat | 0.351 β 0.477 | 0.146 β 0.337 | 0.185 β 0.343 |
| Llama-2-13b-chat | 0.328 β 0.476 | 0.148 β 0.340 | 0.214 β 0.347 |
| Mistral-7B-Instruct-v0.2 | 0.366 β 0.514 | 0.144 β 0.375 | 0.205 β 0.369 |
Usage
import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("mistralai/Mistral-7B-Instruct-v0.2")
model = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-Instruct-v0.2", torch_dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(model, "Kamyar-zeinalipour/Mistral7B-Syntax-Instruct").merge_and_unload()
Training
LoRA r = 32, Ξ± = 64, dropout 0.1; 3 epochs; AdamW, learning rate 1e-4 with cosine decay;
8 sequences per GPU Γ 4 accumulation steps; up to 2,048 tokens; bf16, DeepSpeed.
Script: scripts/.
Models from this paper
| Base model | Adapter | ROUGE-L (base β fine-tuned) |
|---|---|---|
| Llama-2-7b-chat | Llama2-7B-Syntax-Instruct | 0.185 β 0.343 |
| Llama-2-13b-chat | Llama2-13B-Syntax-Instruct | 0.214 β 0.347 |
| Mistral-7B-Instruct-v0.2 | Mistral7B-Syntax-Instruct (this model) | 0.205 β 0.369 |
Citation
@inproceedings{zeinalipour2024advancing,
title = {Advancing Student Writing Through Automated Syntax Feedback},
author = {Kamyar Zeinalipour and Mehak Mehak and Fatemeh Parsamotamed and Marco Maggini and Marco Gori},
booktitle = {AI in Education and Educational Research (AIEER 2024)},
pages = {52--66},
year = {2024},
publisher = {Springer},
doi = {10.1007/978-3-031-93409-4_5},
url = {https://arxiv.org/abs/2501.07740}
}
License
Released under the base model's license (apache-2.0).