File size: 3,906 Bytes
7e3fdc8 063fa24 7e3fdc8 063fa24 7e3fdc8 063fa24 7e3fdc8 063fa24 7e3fdc8 063fa24 7e3fdc8 063fa24 7e3fdc8 063fa24 7e3fdc8 063fa24 7e3fdc8 063fa24 7e3fdc8 063fa24 7e3fdc8 063fa24 7e3fdc8 063fa24 7e3fdc8 063fa24 7e3fdc8 063fa24 7e3fdc8 063fa24 7e3fdc8 063fa24 7e3fdc8 063fa24 7e3fdc8 063fa24 7e3fdc8 063fa24 7e3fdc8 063fa24 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 | ---
base_model: mistralai/Mistral-7B-Instruct-v0.2
library_name: peft
pipeline_tag: text-generation
license: apache-2.0
language:
- en
datasets:
- Kamyar-zeinalipour/Essay-Syntax-Instruct
tags:
- education
- writing-feedback
- syntax
- lora
- peft
---
<div align="center">
# Mistral7B-Syntax-Instruct
**Mistral-7B-Instruct-v0.2 fine-tuned to give students feedback on essay syntax**
[](https://arxiv.org/abs/2501.07740) [](https://github.com/KamyarZeinalipour/Essay-Syntax-Feedback) [](https://huggingface.co/collections/Kamyar-zeinalipour/automated-syntax-feedback-aieer-2024-6af487599293af6e63c227f9)
</div>
Reads an English student essay and returns structured feedback on its syntax across eight
categories β spelling, conjunctions and connectives, modifiers, prepositions, modal verbs,
punctuation, articles and pronouns β with examples from the essay and suggestions for each.
A LoRA adapter for [Mistral-7B-Instruct-v0.2](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2) trained on
[Essay-Syntax-Instruct](https://huggingface.co/datasets/Kamyar-zeinalipour/Essay-Syntax-Instruct).
From [*Advancing Student Writing Through Automated Syntax Feedback*](https://arxiv.org/abs/2501.07740) (AIEER 2024). Code: [Essay-Syntax-Feedback](https://github.com/KamyarZeinalipour/Essay-Syntax-Feedback).
## Results
ROUGE against the reference feedback, base β fine-tuned:
| Model | ROUGE-1 | ROUGE-2 | ROUGE-L |
|---|---|---|---|
| Llama-2-7b-chat | 0.351 β **0.477** | 0.146 β **0.337** | 0.185 β **0.343** |
| Llama-2-13b-chat | 0.328 β **0.476** | 0.148 β **0.340** | 0.214 β **0.347** |
| Mistral-7B-Instruct-v0.2 | 0.366 β **0.514** | 0.144 β **0.375** | 0.205 β **0.369** |
## Usage
```python
import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("mistralai/Mistral-7B-Instruct-v0.2")
model = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-Instruct-v0.2", torch_dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(model, "Kamyar-zeinalipour/Mistral7B-Syntax-Instruct").merge_and_unload()
```
## Training
LoRA r = 32, Ξ± = 64, dropout 0.1; 3 epochs; AdamW, learning rate 1e-4 with cosine decay;
8 sequences per GPU Γ 4 accumulation steps; up to 2,048 tokens; bf16, DeepSpeed.
Script: [`scripts/`](https://github.com/KamyarZeinalipour/Essay-Syntax-Feedback/tree/main/scripts).
## Models from this paper
| Base model | Adapter | ROUGE-L (base β fine-tuned) |
|---|---|---|
| [Llama-2-7b-chat](https://huggingface.co/meta-llama/Llama-2-7b-chat-hf) | [Llama2-7B-Syntax-Instruct](https://huggingface.co/Kamyar-zeinalipour/Llama2-7B-Syntax-Instruct) | 0.185 β **0.343** |
| [Llama-2-13b-chat](https://huggingface.co/meta-llama/Llama-2-13b-chat-hf) | [Llama2-13B-Syntax-Instruct](https://huggingface.co/Kamyar-zeinalipour/Llama2-13B-Syntax-Instruct) | 0.214 β **0.347** |
| [Mistral-7B-Instruct-v0.2](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2) | **Mistral7B-Syntax-Instruct** (this model) | 0.205 β **0.369** |
## Citation
```bibtex
@inproceedings{zeinalipour2024advancing,
title = {Advancing Student Writing Through Automated Syntax Feedback},
author = {Kamyar Zeinalipour and Mehak Mehak and Fatemeh Parsamotamed and Marco Maggini and Marco Gori},
booktitle = {AI in Education and Educational Research (AIEER 2024)},
pages = {52--66},
year = {2024},
publisher = {Springer},
doi = {10.1007/978-3-031-93409-4_5},
url = {https://arxiv.org/abs/2501.07740}
}
```
## License
Released under the base model's license (`apache-2.0`).
|