basilepp19/mt5_base_inclusive

FAIR-ISLE: A Bilingual Platform for AI-Mediated Inclusive Rewriting and Critical Language Awareness

Official repository for the paper: "FAIR-ISLE: A Bilingual Platform for AI-mediated Inclusive Rewriting and Critical Language Awareness", accepted at the 12th Italian Conference on Computational Linguistics (CLiC-it 2026), Palermo, Italy.

---

📌 Overview

FAIR-ISLE (Female Artificial Intelligence Resource for a Safe Learning Environment) is an interdisciplinary bilingual (Italian–English) platform designed to promote critical language awareness and identify/rewrite identity-based linguistic biases using Large Language Models (LLMs).

Unlike conventional grammar correctors, FAIR-ISLE integrates a Human-in-the-Loop (HITL) interaction mechanism to position users as active evaluators rather than passive recipients of automated corrections, fostering metalinguistic reflection and digital literacy in educational contexts.

Key Features

• 🌐 Bilingual Rewriting (Italian & English): Handles English neutralization strategies as well as complex Italian morphosyntactic structures.
• 🎯 Broad Bias Coverage: Mitigates gender bias, cisnormative bias, ageism, racial bias, xenophobia, sexual-orientation bias, ableism, socioeconomic bias, and intersectional bias.
• ⚡ Parameter-Efficient SFT: Fine-tuned Llama-3-8B-Instruct using Unsloth and LoRA (4-bit NF4 quantization) targeting all linear layers.
• 🖥️ Interactive Web Interface: Lightweight Flask app providing sentence-level tokenization (NLTK), low-latency inference ($T=0.1$), and structured JSON feedback logging for DPO fine-tuning loops.

---

📊 Dataset & Model Performance

Dataset

• Total Samples: 10,391 parallel sentence pairs generated via teacher-student distillation (GPT-4.1) and validated by expert human annotators.
• Split: 90% Training / 10% Testing.
• Includes explicitly tagged neutral sentences to train identity mappings.

In-Vitro Evaluation

Model	sacreBLEU	chrF	METEOR	COMET
Llama-3-8B-Instruct (LoRA FT)	82.42	87.47	0.83	0.91
mT5-base (Full FT)	75.00	80.19	0.73	0.86

Real-World In-Vivo Evaluation (250 Undergraduate Students, 540 Samples)

• User Acceptance Rate (UAR): 77.4% ( rewrites accepted without modification )
• User Correction Rate (UCR): 18.5% ( manually edited by users )
• Rejection Rate (RR): 4.1% ( dismissed suggestions )

---

🛠️ Hyperparameter Configuration

Parameter	Value
Base Model	Llama-3-8B-Instruct (Unsloth 4-bit NF4)
Max Sequence Length	2048
LoRA Rank ($r$) / Alpha ($ lpha$)	16 / 16
Target Modules	$q, k, v, o, gate, up, down$
Optimizer / LR	AdamW (8-bit) / $2 imes 10^{-4}$ (Linear Scheduler)
Batch Size / Effective Batch Size	8 per device (Grad Accum = 2) $
ightarrow$ 16
Epochs	5

---

🚀 Quick Start

1. Installation

git clone https://github.com/uniba-nlp/FAIR-ISLE.git
cd FAIR-ISLE

conda create -n fair-isle python=3.10 -y
conda activate fair-isle

pip install -r requirements.txt

2. Running the Interactive Web Interface

Launch the Flask server to interact with the FAIR-ISLE Inclusive Rewriter:

python app.py --model_path uniba-nlp/Llama-3-8B-FAIR-ISLE --port 5000

Open your browser at http://localhost:5000.

3. Model Fine-Tuning with Unsloth

To replicate the training using LoRA fine-tuning:

python train.py \
    --base_model "unsloth/llama-3-8b-instruct-bnb-4bit" \
    --dataset_path "./data/fair_isle_dataset.json" \
    --output_dir "./models/fair-isle-lora" \
    --epochs 5 \
    --batch_size 8 \
    --learning_rate 2e-4

---

💡 Prompt Template

The fine-tuned model follows the Llama-3 instruction format:

<|begin_of_text|><|start_header_id|>system<|end_header_id|>

You are an assistant skilled in inclusive language. Rewrite the sentence provided by the user to make it inclusive. If the sentence is already correct or neutral, return it unchanged.<|eot_id|>
<|start_header_id|>user<|end_header_id|>

Original sentence: {input_sentence}<|eot_id|>
<|start_header_id|>assistant<|end_header_id|>

Rewritten sentence: {output_sentence}<|eot_id|>

---

📜 Citation

If you use FAIR-ISLE in your research or application, please cite our paper:

@inproceedings{filograsso2026fairisle,
  title     = {FAIR-ISLE: A Bilingual Platform for AI-mediated Inclusive Rewriting and Critical Language Awareness},
  author    = {Filograsso, Francesca and Siciliani, Lucia and Maglie, Rosita Belinda and Basile, Pierpaolo and de Gemmis, Marco},
  booktitle = {Proceedings of the Twelfth Italian Conference on Computational Linguistics (CLiC-it 2026)},
  year      = {2026},
  address   = {Palermo, Italy},
  publisher = {CEUR Workshop Proceedings}
}

---

👥 Authors & Affiliations

• Francesca Filograsso* (francesca.filograsso@uniba.it) — Department of Education, Psychology and Communication Sciences, University of Bari Aldo Moro
• Lucia Siciliani* (lucia.siciliani@uniba.it) — Department of Computer Science, University of Bari Aldo Moro
• Rosita Belinda Maglie (rosita.maglie@uniba.it) — Department of Education, Psychology and Communication Sciences, University of Bari Aldo Moro
• Pierpaolo Basile (pierpaolo.basile@uniba.it) — Department of Computer Science, University of Bari Aldo Moro
• Marco de Gemmis (marco.degemmis@uniba.it) — Department of Computer Science, University of Bari Aldo Moro

---

📄 License

This project is licensed under the Creative Commons Attribution 4.0 International License (CC BY 4.0).