RoBERTa-base + LoRA โ€” IMDB Sentiment Classification

LoRA-adapted roberta-base for binary sentiment classification on the IMDB Large Movie Review Dataset. One of four base models in a parameter-efficient stacking ensemble described in "Parameter-Efficient Fine-Tuning Ensembles for Sentiment Analysis" (IDAP 2026).

Results (25,000-example IMDB test set)

Metric Value
Accuracy 95.60%
F1 95.62%
Precision 95.23%
Recall 96.01%
ROC-AUC 99.00%

LoRA Configuration

Hyperparameter Value
Rank (r) 16
Alpha 32
Dropout 0.1
Target modules query, value
Trainable parameters 1,181,954 (0.94% of 125.8M total)
Precision fp16

Loading

from transformers import RobertaTokenizer, RobertaForSequenceClassification
from peft import PeftModel

tokenizer = RobertaTokenizer.from_pretrained("roberta-base")
base = RobertaForSequenceClassification.from_pretrained("roberta-base", num_labels=2)
model = PeftModel.from_pretrained(base, "Denizhaan/imdb-roberta-lora-v2")
model.eval()

Citation

If you use this model, please cite our paper (details in the GitHub repository).

Framework versions

  • PEFT 0.19.1
Downloads last month
5
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for Denizhaan/imdb-roberta-lora-v2

Adapter
(329)
this model