Hooshaai/distilbert-sst2

DistilBERT fine-tuned on SST-2 (binary sentiment)

This checkpoint was fully fine-tuned (backbone + classification head) on the full SST2 training set as part of the A3 SVD transformer compression project.

Results

Metric	Value
Accuracy	0.9106
F1 (macro)	0.9137

Model Details

• Base model: distilbert-base-uncased
• Architecture: 6 layers, 768 hidden, 12 heads
• Number of labels: 2
• Task: SST-2 (binary sentiment)

Usage

from transformers import AutoModelForSequenceClassification, AutoTokenizer

model = AutoModelForSequenceClassification.from_pretrained("TahaaaaM/distilbert-sst2-a3")
tokenizer = AutoTokenizer.from_pretrained("TahaaaaM/distilbert-sst2-a3")

inputs = tokenizer("Your input text here", return_tensors="pt")
logits = model(**inputs).logits
predicted_class = logits.argmax().item()

A3 Compression Context

This checkpoint serves as the starting point for A3 SVD compression experiments. Results of compressed variants will be reported in the accompanying paper.

Citation

Part of the A3 SVD transformer compression project.