car-review-absa

Aspect-level sentiment for car owner reviews. One forward pass returns the reviewer's opinion on eight aspects — price, performance, comfort, reliability, design, safety, fuel economy, technology — each as not mentioned / positive / negative / mixed, plus an overall verdict (positive / mixed / negative).

Fine-tuned from ModernBERT-base on 3,872 synthetic reviews written and blind-checked by a local Qwen3-8B teacher. Try it in the live demo; code, data generation and evaluation are on GitHub.

Results

330 test reviews about six car models that never appear in training. Latency: median per review on CPU, 2 threads.

Model	Aspect macro-F1	Mention F1	Sentiment acc. (mentioned)	All 8 aspects right	Overall verdict	Latency
DeBERTa-v3 ABSA + DistilBERT-SST2 (previous pipeline)	0.183	0.401	88.2%	0.0%	74.2%	2,366 ms
car-review-absa	0.921	0.991	95.6%	92.4%	90.6%	184 ms

The previous pipeline asks a general ABSA model about each aspect in turn; it always returns a polarity, so it cannot tell that an aspect isn't discussed. Weakest aspect: design (macro-F1 0.73), where mixed opinions are rare in the data.

Usage

The 35 outputs are eight groups of four (one per aspect, in the order above) followed by three overall classes; config.id2label names every output.

import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer

name = "Phoenixak99/car-review-absa"
tok = AutoTokenizer.from_pretrained(name)
model = AutoModelForSequenceClassification.from_pretrained(name).eval()

review = "Roomy and comfortable, and we average 34 mpg, but the touchscreen freezes every week."
with torch.no_grad():
    logits = model(**tok(review, return_tensors="pt")).logits[0]

aspects = ["price", "performance", "comfort", "reliability", "design", "safety", "fuel_economy", "technology"]
states = ["none", "positive", "negative", "mixed"]
probs = logits[:32].reshape(8, 4).softmax(-1)
print({a: states[p.argmax()] for a, p in zip(aspects, probs) if p.argmax() > 0})
print("overall:", ["negative", "mixed", "positive"][logits[32:].argmax()])

The GitHub repo's review_intel.infer.Analyzer adds the sentences behind each aspect and GLiNER entities.

Training data

No car-review dataset with a clear licence was available (the public sets are scraped review sites), so the data is synthetic and labelled by construction:

1. A spec is sampled: car (real make and model), reviewer persona, length, tone, 1–4 aspects with a sentiment each.
2. Qwen3-8B (local, llama.cpp) writes a review expressing exactly that spec.
3. Qwen3-8B labels the review again without seeing the spec, with JSON-schema-constrained output.
4. A review is kept only if the blind labels match the spec on every aspect (3,872 of 6,000, 64.5%); the overall verdict comes from the blind reading.

A manual check of 40 random kept reviews found 35 labelled exactly right; the other five each missed a faint passing mention (for example a "quiet cabin" aside in a reliability review) rather than flipping a sentiment. Train / validation / test are split by car model.

Limitations

• Trained on synthetic English reviews: real reviews with slang, typos, sarcasm or non-car tangents will be harder. Treat scores on real data as unmeasured.
• Opinions in the training text are invented, even though car names are real; the model says nothing about actual cars.
• Faint or implicit mentions are the most common miss.
• Eight fixed aspects; anything else (dealership experience, towing as its own topic…) is folded into the nearest aspect or ignored.

Training

ModernBERT-base, 35-way head trained with one cross-entropy per aspect group plus 0.5 × the overall-verdict loss; AdamW, lr 5e-5 for two epochs then one epoch at 2e-5, batch 16, max length 256, on CPU.