FoodReviewAI - BERTimbau Sentiment Classifier
Model Description
FoodReviewAI-BERTimbau is a fine-tuned Transformer model developed to classify Brazilian Portuguese restaurant reviews as positive or negative.
The model is based on neuralmind/bert-base-portuguese-cased and was trained using PyTorch and Hugging Face Transformers.
This project compares traditional Machine Learning techniques with Transformer-based models for sentiment analysis.
Dataset
The model uses the StarRest dataset, containing restaurant reviews from Brazil.
After preprocessing, 16,293 Portuguese reviews were selected:
| Sentiment | Reviews |
|---|---|
| Positive | 15,207 |
| Negative | 1,086 |
| Total | 16,293 |
Reviews rated 1β2 were labeled negative, while reviews rated 4β5 were labeled positive. Reviews rated 3 were excluded.
Only reviews from platforms 1, 2 and 3 were used.
The sentiment labels were derived from customer ratings rather than manual sentiment annotations.
Training Configuration
- Base model: BERTimbau Base
- Framework: PyTorch / Transformers
- Hardware: NVIDIA Tesla T4
- Training epochs: 2
- Learning rate: 2e-5
- Training batch size: 16
- Evaluation batch size: 32
- Maximum sequence length: 128 tokens
- Optimized metric: Negative-class F1-score
- Class balancing: Weighted cross-entropy
- Random seed: 42
The dataset was split into 80% training and 20% testing. An additional 10% of the training portion was reserved for validation.
Evaluation Results
The final model was evaluated on 3,259 held-out reviews.
| Metric | Logistic Regression | BERTimbau |
|---|---|---|
| Accuracy | 95.58% | 97.64% |
| Negative Precision | 0.61 | 0.80 |
| Negative Recall | 0.91 | 0.86 |
| Negative F1-score | 0.73 | 0.83 |
| Positive F1-score | 0.98 | 0.99 |
BERTimbau Confusion Matrix
| Predicted Negative | Predicted Positive | |
|---|---|---|
| Actual Negative | 186 | 31 |
| Actual Positive | 46 | 2,996 |
BERTimbau achieved a higher negative-class F1-score and precision than Logistic Regression. However, Logistic Regression achieved higher recall for negative reviews.
This illustrates the trade-off between identifying more dissatisfied customers and reducing false alarms.
Usage
Install the dependencies:
pip install torch transformers
Load the trained model:
from transformers import pipeline
classificador = pipeline(
"text-classification",
model="felipermtrindade/FoodReviewAI-BERTimbau",
truncation=True,
max_length=128
)
texto = "A comida estava maravilhosa e o atendimento foi excelente!"
resultado = classificador(texto)
print(resultado)
Limitations
- The dataset is highly imbalanced, containing approximately 93% positive reviews.
- Sentiment labels were inferred from ratings and may contain labeling errors.
- Reviews containing mixed sentiments may be difficult to classify correctly.
- Inputs longer than 128 tokens are truncated.
- The model was trained primarily for Brazilian Portuguese restaurant reviews.
- Confidence scores have not been independently calibrated.
- Generalization to other datasets and real-world applications has not yet been established.
Source Code
Dataset
Base Model
Future Improvements
- Experiment with different class weights.
- Evaluate model performance across different restaurants.
- Investigate incorrectly classified reviews.
- Test on external datasets.
- Compare inference time and memory consumption between models.
- Integrate the Transformer into the FoodReviewAI API.
- Downloads last month
- 43
Model tree for felipermtrindade/FoodReviewAI-BERTimbau
Base model
neuralmind/bert-base-portuguese-cased