FoodReviewAI - BERTimbau Sentiment Classifier

Model Description

FoodReviewAI-BERTimbau is a fine-tuned Transformer model developed to classify Brazilian Portuguese restaurant reviews as positive or negative.

The model is based on neuralmind/bert-base-portuguese-cased and was trained using PyTorch and Hugging Face Transformers.

This project compares traditional Machine Learning techniques with Transformer-based models for sentiment analysis.

Dataset

The model uses the StarRest dataset, containing restaurant reviews from Brazil.

After preprocessing, 16,293 Portuguese reviews were selected:

Sentiment Reviews
Positive 15,207
Negative 1,086
Total 16,293

Reviews rated 1–2 were labeled negative, while reviews rated 4–5 were labeled positive. Reviews rated 3 were excluded.

Only reviews from platforms 1, 2 and 3 were used.

The sentiment labels were derived from customer ratings rather than manual sentiment annotations.

Training Configuration

  • Base model: BERTimbau Base
  • Framework: PyTorch / Transformers
  • Hardware: NVIDIA Tesla T4
  • Training epochs: 2
  • Learning rate: 2e-5
  • Training batch size: 16
  • Evaluation batch size: 32
  • Maximum sequence length: 128 tokens
  • Optimized metric: Negative-class F1-score
  • Class balancing: Weighted cross-entropy
  • Random seed: 42

The dataset was split into 80% training and 20% testing. An additional 10% of the training portion was reserved for validation.

Evaluation Results

The final model was evaluated on 3,259 held-out reviews.

Metric Logistic Regression BERTimbau
Accuracy 95.58% 97.64%
Negative Precision 0.61 0.80
Negative Recall 0.91 0.86
Negative F1-score 0.73 0.83
Positive F1-score 0.98 0.99

BERTimbau Confusion Matrix

Predicted Negative Predicted Positive
Actual Negative 186 31
Actual Positive 46 2,996

BERTimbau achieved a higher negative-class F1-score and precision than Logistic Regression. However, Logistic Regression achieved higher recall for negative reviews.

This illustrates the trade-off between identifying more dissatisfied customers and reducing false alarms.

Usage

Install the dependencies:

pip install torch transformers

Load the trained model:

from transformers import pipeline

classificador = pipeline(
    "text-classification",
    model="felipermtrindade/FoodReviewAI-BERTimbau",
    truncation=True,
    max_length=128
)

texto = "A comida estava maravilhosa e o atendimento foi excelente!"

resultado = classificador(texto)

print(resultado)

Limitations

  • The dataset is highly imbalanced, containing approximately 93% positive reviews.
  • Sentiment labels were inferred from ratings and may contain labeling errors.
  • Reviews containing mixed sentiments may be difficult to classify correctly.
  • Inputs longer than 128 tokens are truncated.
  • The model was trained primarily for Brazilian Portuguese restaurant reviews.
  • Confidence scores have not been independently calibrated.
  • Generalization to other datasets and real-world applications has not yet been established.

Source Code

FoodReviewAI - GitHub

Dataset

StarRest - Mendeley Data

Base Model

NeuralMind BERTimbau Base

Future Improvements

  • Experiment with different class weights.
  • Evaluate model performance across different restaurants.
  • Investigate incorrectly classified reviews.
  • Test on external datasets.
  • Compare inference time and memory consumption between models.
  • Integrate the Transformer into the FoodReviewAI API.
Downloads last month
43
Safetensors
Model size
0.1B params
Tensor type
F32
Β·
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for felipermtrindade/FoodReviewAI-BERTimbau

Finetuned
(260)
this model