Hugging Face's logo

Spaces:
buzzyss
/
SentimentMatcha
Running

App Files Community
SentimentMatcha
File size: 5,375 Bytes
541fa1f
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
---
title: Matcha Sentiment
emoji: 🍵
colorFrom: green
colorTo: yellow
sdk: gradio
sdk_version: 5.0.1
app_file: app.py
python_version: "3.12"
pinned: false
license: mit
---

# Matcha Sentiment

Sentiment analysis bahasa Indonesia untuk review Matchaya/IKUYO. Dataset dibersihkan menjadi klasifikasi biner `Negatif` dan `Positif`, lalu dibandingkan dengan baseline machine learning klasik dan fine-tuning 9 model Transformer Indonesia.

![Dashboard](docs/images/dashboard_home_wide.png)

## Ringkasan

| Area | Hasil |
| --- | --- |
| Dataset final | 2028 review |
| Label | 1014 `Negatif`, 1014 `Positif` |
| Label dihapus | 14 `Netral` |
| Duplikat dibuang | 219 teks |
| Best classical | `TF-IDF + Linear SVM` |
| Best Transformer | `indolem/indobert-base-uncased` |
| Runtime | Docker + NVIDIA GPU |
| Dashboard | Gradio, siap Hugging Face Spaces |

> Catatan push: model Transformer terbaik disimpan di `models/best_transformer` dan ditrack lewat Git LFS. Weight kandidat di `models/transformers/*/model/model.safetensors` di-ignore karena bisa dibuat ulang dari pipeline training.

## Hasil Utama

### Transformer

| Model | Accuracy | Precision | Recall | F1 | ROC AUC |
| --- | ---: | ---: | ---: | ---: | ---: |
| `indolem/indobert-base-uncased` | 0.9951 | 0.9902 | 1.0000 | 0.9951 | 0.9998 |
| `naufalihsan/indonesian-sbert-large` | 0.9901 | 0.9806 | 1.0000 | 0.9902 | 0.9998 |
| `flax-community/indonesian-roberta-base` | 0.9901 | 0.9806 | 1.0000 | 0.9902 | 0.9997 |
| `sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2` | 0.9901 | 0.9806 | 1.0000 | 0.9902 | 0.9996 |
| `indobenchmark/indobert-base-p1` | 0.9901 | 0.9806 | 1.0000 | 0.9902 | 0.9989 |
| `ChristopherA08/IndoELECTRA` | 0.9901 | 0.9806 | 1.0000 | 0.9902 | 0.9989 |
| `cahya/distilbert-base-indonesian` | 0.9901 | 0.9901 | 0.9901 | 0.9901 | 0.9998 |
| `indolem/indobertweet-base-uncased` | 0.9852 | 0.9804 | 0.9901 | 0.9852 | 0.9994 |
| `w11wo/indonesian-roberta-base-sentiment-classifier` | 0.9803 | 0.9619 | 1.0000 | 0.9806 | 1.0000 |

Model terbaik sudah disimpan di:

```text
models/best_transformer
```

### Machine Learning Klasik

TF-IDF dan Word2Vec diuji dengan Stratified 10-fold cross validation. Hasil lengkap:

| Feature | Model | Accuracy | Precision | Recall | F1 | ROC AUC |
| --- | --- | ---: | ---: | ---: | ---: | ---: |
| TF-IDF | Linear SVM | 0.9684 | 0.9788 | 0.9576 | 0.9681 | 0.9951 |
| Word2Vec | Logistic Regression | 0.9635 | 0.9663 | 0.9606 | 0.9634 | 0.9939 |
| Word2Vec | Extra Trees | 0.9625 | 0.9653 | 0.9596 | 0.9624 | 0.9940 |
| TF-IDF | Logistic Regression | 0.9610 | 0.9756 | 0.9458 | 0.9604 | 0.9933 |
| Word2Vec | Linear SVM | 0.9596 | 0.9660 | 0.9527 | 0.9593 | 0.9924 |
| Word2Vec | Random Forest | 0.9591 | 0.9632 | 0.9546 | 0.9589 | 0.9927 |
| Word2Vec | Gradient Boosting | 0.9571 | 0.9603 | 0.9536 | 0.9570 | 0.9933 |
| TF-IDF | Extra Trees | 0.9522 | 0.9580 | 0.9458 | 0.9519 | 0.9918 |
| TF-IDF | Random Forest | 0.9443 | 0.9353 | 0.9546 | 0.9449 | 0.9882 |
| TF-IDF | Gradient Boosting | 0.9147 | 0.9304 | 0.8964 | 0.9131 | 0.9735 |

## Visual Evaluasi

![Results Gallery](docs/images/results_gallery.png)

### Dashboard

| Prediksi | Visual | Kata Kunci |
| --- | --- | --- |
| ![Dashboard Home](docs/images/dashboard_home_wide.png) | ![Dashboard Visual](docs/images/dashboard_visual_tab.png) | ![Dashboard Keywords](docs/images/dashboard_keywords_tab.png) |

### Detail Plot

| Training Loss | Confusion Matrix | ROC AUC |
| --- | --- | --- |
| ![Training Loss](docs/images/transformer_best_training_loss.png) | ![Confusion Matrix](docs/images/transformer_best_confusion_matrix.png) | ![ROC AUC](docs/images/transformer_best_roc_auc.png) |

| Top Words | Word Cloud Positif | Word Cloud Negatif |
| --- | --- | --- |
| ![Top Words](docs/images/top_words_tfidf.png) | ![Word Cloud Positif](docs/images/wordcloud_positif.png) | ![Word Cloud Negatif](docs/images/wordcloud_negatif.png) |

## Kata Kunci Bermakna

Beberapa kata yang paling membantu membaca arah sentimen:

| Kata | Positif Docs | Negatif Docs | Dominan |
| --- | ---: | ---: | --- |
| `enak` | 173 | 10 | Positif |
| `nyaman` | 54 | 10 | Positif |
| `ramah` | 37 | 9 | Positif |
| `terbaik` | 18 | 0 | Positif |
| `mahal` | 10 | 24 | Negatif |
| `harga` | 10 | 28 | Negatif |
| `buruk` | 0 | 19 | Negatif |
| `antrean` | 0 | 19 | Negatif |
| `lama` | 1 | 16 | Negatif |

File lengkapnya ada di:

```text
artifacts/classical/keyword_counts.csv
```

## Struktur Proyek

```text
.
├── app.py
├── Dockerfile
├── docker-compose.yml
├── INSTALL_DOCKER.md
├── data/processed/matcha_sentiment_binary.csv
├── docs/images/
├── artifacts/
├── models/best_transformer/
├── models/classical/best_model.joblib
├── scripts/
└── src/matcha_sentiment/
```

## Quick Start

```bash
docker build -t matcha-sentiment .
docker run --rm --gpus all -p 7860:7860 -v "${PWD}:/workspace" matcha-sentiment
```

Buka:

```text
http://localhost:7860
```

Panduan dari nol sampai deploy ada di [INSTALL_DOCKER.md](INSTALL_DOCKER.md).

## Catatan

Skor evaluasi sangat tinggi karena dataset masih kecil dan domainnya sempit. Model ini sudah bagus untuk demo, dashboard, dan eksperimen sentiment analysis review matcha, tetapi untuk production lintas brand atau lintas kategori sebaiknya ditambah data baru yang lebih beragam.
# matchaSentiment