Spaces:
Paused
Paused
Upload 10 files
Browse files- README.md +107 -10
- config.py +29 -0
- database.py +155 -0
- docker-compose.yml +49 -0
- main.py +188 -0
- models.py +49 -0
- queue.py +80 -0
- requirements.txt +8 -0
- sentiment.py +93 -0
- test_webhook.py +97 -0
README.md
CHANGED
|
@@ -1,13 +1,110 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
-
|
| 3 |
-
|
| 4 |
-
|
| 5 |
-
|
| 6 |
-
|
| 7 |
-
|
| 8 |
-
|
| 9 |
-
|
| 10 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 11 |
---
|
| 12 |
|
| 13 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# LINE Webhook β HuggingFace Sentiment Server
|
| 2 |
+
|
| 3 |
+
ΰΈ£ΰΈ±ΰΈ message event ΰΈΰΈ²ΰΈ LINE Official Account β ΰΈ§ΰΈ΄ΰΉΰΈΰΈ£ΰΈ²ΰΈ°ΰΈ«ΰΉ sentiment β ΰΈͺΰΉΰΈΰΈΰΉΰΈΰΉΰΈ«ΰΉ Lovable AI Dashboard
|
| 4 |
+
|
| 5 |
---
|
| 6 |
+
|
| 7 |
+
## Architecture
|
| 8 |
+
|
| 9 |
+
```
|
| 10 |
+
LINE OA β Webhook Server β HuggingFace Inference API
|
| 11 |
+
β β
|
| 12 |
+
PostgreSQL ββββββββ sentiment result
|
| 13 |
+
β
|
| 14 |
+
Redis Pub/Sub
|
| 15 |
+
β
|
| 16 |
+
Lovable AI Dashboard (real-time)
|
| 17 |
+
```
|
| 18 |
+
|
| 19 |
+
---
|
| 20 |
+
|
| 21 |
+
## Quick Start
|
| 22 |
+
|
| 23 |
+
### 1. Clone & ΰΈΰΈ΄ΰΈΰΈΰΈ±ΰΉΰΈ
|
| 24 |
+
|
| 25 |
+
```bash
|
| 26 |
+
git clone <repo>
|
| 27 |
+
cd line-webhook
|
| 28 |
+
cp .env.example .env # ΰΉΰΈΰΉΰΉΰΈΰΈΰΉΰΈ²ΰΉΰΈ .env
|
| 29 |
+
```
|
| 30 |
+
|
| 31 |
+
### 2. ΰΉΰΈͺΰΉΰΈΰΉΰΈ²ΰΉΰΈ .env
|
| 32 |
+
|
| 33 |
+
```env
|
| 34 |
+
LINE_CHANNEL_SECRET=<ΰΈΰΈ²ΰΈ LINE Developers Console>
|
| 35 |
+
LINE_CHANNEL_ACCESS_TOKEN=<ΰΈΰΈ²ΰΈ LINE Developers Console>
|
| 36 |
+
HF_API_TOKEN=<ΰΈΰΈ²ΰΈ huggingface.co/settings/tokens>
|
| 37 |
+
```
|
| 38 |
+
|
| 39 |
+
### 3. ΰΈ£ΰΈ±ΰΈΰΈΰΉΰΈ§ΰΈ’ Docker Compose
|
| 40 |
+
|
| 41 |
+
```bash
|
| 42 |
+
docker compose up -d
|
| 43 |
+
```
|
| 44 |
+
|
| 45 |
+
Server ΰΈΰΈ£ΰΉΰΈΰΈ‘ΰΈΰΈ΅ΰΉ `http://localhost:8000`
|
| 46 |
+
|
| 47 |
+
### 4. ΰΈ£ΰΈ±ΰΈ Local (ΰΉΰΈ‘ΰΉΰΉΰΈΰΉ Docker)
|
| 48 |
+
|
| 49 |
+
```bash
|
| 50 |
+
python -m venv venv
|
| 51 |
+
source venv/bin/activate # Windows: venv\Scripts\activate
|
| 52 |
+
pip install -r requirements.txt
|
| 53 |
+
uvicorn app.main:app --reload
|
| 54 |
+
```
|
| 55 |
+
|
| 56 |
---
|
| 57 |
|
| 58 |
+
## ΰΈΰΈ±ΰΉΰΈΰΈΰΉΰΈ² LINE Webhook URL
|
| 59 |
+
|
| 60 |
+
1. ΰΉΰΈΰΈΰΈ΅ΰΉ [LINE Developers Console](https://developers.line.biz/console/)
|
| 61 |
+
2. ΰΉΰΈ₯ΰΈ·ΰΈΰΈ Channel β **Messaging API**
|
| 62 |
+
3. ΰΈΰΈ±ΰΉΰΈ Webhook URL ΰΉΰΈΰΉΰΈ:
|
| 63 |
+
```
|
| 64 |
+
https://your-domain.com/webhook
|
| 65 |
+
```
|
| 66 |
+
> ΰΉΰΈΰΉ **ngrok** ΰΈͺΰΈ³ΰΈ«ΰΈ£ΰΈ±ΰΈ local dev:
|
| 67 |
+
> ```bash
|
| 68 |
+
> ngrok http 8000
|
| 69 |
+
> ```
|
| 70 |
+
> ΰΉΰΈ₯ΰΉΰΈ§ΰΉΰΈΰΉ URL ΰΈΰΈ΅ΰΉ ngrok ΰΉΰΈ«ΰΉ ΰΉΰΈΰΉΰΈ `https://abc123.ngrok.io/webhook`
|
| 71 |
+
4. ΰΉΰΈΰΈ΄ΰΈ **Use webhook** β Verify
|
| 72 |
+
|
| 73 |
+
---
|
| 74 |
+
|
| 75 |
+
## API Endpoints
|
| 76 |
+
|
| 77 |
+
| Method | Path | ΰΈΰΈ³ΰΈΰΈΰΈ΄ΰΈΰΈ²ΰΈ’ |
|
| 78 |
+
|--------|------|-----------|
|
| 79 |
+
| `POST` | `/webhook` | ΰΈ£ΰΈ±ΰΈ event ΰΈΰΈ²ΰΈ LINE (signature verified) |
|
| 80 |
+
| `GET` | `/health` | Health check |
|
| 81 |
+
| `GET` | `/api/conversations` | ΰΈΰΈΆΰΈ conversation list ΰΈΰΈ£ΰΉΰΈΰΈ‘ sentiment |
|
| 82 |
+
| `GET` | `/api/sentiment-summary` | ΰΈͺΰΈ£ΰΈΈΰΈ sentiment ΰΈ ΰΈ²ΰΈΰΈ£ΰΈ§ΰΈ‘ 7 ΰΈ§ΰΈ±ΰΈ |
|
| 83 |
+
|
| 84 |
+
---
|
| 85 |
+
|
| 86 |
+
## HuggingFace Model
|
| 87 |
+
|
| 88 |
+
ΰΉΰΈΰΈ°ΰΈΰΈ³ΰΉΰΈΰΉ model ΰΉΰΈ«ΰΈ₯ΰΉΰΈ²ΰΈΰΈ΅ΰΉ (ΰΈΰΈ±ΰΉΰΈΰΉΰΈ `HF_MODEL_NAME`):
|
| 89 |
+
|
| 90 |
+
| Model | ΰΈ ΰΈ²ΰΈ©ΰΈ² | ΰΈΰΈ§ΰΈ²ΰΈ‘ΰΉΰΈ‘ΰΉΰΈΰΈ’ΰΈ³ |
|
| 91 |
+
|-------|-------|-----------|
|
| 92 |
+
| `cardiffnlp/twitter-xlm-roberta-base-sentiment` | Multilingual (ΰΉΰΈΰΈ’ΰΉΰΈΰΉ) | ΰΈΰΈ΅ |
|
| 93 |
+
| `airesearch/wangchanberta-base-att-spm-uncased` | ΰΉΰΈΰΈ’ΰΉΰΈΰΉΰΈΰΈ«ΰΈ₯ΰΈ±ΰΈ | ΰΈΰΈ΅ΰΈ‘ΰΈ²ΰΈΰΈͺΰΈ³ΰΈ«ΰΈ£ΰΈ±ΰΈΰΉΰΈΰΈ’ |
|
| 94 |
+
|
| 95 |
+
---
|
| 96 |
+
|
| 97 |
+
## Alert System
|
| 98 |
+
|
| 99 |
+
ΰΉΰΈ‘ΰΈ·ΰΉΰΈ negative score β₯ `NEGATIVE_ALERT_THRESHOLD` (default 0.80):
|
| 100 |
+
- Push event ΰΉΰΈΰΈΰΈ΅ΰΉ Redis channel `sentiment:alert_live`
|
| 101 |
+
- Lovable Dashboard ΰΈ£ΰΈ±ΰΈ alert ΰΉΰΈͺΰΈΰΈ notification ΰΈΰΈ±ΰΈΰΈΰΈ΅
|
| 102 |
+
|
| 103 |
+
---
|
| 104 |
+
|
| 105 |
+
## ΰΈ£ΰΈ±ΰΈ Tests
|
| 106 |
+
|
| 107 |
+
```bash
|
| 108 |
+
pip install pytest pytest-asyncio
|
| 109 |
+
pytest tests/ -v
|
| 110 |
+
```
|
config.py
ADDED
|
@@ -0,0 +1,29 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
Configuration β ΰΈΰΉΰΈ²ΰΈΰΈΰΉΰΈ²ΰΈΰΈ²ΰΈ Environment Variables (.env)
|
| 3 |
+
"""
|
| 4 |
+
|
| 5 |
+
from pydantic_settings import BaseSettings, SettingsConfigDict
|
| 6 |
+
|
| 7 |
+
|
| 8 |
+
class Settings(BaseSettings):
|
| 9 |
+
model_config = SettingsConfigDict(env_file=".env", env_file_encoding="utf-8")
|
| 10 |
+
|
| 11 |
+
# LINE
|
| 12 |
+
LINE_CHANNEL_SECRET: str
|
| 13 |
+
LINE_CHANNEL_ACCESS_TOKEN: str
|
| 14 |
+
|
| 15 |
+
# HuggingFace
|
| 16 |
+
HF_API_TOKEN: str
|
| 17 |
+
HF_MODEL_NAME: str = "cardiffnlp/twitter-xlm-roberta-base-sentiment"
|
| 18 |
+
|
| 19 |
+
# Database
|
| 20 |
+
DATABASE_URL: str = "postgresql://user:password@localhost:5432/sentiment_db"
|
| 21 |
+
|
| 22 |
+
# Redis
|
| 23 |
+
REDIS_URL: str = "redis://localhost:6379"
|
| 24 |
+
|
| 25 |
+
# Alert threshold β negative score β₯ ΰΈΰΉΰΈ²ΰΈΰΈ΅ΰΉΰΈΰΈ° trigger alert
|
| 26 |
+
NEGATIVE_ALERT_THRESHOLD: float = 0.80
|
| 27 |
+
|
| 28 |
+
|
| 29 |
+
settings = Settings()
|
database.py
ADDED
|
@@ -0,0 +1,155 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
Database Layer β PostgreSQL (asyncpg)
|
| 3 |
+
ββββββββββββββββββββββββββββββββββββββ
|
| 4 |
+
ΰΉΰΈΰΉΰΈ conversation history ΰΉΰΈ₯ΰΈ° sentiment results
|
| 5 |
+
"""
|
| 6 |
+
|
| 7 |
+
import logging
|
| 8 |
+
import asyncpg
|
| 9 |
+
from datetime import datetime, timezone
|
| 10 |
+
from app.config import settings
|
| 11 |
+
|
| 12 |
+
logger = logging.getLogger(__name__)
|
| 13 |
+
|
| 14 |
+
|
| 15 |
+
CREATE_TABLE_SQL = """
|
| 16 |
+
CREATE TABLE IF NOT EXISTS conversations (
|
| 17 |
+
id SERIAL PRIMARY KEY,
|
| 18 |
+
conv_id TEXT NOT NULL UNIQUE,
|
| 19 |
+
user_id TEXT NOT NULL,
|
| 20 |
+
message TEXT NOT NULL,
|
| 21 |
+
source_type TEXT DEFAULT 'user',
|
| 22 |
+
timestamp BIGINT NOT NULL, -- LINE timestamp (ms)
|
| 23 |
+
created_at TIMESTAMPTZ DEFAULT NOW(),
|
| 24 |
+
|
| 25 |
+
-- Sentiment fields (nullable ΰΈΰΉΰΈΰΈ analyze ΰΉΰΈͺΰΈ£ΰΉΰΈ)
|
| 26 |
+
sentiment_label TEXT, -- positive | neutral | negative
|
| 27 |
+
sentiment_score FLOAT, -- confidence score 0β1
|
| 28 |
+
raw_scores JSONB -- {"positive": 0.8, ...}
|
| 29 |
+
);
|
| 30 |
+
|
| 31 |
+
CREATE INDEX IF NOT EXISTS idx_conv_user ON conversations(user_id);
|
| 32 |
+
CREATE INDEX IF NOT EXISTS idx_conv_created ON conversations(created_at DESC);
|
| 33 |
+
CREATE INDEX IF NOT EXISTS idx_conv_sentiment ON conversations(sentiment_label);
|
| 34 |
+
"""
|
| 35 |
+
|
| 36 |
+
|
| 37 |
+
class Database:
|
| 38 |
+
def __init__(self):
|
| 39 |
+
self._pool: asyncpg.Pool | None = None
|
| 40 |
+
|
| 41 |
+
async def connect(self):
|
| 42 |
+
self._pool = await asyncpg.create_pool(
|
| 43 |
+
dsn=settings.DATABASE_URL,
|
| 44 |
+
min_size=2,
|
| 45 |
+
max_size=10,
|
| 46 |
+
)
|
| 47 |
+
logger.info("β
Database pool created")
|
| 48 |
+
|
| 49 |
+
async def disconnect(self):
|
| 50 |
+
if self._pool:
|
| 51 |
+
await self._pool.close()
|
| 52 |
+
|
| 53 |
+
async def _pool_ok(self) -> asyncpg.Pool:
|
| 54 |
+
if not self._pool:
|
| 55 |
+
await self.connect()
|
| 56 |
+
return self._pool
|
| 57 |
+
|
| 58 |
+
# ββ Write ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 59 |
+
|
| 60 |
+
async def save_conversation(
|
| 61 |
+
self,
|
| 62 |
+
conv_id: str,
|
| 63 |
+
user_id: str,
|
| 64 |
+
message: str,
|
| 65 |
+
timestamp: int,
|
| 66 |
+
source_type: str = "user",
|
| 67 |
+
) -> dict:
|
| 68 |
+
pool = await self._pool_ok()
|
| 69 |
+
async with pool.acquire() as conn:
|
| 70 |
+
row = await conn.fetchrow(
|
| 71 |
+
"""
|
| 72 |
+
INSERT INTO conversations (conv_id, user_id, message, source_type, timestamp)
|
| 73 |
+
VALUES ($1, $2, $3, $4, $5)
|
| 74 |
+
ON CONFLICT (conv_id) DO NOTHING
|
| 75 |
+
RETURNING *
|
| 76 |
+
""",
|
| 77 |
+
conv_id, user_id, message, source_type, timestamp,
|
| 78 |
+
)
|
| 79 |
+
return dict(row) if row else {}
|
| 80 |
+
|
| 81 |
+
async def update_sentiment(
|
| 82 |
+
self,
|
| 83 |
+
conv_id: str,
|
| 84 |
+
label: str,
|
| 85 |
+
score: float,
|
| 86 |
+
raw_scores: dict,
|
| 87 |
+
) -> None:
|
| 88 |
+
import json
|
| 89 |
+
pool = await self._pool_ok()
|
| 90 |
+
async with pool.acquire() as conn:
|
| 91 |
+
await conn.execute(
|
| 92 |
+
"""
|
| 93 |
+
UPDATE conversations
|
| 94 |
+
SET sentiment_label = $1,
|
| 95 |
+
sentiment_score = $2,
|
| 96 |
+
raw_scores = $3
|
| 97 |
+
WHERE conv_id = $4
|
| 98 |
+
""",
|
| 99 |
+
label, score, json.dumps(raw_scores), conv_id,
|
| 100 |
+
)
|
| 101 |
+
|
| 102 |
+
# ββ Read βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 103 |
+
|
| 104 |
+
async def get_conversations(self, limit: int = 50, offset: int = 0) -> list[dict]:
|
| 105 |
+
pool = await self._pool_ok()
|
| 106 |
+
async with pool.acquire() as conn:
|
| 107 |
+
rows = await conn.fetch(
|
| 108 |
+
"""
|
| 109 |
+
SELECT conv_id, user_id, message, sentiment_label,
|
| 110 |
+
sentiment_score, raw_scores, created_at
|
| 111 |
+
FROM conversations
|
| 112 |
+
ORDER BY created_at DESC
|
| 113 |
+
LIMIT $1 OFFSET $2
|
| 114 |
+
""",
|
| 115 |
+
limit, offset,
|
| 116 |
+
)
|
| 117 |
+
return [dict(r) for r in rows]
|
| 118 |
+
|
| 119 |
+
async def get_sentiment_summary(self) -> dict:
|
| 120 |
+
"""ΰΈͺΰΈ£ΰΈΈΰΈΰΈ’ΰΈΰΈΰΈ£ΰΈ§ΰΈ‘ sentiment ΰΈͺΰΈ³ΰΈ«ΰΈ£ΰΈ±ΰΈ stat cards"""
|
| 121 |
+
pool = await self._pool_ok()
|
| 122 |
+
async with pool.acquire() as conn:
|
| 123 |
+
rows = await conn.fetch(
|
| 124 |
+
"""
|
| 125 |
+
SELECT
|
| 126 |
+
COUNT(*) FILTER (WHERE sentiment_label = 'positive') AS positive,
|
| 127 |
+
COUNT(*) FILTER (WHERE sentiment_label = 'neutral') AS neutral,
|
| 128 |
+
COUNT(*) FILTER (WHERE sentiment_label = 'negative') AS negative,
|
| 129 |
+
COUNT(*) AS total
|
| 130 |
+
FROM conversations
|
| 131 |
+
WHERE created_at >= NOW() - INTERVAL '7 days'
|
| 132 |
+
"""
|
| 133 |
+
)
|
| 134 |
+
row = dict(rows[0])
|
| 135 |
+
total = row["total"] or 1 # ΰΈΰΉΰΈΰΈΰΈΰΈ±ΰΈ div/0
|
| 136 |
+
return {
|
| 137 |
+
"positive": {"count": row["positive"], "pct": round(row["positive"] / total * 100, 1)},
|
| 138 |
+
"neutral": {"count": row["neutral"], "pct": round(row["neutral"] / total * 100, 1)},
|
| 139 |
+
"negative": {"count": row["negative"], "pct": round(row["negative"] / total * 100, 1)},
|
| 140 |
+
"total": row["total"],
|
| 141 |
+
"period": "last 7 days",
|
| 142 |
+
}
|
| 143 |
+
|
| 144 |
+
|
| 145 |
+
async def init_db():
|
| 146 |
+
"""ΰΈͺΰΈ£ΰΉΰΈ²ΰΈ table ΰΈΰΉΰΈ²ΰΈ’ΰΈ±ΰΈΰΉΰΈ‘ΰΉΰΈ‘ΰΈ΅ (ΰΈ£ΰΈ±ΰΈΰΈΰΈΰΈ startup)"""
|
| 147 |
+
pool = await asyncpg.create_pool(dsn=settings.DATABASE_URL)
|
| 148 |
+
async with pool.acquire() as conn:
|
| 149 |
+
await conn.execute(CREATE_TABLE_SQL)
|
| 150 |
+
await pool.close()
|
| 151 |
+
logger.info("β
Database initialized")
|
| 152 |
+
|
| 153 |
+
|
| 154 |
+
# Singleton
|
| 155 |
+
db = Database()
|
docker-compose.yml
ADDED
|
@@ -0,0 +1,49 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version: "3.9"
|
| 2 |
+
|
| 3 |
+
services:
|
| 4 |
+
|
| 5 |
+
# ββ Webhook Server ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 6 |
+
webhook:
|
| 7 |
+
build: .
|
| 8 |
+
ports:
|
| 9 |
+
- "8000:8000"
|
| 10 |
+
env_file: .env
|
| 11 |
+
depends_on:
|
| 12 |
+
postgres:
|
| 13 |
+
condition: service_healthy
|
| 14 |
+
redis:
|
| 15 |
+
condition: service_healthy
|
| 16 |
+
volumes:
|
| 17 |
+
- ./app:/app/app # hot-reload ΰΉΰΈ dev mode
|
| 18 |
+
command: uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload
|
| 19 |
+
|
| 20 |
+
# ββ PostgreSQL ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 21 |
+
postgres:
|
| 22 |
+
image: postgres:16-alpine
|
| 23 |
+
environment:
|
| 24 |
+
POSTGRES_USER: user
|
| 25 |
+
POSTGRES_PASSWORD: password
|
| 26 |
+
POSTGRES_DB: sentiment_db
|
| 27 |
+
ports:
|
| 28 |
+
- "5432:5432"
|
| 29 |
+
volumes:
|
| 30 |
+
- postgres_data:/var/lib/postgresql/data
|
| 31 |
+
healthcheck:
|
| 32 |
+
test: ["CMD-SHELL", "pg_isready -U user -d sentiment_db"]
|
| 33 |
+
interval: 5s
|
| 34 |
+
timeout: 5s
|
| 35 |
+
retries: 5
|
| 36 |
+
|
| 37 |
+
# ββ Redis βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 38 |
+
redis:
|
| 39 |
+
image: redis:7-alpine
|
| 40 |
+
ports:
|
| 41 |
+
- "6379:6379"
|
| 42 |
+
healthcheck:
|
| 43 |
+
test: ["CMD", "redis-cli", "ping"]
|
| 44 |
+
interval: 5s
|
| 45 |
+
timeout: 3s
|
| 46 |
+
retries: 5
|
| 47 |
+
|
| 48 |
+
volumes:
|
| 49 |
+
postgres_data:
|
main.py
ADDED
|
@@ -0,0 +1,188 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
LINE Official Account Webhook Server
|
| 3 |
+
ββββββββββββββββββββββββββββββββββββ
|
| 4 |
+
ΰΈ£ΰΈ±ΰΈ event ΰΈΰΈ²ΰΈ LINE β ΰΈΰΈ£ΰΈ§ΰΈΰΈͺΰΈΰΈ signature β ΰΈͺΰΉΰΈΰΈΰΉΰΈΰΉΰΈΰΈ’ΰΈ±ΰΈ HuggingFace Sentiment API
|
| 5 |
+
"""
|
| 6 |
+
|
| 7 |
+
import hashlib
|
| 8 |
+
import hmac
|
| 9 |
+
import base64
|
| 10 |
+
import logging
|
| 11 |
+
from contextlib import asynccontextmanager
|
| 12 |
+
|
| 13 |
+
from fastapi import FastAPI, Request, HTTPException, BackgroundTasks
|
| 14 |
+
from fastapi.responses import JSONResponse
|
| 15 |
+
|
| 16 |
+
from app.config import settings
|
| 17 |
+
from app.models import LineWebhookPayload
|
| 18 |
+
from app.queue import message_queue
|
| 19 |
+
from app.sentiment import analyze_sentiment
|
| 20 |
+
from app.database import db, init_db
|
| 21 |
+
|
| 22 |
+
logging.basicConfig(level=logging.INFO, format="%(asctime)s [%(levelname)s] %(message)s")
|
| 23 |
+
logger = logging.getLogger(__name__)
|
| 24 |
+
|
| 25 |
+
|
| 26 |
+
@asynccontextmanager
|
| 27 |
+
async def lifespan(app: FastAPI):
|
| 28 |
+
"""Startup / Shutdown events"""
|
| 29 |
+
logger.info("π Starting LINE Webhook Server...")
|
| 30 |
+
await init_db()
|
| 31 |
+
yield
|
| 32 |
+
logger.info("π Shutting down...")
|
| 33 |
+
|
| 34 |
+
|
| 35 |
+
app = FastAPI(
|
| 36 |
+
title="LINE Webhook Sentiment Server",
|
| 37 |
+
version="1.0.0",
|
| 38 |
+
lifespan=lifespan,
|
| 39 |
+
)
|
| 40 |
+
|
| 41 |
+
|
| 42 |
+
# βββ Signature Verification ββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 43 |
+
|
| 44 |
+
def verify_line_signature(body: bytes, signature: str) -> bool:
|
| 45 |
+
"""
|
| 46 |
+
ΰΈΰΈ£ΰΈ§ΰΈΰΈͺΰΈΰΈ X-Line-Signature header ΰΈΰΈ²ΰΈ LINE
|
| 47 |
+
ΰΈΰΉΰΈΰΈΰΈΰΈ±ΰΈ request ΰΈΰΈ₯ΰΈΰΈ‘ΰΈΰΈ΅ΰΉΰΉΰΈ‘ΰΉΰΉΰΈΰΉΰΈ‘ΰΈ²ΰΈΰΈ²ΰΈ LINE Platform
|
| 48 |
+
"""
|
| 49 |
+
channel_secret = settings.LINE_CHANNEL_SECRET.encode("utf-8")
|
| 50 |
+
hash_value = hmac.new(channel_secret, body, hashlib.sha256).digest()
|
| 51 |
+
expected = base64.b64encode(hash_value).decode("utf-8")
|
| 52 |
+
return hmac.compare_digest(expected, signature)
|
| 53 |
+
|
| 54 |
+
|
| 55 |
+
# βββ Webhook Endpoint ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 56 |
+
|
| 57 |
+
@app.post("/webhook")
|
| 58 |
+
async def line_webhook(request: Request, background_tasks: BackgroundTasks):
|
| 59 |
+
"""
|
| 60 |
+
ΰΈ£ΰΈ±ΰΈ webhook events ΰΈΰΈ²ΰΈ LINE Platform
|
| 61 |
+
|
| 62 |
+
Flow:
|
| 63 |
+
1. ΰΈΰΈ£ΰΈ§ΰΈΰΈͺΰΈΰΈ LINE Signature
|
| 64 |
+
2. Parse JSON payload
|
| 65 |
+
3. ΰΈΰΈ£ΰΈΰΈΰΉΰΈΰΈΰΈ²ΰΈ° message events (text)
|
| 66 |
+
4. ΰΈͺΰΉΰΈΰΉΰΈΰΈΰΈ£ΰΈ°ΰΈ‘ΰΈ§ΰΈ₯ΰΈΰΈ₯ sentiment ΰΉΰΈ background
|
| 67 |
+
"""
|
| 68 |
+
# 1. ΰΈΰΈΆΰΈ raw body ΰΈͺΰΈ³ΰΈ«ΰΈ£ΰΈ±ΰΈΰΈΰΈ£ΰΈ§ΰΈΰΈͺΰΈΰΈ signature
|
| 69 |
+
body = await request.body()
|
| 70 |
+
signature = request.headers.get("X-Line-Signature", "")
|
| 71 |
+
|
| 72 |
+
if not signature:
|
| 73 |
+
logger.warning("β οΈ Missing X-Line-Signature header")
|
| 74 |
+
raise HTTPException(status_code=400, detail="Missing signature")
|
| 75 |
+
|
| 76 |
+
if not verify_line_signature(body, signature):
|
| 77 |
+
logger.warning("β οΈ Invalid signature β request rejected")
|
| 78 |
+
raise HTTPException(status_code=403, detail="Invalid signature")
|
| 79 |
+
|
| 80 |
+
# 2. Parse payload
|
| 81 |
+
try:
|
| 82 |
+
payload = LineWebhookPayload.model_validate_json(body)
|
| 83 |
+
except Exception as e:
|
| 84 |
+
logger.error(f"β Failed to parse payload: {e}")
|
| 85 |
+
raise HTTPException(status_code=400, detail="Invalid payload")
|
| 86 |
+
|
| 87 |
+
# 3. Process events ΰΉΰΈ background (ΰΉΰΈ‘ΰΉ block LINE server)
|
| 88 |
+
for event in payload.events:
|
| 89 |
+
if event.type == "message" and event.message.type == "text":
|
| 90 |
+
background_tasks.add_task(process_message_event, event)
|
| 91 |
+
else:
|
| 92 |
+
logger.debug(f"Skipped event type: {event.type}")
|
| 93 |
+
|
| 94 |
+
# LINE ΰΈΰΉΰΈΰΈΰΈΰΈ²ΰΈ£ HTTP 200 ΰΈΰΈ±ΰΈΰΈΰΈ΅ ΰΉΰΈ‘ΰΉΰΈΰΈ±ΰΉΰΈ retry
|
| 95 |
+
return JSONResponse(content={"status": "ok"})
|
| 96 |
+
|
| 97 |
+
|
| 98 |
+
# βββ Background Processing βββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 99 |
+
|
| 100 |
+
async def process_message_event(event):
|
| 101 |
+
"""
|
| 102 |
+
ΰΈΰΈ£ΰΈ°ΰΈ‘ΰΈ§ΰΈ₯ΰΈΰΈ₯ΰΉΰΈΰΉΰΈ₯ΰΈ° message event:
|
| 103 |
+
1. ΰΈΰΈ±ΰΈΰΈΰΈΆΰΈ raw conversation ΰΈ₯ΰΈ DB
|
| 104 |
+
2. ΰΈͺΰΉΰΈΰΉΰΈ HuggingFace ΰΈ§ΰΈ΄ΰΉΰΈΰΈ£ΰΈ²ΰΈ°ΰΈ«ΰΉ sentiment
|
| 105 |
+
3. ΰΈΰΈ±ΰΈΰΉΰΈΰΈ result ΰΈΰΈ₯ΰΈ±ΰΈΰΉΰΈ DB
|
| 106 |
+
4. Push ΰΈΰΉΰΈ²ΰΈ queue ΰΉΰΈΰΈ’ΰΈ±ΰΈ Dashboard
|
| 107 |
+
"""
|
| 108 |
+
user_id = event.source.userId
|
| 109 |
+
msg_text = event.message.text
|
| 110 |
+
conv_id = event.replyToken or f"{user_id}_{event.timestamp}"
|
| 111 |
+
|
| 112 |
+
logger.info(f"π¨ New message | user={user_id} | text={msg_text[:50]}...")
|
| 113 |
+
|
| 114 |
+
# Step 1 β ΰΈΰΈ±ΰΈΰΈΰΈΆΰΈ conversation
|
| 115 |
+
conversation = await db.save_conversation(
|
| 116 |
+
conv_id=conv_id,
|
| 117 |
+
user_id=user_id,
|
| 118 |
+
message=msg_text,
|
| 119 |
+
timestamp=event.timestamp,
|
| 120 |
+
source_type=event.source.type,
|
| 121 |
+
)
|
| 122 |
+
|
| 123 |
+
# Step 2 β ΰΈ§ΰΈ΄ΰΉΰΈΰΈ£ΰΈ²ΰΈ°ΰΈ«ΰΉ sentiment ΰΈΰΉΰΈ²ΰΈ HuggingFace
|
| 124 |
+
sentiment_result = await analyze_sentiment(msg_text)
|
| 125 |
+
logger.info(
|
| 126 |
+
f"π§ Sentiment | label={sentiment_result.label} "
|
| 127 |
+
f"score={sentiment_result.score:.3f}"
|
| 128 |
+
)
|
| 129 |
+
|
| 130 |
+
# Step 3 β ΰΈοΏ½οΏ½οΏ½ΰΈΰΉΰΈΰΈ sentiment ΰΉΰΈ DB
|
| 131 |
+
await db.update_sentiment(
|
| 132 |
+
conv_id=conv_id,
|
| 133 |
+
label=sentiment_result.label,
|
| 134 |
+
score=sentiment_result.score,
|
| 135 |
+
raw_scores=sentiment_result.raw_scores,
|
| 136 |
+
)
|
| 137 |
+
|
| 138 |
+
# Step 4 β Push ΰΉΰΈ queue β Dashboard ΰΈ£ΰΈ±ΰΈΰΉΰΈΰΈ real-time
|
| 139 |
+
await message_queue.push({
|
| 140 |
+
"conv_id": conv_id,
|
| 141 |
+
"user_id": user_id,
|
| 142 |
+
"message": msg_text,
|
| 143 |
+
"sentiment": sentiment_result.label,
|
| 144 |
+
"score": sentiment_result.score,
|
| 145 |
+
"timestamp": event.timestamp,
|
| 146 |
+
})
|
| 147 |
+
|
| 148 |
+
# Alert ΰΈΰΉΰΈ² negative ΰΈͺΰΈΉΰΈΰΉΰΈΰΈ΄ΰΈ threshold
|
| 149 |
+
if (
|
| 150 |
+
sentiment_result.label == "negative"
|
| 151 |
+
and sentiment_result.score >= settings.NEGATIVE_ALERT_THRESHOLD
|
| 152 |
+
):
|
| 153 |
+
logger.warning(
|
| 154 |
+
f"π¨ HIGH NEGATIVE ALERT | user={user_id} | score={sentiment_result.score:.3f}"
|
| 155 |
+
)
|
| 156 |
+
await message_queue.push_alert({
|
| 157 |
+
"type": "high_negative",
|
| 158 |
+
"conv_id": conv_id,
|
| 159 |
+
"user_id": user_id,
|
| 160 |
+
"score": sentiment_result.score,
|
| 161 |
+
"message": msg_text,
|
| 162 |
+
})
|
| 163 |
+
|
| 164 |
+
|
| 165 |
+
# βββ Health & Dashboard Endpoints βββββββββββββββββββββββββββββββββββββββββββββ
|
| 166 |
+
|
| 167 |
+
@app.get("/health")
|
| 168 |
+
async def health_check():
|
| 169 |
+
"""Health check ΰΈͺΰΈ³ΰΈ«ΰΈ£ΰΈ±ΰΈ load balancer / monitoring"""
|
| 170 |
+
return {"status": "healthy", "version": app.version}
|
| 171 |
+
|
| 172 |
+
|
| 173 |
+
@app.get("/api/conversations")
|
| 174 |
+
async def get_conversations(limit: int = 50, offset: int = 0):
|
| 175 |
+
"""
|
| 176 |
+
ΰΈΰΈΆΰΈ conversation list ΰΈΰΈ£ΰΉΰΈΰΈ‘ sentiment ΰΈͺΰΈ³ΰΈ«ΰΈ£ΰΈ±ΰΈ Lovable Dashboard
|
| 177 |
+
"""
|
| 178 |
+
data = await db.get_conversations(limit=limit, offset=offset)
|
| 179 |
+
return {"conversations": data, "total": len(data)}
|
| 180 |
+
|
| 181 |
+
|
| 182 |
+
@app.get("/api/sentiment-summary")
|
| 183 |
+
async def get_sentiment_summary():
|
| 184 |
+
"""
|
| 185 |
+
ΰΈͺΰΈ£ΰΈΈΰΈ sentiment ΰΈ ΰΈ²ΰΈΰΈ£ΰΈ§ΰΈ‘ΰΈͺΰΈ³ΰΈ«ΰΈ£ΰΈ±ΰΈ stat cards ΰΈΰΈ Dashboard
|
| 186 |
+
"""
|
| 187 |
+
summary = await db.get_sentiment_summary()
|
| 188 |
+
return summary
|
models.py
ADDED
|
@@ -0,0 +1,49 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
Pydantic models β LINE Webhook Payload
|
| 3 |
+
ΰΈΰΈ£ΰΈΰΈΰΈΰΈ₯ΰΈΈΰΈ‘ event types ΰΈΰΈ΅ΰΉΰΈΰΈ³ΰΉΰΈΰΉΰΈΰΈͺΰΈ³ΰΈ«ΰΈ£ΰΈ±ΰΈ chat conversation
|
| 4 |
+
"""
|
| 5 |
+
|
| 6 |
+
from typing import Optional, Literal
|
| 7 |
+
from pydantic import BaseModel
|
| 8 |
+
|
| 9 |
+
|
| 10 |
+
# βββ Source βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 11 |
+
|
| 12 |
+
class LineSource(BaseModel):
|
| 13 |
+
type: Literal["user", "group", "room"]
|
| 14 |
+
userId: Optional[str] = None
|
| 15 |
+
groupId: Optional[str] = None
|
| 16 |
+
roomId: Optional[str] = None
|
| 17 |
+
|
| 18 |
+
|
| 19 |
+
# βββ Message ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 20 |
+
|
| 21 |
+
class LineMessage(BaseModel):
|
| 22 |
+
id: str
|
| 23 |
+
type: Literal["text", "image", "sticker", "location", "audio", "video", "file"]
|
| 24 |
+
text: Optional[str] = None # ΰΈ‘ΰΈ΅ΰΉΰΈΰΈΰΈ²ΰΈ° type=text
|
| 25 |
+
|
| 26 |
+
|
| 27 |
+
# βββ Events βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 28 |
+
|
| 29 |
+
class LineEvent(BaseModel):
|
| 30 |
+
type: Literal["message", "follow", "unfollow", "join", "leave", "postback"]
|
| 31 |
+
timestamp: int
|
| 32 |
+
replyToken: Optional[str] = None
|
| 33 |
+
source: LineSource
|
| 34 |
+
message: Optional[LineMessage] = None
|
| 35 |
+
|
| 36 |
+
|
| 37 |
+
# βββ Root Payload βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 38 |
+
|
| 39 |
+
class LineWebhookPayload(BaseModel):
|
| 40 |
+
destination: str # Channel ID ΰΈΰΈ₯ΰΈ²ΰΈ’ΰΈΰΈ²ΰΈ
|
| 41 |
+
events: list[LineEvent]
|
| 42 |
+
|
| 43 |
+
|
| 44 |
+
# βββ Sentiment Result βββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 45 |
+
|
| 46 |
+
class SentimentResult(BaseModel):
|
| 47 |
+
label: Literal["positive", "neutral", "negative"]
|
| 48 |
+
score: float # confidence ΰΈΰΈΰΈ label ΰΈ«ΰΈ₯ΰΈ±ΰΈ (0β1)
|
| 49 |
+
raw_scores: dict[str, float] # {"positive": 0.8, "neutral": 0.1, "negative": 0.1}
|
queue.py
ADDED
|
@@ -0,0 +1,80 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
Message Queue β Redis
|
| 3 |
+
βββββββββββββββββββββ
|
| 4 |
+
Push sentiment results β Dashboard (Lovable AI) ΰΈ£ΰΈ±ΰΈΰΉΰΈΰΈ real-time
|
| 5 |
+
|
| 6 |
+
ΰΉΰΈΰΉ Redis List + Pub/Sub:
|
| 7 |
+
- LPUSH sentiment:results β queue ΰΈͺΰΈ³ΰΈ«ΰΈ£ΰΈ±ΰΈ polling
|
| 8 |
+
- PUBLISH sentiment:live β pub/sub ΰΈͺΰΈ³ΰΈ«ΰΈ£ΰΈ±ΰΈ WebSocket / SSE
|
| 9 |
+
"""
|
| 10 |
+
|
| 11 |
+
import json
|
| 12 |
+
import logging
|
| 13 |
+
import redis.asyncio as aioredis
|
| 14 |
+
from app.config import settings
|
| 15 |
+
|
| 16 |
+
logger = logging.getLogger(__name__)
|
| 17 |
+
|
| 18 |
+
QUEUE_KEY = "sentiment:results"
|
| 19 |
+
ALERT_KEY = "sentiment:alerts"
|
| 20 |
+
PUBSUB_CH = "sentiment:live"
|
| 21 |
+
ALERT_CH = "sentiment:alert_live"
|
| 22 |
+
|
| 23 |
+
|
| 24 |
+
class MessageQueue:
|
| 25 |
+
def __init__(self):
|
| 26 |
+
self._redis: aioredis.Redis | None = None
|
| 27 |
+
|
| 28 |
+
async def _get_redis(self) -> aioredis.Redis:
|
| 29 |
+
if self._redis is None:
|
| 30 |
+
self._redis = await aioredis.from_url(
|
| 31 |
+
settings.REDIS_URL,
|
| 32 |
+
encoding="utf-8",
|
| 33 |
+
decode_responses=True,
|
| 34 |
+
)
|
| 35 |
+
return self._redis
|
| 36 |
+
|
| 37 |
+
async def push(self, data: dict) -> None:
|
| 38 |
+
"""
|
| 39 |
+
Push sentiment result ΰΉΰΈΰΉΰΈ² queue ΰΉΰΈ₯ΰΈ° publish ΰΉΰΈ«ΰΉ subscribers
|
| 40 |
+
"""
|
| 41 |
+
try:
|
| 42 |
+
r = await self._get_redis()
|
| 43 |
+
serialized = json.dumps(data, ensure_ascii=False)
|
| 44 |
+
|
| 45 |
+
# List (Lovable polling ΰΈ«ΰΈ£ΰΈ·ΰΈ worker ΰΈΰΈΆΰΈΰΉΰΈΰΈΰΈ£ΰΈ°ΰΈ‘ΰΈ§ΰΈ₯ΰΈΰΈ₯ΰΈΰΉΰΈ)
|
| 46 |
+
await r.lpush(QUEUE_KEY, serialized)
|
| 47 |
+
await r.ltrim(QUEUE_KEY, 0, 999) # ΰΉΰΈΰΉΰΈΰΉΰΈΰΉ 1,000 ΰΈ£ΰΈ²ΰΈ’ΰΈΰΈ²ΰΈ£ΰΈ₯ΰΉΰΈ²ΰΈͺΰΈΈΰΈ
|
| 48 |
+
|
| 49 |
+
# Pub/Sub (real-time Dashboard WebSocket)
|
| 50 |
+
await r.publish(PUBSUB_CH, serialized)
|
| 51 |
+
|
| 52 |
+
logger.debug(f"β
Queued: conv_id={data.get('conv_id')}")
|
| 53 |
+
|
| 54 |
+
except Exception as e:
|
| 55 |
+
logger.error(f"β Queue push failed: {e}")
|
| 56 |
+
|
| 57 |
+
async def push_alert(self, data: dict) -> None:
|
| 58 |
+
"""Push high-negative alert ΰΉΰΈ’ΰΈ channel"""
|
| 59 |
+
try:
|
| 60 |
+
r = await self._get_redis()
|
| 61 |
+
serialized = json.dumps(data, ensure_ascii=False)
|
| 62 |
+
await r.lpush(ALERT_KEY, serialized)
|
| 63 |
+
await r.publish(ALERT_CH, serialized)
|
| 64 |
+
logger.warning(f"π¨ Alert pushed: {data}")
|
| 65 |
+
except Exception as e:
|
| 66 |
+
logger.error(f"β Alert push failed: {e}")
|
| 67 |
+
|
| 68 |
+
async def get_recent(self, count: int = 50) -> list[dict]:
|
| 69 |
+
"""ΰΈΰΈΆΰΈ result ΰΈ₯ΰΉΰΈ²ΰΈͺΰΈΈΰΈΰΈΰΈ²ΰΈ queue ΰΈͺΰΈ³ΰΈ«ΰΈ£ΰΈ±ΰΈ initial load"""
|
| 70 |
+
try:
|
| 71 |
+
r = await self._get_redis()
|
| 72 |
+
items = await r.lrange(QUEUE_KEY, 0, count - 1)
|
| 73 |
+
return [json.loads(i) for i in items]
|
| 74 |
+
except Exception as e:
|
| 75 |
+
logger.error(f"β Queue read failed: {e}")
|
| 76 |
+
return []
|
| 77 |
+
|
| 78 |
+
|
| 79 |
+
# Singleton instance
|
| 80 |
+
message_queue = MessageQueue()
|
requirements.txt
ADDED
|
@@ -0,0 +1,8 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
fastapi==0.115.5
|
| 2 |
+
uvicorn[standard]==0.32.1
|
| 3 |
+
httpx==0.28.0
|
| 4 |
+
pydantic==2.10.3
|
| 5 |
+
pydantic-settings==2.6.1
|
| 6 |
+
asyncpg==0.30.0
|
| 7 |
+
redis==5.2.1
|
| 8 |
+
python-dotenv==1.0.1
|
sentiment.py
ADDED
|
@@ -0,0 +1,93 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
HuggingFace Sentiment Analysis
|
| 3 |
+
βββββββββββββββββββββββββββββ
|
| 4 |
+
ΰΉΰΈ£ΰΈ΅ΰΈ’ΰΈ HuggingFace Inference API ΰΉΰΈΰΈ·ΰΉΰΈΰΈ§ΰΈ΄ΰΉΰΈΰΈ£ΰΈ²ΰΈ°ΰΈ«ΰΉΰΈΰΈ§ΰΈ²ΰΈ‘ΰΈ£ΰΈΉΰΉΰΈͺΰΈΆΰΈΰΈΰΈΰΈΰΈΰΉΰΈΰΈΰΈ§ΰΈ²ΰΈ‘ΰΈ ΰΈ²ΰΈ©ΰΈ²ΰΉΰΈΰΈ’
|
| 5 |
+
|
| 6 |
+
Model ΰΈΰΈ΅ΰΉΰΉΰΈΰΈ°ΰΈΰΈ³:
|
| 7 |
+
- airesearch/wangchanberta-base-att-spm-uncased (ΰΈ ΰΈ²ΰΈ©ΰΈ²ΰΉΰΈΰΈ’ ΰΈΰΈ±ΰΉΰΈ§ΰΉΰΈ)
|
| 8 |
+
- pythainlp/thai-sentiment-analysis (ΰΈΰΉΰΈ²ΰΈ‘ΰΈ΅ fine-tune)
|
| 9 |
+
- ΰΈΰΈ²ΰΈΰΉΰΈΰΉ multilingual model: cardiffnlp/twitter-xlm-roberta-base-sentiment
|
| 10 |
+
"""
|
| 11 |
+
|
| 12 |
+
import httpx
|
| 13 |
+
import logging
|
| 14 |
+
from app.config import settings
|
| 15 |
+
from app.models import SentimentResult
|
| 16 |
+
|
| 17 |
+
logger = logging.getLogger(__name__)
|
| 18 |
+
|
| 19 |
+
HF_API_URL = (
|
| 20 |
+
f"https://api-inference.huggingface.co/models/{settings.HF_MODEL_NAME}"
|
| 21 |
+
)
|
| 22 |
+
|
| 23 |
+
# Label mapping ΰΈΰΈ²ΰΈ model output β ΰΈΰΈ·ΰΉΰΈ standard
|
| 24 |
+
LABEL_MAP = {
|
| 25 |
+
# WangchanBERTa / Thai models
|
| 26 |
+
"pos": "positive",
|
| 27 |
+
"neu": "neutral",
|
| 28 |
+
"neg": "negative",
|
| 29 |
+
# cardiffnlp / XLM-RoBERTa
|
| 30 |
+
"positive": "positive",
|
| 31 |
+
"neutral": "neutral",
|
| 32 |
+
"negative": "negative",
|
| 33 |
+
# LABEL_0/1/2 fallback (ΰΈΰΈΆΰΉΰΈΰΈΰΈ±ΰΈ model)
|
| 34 |
+
"LABEL_0": "negative",
|
| 35 |
+
"LABEL_1": "neutral",
|
| 36 |
+
"LABEL_2": "positive",
|
| 37 |
+
}
|
| 38 |
+
|
| 39 |
+
|
| 40 |
+
async def analyze_sentiment(text: str) -> SentimentResult:
|
| 41 |
+
"""
|
| 42 |
+
ΰΈͺΰΉΰΈΰΈΰΉΰΈΰΈΰΈ§ΰΈ²ΰΈ‘ΰΉΰΈΰΈ§ΰΈ΄ΰΉΰΈΰΈ£ΰΈ²ΰΈ°ΰΈ«ΰΉΰΈΰΈ΅ΰΉ HuggingFace ΰΉΰΈ₯ΰΉΰΈ§ΰΈΰΈ·ΰΈ SentimentResult
|
| 43 |
+
|
| 44 |
+
HuggingFace Inference API ΰΈΰΈ·ΰΈΰΈ£ΰΈΉΰΈΰΉΰΈΰΈ:
|
| 45 |
+
[[{"label": "pos", "score": 0.91}, {"label": "neg", "score": 0.05}, ...]]
|
| 46 |
+
"""
|
| 47 |
+
headers = {"Authorization": f"Bearer {settings.HF_API_TOKEN}"}
|
| 48 |
+
payload = {"inputs": text, "options": {"wait_for_model": True}}
|
| 49 |
+
|
| 50 |
+
try:
|
| 51 |
+
async with httpx.AsyncClient(timeout=15.0) as client:
|
| 52 |
+
response = await client.post(HF_API_URL, headers=headers, json=payload)
|
| 53 |
+
response.raise_for_status()
|
| 54 |
+
data = response.json()
|
| 55 |
+
|
| 56 |
+
# Flatten nested list ΰΈΰΈ΅ΰΉ HF ΰΈΰΈ·ΰΈΰΈ‘ΰΈ²
|
| 57 |
+
scores = data[0] if isinstance(data[0], list) else data
|
| 58 |
+
|
| 59 |
+
# ΰΉΰΈΰΈ₯ΰΈ label ΰΉΰΈ₯ΰΈ°ΰΈͺΰΈ£ΰΉΰΈ²ΰΈ raw_scores dict
|
| 60 |
+
raw_scores: dict[str, float] = {}
|
| 61 |
+
for item in scores:
|
| 62 |
+
normalized = LABEL_MAP.get(item["label"], item["label"].lower())
|
| 63 |
+
raw_scores[normalized] = round(item["score"], 4)
|
| 64 |
+
|
| 65 |
+
# ΰΈ«ΰΈ² label ΰΈΰΈ΅ΰΉΰΈ‘ΰΈ΅ score ΰΈͺΰΈΉΰΈΰΈͺΰΈΈΰΈ
|
| 66 |
+
best_label = max(raw_scores, key=raw_scores.get)
|
| 67 |
+
|
| 68 |
+
return SentimentResult(
|
| 69 |
+
label=best_label,
|
| 70 |
+
score=raw_scores[best_label],
|
| 71 |
+
raw_scores=raw_scores,
|
| 72 |
+
)
|
| 73 |
+
|
| 74 |
+
except httpx.HTTPStatusError as e:
|
| 75 |
+
logger.error(f"HuggingFace API error {e.response.status_code}: {e.response.text}")
|
| 76 |
+
return _fallback_result()
|
| 77 |
+
|
| 78 |
+
except httpx.TimeoutException:
|
| 79 |
+
logger.error("HuggingFace API timeout")
|
| 80 |
+
return _fallback_result()
|
| 81 |
+
|
| 82 |
+
except Exception as e:
|
| 83 |
+
logger.error(f"Unexpected error in sentiment analysis: {e}")
|
| 84 |
+
return _fallback_result()
|
| 85 |
+
|
| 86 |
+
|
| 87 |
+
def _fallback_result() -> SentimentResult:
|
| 88 |
+
"""ΰΈΰΈ·ΰΈ neutral ΰΉΰΈ‘ΰΈ·ΰΉΰΈ API ΰΈ₯ΰΉΰΈ‘ΰΉΰΈ«ΰΈ₯ΰΈ§ ΰΉΰΈΰΈ·ΰΉΰΈΰΉΰΈ‘ΰΉΰΉΰΈ«ΰΉΰΈ£ΰΈ°ΰΈΰΈΰΈ«ΰΈ’ΰΈΈΰΈ"""
|
| 89 |
+
return SentimentResult(
|
| 90 |
+
label="neutral",
|
| 91 |
+
score=0.0,
|
| 92 |
+
raw_scores={"positive": 0.0, "neutral": 0.0, "negative": 0.0},
|
| 93 |
+
)
|
test_webhook.py
ADDED
|
@@ -0,0 +1,97 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
Unit Tests β Webhook Server
|
| 3 |
+
"""
|
| 4 |
+
|
| 5 |
+
import hashlib
|
| 6 |
+
import hmac
|
| 7 |
+
import base64
|
| 8 |
+
import pytest
|
| 9 |
+
from fastapi.testclient import TestClient
|
| 10 |
+
from unittest.mock import AsyncMock, patch
|
| 11 |
+
|
| 12 |
+
|
| 13 |
+
# βββ Signature Helper ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 14 |
+
|
| 15 |
+
def make_signature(body: bytes, secret: str) -> str:
|
| 16 |
+
h = hmac.new(secret.encode(), body, hashlib.sha256).digest()
|
| 17 |
+
return base64.b64encode(h).decode()
|
| 18 |
+
|
| 19 |
+
|
| 20 |
+
# βββ Fixture: mock settings βββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 21 |
+
|
| 22 |
+
@pytest.fixture(autouse=True)
|
| 23 |
+
def mock_settings(monkeypatch):
|
| 24 |
+
monkeypatch.setenv("LINE_CHANNEL_SECRET", "test_secret")
|
| 25 |
+
monkeypatch.setenv("LINE_CHANNEL_ACCESS_TOKEN", "test_token")
|
| 26 |
+
monkeypatch.setenv("HF_API_TOKEN", "hf_test")
|
| 27 |
+
monkeypatch.setenv("DATABASE_URL", "postgresql://u:p@localhost/test")
|
| 28 |
+
monkeypatch.setenv("REDIS_URL", "redis://localhost:6379")
|
| 29 |
+
|
| 30 |
+
|
| 31 |
+
# βββ Tests ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 32 |
+
|
| 33 |
+
class TestSignatureVerification:
|
| 34 |
+
def test_valid_signature(self):
|
| 35 |
+
from app.main import verify_line_signature
|
| 36 |
+
body = b'{"test": "data"}'
|
| 37 |
+
sig = make_signature(body, "test_secret")
|
| 38 |
+
assert verify_line_signature(body, sig) is True
|
| 39 |
+
|
| 40 |
+
def test_invalid_signature(self):
|
| 41 |
+
from app.main import verify_line_signature
|
| 42 |
+
body = b'{"test": "data"}'
|
| 43 |
+
assert verify_line_signature(body, "wrong_sig") is False
|
| 44 |
+
|
| 45 |
+
def test_empty_signature(self):
|
| 46 |
+
from app.main import verify_line_signature
|
| 47 |
+
assert verify_line_signature(b"body", "") is False
|
| 48 |
+
|
| 49 |
+
|
| 50 |
+
class TestPayloadParsing:
|
| 51 |
+
def test_parse_text_message(self):
|
| 52 |
+
from app.models import LineWebhookPayload
|
| 53 |
+
raw = """{
|
| 54 |
+
"destination": "U123",
|
| 55 |
+
"events": [{
|
| 56 |
+
"type": "message",
|
| 57 |
+
"timestamp": 1700000000000,
|
| 58 |
+
"replyToken": "token123",
|
| 59 |
+
"source": {"type": "user", "userId": "Uabc"},
|
| 60 |
+
"message": {"id": "m1", "type": "text", "text": "ΰΈͺΰΈ§ΰΈ±ΰΈͺΰΈΰΈ΅ΰΈΰΈ£ΰΈ±ΰΈ"}
|
| 61 |
+
}]
|
| 62 |
+
}"""
|
| 63 |
+
payload = LineWebhookPayload.model_validate_json(raw)
|
| 64 |
+
assert len(payload.events) == 1
|
| 65 |
+
assert payload.events[0].message.text == "ΰΈͺΰΈ§ΰΈ±ΰΈͺΰΈΰΈ΅ΰΈΰΈ£ΰΈ±ΰΈ"
|
| 66 |
+
|
| 67 |
+
def test_skip_non_text_event(self):
|
| 68 |
+
from app.models import LineWebhookPayload
|
| 69 |
+
raw = """{
|
| 70 |
+
"destination": "U123",
|
| 71 |
+
"events": [{
|
| 72 |
+
"type": "follow",
|
| 73 |
+
"timestamp": 1700000000000,
|
| 74 |
+
"source": {"type": "user", "userId": "Uabc"}
|
| 75 |
+
}]
|
| 76 |
+
}"""
|
| 77 |
+
payload = LineWebhookPayload.model_validate_json(raw)
|
| 78 |
+
assert payload.events[0].type == "follow"
|
| 79 |
+
assert payload.events[0].message is None
|
| 80 |
+
|
| 81 |
+
|
| 82 |
+
class TestSentimentResult:
|
| 83 |
+
def test_label_mapping_positive(self):
|
| 84 |
+
from app.models import SentimentResult
|
| 85 |
+
r = SentimentResult(
|
| 86 |
+
label="positive",
|
| 87 |
+
score=0.91,
|
| 88 |
+
raw_scores={"positive": 0.91, "neutral": 0.07, "negative": 0.02},
|
| 89 |
+
)
|
| 90 |
+
assert r.label == "positive"
|
| 91 |
+
assert r.score == 0.91
|
| 92 |
+
|
| 93 |
+
def test_fallback_is_neutral(self):
|
| 94 |
+
from app.sentiment import _fallback_result
|
| 95 |
+
r = _fallback_result()
|
| 96 |
+
assert r.label == "neutral"
|
| 97 |
+
assert r.score == 0.0
|