Fetching metadata from the HF Docker repository...
-
Wave 2: compare_destinations data + NER fixes + evaluation update
Intent:
- Add 120 compare_destinations samples (80 VI + 40 EN) to intent_train_enhanced.json
- Re-split: train=5102, val=1094, test=1094 (seed=42)
- Retrain: macro F1 0.8661 -> 0.9412 on 1094 held-out samples
- compare_destinations: F1 0.00 -> 0.92
NER model.py:
- origin: HIGH_CONF end-of-sentence 'tu X' pattern + LOW_CONF gate when 'tu' present
- hotel_type: suppress 'accommodation' in budget-allocation context (tiet kiem, chi nhieu)
- hotel_type: suppress EN 'budget' when monetary amount present
- budget: add 'ruoi' (3 trieu ruoi = 3.5M VND) pattern
- budget_priority: broaden VI patterns (tiet kiem tien an, chi nhieu hon cho, uu tien)
- cuisine: skip cuisine keyword when dietary negation phrase overlaps
- transport: add 'mrt'/'gan mrt' to VI keywords
- time_preference: suppress short-keyword matches in duration context (2 dem, 2 nights)
- _match_keyword_dict: consumed-text approach prevents substring double-match
- FALSE_POSITIVE_BLOCKLIST: add 'massa'/'massage'
evaluate.py:
- Add … -
eval(chatbot): Wave 1 retrain + held-out evaluation results
- Produced intent_test.json / intent_val.json via split_dataset.py
(stratified 15/15/70 split, seed=42, 7170 total samples, 12 intents)
- Retrained IntentClassifier with FeatureUnion pipeline:
Held-out accuracy=0.9377, macro F1=0.8661
greeting precision: 0.59 -> 0.72 (OOD rejector working)
food<->activity confusion: 31 -> 9 errors (word TF-IDF working)
- NER re-evaluated after regex fixes:
Micro F1: 0.903 -> 0.923
hotel_type F1: 0.40 -> 0.80 (priority-based selection)
dietary F1: 0.88 -> 0.94 (broadened alias)
- Updated evaluation_report.md with all Wave 1 results
- Updated eval_intent.json and eval_ner.json
