Fetching metadata from the HF Docker repository...
-
Wave 3: expand NER fixture 85->102, time_preference F1 0.40->0.98, people/duration fixes
- NER fixture: 85 -> 102 samples (+12 time_preference + 5 people edge cases)
- time_preference F1: 0.40 -> 0.98 (was 0.00 in Wave 1)
* Add 'buoi sang som' -> early_morning keyword (highest priority)
* VI compound scrubbing: 'an sang' (breakfast), 'cho dem' (night market VI)
* EN compound scrubbing: night market/tour/life/show/club/bus
* Fix 4 gold annotation gaps (buoi toi, hoang hon, buoi chieu samples)
- duration: add EN natural-language week patterns (a week, one/two/three weeks)
- people: add VI vo chong, co N tre em; EN group of N, N of us, solo word boundary
- location: extend FP blocklist (temple, shrine, sunrise, batu, anh, market, beach)
* Location FP: 24 -> 7 (also fixed dual location+destination annotation in fixture)
- NER micro F1: 0.912 (85 samples) -> 0.920 (102 samples)
- evaluation_report.md: add Wave 3 section with full metrics table -
Wave 2: compare_destinations data + NER fixes + evaluation update
Intent:
- Add 120 compare_destinations samples (80 VI + 40 EN) to intent_train_enhanced.json
- Re-split: train=5102, val=1094, test=1094 (seed=42)
- Retrain: macro F1 0.8661 -> 0.9412 on 1094 held-out samples
- compare_destinations: F1 0.00 -> 0.92
NER model.py:
- origin: HIGH_CONF end-of-sentence 'tu X' pattern + LOW_CONF gate when 'tu' present
- hotel_type: suppress 'accommodation' in budget-allocation context (tiet kiem, chi nhieu)
- hotel_type: suppress EN 'budget' when monetary amount present
- budget: add 'ruoi' (3 trieu ruoi = 3.5M VND) pattern
- budget_priority: broaden VI patterns (tiet kiem tien an, chi nhieu hon cho, uu tien)
- cuisine: skip cuisine keyword when dietary negation phrase overlaps
- transport: add 'mrt'/'gan mrt' to VI keywords
- time_preference: suppress short-keyword matches in duration context (2 dem, 2 nights)
- _match_keyword_dict: consumed-text approach prevents substring double-match
- FALSE_POSITIVE_BLOCKLIST: add 'massa'/'massage'
evaluate.py:
- Add … -
eval(chatbot): Wave 1 retrain + held-out evaluation results
- Produced intent_test.json / intent_val.json via split_dataset.py
(stratified 15/15/70 split, seed=42, 7170 total samples, 12 intents)
- Retrained IntentClassifier with FeatureUnion pipeline:
Held-out accuracy=0.9377, macro F1=0.8661
greeting precision: 0.59 -> 0.72 (OOD rejector working)
food<->activity confusion: 31 -> 9 errors (word TF-IDF working)
- NER re-evaluated after regex fixes:
Micro F1: 0.903 -> 0.923
hotel_type F1: 0.40 -> 0.80 (priority-based selection)
dietary F1: 0.88 -> 0.94 (broadened alias)
- Updated evaluation_report.md with all Wave 1 results
- Updated eval_intent.json and eval_ner.json
