Fetching metadata from the HF Docker repository...
-
Wave 2: compare_destinations data + NER fixes + evaluation update
Intent:
- Add 120 compare_destinations samples (80 VI + 40 EN) to intent_train_enhanced.json
- Re-split: train=5102, val=1094, test=1094 (seed=42)
- Retrain: macro F1 0.8661 -> 0.9412 on 1094 held-out samples
- compare_destinations: F1 0.00 -> 0.92
NER model.py:
- origin: HIGH_CONF end-of-sentence 'tu X' pattern + LOW_CONF gate when 'tu' present
- hotel_type: suppress 'accommodation' in budget-allocation context (tiet kiem, chi nhieu)
- hotel_type: suppress EN 'budget' when monetary amount present
- budget: add 'ruoi' (3 trieu ruoi = 3.5M VND) pattern
- budget_priority: broaden VI patterns (tiet kiem tien an, chi nhieu hon cho, uu tien)
- cuisine: skip cuisine keyword when dietary negation phrase overlaps
- transport: add 'mrt'/'gan mrt' to VI keywords
- time_preference: suppress short-keyword matches in duration context (2 dem, 2 nights)
- _match_keyword_dict: consumed-text approach prevents substring double-match
- FALSE_POSITIVE_BLOCKLIST: add 'massa'/'massage'
evaluate.py:
- Add …
