The terrain/theme redirect used plain substring matching, so 'eco'
matched 'economy' and 'city' matched 'electricity'/'ethnicity', sending
unrelated questions to personalize_recommend. Use the same Unicode
whole-phrase regex (_phrase_regex on NFC-lowercased text) as the food
and suggestion keywords, and add plural forms that substring matching
used to cover implicitly (beaches, islands, mountains, cities, ...).
fix(chatbot-ml): cap request sizes on all ML service endpoints
PredictRequest: bounded ids/language, context limited to 50 keys and
16k chars of JSON, conversation_history turns must be user/assistant
with string content <= 4000 chars (max 20 turns, message <= 2000 chars).
Recommend/plan/multi-destination/compare/budget requests: bounded list
lengths, string lengths, trip days (<= 60), party size (<= 50), month
range and free-form dict sizes. Over-limit input is rejected with 422.
fix(chatbot-ml): apply /predict guardrails to /predict/stream
Factor the pre-generation routing (dialog-state re-routing of follow-up
answers, out-of-domain math guard, off-topic guardrail) into a shared
_pre_generation_guard() helper used by both /predict and /predict/stream.
A blocked stream now emits a single 'done' event carrying the guardrail
reply instead of generating an answer. The stream also clears/sets the
'awaiting' follow-up flag like /predict and no longer leaks exception
text in its error event.
fix(chatbot-ml): party sizes like "2+1 người" / "phòng 2+1" are not arithmetic
The '+' operator rule sent party-size and room answers to the off-topic
reply. The math guard now never fires when the message mentions people,
guests, rooms, nights, days, tickets, seats, beds or stars (vi + en),
matched as whole words.
fix(chatbot-ml): match food / suggestion keywords as whole phrases
Keywords were plain substrings, so "ăn gì" matched inside "văn gì" /
"khăn gì" and bypassed the off-topic guardrail. Match with a
Unicode-aware (?<!\w)...(?!\w) regex on NFC-normalised text (inner
spaces match any whitespace). Replace the bare "must try" (also in "you
must try harder") and the "food special" prefix with explicit food
phrases.
fix(chatbot-ml): flag invented map waypoints as approximate
When a destination's activities have no coordinates, the map card places
them evenly on a ~2.5 km circle around the city centre; the client drew
them as exact pins and offered a coordinate route. Mark each synthesized
point with approximate: true so the frontend can render it differently.
fix(chatbot-ml): guardrail replies use real PredictResponse fields
Both guardrail branches passed context= and generation_method=, which
pydantic silently dropped (the fields are updatedContext and
generationMethod), so guardrail turns returned an empty context and
generationMethod "template", and a stale context['awaiting'] was never
cleared.
- Build guardrail replies in one helper that sets
updatedContext={**context, 'awaiting': None} and generationMethod='guardrail'.
- Top-level response models now use extra='forbid' so wrong kwargs raise
instead of being dropped (EntityResult left lenient: NER may add keys).
- Tests for the field values, stale-awaiting clearing, strictness and
serialization through the real /api/predict route.
fix(chatbot-ml): let food / destination-suggestion questions past the off-topic guardrail
The guardrail returned before the food_recommend and personalize_recommend
overrides ran, so low-confidence 'fallback' queries such as "nên đi đâu",
"gợi ý điểm đến mùa hè" or "ăn gì ngon" always got the off-topic reply.
- Hoist the food / suggestion keyword lists to module constants and count
them as a travel signal in the guardrail.
- Never guardrail a turn whose intent was set by dialog-state routing.
- Apply the overrides to any non-travel classifier label (e.g. goodbye),
not only 'fallback'.
- Tests for both overrides and for a genuinely off-topic message.
fix(chatbot-ml): math guard no longer swallows slot-filling replies
The out-of-domain arithmetic guard matched any "digit op digit" pattern,
so answers such as "2-3 triệu", "3-4 sao", "2-3 người" or "20/11" were
answered with the off-topic message -- even when they were the reply to
the bot's own follow-up question.
- Only +, *, × count as arithmetic on their own; -, /, x, : only when the
message also asks for a result ("=", "bằng mấy", "bằng bao nhiêu",
"equals").
- Skip the guard when context['awaiting'] is set or dialog-state routing
already overrode the intent.
- Add predict() routing tests with stubbed models.
fix(chatbot): destination-suggestion + map follow-up wrongly hit off-topic guardrail
- nearby_attractions/destination_info/travel_tips/transport_info/optimize_budget
were missing from _TRAVEL_INTENTS, so low-confidence travel queries like
'Gợi ý điểm đến mùa hè' were rejected by the off-topic guardrail. Add them.
- Route 'gợi ý điểm đến / nên đi đâu' (no specific place) to personalize_recommend
so it suggests destinations instead of an empty nearby_attractions prompt.
- show_map reply asks 'show nearby attractions too?'; set awaiting='show_attractions'
so a 'Có' answer routes to nearby_attractions instead of the guardrail.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
fix(chatbot): food queries misclassified as activities
food_recommend was trained mostly on food-ordering/delivery samples, so
travel-food queries ("Ăn gì ở Hà Nội", "Ẩm thực đặc sắc tại Hà Nội",
"What should I eat in Hanoi") were misclassified as activity_suggest/
destination_info — notably for Hà Nội.
- Augment intent_train.json with 151 travel-food examples (VI+EN templates ×
cities, excluding the SC-I-17/18 test strings to avoid leakage). Held-out
val accuracy stable (92.05%→92.14%); all 20 SC-I scenarios now pass (was 18).
- Add an application-layer food-intent guard in predict() as a safety net.
Model is gitignored and retrained from this dataset on Space startup.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
fix(chatbot): set awaiting after nearby_attractions so 'có' plans a trip
The nearby_attractions reply asks 'lên lịch trình kết hợp các điểm này không?'
but did not set context.awaiting, so a 'có' answer was classified blindly and
hit the off-topic guardrail. Treat nearby_attractions like activity_suggest
(awaiting='activity_selection') → an affirmative now routes to plan_trip.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
feat(chatbot): surface real hotels and attractions from KB
- find_hotel: after criteria, list top 3 hotels (name, price, rating) from the
destination's top_hotels_booking instead of only pointing to the website
- nearby_attractions: render the actual attraction list (name + cost) from
popular_activities in the message text (the rich card alone wasn't showing)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- dietary follow-ups now stay in the dietary flow (awaiting='dietary' kept),
so 'dị ứng tinh bột nữa' after 'dị ứng hải sản' is no longer rejected by the
off-topic guardrail; restrictions accumulate across turns and the food list
is filtered by all of them
- NER recognises 'tinh bột' (no_starch)
- weather_info now fetches real-time current weather via RealtimeDataService
and falls back to static KB climate if unavailable
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
feat(chatbot): OOD guard for arithmetic + find_hotel slot-filling
- Hard out-of-domain guard: arithmetic like '1+1 bằng mấy?' now returns the
off-topic response instead of being mislabelled as find_hotel (or erroring)
- find_hotel uses a direct template (skips the LLM call) to fix the 20-30s
latency, and sets context.awaiting='hotel_criteria'
- the follow-up criteria reply ('4 sao, 5 triệu...') is acknowledged and routed
to the hotel search instead of repeating the same clarifying question
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
feat(chatbot): lightweight dialog-state for follow-up replies
- Track context['awaiting'] after the bot asks an activity/dietary question;
interpret the next short reply in that context instead of classifying blindly
- 'tất cả' (and affirmatives) after an activity list now build a trip plan
instead of being misread as 'thank' → goodbye
- dietary replies (e.g. 'dị ứng tôm và thịt bò') route to a dietary-aware
response that filters the destination's food list by allergen (no LLM, no
external cuisine DB needed)
- NER: recognise tôm/thịt bò allergens (no_seafood/no_beef)
- always reset 'awaiting' (incl. None) so server/client context merges clear it
Context already round-trips via backend conversation + ChatWindow, so no FE change.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
fix(chatbot): simulate multi-pin route map for destinations without per-activity coords
- Add HotelMigration CommandLineRunner to insert "Free WiFi" into all
live MongoDB hotel documents missing it via $addToSet at startup
- Add "Free WiFi" to all 129 hotels in hotels.json seed data that lacked it
- Fix chatbot map search_query using full user text instead of destination name
- Add lat/lon coordinates to activity objects in TripOptimizer output
- Build route waypoints for map card from trip plan activity coordinates,
replacing single-pin map with multi-stop route visualization
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add module-level constants (_TRAVEL_INTENTS, _TRAVEL_KEYWORDS_RE, _TERRAIN_KEYWORDS,
_LOCATION_REQUIRED_INTENTS, _DESTINATION_CARD_INTENTS, _INTENT_DISPLAY_MAP) compiled
ONCE at import time instead of being recreated on every request
- Add _load_user_memory, _classify_intent_safe, _extract_entities_safe helpers and
run them via asyncio.gather — cache miss now triggers 3-way parallel IO
(memory load + intent classification + NER) instead of sequential awaits
- Extract _build_destination_info_card, _build_nearby_attractions_card,
_build_travel_tips_card, _build_destination_extra_cards as pure helpers —
individually testable and reusable by predict_stream
- predict() body: 851 → 648 lines (-203 lines, ~24% reduction)
chatbot-ml-service/app/models/intent_classifier/model.py:
- Add Counter-based validation before train_test_split:
raises ValueError if < 2 classes or texts/labels length mismatch
- Compute _can_split flag: requires min_per_class >= 2 AND
total >= max(7, n_classes*2) so stratify never crashes
- When _can_split=False: train on full set, skip eval, return
{accuracy: None, note: '...'} instead of crashing
- Add warnings.warn so operator sees the degraded-mode message
chatbot-ml-service/app/api/routes.py:
- Add module-level _BACKGROUND_TASKS: set to hold strong refs
to fire-and-forget asyncio.create_task() tasks
- Each task adds itself to _BACKGROUND_TASKS before scheduling
- Done-callback calls _BACKGROUND_TASKS.discard(task) to release
the reference once the task completes or is cancelled
- Prevents CPython GC from collecting tasks before the event loop
runs them (seen as silent no-op memory update drops)
fix: resolve remaining HIGH/MEDIUM issues from code review
Chatbot ML Service:
- Replace all print() calls with logger.info/warning in load_models() startup path
- Remove LLM provider name and model ID from unauthenticated /health endpoint;
replace with generic llm_ready boolean to prevent vendor enumeration
- Reuse shared httpx.AsyncClient across weather requests (lazy-init property +
close() called in lifespan shutdown); eliminates per-request TCP handshake
- Cap ChromaDB query input at 256 chars before embedding for retrieval quality
- Add optional language param to IntentClassifier.predict() to skip redundant
language detection when caller has already detected it; update both call sites
in routes.py predict and stream handlers
- Remove 'anh' from NER false-positive blocklist so England/London alias resolves
correctly; add comment explaining deliberate omission
- Replace hardcoded timestamped TripAdvisor dataset filename with glob-based
auto-discovery of latest matching file in scraped_data/TripAdvisor/
Frontend:
- PaginationUI…
refactor: fix HIGH issues - constructor injection, i18n, ML cache & preprocessing
Backend (12 controllers):
- Replace @Autowired field injection with @RequiredArgsConstructor + private final
in ChatbotController, FlightController, FlightSeatController, HotelController,
LocationController, ProfileController, PromotionController, RoomController,
StripeWebhookController, UserController, UserVoucherController, VisaArticleController
- @Value fields in StripeWebhookController preserved as non-final (Spring property binding)
Frontend (4 pages):
- Replace hardcoded "Vietnam"/"Việt Nam" strings with t('common.vietnam') in
HotelListPage, CarRentalLandingPage, ActivitiesPage, VisaApplicationPage
- Translation key already existed in both en.json and vi.json
Chatbot ML Service:
- Add user_id to response cache key to prevent authenticated users from receiving
each other's personalized cached responses (privacy leak)
- Call preprocess(text, language) before NER extraction so abbreviations like
HCM, HN, 3N2D are expanded before entity recognition
- Update cache.get/se…
security: fix 6 critical vulnerabilities across chatbot and backend
Chatbot ML Service:
- Remove hardcoded fallback API key 'wanderlust-internal-dev-key'; service
now returns 503 when CHATBOT_ML_SERVICE_API_KEY env var is unset
- Fix MongoDB UserMemoryService always reporting _ready=True; add async
_ensure_ready() probe with real admin.command('ping') check
- Delete duplicate function definitions in postprocessor.py (lines 148-237);
RAG_EMPTY stripping, N/A scrubbing, and POI bracket normalization now
actually execute instead of being dead code
- Add _scrub() helper in context_builder.py to sanitize all user-derived
values interpolated into LLM system prompt, preventing prompt injection
- Extend rate limiter middleware to cover /api/predict/stream (was only
covering /api/predict); add X-Forwarded-For IP detection for proxy setups
- Fix off-topic guardrail AND logic bug in routes.py; now fires on low
confidence OR fallback intent when no travel signal present
Backend:
- Move WalletTransaction query from controller to WalletService; return…
security: remove hardcoded ML service API key - require env var (CHATBOT_ML_SERVICE_API_KEY)
- Fix $inc: conversationCount: 0 → 1 (counter was never incrementing)
- Add done callbacks to fire-and-forget async memory tasks for exception visibility
- Replace hardcoded HF username with vars.HF_USERNAME in both deploy workflows
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
fix(chatbot): resolve NameError in guardrail and cache UserMemoryService init failure
Move `context = request.context or {}` before the off-topic guardrail block so
the `PredictResponse(context=context, ...)` early return no longer raises a
NameError. Also introduce a `_USER_MEMORY_INIT_FAILED` sentinel so repeated
requests when MongoDB is down skip re-initialization instead of re-attempting
and logging a warning on every call.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
feat(chatbot): integrate UserMemoryService into predict endpoint for cross-session memory
Problem 1 - blocked iframe:
OSM export/embed.html sends X-Frame-Options: SAMEORIGIN header
→ browser blocks the iframe entirely showing 'content blocked'
Fix: render OSM tile images directly as <img> tags in a 3x3 grid
(tile.openstreetmap.org is a CDN image endpoint, not blocked)
Map area is clickable → opens Google Maps in new tab
Problem 2 - wrong location (city instead of specific POI):
map_data only had destination city coords + name
→ Google Maps URL used lat/lon pointing to city center
→ 'bãi biển sơn trà đà nẵng' opened as just 'Đà Nẵng'
Fix: pass original user query text as search_query in map_data
→ Google Maps URL now: /search?query=bãi biển sơn trà đà nẵng
→ OSM URL now: /search?query=bãi biển sơn trà đà nẵng
- Added 80 training samples for show_map intent (VI + EN, including
'from saved plan' phrasings like 'dựa vào kế hoạch đã lưu thì bật map')
- Added show_map handler in routes.py: falls back to context destination_id
when NER finds no location (covers 'from saved plan' use case)
- Added show_map to additional_rc map trigger list so map card is rendered
- Added show_map response templates to responses_vi.json + responses_en.json
- Added show_map quick replies to ResponseGenerator
- Retrained classifier: overall accuracy 94.2%, show_map P/R/F1=1.00
- GlobalExceptionHandler: add handlers for IllegalStateException (400),
HttpMessageNotReadableException (400), NoHandlerFoundException (404),
HttpRequestMethodNotSupportedException (405)
- TopUpRequestDTO: add @NotNull, @DecimalMin(10000), @DecimalMax(50000000),
@NotBlank(paymentMethod) constraints; validation now returns 400 instead of 500
- RefundService: inject MoneyTransferService and call processRefund() in approveRefund()
so wallet is credited back when admin approves a refund (financial bug fix)
- RefundService: add idempotency guard (status == PENDING check before processing)
- RefundService: replace RuntimeException with ResourceNotFoundException and
IllegalArgumentException throughout
- Tests: remove xfail markers from 6 wallet and vendor tests that are now fixed
feat(chatbot): add chain-of-thought logic for personalize_recommend intent
Wanderlust Dev
3ebc772
When user asks for destination suggestions (e.g. 'Gợi ý điểm đến') without
providing any preferences, the chatbot now:
1. Detects missing preferences (travel style, budget, activities, etc.)
2. Asks clarifying questions instead of faking 'analyzed your preferences'
3. Provides quick replies like 'Tôi thích biển/núi' to guide the user
4. Only gives personalized recommendations when real preferences exist
Changes:
- routes.py: Add preference-checking logic (7.6) before generating response.
Checks entities, user profile, and conversation context for any signals.
Overrides intent to 'missing_preferences' when no info is available.
Adds actual recommendation handler using rec.recommend() when prefs exist.
- responses_vi.json: Fix templates to not claim 'analyzed your preferences'.
Add 'missing_preferences' templates with friendly preference questions.
- responses_en.json: Same fixes for English templates.
- generator.py: Add 'missing_preferences' quick replies (beach/mountain/
b…
Round 15: Dark mode page backgrounds + Chatbot fixes (NER/weather/events/templates/currency)
Wanderlust Dev
fa626b2
- Dark mode: Add dark:bg-gray-900/950 to 36+ page wrappers (bg-gray-50, bg-white, bg-slate-50, gradients)
- NER: Add word-boundary matching for short names + blocklist to prevent false positives
- Weather: Load coords from KB (9,474 destinations) instead of hardcoded 12-city map
- Routes: _get_coords_for_id() now uses KB data via realtime_service
- Events: Fix Hoi An Lantern Festival month:0 crash + add monthly recurring support
- Templates: Add 2-3 new templates per intent for 7 intents (vi+en)
- Currency: Update 16 rates + add 5 new currencies (CNY, CAD, NZD, CHF, SEK)
- CHANGELOG: Update Round 15 section with all fixes
- #1 Fix render-time state mutation → useEffect in ChatWindow
- #2 Add ownership check on DELETE /conversation/{id}
- #3 Add ownership check on GET /plan/{id}
- #4 Add max_length=2000 to PredictRequest.text
- #5 Restrict CORS origins in ML service
- #8 Add dark mode classes to all chatbot components
- #9 Add AbortController + 30s timeout to chatbotApi
- #10 Add maxLength={2000} to ChatInput textarea
- #11 Add retry button on error (quick reply)
- #12 Add rate limiting middleware (30 req/min)
- #15 Fix QuickReplies lost on language toggle
- #16 Add double-click guard on send button
- #25 Improve error messages with HTTP status codes
- #26 Add max_length annotation to PredictRequest
12 files changed across Frontend, Backend, ML service
fix: chatbot direct ML service call for guest users, fix routes syntax, add RUN_GUIDE
- Add APIProvider for cloud LLM services (Groq, OpenRouter, HuggingFace, OpenAI)
- Add ResponseCache (LRU+TTL) and ConversationMemory for multi-turn context
- Add 12 intent-specific system prompts (VI+EN) in ContextBuilder
- Add safety filter with blocklist patterns in PostProcessor
- Add smart quick replies per intent (12 intents x 2 languages)
- Expand DESTINATION_COORDS from 12 to 38 destinations
- Provider routing: local GPU vs cloud API via LLM_PROVIDER env var
- Pass conversation_id through routes for multi-turn conversations
- Update .gitignore for Python __pycache__ files
- Version bump: 3.1.0 -> 3.2.0
- Add 7 new destinations (hanoi, hcmc, bangkok, singapore, bali, tokyo, seoul, paris) to knowledge base
- Fix origin/destination pipeline in routes.py using NER output
- Implement _estimate_transport_cost() with KB lookup + Haversine fallback
- Fix trip optimizer _get_arrival_info() to use actual origin instead of hardcoded HCM
- Add dynamic trip plan summary builder in response generator (VI + EN)
- Create augment_intent_dataset.py for balancing class imbalance (500 target/intent)
- Consolidate dataset research into Update.md section 11
- Delete separate DATASET_RESEARCH.md (content moved to Update.md)
feat: integrate chatbot ML service with backend and frontend
