Spaces:
Sleeping
Sleeping
Commit History
-
docs: Cluster A (count drift) + Cluster B (deleted-module refs) sweep
Cluster A — Align pre-ADR-044 count snapshots with current 148/21 state: - CLAUDE.md L7: "19 insurers / 166 cards / 4-gate faithfulness" → "21 insurer slugs / 148 catalogued cards / structural grounding" - README.md L61: "~150 plans across 20 insurers" → "~148 plans across 21 insurers" - rag/README.md L30: "19 insurers" → "21 insurer slugs; dedup to 148 marketplace cards" - 70-docs/80-audit/reviews-source-map.md ×4: "20 insurers" → "21" - ADR-002 + ADR-009 + 60-decisions/README.md: kept historical scope numbers as ADR history-of-record; added forward-pointer footnotes to current 21/148 per ADR-044 Cluster B — Replace references to modules deleted in KI-225 single-brain consolidation: - kb/AUDIT_TRAIL.md: faithfulness.py / persona.py / fact_find_brain.py / sales_brain.py / qa_brain.py / translator.py / profile_extractor.py / orchestrator.py → single_brain.py - backend/providers/README.md L13: removed translator.py reference (sarvam_llm.py now used directly by …4c728a9
-
fix(batch): voice interim, TTS truncation, RULE2.5 re-ask, faithfulness regression, markdown, compare-reviews, policy-name
Eight user-reported defects fixed together (all root-caused, all verified): #17 page.tsx PTT live transcript: separate inline SpeechRecognition had continuous=false + no-op onend (died after first pause, never restarted) + replace-not-accumulate. Added cross-session final accumulator, want-gated onend restart + 3s heartbeat, trailing flush. #19 sarvam_tts.py: bulbul:v2 silently voices only the first sentence of an over-real-limit single request; hard-coded 1500 ceiling meant the chunker never engaged for 370-720-char replies. Ceiling 1500->500, dropped undocumented enable_preprocessing, added a WAV-duration truncation guard (fail-loud). RED->GREEN proven. #18 single_brain.py: _profile_to_snapshot never exported smoker/ copay_pct/family_medical_history/parents_age_max, so the model couldn't know they were captured -> RULE 2.5 re-asked them. Snapshot widened + RULE 2.5 'omit captured items' directive. #23 single_brain.py: get_policy_facts policies now regis…8828a2e -
fix(#55+#56+#53/#54-engine): TTS full natural readout + voice warm-stream/pre-roll
#55 TTS 10s truncation: voice_format._truncate_for_voice(max_words=55) chopped every advisor reply to ~55 words (~10s) before it reached Sarvam. Cap removed (max_words now ignored unless >=100000). Latent Sarvam Bulbul v2 1500-char cap covered: sarvam_tts now char-ceiling chunks at item/sentence/comma seams, synthesizes sequentially, gapless WAV concat, propagates any chunk error (no silent partial) — STT-style. #56 TTS normalization: _normalize_abbreviations + _normalize_ranges — e.g./i.e./etc. spoken as words; '/' → 'or'; ₹/L/Cr/K/% and en-dash ranges expanded; markdown/list numbering stripped. Verified on the exact 6-question pricing message (all 6 questions present, artifacts gone). #53/#54 ENGINE: useStreamingVoice warm pre-armed mic + 800ms PreRollRing + 200ms deliberate-hold gate + beginPushToTalk/endPushToTalk/ consumePreRollChunks API. Keeps the OS mic hot so page.tsx getUserMedia is ~10-50ms not 200-700ms (latency #54) and the first word is captured (head-clip #53). page.ts…7d87d62
-
fix(#40+voice+upload): SSOT scorecard parity + Sarvam 30s-truncation + uploaded-PDF retrieval gate
#40 SINGLE SOURCE OF TRUTH: extract the marketplace catalogue into backend.main._marketplace_catalogue + marketplace_grade(); brain_tools ._scorecard_signal now delegates (deleted the parallel doctype-rank/_merge re-implementation). Rec-card grade == marketplace grade for all 148 by construction — deterministic oracle 790 ids / 0 mismatches; parity test rewritten to assert the full id universe + alias/doctype edges. VOICE: Sarvam saarika REST has a hard ~30s limit and silently returns only the first 30s (HTTP 200). sarvam_stt now decodes to 16k mono and splits >25s audio on silence, transcribes sequentially, concatenates, raises on error (no silent failure). Frontend POST timeout scales with blob size. Regression test: 92s utterance -> all 92 words. UPLOAD: brain_tools profile-gate silently returned 0 chunks for a freshly uploaded PDF asked about before fact-find (contradicted the 'searchable now' UI promise). Added session-scoped quarantine bypass + single_brain UPLOADED-DOC prompt …a5d9bfd -
chore(cleanup): purge stale narrative/tombstones/dead code — codebase reads as the current standard
Comprehensive stale-content purge so code/docs read as if written correctly from the start; history of record stays only in ADRs (70-docs/60-decisions), KIs (80-audit), and the README. Code (.py): removed mistake/removal/incident narration and tombstone comments ("REMOVED/VESTIGIAL", dated fix-narration, "Z2 fix/W1 Turn/Priya/ Vikram", "the bug was", "used to ... → loop", "do NOT reintroduce"), rewrote them as clean current-behavior comments; deleted the dead SessionState._recall_sniff_done field (zero readers; in-memory-only, verified safe); cleaned stale references to removed systems (sales_brain/qa_brain/orchestrator) in comments, docstrings and a log string. NO logic changed: 29/31 modified .py are AST-equal modulo docstrings (provably comment-only); the other 2 are the verified-safe field deletion + log/docstring string text only. Docs: 70-docs/** + ARCHITECTURE.md rewritten to the current single-LLM-with-tools standard (gemini-2.5-flash + save_profile_field/ retrieve_policies/m…23b8fad -
fix(single_brain): restore broken core flow — model revert + thinkingConfig + rip deterministic scaffolding
Root causes of the live "new user never gets a recommendation" regression: - KI-183 (132a829) silently set the brain to the weak gemini-2.5-flash-lite while migrating off retired gemini-2.0-flash. Reverted DEFAULT_MODEL + _FALLBACK_MODEL to gemini-2.5-flash. - gemini-2.5-flash is a THINKING model; with no thinkingConfig it returns EMPTY completions (zero output tokens, no parts) — proven via raw A/B payload test. Added generationConfig.thinkingConfig.thinkingBudget=0. - b87bd2d's RULE-0 "ASK FOR THE NAME FIRST" overlay contradicted RULE 1 ("save the name the user gave"); removed (restores pre-b87bd2d prompt). - Ripped out leftover pre-Path-B deterministic fact-find scaffolding (the old rule/3-brain/judge defect): _synthesise_fallback + _FALLBACK_SLOT_QUESTIONS (the canned "What's your name?" mask) and the entire in-conversation recall-confirm/sniff (_classify_recall_answer, extract_name_in_context, _recall_sniff_done, STEP1/STEP2). Empty-LLM now surfaces an honest e…5a4b5e8
-
recovery: integrate stalled-session work + de-stale/cleanup (pytest 215 green) [build-fix]
Resumes the session whose 3 background agents stalled on the 600s watchdog; captures the prior session's uncommitted #42–#110 work + data exercise plus this session's verified recovery changes. (Rebuilt from the first attempt to store frontend/public/insurer-logos/*.png via Git LFS — HF Spaces' pre-receive hook requires it; .gitattributes updated, tools/.pdf_text_cache gitignored.) Prior-session work captured (was uncommitted, behaviourally live-tested): - single_brain rewrite, full frontend rebuild, compare/scorecard/premium UI, ~77 40-data/policy_facts sourced-data edits, new tests/tools, render.yaml drop. This session's recovery (surgical, verified, behaviour-preserving): - Fix app-wide HTTP 500: a `*/` inside a CSS comment killed the stylesheet. - Remove dead get_judge_llm everywhere (judge retired in single-brain rewrite). - eval/run.py: repair imports + hard-guard against stale results.json. - Centralize ~13 hardcoded "40-data" literals to one settings.DATA_DIR (resolves id…b87bd2d -
data+scoring: verbatim-source all policy_facts, recalibrate scorecard, fix recommendation
- Re-sourced ~2900 cells to verbatim PDF clauses (v1/v2/v3 fleets); 0 fabrication, 0 legacy-provenance, ~250 wrong values corrected; image-only Acko cells dropped. - Removed max_renewal_age from scoring (IRDAI lifelong is universal); root-caused 999. - Recalibrated 6 sub-scores + thresholds: real A/B/C/D/F spread (was ~90% B); fixed-benefit N/A reweight. - #9 citations = exactly the prose-named policies (no slice cap). - #10 eligibility gates (no top-up for first-timers, zero-copay, SI floor) + profile-fit rank. - Voice TTS silent-failure surfaced + codec fix; 14 dead source URLs replaced. - scorecard/compare/bulk now merge corrected policy_facts (divergence fix). - 62/62 tests; two-part verify: 4113 verbatim-verified, 66 documented residual, 0 fabrication. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>7081aaa
-
fix(ux+voice): KI-274 — landing page enterprise redesign + Sarvam 429 friendly error
Two parallel bundles committed together: A — Landing page enterprise 3-section redesign (frontend/src/app/page.tsx): - Removed: 4 example suggestion chips ("nonsense"), yellow standalone callout "Tell me the truth", "You can type below" narrative, "166 policies" standalone line, header EN/HI lang toggle (already removed in KI-257). - Section 1 "How this works": 3-step grid with numbered teal circles — (1) ask Qs → profile, (2) match 150+ policies, (3) customised premium estimate. Lightbulb icon header. - Section 2 "Using the bot": 4-mode card grid — Type / Push-to-talk / Hold SPACE / Live (BETA). Mic icon header. Note about Sarvam TTS + Hindi. - Section 3 "Two things before we start": 2 callouts side-by-side — Left (amber/teal): "Tell me the truth, even on hard things" (absorbed yellow callout). Right (muted): "Speaking to the bot" — speak slowly, English OR Hindi, the bot mirrors. Shield icon header. - Visual: rounded cards with subtle borders + shadows, hero gradient…ce7f6d0 -
refactor: KI-268 — Path B cleanup: -5,219 LOC legacy + nim_fallback + chain collapse + modal wiring
CLEAN1 — nim_fallback replaces legacy orchestrator fallback (+190 LOC, NEW): - backend/nim_fallback.py: handle_turn_fallback(session, user_text, chat_history) picks NIM via llm_health.get_primary("brain") with BRAIN_CHAIN[0] last-resort. One NvidiaNimLLM.chat call with 15s timeout. Never raises — graceful TurnResult on timeout/exception. - main.py: removed `from backend.orchestrator import handle_turn`. Both fallback branches (SingleBrainError non-sticky + USE_SINGLE_BRAIN=False) now call nim_fallback.handle_turn_fallback. Sticky-graceful-retry preserved. CLEAN2 — Port _normalize_dependents into brain_tools, delete 9 legacy modules: - brain_tools._coerce_dependents now uses ported _normalize_dependents_inline + _CANONICAL_VALUES tuple. Zero imports from sales_brain_normalizer. - DELETED (5,219 LOC total): * orchestrator.py (1,832) * sales_brain_normalizer.py (855) * sales_brain.py (819) * faithfulness.py (616) * providers/tiered_brain_llm.py (257) * persona.py …55fb938 -
fix(arch+wiring): KI-237..KI-241 — Phase Y bundle (Y1+Y2+Y3+Y4 + profile-gate)
Y1 — BrainProviderError + soft_hints consumer: - tiered_brain_llm catches BrainProviderError specifically: retryable=False fails fast (auth/safety), retryable=True falls through to NIM. - orchestrator soft-hint retry block (max 1 per turn): missing_citation re-prompts with [Source:N] addendum; hallucinated_policy constrains brain to cited_policies from X5's faithfulness verdict. Y2 — turn_idx stamping on policy events: - session_state.SessionState.turn_idx int field, incremented at handle_turn start in both orchestrator and single_brain. - All shown_policies / selected_policies / rejected_policies events stamped with turn_idx so admin /api/admin/recommendation-history can show conversation_turn instead of "—". - brain_tools.mark_recommendation records a shown_policies event via profile_store. Y3 — Backend honors X-Preferred-Codec for Safari/Webkit: - main.py chat() now accepts FastAPI Request; reads X-Preferred-Codec header (whitelisted: audio/wav, audio/mp4, audio/webm; else wav). -…2524073 -
fix(arch+robustness): KI-225..KI-235 — Path B single-LLM rewrite + audit bundle
Architectural simplification (KI-225, Path B) — replaces the two-brain sales_brain/qa_brain handoff with a single Gemini Flash call per turn using native function-calling. Kills 40+ scaffolding failure modes at once: - No fact_find vs QA routing boundary (every confirmation/knowledge edge case in should_route_to_fact_find disappears). - No faithfulness gate rejecting natural phrasing ("yes, that sounds correct"). - No separate profile_extractor pass overwriting captured slots with null (KI-091/094 root cause permanently fixed via save_profile_field tool guard). - No rigid normalizer enum — accepts what Gemini emits, light coerce only. - No 700-token JSON contract — Gemini function-calling separates tool calls from prose, no truncation trade-off. Feature-flagged: USE_SINGLE_BRAIN=true activates the new path; default off, legacy orchestrator preserved as fallback. SingleBrainError falls back to orchestrator transparently. New files: - backend/single_brain.py — Gemini function-cal…f15911b -
feat(voice+session+llm): KI-195 + KI-196 + KI-197 + KI-199 bundle
THE "ONE LAST REBUILD" — adaptive voice calibration + clean session lifecycle + 3 latency cheap wins + Gemini context caching. KI-195 — Adaptive TTS volume calibration vs user speech level. Per user idea: bot TTS should ALWAYS be quieter than user speech at the mic so echo loop is mathematically impossible and barge-in always wins. Implementation: - userSpeechRmsRef: rolling EMA of peak mic RMS while user speaks (recording active, not TTS); rAF loop tracks samples > 0.02 floor. - calibrateBotVolume: every 300ms during TTS, if bot_rms_at_mic > userSpeechRms × 0.35, multiply el.volume by 0.8 (floor 0.15). - calibratedVolumes Map persists final levels across turns. KI-196 — Clean session/profile lifecycle (ADR-041). CRITICAL UX fix: silent KI-040 name-based profile recall confused users into thinking the bot was "remembering" them from a prior session. Implementation: - clear_session() in session_state.py (in-memory evict only; on-disk profile JSON untouched). - …96ddaab -
feat(llm+docs): KI-177 + KI-179 + KI-183 + ADR-040 docs cascade
THREE-TIER LLM CHAIN — Google Gemini PRIMARY, NIM Tier 1, OR Tier 2. Live audit (KI-178) of OpenRouter's free-tier catalog killed my earlier recommendation: gemini-2.0-flash isn't in OR's catalog, llama-3.3-70b- :free and hermes-3-405b:free don't expose response_format. So OR's free-with-JSON pool is only nemotron-3-super-120b / qwen3-next-80b / gemma-4-31b. User obtained a Google AI Studio key, set up billing ($0 historical spend, $10 cap, free 1500/day quota active), confirmed working live on gemini-2.5-flash + gemini-2.5-flash-lite. NEW MODEL TIERS: Brain Fast (sales_brain): Tier 0 Google gemini-2.5-flash-lite (KI-183 — 2.0-flash retired) Tier 1 NIM qwen3-next-80b → mistral-large-3 → llama-4-maverick Tier 2 OR nemotron-3-super-120b:free → qwen3-next-80b:free → gemma-4-31b:free Last NIM nemotron-super-49b (only when all else dies) Brain Main (orchestrator): Tier 0 Google gemini-2.5-flash Tier 1+ same NIM/OR cascade Judg…132a829 -
fix(llm): KI-175 — demote nemotron to last resort in all chains
User verdict on nemotron: "the worst model, barely usable, last resort." Live admin showed nemotron elected as primary for Brain Fast + Brain Main + Judge — because the chain election was picking the first HEALTHY model in the chain, and nemotron sat at position #2 (BRAIN/FAST_BRAIN) or #3 (JUDGE). Whenever qwen had a transient blip the election fell to nemotron and the sticky-primary held it there. Fix: reorder chains so nemotron is always LAST. It now only serves when every other chain member is simultaneously unavailable — fail-loud is better than serving the worst model by default. BRAIN_CHAIN: qwen3-next-80b → mistral-large-3 → llama-4-maverick → nemotron FAST_BRAIN_CHAIN: qwen3-next-80b → mistral-large-3 → llama-4-maverick → nemotron JUDGE_CHAIN: mistral-large-3 → llama-4-maverick → nemotron (judge order was already correct; chain unchanged) Llama-4 Maverick added as a 3rd-fallback to FAST_BRAIN_CHAIN (previously only in BRAIN+JUDGE). A…dc57a46 -
fix(voice+chat+llm+docs): KI-155 + KI-156 + KI-157 + KI-158 + KI-159 + KI-160 bundle
Six conversation-pipeline + LLM-chain fixes addressing every live-test defect surfaced today, plus the documentation cascade. KI-155 — enforce <FF> trailer emission. Strengthened fact_find_brain system prompt with absolute final-instruction block + edge-case few-shots ('Hi' → still emit trailer, 'ok' → still emit trailer) so Groq Llama-3.3 (was elected as cross-provider fallback, was silently dropping the trailer) now complies. Smoke: My name is Rohit / I am 29 years old / to buy a health insurance policy — all three return fact_find_brain:: continue, no more ::fallback:no_trailer. KI-156 — fix bot non-response + 'not having any policy' uncaptured. Three sub-fixes: Fix A: extend negation regex in fact_find_normalizer._parse_existing_cover to cover 'not having', 'not got', 'not have', 'without (any|a)'. Fix B: stop-list in fact_find_brain._validate_capture for name field — reject hallucinated names that start with 'currently', 'not', 'no', 'looking', etc. (live bug: LL…1c5baae -
perf(ingest): KI-125 — bump MPS embedding batch 64→128 for bulk re-ingest
Doubles batch size on Mac M-series MPS device. ~100 MB per batch (was ~50 MB), still well within unified-memory budget. Halves GPU kernel launches when ingesting all 206 policies + 18 regulatory docs from scratch (motivated by the 2026-05-15 re-ingest). CPU path unchanged at batch_size=32 to keep peak RSS bounded on machines without a GPU.9519893
-
fix(latency): KI-099 — bound every per-turn LLM call
KI-098 audit surfaced 5 holes where a per-turn LLM call could hang the user-facing chat for 60-120s. This commit installs explicit budgets: backend/orchestrator.py: - extract_profile_updates → asyncio.wait_for(12s) + TimeoutError handler - main brain pick.provider.chat → asyncio.wait_for(30s) - faithfulness cross-check retry → asyncio.wait_for(20s) + skip-retry on timeout (rare-path fallback; safer to skip than hang) backend/providers/sarvam_llm.py: - Monolithic timeout=60.0 → httpx.Timeout(connect=2, read=20, write=2, pool=2) mirroring KI-084 on NIM. Constructor kept backward compatible. backend/translator.py: - asyncio.wait_for(20s) around both Sarvam .chat() calls. On timeout, passthrough original text (callers already tolerate via KI-004). Third callsite (back-translate) routes through translate_to_english so it inherits the wrap. backend/providers/nvidia_nim_llm.py: - Budget guard before the synchronous probe_all() refresh inside chat(). If …0e3c560 -
feat(llm-chain): KI-088 — cap NIM outbound concurrency at 2 + serialise probe + drop inner retry
Live probe at 6a47549 measured only 2/10 brain turns succeeding despite KI-080 election + KI-085 credit gating + KI-087 NIM-first + KI-079 escalation all live. The chain architecture is correct; the binding constraint is NIM's free-tier per-key concurrency (~3-5 slots). Root cause: 5 sources of NIM traffic stack on the same key with no global throttle. 1. Probe loop (every 300s) fired ~6 candidates in parallel via asyncio.gather → 6-slot burst. 2. Admin tab polls /api/admin/llm-health every 30s. 3. Per-user chat turn: 1-2 NIM calls. 4. N concurrent users → N simultaneous calls. 5. Inner 4-attempt 429/5xx retry held one slot for up to ~15s on exponential backoff sleeps. 6+ in-flight → queue → 15-25s response times → bot's 12s outer cap fires → user sees fallback. 20% brain success rate measured. Three coordinated fixes: FIX 1 — Module-level asyncio.Semaphore(2) at backend/providers/ nvidia_nim_llm.py shared across every NvidiaNimLLM instance. Wraps ONLY th…14ee008 -
docs: ADR-032 LLM chain architecture reference + KI-080..KI-087 doc sync
Adds new architecture reference ADR consolidating KI-079..KI-087: - 70-docs/60-decisions/ADR-032-llm-chain-architecture.md (NEW, 435 lines) - TL;DR + ASCII data flow diagram - Component reference (every ModelHealth field) - Election algorithm pseudo-code + score formula - Probe loop cadence + OpenRouter credits polling - Per-provider credit signal table (Groq headers / OR endpoint / NIM rate-meter) - Per-phase httpx timeout justification - Failure classification table - KI-079 escalation path - Telemetry surface (brain_used variants) - Performance characteristics + operational runbook Updated docs: - CLAUDE.md "LLM stack" section: KI-080..KI-087 full coverage + NIM-first preference (KI-087), credit gating (KI-085), per-phase timeouts (KI-084), admin tab (KI-086), HF Space secrets (KI-081) - 80-audit/ENTERPRISE_AUDIT.md: 5 new rows (KI-081, KI-084, KI-085, KI-086, KI-087) appended to "Fixes shipped today" table - backend/provi…6a47549 -
feat(llm-chain): KI-085 — proactive credit tracking; election gates by quota not just liveness
WHY: KI-084 (commit 119e0fd) demotes a candidate for 1h AFTER the first HTTP 429 — that's REACTIVE. Cost: one user-facing failure per dead quota before the elector knows to skip it. KI-085 is PROACTIVE: every probe + every real call updates our view of remaining quota from response headers + provider endpoints; election filters by `is_alive AND has_credits` BEFORE suggesting a primary, so a quota-exhausted Groq is sidelined the moment headers say `remaining_tokens_day < 5000`, not after a 429 hits. SIGNAL SOURCES (priority order): 1) GROQ — response headers on every successful call. Parse `x-ratelimit-remaining-tokens-day` (the one that bit us today; free-tier daily TPD cap) + `x-ratelimit-reset-tokens-day` (duration string `"1h2m"` / bare seconds / unix epoch — all three shapes handled). Low-water 5000 tokens (≥ one fact-find ~2K-input + ~400- output round-trip with margin). 2) OPENROUTER — dedicated GET /api/v1/credits endpoint, polled every ~10 min…8fc7979 -
feat(llm-chain): KI-084 — cut probe token consumption 50× + 429-demote for 1h + per-phase httpx timeouts
Live diagnosis (2026-05-15): Groq returned HTTP 429 — body "Rate limit reached for model llama-3.3-70b-versatile on tokens per day (TPD): Limit 100000, Used 99454". ~30-50K of that quota was the probe loop alone (60s × 25 candidates × ~50 tokens/probe). Election kept picking Groq because its probe latency stayed best, so every chat call hit 429, fell to NIM Qwen which was also under load, hit the 25s outer timeout, and even the KI-079 escalation failed. Three coordinated fixes in one commit: 1. Probe overhead cut ~250× (backend/llm_health.py): - PROBE_INTERVAL_SEC 60 → 300 (5 min cadence) - PROBE_MAX_TOKENS 5 → 1 (same 200 envelope, content unused) - HEALTHY_PROBE_AGE_SEC 90 → 600 (tracks new cadence + headroom) Expected probe-driven Groq token spend drops from ~30-50K/day to ~3K/day baseline — well within the 100K free-tier daily quota. Election still adapts within ~5min of any provider degradation, which is more than fast enough at our chat volume. 2. 4…119e0fd -
docs: KI-080 — sync CLAUDE.md + ENTERPRISE_AUDIT + new ADR-031 + 11 stale KI-025/50-50 surface sweep
Documents the KI-080 architectural shift from "iterate every chain candidate per call" (ADR-026 / KI-025 50/50 NIM ↔ Groq rotation) to "probe-elected sticky primary with cross-provider backup" (ADR-031). Files updated: - CLAUDE.md — "LLM stack" section rewritten for KI-080 election; KI-025 rotation marked deprecated. - 80-audit/ENTERPRISE_AUDIT.md — KI-078/KI-079/KI-080 rows added to "Fixes shipped today" table; executive-scorecard UX-latency + test- coverage rows updated to cite KI-080 (supersedes KI-025). - 70-docs/60-decisions/ADR-031-sticky-primary-election.md — NEW. Full ADR following ADR-030 shape: Context, Decision, Alternatives, Consequences, Files touched, Revisit at scale. - 70-docs/60-decisions/ADR-026-provider-load-balancing.md — header updated to "Superseded by ADR-031"; body retained as historical record per ADR convention. - 70-docs/60-decisions/ADR-030-llm-driven-fact-find.md — pointer to ADR-026 swapped for ADR-031. - README…c592055 -
feat(llm-chain): KI-080 — sticky primary election; 1-2 LLM calls per turn instead of 5-6
Architectural refactor: the chain list is now the CANDIDATE POOL for probe-driven election, not a per-call sequence. Pre-KI-080, NimChainLLM.chat iterated the full chain sequentially every turn. Under NIM per-key concurrency throttling, the first 5 NIM-hosted candidates queued together inside ONE turn, burning the 22s budget before any cross-provider fallback (Groq / OpenRouter) was ever reached. The 10-turn live probe at commit 078ff45 showed 7/10 fact-find turns timing out at exactly 26.6s — every failure a timeout, never a parse error. ELECTION ======== backend/llm_health.py: - Probe cadence tightened 5min → 60s so election reflects pool health fast enough that a NIM brownout rotates out within a minute. - Each candidate carries probe_history (last 5 ok/latency pairs) and a degraded_until_monotonic sin-bin window. - score = (1/max(50, latency_ms)) * success_rate_last_5 - get_primary(chain_name) → highest scorer with a probe within 90s. - get_backup(chain_name) → …6159c54 -
feat(llm-chain): KI-079 — Groq earlier in FAST_BRAIN_CHAIN + escalate to BRAIN_CHAIN on timeout before canonical fallback
Live 10-turn probe (commit 078ff45) showed 7/10 fact-find turns timing out at exactly 26.6s with _fallback_reason="timeout" — ALL failures are real timeouts, not parse errors. Root cause: FAST_BRAIN_CHAIN's first 5 candidates were all NIM-hosted; under NIM per-key concurrency limits they queued together and the chain burned its 22s total_budget_s inside NIM before reaching Groq Llama-3.3 70B at the chain bottom. FIX 1 (nvidia_nim_llm.py) - Move groq:llama-3.3-70b-versatile from FAST_BRAIN_CHAIN[5] (last) to FAST_BRAIN_CHAIN[1] (right after Nemotron primary). All other candidates kept. Nemotron remains primary (KI-035 latency win). - New worst-case time-to-cross-provider on a NIM hang: ~6-7s (per-link 6s timeout × 1 link) vs the prior ~22s. FIX 2 (fact_find_brain.py) - Add _TIMEOUT_S_ESCALATION = 15.0 alongside existing _TIMEOUT_S = 25.0. - On asyncio.TimeoutError from FAST_BRAIN_CHAIN, escalate ONCE to get_brain_llm() (BRAIN_CHAIN — Qwen 80B primary + 8 fallbacks incl. Open…87ee522 -
feat(llm-chain): KI-078 — tighten per-link timeout + narrow exception catch + stamp fallback reason for telemetry
Three related LLM-chain hardening fixes that together close the loop on KI-075's fact-find brain failure root cause: 1. nvidia_nim_llm.py — per-link timeout 12s → 6s in get_fast_brain_llm(). With a 22s total chain budget, a 12s per-link meant only 1 candidate could complete before the budget expired — i.e. the entire fallback chain was effectively dead. Healthy paths (Groq LPU, NIM Nemotron Nano, NIM Qwen) all TTFT in <2s, so 6s catches a degraded link fast and lets the chain try 3-4 candidates inside the 22s budget. NIM cold-start still has Groq's cross-provider link (no NIM cold-start exposure) + the outer fact_find_brain wait_for=25s. 2. nvidia_nim_llm.py — narrow the broad `except Exception` in NimChainLLM.chat() loop to re-raise asyncio.CancelledError, KeyboardInterrupt, SystemExit. The previous broad catch was swallowing CancelledError, so when fact_find_brain's outer asyncio.wait_for(_TIMEOUT_S=25s) fired, this loop kept burning the full cha…078ff45 -
docs: KI-071 — reconcile docs with code reality (Qwen 80B / Nemotron 30B / Mistral 675B as current primaries)
Source-of-truth audit found the production code on `main` runs Qwen 3-Next 80B (brain primary) + Nemotron Nano 30B (fast brain primary) + Mistral Large 3 675B (judge primary) per the D-022 inline brain swap on 2026-05-14 (NIM DeepSeek + Llama-4 Maverick pools were timing out for hours). The ADR-019 decision text + multiple docs still referenced DeepSeek V4-Pro / V4-Flash + Llama-4 Maverick as primaries. This commit aligns 12 files with the actual chain definitions in `backend/providers/nvidia_nim_llm.py`. DeepSeek V4-Pro / V4-Flash + Llama-4 Maverick are correctly documented now as **fallback chain entries** that re-enter rotation when NIM's pools recover. Files reconciled (+64 / -45): - `README.md` — §1.2 "Honest model picks" paragraph + §4.3 Model stack table rewritten. Heavy brain row now: "Qwen 3-Next 80B *or* Groq Llama-3.3-70B (50/50 rotation)". Fast brain: "Nemotron Nano 30B". Judge: "Mistral Large 3 675B". Full fallback chains documented. - `70-docs/60-decisio…2e60476 -
refactor: KI-050 — complete data/ → 40-data/ rename across all Python refs
The data/ → 40-data/ git mv landed in the previous commit (c8bf1a1) but the corresponding string-path updates in Python code did not — Python was still resolving _DATA_ROOT, _PROFILES_DIR, PREMIUM_DATA, HEALTH_FILE, etc. to the now-non-existent data/ path. The HF Space rebuild would have shipped a broken backend. This commit sweeps the 23 files that referenced data/ as a string path: • `"data/*` → `"40-data/*` • `/data/*` → `/40-data/*` • `(data/` → `(40-data/` • `Path("data" ...)` → `Path("40-data" ...)` • `ROOT / "data"` → `ROOT / "40-data"` Touched: backend/{session_state, profile_store, premium_calculator, llm_health, admin, main, scorecard, providers/nvidia_nim_llm}, rag/build_kb, tools/{check_link_rot, info_source_map, refresh_premiums, generate_policy_facts, curate_*, ingest_reviews, ingest_kb_summaries}, backend's per-folder docs, etc. Dockerfile: `COPY data ./data` → `COPY 40-data ./40-data` — container path mirrors host. Verified by: • Python smoke: _DATA_ROO…52c6351 -
refactor: KI-047 — bucket reorg (Option A safe subset)
User picked numbered-bucket layout. Executing the SAFE subset that does NOT touch Python imports or string path refs: • docs/ → 70-docs/ • audit_results/ → 80-audit/ Kept as-is (renaming would need ~25 Python files of string-path updates — deferred to follow-up): • data/ — code-referenced (backend/{premium_calculator, session_state, profile_store, scorecard, admin, llm_health, main}, rag/build_kb, tools/{check_link_rot, info_source_map, curate_*, refresh_premiums, generate_policy_facts}). • backend/, frontend/, rag/, tools/, eval/, tests/, kb/ — Python directly forbids leading-digit / hyphen package names. All references to the moved folders updated in one sed pass across *.md, *.py, *.sh, *.ts, *.tsx, *.json, Dockerfile. Verified by: • tests/test_routing_regression.py — 15/15 still pass. • No residual (docs/ or [docs/ or /docs/ or ^docs/ refs. • Same for audit_results/. Doc…e2d09f9 -
feat: KI-044 PCM pre-roll via AudioWorklet + 11 folder READMEs
KI-044 — Proper first-word capture in Live mode. Previous MediaRecorder approach started recording only AFTER VAD declared speech-start, missing the first 50-100 ms of the opening word. User report: "ensure that the first word uttered is also captured, especially in live." New approach (frontend/src/lib/useLiveConversation.ts): • Inline AudioWorkletProcessor (Blob-URL registered, no static asset routing) taps the raw PCM from the mic source. Every render quantum (128 samples) is posted to the main thread as Float32. • Main thread maintains a circular preroll buffer (~300 ms at the AudioContext sample rate) while no utterance is in progress. • When VAD fires speech-start: snapshot the preroll into the active utterance buffer; continue accumulating subsequent samples. • When VAD fires silence-end: encode utterance as 16-bit PCM WAV (Sarvam Saarika native format) and post to onUtterance. • Worklet stays running via source → workletNode → silentSink (zero…86fcc31 -
perf: KI-034 retrieval cache + KI-035 reorder FAST_BRAIN_CHAIN
KI-034 — Retrieval cache. In-process LRU keyed by (query_text_normalized, top_k, sorted policy_ids, sorted insurer_slugs). Cap 256 entries; FIFO/LRU eviction so memory stays bounded across long sessions. Skips both the Voyage embed call AND the Chroma query + regulatory/review boost passes on cache hit. Typical chat session has lots of near-identical follow-ups ("waiting period?" → "PED waiting period?" → "diabetes waiting period?"). Cache key is lowercased + stripped so trivial reformatting hits the cache too. Cache invalidates implicitly on process restart (HF Space rebuild). Smoke: _cache_key("what is the waiting period?", 5, None, None) == _cache_key("WHAT IS the Waiting Period? ", 5, None, None) → True _cache_get returns ['mock_chunk'] for either form. KI-035 — FAST_BRAIN_CHAIN reorder. Fast brain serves the latency-sensitive roles: fact-find, QA, paraphrase, normalize, extract. Throughput-wise they're all sub-second by content size, so TTFT dominates per…844ed03 -
perf: KI-025 — load-balance brain calls across NIM Qwen ↔ Groq Llama
The NIM 40 req/min rate cap is the upstream bottleneck on every audit and eval. Groq has independent rate quota and a Llama-3.3-70B LPU primary already in BRAIN_CHAIN/FAST_BRAIN_CHAIN as a cross-provider fallback. New _balanced_brain_chain() hoists the Groq entry to position 0 on 50% of calls (per-call random.random(), no shared state, no GIL races, fair in aggregate) so half the brain traffic goes to each provider — effectively 2× sustained brain throughput. The full fallback chain is preserved when Groq is primary (Groq → NIM Qwen → other NIM models → ...), so reliability is unchanged. Groq LPU typically has sub-1s TTFT, often faster than NIM Qwen — so for fact-find / QA this rotation is also a strict latency win. Tests pin: (a) probability=0 keeps NIM primary, (b) probability=1 hoists Groq, (c) rotation preserves the full chain (no candidates lost), (d) in aggregate over 1000 calls with seeded random, ~50% land on Groq primary. Co-Authored-By: Claude Opus 4.7 (1M context) <norepl…a04c17a -
fix: KI-018..022 — routing, profile telemetry, NIM timeouts, judge JSON, chat clear
Five interlocking enterprise-readiness fixes plus a master audit report. KI-018 — orchestrator.py: stop force-routing QA intent to fact-find when the session profile is empty. The KI-013 guard was over-correcting: every QA question ("What's the waiting period for PED in Activ Assure?") was being answered with "Happy to help. First, your age?". Restrict the empty-profile guard to {recommendation, comparison} intents only. 5-Q post-fix smoke: factual accuracy 0% → 60%; nim-chain serving 100% of QA (was 0%). Full re-eval owed. KI-019 — orchestrator.py: populate TurnResult.profile_updates from the fact-find branch (previously only the free-form branch populated it). The 100-persona audit appeared to show age captured for only 12/100 personas when the slot-filler was actually working — confirmed by the fact_find_complete readback "42 years old; covering self+spouse+kids+ parents; income 25L+". This was a telemetry bug, not a slot-filler bug. KI-020 — main.py / session_state.py / frontend…bcb7079 -
fix(stt): transcode browser webm/opus → wav before Sarvam (D-024)
Root cause: Browser MediaRecorder emits audio/webm;codecs=opus by default. Sarvam STT only accepts WAV/MP3/FLAC/OGG/M4A. The /api/transcribe endpoint silently coerced webm→wav in the Content-Type but uploaded the raw webm bytes, so Sarvam returned HTTP 400 Bad Request: STT failed: HTTPStatusError: Client error '400 Bad Request' for url 'https://api.sarvam.ai/speech-to-text' Fix in three layers: 1) sarvam_stt.py — _transcode_to_wav() helper using pydub (ffmpeg-backed) converts any non-native container to 16 kHz mono WAV before upload. Also short-audio guard: <1 KB bytes return empty STTResult instead of 500ing on Sarvam's silence rejection. 2) main.py — extend the format whitelist to include webm/opus/mp4 so the real extension is passed through to the provider (provider then decides to transcode). 3) Dockerfile — add ffmpeg apt package (pydub's runtime dependency). requirements.txt — add pydub==0.25.1. This is the first of three live-product issues surfaced t…a777198 -
Deploy: Stack A (NIM brain + Maverick judge + Sarvam voice). D-019.
Verified signaturecd72f85 -
Deploy: Stack A (NIM brain + Maverick judge + Sarvam voice). D-019.
Verified signature5de86d2 -
Deploy: Stack A (NIM brain + Maverick judge + Sarvam voice). D-019.
Verified signature629dfe2 -
Deploy: Stack A (NIM brain + Maverick judge + Sarvam voice). D-019.
Verified signature13779e8
-
Clean Space: delete 263 legacy files (4 retired providers + corpus/vectors/extracted moved to dataset). D-019 Stack A.
Verified signature35cff13 -
Stack A: NIM brain + Maverick judge + Sarvam voice/Indic (D-019). Data moved to insurance-bot-data dataset; Space is now code-only.
Verified signaturec32bddc -
Deploy v1 — single-Docker FastAPI + Next.js + RAG + voice + faithfulness
Verified signaturebd7ade9
-
Deploy v1 — single-Docker FastAPI + Next.js + RAG + voice + faithfulness
Verified signature5dbdf6e