Spaces:
Sleeping
docs: Cluster A (count drift) + Cluster B (deleted-module refs) sweep
Browse filesCluster A β Align pre-ADR-044 count snapshots with current 148/21 state:
- CLAUDE.md L7: "19 insurers / 166 cards / 4-gate faithfulness" β "21 insurer slugs / 148 catalogued cards / structural grounding"
- README.md L61: "~150 plans across 20 insurers" β "~148 plans across 21 insurers"
- rag/README.md L30: "19 insurers" β "21 insurer slugs; dedup to 148 marketplace cards"
- 70-docs/80-audit/reviews-source-map.md Γ4: "20 insurers" β "21"
- ADR-002 + ADR-009 + 60-decisions/README.md: kept historical scope numbers as ADR
history-of-record; added forward-pointer footnotes to current 21/148 per ADR-044
Cluster B β Replace references to modules deleted in KI-225 single-brain consolidation:
- kb/AUDIT_TRAIL.md: faithfulness.py / persona.py / fact_find_brain.py / sales_brain.py /
qa_brain.py / translator.py / profile_extractor.py / orchestrator.py β single_brain.py
- backend/providers/README.md L13: removed translator.py reference (sarvam_llm.py now
used directly by single_brain.py for Indic)
- backend/README.md L17-18: sales_brain.py row replaced by single_brain.py row
(one Gemini-with-function-calling call per turn, ~5,200 LOC net removed in KI-225)
- kb/security/INDEX.md L28: "Hallucination defense β 5 gates" β "structural grounding
(post-KI-225)" β faithfulness.py was removed; brain quotes only what tools returned
- CLAUDE.md L58-60: KI-167 sales_brain.py paragraph extended with KI-225 consolidation
note; nim_fallback.py described as ~100 LOC fallback module (not per-turn chain)
11 docs files. Zero Python touched. Zero data files touched. Zero build config touched.
HF Space rebuilds on push but runtime is byte-identical.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- 70-docs/60-decisions/ADR-002-health-category-vertical.md +4 -0
- 70-docs/60-decisions/ADR-009-19-insurer-comprehensive-schema.md +1 -1
- 70-docs/60-decisions/README.md +1 -1
- 70-docs/80-audit/reviews-source-map.md +5 -5
- CLAUDE.md +3 -3
- README.md +1 -1
- backend/README.md +2 -2
- backend/providers/README.md +1 -1
- kb/AUDIT_TRAIL.md +3 -3
- kb/security/INDEX.md +7 -11
- rag/README.md +1 -1
|
@@ -34,3 +34,7 @@ Given the vertical-slice scope (ADR-001), one insurance category had to be chose
|
|
| 34 |
## Revisit at scale
|
| 35 |
|
| 36 |
v2 adds Life (term + ULIP + endowment) and Motor (private car + two-wheeler). The 48-field Health schema generalises with ~70% reuse β most additions are category-specific waiting/exclusion fields.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 34 |
## Revisit at scale
|
| 35 |
|
| 36 |
v2 adds Life (term + ULIP + endowment) and Motor (private car + two-wheeler). The 48-field Health schema generalises with ~70% reuse β most additions are category-specific waiting/exclusion fields.
|
| 37 |
+
|
| 38 |
+
---
|
| 39 |
+
|
| 40 |
+
*(Current state per [ADR-044](ADR-044-uploaded-pdf-parity.md): corpus expanded to 21 catalogued insurer slugs / 148 marketplace cards; the 19-insurer baseline this ADR was scoped against captured the original launch corpus.)*
|
|
@@ -1,6 +1,6 @@
|
|
| 1 |
# ADR-009: 19 insurers Γ all health policies; 48-field structured schema
|
| 2 |
|
| 3 |
-
**Status:** Locked
|
| 4 |
**Date:** 2026-05-13
|
| 5 |
|
| 6 |
## Context
|
|
|
|
| 1 |
# ADR-009: 19 insurers Γ all health policies; 48-field structured schema
|
| 2 |
|
| 3 |
+
**Status:** Locked (historical scope). Current state per [ADR-044](ADR-044-uploaded-pdf-parity.md): 21 catalogued insurer slugs / 148 marketplace cards / 7,317 Chroma chunks. The 48-field structured schema (`rag/schema.py::HealthPolicy`) is unchanged.
|
| 4 |
**Date:** 2026-05-13
|
| 5 |
|
| 6 |
## Context
|
|
@@ -16,7 +16,7 @@ ADR numbers map 1:1 to the original `D-NNN` entries that lived in the legacy `70
|
|
| 16 |
| [ADR-006](ADR-006-sarvam-first-stack.md) | Sarvam-first STT/TTS/LLM defaults | Partially superseded by ADR-019 |
|
| 17 |
| [ADR-007](ADR-007-illustrative-pricing.md) | Illustrative pricing, not real-time quotes | Locked |
|
| 18 |
| [ADR-008](ADR-008-consultative-advisor-persona.md) | Consultative-advisor persona | Locked |
|
| 19 |
-
| [ADR-009](ADR-009-19-insurer-comprehensive-schema.md) | 19 insurers Γ all health policies; 48-field schema | Locked |
|
| 20 |
| [ADR-010](ADR-010-secret-handling.md) | Secret handling: `.env` chmod 600, gitignored | Locked |
|
| 21 |
| [ADR-011](ADR-011-bge-local-embeddings.md) | Local BGE-small embeddings (Voyage was original) | Locked |
|
| 22 |
| [ADR-012](ADR-012-render-then-hf-space-deploy.md) | Render β HF Space migration | Superseded |
|
|
|
|
| 16 |
| [ADR-006](ADR-006-sarvam-first-stack.md) | Sarvam-first STT/TTS/LLM defaults | Partially superseded by ADR-019 |
|
| 17 |
| [ADR-007](ADR-007-illustrative-pricing.md) | Illustrative pricing, not real-time quotes | Locked |
|
| 18 |
| [ADR-008](ADR-008-consultative-advisor-persona.md) | Consultative-advisor persona | Locked |
|
| 19 |
+
| [ADR-009](ADR-009-19-insurer-comprehensive-schema.md) | 19 insurers Γ all health policies; 48-field schema | Locked (historical scope; current 21/148) |
|
| 20 |
| [ADR-010](ADR-010-secret-handling.md) | Secret handling: `.env` chmod 600, gitignored | Locked |
|
| 21 |
| [ADR-011](ADR-011-bge-local-embeddings.md) | Local BGE-small embeddings (Voyage was original) | Locked |
|
| 22 |
| [ADR-012](ADR-012-render-then-hf-space-deploy.md) | Render β HF Space migration | Superseded |
|
|
@@ -3,7 +3,7 @@
|
|
| 3 |
| Field | Value |
|
| 4 |
| --- | --- |
|
| 5 |
| Document type | Source-methodology catalog (data-provenance audit) |
|
| 6 |
-
| Subject data files | `40-data/reviews/<slug>.json` (
|
| 7 |
| Companion data | `40-data/reviews/INDEX.md` (leaderboard) |
|
| 8 |
| Generated (this doc) | 2026-05-18 |
|
| 9 |
| Premium analogue | [`premium-source-map.md`](premium-source-map.md) |
|
|
@@ -22,7 +22,7 @@ This document does **not** modify any JSON or code β it is read-only documenta
|
|
| 22 |
|
| 23 |
| Metric | Value |
|
| 24 |
| --- | --- |
|
| 25 |
-
| Insurer review files documented |
|
| 26 |
| Distinct source domains (CSR provenance) | 7 |
|
| 27 |
| Files MISSING a source_url | 0 (none) |
|
| 28 |
| Files MISSING a CSR data year | 0 (none) |
|
|
@@ -30,7 +30,7 @@ This document does **not** modify any JSON or code β it is read-only documenta
|
|
| 30 |
| Marketplace policies replayed (Deliverable 2) | 148 |
|
| 31 |
| Policies with a non-null Claim-Experience sub-score | 148 / 148 |
|
| 32 |
| Distinct Claim-Experience sub-score values observed | 19 (differentiates: YES) |
|
| 33 |
-
| CSRβ β CEβ monotonicity | CSR-COMPONENT STRICTLY MONOTONE (the CSR term of the sub-score is non-decreasing in CSR across all
|
| 34 |
|
| 35 |
Source domains: `irdai.gov.in`, `joinditto.in`, `web.archive.org`, `www.beshak.org`, `www.business-standard.com`, `www.policybazaar.com`, `www.policyx.com`.
|
| 36 |
|
|
@@ -222,9 +222,9 @@ Each row is one marketplace policy from `asyncio.run(policies_all()).dict()['pol
|
|
| 222 |
|
| 223 |
- **Non-null contribution:** PASS β 148/148 policies yield a non-null, non-zeroed Claim-Experience sub-score.
|
| 224 |
- **Differentiation:** PASS β 19 distinct sub-score values across the 148 policies (min 47, max 100); the contribution is NOT uniform across insurers.
|
| 225 |
-
- **CSR consistency (monotone):** PASS β CSR-COMPONENT STRICTLY MONOTONE (the CSR term of the sub-score is non-decreasing in CSR across all
|
| 226 |
|
| 227 |
-
The load-bearing reconciliation is the **CSR component in isolation**: replaying the exact CSR branch (`scorecard.py` Β§332-337) over all
|
| 228 |
|
| 229 |
The 61 raw mean-CE pairwise non-monotonicities below are EXPECTED and correct: the sub-score also blends complaints/10k (Β±16), cashless support (Β±18) and network-hospital count (Β±18) (`scorecard.py` Β§315-363), which vary independently of CSR. e.g. Bajaj Allianz (CSR 92.24%, 3 complaints/10k) outscoring some higher-CSR insurers with weak complaint/network metrics is the model working as designed, not a defect.
|
| 230 |
|
|
|
|
| 3 |
| Field | Value |
|
| 4 |
| --- | --- |
|
| 5 |
| Document type | Source-methodology catalog (data-provenance audit) |
|
| 6 |
+
| Subject data files | `40-data/reviews/<slug>.json` (21 insurer files) |
|
| 7 |
| Companion data | `40-data/reviews/INDEX.md` (leaderboard) |
|
| 8 |
| Generated (this doc) | 2026-05-18 |
|
| 9 |
| Premium analogue | [`premium-source-map.md`](premium-source-map.md) |
|
|
|
|
| 22 |
|
| 23 |
| Metric | Value |
|
| 24 |
| --- | --- |
|
| 25 |
+
| Insurer review files documented | 21 |
|
| 26 |
| Distinct source domains (CSR provenance) | 7 |
|
| 27 |
| Files MISSING a source_url | 0 (none) |
|
| 28 |
| Files MISSING a CSR data year | 0 (none) |
|
|
|
|
| 30 |
| Marketplace policies replayed (Deliverable 2) | 148 |
|
| 31 |
| Policies with a non-null Claim-Experience sub-score | 148 / 148 |
|
| 32 |
| Distinct Claim-Experience sub-score values observed | 19 (differentiates: YES) |
|
| 33 |
+
| CSRβ β CEβ monotonicity | CSR-COMPONENT STRICTLY MONOTONE (the CSR term of the sub-score is non-decreasing in CSR across all 21 insurers β verified PASS); 61 raw mean-CE pairwise non-monotonicities remain and are EXPECTED β the sub-score also blends complaints/10k, cashless support and network size, which vary independently of CSR (scorecard.py Β§315-363) |
|
| 34 |
|
| 35 |
Source domains: `irdai.gov.in`, `joinditto.in`, `web.archive.org`, `www.beshak.org`, `www.business-standard.com`, `www.policybazaar.com`, `www.policyx.com`.
|
| 36 |
|
|
|
|
| 222 |
|
| 223 |
- **Non-null contribution:** PASS β 148/148 policies yield a non-null, non-zeroed Claim-Experience sub-score.
|
| 224 |
- **Differentiation:** PASS β 19 distinct sub-score values across the 148 policies (min 47, max 100); the contribution is NOT uniform across insurers.
|
| 225 |
+
- **CSR consistency (monotone):** PASS β CSR-COMPONENT STRICTLY MONOTONE (the CSR term of the sub-score is non-decreasing in CSR across all 21 insurers β verified PASS); 61 raw mean-CE pairwise non-monotonicities remain and are EXPECTED β the sub-score also blends complaints/10k, cashless support and network size, which vary independently of CSR (scorecard.py Β§315-363).
|
| 226 |
|
| 227 |
+
The load-bearing reconciliation is the **CSR component in isolation**: replaying the exact CSR branch (`scorecard.py` Β§332-337) over all 21 insurers sorted by ascending CSR yields a strictly non-decreasing point contribution (β20 β β6 β +5 β +12 β +20 across the 75/85/90/95 bands). So a higher CSR can only *raise* (never lower) the Claim-Experience sub-score, all else equal β the parity fix holds.
|
| 228 |
|
| 229 |
The 61 raw mean-CE pairwise non-monotonicities below are EXPECTED and correct: the sub-score also blends complaints/10k (Β±16), cashless support (Β±18) and network-hospital count (Β±18) (`scorecard.py` Β§315-363), which vary independently of CSR. e.g. Bajaj Allianz (CSR 92.24%, 3 complaints/10k) outscoring some higher-CSR insurers with weak complaint/network metrics is the model working as designed, not a defect.
|
| 230 |
|
|
@@ -4,7 +4,7 @@ This file is read by Claude Code (and any compatible AI tool) at the start of a
|
|
| 4 |
|
| 5 |
## Project at a glance
|
| 6 |
|
| 7 |
-
- **What:** a voice-first AI advisor for Indian health insurance β RAG over a curated 206-document corpus (188 product PDFs across
|
| 8 |
- **Live:** https://rohitsar567-insurancebot.hf.space (HF Space; rebuild triggered on every push to `origin main`).
|
| 9 |
- **Repos:** `origin` is the HF Space at `huggingface.co/spaces/rohitsar567/InsuranceBot`. `github` is the mirror at `github.com/rohitsar567/insurance-sales-bot`. Data lives separately at `huggingface.co/datasets/rohitsar567/insurance-bot-data` (with a GitHub mirror that uses LFS).
|
| 10 |
- **Local dev path:** `~/Developer/Insurance Sales Bot/` (NOT `~/Documents/Personal/AI Work/...` β the older path that occasionally shows up in stale scripts; iCloud-synced + TCC-restricted).
|
|
@@ -55,9 +55,9 @@ Every LLM role is a `NimChainLLM` candidate pool, NOT a hardcoded single model.
|
|
| 55 |
|
| 56 |
## Fact-find loop (ADR-039 + ADR-040, supersedes ADR-030 / ADR-027) β KI-167 / KI-179
|
| 57 |
|
| 58 |
-
**One LLM call per turn, structured-output JSON, no scripted prompts, no fallback prose.** KI-167 ripped out `backend/fact_find_brain.py` (the `<FF>{...}</FF>` trailer convention + `_canonical_fallback`
|
| 59 |
|
| 60 |
-
- **Single
|
| 61 |
- **System prompt** carries the 9-slot schema + current profile state. The LLM is free to ask in any order, in any voice, multi-fact in one turn or one slot at a time. No scripted opener, no acknowledger prefix, no `prompt_en` template.
|
| 62 |
- **Deterministic post-processor** (`backend/sales_brain_normalizer.py`) takes the LLM's loose `captures` dict and emits a `{canonical_field: validated_value}` map: alias resolution (`location` β `location_tier`), enum normalization (`Bangalore` β `metro`), INR-amount parsing, null/empty drop, type/bounds validation. No LLM calls β pure rules. Override: if LLM sets `complete: true` while any required slot is empty, force `complete: false`.
|
| 63 |
- **Profile persistence (post-ADR-043, 2026-05-27).** Captures flow through `session.update_profile_field()` into the in-memory `SessionState.profile` only. The previous `backend/profile_store.save_profile()` disk write and `backend/profile_rag.upsert_profile_chunk()` Chroma write were **removed entirely** when cross-session recall was retired (ADR-043). Closing the tab discards the profile.
|
|
|
|
| 4 |
|
| 5 |
## Project at a glance
|
| 6 |
|
| 7 |
+
- **What:** a voice-first AI advisor for Indian health insurance β RAG over a curated 206-document corpus (188 product PDFs across 21 insurer slugs + 18 regulatory IRDAI/NHA docs, **7,317 Chroma chunks** post-KI-125β127 rebuild β wordings 5,401 Β· brochure 611 Β· regulatory 498 Β· prospectus 483 Β· cis 302 Β· curated 21 Β· profile 1), Sarvam STT/TTS, structural grounding (the single LLM brain quotes only what its tools returned), 21-insurer scorecard (regulatory tracked separately). Marketplace surfaces **148 catalogued cards** across the 21 real insurer slugs (one card per IRDAI-filed product after KI-133 / KI-141 / KI-142 / KI-145 dedup, IndusInd-General slug per KI-144); 201 extracted JSONs + 253 curated `policy_facts` JSONs feed the structured side. The 21 internal Chroma slugs = 20 user-facing insurers + IndusInd-General + 1 `regulatory` bucket; the regulatory + `profile` slugs are filtered out of every user-facing count (KI-129 / KI-130 / KI-132).
|
| 8 |
- **Live:** https://rohitsar567-insurancebot.hf.space (HF Space; rebuild triggered on every push to `origin main`).
|
| 9 |
- **Repos:** `origin` is the HF Space at `huggingface.co/spaces/rohitsar567/InsuranceBot`. `github` is the mirror at `github.com/rohitsar567/insurance-sales-bot`. Data lives separately at `huggingface.co/datasets/rohitsar567/insurance-bot-data` (with a GitHub mirror that uses LFS).
|
| 10 |
- **Local dev path:** `~/Developer/Insurance Sales Bot/` (NOT `~/Documents/Personal/AI Work/...` β the older path that occasionally shows up in stale scripts; iCloud-synced + TCC-restricted).
|
|
|
|
| 55 |
|
| 56 |
## Fact-find loop (ADR-039 + ADR-040, supersedes ADR-030 / ADR-027) β KI-167 / KI-179
|
| 57 |
|
| 58 |
+
**One LLM call per turn, structured-output JSON, no scripted prompts, no fallback prose.** KI-167 (2026-05-15) ripped out `backend/fact_find_brain.py` (the `<FF>{...}</FF>` trailer convention + `_canonical_fallback` + scripted `Question.prompt_en`) and replaced it with `backend/sales_brain.py` β a single LLM-driven sales agent. **KI-225 (2026-05-15) then consolidated further:** `sales_brain.py` + `qa_brain.py` + `faithfulness.py` + `persona.py` + `translator.py` + `profile_extractor.py` + `orchestrator.py` were all removed (~5,200 LOC net) and merged into the current `backend/single_brain.py` β ONE Gemini-with-function-calling call per turn with tools `save_profile_field`, `retrieve_policies`, `get_policy_facts`, `mark_recommendation`. Faithfulness is now structural (the brain can only quote what its tools returned).
|
| 59 |
|
| 60 |
+
- **Single brain call per turn** via `single_brain.handle_turn()` calling `GoogleGeminiLLM` through the function-calling API. The pre-KI-167 architecture had accreted eight patches (KI-090 / KI-091 / KI-094 / KI-103 / KI-150 / KI-155 / KI-156 / KI-158 / KI-161) working around the fragility of pattern-matching structure out of prose; KI-225 collapsed the whole brain stack onto a single Gemini call with tools. NIM remains as fallback via `backend/nim_fallback.py` (~100 LOC) only when `GOOGLE_API_KEY` is missing or Gemini hard-errors β NOT as a per-turn chain. Indic queries route to Sarvam-M via single_brain when `_detect_language` returns `'indic'`.
|
| 61 |
- **System prompt** carries the 9-slot schema + current profile state. The LLM is free to ask in any order, in any voice, multi-fact in one turn or one slot at a time. No scripted opener, no acknowledger prefix, no `prompt_en` template.
|
| 62 |
- **Deterministic post-processor** (`backend/sales_brain_normalizer.py`) takes the LLM's loose `captures` dict and emits a `{canonical_field: validated_value}` map: alias resolution (`location` β `location_tier`), enum normalization (`Bangalore` β `metro`), INR-amount parsing, null/empty drop, type/bounds validation. No LLM calls β pure rules. Override: if LLM sets `complete: true` while any required slot is empty, force `complete: false`.
|
| 63 |
- **Profile persistence (post-ADR-043, 2026-05-27).** Captures flow through `session.update_profile_field()` into the in-memory `SessionState.profile` only. The previous `backend/profile_store.save_profile()` disk write and `backend/profile_rag.upsert_profile_chunk()` Chroma write were **removed entirely** when cross-session recall was retired (ADR-043). Closing the tab discards the profile.
|
|
@@ -58,7 +58,7 @@ mobile.
|
|
| 58 |
Buying health insurance in India is hard for an ordinary person. A
|
| 59 |
first-time buyer faces three concrete problems:
|
| 60 |
|
| 61 |
-
1. **Too much to compare.** ~
|
| 62 |
dozens of decision-relevant fields (waiting periods, room-rent caps,
|
| 63 |
co-pay, maternity, sub-limits, network size). No human reads them all.
|
| 64 |
2. **The truth is buried.** The number that decides whether a plan is
|
|
|
|
| 58 |
Buying health insurance in India is hard for an ordinary person. A
|
| 59 |
first-time buyer faces three concrete problems:
|
| 60 |
|
| 61 |
+
1. **Too much to compare.** ~148 plans across 21 insurers, each with
|
| 62 |
dozens of decision-relevant fields (waiting periods, room-rent caps,
|
| 63 |
co-pay, maternity, sub-limits, network size). No human reads them all.
|
| 64 |
2. **The truth is buried.** The number that decides whether a plan is
|
|
@@ -14,8 +14,8 @@ FastAPI + Pydantic service that fronts every chat turn. The HTTP entry point is
|
|
| 14 |
|
| 15 |
| File | Role | Related ADR |
|
| 16 |
| --- | --- | --- |
|
| 17 |
-
| `needs_finder.py` | 9-slot fact-find SCHEMA (`GRAPH`). Post-KI-167 the `prompt_en` / `prompt_hi` strings are dead text β the LLM owns voice + cadence end-to-end via its system prompt. The data structure is retained as the schema source for `
|
| 18 |
-
| `
|
| 19 |
| `sales_brain_normalizer.py` | **KI-167 deterministic post-processor.** Pure-rule mapping of the LLM's loose `captures` dict to canonical `{field: validated_value}`: alias resolution (`location` β `location_tier`), enum coercion (`Bangalore` β `metro`), INR parsing, type / bounds validation, KI-094 null-drop. No LLM calls. | [ADR-039](../70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md) |
|
| 20 |
| ~~`fact_find_brain.py`~~ *(deleted in KI-167)* | Was the ADR-030 one-call brain with `<FF>...</FF>` trailer convention + `_canonical_fallback`. Removed entirely. | superseded by [ADR-039](../70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md) |
|
| 21 |
| ~~`question_paraphraser.py`~~ *(deleted in KI-070)* | Was an LLM rewrite of canonical slot questions (ADR-027). Superseded first by `fact_find_brain.py`, now by `sales_brain.py`. | superseded by [ADR-039](../70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md) |
|
|
|
|
| 14 |
|
| 15 |
| File | Role | Related ADR |
|
| 16 |
| --- | --- | --- |
|
| 17 |
+
| `needs_finder.py` | 9-slot fact-find SCHEMA (`GRAPH`). Post-KI-167 the `prompt_en` / `prompt_hi` strings are dead text β the LLM owns voice + cadence end-to-end via its system prompt. The data structure is retained as the schema source for `single_brain.py`. | [ADR-039](../70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md), [ADR-040](../70-docs/60-decisions/ADR-040-google-gemini-primary.md) |
|
| 18 |
+
| `single_brain.py` | **KI-225 single-LLM brain (2026-05-15).** One Gemini-with-function-calling call per turn. Tools: `save_profile_field`, `retrieve_policies`, `get_policy_facts`, `mark_recommendation`. Replaced the multi-brain stack (`sales_brain` + `qa_brain` + `faithfulness` + `persona` + `translator` + `profile_extractor` + `orchestrator`, ~5,200 LOC removed). | [ADR-040](../70-docs/60-decisions/ADR-040-google-gemini-primary.md), KI-225 |
|
| 19 |
| `sales_brain_normalizer.py` | **KI-167 deterministic post-processor.** Pure-rule mapping of the LLM's loose `captures` dict to canonical `{field: validated_value}`: alias resolution (`location` β `location_tier`), enum coercion (`Bangalore` β `metro`), INR parsing, type / bounds validation, KI-094 null-drop. No LLM calls. | [ADR-039](../70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md) |
|
| 20 |
| ~~`fact_find_brain.py`~~ *(deleted in KI-167)* | Was the ADR-030 one-call brain with `<FF>...</FF>` trailer convention + `_canonical_fallback`. Removed entirely. | superseded by [ADR-039](../70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md) |
|
| 21 |
| ~~`question_paraphraser.py`~~ *(deleted in KI-070)* | Was an LLM rewrite of canonical slot questions (ADR-027). Superseded first by `fact_find_brain.py`, now by `sales_brain.py`. | superseded by [ADR-039](../70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md) |
|
|
@@ -10,7 +10,7 @@ Every external model is fronted by a small typed client here. The orchestrator a
|
|
| 10 |
| `nvidia_nim_llm.py` | NVIDIA NIM | Core chain runner β `NimChainLLM(chain=[...])` uses probe-driven primary election (KI-080): calls the elected PRIMARY once per turn, falls to elected BACKUP on real-time failure. Exposes `get_brain_llm()` (the separate `get_fast_brain_llm()` / `get_judge_llm()` accessors were collapsed into it in the 2026-05-15 three-chain consolidation). Legacy `_balanced_brain_chain()` (50/50 NIM β Groq rotator) retained as a bypassed feature-flag branch for one-release rollback. | [ADR-019](../../70-docs/60-decisions/ADR-019-nim-single-provider-consolidation.md), [ADR-031](../../70-docs/60-decisions/ADR-031-sticky-primary-election.md) (supersedes [ADR-026](../../70-docs/60-decisions/ADR-026-provider-load-balancing.md)) |
|
| 11 |
| `groq_llm.py` | Groq | Single-call Llama-3.3-70B client. Used as cross-provider backup election candidate (KI-080) for both brain + fast-brain chains. | [ADR-031](../../70-docs/60-decisions/ADR-031-sticky-primary-election.md) |
|
| 12 |
| `openrouter_llm.py` | OpenRouter | Multi-model fallback rung (DeepSeek-V3 etc.) for chains; rarely the primary in production. | β |
|
| 13 |
-
| `sarvam_llm.py` | Sarvam-M | Indic-aware LLM
|
| 14 |
| `sarvam_stt.py` | Sarvam Saarika v2.5 | Speech-to-text (10 Indic languages + English). | ADR-006 |
|
| 15 |
| `sarvam_tts.py` | Sarvam Bulbul v2 | Text-to-speech; returns base64 WAV the frontend mounts in the in-DOM `<audio>` element. | ADR-006 |
|
| 16 |
| `voyage_embeddings.py` | Voyage AI | Original ingest-time embedder. **Not on the hot path** β query-time uses Chroma vectors directly. Configured in `.env` for occasional re-ingest. | [ADR-011](../../70-docs/60-decisions/ADR-011-bge-local-embeddings.md) |
|
|
|
|
| 10 |
| `nvidia_nim_llm.py` | NVIDIA NIM | Core chain runner β `NimChainLLM(chain=[...])` uses probe-driven primary election (KI-080): calls the elected PRIMARY once per turn, falls to elected BACKUP on real-time failure. Exposes `get_brain_llm()` (the separate `get_fast_brain_llm()` / `get_judge_llm()` accessors were collapsed into it in the 2026-05-15 three-chain consolidation). Legacy `_balanced_brain_chain()` (50/50 NIM β Groq rotator) retained as a bypassed feature-flag branch for one-release rollback. | [ADR-019](../../70-docs/60-decisions/ADR-019-nim-single-provider-consolidation.md), [ADR-031](../../70-docs/60-decisions/ADR-031-sticky-primary-election.md) (supersedes [ADR-026](../../70-docs/60-decisions/ADR-026-provider-load-balancing.md)) |
|
| 11 |
| `groq_llm.py` | Groq | Single-call Llama-3.3-70B client. Used as cross-provider backup election candidate (KI-080) for both brain + fast-brain chains. | [ADR-031](../../70-docs/60-decisions/ADR-031-sticky-primary-election.md) |
|
| 12 |
| `openrouter_llm.py` | OpenRouter | Multi-model fallback rung (DeepSeek-V3 etc.) for chains; rarely the primary in production. | β |
|
| 13 |
+
| `sarvam_llm.py` | Sarvam-M | Indic-aware LLM. Used directly by `backend/single_brain.py` for Indic outputs β Sarvam-M routing is conditional on `_detect_language(user_text)` returning `'indic'`. | [ADR-006](../../70-docs/60-decisions/ADR-006-sarvam-first-stack.md) |
|
| 14 |
| `sarvam_stt.py` | Sarvam Saarika v2.5 | Speech-to-text (10 Indic languages + English). | ADR-006 |
|
| 15 |
| `sarvam_tts.py` | Sarvam Bulbul v2 | Text-to-speech; returns base64 WAV the frontend mounts in the in-DOM `<audio>` element. | ADR-006 |
|
| 16 |
| `voyage_embeddings.py` | Voyage AI | Original ingest-time embedder. **Not on the hot path** β query-time uses Chroma vectors directly. Configured in `.env` for occasional re-ingest. | [ADR-011](../../70-docs/60-decisions/ADR-011-bge-local-embeddings.md) |
|
|
@@ -75,7 +75,7 @@ RESPONSE TTS UI RENDER
|
|
| 75 |
| 9 SCORECARD | per-policy grade | `kb/policies/<policy_id>.md` (live in code via `backend/scorecard.py`) | `rag/build_kb.py` + `backend/scorecard.py` | re-run `build_kb` |
|
| 76 |
| 10 RETRIEVAL (runtime) | top-k chunks | _ephemeral, logged_ | `rag/retrieve.py` | replay query against Chroma |
|
| 77 |
| 11 ORCHESTRATION (runtime) | classified intent + brain | _logged_ | `backend/orchestrator.py` | `logs/turns.jsonl` |
|
| 78 |
-
| 12
|
| 79 |
| 13 RESPONSE | bot reply + cited chunks | _returned to UI_ | `backend/main.py:/api/chat` | curl the API |
|
| 80 |
| 14 TTS (runtime) | base64 WAV | _returned to UI_ | `backend/providers/sarvam_tts.py` | save + play |
|
| 81 |
| 15 UI (runtime) | rendered chat | _browser DOM_ | `frontend/src/app/page.tsx` | browser dev-tools |
|
|
@@ -120,7 +120,7 @@ Every architectural decision in `70-docs/decisions.md` produces a specific artif
|
|
| 120 |
| D-005 Next.js + FastAPI | `frontend/` + `backend/main.py` |
|
| 121 |
| D-006 Sarvam-first | `backend/providers/sarvam_*.py` |
|
| 122 |
| D-007 Pricing illustrative only | (no `actual_premium` field anywhere) |
|
| 123 |
-
| D-008 Consultative persona | `backend/
|
| 124 |
| D-009 10 insurers / all health policies | `40-data/corpus_urls.md` (75 URLs) |
|
| 125 |
| D-010 Secret handling | `.env` (chmod 600 + gitignored) |
|
| 126 |
| D-011 Voyage embeddings | _superseded by local BGE; see D-011 revision_ |
|
|
@@ -206,7 +206,7 @@ Three back-to-back curation passes brought the `40-data/policy_facts/` directory
|
|
| 206 |
|
| 207 |
The day's sprint produced a chain-level rewrite, a provider integration, and a fact-find rip-out. See [`70-docs/40-evaluation/quality-sprint-2026-05-14.md`](../70-docs/40-evaluation/quality-sprint-2026-05-14.md) (Follow-on sprint section) for the full KI list.
|
| 208 |
|
| 209 |
-
- **KI-167 / [ADR-039](../70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md)** β Ripped out `backend/fact_find_brain.py` (441 LOC) + `_canonical_fallback` + the `<FF>{...}</FF>` trailer convention. Replaced with `backend/sales_brain.py` β one LLM call per turn against `FAST_BRAIN_CHAIN` using native provider JSON mode (`response_mime_type=application/json` on Gemini, `response_format={"type":"json_object"}` on NIM). New deterministic post-processor `backend/sales_brain_normalizer.py` validates and normalises captured fields.
|
| 210 |
- **KI-168** β Hybrid voice capture: Web Speech API streams interim transcripts to chat input (live UX feel) while `MediaRecorder` runs in parallel; on browser silence-detect the audio blob is POSTed to `/api/transcribe` (Sarvam Saarika v2.5) for the authoritative transcript. Auto-submits the Sarvam result; falls back to Web Speech only if Sarvam errors.
|
| 211 |
- **KI-171** β Skip faithfulness Gate 4 (LLM judge) on `fact_find` + `recommendation` intents β no retrieved context to grade against on those turns.
|
| 212 |
- **KI-173 / KI-174** β Voice heartbeat + `visibilitychange` / `focus` revival keeps the mic alive across tab / app switches.
|
|
|
|
| 75 |
| 9 SCORECARD | per-policy grade | `kb/policies/<policy_id>.md` (live in code via `backend/scorecard.py`) | `rag/build_kb.py` + `backend/scorecard.py` | re-run `build_kb` |
|
| 76 |
| 10 RETRIEVAL (runtime) | top-k chunks | _ephemeral, logged_ | `rag/retrieve.py` | replay query against Chroma |
|
| 77 |
| 11 ORCHESTRATION (runtime) | classified intent + brain | _logged_ | `backend/orchestrator.py` | `logs/turns.jsonl` |
|
| 78 |
+
| 12 STRUCTURAL GROUNDING (runtime) | brain output | _structural β brain quotes only what tools returned_ | `backend/single_brain.py` | `logs/turns.jsonl` |
|
| 79 |
| 13 RESPONSE | bot reply + cited chunks | _returned to UI_ | `backend/main.py:/api/chat` | curl the API |
|
| 80 |
| 14 TTS (runtime) | base64 WAV | _returned to UI_ | `backend/providers/sarvam_tts.py` | save + play |
|
| 81 |
| 15 UI (runtime) | rendered chat | _browser DOM_ | `frontend/src/app/page.tsx` | browser dev-tools |
|
|
|
|
| 120 |
| D-005 Next.js + FastAPI | `frontend/` + `backend/main.py` |
|
| 121 |
| D-006 Sarvam-first | `backend/providers/sarvam_*.py` |
|
| 122 |
| D-007 Pricing illustrative only | (no `actual_premium` field anywhere) |
|
| 123 |
+
| D-008 Consultative persona | `backend/single_brain.py` (SYSTEM_PROMPT β consultative persona is folded into the single-brain prompt) |
|
| 124 |
| D-009 10 insurers / all health policies | `40-data/corpus_urls.md` (75 URLs) |
|
| 125 |
| D-010 Secret handling | `.env` (chmod 600 + gitignored) |
|
| 126 |
| D-011 Voyage embeddings | _superseded by local BGE; see D-011 revision_ |
|
|
|
|
| 206 |
|
| 207 |
The day's sprint produced a chain-level rewrite, a provider integration, and a fact-find rip-out. See [`70-docs/40-evaluation/quality-sprint-2026-05-14.md`](../70-docs/40-evaluation/quality-sprint-2026-05-14.md) (Follow-on sprint section) for the full KI list.
|
| 208 |
|
| 209 |
+
- **KI-167 / [ADR-039](../70-docs/60-decisions/ADR-039-llm-driven-sales-brain.md)** β Ripped out `backend/fact_find_brain.py` (441 LOC) + `_canonical_fallback` + the `<FF>{...}</FF>` trailer convention. Replaced with `backend/sales_brain.py` β one LLM call per turn against `FAST_BRAIN_CHAIN` using native provider JSON mode (`response_mime_type=application/json` on Gemini, `response_format={"type":"json_object"}` on NIM). New deterministic post-processor `backend/sales_brain_normalizer.py` validates and normalises captured fields. **Subsequent KI-225 (2026-05-15) further consolidated:** `sales_brain.py` + `qa_brain.py` + `faithfulness.py` + `persona.py` + `translator.py` + `profile_extractor.py` + `orchestrator.py` were all removed (~5,200 LOC net) and replaced by `backend/single_brain.py` β one Gemini-with-function-calling call per turn. Structural grounding replaced the faithfulness gate.
|
| 210 |
- **KI-168** β Hybrid voice capture: Web Speech API streams interim transcripts to chat input (live UX feel) while `MediaRecorder` runs in parallel; on browser silence-detect the audio blob is POSTed to `/api/transcribe` (Sarvam Saarika v2.5) for the authoritative transcript. Auto-submits the Sarvam result; falls back to Web Speech only if Sarvam errors.
|
| 211 |
- **KI-171** β Skip faithfulness Gate 4 (LLM judge) on `fact_find` + `recommendation` intents β no retrieved context to grade against on those turns.
|
| 212 |
- **KI-173 / KI-174** β Voice heartbeat + `visibilitychange` / `focus` revival keeps the mic alive across tab / app switches.
|
|
@@ -25,20 +25,16 @@ short-circuit to the catalogued card.
|
|
| 25 |
|
| 26 |
All gates run for EVERY upload. Block on any failure; the audit trail captures the reason set. See README Β§2.8 and `70-docs/60-decisions/ADR-044-uploaded-pdf-parity.md` for the dual-write model and the heuristic-floor / Gemini extraction chain.
|
| 27 |
|
| 28 |
-
## Hallucination defense β
|
| 29 |
|
| 30 |
-
|
| 31 |
-
| --- | --- | --- |
|
| 32 |
-
| 1 | Retrieval floor | Top-1 cosine < 0.30 OR avg top-5 < 0.22 β refuse outright |
|
| 33 |
-
| 2 | Citation integrity | Any `[Source:β¦]` in the bot's reply must point to a real retrieved chunk's policy_name |
|
| 34 |
-
| 3 | Numeric grounding | Every βΉ, %, day/month/year in the reply must appear in retrieved chunks (regex) |
|
| 35 |
-
| 4 | LLM-judge faithfulness | NIM Mistral Large 3 675B (primary judge per D-022) + Llama-4 Maverick (fallback) inspects the reply against retrieved chunks; outputs strict JSON; different family from Qwen 3-Next 80B brain (primary per D-022) + DeepSeek-V4 (fallback) β non-circular eval (D-019). |
|
| 36 |
-
| 5 (Indic) | Hinglish drift LLM-judge | Same idea on the Hinglish back-translation vs the English source |
|
| 37 |
|
| 38 |
-
|
| 39 |
-
|
|
|
|
|
|
|
| 40 |
|
| 41 |
-
|
| 42 |
|
| 43 |
## What WE can't (yet) check
|
| 44 |
|
|
|
|
| 25 |
|
| 26 |
All gates run for EVERY upload. Block on any failure; the audit trail captures the reason set. See README Β§2.8 and `70-docs/60-decisions/ADR-044-uploaded-pdf-parity.md` for the dual-write model and the heuristic-floor / Gemini extraction chain.
|
| 27 |
|
| 28 |
+
## Hallucination defense β structural grounding (post-KI-225, 2026-05-15)
|
| 29 |
|
| 30 |
+
The single brain (`backend/single_brain.py`) quotes only what its tools returned:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 31 |
|
| 32 |
+
| Tool | What it returns | Where the brain reads it |
|
| 33 |
+
|---|---|---|
|
| 34 |
+
| `retrieve_policies` | top-k policy-wording chunks from Chroma | `backend/brain_tools.py::retrieve_policies` |
|
| 35 |
+
| `get_policy_facts` | curated structured facts + verbatim `source_quote` | `backend/brain_tools.py::get_policy_facts` |
|
| 36 |
|
| 37 |
+
The brain's system prompt enforces "cite only what the tools returned" as a structural invariant. The pre-KI-225 architecture had a separate `backend/faithfulness.py` 4-gate post-hoc verifier β that module was removed in the single-brain consolidation because the single LLM's tool-grounded output flow makes the post-hoc verification structurally unnecessary. Source: ADR-040 + KI-225.
|
| 38 |
|
| 39 |
## What WE can't (yet) check
|
| 40 |
|
|
@@ -27,7 +27,7 @@ Lineage for every artefact below is documented in [`kb/AUDIT_TRAIL.md`](../kb/AU
|
|
| 27 |
|
| 28 |
| Path | Source of truth | Notes |
|
| 29 |
| --- | --- | --- |
|
| 30 |
-
| `rag/corpus/<insurer>/*.pdf` | insurer CDNs | 206 PDFs (188 product PDFs across
|
| 31 |
| `rag/extracted/<policy_id>.json` | `extract.py` | 201 JSONs, one per policy, conforming to `schema.HealthPolicy`. Generated; never hand-edit. |
|
| 32 |
| `rag/vectors/chroma.sqlite3` + HNSW binaries | `ingest.py` | Persistent Chroma store. Symlinked to `rag/_hf_dataset_backup/rag/vectors/` for the offline canonical copy. |
|
| 33 |
| `rag/policies.duckdb` | `extract.py` | DuckDB rollup of the 62-field JSONs; used for SQL-style filters in `backend/main.py`. |
|
|
|
|
| 27 |
|
| 28 |
| Path | Source of truth | Notes |
|
| 29 |
| --- | --- | --- |
|
| 30 |
+
| `rag/corpus/<insurer>/*.pdf` | insurer CDNs | 206 PDFs (188 product PDFs across 21 insurer slugs + 18 regulatory IRDAI/NHA docs; dedup to 148 marketplace cards). Not in git β hydrated at Docker build from the companion HF dataset. |
|
| 31 |
| `rag/extracted/<policy_id>.json` | `extract.py` | 201 JSONs, one per policy, conforming to `schema.HealthPolicy`. Generated; never hand-edit. |
|
| 32 |
| `rag/vectors/chroma.sqlite3` + HNSW binaries | `ingest.py` | Persistent Chroma store. Symlinked to `rag/_hf_dataset_backup/rag/vectors/` for the offline canonical copy. |
|
| 33 |
| `rag/policies.duckdb` | `extract.py` | DuckDB rollup of the 62-field JSONs; used for SQL-style filters in `backend/main.py`. |
|