rohitsar567 Claude Opus 4.7 (1M context) commited on
Commit
2e60476
·
1 Parent(s): c53381c

docs: KI-071 — reconcile docs with code reality (Qwen 80B / Nemotron 30B / Mistral 675B as current primaries)

Browse files

Source-of-truth audit found the production code on `main` runs Qwen
3-Next 80B (brain primary) + Nemotron Nano 30B (fast brain primary) +
Mistral Large 3 675B (judge primary) per the D-022 inline brain swap on
2026-05-14 (NIM DeepSeek + Llama-4 Maverick pools were timing out for
hours). The ADR-019 decision text + multiple docs still referenced
DeepSeek V4-Pro / V4-Flash + Llama-4 Maverick as primaries.

This commit aligns 12 files with the actual chain definitions in
`backend/providers/nvidia_nim_llm.py`. DeepSeek V4-Pro / V4-Flash +
Llama-4 Maverick are correctly documented now as **fallback chain
entries** that re-enter rotation when NIM's pools recover.

Files reconciled (+64 / -45):
- `README.md` — §1.2 "Honest model picks" paragraph + §4.3 Model stack
table rewritten. Heavy brain row now: "Qwen 3-Next 80B *or* Groq
Llama-3.3-70B (50/50 rotation)". Fast brain: "Nemotron Nano 30B".
Judge: "Mistral Large 3 675B". Full fallback chains documented.
- `70-docs/60-decisions/ADR-019` — partial-supersession note added
under Status: documents the D-022 brain swap, points to current code
for source of truth.
- `kb/security/INDEX.md` + `kb/eval/INDEX.md` — Gate-4 / grader rows
now name Mistral 675B primary + Qwen brain (was DeepSeek-V4 + Llama-4).
- `frontend/README.md` — 3× wrong ADR-028 filename fixed; env var
`NEXT_PUBLIC_API_BASE` → `NEXT_PUBLIC_BACKEND_URL` (matches Dockerfile).
- `rag/README.md` — extractor line now cites `get_brain_llm()` /
`get_fast_brain_llm()` instead of "Sarvam-M + DeepSeek-V3 fallback"
(deeply stale); 2× ADR-029 filename fixed.
- `eval/README.md` — judge label corrected to Mistral 675B primary
(Groq is 6th fallback in JUDGE_CHAIN, not primary as it was labelled).
- `frontend/src/lib/i18n.ts` — dead `input.hands_free` keys (EN + HI)
removed (KI-027 retired Hands-free mode). `welcome.subtitle` (EN +
HI) softened from "8-10 short questions" to "a few short questions"
since post-KI-070 turn count is variable.
- `tools/README.md` — 2× ADR-029 broken link fixed.
- `40-data/profiles/README.md` — 1× ADR-029 broken link fixed.
- `backend/providers/__init__.py` — module docstring rewritten:
chain primaries listed, D-022 swap acknowledged, openrouter + groq
documented as cross-provider fallbacks (they exist on disk, were
incorrectly described as "retired" in the old docstring).
- `rag/extract.py:341-342` — inline primary/fallback comments fixed.

Verified no `DeepSeek V4-Pro` claim remains as PRIMARY in any doc;
DeepSeek mentions appear only in fallback chain context.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

40-data/profiles/README.md CHANGED
@@ -49,7 +49,7 @@ The original design considered embedding-only. The "why JSON" trade-offs:
49
  - **Deterministic name lookup.** `Rohit` → `rohit.json` is exact; vector search is approximate and can collide on common first names.
50
  - **Human-readable.** A BFSI auditor can `cat` the file and see the full profile.
51
  - **Manually editable.** Quick repair without a re-embed pipeline.
52
- - **No HNSW bloat exposure.** Profile updates do not touch the policy vector store ([ADR-029](../70-docs/60-decisions/ADR-029-disk-storage-hardening.md)).
53
 
54
  The Chroma chunk is purely a retrieval-time view of the canonical JSON.
55
 
 
49
  - **Deterministic name lookup.** `Rohit` → `rohit.json` is exact; vector search is approximate and can collide on common first names.
50
  - **Human-readable.** A BFSI auditor can `cat` the file and see the full profile.
51
  - **Manually editable.** Quick repair without a re-embed pipeline.
52
+ - **No HNSW bloat exposure.** Profile updates do not touch the policy vector store ([ADR-029](../70-docs/60-decisions/ADR-029-hnsw-bloat-tripwire.md)).
53
 
54
  The Chroma chunk is purely a retrieval-time view of the canonical JSON.
55
 
70-docs/60-decisions/ADR-019-nim-single-provider-consolidation.md CHANGED
@@ -3,6 +3,8 @@
3
  **Status:** Locked
4
  **Date:** 2026-05-14
5
 
 
 
6
  ## Context
7
 
8
  By mid-May 2026 the LLM stack had accreted **four third-party providers** in overlapping roles, each with its own free-tier ceiling that surfaced as quality problems:
 
3
  **Status:** Locked
4
  **Date:** 2026-05-14
5
 
6
+ **Partial supersession (D-022, 2026-05-14):** NIM's DeepSeek V4-Pro / V4-Flash and Llama-4 Maverick inference pools degraded with hours-long timeouts during a production incident. The chains were re-ordered to put **Qwen 3-Next 80B** (brain primary), **Nemotron Nano 30B** (fast brain primary), and **Mistral Large 3 675B** (judge primary) at the head, with DeepSeek + Maverick retained as fallback chain entries that re-enter rotation when those pools recover. The single-provider consolidation decision in this ADR still stands (NIM remains the primary inference provider); only the *specific model picks* were swapped. See `backend/providers/nvidia_nim_llm.py` for the current chain definitions.
7
+
8
  ## Context
9
 
10
  By mid-May 2026 the LLM stack had accreted **four third-party providers** in overlapping roles, each with its own free-tier ceiling that surfaced as quality problems:
README.md CHANGED
@@ -71,7 +71,15 @@ A take-home is a sample of how the engineer thinks under constraint. Three thing
71
 
72
  2. **Hallucination defense and refusal as product features.** BFSI deployments get fined for mis-selling; the bot is biased toward refusal over confident wrong answers. The 4 faithfulness gates + cross-check retry + 3 Indic drift checks + audit log are the BFSI-compliance-grade version of "we shipped a chatbot." When the eval shows a headline accuracy below 100% because the gates are aggressive, the right response is to soften the gates carefully — not to ship a higher number by relaxing the verifier.
73
 
74
- 3. **Honest model picks — Sarvam where Sarvam is uniquely strong, open-weights frontier for reasoning.** Sarvam Saarika v2.5 STT, Bulbul v2 TTS, and Sarvam-M Indic translation are *non-substitutable* — no closed-source frontier matches them on Indian voice or Hinglish. Reasoning is a different problem; the brain runs a fallback chain whose primary rotates 50/50 between **NIM Qwen 3-Next 80B** and **Groq Llama-3.3-70B** (ADR-026 — 2× sustained throughput across two free-tier providers), with the judge on **Mistral Large 3 675B** for non-circular grading. A Sarvam customer deploying this stack gets a product that *uses Sarvam exactly where Sarvam beats the world* and uses MIT-licensed frontier weights for everything else — open-weights only, $0 inference, single API key per provider for the entire non-voice stack.
 
 
 
 
 
 
 
 
75
 
76
  The rest is craftsmanship. The 8-section KB ([`kb/`](kb/)) is regeneratable from primary sources in <40 minutes for <$2 cold. Every numeric value in every reviewer-facing artifact traces to a source PDF + page + clause. Every architectural decision is in [`70-docs/60-decisions/`](70-docs/60-decisions/) with alternatives and revisit-at-scale notes. Every production-readiness defect is in [`80-audit/ENTERPRISE_AUDIT.md`](80-audit/ENTERPRISE_AUDIT.md). The repo is structured so a new engineer joining on Monday could ship v1.1 by Friday.
77
 
@@ -540,23 +548,24 @@ The bot is two flows running together — the customer's experience and the tech
540
 
541
  ### 4.3 Model stack
542
 
543
- Every LLM role is served by a **fallback chain** of candidate models (`backend/providers/nvidia_nim_llm.py::NimChainLLM`), not a single hardcoded model. Chains preserve brain↔judge family diversity (Qwen brain ↔ Mistral judge) so failovers can't accidentally produce circular grading.
544
 
545
- | Role | Primary | Fallback chain | Provider mix |
546
- |---|---|---|---|
547
- | STT | Saarika v2.5 | — | Sarvam |
548
- | TTS | Bulbul v2 (`anushka`) | — | Sarvam |
549
- | Indic translation | Sarvam-M | — | Sarvam |
550
- | **Heavy brain** (comparison + recommendation) | Qwen 3-Next 80B *or* Groq Llama-3.3-70B (50/50 rotation) | Qwen 122B → GPT-OSS 120B → Mistral 675B → Nemotron-Super 49B → Llama-3.3-70B → the brain chain (Qwen 80B primary, 50/50 rotated with Groq Llama-3.3) → OpenRouter GPT-OSS → Groq Llama | NIM + Groq + OpenRouter |
551
- | **Fast brain** (fact-find + QA + paraphrase + normalize + extract) | Nemotron Nano 30B *or* Groq Llama-3.3-70B (50/50 rotation) | Qwen 80B → GPT-OSS 120B → Qwen 122B → DeepSeek V4-Flash → Groq | NIM + Groq |
552
- | **Judge** (faithfulness gate + grader) | Mistral Large 3 675B | GPT-OSS 120B → Kimi K2 → MiniMax M2.5 → Mistral Large 3 675B + cross-provider | NIM + cross-provider |
553
- | Embeddings | BGE-small-en-v1.5 (384d) | — | Local (`LocalEmbeddings`) |
554
- | Vector store | Chroma 1.5 (HNSW + sqlite) | — | Local |
555
- | Structured store | DuckDB 1.1.3 | — | Local |
556
-
557
- Per-call budgets: brain `20s × 35s total`, fast-brain `12s × 22s total`, judge `30s × 75s total`. The per-link timeout is dynamically clipped to remaining budget so a single fallback can never blow past the chain ceiling.
558
-
559
- **Provider load balancing.** The brain chain primary rotates 50/50 per call between NIM Qwen and Groq Llama-3.3 (`_balanced_brain_chain` in `backend/providers/nvidia_nim_llm.py`). Spreads load across two independent rate caps (NIM 40 req/min + Groq's separate quota) — effectively 2× sustained brain throughput. See [`ADR-026`](70-docs/60-decisions/ADR-026-provider-load-balancing.md).
 
560
 
561
  ### 4.4 Major design decisions (the short list)
562
 
 
71
 
72
  2. **Hallucination defense and refusal as product features.** BFSI deployments get fined for mis-selling; the bot is biased toward refusal over confident wrong answers. The 4 faithfulness gates + cross-check retry + 3 Indic drift checks + audit log are the BFSI-compliance-grade version of "we shipped a chatbot." When the eval shows a headline accuracy below 100% because the gates are aggressive, the right response is to soften the gates carefully — not to ship a higher number by relaxing the verifier.
73
 
74
+ 3. **Honest model picks — Sarvam where Sarvam is uniquely strong, open-weights frontier for reasoning.** Voice and Indic are non-substitutable: **Sarvam Saarika v2.5** for speech-to-text, **Sarvam Bulbul v2** (speaker `anushka`) for text-to-speech, and **Sarvam-M** for Hindi/Hinglish/vernacular translation — no closed-source frontier matches Sarvam on Indian accents or code-mixed Hinglish. Reasoning is a different problem and runs on open-weights frontier models behind a fallback chain, not a single hardcoded brain. The chain architecture has three roles, each a `NimChainLLM` (`backend/providers/nvidia_nim_llm.py`) with a NIM-primary preference order plus OpenRouter and Groq cross-provider fallbacks:
75
+
76
+ - **`BRAIN_CHAIN`** (comparison, recommendation, synthesis) — primary **Qwen 3-Next 80B** (`qwen/qwen3-next-80b-a3b-instruct`, 80B / 3B-active MoE, multilingual, ~2s TTFT on NIM free tier), with **Qwen 3.5 122B**, **OpenAI GPT-OSS 120B**, **Mistral Large 3 675B**, **NVIDIA Nemotron-Super 49B**, **Meta Llama-3.3 70B**, and **DeepSeek V4-Pro** as in-NIM fallbacks, then **OpenRouter GPT-OSS 120B** and **Groq Llama-3.3-70B** as cross-provider fallbacks for full-NIM-outage survival.
77
+ - **`FAST_BRAIN_CHAIN`** (fact-find turns, QA, paraphrase, normalize, extract — every latency-sensitive role) — primary **NVIDIA Nemotron Nano 30B** (`nvidia/nemotron-3-nano-30b-a3b`, ~1.6s TTFT — bottleneck is TTFT not capability on these jobs), with Qwen 3-Next 80B, GPT-OSS 120B, Qwen 3.5 122B, and **DeepSeek V4-Flash** as NIM fallbacks, then Groq Llama-3.3-70B as the cross-provider fallback.
78
+ - **`JUDGE_CHAIN`** (faithfulness Gate 4, Hinglish drift LLM-judge, eval grader) — primary **Mistral Large 3 675B** (`mistralai/mistral-large-3-675b-instruct-2512`, MIT). Mistral is a deliberately different model family from the Qwen brain so the judge does not mark its own homework. Fallbacks: GPT-OSS 120B, Moonshot Kimi K2, MiniMax M2.5, and **Meta Llama-4 Maverick 17B/128E** (the original D-019 judge — kept in the chain for when NIM's Llama pool recovers), then OpenRouter and Groq cross-provider entries.
79
+
80
+ **Provider load-balancing (KI-025, [ADR-026](70-docs/60-decisions/ADR-026-provider-load-balancing.md)).** The brain and fast-brain chains' *primary* slot rotates 50/50 per call between the NIM Qwen entry and the Groq Llama-3.3-70B entry via `_balanced_brain_chain` (per-call `random.random()`, no shared cycle state). NIM's free-tier rate cap is 40 req/min shared across every model on a key; splitting brain load across two independent free-tier quotas (NIM + Groq) effectively doubles sustained brain throughput. Groq's LPU TTFT (~1s) is often *lower* than NIM Qwen's, so the rotation is a strict latency win on top of the throughput win. A Groq-primary call that fails still gets the full NIM fallback ladder underneath, so reliability is unchanged.
81
+
82
+ The result: a Sarvam customer deploying this stack gets a product that *uses Sarvam exactly where Sarvam beats the world* and uses MIT-licensed open-weights frontier models for everything else — $0 inference, two independent free-tier providers, single-key-per-provider for the entire non-voice stack.
83
 
84
  The rest is craftsmanship. The 8-section KB ([`kb/`](kb/)) is regeneratable from primary sources in <40 minutes for <$2 cold. Every numeric value in every reviewer-facing artifact traces to a source PDF + page + clause. Every architectural decision is in [`70-docs/60-decisions/`](70-docs/60-decisions/) with alternatives and revisit-at-scale notes. Every production-readiness defect is in [`80-audit/ENTERPRISE_AUDIT.md`](80-audit/ENTERPRISE_AUDIT.md). The repo is structured so a new engineer joining on Monday could ship v1.1 by Friday.
85
 
 
548
 
549
  ### 4.3 Model stack
550
 
551
+ Every LLM role is served by a **fallback chain** of candidate models (`backend/providers/nvidia_nim_llm.py::NimChainLLM`), never a single hardcoded model. Chains were curated to preserve brain ↔ judge family diversity (Qwen brain ↔ Mistral judge) so any failover still produces non-circular grading. Cross-provider entries (OpenRouter, Groq) sit at the bottom of every chain so the whole reasoning stack survives a full NIM regional outage.
552
 
553
+ | Role | Primary | Fallback chain (in order) | Provider(s) | Why this primary |
554
+ |---|---|---|---|---|
555
+ | **Heavy brain** (comparison, recommendation, synthesis) | Qwen 3-Next 80B *or* Groq Llama-3.3-70B (50/50 per-call rotation, KI-025) | Qwen 3.5 122B → GPT-OSS 120B → Mistral Large 3 675B → Nemotron-Super 49B → Llama-3.3-70B → DeepSeek V4-Pro → OpenRouter GPT-OSS 120B → Groq Llama-3.3-70B | NIM + Groq + OpenRouter | Qwen 80B is the fastest reliable frontier-tier MoE on NIM free tier (~2s TTFT, clean JSON, multilingual). Groq LPU is the lowest-TTFT free-tier option; 50/50 split doubles sustained throughput across two independent rate caps ([ADR-026](70-docs/60-decisions/ADR-026-provider-load-balancing.md)). |
556
+ | **Fast brain** (voice turns, fact-find, QA, paraphrase, normalize, extract) | Nemotron Nano 30B *or* Groq Llama-3.3-70B (50/50 per-call rotation) | Qwen 3-Next 80B → GPT-OSS 120B → Qwen 3.5 122B → DeepSeek V4-Flash → Groq Llama-3.3-70B | NIM + Groq | Bottleneck on these short jobs is TTFT, not capability. Nemotron Nano hits ~1.6s; Qwen 80B is ~2-3s. Reordered for latency in KI-035. |
557
+ | **Judge** (faithfulness Gate 4, Hinglish drift, eval grader) | Mistral Large 3 675B | GPT-OSS 120B → Kimi K2 → MiniMax M2.5 → Llama-4 Maverick 17B/128E → OpenRouter GPT-OSS 120B → Groq Llama-3.3-70B | NIM + OpenRouter + Groq | Different family from the Qwen brain (Mistral, not Qwen / DeepSeek / Llama family) so the judge sees the brain's output from a genuinely different decision surface. 675B dense, MIT, ~4.3s on NIM. |
558
+ | **Profile extractor** (free-form profile updates → 9-slot schema, ADR-022) | Fast brain chain (Nemotron Nano 30B primary) | inherits FAST_BRAIN_CHAIN | NIM + Groq | Short prompt, structured JSON out — Nemotron Nano is fast and reliable enough; same chain as fact-find. |
559
+ | **Fact-find normalizer** (user answer → typed slot value) | Fast brain chain (Nemotron Nano 30B primary) | inherits FAST_BRAIN_CHAIN | NIM + Groq | Same shape as profile extractor — narrow input, narrow JSON output. |
560
+ | **Question paraphraser** *(deprecated by KI-070)* | Fast brain chain (Nemotron Nano 30B primary), 4s per-link / 6s total budget | inherits FAST_BRAIN_CHAIN | NIM + Groq | ADR-027 layered an LLM paraphraser on top of the hardcoded GRAPH to make fact-find sound less robotic. **KI-070 replaces the paraphraser + opener-rotation + acknowledger machinery with a single LLM brain call per fact-find turn (ADR-030).** The paraphraser module is scheduled for deletion in the KI-070 sync sweep. |
561
+ | **Indic translation** (English ↔ Hindi / Hinglish / vernacular) | Sarvam-M | — (no fallback; Indic is non-substitutable) | Sarvam | Best-in-class on Hinglish and code-mixed Indian languages; no open-weights frontier model is competitive here. |
562
+ | **STT** (speech-to-text) | Sarvam Saarika v2.5 (`saarika:v2.5`) | — | Sarvam | Best-in-class on Indian-accented English and Hinglish; WebM transcoded to 16kHz mono WAV at the gateway (`backend/providers/sarvam_stt.py`). |
563
+ | **TTS** (text-to-speech) | Sarvam Bulbul v2 (`bulbul:v2`, speaker `anushka`) | — | Sarvam | Best-in-class on Hinglish prosody; 22.05kHz output, base64 WAV from Sarvam endpoint. |
564
+ | **Embeddings** | Voyage `voyage-3` (legacy ingest only; hot path uses BGE) | BGE-small-en-v1.5 (384d, local CPU) | Voyage (offline) + Local (hot path) | BGE on-CPU avoids network on every retrieval ([ADR-011](70-docs/60-decisions/ADR-011-bge-local-embeddings.md)); Voyage retained for back-compat with already-extracted artifacts. |
565
+
566
+ **Chain budgets.** Brain `20s` per-link / `35s` total; fast brain `12s` / `22s`; judge `30s` / `75s`. The per-link timeout is dynamically clipped to the remaining chain budget so a single fallback can never blow past the ceiling (KI-021).
567
+
568
+ **Health-aware filtering.** A background probe loop (`backend/llm_health.py`) marks individual NIM models as up/down. `NimChainLLM.filter_chain()` skips known-down models before the first call; if every candidate is filtered, the full chain is tried anyway since the probe may be stale. On total chain exhaustion, one synchronous probe refresh is triggered before raising.
569
 
570
  ### 4.4 Major design decisions (the short list)
571
 
backend/providers/__init__.py CHANGED
@@ -1,16 +1,26 @@
1
  """Provider clients — thin, async, behind a common interface.
2
 
3
- After the 2026-05-14 Stack A consolidation (D-019), the provider stack is:
 
4
 
5
  sarvam_stt.py — Sarvam Saarika v2.5 (speech-to-text)
6
- sarvam_tts.py — Sarvam Bulbul v2 (text-to-speech)
7
- sarvam_llm.py — Sarvam-M (Indic translation IN and OUT; not brain anymore)
8
  local_embeddings.py — BGE-small-en-v1.5 (local CPU embeddings)
9
- nvidia_nim_llm.py — NIM Llama-3.3-70B-Instruct (BRAIN) + Llama-4 Maverick (JUDGE)
 
 
 
 
 
 
 
 
 
 
10
 
11
- Four legacy providers were retired in the same change: openrouter_llm.py,
12
- deepseek_llm.py, cerebras_llm.py, groq_llm.py. Single NIM key replaces all
13
- four. See 70-docs/decisions.md D-019.
14
 
15
  All clients are async (use httpx.AsyncClient) so the FastAPI handlers can
16
  parallelize provider calls without blocking the event loop.
 
1
  """Provider clients — thin, async, behind a common interface.
2
 
3
+ After the 2026-05-14 Stack A consolidation (D-019, partially superseded by
4
+ D-022 on 2026-05-14 — NIM brain pool swap), the provider stack is:
5
 
6
  sarvam_stt.py — Sarvam Saarika v2.5 (speech-to-text)
7
+ sarvam_tts.py — Sarvam Bulbul v2 (text-to-speech, speaker anushka)
8
+ sarvam_llm.py — Sarvam-M (Indic translation IN and OUT; not brain)
9
  local_embeddings.py — BGE-small-en-v1.5 (local CPU embeddings)
10
+ nvidia_nim_llm.py — Chain pattern via NimChainLLM. Current primaries (D-022):
11
+ BRAIN_CHAIN[0] = Qwen 3-Next 80B
12
+ FAST_BRAIN_CHAIN[0] = Nemotron Nano 30B
13
+ JUDGE_CHAIN[0] = Mistral Large 3 675B
14
+ DeepSeek V4-Pro / V4-Flash + Llama-4 Maverick (the
15
+ original D-019 picks) are retained as fallback chain
16
+ entries that re-enter rotation when NIM's pools
17
+ recover from the 2026-05-14 outage.
18
+ openrouter_llm.py — OpenRouter cross-provider fallback (bottom of every chain).
19
+ groq_llm.py — Groq LPU; brain primary rotates 50/50 NIM ↔ Groq
20
+ per call (KI-025 / ADR-026).
21
 
22
+ See 70-docs/60-decisions/ADR-019, ADR-026, ADR-030 and 80-audit/ENTERPRISE_AUDIT.md
23
+ (D-022 row) for the full provenance.
 
24
 
25
  All clients are async (use httpx.AsyncClient) so the FastAPI handlers can
26
  parallelize provider calls without blocking the event loop.
eval/README.md CHANGED
@@ -10,7 +10,7 @@ Numerical accuracy + grounding eval. Walks a fixed list of curated questions thr
10
  | --- | --- |
11
  | `generate_gold.py` | Builds `gold_qa.json` from `40-data/policy_facts/`: for each curated field with a verbatim quote, emits a natural-language question + expected answer + expected citation. |
12
  | `gold_qa.json` | 96-Q gold set. Each entry: `{policy_id, question, expected_answer, expected_regex, source_quote, source_pdf}`. |
13
- | `run.py` | Runner. For each pair: calls `backend.orchestrator.handle_turn` with `policy_filter_ids=[pair.policy_id]` so retrieval is scoped to the policy under test. Grades each reply twice — regex hard-facts + Groq Llama judge (different family from the NIM brain → non-circular). |
14
  | `results.json` | Machine-readable last-run results. |
15
  | `results.md` | Human-readable last-run report — per-question pass/fail, per-category rollup, hallucination breakdown. |
16
  | `info_source_map.json` | Generated by `tools/info_source_map.py`. Claim → URL → verdict (✅ 798 / ⚠️ 321 / ❌ 0 / ⏳ 1385 as of 2026-05-14). The canonical source-grounding KPI. |
@@ -39,7 +39,7 @@ python -m eval.generate_gold
39
  | Layer | What it checks | Output field |
40
  | --- | --- | --- |
41
  | Regex hard-facts | Expected numeric / boolean / waiting-period value appears verbatim in `reply_text`. | `regex_pass` |
42
- | LLM judge (Groq Llama-3.3-70B) | Reply *semantically* answers the question and cites the expected source. Different family from the NIM Qwen brain → non-circular. | `judge_pass` |
43
  | Faithfulness gate | The 4-gate guard in `backend/faithfulness.py` already ran during `handle_turn`; eval records whether it blocked. | `blocked` |
44
 
45
  ## Acceptance bar
 
10
  | --- | --- |
11
  | `generate_gold.py` | Builds `gold_qa.json` from `40-data/policy_facts/`: for each curated field with a verbatim quote, emits a natural-language question + expected answer + expected citation. |
12
  | `gold_qa.json` | 96-Q gold set. Each entry: `{policy_id, question, expected_answer, expected_regex, source_quote, source_pdf}`. |
13
+ | `run.py` | Runner. For each pair: calls `backend.orchestrator.handle_turn` with `policy_filter_ids=[pair.policy_id]` so retrieval is scoped to the policy under test. Grades each reply twice — regex hard-facts + NIM Mistral Large 3 judge chain (different family from the NIM Qwen brain → non-circular). Falls through to Groq Llama only at the cross-provider tail of `JUDGE_CHAIN`. |
14
  | `results.json` | Machine-readable last-run results. |
15
  | `results.md` | Human-readable last-run report — per-question pass/fail, per-category rollup, hallucination breakdown. |
16
  | `info_source_map.json` | Generated by `tools/info_source_map.py`. Claim → URL → verdict (✅ 798 / ⚠️ 321 / ❌ 0 / ⏳ 1385 as of 2026-05-14). The canonical source-grounding KPI. |
 
39
  | Layer | What it checks | Output field |
40
  | --- | --- | --- |
41
  | Regex hard-facts | Expected numeric / boolean / waiting-period value appears verbatim in `reply_text`. | `regex_pass` |
42
+ | LLM judge (NIM Mistral Large 3 675B primary; OpenAI GPT-OSS 120B / Moonshot Kimi K2 / Llama-4 Maverick / Groq Llama-3.3-70B fallbacks) | Reply *semantically* answers the question and cites the expected source. Different family from the NIM Qwen brain → non-circular. | `judge_pass` |
43
  | Faithfulness gate | The 4-gate guard in `backend/faithfulness.py` already ran during `handle_turn`; eval records whether it blocked. | `blocked` |
44
 
45
  ## Acceptance bar
frontend/README.md CHANGED
@@ -12,10 +12,10 @@ For Claude / agent-specific rules (e.g. the "this is NOT the Next.js you know" n
12
  | `src/app/layout.tsx` | Root layout + font wiring (Geist via `next/font`). |
13
  | `src/app/globals.css` | Tailwind v4 entrypoint + the shadcn/ui token layer ([ADR-013](../70-docs/60-decisions/ADR-013-tailwind-shadcn-ui.md)). |
14
  | `src/lib/api.ts` | Typed API client. Generated types via `openapi-typescript` from the FastAPI OpenAPI schema ([ADR-015](../70-docs/60-decisions/ADR-015-openapi-typescript-codegen.md)). |
15
- | `src/lib/useLiveConversation.ts` | The continuously-open-mic VAD hook that powers Live mode + barge-in. State persists in `localStorage.insurance_live_pref` ([ADR-028](../70-docs/60-decisions/ADR-028-voice-ux-live-default.md)). |
16
  | `src/lib/i18n.ts` | EN ↔ हिं strings + the 13-term `GLOSSARY` mirrored to [`kb/methodology/glossary.json`](../kb/methodology/glossary.json). |
17
 
18
- ## Voice UX invariants ([ADR-028](../70-docs/60-decisions/ADR-028-voice-ux-live-default.md))
19
 
20
  - **One default voice mode + one fallback.** Live mode is the default; the toolbar pill toggles it (green = on, red = off). The 🎤 push-to-talk button suspends Live for one turn → captures with VAD silence-cutoff → resumes Live if the user preference is still on.
21
  - **Hands-free mode was removed in KI-027.** Any reference to it in code or docs is stale.
@@ -40,11 +40,11 @@ npm install # or pnpm / bun
40
  npm run dev # localhost:3000 — points at the FastAPI on :8000 by default
41
  ```
42
 
43
- Point at a different backend with `NEXT_PUBLIC_API_BASE=...` (see `src/lib/api.ts`).
44
 
45
  ## Related
46
 
47
  - `AGENTS.md` (this folder) — required reading for AI agents touching this code
48
  - Root `CLAUDE.md` § Voice UX (ADR-028) — the canonical voice-UX cheat-sheet
49
- - [ADR-005](../70-docs/60-decisions/ADR-005-nextjs-fastapi-frontend.md) · [ADR-013](../70-docs/60-decisions/ADR-013-tailwind-shadcn-ui.md) · [ADR-015](../70-docs/60-decisions/ADR-015-openapi-typescript-codegen.md) · [ADR-021](../70-docs/60-decisions/ADR-021-view-aware-system-prompt.md) · [ADR-028](../70-docs/60-decisions/ADR-028-voice-ux-live-default.md)
50
  - [`backend/main.py`](../backend/main.py) — the API the frontend talks to
 
12
  | `src/app/layout.tsx` | Root layout + font wiring (Geist via `next/font`). |
13
  | `src/app/globals.css` | Tailwind v4 entrypoint + the shadcn/ui token layer ([ADR-013](../70-docs/60-decisions/ADR-013-tailwind-shadcn-ui.md)). |
14
  | `src/lib/api.ts` | Typed API client. Generated types via `openapi-typescript` from the FastAPI OpenAPI schema ([ADR-015](../70-docs/60-decisions/ADR-015-openapi-typescript-codegen.md)). |
15
+ | `src/lib/useLiveConversation.ts` | The continuously-open-mic VAD hook that powers Live mode + barge-in. State persists in `localStorage.insurance_live_pref` ([ADR-028](../70-docs/60-decisions/ADR-028-voice-ux-single-default-mode.md)). |
16
  | `src/lib/i18n.ts` | EN ↔ हिं strings + the 13-term `GLOSSARY` mirrored to [`kb/methodology/glossary.json`](../kb/methodology/glossary.json). |
17
 
18
+ ## Voice UX invariants ([ADR-028](../70-docs/60-decisions/ADR-028-voice-ux-single-default-mode.md))
19
 
20
  - **One default voice mode + one fallback.** Live mode is the default; the toolbar pill toggles it (green = on, red = off). The 🎤 push-to-talk button suspends Live for one turn → captures with VAD silence-cutoff → resumes Live if the user preference is still on.
21
  - **Hands-free mode was removed in KI-027.** Any reference to it in code or docs is stale.
 
40
  npm run dev # localhost:3000 — points at the FastAPI on :8000 by default
41
  ```
42
 
43
+ Point at a different backend with `NEXT_PUBLIC_BACKEND_URL=...` (see `src/lib/api.ts`).
44
 
45
  ## Related
46
 
47
  - `AGENTS.md` (this folder) — required reading for AI agents touching this code
48
  - Root `CLAUDE.md` § Voice UX (ADR-028) — the canonical voice-UX cheat-sheet
49
+ - [ADR-005](../70-docs/60-decisions/ADR-005-nextjs-fastapi-frontend.md) · [ADR-013](../70-docs/60-decisions/ADR-013-tailwind-shadcn-ui.md) · [ADR-015](../70-docs/60-decisions/ADR-015-openapi-typescript-codegen.md) · [ADR-021](../70-docs/60-decisions/ADR-021-view-aware-system-prompt.md) · [ADR-028](../70-docs/60-decisions/ADR-028-voice-ux-single-default-mode.md)
50
  - [`backend/main.py`](../backend/main.py) — the API the frontend talks to
frontend/src/lib/i18n.ts CHANGED
@@ -24,7 +24,7 @@ export const UI_STRINGS = {
24
  "welcome.heading_a": "Find a health policy that genuinely fits ",
25
  "welcome.heading_b": "you",
26
  "welcome.heading_c": ".",
27
- "welcome.subtitle": "I'll ask 8–10 short questions, then show you 3 policies that match — with the exact reasons each ranked well.",
28
  "welcome.no_commissions": "No broker commissions in my ranking.",
29
  "welcome.source_link": "Every fact you see has a source link.",
30
  "welcome.trust_title": "Tell me the truth — even on the hard things.",
@@ -35,7 +35,6 @@ export const UI_STRINGS = {
35
  "input.placeholder": "Ask about coverage, waiting periods, exclusions, or compare policies…",
36
  "input.send": "Send",
37
  "input.voice_reply": "Voice reply",
38
- "input.hands_free": "Hands-free",
39
  "input.lang_label": "Lang:",
40
  "input.voice_input": "Voice input",
41
  "input.upload": "Upload your own policy PDF",
@@ -142,7 +141,7 @@ export const UI_STRINGS = {
142
  "welcome.heading_a": "आपके लिए सही स्वास्थ्य पॉलिसी खोजें — ",
143
  "welcome.heading_b": "बस आपके लिए",
144
  "welcome.heading_c": "।",
145
- "welcome.subtitle": "मैं 8–10 छोटे सवाल पूछूंगा, फिर 3 पॉलिसियाँ दिखाऊंगा जो आपके लिए सबसे सही हैं — सटीक कारणों के साथ।",
146
  "welcome.no_commissions": "मेरी रैंकिंग में कोई ब्रोकर कमीशन नहीं।",
147
  "welcome.source_link": "हर तथ्य का source link है।",
148
  "welcome.trust_title": "सच बताइए — मुश्किल बातें भी।",
@@ -152,7 +151,6 @@ export const UI_STRINGS = {
152
  "input.placeholder": "Coverage, waiting period, exclusion, या तुलना के बारे में पूछिए…",
153
  "input.send": "भेजें",
154
  "input.voice_reply": "आवाज़ में जवाब",
155
- "input.hands_free": "हैंड्स-फ्री",
156
  "input.lang_label": "भाषा:",
157
  "input.voice_input": "आवाज़ input",
158
  "input.upload": "अपनी policy PDF upload करें",
 
24
  "welcome.heading_a": "Find a health policy that genuinely fits ",
25
  "welcome.heading_b": "you",
26
  "welcome.heading_c": ".",
27
+ "welcome.subtitle": "I'll ask a few short questions, then show you 3 policies that match — with the exact reasons each ranked well.",
28
  "welcome.no_commissions": "No broker commissions in my ranking.",
29
  "welcome.source_link": "Every fact you see has a source link.",
30
  "welcome.trust_title": "Tell me the truth — even on the hard things.",
 
35
  "input.placeholder": "Ask about coverage, waiting periods, exclusions, or compare policies…",
36
  "input.send": "Send",
37
  "input.voice_reply": "Voice reply",
 
38
  "input.lang_label": "Lang:",
39
  "input.voice_input": "Voice input",
40
  "input.upload": "Upload your own policy PDF",
 
141
  "welcome.heading_a": "आपके लिए सही स्वास्थ्य पॉलिसी खोजें — ",
142
  "welcome.heading_b": "बस आपके लिए",
143
  "welcome.heading_c": "।",
144
+ "welcome.subtitle": "मैं कुछ छोटे सवाल पूछूंगा, फिर 3 पॉलिसियाँ दिखाऊंगा जो आपके लिए सबसे सही हैं — सटीक कारणों के साथ।",
145
  "welcome.no_commissions": "मेरी रैंकिंग में कोई ब्रोकर कमीशन नहीं।",
146
  "welcome.source_link": "हर तथ्य का source link है।",
147
  "welcome.trust_title": "सच बताइए — मुश्किल बातें भी।",
 
151
  "input.placeholder": "Coverage, waiting period, exclusion, या तुलना के बारे में पूछिए…",
152
  "input.send": "भेजें",
153
  "input.voice_reply": "आवाज़ में जवाब",
 
154
  "input.lang_label": "भाषा:",
155
  "input.voice_input": "आवाज़ input",
156
  "input.upload": "अपनी policy PDF upload करें",
kb/eval/INDEX.md CHANGED
@@ -27,6 +27,6 @@ _Auto-generated. Source: `eval/gold_qa.json` + `eval/results.json` + `eval/run.p
27
  ## Methodology
28
 
29
  - Gold Q&A built by 3 pipelines: auto-from-extraction (templated), LLM-drafted (human-verified), hand-crafted adversarial. See `70-docs/03-eval-plan.md`.
30
- - Grader: NIM Llama-4 Maverick (Meta MoE) — different family from the DeepSeek-V4 brain → non-circular (D-019, 2026-05-14). The earlier Groq Llama grader was retired in the same consolidation.
31
  - Re-run: `python -m eval.run [--limit N] [--policy <id>]`.
32
  - CI gate: `.github/workflows/eval.yml` runs eval on every PR; blocks merge if factual_accuracy < 0.65 or citation_accuracy < 0.55.
 
27
  ## Methodology
28
 
29
  - Gold Q&A built by 3 pipelines: auto-from-extraction (templated), LLM-drafted (human-verified), hand-crafted adversarial. See `70-docs/03-eval-plan.md`.
30
+ - Grader: NIM Mistral Large 3 675B (Mistral; primary judge per D-022) — different family from the Qwen 3-Next 80B brain → non-circular (D-019, 2026-05-14). The earlier Groq Llama grader was retired in the same consolidation.
31
  - Re-run: `python -m eval.run [--limit N] [--policy <id>]`.
32
  - CI gate: `.github/workflows/eval.yml` runs eval on every PR; blocks merge if factual_accuracy < 0.65 or citation_accuracy < 0.55.
kb/security/INDEX.md CHANGED
@@ -24,7 +24,7 @@ All gates run for EVERY upload. Block on any failure; the audit trail captures t
24
  | 1 | Retrieval floor | Top-1 cosine < 0.30 OR avg top-5 < 0.22 → refuse outright |
25
  | 2 | Citation integrity | Any `[Source:…]` in the bot's reply must point to a real retrieved chunk's policy_name |
26
  | 3 | Numeric grounding | Every ₹, %, day/month/year in the reply must appear in retrieved chunks (regex) |
27
- | 4 | LLM-judge faithfulness | NIM Llama-4 Maverick (Meta MoE) inspects the reply against retrieved chunks; outputs strict JSON; different family from DeepSeek-V4 brain → non-circular eval (D-019). |
28
  | 5 (Indic) | Hinglish drift LLM-judge | Same idea on the Hinglish back-translation vs the English source |
29
 
30
  Plus **regex anchors + back-translate cosine** as additional drift checks
 
24
  | 1 | Retrieval floor | Top-1 cosine < 0.30 OR avg top-5 < 0.22 → refuse outright |
25
  | 2 | Citation integrity | Any `[Source:…]` in the bot's reply must point to a real retrieved chunk's policy_name |
26
  | 3 | Numeric grounding | Every ₹, %, day/month/year in the reply must appear in retrieved chunks (regex) |
27
+ | 4 | LLM-judge faithfulness | NIM Mistral Large 3 675B (primary judge per D-022) + Llama-4 Maverick (fallback) inspects the reply against retrieved chunks; outputs strict JSON; different family from Qwen 3-Next 80B brain (primary per D-022) + DeepSeek-V4 (fallback) → non-circular eval (D-019). |
28
  | 5 (Indic) | Hinglish drift LLM-judge | Same idea on the Hinglish back-translation vs the English source |
29
 
30
  Plus **regex anchors + back-translate cosine** as additional drift checks
rag/README.md CHANGED
@@ -11,8 +11,8 @@ Lineage for every artefact below is documented in [`kb/AUDIT_TRAIL.md`](../kb/AU
11
  | `download_corpus.py` | 1. SOURCE → 3. DOWNLOAD | `rag/corpus/<insurer>/*.pdf` + `_manifest.json`. HEAD-checks + magic-byte sniff + retry on 403/timeout. |
12
  | `download_retry.py` | 3. DOWNLOAD | Retries the failures from the previous run. |
13
  | `download_regulatory.py` | 3. DOWNLOAD | IRDAI / regulatory PDFs (deferred from v1; see [ADR-017](../70-docs/60-decisions/ADR-017-irdai-corpus-playwright-rescue.md)). |
14
- | `ingest.py` | 4. PARSE → 5. CHUNK → 6. EMBED → 7. INDEX | The big one. `read_pdf_pages` (pdfplumber) → `chunk_pages` (800-tok / 120 overlap, sentence-aware) → BGE embed → `chromadb.PersistentClient.add(...)`. Carries the in-process HNSW bloat tripwire ([ADR-029](../70-docs/60-decisions/ADR-029-disk-storage-hardening.md)). |
15
- | `extract.py` | 8. STRUCTURED EXTRACTION | LLM extraction over each PDF using `schema.py::HealthPolicy` (62 fields). Sarvam-M primary → DeepSeek-V3 fallback. Writes `rag/extracted/<policy_id>.json` + upserts `policies.duckdb`. |
16
  | `build_kb.py` | 9. SCORECARD → KB MIRROR | Runs `backend/scorecard.py` per policy and regenerates the human-readable `kb/policies/<id>.md` tree. |
17
  | `source_map.py` | post-build | Builds `source_map.json` — every chunk → (PDF path, page, span) for citation rendering. |
18
 
@@ -62,5 +62,5 @@ Total cost from cold: < $2 (BGE local + ~80 LLM extractions). Wall-time: ~30-40
62
  - [ADR-011](../70-docs/60-decisions/ADR-011-bge-local-embeddings.md) — why local BGE replaced Voyage
63
  - [ADR-018](../70-docs/60-decisions/ADR-018-chunk-size-sweep-deferred.md) — 800/120 chunk-size baseline
64
  - [ADR-020](../70-docs/60-decisions/ADR-020-code-data-split-hf-dataset.md) — code-vs-data repo split
65
- - [ADR-029](../70-docs/60-decisions/ADR-029-disk-storage-hardening.md) — HNSW bloat tripwire (3-layer defence)
66
  - [`kb/AUDIT_TRAIL.md`](../kb/AUDIT_TRAIL.md) — end-to-end lineage doc
 
11
  | `download_corpus.py` | 1. SOURCE → 3. DOWNLOAD | `rag/corpus/<insurer>/*.pdf` + `_manifest.json`. HEAD-checks + magic-byte sniff + retry on 403/timeout. |
12
  | `download_retry.py` | 3. DOWNLOAD | Retries the failures from the previous run. |
13
  | `download_regulatory.py` | 3. DOWNLOAD | IRDAI / regulatory PDFs (deferred from v1; see [ADR-017](../70-docs/60-decisions/ADR-017-irdai-corpus-playwright-rescue.md)). |
14
+ | `ingest.py` | 4. PARSE → 5. CHUNK → 6. EMBED → 7. INDEX | The big one. `read_pdf_pages` (pdfplumber) → `chunk_pages` (800-tok / 120 overlap, sentence-aware) → BGE embed → `chromadb.PersistentClient.add(...)`. Carries the in-process HNSW bloat tripwire ([ADR-029](../70-docs/60-decisions/ADR-029-hnsw-bloat-tripwire.md)). |
15
+ | `extract.py` | 8. STRUCTURED EXTRACTION | LLM extraction over each PDF using `schema.py::HealthPolicy` (62 fields). `get_brain_llm()` (Qwen 3-Next 80B primary) + `get_fast_brain_llm()` (Nemotron Nano 30B) — both `NimChainLLM` fallback chains. Writes `rag/extracted/<policy_id>.json` + upserts `policies.duckdb`. |
16
  | `build_kb.py` | 9. SCORECARD → KB MIRROR | Runs `backend/scorecard.py` per policy and regenerates the human-readable `kb/policies/<id>.md` tree. |
17
  | `source_map.py` | post-build | Builds `source_map.json` — every chunk → (PDF path, page, span) for citation rendering. |
18
 
 
62
  - [ADR-011](../70-docs/60-decisions/ADR-011-bge-local-embeddings.md) — why local BGE replaced Voyage
63
  - [ADR-018](../70-docs/60-decisions/ADR-018-chunk-size-sweep-deferred.md) — 800/120 chunk-size baseline
64
  - [ADR-020](../70-docs/60-decisions/ADR-020-code-data-split-hf-dataset.md) — code-vs-data repo split
65
+ - [ADR-029](../70-docs/60-decisions/ADR-029-hnsw-bloat-tripwire.md) — HNSW bloat tripwire (3-layer defence)
66
  - [`kb/AUDIT_TRAIL.md`](../kb/AUDIT_TRAIL.md) — end-to-end lineage doc
rag/extract.py CHANGED
@@ -338,10 +338,10 @@ async def main():
338
  # different MoE routing path so a Pro timeout doesn't always reproduce
339
  # in Flash. Sarvam-M was the old fallback but its <think> tags ate the
340
  # 2048 output cap before reaching JSON, causing ~20 truncation failures.
341
- primary = get_brain_llm() # DeepSeek-V4-Pro
342
- fallback = get_fast_brain_llm() # DeepSeek-V4-Flash
343
 
344
- print(f"Extracting {len(pdfs)} policies. Primary=NIM V4-Pro, Fallback=NIM V4-Flash.\n")
345
  t0 = time.time()
346
  ok = 0
347
  for i, pdf in enumerate(pdfs, 1):
 
338
  # different MoE routing path so a Pro timeout doesn't always reproduce
339
  # in Flash. Sarvam-M was the old fallback but its <think> tags ate the
340
  # 2048 output cap before reaching JSON, causing ~20 truncation failures.
341
+ primary = get_brain_llm() # BRAIN_CHAIN — Qwen 3-Next 80B primary (D-022)
342
+ fallback = get_fast_brain_llm() # FAST_BRAIN_CHAIN — Nemotron Nano 30B primary (D-022)
343
 
344
+ print(f"Extracting {len(pdfs)} policies. Primary=BRAIN_CHAIN (Qwen 80B), Fallback=FAST_BRAIN_CHAIN (Nemotron 30B).\n")
345
  t0 = time.time()
346
  ok = 0
347
  for i, pdf in enumerate(pdfs, 1):
tools/README.md CHANGED
@@ -2,7 +2,7 @@
2
 
3
  Loose collection of CLI scripts: corpus operations, data uploads, probes, KB regeneration, scheduled-job runners. Nothing under `tools/` is imported by the live server — `backend/` and `rag/` are the runtime surface.
4
 
5
- Scheduling for the long-running ones is wired via macOS LaunchAgents — see `CRON_README.md` in this folder for cadence + script paths, and [ADR-029](../70-docs/60-decisions/ADR-029-disk-storage-hardening.md) for the disk-safety LaunchAgents.
6
 
7
  ## Corpus + extraction batch ops
8
 
@@ -76,5 +76,5 @@ Scheduling for the long-running ones is wired via macOS LaunchAgents — see `CR
76
  ## Related
77
 
78
  - `CRON_README.md` (this folder) — LaunchAgent cadence reference
79
- - [ADR-020](../70-docs/60-decisions/ADR-020-code-data-split-hf-dataset.md), [ADR-024](../70-docs/60-decisions/ADR-024-triple-mirror-code-and-data.md), [ADR-029](../70-docs/60-decisions/ADR-029-disk-storage-hardening.md)
80
  - `80-audit/ENTERPRISE_AUDIT.md` — defect register, including silent-LaunchAgent regressions (D-002)
 
2
 
3
  Loose collection of CLI scripts: corpus operations, data uploads, probes, KB regeneration, scheduled-job runners. Nothing under `tools/` is imported by the live server — `backend/` and `rag/` are the runtime surface.
4
 
5
+ Scheduling for the long-running ones is wired via macOS LaunchAgents — see `CRON_README.md` in this folder for cadence + script paths, and [ADR-029](../70-docs/60-decisions/ADR-029-hnsw-bloat-tripwire.md) for the disk-safety LaunchAgents.
6
 
7
  ## Corpus + extraction batch ops
8
 
 
76
  ## Related
77
 
78
  - `CRON_README.md` (this folder) — LaunchAgent cadence reference
79
+ - [ADR-020](../70-docs/60-decisions/ADR-020-code-data-split-hf-dataset.md), [ADR-024](../70-docs/60-decisions/ADR-024-triple-mirror-code-and-data.md), [ADR-029](../70-docs/60-decisions/ADR-029-hnsw-bloat-tripwire.md)
80
  - `80-audit/ENTERPRISE_AUDIT.md` — defect register, including silent-LaunchAgent regressions (D-002)