Spaces:
Sleeping
docs: KI-071 — reconcile docs with code reality (Qwen 80B / Nemotron 30B / Mistral 675B as current primaries)
Browse filesSource-of-truth audit found the production code on `main` runs Qwen
3-Next 80B (brain primary) + Nemotron Nano 30B (fast brain primary) +
Mistral Large 3 675B (judge primary) per the D-022 inline brain swap on
2026-05-14 (NIM DeepSeek + Llama-4 Maverick pools were timing out for
hours). The ADR-019 decision text + multiple docs still referenced
DeepSeek V4-Pro / V4-Flash + Llama-4 Maverick as primaries.
This commit aligns 12 files with the actual chain definitions in
`backend/providers/nvidia_nim_llm.py`. DeepSeek V4-Pro / V4-Flash +
Llama-4 Maverick are correctly documented now as **fallback chain
entries** that re-enter rotation when NIM's pools recover.
Files reconciled (+64 / -45):
- `README.md` — §1.2 "Honest model picks" paragraph + §4.3 Model stack
table rewritten. Heavy brain row now: "Qwen 3-Next 80B *or* Groq
Llama-3.3-70B (50/50 rotation)". Fast brain: "Nemotron Nano 30B".
Judge: "Mistral Large 3 675B". Full fallback chains documented.
- `70-docs/60-decisions/ADR-019` — partial-supersession note added
under Status: documents the D-022 brain swap, points to current code
for source of truth.
- `kb/security/INDEX.md` + `kb/eval/INDEX.md` — Gate-4 / grader rows
now name Mistral 675B primary + Qwen brain (was DeepSeek-V4 + Llama-4).
- `frontend/README.md` — 3× wrong ADR-028 filename fixed; env var
`NEXT_PUBLIC_API_BASE` → `NEXT_PUBLIC_BACKEND_URL` (matches Dockerfile).
- `rag/README.md` — extractor line now cites `get_brain_llm()` /
`get_fast_brain_llm()` instead of "Sarvam-M + DeepSeek-V3 fallback"
(deeply stale); 2× ADR-029 filename fixed.
- `eval/README.md` — judge label corrected to Mistral 675B primary
(Groq is 6th fallback in JUDGE_CHAIN, not primary as it was labelled).
- `frontend/src/lib/i18n.ts` — dead `input.hands_free` keys (EN + HI)
removed (KI-027 retired Hands-free mode). `welcome.subtitle` (EN +
HI) softened from "8-10 short questions" to "a few short questions"
since post-KI-070 turn count is variable.
- `tools/README.md` — 2× ADR-029 broken link fixed.
- `40-data/profiles/README.md` — 1× ADR-029 broken link fixed.
- `backend/providers/__init__.py` — module docstring rewritten:
chain primaries listed, D-022 swap acknowledged, openrouter + groq
documented as cross-provider fallbacks (they exist on disk, were
incorrectly described as "retired" in the old docstring).
- `rag/extract.py:341-342` — inline primary/fallback comments fixed.
Verified no `DeepSeek V4-Pro` claim remains as PRIMARY in any doc;
DeepSeek mentions appear only in fallback chain context.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- 40-data/profiles/README.md +1 -1
- 70-docs/60-decisions/ADR-019-nim-single-provider-consolidation.md +2 -0
- README.md +26 -17
- backend/providers/__init__.py +17 -7
- eval/README.md +2 -2
- frontend/README.md +4 -4
- frontend/src/lib/i18n.ts +2 -4
- kb/eval/INDEX.md +1 -1
- kb/security/INDEX.md +1 -1
- rag/README.md +3 -3
- rag/extract.py +3 -3
- tools/README.md +2 -2
|
@@ -49,7 +49,7 @@ The original design considered embedding-only. The "why JSON" trade-offs:
|
|
| 49 |
- **Deterministic name lookup.** `Rohit` → `rohit.json` is exact; vector search is approximate and can collide on common first names.
|
| 50 |
- **Human-readable.** A BFSI auditor can `cat` the file and see the full profile.
|
| 51 |
- **Manually editable.** Quick repair without a re-embed pipeline.
|
| 52 |
-
- **No HNSW bloat exposure.** Profile updates do not touch the policy vector store ([ADR-029](../70-docs/60-decisions/ADR-029-
|
| 53 |
|
| 54 |
The Chroma chunk is purely a retrieval-time view of the canonical JSON.
|
| 55 |
|
|
|
|
| 49 |
- **Deterministic name lookup.** `Rohit` → `rohit.json` is exact; vector search is approximate and can collide on common first names.
|
| 50 |
- **Human-readable.** A BFSI auditor can `cat` the file and see the full profile.
|
| 51 |
- **Manually editable.** Quick repair without a re-embed pipeline.
|
| 52 |
+
- **No HNSW bloat exposure.** Profile updates do not touch the policy vector store ([ADR-029](../70-docs/60-decisions/ADR-029-hnsw-bloat-tripwire.md)).
|
| 53 |
|
| 54 |
The Chroma chunk is purely a retrieval-time view of the canonical JSON.
|
| 55 |
|
|
@@ -3,6 +3,8 @@
|
|
| 3 |
**Status:** Locked
|
| 4 |
**Date:** 2026-05-14
|
| 5 |
|
|
|
|
|
|
|
| 6 |
## Context
|
| 7 |
|
| 8 |
By mid-May 2026 the LLM stack had accreted **four third-party providers** in overlapping roles, each with its own free-tier ceiling that surfaced as quality problems:
|
|
|
|
| 3 |
**Status:** Locked
|
| 4 |
**Date:** 2026-05-14
|
| 5 |
|
| 6 |
+
**Partial supersession (D-022, 2026-05-14):** NIM's DeepSeek V4-Pro / V4-Flash and Llama-4 Maverick inference pools degraded with hours-long timeouts during a production incident. The chains were re-ordered to put **Qwen 3-Next 80B** (brain primary), **Nemotron Nano 30B** (fast brain primary), and **Mistral Large 3 675B** (judge primary) at the head, with DeepSeek + Maverick retained as fallback chain entries that re-enter rotation when those pools recover. The single-provider consolidation decision in this ADR still stands (NIM remains the primary inference provider); only the *specific model picks* were swapped. See `backend/providers/nvidia_nim_llm.py` for the current chain definitions.
|
| 7 |
+
|
| 8 |
## Context
|
| 9 |
|
| 10 |
By mid-May 2026 the LLM stack had accreted **four third-party providers** in overlapping roles, each with its own free-tier ceiling that surfaced as quality problems:
|
|
@@ -71,7 +71,15 @@ A take-home is a sample of how the engineer thinks under constraint. Three thing
|
|
| 71 |
|
| 72 |
2. **Hallucination defense and refusal as product features.** BFSI deployments get fined for mis-selling; the bot is biased toward refusal over confident wrong answers. The 4 faithfulness gates + cross-check retry + 3 Indic drift checks + audit log are the BFSI-compliance-grade version of "we shipped a chatbot." When the eval shows a headline accuracy below 100% because the gates are aggressive, the right response is to soften the gates carefully — not to ship a higher number by relaxing the verifier.
|
| 73 |
|
| 74 |
-
3. **Honest model picks — Sarvam where Sarvam is uniquely strong, open-weights frontier for reasoning.** Sarvam Saarika v2.5
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 75 |
|
| 76 |
The rest is craftsmanship. The 8-section KB ([`kb/`](kb/)) is regeneratable from primary sources in <40 minutes for <$2 cold. Every numeric value in every reviewer-facing artifact traces to a source PDF + page + clause. Every architectural decision is in [`70-docs/60-decisions/`](70-docs/60-decisions/) with alternatives and revisit-at-scale notes. Every production-readiness defect is in [`80-audit/ENTERPRISE_AUDIT.md`](80-audit/ENTERPRISE_AUDIT.md). The repo is structured so a new engineer joining on Monday could ship v1.1 by Friday.
|
| 77 |
|
|
@@ -540,23 +548,24 @@ The bot is two flows running together — the customer's experience and the tech
|
|
| 540 |
|
| 541 |
### 4.3 Model stack
|
| 542 |
|
| 543 |
-
Every LLM role is served by a **fallback chain** of candidate models (`backend/providers/nvidia_nim_llm.py::NimChainLLM`),
|
| 544 |
|
| 545 |
-
| Role | Primary | Fallback chain | Provider
|
| 546 |
-
|---|---|---|---|
|
| 547 |
-
|
|
| 548 |
-
|
|
| 549 |
-
|
|
| 550 |
-
| **
|
| 551 |
-
| **
|
| 552 |
-
| **
|
| 553 |
-
|
|
| 554 |
-
|
|
| 555 |
-
|
|
| 556 |
-
|
| 557 |
-
|
| 558 |
-
|
| 559 |
-
|
|
|
|
| 560 |
|
| 561 |
### 4.4 Major design decisions (the short list)
|
| 562 |
|
|
|
|
| 71 |
|
| 72 |
2. **Hallucination defense and refusal as product features.** BFSI deployments get fined for mis-selling; the bot is biased toward refusal over confident wrong answers. The 4 faithfulness gates + cross-check retry + 3 Indic drift checks + audit log are the BFSI-compliance-grade version of "we shipped a chatbot." When the eval shows a headline accuracy below 100% because the gates are aggressive, the right response is to soften the gates carefully — not to ship a higher number by relaxing the verifier.
|
| 73 |
|
| 74 |
+
3. **Honest model picks — Sarvam where Sarvam is uniquely strong, open-weights frontier for reasoning.** Voice and Indic are non-substitutable: **Sarvam Saarika v2.5** for speech-to-text, **Sarvam Bulbul v2** (speaker `anushka`) for text-to-speech, and **Sarvam-M** for Hindi/Hinglish/vernacular translation — no closed-source frontier matches Sarvam on Indian accents or code-mixed Hinglish. Reasoning is a different problem and runs on open-weights frontier models behind a fallback chain, not a single hardcoded brain. The chain architecture has three roles, each a `NimChainLLM` (`backend/providers/nvidia_nim_llm.py`) with a NIM-primary preference order plus OpenRouter and Groq cross-provider fallbacks:
|
| 75 |
+
|
| 76 |
+
- **`BRAIN_CHAIN`** (comparison, recommendation, synthesis) — primary **Qwen 3-Next 80B** (`qwen/qwen3-next-80b-a3b-instruct`, 80B / 3B-active MoE, multilingual, ~2s TTFT on NIM free tier), with **Qwen 3.5 122B**, **OpenAI GPT-OSS 120B**, **Mistral Large 3 675B**, **NVIDIA Nemotron-Super 49B**, **Meta Llama-3.3 70B**, and **DeepSeek V4-Pro** as in-NIM fallbacks, then **OpenRouter GPT-OSS 120B** and **Groq Llama-3.3-70B** as cross-provider fallbacks for full-NIM-outage survival.
|
| 77 |
+
- **`FAST_BRAIN_CHAIN`** (fact-find turns, QA, paraphrase, normalize, extract — every latency-sensitive role) — primary **NVIDIA Nemotron Nano 30B** (`nvidia/nemotron-3-nano-30b-a3b`, ~1.6s TTFT — bottleneck is TTFT not capability on these jobs), with Qwen 3-Next 80B, GPT-OSS 120B, Qwen 3.5 122B, and **DeepSeek V4-Flash** as NIM fallbacks, then Groq Llama-3.3-70B as the cross-provider fallback.
|
| 78 |
+
- **`JUDGE_CHAIN`** (faithfulness Gate 4, Hinglish drift LLM-judge, eval grader) — primary **Mistral Large 3 675B** (`mistralai/mistral-large-3-675b-instruct-2512`, MIT). Mistral is a deliberately different model family from the Qwen brain so the judge does not mark its own homework. Fallbacks: GPT-OSS 120B, Moonshot Kimi K2, MiniMax M2.5, and **Meta Llama-4 Maverick 17B/128E** (the original D-019 judge — kept in the chain for when NIM's Llama pool recovers), then OpenRouter and Groq cross-provider entries.
|
| 79 |
+
|
| 80 |
+
**Provider load-balancing (KI-025, [ADR-026](70-docs/60-decisions/ADR-026-provider-load-balancing.md)).** The brain and fast-brain chains' *primary* slot rotates 50/50 per call between the NIM Qwen entry and the Groq Llama-3.3-70B entry via `_balanced_brain_chain` (per-call `random.random()`, no shared cycle state). NIM's free-tier rate cap is 40 req/min shared across every model on a key; splitting brain load across two independent free-tier quotas (NIM + Groq) effectively doubles sustained brain throughput. Groq's LPU TTFT (~1s) is often *lower* than NIM Qwen's, so the rotation is a strict latency win on top of the throughput win. A Groq-primary call that fails still gets the full NIM fallback ladder underneath, so reliability is unchanged.
|
| 81 |
+
|
| 82 |
+
The result: a Sarvam customer deploying this stack gets a product that *uses Sarvam exactly where Sarvam beats the world* and uses MIT-licensed open-weights frontier models for everything else — $0 inference, two independent free-tier providers, single-key-per-provider for the entire non-voice stack.
|
| 83 |
|
| 84 |
The rest is craftsmanship. The 8-section KB ([`kb/`](kb/)) is regeneratable from primary sources in <40 minutes for <$2 cold. Every numeric value in every reviewer-facing artifact traces to a source PDF + page + clause. Every architectural decision is in [`70-docs/60-decisions/`](70-docs/60-decisions/) with alternatives and revisit-at-scale notes. Every production-readiness defect is in [`80-audit/ENTERPRISE_AUDIT.md`](80-audit/ENTERPRISE_AUDIT.md). The repo is structured so a new engineer joining on Monday could ship v1.1 by Friday.
|
| 85 |
|
|
|
|
| 548 |
|
| 549 |
### 4.3 Model stack
|
| 550 |
|
| 551 |
+
Every LLM role is served by a **fallback chain** of candidate models (`backend/providers/nvidia_nim_llm.py::NimChainLLM`), never a single hardcoded model. Chains were curated to preserve brain ↔ judge family diversity (Qwen brain ↔ Mistral judge) so any failover still produces non-circular grading. Cross-provider entries (OpenRouter, Groq) sit at the bottom of every chain so the whole reasoning stack survives a full NIM regional outage.
|
| 552 |
|
| 553 |
+
| Role | Primary | Fallback chain (in order) | Provider(s) | Why this primary |
|
| 554 |
+
|---|---|---|---|---|
|
| 555 |
+
| **Heavy brain** (comparison, recommendation, synthesis) | Qwen 3-Next 80B *or* Groq Llama-3.3-70B (50/50 per-call rotation, KI-025) | Qwen 3.5 122B → GPT-OSS 120B → Mistral Large 3 675B → Nemotron-Super 49B → Llama-3.3-70B → DeepSeek V4-Pro → OpenRouter GPT-OSS 120B → Groq Llama-3.3-70B | NIM + Groq + OpenRouter | Qwen 80B is the fastest reliable frontier-tier MoE on NIM free tier (~2s TTFT, clean JSON, multilingual). Groq LPU is the lowest-TTFT free-tier option; 50/50 split doubles sustained throughput across two independent rate caps ([ADR-026](70-docs/60-decisions/ADR-026-provider-load-balancing.md)). |
|
| 556 |
+
| **Fast brain** (voice turns, fact-find, QA, paraphrase, normalize, extract) | Nemotron Nano 30B *or* Groq Llama-3.3-70B (50/50 per-call rotation) | Qwen 3-Next 80B → GPT-OSS 120B → Qwen 3.5 122B → DeepSeek V4-Flash → Groq Llama-3.3-70B | NIM + Groq | Bottleneck on these short jobs is TTFT, not capability. Nemotron Nano hits ~1.6s; Qwen 80B is ~2-3s. Reordered for latency in KI-035. |
|
| 557 |
+
| **Judge** (faithfulness Gate 4, Hinglish drift, eval grader) | Mistral Large 3 675B | GPT-OSS 120B → Kimi K2 → MiniMax M2.5 → Llama-4 Maverick 17B/128E → OpenRouter GPT-OSS 120B → Groq Llama-3.3-70B | NIM + OpenRouter + Groq | Different family from the Qwen brain (Mistral, not Qwen / DeepSeek / Llama family) so the judge sees the brain's output from a genuinely different decision surface. 675B dense, MIT, ~4.3s on NIM. |
|
| 558 |
+
| **Profile extractor** (free-form profile updates → 9-slot schema, ADR-022) | Fast brain chain (Nemotron Nano 30B primary) | inherits FAST_BRAIN_CHAIN | NIM + Groq | Short prompt, structured JSON out — Nemotron Nano is fast and reliable enough; same chain as fact-find. |
|
| 559 |
+
| **Fact-find normalizer** (user answer → typed slot value) | Fast brain chain (Nemotron Nano 30B primary) | inherits FAST_BRAIN_CHAIN | NIM + Groq | Same shape as profile extractor — narrow input, narrow JSON output. |
|
| 560 |
+
| **Question paraphraser** *(deprecated by KI-070)* | Fast brain chain (Nemotron Nano 30B primary), 4s per-link / 6s total budget | inherits FAST_BRAIN_CHAIN | NIM + Groq | ADR-027 layered an LLM paraphraser on top of the hardcoded GRAPH to make fact-find sound less robotic. **KI-070 replaces the paraphraser + opener-rotation + acknowledger machinery with a single LLM brain call per fact-find turn (ADR-030).** The paraphraser module is scheduled for deletion in the KI-070 sync sweep. |
|
| 561 |
+
| **Indic translation** (English ↔ Hindi / Hinglish / vernacular) | Sarvam-M | — (no fallback; Indic is non-substitutable) | Sarvam | Best-in-class on Hinglish and code-mixed Indian languages; no open-weights frontier model is competitive here. |
|
| 562 |
+
| **STT** (speech-to-text) | Sarvam Saarika v2.5 (`saarika:v2.5`) | — | Sarvam | Best-in-class on Indian-accented English and Hinglish; WebM transcoded to 16kHz mono WAV at the gateway (`backend/providers/sarvam_stt.py`). |
|
| 563 |
+
| **TTS** (text-to-speech) | Sarvam Bulbul v2 (`bulbul:v2`, speaker `anushka`) | — | Sarvam | Best-in-class on Hinglish prosody; 22.05kHz output, base64 WAV from Sarvam endpoint. |
|
| 564 |
+
| **Embeddings** | Voyage `voyage-3` (legacy ingest only; hot path uses BGE) | BGE-small-en-v1.5 (384d, local CPU) | Voyage (offline) + Local (hot path) | BGE on-CPU avoids network on every retrieval ([ADR-011](70-docs/60-decisions/ADR-011-bge-local-embeddings.md)); Voyage retained for back-compat with already-extracted artifacts. |
|
| 565 |
+
|
| 566 |
+
**Chain budgets.** Brain `20s` per-link / `35s` total; fast brain `12s` / `22s`; judge `30s` / `75s`. The per-link timeout is dynamically clipped to the remaining chain budget so a single fallback can never blow past the ceiling (KI-021).
|
| 567 |
+
|
| 568 |
+
**Health-aware filtering.** A background probe loop (`backend/llm_health.py`) marks individual NIM models as up/down. `NimChainLLM.filter_chain()` skips known-down models before the first call; if every candidate is filtered, the full chain is tried anyway since the probe may be stale. On total chain exhaustion, one synchronous probe refresh is triggered before raising.
|
| 569 |
|
| 570 |
### 4.4 Major design decisions (the short list)
|
| 571 |
|
|
@@ -1,16 +1,26 @@
|
|
| 1 |
"""Provider clients — thin, async, behind a common interface.
|
| 2 |
|
| 3 |
-
After the 2026-05-14 Stack A consolidation (D-019
|
|
|
|
| 4 |
|
| 5 |
sarvam_stt.py — Sarvam Saarika v2.5 (speech-to-text)
|
| 6 |
-
sarvam_tts.py — Sarvam Bulbul v2 (text-to-speech)
|
| 7 |
-
sarvam_llm.py — Sarvam-M (Indic translation IN and OUT; not brain
|
| 8 |
local_embeddings.py — BGE-small-en-v1.5 (local CPU embeddings)
|
| 9 |
-
nvidia_nim_llm.py —
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 10 |
|
| 11 |
-
|
| 12 |
-
|
| 13 |
-
four. See 70-docs/decisions.md D-019.
|
| 14 |
|
| 15 |
All clients are async (use httpx.AsyncClient) so the FastAPI handlers can
|
| 16 |
parallelize provider calls without blocking the event loop.
|
|
|
|
| 1 |
"""Provider clients — thin, async, behind a common interface.
|
| 2 |
|
| 3 |
+
After the 2026-05-14 Stack A consolidation (D-019, partially superseded by
|
| 4 |
+
D-022 on 2026-05-14 — NIM brain pool swap), the provider stack is:
|
| 5 |
|
| 6 |
sarvam_stt.py — Sarvam Saarika v2.5 (speech-to-text)
|
| 7 |
+
sarvam_tts.py — Sarvam Bulbul v2 (text-to-speech, speaker anushka)
|
| 8 |
+
sarvam_llm.py — Sarvam-M (Indic translation IN and OUT; not brain)
|
| 9 |
local_embeddings.py — BGE-small-en-v1.5 (local CPU embeddings)
|
| 10 |
+
nvidia_nim_llm.py — Chain pattern via NimChainLLM. Current primaries (D-022):
|
| 11 |
+
BRAIN_CHAIN[0] = Qwen 3-Next 80B
|
| 12 |
+
FAST_BRAIN_CHAIN[0] = Nemotron Nano 30B
|
| 13 |
+
JUDGE_CHAIN[0] = Mistral Large 3 675B
|
| 14 |
+
DeepSeek V4-Pro / V4-Flash + Llama-4 Maverick (the
|
| 15 |
+
original D-019 picks) are retained as fallback chain
|
| 16 |
+
entries that re-enter rotation when NIM's pools
|
| 17 |
+
recover from the 2026-05-14 outage.
|
| 18 |
+
openrouter_llm.py — OpenRouter cross-provider fallback (bottom of every chain).
|
| 19 |
+
groq_llm.py — Groq LPU; brain primary rotates 50/50 NIM ↔ Groq
|
| 20 |
+
per call (KI-025 / ADR-026).
|
| 21 |
|
| 22 |
+
See 70-docs/60-decisions/ADR-019, ADR-026, ADR-030 and 80-audit/ENTERPRISE_AUDIT.md
|
| 23 |
+
(D-022 row) for the full provenance.
|
|
|
|
| 24 |
|
| 25 |
All clients are async (use httpx.AsyncClient) so the FastAPI handlers can
|
| 26 |
parallelize provider calls without blocking the event loop.
|
|
@@ -10,7 +10,7 @@ Numerical accuracy + grounding eval. Walks a fixed list of curated questions thr
|
|
| 10 |
| --- | --- |
|
| 11 |
| `generate_gold.py` | Builds `gold_qa.json` from `40-data/policy_facts/`: for each curated field with a verbatim quote, emits a natural-language question + expected answer + expected citation. |
|
| 12 |
| `gold_qa.json` | 96-Q gold set. Each entry: `{policy_id, question, expected_answer, expected_regex, source_quote, source_pdf}`. |
|
| 13 |
-
| `run.py` | Runner. For each pair: calls `backend.orchestrator.handle_turn` with `policy_filter_ids=[pair.policy_id]` so retrieval is scoped to the policy under test. Grades each reply twice — regex hard-facts +
|
| 14 |
| `results.json` | Machine-readable last-run results. |
|
| 15 |
| `results.md` | Human-readable last-run report — per-question pass/fail, per-category rollup, hallucination breakdown. |
|
| 16 |
| `info_source_map.json` | Generated by `tools/info_source_map.py`. Claim → URL → verdict (✅ 798 / ⚠️ 321 / ❌ 0 / ⏳ 1385 as of 2026-05-14). The canonical source-grounding KPI. |
|
|
@@ -39,7 +39,7 @@ python -m eval.generate_gold
|
|
| 39 |
| Layer | What it checks | Output field |
|
| 40 |
| --- | --- | --- |
|
| 41 |
| Regex hard-facts | Expected numeric / boolean / waiting-period value appears verbatim in `reply_text`. | `regex_pass` |
|
| 42 |
-
| LLM judge (Groq Llama-3.3-70B) | Reply *semantically* answers the question and cites the expected source. Different family from the NIM Qwen brain → non-circular. | `judge_pass` |
|
| 43 |
| Faithfulness gate | The 4-gate guard in `backend/faithfulness.py` already ran during `handle_turn`; eval records whether it blocked. | `blocked` |
|
| 44 |
|
| 45 |
## Acceptance bar
|
|
|
|
| 10 |
| --- | --- |
|
| 11 |
| `generate_gold.py` | Builds `gold_qa.json` from `40-data/policy_facts/`: for each curated field with a verbatim quote, emits a natural-language question + expected answer + expected citation. |
|
| 12 |
| `gold_qa.json` | 96-Q gold set. Each entry: `{policy_id, question, expected_answer, expected_regex, source_quote, source_pdf}`. |
|
| 13 |
+
| `run.py` | Runner. For each pair: calls `backend.orchestrator.handle_turn` with `policy_filter_ids=[pair.policy_id]` so retrieval is scoped to the policy under test. Grades each reply twice — regex hard-facts + NIM Mistral Large 3 judge chain (different family from the NIM Qwen brain → non-circular). Falls through to Groq Llama only at the cross-provider tail of `JUDGE_CHAIN`. |
|
| 14 |
| `results.json` | Machine-readable last-run results. |
|
| 15 |
| `results.md` | Human-readable last-run report — per-question pass/fail, per-category rollup, hallucination breakdown. |
|
| 16 |
| `info_source_map.json` | Generated by `tools/info_source_map.py`. Claim → URL → verdict (✅ 798 / ⚠️ 321 / ❌ 0 / ⏳ 1385 as of 2026-05-14). The canonical source-grounding KPI. |
|
|
|
|
| 39 |
| Layer | What it checks | Output field |
|
| 40 |
| --- | --- | --- |
|
| 41 |
| Regex hard-facts | Expected numeric / boolean / waiting-period value appears verbatim in `reply_text`. | `regex_pass` |
|
| 42 |
+
| LLM judge (NIM Mistral Large 3 675B primary; OpenAI GPT-OSS 120B / Moonshot Kimi K2 / Llama-4 Maverick / Groq Llama-3.3-70B fallbacks) | Reply *semantically* answers the question and cites the expected source. Different family from the NIM Qwen brain → non-circular. | `judge_pass` |
|
| 43 |
| Faithfulness gate | The 4-gate guard in `backend/faithfulness.py` already ran during `handle_turn`; eval records whether it blocked. | `blocked` |
|
| 44 |
|
| 45 |
## Acceptance bar
|
|
@@ -12,10 +12,10 @@ For Claude / agent-specific rules (e.g. the "this is NOT the Next.js you know" n
|
|
| 12 |
| `src/app/layout.tsx` | Root layout + font wiring (Geist via `next/font`). |
|
| 13 |
| `src/app/globals.css` | Tailwind v4 entrypoint + the shadcn/ui token layer ([ADR-013](../70-docs/60-decisions/ADR-013-tailwind-shadcn-ui.md)). |
|
| 14 |
| `src/lib/api.ts` | Typed API client. Generated types via `openapi-typescript` from the FastAPI OpenAPI schema ([ADR-015](../70-docs/60-decisions/ADR-015-openapi-typescript-codegen.md)). |
|
| 15 |
-
| `src/lib/useLiveConversation.ts` | The continuously-open-mic VAD hook that powers Live mode + barge-in. State persists in `localStorage.insurance_live_pref` ([ADR-028](../70-docs/60-decisions/ADR-028-voice-ux-
|
| 16 |
| `src/lib/i18n.ts` | EN ↔ हिं strings + the 13-term `GLOSSARY` mirrored to [`kb/methodology/glossary.json`](../kb/methodology/glossary.json). |
|
| 17 |
|
| 18 |
-
## Voice UX invariants ([ADR-028](../70-docs/60-decisions/ADR-028-voice-ux-
|
| 19 |
|
| 20 |
- **One default voice mode + one fallback.** Live mode is the default; the toolbar pill toggles it (green = on, red = off). The 🎤 push-to-talk button suspends Live for one turn → captures with VAD silence-cutoff → resumes Live if the user preference is still on.
|
| 21 |
- **Hands-free mode was removed in KI-027.** Any reference to it in code or docs is stale.
|
|
@@ -40,11 +40,11 @@ npm install # or pnpm / bun
|
|
| 40 |
npm run dev # localhost:3000 — points at the FastAPI on :8000 by default
|
| 41 |
```
|
| 42 |
|
| 43 |
-
Point at a different backend with `
|
| 44 |
|
| 45 |
## Related
|
| 46 |
|
| 47 |
- `AGENTS.md` (this folder) — required reading for AI agents touching this code
|
| 48 |
- Root `CLAUDE.md` § Voice UX (ADR-028) — the canonical voice-UX cheat-sheet
|
| 49 |
-
- [ADR-005](../70-docs/60-decisions/ADR-005-nextjs-fastapi-frontend.md) · [ADR-013](../70-docs/60-decisions/ADR-013-tailwind-shadcn-ui.md) · [ADR-015](../70-docs/60-decisions/ADR-015-openapi-typescript-codegen.md) · [ADR-021](../70-docs/60-decisions/ADR-021-view-aware-system-prompt.md) · [ADR-028](../70-docs/60-decisions/ADR-028-voice-ux-
|
| 50 |
- [`backend/main.py`](../backend/main.py) — the API the frontend talks to
|
|
|
|
| 12 |
| `src/app/layout.tsx` | Root layout + font wiring (Geist via `next/font`). |
|
| 13 |
| `src/app/globals.css` | Tailwind v4 entrypoint + the shadcn/ui token layer ([ADR-013](../70-docs/60-decisions/ADR-013-tailwind-shadcn-ui.md)). |
|
| 14 |
| `src/lib/api.ts` | Typed API client. Generated types via `openapi-typescript` from the FastAPI OpenAPI schema ([ADR-015](../70-docs/60-decisions/ADR-015-openapi-typescript-codegen.md)). |
|
| 15 |
+
| `src/lib/useLiveConversation.ts` | The continuously-open-mic VAD hook that powers Live mode + barge-in. State persists in `localStorage.insurance_live_pref` ([ADR-028](../70-docs/60-decisions/ADR-028-voice-ux-single-default-mode.md)). |
|
| 16 |
| `src/lib/i18n.ts` | EN ↔ हिं strings + the 13-term `GLOSSARY` mirrored to [`kb/methodology/glossary.json`](../kb/methodology/glossary.json). |
|
| 17 |
|
| 18 |
+
## Voice UX invariants ([ADR-028](../70-docs/60-decisions/ADR-028-voice-ux-single-default-mode.md))
|
| 19 |
|
| 20 |
- **One default voice mode + one fallback.** Live mode is the default; the toolbar pill toggles it (green = on, red = off). The 🎤 push-to-talk button suspends Live for one turn → captures with VAD silence-cutoff → resumes Live if the user preference is still on.
|
| 21 |
- **Hands-free mode was removed in KI-027.** Any reference to it in code or docs is stale.
|
|
|
|
| 40 |
npm run dev # localhost:3000 — points at the FastAPI on :8000 by default
|
| 41 |
```
|
| 42 |
|
| 43 |
+
Point at a different backend with `NEXT_PUBLIC_BACKEND_URL=...` (see `src/lib/api.ts`).
|
| 44 |
|
| 45 |
## Related
|
| 46 |
|
| 47 |
- `AGENTS.md` (this folder) — required reading for AI agents touching this code
|
| 48 |
- Root `CLAUDE.md` § Voice UX (ADR-028) — the canonical voice-UX cheat-sheet
|
| 49 |
+
- [ADR-005](../70-docs/60-decisions/ADR-005-nextjs-fastapi-frontend.md) · [ADR-013](../70-docs/60-decisions/ADR-013-tailwind-shadcn-ui.md) · [ADR-015](../70-docs/60-decisions/ADR-015-openapi-typescript-codegen.md) · [ADR-021](../70-docs/60-decisions/ADR-021-view-aware-system-prompt.md) · [ADR-028](../70-docs/60-decisions/ADR-028-voice-ux-single-default-mode.md)
|
| 50 |
- [`backend/main.py`](../backend/main.py) — the API the frontend talks to
|
|
@@ -24,7 +24,7 @@ export const UI_STRINGS = {
|
|
| 24 |
"welcome.heading_a": "Find a health policy that genuinely fits ",
|
| 25 |
"welcome.heading_b": "you",
|
| 26 |
"welcome.heading_c": ".",
|
| 27 |
-
"welcome.subtitle": "I'll ask
|
| 28 |
"welcome.no_commissions": "No broker commissions in my ranking.",
|
| 29 |
"welcome.source_link": "Every fact you see has a source link.",
|
| 30 |
"welcome.trust_title": "Tell me the truth — even on the hard things.",
|
|
@@ -35,7 +35,6 @@ export const UI_STRINGS = {
|
|
| 35 |
"input.placeholder": "Ask about coverage, waiting periods, exclusions, or compare policies…",
|
| 36 |
"input.send": "Send",
|
| 37 |
"input.voice_reply": "Voice reply",
|
| 38 |
-
"input.hands_free": "Hands-free",
|
| 39 |
"input.lang_label": "Lang:",
|
| 40 |
"input.voice_input": "Voice input",
|
| 41 |
"input.upload": "Upload your own policy PDF",
|
|
@@ -142,7 +141,7 @@ export const UI_STRINGS = {
|
|
| 142 |
"welcome.heading_a": "आपके लिए सही स्वास्थ्य पॉलिसी खोजें — ",
|
| 143 |
"welcome.heading_b": "बस आपके लिए",
|
| 144 |
"welcome.heading_c": "।",
|
| 145 |
-
"welcome.subtitle": "मैं
|
| 146 |
"welcome.no_commissions": "मेरी रैंकिंग में कोई ब्रोकर कमीशन नहीं।",
|
| 147 |
"welcome.source_link": "हर तथ्य का source link है।",
|
| 148 |
"welcome.trust_title": "सच बताइए — मुश्किल बातें भी।",
|
|
@@ -152,7 +151,6 @@ export const UI_STRINGS = {
|
|
| 152 |
"input.placeholder": "Coverage, waiting period, exclusion, या तुलना के बारे में पूछिए…",
|
| 153 |
"input.send": "भेजें",
|
| 154 |
"input.voice_reply": "आवाज़ में जवाब",
|
| 155 |
-
"input.hands_free": "हैंड्स-फ्री",
|
| 156 |
"input.lang_label": "भाषा:",
|
| 157 |
"input.voice_input": "आवाज़ input",
|
| 158 |
"input.upload": "अपनी policy PDF upload करें",
|
|
|
|
| 24 |
"welcome.heading_a": "Find a health policy that genuinely fits ",
|
| 25 |
"welcome.heading_b": "you",
|
| 26 |
"welcome.heading_c": ".",
|
| 27 |
+
"welcome.subtitle": "I'll ask a few short questions, then show you 3 policies that match — with the exact reasons each ranked well.",
|
| 28 |
"welcome.no_commissions": "No broker commissions in my ranking.",
|
| 29 |
"welcome.source_link": "Every fact you see has a source link.",
|
| 30 |
"welcome.trust_title": "Tell me the truth — even on the hard things.",
|
|
|
|
| 35 |
"input.placeholder": "Ask about coverage, waiting periods, exclusions, or compare policies…",
|
| 36 |
"input.send": "Send",
|
| 37 |
"input.voice_reply": "Voice reply",
|
|
|
|
| 38 |
"input.lang_label": "Lang:",
|
| 39 |
"input.voice_input": "Voice input",
|
| 40 |
"input.upload": "Upload your own policy PDF",
|
|
|
|
| 141 |
"welcome.heading_a": "आपके लिए सही स्वास्थ्य पॉलिसी खोजें — ",
|
| 142 |
"welcome.heading_b": "बस आपके लिए",
|
| 143 |
"welcome.heading_c": "।",
|
| 144 |
+
"welcome.subtitle": "मैं कुछ छोटे सवाल पूछूंगा, फिर 3 पॉलिसियाँ दिखाऊंगा जो आपके लिए सबसे सही हैं — सटीक कारणों के साथ।",
|
| 145 |
"welcome.no_commissions": "मेरी रैंकिंग में कोई ब्रोकर कमीशन नहीं।",
|
| 146 |
"welcome.source_link": "हर तथ्य का source link है।",
|
| 147 |
"welcome.trust_title": "सच बताइए — मुश्किल बातें भी।",
|
|
|
|
| 151 |
"input.placeholder": "Coverage, waiting period, exclusion, या तुलना के बारे में पूछिए…",
|
| 152 |
"input.send": "भेजें",
|
| 153 |
"input.voice_reply": "आवाज़ में जवाब",
|
|
|
|
| 154 |
"input.lang_label": "भाषा:",
|
| 155 |
"input.voice_input": "आवाज़ input",
|
| 156 |
"input.upload": "अपनी policy PDF upload करें",
|
|
@@ -27,6 +27,6 @@ _Auto-generated. Source: `eval/gold_qa.json` + `eval/results.json` + `eval/run.p
|
|
| 27 |
## Methodology
|
| 28 |
|
| 29 |
- Gold Q&A built by 3 pipelines: auto-from-extraction (templated), LLM-drafted (human-verified), hand-crafted adversarial. See `70-docs/03-eval-plan.md`.
|
| 30 |
-
- Grader: NIM
|
| 31 |
- Re-run: `python -m eval.run [--limit N] [--policy <id>]`.
|
| 32 |
- CI gate: `.github/workflows/eval.yml` runs eval on every PR; blocks merge if factual_accuracy < 0.65 or citation_accuracy < 0.55.
|
|
|
|
| 27 |
## Methodology
|
| 28 |
|
| 29 |
- Gold Q&A built by 3 pipelines: auto-from-extraction (templated), LLM-drafted (human-verified), hand-crafted adversarial. See `70-docs/03-eval-plan.md`.
|
| 30 |
+
- Grader: NIM Mistral Large 3 675B (Mistral; primary judge per D-022) — different family from the Qwen 3-Next 80B brain → non-circular (D-019, 2026-05-14). The earlier Groq Llama grader was retired in the same consolidation.
|
| 31 |
- Re-run: `python -m eval.run [--limit N] [--policy <id>]`.
|
| 32 |
- CI gate: `.github/workflows/eval.yml` runs eval on every PR; blocks merge if factual_accuracy < 0.65 or citation_accuracy < 0.55.
|
|
@@ -24,7 +24,7 @@ All gates run for EVERY upload. Block on any failure; the audit trail captures t
|
|
| 24 |
| 1 | Retrieval floor | Top-1 cosine < 0.30 OR avg top-5 < 0.22 → refuse outright |
|
| 25 |
| 2 | Citation integrity | Any `[Source:…]` in the bot's reply must point to a real retrieved chunk's policy_name |
|
| 26 |
| 3 | Numeric grounding | Every ₹, %, day/month/year in the reply must appear in retrieved chunks (regex) |
|
| 27 |
-
| 4 | LLM-judge faithfulness | NIM Llama-4 Maverick (
|
| 28 |
| 5 (Indic) | Hinglish drift LLM-judge | Same idea on the Hinglish back-translation vs the English source |
|
| 29 |
|
| 30 |
Plus **regex anchors + back-translate cosine** as additional drift checks
|
|
|
|
| 24 |
| 1 | Retrieval floor | Top-1 cosine < 0.30 OR avg top-5 < 0.22 → refuse outright |
|
| 25 |
| 2 | Citation integrity | Any `[Source:…]` in the bot's reply must point to a real retrieved chunk's policy_name |
|
| 26 |
| 3 | Numeric grounding | Every ₹, %, day/month/year in the reply must appear in retrieved chunks (regex) |
|
| 27 |
+
| 4 | LLM-judge faithfulness | NIM Mistral Large 3 675B (primary judge per D-022) + Llama-4 Maverick (fallback) inspects the reply against retrieved chunks; outputs strict JSON; different family from Qwen 3-Next 80B brain (primary per D-022) + DeepSeek-V4 (fallback) → non-circular eval (D-019). |
|
| 28 |
| 5 (Indic) | Hinglish drift LLM-judge | Same idea on the Hinglish back-translation vs the English source |
|
| 29 |
|
| 30 |
Plus **regex anchors + back-translate cosine** as additional drift checks
|
|
@@ -11,8 +11,8 @@ Lineage for every artefact below is documented in [`kb/AUDIT_TRAIL.md`](../kb/AU
|
|
| 11 |
| `download_corpus.py` | 1. SOURCE → 3. DOWNLOAD | `rag/corpus/<insurer>/*.pdf` + `_manifest.json`. HEAD-checks + magic-byte sniff + retry on 403/timeout. |
|
| 12 |
| `download_retry.py` | 3. DOWNLOAD | Retries the failures from the previous run. |
|
| 13 |
| `download_regulatory.py` | 3. DOWNLOAD | IRDAI / regulatory PDFs (deferred from v1; see [ADR-017](../70-docs/60-decisions/ADR-017-irdai-corpus-playwright-rescue.md)). |
|
| 14 |
-
| `ingest.py` | 4. PARSE → 5. CHUNK → 6. EMBED → 7. INDEX | The big one. `read_pdf_pages` (pdfplumber) → `chunk_pages` (800-tok / 120 overlap, sentence-aware) → BGE embed → `chromadb.PersistentClient.add(...)`. Carries the in-process HNSW bloat tripwire ([ADR-029](../70-docs/60-decisions/ADR-029-
|
| 15 |
-
| `extract.py` | 8. STRUCTURED EXTRACTION | LLM extraction over each PDF using `schema.py::HealthPolicy` (62 fields).
|
| 16 |
| `build_kb.py` | 9. SCORECARD → KB MIRROR | Runs `backend/scorecard.py` per policy and regenerates the human-readable `kb/policies/<id>.md` tree. |
|
| 17 |
| `source_map.py` | post-build | Builds `source_map.json` — every chunk → (PDF path, page, span) for citation rendering. |
|
| 18 |
|
|
@@ -62,5 +62,5 @@ Total cost from cold: < $2 (BGE local + ~80 LLM extractions). Wall-time: ~30-40
|
|
| 62 |
- [ADR-011](../70-docs/60-decisions/ADR-011-bge-local-embeddings.md) — why local BGE replaced Voyage
|
| 63 |
- [ADR-018](../70-docs/60-decisions/ADR-018-chunk-size-sweep-deferred.md) — 800/120 chunk-size baseline
|
| 64 |
- [ADR-020](../70-docs/60-decisions/ADR-020-code-data-split-hf-dataset.md) — code-vs-data repo split
|
| 65 |
-
- [ADR-029](../70-docs/60-decisions/ADR-029-
|
| 66 |
- [`kb/AUDIT_TRAIL.md`](../kb/AUDIT_TRAIL.md) — end-to-end lineage doc
|
|
|
|
| 11 |
| `download_corpus.py` | 1. SOURCE → 3. DOWNLOAD | `rag/corpus/<insurer>/*.pdf` + `_manifest.json`. HEAD-checks + magic-byte sniff + retry on 403/timeout. |
|
| 12 |
| `download_retry.py` | 3. DOWNLOAD | Retries the failures from the previous run. |
|
| 13 |
| `download_regulatory.py` | 3. DOWNLOAD | IRDAI / regulatory PDFs (deferred from v1; see [ADR-017](../70-docs/60-decisions/ADR-017-irdai-corpus-playwright-rescue.md)). |
|
| 14 |
+
| `ingest.py` | 4. PARSE → 5. CHUNK → 6. EMBED → 7. INDEX | The big one. `read_pdf_pages` (pdfplumber) → `chunk_pages` (800-tok / 120 overlap, sentence-aware) → BGE embed → `chromadb.PersistentClient.add(...)`. Carries the in-process HNSW bloat tripwire ([ADR-029](../70-docs/60-decisions/ADR-029-hnsw-bloat-tripwire.md)). |
|
| 15 |
+
| `extract.py` | 8. STRUCTURED EXTRACTION | LLM extraction over each PDF using `schema.py::HealthPolicy` (62 fields). `get_brain_llm()` (Qwen 3-Next 80B primary) + `get_fast_brain_llm()` (Nemotron Nano 30B) — both `NimChainLLM` fallback chains. Writes `rag/extracted/<policy_id>.json` + upserts `policies.duckdb`. |
|
| 16 |
| `build_kb.py` | 9. SCORECARD → KB MIRROR | Runs `backend/scorecard.py` per policy and regenerates the human-readable `kb/policies/<id>.md` tree. |
|
| 17 |
| `source_map.py` | post-build | Builds `source_map.json` — every chunk → (PDF path, page, span) for citation rendering. |
|
| 18 |
|
|
|
|
| 62 |
- [ADR-011](../70-docs/60-decisions/ADR-011-bge-local-embeddings.md) — why local BGE replaced Voyage
|
| 63 |
- [ADR-018](../70-docs/60-decisions/ADR-018-chunk-size-sweep-deferred.md) — 800/120 chunk-size baseline
|
| 64 |
- [ADR-020](../70-docs/60-decisions/ADR-020-code-data-split-hf-dataset.md) — code-vs-data repo split
|
| 65 |
+
- [ADR-029](../70-docs/60-decisions/ADR-029-hnsw-bloat-tripwire.md) — HNSW bloat tripwire (3-layer defence)
|
| 66 |
- [`kb/AUDIT_TRAIL.md`](../kb/AUDIT_TRAIL.md) — end-to-end lineage doc
|
|
@@ -338,10 +338,10 @@ async def main():
|
|
| 338 |
# different MoE routing path so a Pro timeout doesn't always reproduce
|
| 339 |
# in Flash. Sarvam-M was the old fallback but its <think> tags ate the
|
| 340 |
# 2048 output cap before reaching JSON, causing ~20 truncation failures.
|
| 341 |
-
primary = get_brain_llm() #
|
| 342 |
-
fallback = get_fast_brain_llm() #
|
| 343 |
|
| 344 |
-
print(f"Extracting {len(pdfs)} policies. Primary=
|
| 345 |
t0 = time.time()
|
| 346 |
ok = 0
|
| 347 |
for i, pdf in enumerate(pdfs, 1):
|
|
|
|
| 338 |
# different MoE routing path so a Pro timeout doesn't always reproduce
|
| 339 |
# in Flash. Sarvam-M was the old fallback but its <think> tags ate the
|
| 340 |
# 2048 output cap before reaching JSON, causing ~20 truncation failures.
|
| 341 |
+
primary = get_brain_llm() # BRAIN_CHAIN — Qwen 3-Next 80B primary (D-022)
|
| 342 |
+
fallback = get_fast_brain_llm() # FAST_BRAIN_CHAIN — Nemotron Nano 30B primary (D-022)
|
| 343 |
|
| 344 |
+
print(f"Extracting {len(pdfs)} policies. Primary=BRAIN_CHAIN (Qwen 80B), Fallback=FAST_BRAIN_CHAIN (Nemotron 30B).\n")
|
| 345 |
t0 = time.time()
|
| 346 |
ok = 0
|
| 347 |
for i, pdf in enumerate(pdfs, 1):
|
|
@@ -2,7 +2,7 @@
|
|
| 2 |
|
| 3 |
Loose collection of CLI scripts: corpus operations, data uploads, probes, KB regeneration, scheduled-job runners. Nothing under `tools/` is imported by the live server — `backend/` and `rag/` are the runtime surface.
|
| 4 |
|
| 5 |
-
Scheduling for the long-running ones is wired via macOS LaunchAgents — see `CRON_README.md` in this folder for cadence + script paths, and [ADR-029](../70-docs/60-decisions/ADR-029-
|
| 6 |
|
| 7 |
## Corpus + extraction batch ops
|
| 8 |
|
|
@@ -76,5 +76,5 @@ Scheduling for the long-running ones is wired via macOS LaunchAgents — see `CR
|
|
| 76 |
## Related
|
| 77 |
|
| 78 |
- `CRON_README.md` (this folder) — LaunchAgent cadence reference
|
| 79 |
-
- [ADR-020](../70-docs/60-decisions/ADR-020-code-data-split-hf-dataset.md), [ADR-024](../70-docs/60-decisions/ADR-024-triple-mirror-code-and-data.md), [ADR-029](../70-docs/60-decisions/ADR-029-
|
| 80 |
- `80-audit/ENTERPRISE_AUDIT.md` — defect register, including silent-LaunchAgent regressions (D-002)
|
|
|
|
| 2 |
|
| 3 |
Loose collection of CLI scripts: corpus operations, data uploads, probes, KB regeneration, scheduled-job runners. Nothing under `tools/` is imported by the live server — `backend/` and `rag/` are the runtime surface.
|
| 4 |
|
| 5 |
+
Scheduling for the long-running ones is wired via macOS LaunchAgents — see `CRON_README.md` in this folder for cadence + script paths, and [ADR-029](../70-docs/60-decisions/ADR-029-hnsw-bloat-tripwire.md) for the disk-safety LaunchAgents.
|
| 6 |
|
| 7 |
## Corpus + extraction batch ops
|
| 8 |
|
|
|
|
| 76 |
## Related
|
| 77 |
|
| 78 |
- `CRON_README.md` (this folder) — LaunchAgent cadence reference
|
| 79 |
+
- [ADR-020](../70-docs/60-decisions/ADR-020-code-data-split-hf-dataset.md), [ADR-024](../70-docs/60-decisions/ADR-024-triple-mirror-code-and-data.md), [ADR-029](../70-docs/60-decisions/ADR-029-hnsw-bloat-tripwire.md)
|
| 80 |
- `80-audit/ENTERPRISE_AUDIT.md` — defect register, including silent-LaunchAgent regressions (D-002)
|