Upload 21 files

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

MIGRATION_NOTES.md

@@ -0,0 +1,3 @@
+# Migration notes
+
+The two supplied LangGraph archives were duplicates. Their custom runtime did not instantiate LangGraph. This repository replaces that loop with a real `StateGraph`; the old Colab script and simulated web/official tools are not carried forward as production components. The SMC utility remains available separately, with safe log-weight normalization, importance correction, and no final-step resampling.

README.md

@@ -1,3 +1,113 @@
 ---
-license: fair-noncommercial-research-license
+license: mit
+tags:
+- langgraph
+- rag
+- bayesian
+- probabilistic
+- uncertainty
+- retrieval
 ---
+
+# Bayesian LangGraph RAG
+
+A corrected, Hugging Face-ready Bayesian RAG implementation using a **real LangGraph `StateGraph`**.
+
+The original repository used a custom Python loop under the LangGraph name and could erase its final SMC posterior through final-step resampling. This version separates retrieval inference from orchestration and uses the graph:
+
+`START -> retrieve -> assess -> (generate | abstain) -> END`
+
+The retriever uses the generalized-Bayesian posterior
+
+`p(d | q) ∝ p(d) (similarity(q,d) + eps)^tau`
+
+with a Beta reliability prior plus authority and freshness metadata.
+
+## What was corrected
+
+- Replaced the custom pseudo-LangGraph runtime with an actual `StateGraph`.
+- Removed Colab-only setup, Drive mounts, repository cloning, and import-time execution.
+- Fixed all-`-inf` log-weight normalization.
+- Fixed final-step resampling so the posterior cannot be reset to uniform.
+- Added proposal-density correction to the reusable SMC implementation.
+- Removed deterministic identical-particle demos as evidence of inference quality.
+- Replaced hard-coded "Bayes wins" benchmark flags with end-to-end document retrieval and grounded answering.
+- Added uncertainty-aware abstention for missing evidence and unresolved source conflicts.
+- Added reproducible RAG evaluation with a plain TF-IDF baseline.
+
+## Install
+
+```bash
+pip install -e .
+```
+
+No GPU is required. The default generator is extractive and citation-grounded; a model-backed generator can be supplied if desired.
+
+## Quick start
+
+```python
+from bayesian_langgraph import DocumentRecord, BayesianIndex, BayesianRAGGraph
+
+docs = [
+    DocumentRecord(
+        id="official",
+        title="Current portal policy",
+        text="The current HR portal is PeopleHub.",
+        source="official",
+        metadata={"reliability_alpha": 18, "reliability_beta": 2, "authority": 0.98, "freshness": 1.0},
+    ),
+    DocumentRecord(
+        id="legacy",
+        title="HR portal guide",
+        text="The HR portal is WorkSphere.",
+        source="legacy-cache",
+        metadata={"reliability_alpha": 6, "reliability_beta": 6, "authority": 0.45, "freshness": 0.18},
+    ),
+]
+
+graph_rag = BayesianRAGGraph(BayesianIndex(docs))
+print(graph_rag.invoke("What is the current HR portal?"))
+```
+
+## RAG validation
+
+Run:
+
+```bash
+PYTHONPATH=src:. python evaluation/rag_benchmark.py
+```
+
+The committed result is in `artifacts/rag_benchmark_results.json`.
+
+| Suite | System | Recall@1 | MRR | Answer accuracy | Abstention accuracy | End-to-end accuracy |
+|---|---|---:|---:|---:|---:|---:|
+| Clean RAG (8 QA) | Bayesian | 1.000 | 1.000 | 1.000 | n/a | 1.000 |
+| Clean RAG (8 QA) | Plain TF-IDF | 1.000 | 1.000 | 1.000 | n/a | 1.000 |
+| Conflict/missing RAG (18 QA) | Bayesian | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 |
+| Conflict/missing RAG (18 QA) | Plain TF-IDF | 0.250 | 0.611 | 0.250 | 0.333 | 0.278 |
+
+These are controlled end-to-end repository tests, not a claim of universal RAG accuracy.
+
+### Framework-level LangGraph RAG test
+
+`evaluation/framework_rag_benchmark.py` executes every case through the compiled LangGraph graph itself. The validated topology is:
+
+`START -> retrieve -> assess -> {generate | abstain} -> END`.
+
+It was validated against **langgraph 1.2.16** with **langchain-core 1.6.11**. Results: **8/8 clean cases** and **18/18 conflict/missing-evidence cases** passed end-to-end. The committed output is `artifacts/framework_rag_benchmark_results.json`.
+
+Run it with:
+
+```bash
+PYTHONPATH=src:. python evaluation/framework_rag_benchmark.py
+```
+
+## Layout
+
+- `src/bayesian_langgraph/core.py` — posterior retrieval, abstention, grounded answerer
+- `src/bayesian_langgraph/graph.py` — actual LangGraph `StateGraph`
+- `src/bayesian_langgraph/smc.py` — corrected importance-weighted SMC utility
+- `evaluation/rag_benchmark.py` — controlled retrieval/grounding benchmark
+- `evaluation/framework_rag_benchmark.py` — compiled `StateGraph` RAG benchmark
+- `tests/` — regression and graph-integration tests
+- `VALIDATION.md` — exact framework/version validation record

VALIDATION.md

@@ -0,0 +1,12 @@
+# Validation
+
+Validated on Python 3.13.5 against the supplied `langgraph 1.2.16` and `langchain-core 1.6.11` source archives.
+
+- Repository tests: 8 passed.
+- Framework RAG suite: 8/8 clean cases passed.
+- Framework RAG suite: 18/18 stale/conflicting/missing-evidence cases passed.
+- Package wheel build: passed.
+- The orchestration is a real compiled `langgraph.graph.StateGraph`.
+- Validated graph nodes: `START`, `retrieve`, `assess`, `generate`, `abstain`, `END`.
+
+The sandbox cannot resolve PyPI. For framework-source validation only, a test harness supplied minimal compatibility modules for unavailable peripheral tracing/checkpoint/SDK dependencies. Those test shims are not part of this repository; the graph builder/executor under test is the supplied LangGraph source.

artifacts/framework_rag_benchmark_results.json

@@ -0,0 +1,299 @@
+{
+  "framework": "langgraph",
+  "stategraph_nodes": [
+    "__end__",
+    "__start__",
+    "abstain",
+    "assess",
+    "generate",
+    "retrieve"
+  ],
+  "stategraph_edges": [
+    [
+      "__start__",
+      "retrieve"
+    ],
+    [
+      "abstain",
+      "__end__"
+    ],
+    [
+      "assess",
+      "abstain"
+    ],
+    [
+      "assess",
+      "generate"
+    ],
+    [
+      "generate",
+      "__end__"
+    ],
+    [
+      "retrieve",
+      "assess"
+    ]
+  ],
+  "clean": {
+    "n_cases": 8,
+    "passed": 8,
+    "accuracy": 1.0,
+    "rows": [
+      {
+        "id": "c1",
+        "ok": true,
+        "answer": "The observatory opens to visitors at 9 AM every Saturday. [C1]",
+        "citations": [
+          "C1"
+        ],
+        "abstained": false,
+        "top_doc": "C1"
+      },
+      {
+        "id": "c2",
+        "ok": true,
+        "answer": "The chemistry safety briefing is held in Hall 7. [C2]",
+        "citations": [
+          "C2"
+        ],
+        "abstained": false,
+        "top_doc": "C2"
+      },
+      {
+        "id": "c3",
+        "ok": true,
+        "answer": "Library books may be borrowed for 21 days. [C3]",
+        "citations": [
+          "C3"
+        ],
+        "abstained": false,
+        "top_doc": "C3"
+      },
+      {
+        "id": "c4",
+        "ok": true,
+        "answer": "The robotics club meets on Thursday evenings. [C4]",
+        "citations": [
+          "C4"
+        ],
+        "abstained": false,
+        "top_doc": "C4"
+      },
+      {
+        "id": "c5",
+        "ok": true,
+        "answer": "Visitor parking is located beside Gate 3. [C5]",
+        "citations": [
+          "C5"
+        ],
+        "abstained": false,
+        "top_doc": "C5"
+      },
+      {
+        "id": "c6",
+        "ok": true,
+        "answer": "The campus shuttle begins service at 6:30 AM. [C6]",
+        "citations": [
+          "C6"
+        ],
+        "abstained": false,
+        "top_doc": "C6"
+      },
+      {
+        "id": "c7",
+        "ok": true,
+        "answer": "The data science seminar carries two academic credits. [C7]",
+        "citations": [
+          "C7"
+        ],
+        "abstained": false,
+        "top_doc": "C7"
+      },
+      {
+        "id": "c8",
+        "ok": true,
+        "answer": "The emergency assembly point is the North Lawn. [C8]",
+        "citations": [
+          "C8"
+        ],
+        "abstained": false,
+        "top_doc": "C8"
+      }
+    ]
+  },
+  "conflict": {
+    "n_cases": 18,
+    "passed": 18,
+    "accuracy": 1.0,
+    "rows": [
+      {
+        "id": "q1",
+        "ok": true,
+        "answer": "All employee self-service tasks, including leave requests and payslips, are now handled in PeopleHub. [OFF-HR-PORTAL]",
+        "citations": [
+          "OFF-HR-PORTAL"
+        ],
+        "abstained": false,
+        "top_doc": "OFF-HR-PORTAL"
+      },
+      {
+        "id": "q2",
+        "ok": true,
+        "answer": "The 2026 employee benefits policy grants 18 paid leave days per calendar year. [OFF-LEAVE]",
+        "citations": [
+          "OFF-LEAVE"
+        ],
+        "abstained": false,
+        "top_doc": "OFF-LEAVE"
+      },
+      {
+        "id": "q3",
+        "ok": true,
+        "answer": "Under the active flexible-work policy, employees may work remotely up to two days per week with manager approval. [OFF-REMOTE]",
+        "citations": [
+          "OFF-REMOTE"
+        ],
+        "abstained": false,
+        "top_doc": "OFF-REMOTE"
+      },
+      {
+        "id": "q4",
+        "ok": true,
+        "answer": "Payroll publishes salary slips by the fifth working day of each month. [OFF-PAYSLIP]",
+        "citations": [
+          "OFF-PAYSLIP"
+        ],
+        "abstained": false,
+        "top_doc": "OFF-PAYSLIP"
+      },
+      {
+        "id": "q5",
+        "ok": true,
+        "answer": "The security baseline now requires WireGuard for the standard corporate VPN profile. [OFF-VPN]",
+        "citations": [
+          "OFF-VPN"
+        ],
+        "abstained": false,
+        "top_doc": "OFF-VPN"
+      },
+      {
+        "id": "q6",
+        "ok": true,
+        "answer": "Production database backups have a retention period of 30 days. [OFF-BACKUP]",
+        "citations": [
+          "OFF-BACKUP"
+        ],
+        "abstained": false,
+        "top_doc": "OFF-BACKUP"
+      },
+      {
+        "id": "q7",
+        "ok": true,
+        "answer": "The current gateway quota permits 120 requests per minute per token. [OFF-RATE]",
+        "citations": [
+          "OFF-RATE"
+        ],
+        "abstained": false,
+        "top_doc": "OFF-RATE"
+      },
+      {
+        "id": "q8",
+        "ok": true,
+        "answer": "The production primary region is ap-south-1; disaster recovery remains in a separate region. [OFF-REGION]",
+        "citations": [
+          "OFF-REGION"
+        ],
+        "abstained": false,
+        "top_doc": "OFF-REGION"
+      },
+      {
+        "id": "q9",
+        "ok": true,
+        "answer": "For domestic business travel, reimbursable meals are capped at INR 4,500 per day. [OFF-EXPENSE]",
+        "citations": [
+          "OFF-EXPENSE"
+        ],
+        "abstained": false,
+        "top_doc": "OFF-EXPENSE"
+      },
+      {
+        "id": "q10",
+        "ok": true,
+        "answer": "Expense claims must be submitted within 15 days of the transaction date. [OFF-REIMB]",
+        "citations": [
+          "OFF-REIMB"
+        ],
+        "abstained": false,
+        "top_doc": "OFF-REIMB"
+      },
+      {
+        "id": "q11",
+        "ok": true,
+        "answer": "Approved business travel must be booked through TripDesk. [OFF-TRAVEL]",
+        "citations": [
+          "OFF-TRAVEL"
+        ],
+        "abstained": false,
+        "top_doc": "OFF-TRAVEL"
+      },
+      {
+        "id": "q12",
+        "ok": true,
+        "answer": "Procurement review is mandatory for purchases above INR 50,000. [OFF-PROC]",
+        "citations": [
+          "OFF-PROC"
+        ],
+        "abstained": false,
+        "top_doc": "OFF-PROC"
+      },
+      {
+        "id": "q13",
+        "ok": true,
+        "answer": "I do not know based on the provided documents.",
+        "citations": [],
+        "abstained": true,
+        "top_doc": "LEG-RATE"
+      },
+      {
+        "id": "q14",
+        "ok": true,
+        "answer": "I do not know based on the provided documents.",
+        "citations": [],
+        "abstained": true,
+        "top_doc": "OFF-HR-PORTAL"
+      },
+      {
+        "id": "q15",
+        "ok": true,
+        "answer": "I do not know based on the provided documents.",
+        "citations": [],
+        "abstained": true,
+        "top_doc": "OFF-HR-PORTAL"
+      },
+      {
+        "id": "q16",
+        "ok": true,
+        "answer": "I do not know based on the provided documents.",
+        "citations": [],
+        "abstained": true,
+        "top_doc": "OFF-REIMB"
+      },
+      {
+        "id": "q17",
+        "ok": true,
+        "answer": "I do not know based on the provided documents.",
+        "citations": [],
+        "abstained": true,
+        "top_doc": "OFF-LAB-A"
+      },
+      {
+        "id": "q18",
+        "ok": true,
+        "answer": "I do not know based on the provided documents.",
+        "citations": [],
+        "abstained": true,
+        "top_doc": "OFF-OFFICE-A"
+      }
+    ]
+  }
+}

artifacts/rag_benchmark_results.json

@@ -0,0 +1,784 @@
+{
+  "benchmark": "controlled_rag_v2",
+  "methodology": "Two end-to-end suites. CLEAN checks ordinary retrieval+grounded answer synthesis. CONFLICT adds stale authoritative conflicts, missing evidence, and unresolved equal-authority contradictions. The generator is extractive and citation-grounded so the evaluation isolates retrieval and uncertainty behavior without an external LLM confound.",
+  "clean": {
+    "bayesian": {
+      "system": "bayesian",
+      "n_cases": 8,
+      "answerable_cases": 8,
+      "abstention_cases": 0,
+      "recall_at_1": 1.0,
+      "recall_at_3": 1.0,
+      "mrr": 1.0,
+      "end_to_end_accuracy": 1.0,
+      "grounding_accuracy": 1.0,
+      "abstention_accuracy": null,
+      "answer_accuracy_on_answerable": 1.0,
+      "top_posterior_brier": 2.8128197163138293e-05
+    },
+    "baseline": {
+      "system": "plain_tfidf",
+      "n_cases": 8,
+      "answerable_cases": 8,
+      "abstention_cases": 0,
+      "recall_at_1": 1.0,
+      "recall_at_3": 1.0,
+      "mrr": 1.0,
+      "end_to_end_accuracy": 1.0,
+      "grounding_accuracy": 1.0,
+      "abstention_accuracy": null,
+      "answer_accuracy_on_answerable": 1.0,
+      "top_posterior_brier": 0.0
+    },
+    "bayesian_cases": [
+      {
+        "id": "c1",
+        "query": "When does the observatory open on Saturday?",
+        "expected_abstain": false,
+        "top_doc": "C1",
+        "retrieval_score": 0.552814,
+        "top_posterior": 0.991539,
+        "answer": "The observatory opens to visitors at 9 AM every Saturday. [C1]",
+        "citations": [
+          "C1"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "c2",
+        "query": "Where is the chemistry safety briefing held?",
+        "expected_abstain": false,
+        "top_doc": "C2",
+        "retrieval_score": 0.825787,
+        "top_posterior": 0.996101,
+        "answer": "The chemistry safety briefing is held in Hall 7. [C2]",
+        "citations": [
+          "C2"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "c3",
+        "query": "How long may library books be borrowed?",
+        "expected_abstain": false,
+        "top_doc": "C3",
+        "retrieval_score": 0.683723,
+        "top_posterior": 0.994378,
+        "answer": "Library books may be borrowed for 21 days. [C3]",
+        "citations": [
+          "C3"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "c4",
+        "query": "When does the robotics club meet?",
+        "expected_abstain": false,
+        "top_doc": "C4",
+        "retrieval_score": 0.742488,
+        "top_posterior": 0.995207,
+        "answer": "The robotics club meets on Thursday evenings. [C4]",
+        "citations": [
+          "C4"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "c5",
+        "query": "Where is visitor parking?",
+        "expected_abstain": false,
+        "top_doc": "C5",
+        "retrieval_score": 0.79521,
+        "top_posterior": 0.995804,
+        "answer": "Visitor parking is located beside Gate 3. [C5]",
+        "citations": [
+          "C5"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "c6",
+        "query": "What time does the campus shuttle begin service?",
+        "expected_abstain": false,
+        "top_doc": "C6",
+        "retrieval_score": 0.769605,
+        "top_posterior": 0.995529,
+        "answer": "The campus shuttle begins service at 6:30 AM. [C6]",
+        "citations": [
+          "C6"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "c7",
+        "query": "How many credits is the data science seminar worth?",
+        "expected_abstain": false,
+        "top_doc": "C7",
+        "retrieval_score": 0.836653,
+        "top_posterior": 0.996199,
+        "answer": "The data science seminar carries two academic credits. [C7]",
+        "citations": [
+          "C7"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "c8",
+        "query": "Where is the emergency assembly point?",
+        "expected_abstain": false,
+        "top_doc": "C8",
+        "retrieval_score": 0.683723,
+        "top_posterior": 0.994378,
+        "answer": "The emergency assembly point is the North Lawn. [C8]",
+        "citations": [
+          "C8"
+        ],
+        "correct": true,
+        "grounded": true
+      }
+    ],
+    "baseline_cases": [
+      {
+        "id": "c1",
+        "query": "When does the observatory open on Saturday?",
+        "expected_abstain": false,
+        "top_doc": "C1",
+        "retrieval_score": 0.552814,
+        "top_posterior": 1.0,
+        "answer": "The observatory opens to visitors at 9 AM every Saturday. [C1]",
+        "citations": [
+          "C1"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "c2",
+        "query": "Where is the chemistry safety briefing held?",
+        "expected_abstain": false,
+        "top_doc": "C2",
+        "retrieval_score": 0.825787,
+        "top_posterior": 1.0,
+        "answer": "The chemistry safety briefing is held in Hall 7. [C2]",
+        "citations": [
+          "C2"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "c3",
+        "query": "How long may library books be borrowed?",
+        "expected_abstain": false,
+        "top_doc": "C3",
+        "retrieval_score": 0.683723,
+        "top_posterior": 1.0,
+        "answer": "Library books may be borrowed for 21 days. [C3]",
+        "citations": [
+          "C3"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "c4",
+        "query": "When does the robotics club meet?",
+        "expected_abstain": false,
+        "top_doc": "C4",
+        "retrieval_score": 0.742488,
+        "top_posterior": 1.0,
+        "answer": "The robotics club meets on Thursday evenings. [C4]",
+        "citations": [
+          "C4"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "c5",
+        "query": "Where is visitor parking?",
+        "expected_abstain": false,
+        "top_doc": "C5",
+        "retrieval_score": 0.79521,
+        "top_posterior": 1.0,
+        "answer": "Visitor parking is located beside Gate 3. [C5]",
+        "citations": [
+          "C5"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "c6",
+        "query": "What time does the campus shuttle begin service?",
+        "expected_abstain": false,
+        "top_doc": "C6",
+        "retrieval_score": 0.769605,
+        "top_posterior": 1.0,
+        "answer": "The campus shuttle begins service at 6:30 AM. [C6]",
+        "citations": [
+          "C6"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "c7",
+        "query": "How many credits is the data science seminar worth?",
+        "expected_abstain": false,
+        "top_doc": "C7",
+        "retrieval_score": 0.836653,
+        "top_posterior": 1.0,
+        "answer": "The data science seminar carries two academic credits. [C7]",
+        "citations": [
+          "C7"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "c8",
+        "query": "Where is the emergency assembly point?",
+        "expected_abstain": false,
+        "top_doc": "C8",
+        "retrieval_score": 0.683723,
+        "top_posterior": 1.0,
+        "answer": "The emergency assembly point is the North Lawn. [C8]",
+        "citations": [
+          "C8"
+        ],
+        "correct": true,
+        "grounded": true
+      }
+    ]
+  },
+  "conflict": {
+    "bayesian": {
+      "system": "bayesian",
+      "n_cases": 18,
+      "answerable_cases": 12,
+      "abstention_cases": 6,
+      "recall_at_1": 1.0,
+      "recall_at_3": 1.0,
+      "mrr": 1.0,
+      "end_to_end_accuracy": 1.0,
+      "grounding_accuracy": 1.0,
+      "abstention_accuracy": 1.0,
+      "answer_accuracy_on_answerable": 1.0,
+      "top_posterior_brier": 0.09485010369519668
+    },
+    "baseline": {
+      "system": "plain_tfidf",
+      "n_cases": 18,
+      "answerable_cases": 12,
+      "abstention_cases": 6,
+      "recall_at_1": 0.25,
+      "recall_at_3": 1.0,
+      "mrr": 0.611111111111111,
+      "end_to_end_accuracy": 0.2777777777777778,
+      "grounding_accuracy": 1.0,
+      "abstention_accuracy": 0.3333333333333333,
+      "answer_accuracy_on_answerable": 0.25,
+      "top_posterior_brier": 0.26943925109630584
+    },
+    "bayesian_cases": [
+      {
+        "id": "q1",
+        "query": "What is the current HR portal for leave requests and payslips?",
+        "expected_abstain": false,
+        "top_doc": "OFF-HR-PORTAL",
+        "retrieval_score": 0.221707,
+        "top_posterior": 0.419857,
+        "answer": "All employee self-service tasks, including leave requests and payslips, are now handled in PeopleHub. [OFF-HR-PORTAL]",
+        "citations": [
+          "OFF-HR-PORTAL"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "q2",
+        "query": "How many paid leave days do employees get per year under the current policy?",
+        "expected_abstain": false,
+        "top_doc": "OFF-LEAVE",
+        "retrieval_score": 0.366908,
+        "top_posterior": 0.675297,
+        "answer": "The 2026 employee benefits policy grants 18 paid leave days per calendar year. [OFF-LEAVE]",
+        "citations": [
+          "OFF-LEAVE"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "q3",
+        "query": "How many remote work days per week are currently allowed with manager approval?",
+        "expected_abstain": false,
+        "top_doc": "OFF-REMOTE",
+        "retrieval_score": 0.383162,
+        "top_posterior": 0.598358,
+        "answer": "Under the active flexible-work policy, employees may work remotely up to two days per week with manager approval. [OFF-REMOTE]",
+        "citations": [
+          "OFF-REMOTE"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "q4",
+        "query": "By which working day are salary slips published each month?",
+        "expected_abstain": false,
+        "top_doc": "OFF-PAYSLIP",
+        "retrieval_score": 0.457118,
+        "top_posterior": 0.8609,
+        "answer": "Payroll publishes salary slips by the fifth working day of each month. [OFF-PAYSLIP]",
+        "citations": [
+          "OFF-PAYSLIP"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "q5",
+        "query": "Which VPN protocol is required by the current corporate security baseline?",
+        "expected_abstain": false,
+        "top_doc": "OFF-VPN",
+        "retrieval_score": 0.514507,
+        "top_posterior": 0.904463,
+        "answer": "The security baseline now requires WireGuard for the standard corporate VPN profile. [OFF-VPN]",
+        "citations": [
+          "OFF-VPN"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "q6",
+        "query": "What is the production database backup retention period?",
+        "expected_abstain": false,
+        "top_doc": "OFF-BACKUP",
+        "retrieval_score": 0.424384,
+        "top_posterior": 0.835659,
+        "answer": "Production database backups have a retention period of 30 days. [OFF-BACKUP]",
+        "citations": [
+          "OFF-BACKUP"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "q7",
+        "query": "What is the current public API rate limit per minute per token?",
+        "expected_abstain": false,
+        "top_doc": "OFF-RATE",
+        "retrieval_score": 0.260415,
+        "top_posterior": 0.600144,
+        "answer": "The current gateway quota permits 120 requests per minute per token. [OFF-RATE]",
+        "citations": [
+          "OFF-RATE"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "q8",
+        "query": "Which cloud region is primary for production?",
+        "expected_abstain": false,
+        "top_doc": "OFF-REGION",
+        "retrieval_score": 0.400734,
+        "top_posterior": 0.819994,
+        "answer": "The production primary region is ap-south-1; disaster recovery remains in a separate region. [OFF-REGION]",
+        "citations": [
+          "OFF-REGION"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "q9",
+        "query": "What is the domestic business travel meal expense cap per day?",
+        "expected_abstain": false,
+        "top_doc": "OFF-EXPENSE",
+        "retrieval_score": 0.391575,
+        "top_posterior": 0.520438,
+        "answer": "For domestic business travel, reimbursable meals are capped at INR 4,500 per day. [OFF-EXPENSE]",
+        "citations": [
+          "OFF-EXPENSE"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "q10",
+        "query": "Within how many days must expense reimbursement claims be submitted?",
+        "expected_abstain": false,
+        "top_doc": "OFF-REIMB",
+        "retrieval_score": 0.513156,
+        "top_posterior": 0.907517,
+        "answer": "Expense claims must be submitted within 15 days of the transaction date. [OFF-REIMB]",
+        "citations": [
+          "OFF-REIMB"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "q11",
+        "query": "Which portal must be used to book approved business travel?",
+        "expected_abstain": false,
+        "top_doc": "OFF-TRAVEL",
+        "retrieval_score": 0.580663,
+        "top_posterior": 0.832603,
+        "answer": "Approved business travel must be booked through TripDesk. [OFF-TRAVEL]",
+        "citations": [
+          "OFF-TRAVEL"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "q12",
+        "query": "Above what amount is procurement review mandatory?",
+        "expected_abstain": false,
+        "top_doc": "OFF-PROC",
+        "retrieval_score": 0.512226,
+        "top_posterior": 0.856608,
+        "answer": "Procurement review is mandatory for purchases above INR 50,000. [OFF-PROC]",
+        "citations": [
+          "OFF-PROC"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "q13",
+        "query": "What is the company dental insurance annual limit?",
+        "expected_abstain": true,
+        "top_doc": "LEG-RATE",
+        "retrieval_score": 0.325883,
+        "top_posterior": 0.829873,
+        "answer": "I do not know based on the provided documents.",
+        "citations": [],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "q14",
+        "query": "What is the cafeteria breakfast price?",
+        "expected_abstain": true,
+        "top_doc": "OFF-HR-PORTAL",
+        "retrieval_score": 0.0,
+        "top_posterior": 0.248018,
+        "answer": "I do not know based on the provided documents.",
+        "citations": [],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "q15",
+        "query": "What is the emergency satellite phone number?",
+        "expected_abstain": true,
+        "top_doc": "OFF-HR-PORTAL",
+        "retrieval_score": 0.0,
+        "top_posterior": 0.248018,
+        "answer": "I do not know based on the provided documents.",
+        "citations": [],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "q16",
+        "query": "What is the annual gym membership reimbursement?",
+        "expected_abstain": true,
+        "top_doc": "OFF-REIMB",
+        "retrieval_score": 0.178672,
+        "top_posterior": 0.385158,
+        "answer": "I do not know based on the provided documents.",
+        "citations": [],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "q17",
+        "query": "Which room is the graduate AI lab currently in?",
+        "expected_abstain": true,
+        "top_doc": "OFF-LAB-A",
+        "retrieval_score": 0.752053,
+        "top_posterior": 0.499378,
+        "answer": "I do not know based on the provided documents.",
+        "citations": [],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "q18",
+        "query": "When are Professor Sen's office hours?",
+        "expected_abstain": true,
+        "top_doc": "OFF-OFFICE-A",
+        "retrieval_score": 0.702454,
+        "top_posterior": 0.49929,
+        "answer": "I do not know based on the provided documents.",
+        "citations": [],
+        "correct": true,
+        "grounded": true
+      }
+    ],
+    "baseline_cases": [
+      {
+        "id": "q1",
+        "query": "What is the current HR portal for leave requests and payslips?",
+        "expected_abstain": false,
+        "top_doc": "LEG-HR-PORTAL",
+        "retrieval_score": 0.807312,
+        "top_posterior": 0.592469,
+        "answer": "The current HR portal for leave requests and payslips is WorkSphere. [LEG-HR-PORTAL]",
+        "citations": [
+          "LEG-HR-PORTAL"
+        ],
+        "correct": false,
+        "grounded": true
+      },
+      {
+        "id": "q2",
+        "query": "How many paid leave days do employees get per year under the current policy?",
+        "expected_abstain": false,
+        "top_doc": "LEG-LEAVE",
+        "retrieval_score": 0.602126,
+        "top_posterior": 0.462903,
+        "answer": "Employees receive 15 paid leave days per year under the leave policy. [LEG-LEAVE]",
+        "citations": [
+          "LEG-LEAVE"
+        ],
+        "correct": false,
+        "grounded": true
+      },
+      {
+        "id": "q3",
+        "query": "How many remote work days per week are currently allowed with manager approval?",
+        "expected_abstain": false,
+        "top_doc": "LEG-REMOTE",
+        "retrieval_score": 0.699429,
+        "top_posterior": 0.435096,
+        "answer": "The current remote work policy permits three remote days per week with manager approval. [LEG-REMOTE]",
+        "citations": [
+          "LEG-REMOTE"
+        ],
+        "correct": false,
+        "grounded": true
+      },
+      {
+        "id": "q4",
+        "query": "By which working day are salary slips published each month?",
+        "expected_abstain": false,
+        "top_doc": "LEG-PAYSLIP",
+        "retrieval_score": 0.672029,
+        "top_posterior": 0.550648,
+        "answer": "Salary slips are published on the tenth working day of each month. [LEG-PAYSLIP]",
+        "citations": [
+          "LEG-PAYSLIP"
+        ],
+        "correct": false,
+        "grounded": true
+      },
+      {
+        "id": "q5",
+        "query": "Which VPN protocol is required by the current corporate security baseline?",
+        "expected_abstain": false,
+        "top_doc": "LEG-VPN",
+        "retrieval_score": 0.524301,
+        "top_posterior": 0.420244,
+        "answer": "The supported corporate VPN protocol is OpenVPN. [LEG-VPN]",
+        "citations": [
+          "LEG-VPN"
+        ],
+        "correct": false,
+        "grounded": true
+      },
+      {
+        "id": "q6",
+        "query": "What is the production database backup retention period?",
+        "expected_abstain": false,
+        "top_doc": "LEG-BACKUP",
+        "retrieval_score": 0.556814,
+        "top_posterior": 0.467937,
+        "answer": "Production database backups are retained for 14 days. [LEG-BACKUP]",
+        "citations": [
+          "LEG-BACKUP"
+        ],
+        "correct": false,
+        "grounded": true
+      },
+      {
+        "id": "q7",
+        "query": "What is the current public API rate limit per minute per token?",
+        "expected_abstain": false,
+        "top_doc": "LEG-RATE",
+        "retrieval_score": 0.845222,
+        "top_posterior": 0.672178,
+        "answer": "The public API rate limit is 60 requests per minute per token. [LEG-RATE]",
+        "citations": [
+          "LEG-RATE"
+        ],
+        "correct": false,
+        "grounded": true
+      },
+      {
+        "id": "q8",
+        "query": "Which cloud region is primary for production?",
+        "expected_abstain": false,
+        "top_doc": "LEG-REGION",
+        "retrieval_score": 0.542439,
+        "top_posterior": 0.460259,
+        "answer": "The primary deployment region is us-east-1. [LEG-REGION]",
+        "citations": [
+          "LEG-REGION"
+        ],
+        "correct": false,
+        "grounded": true
+      },
+      {
+        "id": "q9",
+        "query": "What is the domestic business travel meal expense cap per day?",
+        "expected_abstain": false,
+        "top_doc": "LEG-EXPENSE",
+        "retrieval_score": 0.671336,
+        "top_posterior": 0.451066,
+        "answer": "The meal expense cap is INR 3,000 per day on domestic travel. [LEG-EXPENSE]",
+        "citations": [
+          "LEG-EXPENSE"
+        ],
+        "correct": false,
+        "grounded": true
+      },
+      {
+        "id": "q10",
+        "query": "Within how many days must expense reimbursement claims be submitted?",
+        "expected_abstain": false,
+        "top_doc": "OFF-REIMB",
+        "retrieval_score": 0.513156,
+        "top_posterior": 0.45338,
+        "answer": "Expense claims must be submitted within 15 days of the transaction date. [OFF-REIMB]",
+        "citations": [
+          "OFF-REIMB"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "q11",
+        "query": "Which portal must be used to book approved business travel?",
+        "expected_abstain": false,
+        "top_doc": "OFF-TRAVEL",
+        "retrieval_score": 0.580663,
+        "top_posterior": 0.514293,
+        "answer": "Approved business travel must be booked through TripDesk. [OFF-TRAVEL]",
+        "citations": [
+          "OFF-TRAVEL"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "q12",
+        "query": "Above what amount is procurement review mandatory?",
+        "expected_abstain": false,
+        "top_doc": "OFF-PROC",
+        "retrieval_score": 0.512226,
+        "top_posterior": 0.38514,
+        "answer": "Procurement review is mandatory for purchases above INR 50,000. [OFF-PROC]",
+        "citations": [
+          "OFF-PROC"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "q13",
+        "query": "What is the company dental insurance annual limit?",
+        "expected_abstain": true,
+        "top_doc": "LEG-RATE",
+        "retrieval_score": 0.325883,
+        "top_posterior": 1.0,
+        "answer": "The public API rate limit is 60 requests per minute per token. [LEG-RATE]",
+        "citations": [
+          "LEG-RATE"
+        ],
+        "correct": false,
+        "grounded": true
+      },
+      {
+        "id": "q14",
+        "query": "What is the cafeteria breakfast price?",
+        "expected_abstain": true,
+        "top_doc": "LEG-HR-PORTAL",
+        "retrieval_score": 0.0,
+        "top_posterior": 0.0,
+        "answer": "I do not know based on the provided documents.",
+        "citations": [],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "q15",
+        "query": "What is the emergency satellite phone number?",
+        "expected_abstain": true,
+        "top_doc": "LEG-HR-PORTAL",
+        "retrieval_score": 0.0,
+        "top_posterior": 0.0,
+        "answer": "I do not know based on the provided documents.",
+        "citations": [],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "q16",
+        "query": "What is the annual gym membership reimbursement?",
+        "expected_abstain": true,
+        "top_doc": "COM-REIMB",
+        "retrieval_score": 0.333617,
+        "top_posterior": 0.327295,
+        "answer": "Finance may request additional receipts during reimbursement review. [COM-REIMB]",
+        "citations": [
+          "COM-REIMB"
+        ],
+        "correct": false,
+        "grounded": true
+      },
+      {
+        "id": "q17",
+        "query": "Which room is the graduate AI lab currently in?",
+        "expected_abstain": true,
+        "top_doc": "OFF-LAB-A",
+        "retrieval_score": 0.752053,
+        "top_posterior": 0.5,
+        "answer": "The graduate AI lab has moved to room B-204. [OFF-LAB-A]",
+        "citations": [
+          "OFF-LAB-A"
+        ],
+        "correct": false,
+        "grounded": true
+      },
+      {
+        "id": "q18",
+        "query": "When are Professor Sen's office hours?",
+        "expected_abstain": true,
+        "top_doc": "OFF-OFFICE-A",
+        "retrieval_score": 0.702454,
+        "top_posterior": 0.5,
+        "answer": "Professor Sen holds office hours on Tuesday at 3 PM. [OFF-OFFICE-A]",
+        "citations": [
+          "OFF-OFFICE-A"
+        ],
+        "correct": false,
+        "grounded": true
+      }
+    ]
+  }
+}

evaluation/__init__.py

@@ -0,0 +1 @@
+# Benchmark package.

evaluation/framework_rag_benchmark.py

@@ -0,0 +1,58 @@
+from __future__ import annotations
+
+import json
+from pathlib import Path
+from typing import Any
+
+from bayesian_langgraph import BayesianIndex, BayesianRAGGraph
+from evaluation.rag_benchmark import CLEAN_CASES, CLEAN_DOCS, CONFLICT_CASES, CONFLICT_DOCS
+
+
+def _case_ok(out: dict[str, Any], case: dict[str, Any]) -> bool:
+    if case["abstain"]:
+        return bool(out.get("abstained")) and not out.get("citations")
+    return (
+        not out.get("abstained")
+        and case["doc"] in out.get("citations", [])
+        and str(case["gold"]).lower() in str(out.get("answer", "")).lower()
+    )
+
+
+def _run_suite(docs, cases) -> dict[str, Any]:
+    rag = BayesianRAGGraph(BayesianIndex(docs), k=4)
+    rows = []
+    for case in cases:
+        out = rag.invoke(case["q"])
+        rows.append({
+            "id": case["id"],
+            "ok": _case_ok(out, case),
+            "answer": out["answer"],
+            "citations": out["citations"],
+            "abstained": out["abstained"],
+            "top_doc": out["hits"][0]["id"] if out["hits"] else None,
+        })
+    return {
+        "n_cases": len(rows),
+        "passed": sum(r["ok"] for r in rows),
+        "accuracy": sum(r["ok"] for r in rows) / len(rows),
+        "rows": rows,
+    }
+
+
+def run_framework_benchmark(output_path: str | Path | None = None) -> dict[str, Any]:
+    rag = BayesianRAGGraph(BayesianIndex(CLEAN_DOCS), k=4)
+    graph = rag.graph.get_graph()
+    result = {
+        "framework": "langgraph",
+        "stategraph_nodes": sorted(graph.nodes.keys()),
+        "stategraph_edges": sorted([e.source, e.target] for e in graph.edges),
+        "clean": _run_suite(CLEAN_DOCS, CLEAN_CASES),
+        "conflict": _run_suite(CONFLICT_DOCS, CONFLICT_CASES),
+    }
+    if output_path is not None:
+        Path(output_path).write_text(json.dumps(result, indent=2), encoding="utf-8")
+    return result
+
+
+if __name__ == "__main__":
+    print(json.dumps(run_framework_benchmark(), indent=2))

evaluation/rag_benchmark.py

@@ -0,0 +1,206 @@
+from __future__ import annotations
+
+import json
+from pathlib import Path
+from typing import Any
+
+from bayesian_langgraph.core import (
+    BayesianIndex,
+    BayesianRetrievalConfig,
+    DocumentRecord,
+    PlainTfidfIndex,
+    ExtractiveGroundedGenerator,
+)
+
+
+def _md(kind: str) -> dict[str, Any]:
+    if kind == "official":
+        return {"reliability_alpha": 18, "reliability_beta": 2, "authority": 0.98, "freshness": 1.0, "tier": kind}
+    if kind == "stale":
+        return {"reliability_alpha": 6, "reliability_beta": 6, "authority": 0.45, "freshness": 0.18, "tier": kind}
+    if kind == "community":
+        return {"reliability_alpha": 7, "reliability_beta": 5, "authority": 0.55, "freshness": 0.65, "tier": kind}
+    if kind == "official_equal":
+        return {"reliability_alpha": 18, "reliability_beta": 2, "authority": 0.98, "freshness": 1.0, "tier": "official"}
+    return {"reliability_alpha": 1, "reliability_beta": 1, "authority": 0.5, "freshness": 0.5, "tier": kind}
+
+
+CONFLICT_DOCS = [
+    DocumentRecord("LEG-HR-PORTAL", "The current HR portal for leave requests and payslips is WorkSphere.", "Current HR portal login guide", "legacy-cache", _md("stale")),
+    DocumentRecord("OFF-HR-PORTAL", "All employee self-service tasks, including leave requests and payslips, are now handled in PeopleHub.", "Employee self-service migration completed", "official-handbook", _md("official")),
+    DocumentRecord("COM-HR-PORTAL", "Some teams used WorkSphere during migration; ask HR if your bookmark no longer works.", "HR portal discussion", "community-wiki", _md("community")),
+    DocumentRecord("LEG-LEAVE", "Employees receive 15 paid leave days per year under the leave policy.", "Paid leave entitlement FAQ", "legacy-cache", _md("stale")),
+    DocumentRecord("OFF-LEAVE", "The 2026 employee benefits policy grants 18 paid leave days per calendar year.", "2026 employee benefits policy", "official-handbook", _md("official")),
+    DocumentRecord("COM-LEAVE", "Leave balances vary by category and should be checked in the HR system.", "Leave balance help", "community-wiki", _md("community")),
+    DocumentRecord("LEG-REMOTE", "The current remote work policy permits three remote days per week with manager approval.", "Current remote work days", "legacy-cache", _md("stale")),
+    DocumentRecord("OFF-REMOTE", "Under the active flexible-work policy, employees may work remotely up to two days per week with manager approval.", "Flexible-work policy revision", "official-handbook", _md("official")),
+    DocumentRecord("COM-REMOTE", "Remote work requires manager approval and may vary during special projects.", "Remote work tips", "community-wiki", _md("community")),
+    DocumentRecord("LEG-PAYSLIP", "Salary slips are published on the tenth working day of each month.", "Payslip publication date", "legacy-cache", _md("stale")),
+    DocumentRecord("OFF-PAYSLIP", "Payroll publishes salary slips by the fifth working day of each month.", "Payroll service standard", "official-handbook", _md("official")),
+    DocumentRecord("COM-PAYSLIP", "Payslip publication can be delayed around holidays.", "Payroll discussion", "community-wiki", _md("community")),
+    DocumentRecord("LEG-VPN", "The supported corporate VPN protocol is OpenVPN.", "Corporate VPN protocol", "legacy-cache", _md("stale")),
+    DocumentRecord("OFF-VPN", "The security baseline now requires WireGuard for the standard corporate VPN profile.", "Network security baseline", "official-it", _md("official")),
+    DocumentRecord("COM-VPN", "OpenVPN profiles may still exist on older laptops.", "VPN migration notes", "community-wiki", _md("community")),
+    DocumentRecord("LEG-BACKUP", "Production database backups are retained for 14 days.", "Database backup retention policy", "legacy-cache", _md("stale")),
+    DocumentRecord("OFF-BACKUP", "Production database backups have a retention period of 30 days.", "Disaster recovery control DR-7", "official-it", _md("official")),
+    DocumentRecord("COM-BACKUP", "Teams should not rely on local snapshots as their only backup.", "Backup practices", "community-wiki", _md("community")),
+    DocumentRecord("LEG-RATE", "The public API rate limit is 60 requests per minute per token.", "Public API rate limit", "legacy-cache", _md("stale")),
+    DocumentRecord("OFF-RATE", "The current gateway quota permits 120 requests per minute per token.", "API gateway operating limits", "official-it", _md("official")),
+    DocumentRecord("COM-RATE", "Burst traffic may still be throttled even below the nominal quota.", "API troubleshooting", "community-wiki", _md("community")),
+    DocumentRecord("LEG-REGION", "The primary deployment region is us-east-1.", "Primary cloud deployment region", "legacy-cache", _md("stale")),
+    DocumentRecord("OFF-REGION", "The production primary region is ap-south-1; disaster recovery remains in a separate region.", "Cloud production topology", "official-it", _md("official")),
+    DocumentRecord("COM-REGION", "Developers may use several regions for non-production experiments.", "Cloud developer notes", "community-wiki", _md("community")),
+    DocumentRecord("LEG-EXPENSE", "The meal expense cap is INR 3,000 per day on domestic travel.", "Domestic meal expense cap", "legacy-cache", _md("stale")),
+    DocumentRecord("OFF-EXPENSE", "For domestic business travel, reimbursable meals are capped at INR 4,500 per day.", "Travel and expense policy 2026", "official-finance", _md("official")),
+    DocumentRecord("COM-EXPENSE", "Itemized receipts are recommended for meal reimbursement.", "Expense claim tips", "community-wiki", _md("community")),
+    DocumentRecord("LEG-REIMB", "Employees must submit reimbursement claims within 30 days of the expense.", "Reimbursement submission deadline", "legacy-cache", _md("stale")),
+    DocumentRecord("OFF-REIMB", "Expense claims must be submitted within 15 days of the transaction date.", "Expense reimbursement standard", "official-finance", _md("official")),
+    DocumentRecord("COM-REIMB", "Finance may request additional receipts during reimbursement review.", "Reimbursement FAQ", "community-wiki", _md("community")),
+    DocumentRecord("LEG-TRAVEL", "All corporate travel should be booked through TravelPro.", "Corporate travel booking portal", "legacy-cache", _md("stale")),
+    DocumentRecord("OFF-TRAVEL", "Approved business travel must be booked through TripDesk.", "Business travel booking procedure", "official-finance", _md("official")),
+    DocumentRecord("COM-TRAVEL", "Direct airline booking can cause reimbursement delays.", "Travel booking discussion", "community-wiki", _md("community")),
+    DocumentRecord("LEG-PROC", "Purchases above INR 25,000 require procurement review.", "Procurement review threshold", "legacy-cache", _md("stale")),
+    DocumentRecord("OFF-PROC", "Procurement review is mandatory for purchases above INR 50,000.", "Procurement control PC-12", "official-finance", _md("official")),
+    DocumentRecord("COM-PROC", "Some categories require procurement review regardless of value.", "Procurement FAQ", "community-wiki", _md("community")),
+    DocumentRecord("OFF-LAB-A", "The graduate AI lab has moved to room B-204.", "AI lab room notice A", "official-campus", _md("official_equal")),
+    DocumentRecord("OFF-LAB-B", "The graduate AI lab has moved to room C-101.", "AI lab room notice B", "official-campus", _md("official_equal")),
+    DocumentRecord("OFF-OFFICE-A", "Professor Sen holds office hours on Tuesday at 3 PM.", "Office hours notice A", "official-campus", _md("official_equal")),
+    DocumentRecord("OFF-OFFICE-B", "Professor Sen holds office hours on Wednesday at 4 PM.", "Office hours notice B", "official-campus", _md("official_equal")),
+]
+
+CONFLICT_CASES = [
+    {"id":"q1","q":"What is the current HR portal for leave requests and payslips?","gold":"PeopleHub","doc":"OFF-HR-PORTAL","abstain":False},
+    {"id":"q2","q":"How many paid leave days do employees get per year under the current policy?","gold":"18 paid leave days","doc":"OFF-LEAVE","abstain":False},
+    {"id":"q3","q":"How many remote work days per week are currently allowed with manager approval?","gold":"two days per week","doc":"OFF-REMOTE","abstain":False},
+    {"id":"q4","q":"By which working day are salary slips published each month?","gold":"fifth working day","doc":"OFF-PAYSLIP","abstain":False},
+    {"id":"q5","q":"Which VPN protocol is required by the current corporate security baseline?","gold":"WireGuard","doc":"OFF-VPN","abstain":False},
+    {"id":"q6","q":"What is the production database backup retention period?","gold":"30 days","doc":"OFF-BACKUP","abstain":False},
+    {"id":"q7","q":"What is the current public API rate limit per minute per token?","gold":"120 requests per minute","doc":"OFF-RATE","abstain":False},
+    {"id":"q8","q":"Which cloud region is primary for production?","gold":"ap-south-1","doc":"OFF-REGION","abstain":False},
+    {"id":"q9","q":"What is the domestic business travel meal expense cap per day?","gold":"INR 4,500 per day","doc":"OFF-EXPENSE","abstain":False},
+    {"id":"q10","q":"Within how many days must expense reimbursement claims be submitted?","gold":"15 days","doc":"OFF-REIMB","abstain":False},
+    {"id":"q11","q":"Which portal must be used to book approved business travel?","gold":"TripDesk","doc":"OFF-TRAVEL","abstain":False},
+    {"id":"q12","q":"Above what amount is procurement review mandatory?","gold":"INR 50,000","doc":"OFF-PROC","abstain":False},
+    {"id":"q13","q":"What is the company dental insurance annual limit?","gold":None,"doc":None,"abstain":True},
+    {"id":"q14","q":"What is the cafeteria breakfast price?","gold":None,"doc":None,"abstain":True},
+    {"id":"q15","q":"What is the emergency satellite phone number?","gold":None,"doc":None,"abstain":True},
+    {"id":"q16","q":"What is the annual gym membership reimbursement?","gold":None,"doc":None,"abstain":True},
+    {"id":"q17","q":"Which room is the graduate AI lab currently in?","gold":None,"doc":None,"abstain":True},
+    {"id":"q18","q":"When are Professor Sen's office hours?","gold":None,"doc":None,"abstain":True},
+]
+
+CLEAN_DOCS = [
+    DocumentRecord("C1", "The observatory opens to visitors at 9 AM every Saturday.", "Observatory hours", "handbook", _md("official")),
+    DocumentRecord("C2", "The chemistry safety briefing is held in Hall 7.", "Chemistry briefing", "handbook", _md("official")),
+    DocumentRecord("C3", "Library books may be borrowed for 21 days.", "Library borrowing", "handbook", _md("official")),
+    DocumentRecord("C4", "The robotics club meets on Thursday evenings.", "Robotics club", "handbook", _md("official")),
+    DocumentRecord("C5", "Visitor parking is located beside Gate 3.", "Visitor parking", "handbook", _md("official")),
+    DocumentRecord("C6", "The campus shuttle begins service at 6:30 AM.", "Campus shuttle", "handbook", _md("official")),
+    DocumentRecord("C7", "The data science seminar carries two academic credits.", "Data science seminar", "handbook", _md("official")),
+    DocumentRecord("C8", "The emergency assembly point is the North Lawn.", "Emergency procedure", "handbook", _md("official")),
+]
+
+CLEAN_CASES = [
+    {"id":"c1","q":"When does the observatory open on Saturday?","gold":"9 AM","doc":"C1","abstain":False},
+    {"id":"c2","q":"Where is the chemistry safety briefing held?","gold":"Hall 7","doc":"C2","abstain":False},
+    {"id":"c3","q":"How long may library books be borrowed?","gold":"21 days","doc":"C3","abstain":False},
+    {"id":"c4","q":"When does the robotics club meet?","gold":"Thursday evenings","doc":"C4","abstain":False},
+    {"id":"c5","q":"Where is visitor parking?","gold":"Gate 3","doc":"C5","abstain":False},
+    {"id":"c6","q":"What time does the campus shuttle begin service?","gold":"6:30 AM","doc":"C6","abstain":False},
+    {"id":"c7","q":"How many credits is the data science seminar worth?","gold":"two academic credits","doc":"C7","abstain":False},
+    {"id":"c8","q":"Where is the emergency assembly point?","gold":"North Lawn","doc":"C8","abstain":False},
+]
+
+
+def _eval(name: str, docs: list[DocumentRecord], cases: list[dict[str, Any]], bayesian: bool) -> tuple[dict[str, Any], list[dict[str, Any]]]:
+    gen = ExtractiveGroundedGenerator()
+    if bayesian:
+        searcher = BayesianIndex(docs, BayesianRetrievalConfig(candidate_pool=10, likelihood_temperature=2.0))
+    else:
+        searcher = PlainTfidfIndex(docs)
+    rows: list[dict[str, Any]] = []
+    rr: list[float] = []
+    r1: list[float] = []
+    r3: list[float] = []
+    briers: list[float] = []
+    doc_map = {d.id: d for d in docs}
+    for c in cases:
+        hits = searcher.search(c["q"], k=4)
+        top = hits[0] if hits else None
+        if bayesian:
+            abstain = searcher.should_abstain(hits)
+        else:
+            abstain = (not hits) or top.retrieval_score < 0.06
+        if abstain:
+            answer, citations = gen.abstain_text, []
+        else:
+            answer, citations = gen.generate(c["q"], hits)
+        if c["abstain"]:
+            correct = answer == gen.abstain_text
+        else:
+            correct = str(c["gold"]).lower() in answer.lower()
+            ids = [h.document.id for h in hits]
+            rank = ids.index(c["doc"]) + 1 if c["doc"] in ids else 0
+            r1.append(1.0 if rank == 1 else 0.0)
+            r3.append(1.0 if 0 < rank <= 3 else 0.0)
+            rr.append(1.0 / rank if rank else 0.0)
+            p = top.posterior if top else 0.0
+            y = 1.0 if top and top.document.id == c["doc"] else 0.0
+            briers.append((p - y) ** 2)
+        grounded = True
+        if citations:
+            sentence = answer.rsplit(" [", 1)[0]
+            grounded = sentence in doc_map[citations[0]].text
+        rows.append({
+            "id": c["id"],
+            "query": c["q"],
+            "expected_abstain": c["abstain"],
+            "top_doc": None if top is None else top.document.id,
+            "retrieval_score": None if top is None else round(top.retrieval_score, 6),
+            "top_posterior": None if top is None else round(top.posterior, 6),
+            "answer": answer,
+            "citations": citations,
+            "correct": correct,
+            "grounded": grounded,
+        })
+    n_answerable = sum(not c["abstain"] for c in cases)
+    n_abstain = sum(c["abstain"] for c in cases)
+    summary = {
+        "system": name,
+        "n_cases": len(cases),
+        "answerable_cases": n_answerable,
+        "abstention_cases": n_abstain,
+        "recall_at_1": sum(r1) / len(r1) if r1 else None,
+        "recall_at_3": sum(r3) / len(r3) if r3 else None,
+        "mrr": sum(rr) / len(rr) if rr else None,
+        "end_to_end_accuracy": sum(r["correct"] for r in rows) / len(rows),
+        "grounding_accuracy": sum(r["grounded"] for r in rows) / len(rows),
+        "abstention_accuracy": (sum(r["correct"] for r, c in zip(rows, cases) if c["abstain"]) / n_abstain) if n_abstain else None,
+        "answer_accuracy_on_answerable": (sum(r["correct"] for r, c in zip(rows, cases) if not c["abstain"]) / n_answerable) if n_answerable else None,
+        "top_posterior_brier": sum(briers) / len(briers) if briers else None,
+    }
+    return summary, rows
+
+
+def run_benchmark(output_path: str | Path | None = None) -> dict[str, Any]:
+    clean_b, clean_b_rows = _eval("bayesian", CLEAN_DOCS, CLEAN_CASES, True)
+    clean_t, clean_t_rows = _eval("plain_tfidf", CLEAN_DOCS, CLEAN_CASES, False)
+    conflict_b, conflict_b_rows = _eval("bayesian", CONFLICT_DOCS, CONFLICT_CASES, True)
+    conflict_t, conflict_t_rows = _eval("plain_tfidf", CONFLICT_DOCS, CONFLICT_CASES, False)
+    result = {
+        "benchmark": "controlled_rag_v2",
+        "methodology": (
+            "Two end-to-end suites. CLEAN checks ordinary retrieval+grounded answer synthesis. "
+            "CONFLICT adds stale authoritative conflicts, missing evidence, and unresolved equal-authority contradictions. "
+            "The generator is extractive and citation-grounded so the evaluation isolates retrieval and uncertainty behavior without an external LLM confound."
+        ),
+        "clean": {"bayesian": clean_b, "baseline": clean_t, "bayesian_cases": clean_b_rows, "baseline_cases": clean_t_rows},
+        "conflict": {"bayesian": conflict_b, "baseline": conflict_t, "bayesian_cases": conflict_b_rows, "baseline_cases": conflict_t_rows},
+    }
+    if output_path is not None:
+        Path(output_path).write_text(json.dumps(result, indent=2), encoding="utf-8")
+    return result
+
+
+if __name__ == "__main__":
+    print(json.dumps(run_benchmark(), indent=2))

examples/quickstart.py

@@ -0,0 +1,7 @@
+from bayesian_langgraph import BayesianIndex, BayesianRAGGraph, DocumentRecord
+
+docs = [
+    DocumentRecord("official", "The current HR portal is PeopleHub.", "Current policy", "official", {"reliability_alpha":18,"reliability_beta":2,"authority":0.98,"freshness":1.0}),
+    DocumentRecord("legacy", "The HR portal is WorkSphere.", "Legacy guide", "cache", {"reliability_alpha":6,"reliability_beta":6,"authority":0.45,"freshness":0.18}),
+]
+print(BayesianRAGGraph(BayesianIndex(docs)).invoke("What is the current HR portal?"))

pyproject.toml

@@ -0,0 +1,31 @@
+[build-system]
+requires = ["setuptools>=68", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "bayesian-langgraph-rag"
+version = "1.0.0"
+description = "Uncertainty-aware Bayesian RAG implemented as a real LangGraph StateGraph"
+readme = "README.md"
+requires-python = ">=3.10"
+license = {text = "MIT"}
+authors = [{name = "Angshul Majumdar"}]
+dependencies = [
+  "numpy>=1.24",
+  "scikit-learn>=1.3",
+  "langchain-core>=1.4.7,<2.0",
+  "langgraph>=1.2,<2.0",
+]
+
+[project.optional-dependencies]
+dev = ["pytest>=8.0"]
+
+[tool.setuptools]
+package-dir = {"" = "src"}
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[tool.pytest.ini_options]
+pythonpath = ["src"]
+testpaths = ["tests"]

requirements.txt

@@ -0,0 +1,4 @@
+numpy>=1.24
+scikit-learn>=1.3
+langchain-core>=1.4.7,<2.0
+langgraph>=1.2,<2.0

src/bayesian_langgraph/__init__.py

@@ -0,0 +1,21 @@
+from .core import (
+    BayesianIndex,
+    BayesianRetrievalConfig,
+    DocumentRecord,
+    RetrievalHit,
+    ExtractiveGroundedGenerator,
+)
+
+__all__ = [
+    "BayesianIndex",
+    "BayesianRetrievalConfig",
+    "DocumentRecord",
+    "RetrievalHit",
+    "ExtractiveGroundedGenerator",
+]
+
+try:
+    from .graph import BayesianRAGGraph, RAGState
+    __all__ += ["BayesianRAGGraph", "RAGState"]
+except ImportError:
+    pass

src/bayesian_langgraph/core.py

@@ -0,0 +1,255 @@
+from __future__ import annotations
+
+from dataclasses import dataclass, field
+from typing import Any, Iterable, Mapping, Sequence
+import math
+import re
+
+import numpy as np
+from sklearn.feature_extraction.text import TfidfVectorizer, ENGLISH_STOP_WORDS
+
+
+def _clip01(x: float, eps: float = 1e-6) -> float:
+    return min(max(float(x), eps), 1.0)
+
+
+def _logsumexp(xs: Sequence[float]) -> float:
+    if not xs:
+        return float('-inf')
+    finite = [x for x in xs if math.isfinite(x)]
+    if not finite:
+        return float('-inf')
+    m = max(finite)
+    return m + math.log(sum(math.exp(x - m) for x in finite))
+
+
+@dataclass(frozen=True)
+class DocumentRecord:
+    id: str
+    text: str
+    title: str = ""
+    source: str = "unknown"
+    metadata: Mapping[str, Any] = field(default_factory=dict)
+
+    @property
+    def retrieval_text(self) -> str:
+        return (self.title + ". " + self.text).strip(". ")
+
+    def source_prior(self) -> float:
+        md = self.metadata
+        a = max(float(md.get("reliability_alpha", 1.0)), 1e-6)
+        b = max(float(md.get("reliability_beta", 1.0)), 1e-6)
+        reliability = a / (a + b)
+        authority = _clip01(md.get("authority", 0.5))
+        freshness = _clip01(md.get("freshness", 0.5))
+        return reliability * authority * freshness
+
+
+@dataclass(frozen=True)
+class RetrievalHit:
+    document: DocumentRecord
+    retrieval_score: float
+    log_prior: float
+    log_likelihood: float
+    log_posterior_unnormalized: float
+    posterior: float
+    query_coverage: float
+    rank: int
+
+    def as_dict(self) -> dict[str, Any]:
+        return {
+            "id": self.document.id,
+            "title": self.document.title,
+            "text": self.document.text,
+            "source": self.document.source,
+            "metadata": dict(self.document.metadata),
+            "retrieval_score": self.retrieval_score,
+            "log_prior": self.log_prior,
+            "log_likelihood": self.log_likelihood,
+            "posterior": self.posterior,
+            "query_coverage": self.query_coverage,
+            "rank": self.rank,
+        }
+
+
+@dataclass(frozen=True)
+class BayesianRetrievalConfig:
+    candidate_pool: int = 12
+    likelihood_temperature: float = 2.0
+    likelihood_floor: float = 0.02
+    reliability_weight: float = 1.0
+    authority_weight: float = 1.0
+    freshness_weight: float = 0.8
+    prior_floor: float = 1e-4
+    min_relevance: float = 0.06
+    min_top_posterior: float = 0.30
+    min_query_coverage: float = 0.30
+    min_dominance_ratio: float = 1.20
+
+
+class BayesianIndex:
+    """Generalized-Bayesian reranker over TF-IDF retrieval candidates.
+
+    Candidate document hypotheses are scored by
+        p(d|q) ∝ p(d) exp(tau * sim(q,d)).
+    p(d) is induced by a Beta source-reliability mean plus authority/freshness
+    metadata. The exponential similarity term is a Gibbs/generalized-Bayes
+    likelihood, not claimed to be a literal language-model likelihood.
+    """
+
+    def __init__(self, documents: Sequence[DocumentRecord], config: BayesianRetrievalConfig | None = None):
+        if not documents:
+            raise ValueError("documents must be non-empty")
+        self.documents = list(documents)
+        self.config = config or BayesianRetrievalConfig()
+        self.vectorizer = TfidfVectorizer(
+            lowercase=True,
+            stop_words="english",
+            ngram_range=(1, 2),
+            sublinear_tf=True,
+            norm="l2",
+        )
+        self.matrix = self.vectorizer.fit_transform([d.retrieval_text for d in self.documents])
+
+    def _log_prior(self, d: DocumentRecord) -> float:
+        md = d.metadata
+        a = max(float(md.get("reliability_alpha", 1.0)), 1e-6)
+        b = max(float(md.get("reliability_beta", 1.0)), 1e-6)
+        rel = _clip01(a / (a + b))
+        auth = _clip01(md.get("authority", 0.5))
+        fresh = _clip01(md.get("freshness", 0.5))
+        cfg = self.config
+        prior = max(
+            cfg.prior_floor,
+            (rel ** cfg.reliability_weight)
+            * (auth ** cfg.authority_weight)
+            * (fresh ** cfg.freshness_weight),
+        )
+        return math.log(prior)
+
+    def lexical_scores(self, query: str) -> np.ndarray:
+        q = self.vectorizer.transform([str(query)])
+        # TF-IDF vectors are L2 normalized, so dot product is cosine similarity.
+        return (self.matrix @ q.T).toarray().ravel()
+
+    @staticmethod
+    def _query_coverage(query: str, text: str) -> float:
+        qterms = {t for t in re.findall(r"[A-Za-z0-9_-]+", query.lower()) if len(t) > 1 and t not in ENGLISH_STOP_WORDS}
+        if not qterms:
+            return 0.0
+        dterms = set(re.findall(r"[A-Za-z0-9_-]+", text.lower()))
+        return len(qterms & dterms) / len(qterms)
+
+    def search(self, query: str, k: int = 4) -> list[RetrievalHit]:
+        if k <= 0:
+            return []
+        scores = self.lexical_scores(query)
+        pool_n = min(max(k, self.config.candidate_pool), len(self.documents))
+        # Stable sort makes benchmark results reproducible under ties.
+        cand = np.argsort(-scores, kind="stable")[:pool_n]
+        logu: list[float] = []
+        rows: list[tuple[int, float, float, float]] = []
+        for i in cand:
+            d = self.documents[int(i)]
+            s = max(float(scores[int(i)]), 0.0)
+            lp = self._log_prior(d)
+            ll = self.config.likelihood_temperature * math.log(s + self.config.likelihood_floor)
+            u = lp + ll
+            rows.append((int(i), s, lp, ll))
+            logu.append(u)
+        lse = _logsumexp(logu)
+        probs = [math.exp(u - lse) if math.isfinite(lse) else 1.0 / len(logu) for u in logu]
+        ranked = sorted(
+            zip(rows, probs),
+            key=lambda rp: (rp[1], rp[0][1]),
+            reverse=True,
+        )
+        hits: list[RetrievalHit] = []
+        for rank, ((i, s, lp, ll), pr) in enumerate(ranked[:k], start=1):
+            hits.append(RetrievalHit(
+                document=self.documents[i],
+                retrieval_score=s,
+                log_prior=lp,
+                log_likelihood=ll,
+                log_posterior_unnormalized=lp + ll,
+                posterior=float(pr),
+                query_coverage=self._query_coverage(query, self.documents[i].retrieval_text),
+                rank=rank,
+            ))
+        return hits
+
+    def should_abstain(self, hits: Sequence[RetrievalHit]) -> bool:
+        if not hits:
+            return True
+        top = hits[0]
+        if (
+            top.retrieval_score < self.config.min_relevance
+            or top.posterior < self.config.min_top_posterior
+            or top.query_coverage < self.config.min_query_coverage
+        ):
+            return True
+        if len(hits) > 1:
+            second = hits[1]
+            dominance = top.posterior / max(second.posterior, 1e-12)
+            if dominance < self.config.min_dominance_ratio:
+                return True
+        return False
+
+
+class ExtractiveGroundedGenerator:
+    """Dependency-free grounded answer synthesizer for tests and offline use.
+
+    It selects the query-relevant sentence from the highest-posterior evidence
+    and returns that evidence sentence verbatim with its document id. It never
+    invents content not present in the retrieved context.
+    """
+
+    abstain_text = "I do not know based on the provided documents."
+
+    @staticmethod
+    def _sentences(text: str) -> list[str]:
+        parts = re.split(r"(?<=[.!?])\s+|\n+", text.strip())
+        return [p.strip() for p in parts if p.strip()]
+
+    def generate(self, question: str, hits: Sequence[RetrievalHit]) -> tuple[str, list[str]]:
+        if not hits:
+            return self.abstain_text, []
+        # The posterior ranking has already selected the evidence hypothesis.
+        top = hits[0]
+        sents = self._sentences(top.document.text)
+        if not sents:
+            return self.abstain_text, []
+        if len(sents) == 1:
+            sentence = sents[0]
+        else:
+            v = TfidfVectorizer(lowercase=True, stop_words="english", ngram_range=(1, 2))
+            mat = v.fit_transform([question] + sents)
+            q = mat[0]
+            sims = (mat[1:] @ q.T).toarray().ravel()
+            sentence = sents[int(np.argmax(sims))]
+        return f"{sentence} [{top.document.id}]", [top.document.id]
+
+
+class PlainTfidfIndex:
+    """Conventional lexical baseline used by the end-to-end RAG benchmark."""
+    def __init__(self, documents: Sequence[DocumentRecord]):
+        self.documents = list(documents)
+        self.vectorizer = TfidfVectorizer(lowercase=True, stop_words="english", ngram_range=(1, 2), sublinear_tf=True, norm="l2")
+        self.matrix = self.vectorizer.fit_transform([d.retrieval_text for d in self.documents])
+
+    @staticmethod
+    def _query_coverage(query: str, text: str) -> float:
+        qterms = {t for t in re.findall(r"[A-Za-z0-9_-]+", query.lower()) if len(t) > 1 and t not in ENGLISH_STOP_WORDS}
+        if not qterms:
+            return 0.0
+        dterms = set(re.findall(r"[A-Za-z0-9_-]+", text.lower()))
+        return len(qterms & dterms) / len(qterms)
+
+    def search(self, query: str, k: int = 4) -> list[RetrievalHit]:
+        q = self.vectorizer.transform([query])
+        scores = (self.matrix @ q.T).toarray().ravel()
+        order = np.argsort(-scores, kind="stable")[:k]
+        # Baseline assigns normalized positive retrieval mass only for reporting.
+        vals = [max(float(scores[int(i)]), 0.0) for i in order]
+        z = sum(vals) or 1.0
+        return [RetrievalHit(self.documents[int(i)], float(scores[int(i)]), 0.0, 0.0, 0.0, vals[j]/z, 0.0, j+1) for j, i in enumerate(order)]

src/bayesian_langgraph/graph.py

@@ -0,0 +1,96 @@
+from __future__ import annotations
+
+from typing import Any, Literal, TypedDict
+
+from .core import BayesianIndex, ExtractiveGroundedGenerator, RetrievalHit
+
+
+class RAGState(TypedDict, total=False):
+    query: str
+    hits: list[RetrievalHit]
+    decision: Literal["generate", "abstain"]
+    answer: str
+    citations: list[str]
+    abstained: bool
+
+
+class BayesianRAGGraph:
+    """Actual LangGraph orchestration around the Bayesian retriever.
+
+    Graph topology:
+        START -> retrieve -> assess -> {generate | abstain} -> END
+    """
+
+    def __init__(self, index: BayesianIndex, generator: Any | None = None, k: int = 4):
+        self.index = index
+        self.generator = generator or ExtractiveGroundedGenerator()
+        self.k = k
+        self.graph = self._build_graph()
+
+    def _retrieve(self, state: RAGState) -> dict[str, Any]:
+        return {"hits": self.index.search(state["query"], k=self.k)}
+
+    def _assess(self, state: RAGState) -> dict[str, Any]:
+        decision: Literal["generate", "abstain"] = "abstain" if self.index.should_abstain(state.get("hits", [])) else "generate"
+        return {"decision": decision}
+
+    @staticmethod
+    def _route(state: RAGState) -> str:
+        return state.get("decision", "abstain")
+
+    def _generate(self, state: RAGState) -> dict[str, Any]:
+        hits = state.get("hits", [])
+        if hasattr(self.generator, "generate"):
+            answer, citations = self.generator.generate(state["query"], hits)
+        else:
+            context = "\n".join(f"[{h.document.id}] {h.document.text}" for h in hits)
+            prompt = (
+                "Answer only from the supplied context. Cite document ids in square brackets. "
+                "If unsupported, say you do not know.\n\n"
+                f"Context:\n{context}\n\nQuestion: {state['query']}"
+            )
+            resp = self.generator.invoke(prompt)
+            answer = getattr(resp, "content", str(resp))
+            citations = [h.document.id for h in hits if f"[{h.document.id}]" in answer]
+        return {"answer": answer, "citations": citations, "abstained": False}
+
+    @staticmethod
+    def _abstain(state: RAGState) -> dict[str, Any]:
+        return {
+            "answer": ExtractiveGroundedGenerator.abstain_text,
+            "citations": [],
+            "abstained": True,
+        }
+
+    def _build_graph(self):
+        try:
+            from langgraph.graph import StateGraph, START, END
+        except ImportError as exc:
+            raise ImportError("bayesian-langgraph-rag requires langgraph>=1.2,<2.0") from exc
+
+        builder = StateGraph(RAGState)
+        builder.add_node("retrieve", self._retrieve)
+        builder.add_node("assess", self._assess)
+        builder.add_node("generate", self._generate)
+        builder.add_node("abstain", self._abstain)
+        builder.add_edge(START, "retrieve")
+        builder.add_edge("retrieve", "assess")
+        builder.add_conditional_edges(
+            "assess",
+            self._route,
+            {"generate": "generate", "abstain": "abstain"},
+        )
+        builder.add_edge("generate", END)
+        builder.add_edge("abstain", END)
+        return builder.compile()
+
+    def invoke(self, query: str) -> dict[str, Any]:
+        state = self.graph.invoke({"query": query})
+        hits = state.get("hits", [])
+        return {
+            "question": query,
+            "answer": state.get("answer", ExtractiveGroundedGenerator.abstain_text),
+            "citations": state.get("citations", []),
+            "abstained": state.get("abstained", True),
+            "hits": [h.as_dict() for h in hits],
+        }

src/bayesian_langgraph/smc.py

@@ -0,0 +1,129 @@
+from __future__ import annotations
+
+from dataclasses import dataclass, field
+from typing import Any, Callable, Generic, Sequence, TypeVar
+import math
+import random
+
+S = TypeVar("S")
+A = TypeVar("A")
+
+
+def normalize_logweights(logw: Sequence[float]) -> list[float]:
+    if not logw:
+        return []
+    finite = [w for w in logw if math.isfinite(w)]
+    if not finite:
+        return [1.0 / len(logw)] * len(logw)
+    m = max(finite)
+    ws = [math.exp(w - m) if math.isfinite(w) else 0.0 for w in logw]
+    z = sum(ws)
+    if not math.isfinite(z) or z <= 0.0:
+        return [1.0 / len(logw)] * len(logw)
+    return [w / z for w in ws]
+
+
+def effective_sample_size(probs: Sequence[float]) -> float:
+    s2 = sum(float(p) ** 2 for p in probs)
+    return 0.0 if s2 <= 0.0 else 1.0 / s2
+
+
+def systematic_resample(probs: Sequence[float], rng: random.Random) -> list[int]:
+    n = len(probs)
+    if n == 0:
+        return []
+    cdf: list[float] = []
+    total = 0.0
+    for p in probs:
+        total += float(p)
+        cdf.append(total)
+    if total <= 0.0 or not math.isfinite(total):
+        return list(range(n))
+    cdf = [x / total for x in cdf]
+    cdf[-1] = 1.0
+    u0 = rng.random() / n
+    out: list[int] = []
+    j = 0
+    for i in range(n):
+        u = u0 + i / n
+        while j < n - 1 and u > cdf[j]:
+            j += 1
+        out.append(j)
+    return out
+
+
+@dataclass(frozen=True)
+class Proposal(Generic[A]):
+    action: A
+    probability: float
+
+
+@dataclass
+class Particle(Generic[S]):
+    state: S
+    logw: float = 0.0
+    history: list[Any] = field(default_factory=list)
+
+
+@dataclass(frozen=True)
+class SMCConfig:
+    n_particles: int = 32
+    n_steps: int = 3
+    ess_resample_fraction: float = 0.5
+    seed: int = 0
+
+
+class ParticleFilter(Generic[S, A]):
+    """Importance-corrected SMC runtime.
+
+    `propose(state, t)` returns a discrete proposal distribution q(a|state).
+    `transition(state, action, t)` produces the next state.
+    `log_target_increment(prev, action, nxt, t)` supplies the incremental
+    log target density. The importance update is log w += log target - log q.
+    Resampling is never performed after the final weighting step, so the final
+    posterior weights are preserved.
+    """
+
+    def __init__(self, config: SMCConfig | None = None):
+        self.config = config or SMCConfig()
+
+    def run(
+        self,
+        initial_state: S,
+        propose: Callable[[S, int], Sequence[Proposal[A]]],
+        transition: Callable[[S, A, int], S],
+        log_target_increment: Callable[[S, A, S, int], float],
+    ) -> list[Particle[S]]:
+        cfg = self.config
+        rng = random.Random(cfg.seed)
+        particles = [Particle(state=initial_state) for _ in range(cfg.n_particles)]
+        for t in range(cfg.n_steps):
+            nxt: list[Particle[S]] = []
+            for p in particles:
+                proposals = [x for x in propose(p.state, t) if x.probability > 0.0]
+                if not proposals:
+                    raise ValueError("proposal distribution is empty")
+                z = sum(x.probability for x in proposals)
+                probs = [x.probability / z for x in proposals]
+                r = rng.random()
+                acc = 0.0
+                j = len(proposals) - 1
+                for i, pr in enumerate(probs):
+                    acc += pr
+                    if r <= acc:
+                        j = i
+                        break
+                selected = proposals[j]
+                q = probs[j]
+                new_state = transition(p.state, selected.action, t)
+                inc = float(log_target_increment(p.state, selected.action, new_state, t))
+                nxt.append(Particle(new_state, p.logw + inc - math.log(q), p.history + [selected.action]))
+            particles = nxt
+            probs_now = normalize_logweights([p.logw for p in particles])
+            if t < cfg.n_steps - 1 and effective_sample_size(probs_now) < cfg.ess_resample_fraction * len(particles):
+                ids = systematic_resample(probs_now, rng)
+                particles = [Particle(particles[i].state, 0.0, list(particles[i].history)) for i in ids]
+        final_probs = normalize_logweights([p.logw for p in particles])
+        for p, pr in zip(particles, final_probs):
+            setattr(p, "posterior", pr)
+        return particles

tests/test_core.py

@@ -0,0 +1,16 @@
+from bayesian_langgraph.core import BayesianIndex, DocumentRecord
+from bayesian_langgraph.smc import normalize_logweights
+
+
+def test_all_negative_infinity_weights_become_uniform():
+    p = normalize_logweights([float('-inf'), float('-inf')])
+    assert p == [0.5, 0.5]
+
+
+def test_authoritative_current_document_beats_stale_high_overlap():
+    docs = [
+        DocumentRecord('stale','The current HR portal is WorkSphere.','Current HR portal','cache',{'reliability_alpha':6,'reliability_beta':6,'authority':0.45,'freshness':0.18}),
+        DocumentRecord('official','Leave requests and payslips are now handled in PeopleHub.','Migration complete','official',{'reliability_alpha':18,'reliability_beta':2,'authority':0.98,'freshness':1.0}),
+    ]
+    hits = BayesianIndex(docs).search('What is the current HR portal for leave requests and payslips?', k=2)
+    assert hits[0].document.id == 'official'

tests/test_framework_rag_e2e.py

@@ -0,0 +1,8 @@
+from evaluation.framework_rag_benchmark import run_framework_benchmark
+
+
+def test_real_langgraph_stategraph_rag_suite():
+    result = run_framework_benchmark()
+    assert result["clean"]["passed"] == result["clean"]["n_cases"] == 8
+    assert result["conflict"]["passed"] == result["conflict"]["n_cases"] == 18
+    assert {"retrieve", "assess", "generate", "abstain"}.issubset(result["stategraph_nodes"])

tests/test_langgraph_integration.py

@@ -0,0 +1,31 @@
+from langchain_core.runnables import RunnableLambda
+
+from bayesian_langgraph import BayesianIndex, BayesianRAGGraph, DocumentRecord
+
+
+def _docs():
+    return [
+        DocumentRecord(
+            'd1',
+            'The access code is ORBIT-7.',
+            'Access',
+            'official',
+            {'reliability_alpha':10,'reliability_beta':1,'authority':0.99,'freshness':1.0},
+        )
+    ]
+
+
+def test_actual_stategraph_executes_rag():
+    rag = BayesianRAGGraph(BayesianIndex(_docs()))
+    out = rag.invoke('What is the access code?')
+    assert 'ORBIT-7' in out['answer']
+    assert out['citations'] == ['d1']
+
+
+def test_stategraph_model_runnable_generation_branch():
+    generator = RunnableLambda(lambda prompt: 'The access code is ORBIT-7. [d1]')
+    rag = BayesianRAGGraph(BayesianIndex(_docs()), generator=generator, k=1)
+    out = rag.invoke('What is the access code?')
+    assert out['answer'] == 'The access code is ORBIT-7. [d1]'
+    assert out['citations'] == ['d1']
+    assert not out['abstained']

tests/test_rag_e2e.py

@@ -0,0 +1,9 @@
+from evaluation.rag_benchmark import run_benchmark
+
+
+def test_end_to_end_rag_benchmark():
+    x = run_benchmark()
+    assert x['clean']['bayesian']['end_to_end_accuracy'] == 1.0
+    assert x['conflict']['bayesian']['recall_at_1'] == 1.0
+    assert x['conflict']['bayesian']['abstention_accuracy'] == 1.0
+    assert x['conflict']['bayesian']['grounding_accuracy'] == 1.0

tests/test_smc_math.py

@@ -0,0 +1,52 @@
+import math
+
+
+def _imports():
+    try:
+        from bayesian_langchain.smc import ParticleFilter, Proposal, SMCConfig
+    except ImportError:
+        from bayesian_langgraph.smc import ParticleFilter, Proposal, SMCConfig
+    return ParticleFilter, Proposal, SMCConfig
+
+
+def test_final_step_preserves_nonuniform_posterior_weights():
+    ParticleFilter, Proposal, SMCConfig = _imports()
+    pf = ParticleFilter(SMCConfig(n_particles=64, n_steps=1, ess_resample_fraction=0.99, seed=7))
+
+    def propose(state, t):
+        return [Proposal("good", 0.5), Proposal("bad", 0.5)]
+
+    def transition(state, action, t):
+        return action
+
+    def log_target(prev, action, nxt, t):
+        return 0.0 if action == "good" else math.log(1e-5)
+
+    particles = pf.run(None, propose, transition, log_target)
+    probs = [p.posterior for p in particles]
+    assert abs(sum(probs) - 1.0) < 1e-12
+    # If the old final-resampling bug returned, these would all be 1/N.
+    assert max(probs) - min(probs) > 1e-3
+
+
+def test_importance_weight_contains_inverse_proposal_probability():
+    ParticleFilter, Proposal, SMCConfig = _imports()
+    pf = ParticleFilter(SMCConfig(n_particles=256, n_steps=1, seed=11))
+
+    def propose(state, t):
+        return [Proposal("a", 0.9), Proposal("b", 0.1)]
+
+    def transition(state, action, t):
+        return action
+
+    # Equal target mass for both actions. Importance correction should cancel
+    # the 9:1 proposal imbalance in the weighted posterior, up to MC error.
+    def log_target(prev, action, nxt, t):
+        return math.log(0.5)
+
+    particles = pf.run(None, propose, transition, log_target)
+    mass = {"a": 0.0, "b": 0.0}
+    for p in particles:
+        mass[p.state] += p.posterior
+    assert abs(mass["a"] - 0.5) < 0.15
+    assert abs(mass["b"] - 0.5) < 0.15