Upload 22 files

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

MIGRATION_NOTES.md

@@ -0,0 +1,3 @@
+# Migration notes
+
+The source material contained two different concepts under the same Bayesian/Probabilistic LangChain umbrella: an SMC tool runtime and a heuristic beam framework (`p_langchain`). The beam framework was not retained as Bayesian code because its softmax score was not a posterior and its multi-proposer executor overwrote earlier proposers. The useful SMC idea is retained in `smc.py` with proposal correction and safe final weighting. The Colab RAG export was replaced by a package-native RAG implementation and reproducible benchmark.

README.md

@@ -1,3 +1,113 @@
 ---
-license: fair-noncommercial-research-license
+license: mit
+tags:
+- langchain
+- rag
+- bayesian
+- probabilistic
+- uncertainty
+- retrieval
 ---
+
+# Bayesian LangChain RAG
+
+A corrected, Hugging Face-ready implementation of uncertainty-aware retrieval augmented generation built on **LangChain Core**.
+
+This repository replaces the earlier heuristic "tool succeeded = high likelihood" and beam-style scoring with an explicit generalized-Bayesian retrieval posterior. For a query `q` and candidate document `d`,
+
+`p(d | q) ∝ p(d) (similarity(q,d) + eps)^tau`
+
+where the prior `p(d)` is built from a Beta source-reliability mean, source authority, and freshness metadata. The similarity term is deliberately described as a **generalized-Bayes/Gibbs likelihood**, not as a literal language-model likelihood.
+
+## What was corrected
+
+- Removed notebook/Colab shell commands and hard-coded paths from library code.
+- Removed the non-Bayesian `p_langchain` beam scorer from the Bayesian runtime.
+- Replaced deterministic identical-particle behavior with an importance-corrected SMC utility for workflows that need particles.
+- Added proposal-density correction `log target - log q` to SMC weights.
+- Fixed all-`-inf` log-weight normalization.
+- Preserved final posterior weights by forbidding final-step resampling.
+- Added a real `BaseRetriever` implementation and LangChain `Runnable` entry point.
+- RAG now reasons over retrieved evidence quality rather than rewarding any successful tool call.
+- Added explicit abstention for missing evidence and unresolved near-equal conflicts.
+- Added citation-grounded answer synthesis and a reproducible end-to-end RAG benchmark.
+
+## Install
+
+```bash
+pip install -e .
+```
+
+No GPU is required. The default evaluation generator is extractive and grounded. Any LangChain-compatible model can be supplied for generative answering.
+
+## Quick start
+
+```python
+from bayesian_langchain import DocumentRecord, BayesianIndex, BayesianRAG
+
+docs = [
+    DocumentRecord(
+        id="official",
+        title="Current portal policy",
+        text="The current HR portal is PeopleHub.",
+        source="official",
+        metadata={"reliability_alpha": 18, "reliability_beta": 2, "authority": 0.98, "freshness": 1.0},
+    ),
+    DocumentRecord(
+        id="legacy",
+        title="HR portal guide",
+        text="The HR portal is WorkSphere.",
+        source="legacy-cache",
+        metadata={"reliability_alpha": 6, "reliability_beta": 6, "authority": 0.45, "freshness": 0.18},
+    ),
+]
+
+rag = BayesianRAG(BayesianIndex(docs))
+print(rag.invoke("What is the current HR portal?"))
+```
+
+`BayesianRAG.as_runnable()` returns a standard LangChain `RunnableLambda`.
+
+## RAG validation
+
+Run:
+
+```bash
+PYTHONPATH=src:. python evaluation/rag_benchmark.py
+```
+
+The committed result is in `artifacts/rag_benchmark_results.json`.
+
+| Suite | System | Recall@1 | MRR | Answer accuracy | Abstention accuracy | End-to-end accuracy |
+|---|---|---:|---:|---:|---:|---:|
+| Clean RAG (8 QA) | Bayesian | 1.000 | 1.000 | 1.000 | n/a | 1.000 |
+| Clean RAG (8 QA) | Plain TF-IDF | 1.000 | 1.000 | 1.000 | n/a | 1.000 |
+| Conflict/missing RAG (18 QA) | Bayesian | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 |
+| Conflict/missing RAG (18 QA) | Plain TF-IDF | 0.250 | 0.611 | 0.250 | 0.333 | 0.278 |
+
+The conflict suite deliberately contains stale high-overlap documents, current authoritative documents, questions with no supporting evidence, and equal-authority contradictions. The results are **controlled repository tests**, not a claim of universal RAG accuracy.
+
+### Framework-level LangChain RAG test
+
+The repository also contains `evaluation/framework_rag_benchmark.py`, which routes every case through the real LangChain path:
+
+`BayesianRetriever(BaseRetriever) -> BayesianRAG.as_runnable() -> grounded generation/abstention`.
+
+It was validated against **langchain-core 1.6.11**. Results: **8/8 clean cases** and **18/18 conflict/missing-evidence cases** passed end-to-end. The committed output is `artifacts/framework_rag_benchmark_results.json`.
+
+Run it with:
+
+```bash
+PYTHONPATH=src:. python evaluation/framework_rag_benchmark.py
+```
+
+## Layout
+
+- `src/bayesian_langchain/core.py` — posterior retrieval, abstention, grounded answerer
+- `src/bayesian_langchain/retriever.py` — LangChain `BaseRetriever`
+- `src/bayesian_langchain/rag.py` — end-to-end LangChain RAG and Runnable
+- `src/bayesian_langchain/smc.py` — corrected importance-weighted SMC utility
+- `evaluation/rag_benchmark.py` — controlled retrieval/grounding benchmark
+- `evaluation/framework_rag_benchmark.py` — real LangChain `BaseRetriever`/Runnable RAG benchmark
+- `tests/` — regression and integration tests
+- `VALIDATION.md` — exact framework/version validation record

VALIDATION.md

@@ -0,0 +1,12 @@
+# Validation
+
+Validated on Python 3.13.5 against the supplied `langchain-core 1.6.11` source archive.
+
+- Repository tests: 8 passed.
+- Framework RAG suite: 8/8 clean cases passed.
+- Framework RAG suite: 18/18 stale/conflicting/missing-evidence cases passed.
+- Package wheel build: passed.
+- `BayesianRetriever` is a real `langchain_core.retrievers.BaseRetriever` subclass.
+- End-to-end evaluation invokes the retriever through LangChain and the RAG system through `RunnableLambda`; retrieval is not recomputed outside the LangChain result path.
+
+The sandbox cannot resolve PyPI. For framework-source validation only, a test harness supplied minimal compatibility modules for unavailable peripheral dependencies such as LangSmith tracing. Those test shims are not part of this repository and do not replace the RAG implementation.

artifacts/framework_rag_benchmark_results.json

@@ -0,0 +1,266 @@
+{
+  "framework": "langchain-core",
+  "execution": "BayesianRetriever(BaseRetriever) -> BayesianRAG RunnableLambda -> grounded answer/abstention",
+  "clean": {
+    "n_cases": 8,
+    "passed": 8,
+    "accuracy": 1.0,
+    "rows": [
+      {
+        "id": "c1",
+        "ok": true,
+        "answer": "The observatory opens to visitors at 9 AM every Saturday. [C1]",
+        "citations": [
+          "C1"
+        ],
+        "abstained": false,
+        "top_doc": "C1"
+      },
+      {
+        "id": "c2",
+        "ok": true,
+        "answer": "The chemistry safety briefing is held in Hall 7. [C2]",
+        "citations": [
+          "C2"
+        ],
+        "abstained": false,
+        "top_doc": "C2"
+      },
+      {
+        "id": "c3",
+        "ok": true,
+        "answer": "Library books may be borrowed for 21 days. [C3]",
+        "citations": [
+          "C3"
+        ],
+        "abstained": false,
+        "top_doc": "C3"
+      },
+      {
+        "id": "c4",
+        "ok": true,
+        "answer": "The robotics club meets on Thursday evenings. [C4]",
+        "citations": [
+          "C4"
+        ],
+        "abstained": false,
+        "top_doc": "C4"
+      },
+      {
+        "id": "c5",
+        "ok": true,
+        "answer": "Visitor parking is located beside Gate 3. [C5]",
+        "citations": [
+          "C5"
+        ],
+        "abstained": false,
+        "top_doc": "C5"
+      },
+      {
+        "id": "c6",
+        "ok": true,
+        "answer": "The campus shuttle begins service at 6:30 AM. [C6]",
+        "citations": [
+          "C6"
+        ],
+        "abstained": false,
+        "top_doc": "C6"
+      },
+      {
+        "id": "c7",
+        "ok": true,
+        "answer": "The data science seminar carries two academic credits. [C7]",
+        "citations": [
+          "C7"
+        ],
+        "abstained": false,
+        "top_doc": "C7"
+      },
+      {
+        "id": "c8",
+        "ok": true,
+        "answer": "The emergency assembly point is the North Lawn. [C8]",
+        "citations": [
+          "C8"
+        ],
+        "abstained": false,
+        "top_doc": "C8"
+      }
+    ]
+  },
+  "conflict": {
+    "n_cases": 18,
+    "passed": 18,
+    "accuracy": 1.0,
+    "rows": [
+      {
+        "id": "q1",
+        "ok": true,
+        "answer": "All employee self-service tasks, including leave requests and payslips, are now handled in PeopleHub. [OFF-HR-PORTAL]",
+        "citations": [
+          "OFF-HR-PORTAL"
+        ],
+        "abstained": false,
+        "top_doc": "OFF-HR-PORTAL"
+      },
+      {
+        "id": "q2",
+        "ok": true,
+        "answer": "The 2026 employee benefits policy grants 18 paid leave days per calendar year. [OFF-LEAVE]",
+        "citations": [
+          "OFF-LEAVE"
+        ],
+        "abstained": false,
+        "top_doc": "OFF-LEAVE"
+      },
+      {
+        "id": "q3",
+        "ok": true,
+        "answer": "Under the active flexible-work policy, employees may work remotely up to two days per week with manager approval. [OFF-REMOTE]",
+        "citations": [
+          "OFF-REMOTE"
+        ],
+        "abstained": false,
+        "top_doc": "OFF-REMOTE"
+      },
+      {
+        "id": "q4",
+        "ok": true,
+        "answer": "Payroll publishes salary slips by the fifth working day of each month. [OFF-PAYSLIP]",
+        "citations": [
+          "OFF-PAYSLIP"
+        ],
+        "abstained": false,
+        "top_doc": "OFF-PAYSLIP"
+      },
+      {
+        "id": "q5",
+        "ok": true,
+        "answer": "The security baseline now requires WireGuard for the standard corporate VPN profile. [OFF-VPN]",
+        "citations": [
+          "OFF-VPN"
+        ],
+        "abstained": false,
+        "top_doc": "OFF-VPN"
+      },
+      {
+        "id": "q6",
+        "ok": true,
+        "answer": "Production database backups have a retention period of 30 days. [OFF-BACKUP]",
+        "citations": [
+          "OFF-BACKUP"
+        ],
+        "abstained": false,
+        "top_doc": "OFF-BACKUP"
+      },
+      {
+        "id": "q7",
+        "ok": true,
+        "answer": "The current gateway quota permits 120 requests per minute per token. [OFF-RATE]",
+        "citations": [
+          "OFF-RATE"
+        ],
+        "abstained": false,
+        "top_doc": "OFF-RATE"
+      },
+      {
+        "id": "q8",
+        "ok": true,
+        "answer": "The production primary region is ap-south-1; disaster recovery remains in a separate region. [OFF-REGION]",
+        "citations": [
+          "OFF-REGION"
+        ],
+        "abstained": false,
+        "top_doc": "OFF-REGION"
+      },
+      {
+        "id": "q9",
+        "ok": true,
+        "answer": "For domestic business travel, reimbursable meals are capped at INR 4,500 per day. [OFF-EXPENSE]",
+        "citations": [
+          "OFF-EXPENSE"
+        ],
+        "abstained": false,
+        "top_doc": "OFF-EXPENSE"
+      },
+      {
+        "id": "q10",
+        "ok": true,
+        "answer": "Expense claims must be submitted within 15 days of the transaction date. [OFF-REIMB]",
+        "citations": [
+          "OFF-REIMB"
+        ],
+        "abstained": false,
+        "top_doc": "OFF-REIMB"
+      },
+      {
+        "id": "q11",
+        "ok": true,
+        "answer": "Approved business travel must be booked through TripDesk. [OFF-TRAVEL]",
+        "citations": [
+          "OFF-TRAVEL"
+        ],
+        "abstained": false,
+        "top_doc": "OFF-TRAVEL"
+      },
+      {
+        "id": "q12",
+        "ok": true,
+        "answer": "Procurement review is mandatory for purchases above INR 50,000. [OFF-PROC]",
+        "citations": [
+          "OFF-PROC"
+        ],
+        "abstained": false,
+        "top_doc": "OFF-PROC"
+      },
+      {
+        "id": "q13",
+        "ok": true,
+        "answer": "I do not know based on the provided documents.",
+        "citations": [],
+        "abstained": true,
+        "top_doc": "LEG-RATE"
+      },
+      {
+        "id": "q14",
+        "ok": true,
+        "answer": "I do not know based on the provided documents.",
+        "citations": [],
+        "abstained": true,
+        "top_doc": "OFF-HR-PORTAL"
+      },
+      {
+        "id": "q15",
+        "ok": true,
+        "answer": "I do not know based on the provided documents.",
+        "citations": [],
+        "abstained": true,
+        "top_doc": "OFF-HR-PORTAL"
+      },
+      {
+        "id": "q16",
+        "ok": true,
+        "answer": "I do not know based on the provided documents.",
+        "citations": [],
+        "abstained": true,
+        "top_doc": "OFF-REIMB"
+      },
+      {
+        "id": "q17",
+        "ok": true,
+        "answer": "I do not know based on the provided documents.",
+        "citations": [],
+        "abstained": true,
+        "top_doc": "OFF-LAB-A"
+      },
+      {
+        "id": "q18",
+        "ok": true,
+        "answer": "I do not know based on the provided documents.",
+        "citations": [],
+        "abstained": true,
+        "top_doc": "OFF-OFFICE-A"
+      }
+    ]
+  }
+}

artifacts/rag_benchmark_results.json

@@ -0,0 +1,784 @@
+{
+  "benchmark": "controlled_rag_v2",
+  "methodology": "Two end-to-end suites. CLEAN checks ordinary retrieval+grounded answer synthesis. CONFLICT adds stale authoritative conflicts, missing evidence, and unresolved equal-authority contradictions. The generator is extractive and citation-grounded so the evaluation isolates retrieval and uncertainty behavior without an external LLM confound.",
+  "clean": {
+    "bayesian": {
+      "system": "bayesian",
+      "n_cases": 8,
+      "answerable_cases": 8,
+      "abstention_cases": 0,
+      "recall_at_1": 1.0,
+      "recall_at_3": 1.0,
+      "mrr": 1.0,
+      "end_to_end_accuracy": 1.0,
+      "grounding_accuracy": 1.0,
+      "abstention_accuracy": null,
+      "answer_accuracy_on_answerable": 1.0,
+      "top_posterior_brier": 2.8128197163138293e-05
+    },
+    "baseline": {
+      "system": "plain_tfidf",
+      "n_cases": 8,
+      "answerable_cases": 8,
+      "abstention_cases": 0,
+      "recall_at_1": 1.0,
+      "recall_at_3": 1.0,
+      "mrr": 1.0,
+      "end_to_end_accuracy": 1.0,
+      "grounding_accuracy": 1.0,
+      "abstention_accuracy": null,
+      "answer_accuracy_on_answerable": 1.0,
+      "top_posterior_brier": 0.0
+    },
+    "bayesian_cases": [
+      {
+        "id": "c1",
+        "query": "When does the observatory open on Saturday?",
+        "expected_abstain": false,
+        "top_doc": "C1",
+        "retrieval_score": 0.552814,
+        "top_posterior": 0.991539,
+        "answer": "The observatory opens to visitors at 9 AM every Saturday. [C1]",
+        "citations": [
+          "C1"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "c2",
+        "query": "Where is the chemistry safety briefing held?",
+        "expected_abstain": false,
+        "top_doc": "C2",
+        "retrieval_score": 0.825787,
+        "top_posterior": 0.996101,
+        "answer": "The chemistry safety briefing is held in Hall 7. [C2]",
+        "citations": [
+          "C2"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "c3",
+        "query": "How long may library books be borrowed?",
+        "expected_abstain": false,
+        "top_doc": "C3",
+        "retrieval_score": 0.683723,
+        "top_posterior": 0.994378,
+        "answer": "Library books may be borrowed for 21 days. [C3]",
+        "citations": [
+          "C3"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "c4",
+        "query": "When does the robotics club meet?",
+        "expected_abstain": false,
+        "top_doc": "C4",
+        "retrieval_score": 0.742488,
+        "top_posterior": 0.995207,
+        "answer": "The robotics club meets on Thursday evenings. [C4]",
+        "citations": [
+          "C4"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "c5",
+        "query": "Where is visitor parking?",
+        "expected_abstain": false,
+        "top_doc": "C5",
+        "retrieval_score": 0.79521,
+        "top_posterior": 0.995804,
+        "answer": "Visitor parking is located beside Gate 3. [C5]",
+        "citations": [
+          "C5"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "c6",
+        "query": "What time does the campus shuttle begin service?",
+        "expected_abstain": false,
+        "top_doc": "C6",
+        "retrieval_score": 0.769605,
+        "top_posterior": 0.995529,
+        "answer": "The campus shuttle begins service at 6:30 AM. [C6]",
+        "citations": [
+          "C6"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "c7",
+        "query": "How many credits is the data science seminar worth?",
+        "expected_abstain": false,
+        "top_doc": "C7",
+        "retrieval_score": 0.836653,
+        "top_posterior": 0.996199,
+        "answer": "The data science seminar carries two academic credits. [C7]",
+        "citations": [
+          "C7"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "c8",
+        "query": "Where is the emergency assembly point?",
+        "expected_abstain": false,
+        "top_doc": "C8",
+        "retrieval_score": 0.683723,
+        "top_posterior": 0.994378,
+        "answer": "The emergency assembly point is the North Lawn. [C8]",
+        "citations": [
+          "C8"
+        ],
+        "correct": true,
+        "grounded": true
+      }
+    ],
+    "baseline_cases": [
+      {
+        "id": "c1",
+        "query": "When does the observatory open on Saturday?",
+        "expected_abstain": false,
+        "top_doc": "C1",
+        "retrieval_score": 0.552814,
+        "top_posterior": 1.0,
+        "answer": "The observatory opens to visitors at 9 AM every Saturday. [C1]",
+        "citations": [
+          "C1"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "c2",
+        "query": "Where is the chemistry safety briefing held?",
+        "expected_abstain": false,
+        "top_doc": "C2",
+        "retrieval_score": 0.825787,
+        "top_posterior": 1.0,
+        "answer": "The chemistry safety briefing is held in Hall 7. [C2]",
+        "citations": [
+          "C2"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "c3",
+        "query": "How long may library books be borrowed?",
+        "expected_abstain": false,
+        "top_doc": "C3",
+        "retrieval_score": 0.683723,
+        "top_posterior": 1.0,
+        "answer": "Library books may be borrowed for 21 days. [C3]",
+        "citations": [
+          "C3"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "c4",
+        "query": "When does the robotics club meet?",
+        "expected_abstain": false,
+        "top_doc": "C4",
+        "retrieval_score": 0.742488,
+        "top_posterior": 1.0,
+        "answer": "The robotics club meets on Thursday evenings. [C4]",
+        "citations": [
+          "C4"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "c5",
+        "query": "Where is visitor parking?",
+        "expected_abstain": false,
+        "top_doc": "C5",
+        "retrieval_score": 0.79521,
+        "top_posterior": 1.0,
+        "answer": "Visitor parking is located beside Gate 3. [C5]",
+        "citations": [
+          "C5"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "c6",
+        "query": "What time does the campus shuttle begin service?",
+        "expected_abstain": false,
+        "top_doc": "C6",
+        "retrieval_score": 0.769605,
+        "top_posterior": 1.0,
+        "answer": "The campus shuttle begins service at 6:30 AM. [C6]",
+        "citations": [
+          "C6"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "c7",
+        "query": "How many credits is the data science seminar worth?",
+        "expected_abstain": false,
+        "top_doc": "C7",
+        "retrieval_score": 0.836653,
+        "top_posterior": 1.0,
+        "answer": "The data science seminar carries two academic credits. [C7]",
+        "citations": [
+          "C7"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "c8",
+        "query": "Where is the emergency assembly point?",
+        "expected_abstain": false,
+        "top_doc": "C8",
+        "retrieval_score": 0.683723,
+        "top_posterior": 1.0,
+        "answer": "The emergency assembly point is the North Lawn. [C8]",
+        "citations": [
+          "C8"
+        ],
+        "correct": true,
+        "grounded": true
+      }
+    ]
+  },
+  "conflict": {
+    "bayesian": {
+      "system": "bayesian",
+      "n_cases": 18,
+      "answerable_cases": 12,
+      "abstention_cases": 6,
+      "recall_at_1": 1.0,
+      "recall_at_3": 1.0,
+      "mrr": 1.0,
+      "end_to_end_accuracy": 1.0,
+      "grounding_accuracy": 1.0,
+      "abstention_accuracy": 1.0,
+      "answer_accuracy_on_answerable": 1.0,
+      "top_posterior_brier": 0.09485010369519668
+    },
+    "baseline": {
+      "system": "plain_tfidf",
+      "n_cases": 18,
+      "answerable_cases": 12,
+      "abstention_cases": 6,
+      "recall_at_1": 0.25,
+      "recall_at_3": 1.0,
+      "mrr": 0.611111111111111,
+      "end_to_end_accuracy": 0.2777777777777778,
+      "grounding_accuracy": 1.0,
+      "abstention_accuracy": 0.3333333333333333,
+      "answer_accuracy_on_answerable": 0.25,
+      "top_posterior_brier": 0.26943925109630584
+    },
+    "bayesian_cases": [
+      {
+        "id": "q1",
+        "query": "What is the current HR portal for leave requests and payslips?",
+        "expected_abstain": false,
+        "top_doc": "OFF-HR-PORTAL",
+        "retrieval_score": 0.221707,
+        "top_posterior": 0.419857,
+        "answer": "All employee self-service tasks, including leave requests and payslips, are now handled in PeopleHub. [OFF-HR-PORTAL]",
+        "citations": [
+          "OFF-HR-PORTAL"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "q2",
+        "query": "How many paid leave days do employees get per year under the current policy?",
+        "expected_abstain": false,
+        "top_doc": "OFF-LEAVE",
+        "retrieval_score": 0.366908,
+        "top_posterior": 0.675297,
+        "answer": "The 2026 employee benefits policy grants 18 paid leave days per calendar year. [OFF-LEAVE]",
+        "citations": [
+          "OFF-LEAVE"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "q3",
+        "query": "How many remote work days per week are currently allowed with manager approval?",
+        "expected_abstain": false,
+        "top_doc": "OFF-REMOTE",
+        "retrieval_score": 0.383162,
+        "top_posterior": 0.598358,
+        "answer": "Under the active flexible-work policy, employees may work remotely up to two days per week with manager approval. [OFF-REMOTE]",
+        "citations": [
+          "OFF-REMOTE"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "q4",
+        "query": "By which working day are salary slips published each month?",
+        "expected_abstain": false,
+        "top_doc": "OFF-PAYSLIP",
+        "retrieval_score": 0.457118,
+        "top_posterior": 0.8609,
+        "answer": "Payroll publishes salary slips by the fifth working day of each month. [OFF-PAYSLIP]",
+        "citations": [
+          "OFF-PAYSLIP"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "q5",
+        "query": "Which VPN protocol is required by the current corporate security baseline?",
+        "expected_abstain": false,
+        "top_doc": "OFF-VPN",
+        "retrieval_score": 0.514507,
+        "top_posterior": 0.904463,
+        "answer": "The security baseline now requires WireGuard for the standard corporate VPN profile. [OFF-VPN]",
+        "citations": [
+          "OFF-VPN"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "q6",
+        "query": "What is the production database backup retention period?",
+        "expected_abstain": false,
+        "top_doc": "OFF-BACKUP",
+        "retrieval_score": 0.424384,
+        "top_posterior": 0.835659,
+        "answer": "Production database backups have a retention period of 30 days. [OFF-BACKUP]",
+        "citations": [
+          "OFF-BACKUP"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "q7",
+        "query": "What is the current public API rate limit per minute per token?",
+        "expected_abstain": false,
+        "top_doc": "OFF-RATE",
+        "retrieval_score": 0.260415,
+        "top_posterior": 0.600144,
+        "answer": "The current gateway quota permits 120 requests per minute per token. [OFF-RATE]",
+        "citations": [
+          "OFF-RATE"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "q8",
+        "query": "Which cloud region is primary for production?",
+        "expected_abstain": false,
+        "top_doc": "OFF-REGION",
+        "retrieval_score": 0.400734,
+        "top_posterior": 0.819994,
+        "answer": "The production primary region is ap-south-1; disaster recovery remains in a separate region. [OFF-REGION]",
+        "citations": [
+          "OFF-REGION"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "q9",
+        "query": "What is the domestic business travel meal expense cap per day?",
+        "expected_abstain": false,
+        "top_doc": "OFF-EXPENSE",
+        "retrieval_score": 0.391575,
+        "top_posterior": 0.520438,
+        "answer": "For domestic business travel, reimbursable meals are capped at INR 4,500 per day. [OFF-EXPENSE]",
+        "citations": [
+          "OFF-EXPENSE"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "q10",
+        "query": "Within how many days must expense reimbursement claims be submitted?",
+        "expected_abstain": false,
+        "top_doc": "OFF-REIMB",
+        "retrieval_score": 0.513156,
+        "top_posterior": 0.907517,
+        "answer": "Expense claims must be submitted within 15 days of the transaction date. [OFF-REIMB]",
+        "citations": [
+          "OFF-REIMB"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "q11",
+        "query": "Which portal must be used to book approved business travel?",
+        "expected_abstain": false,
+        "top_doc": "OFF-TRAVEL",
+        "retrieval_score": 0.580663,
+        "top_posterior": 0.832603,
+        "answer": "Approved business travel must be booked through TripDesk. [OFF-TRAVEL]",
+        "citations": [
+          "OFF-TRAVEL"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "q12",
+        "query": "Above what amount is procurement review mandatory?",
+        "expected_abstain": false,
+        "top_doc": "OFF-PROC",
+        "retrieval_score": 0.512226,
+        "top_posterior": 0.856608,
+        "answer": "Procurement review is mandatory for purchases above INR 50,000. [OFF-PROC]",
+        "citations": [
+          "OFF-PROC"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "q13",
+        "query": "What is the company dental insurance annual limit?",
+        "expected_abstain": true,
+        "top_doc": "LEG-RATE",
+        "retrieval_score": 0.325883,
+        "top_posterior": 0.829873,
+        "answer": "I do not know based on the provided documents.",
+        "citations": [],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "q14",
+        "query": "What is the cafeteria breakfast price?",
+        "expected_abstain": true,
+        "top_doc": "OFF-HR-PORTAL",
+        "retrieval_score": 0.0,
+        "top_posterior": 0.248018,
+        "answer": "I do not know based on the provided documents.",
+        "citations": [],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "q15",
+        "query": "What is the emergency satellite phone number?",
+        "expected_abstain": true,
+        "top_doc": "OFF-HR-PORTAL",
+        "retrieval_score": 0.0,
+        "top_posterior": 0.248018,
+        "answer": "I do not know based on the provided documents.",
+        "citations": [],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "q16",
+        "query": "What is the annual gym membership reimbursement?",
+        "expected_abstain": true,
+        "top_doc": "OFF-REIMB",
+        "retrieval_score": 0.178672,
+        "top_posterior": 0.385158,
+        "answer": "I do not know based on the provided documents.",
+        "citations": [],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "q17",
+        "query": "Which room is the graduate AI lab currently in?",
+        "expected_abstain": true,
+        "top_doc": "OFF-LAB-A",
+        "retrieval_score": 0.752053,
+        "top_posterior": 0.499378,
+        "answer": "I do not know based on the provided documents.",
+        "citations": [],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "q18",
+        "query": "When are Professor Sen's office hours?",
+        "expected_abstain": true,
+        "top_doc": "OFF-OFFICE-A",
+        "retrieval_score": 0.702454,
+        "top_posterior": 0.49929,
+        "answer": "I do not know based on the provided documents.",
+        "citations": [],
+        "correct": true,
+        "grounded": true
+      }
+    ],
+    "baseline_cases": [
+      {
+        "id": "q1",
+        "query": "What is the current HR portal for leave requests and payslips?",
+        "expected_abstain": false,
+        "top_doc": "LEG-HR-PORTAL",
+        "retrieval_score": 0.807312,
+        "top_posterior": 0.592469,
+        "answer": "The current HR portal for leave requests and payslips is WorkSphere. [LEG-HR-PORTAL]",
+        "citations": [
+          "LEG-HR-PORTAL"
+        ],
+        "correct": false,
+        "grounded": true
+      },
+      {
+        "id": "q2",
+        "query": "How many paid leave days do employees get per year under the current policy?",
+        "expected_abstain": false,
+        "top_doc": "LEG-LEAVE",
+        "retrieval_score": 0.602126,
+        "top_posterior": 0.462903,
+        "answer": "Employees receive 15 paid leave days per year under the leave policy. [LEG-LEAVE]",
+        "citations": [
+          "LEG-LEAVE"
+        ],
+        "correct": false,
+        "grounded": true
+      },
+      {
+        "id": "q3",
+        "query": "How many remote work days per week are currently allowed with manager approval?",
+        "expected_abstain": false,
+        "top_doc": "LEG-REMOTE",
+        "retrieval_score": 0.699429,
+        "top_posterior": 0.435096,
+        "answer": "The current remote work policy permits three remote days per week with manager approval. [LEG-REMOTE]",
+        "citations": [
+          "LEG-REMOTE"
+        ],
+        "correct": false,
+        "grounded": true
+      },
+      {
+        "id": "q4",
+        "query": "By which working day are salary slips published each month?",
+        "expected_abstain": false,
+        "top_doc": "LEG-PAYSLIP",
+        "retrieval_score": 0.672029,
+        "top_posterior": 0.550648,
+        "answer": "Salary slips are published on the tenth working day of each month. [LEG-PAYSLIP]",
+        "citations": [
+          "LEG-PAYSLIP"
+        ],
+        "correct": false,
+        "grounded": true
+      },
+      {
+        "id": "q5",
+        "query": "Which VPN protocol is required by the current corporate security baseline?",
+        "expected_abstain": false,
+        "top_doc": "LEG-VPN",
+        "retrieval_score": 0.524301,
+        "top_posterior": 0.420244,
+        "answer": "The supported corporate VPN protocol is OpenVPN. [LEG-VPN]",
+        "citations": [
+          "LEG-VPN"
+        ],
+        "correct": false,
+        "grounded": true
+      },
+      {
+        "id": "q6",
+        "query": "What is the production database backup retention period?",
+        "expected_abstain": false,
+        "top_doc": "LEG-BACKUP",
+        "retrieval_score": 0.556814,
+        "top_posterior": 0.467937,
+        "answer": "Production database backups are retained for 14 days. [LEG-BACKUP]",
+        "citations": [
+          "LEG-BACKUP"
+        ],
+        "correct": false,
+        "grounded": true
+      },
+      {
+        "id": "q7",
+        "query": "What is the current public API rate limit per minute per token?",
+        "expected_abstain": false,
+        "top_doc": "LEG-RATE",
+        "retrieval_score": 0.845222,
+        "top_posterior": 0.672178,
+        "answer": "The public API rate limit is 60 requests per minute per token. [LEG-RATE]",
+        "citations": [
+          "LEG-RATE"
+        ],
+        "correct": false,
+        "grounded": true
+      },
+      {
+        "id": "q8",
+        "query": "Which cloud region is primary for production?",
+        "expected_abstain": false,
+        "top_doc": "LEG-REGION",
+        "retrieval_score": 0.542439,
+        "top_posterior": 0.460259,
+        "answer": "The primary deployment region is us-east-1. [LEG-REGION]",
+        "citations": [
+          "LEG-REGION"
+        ],
+        "correct": false,
+        "grounded": true
+      },
+      {
+        "id": "q9",
+        "query": "What is the domestic business travel meal expense cap per day?",
+        "expected_abstain": false,
+        "top_doc": "LEG-EXPENSE",
+        "retrieval_score": 0.671336,
+        "top_posterior": 0.451066,
+        "answer": "The meal expense cap is INR 3,000 per day on domestic travel. [LEG-EXPENSE]",
+        "citations": [
+          "LEG-EXPENSE"
+        ],
+        "correct": false,
+        "grounded": true
+      },
+      {
+        "id": "q10",
+        "query": "Within how many days must expense reimbursement claims be submitted?",
+        "expected_abstain": false,
+        "top_doc": "OFF-REIMB",
+        "retrieval_score": 0.513156,
+        "top_posterior": 0.45338,
+        "answer": "Expense claims must be submitted within 15 days of the transaction date. [OFF-REIMB]",
+        "citations": [
+          "OFF-REIMB"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "q11",
+        "query": "Which portal must be used to book approved business travel?",
+        "expected_abstain": false,
+        "top_doc": "OFF-TRAVEL",
+        "retrieval_score": 0.580663,
+        "top_posterior": 0.514293,
+        "answer": "Approved business travel must be booked through TripDesk. [OFF-TRAVEL]",
+        "citations": [
+          "OFF-TRAVEL"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "q12",
+        "query": "Above what amount is procurement review mandatory?",
+        "expected_abstain": false,
+        "top_doc": "OFF-PROC",
+        "retrieval_score": 0.512226,
+        "top_posterior": 0.38514,
+        "answer": "Procurement review is mandatory for purchases above INR 50,000. [OFF-PROC]",
+        "citations": [
+          "OFF-PROC"
+        ],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "q13",
+        "query": "What is the company dental insurance annual limit?",
+        "expected_abstain": true,
+        "top_doc": "LEG-RATE",
+        "retrieval_score": 0.325883,
+        "top_posterior": 1.0,
+        "answer": "The public API rate limit is 60 requests per minute per token. [LEG-RATE]",
+        "citations": [
+          "LEG-RATE"
+        ],
+        "correct": false,
+        "grounded": true
+      },
+      {
+        "id": "q14",
+        "query": "What is the cafeteria breakfast price?",
+        "expected_abstain": true,
+        "top_doc": "LEG-HR-PORTAL",
+        "retrieval_score": 0.0,
+        "top_posterior": 0.0,
+        "answer": "I do not know based on the provided documents.",
+        "citations": [],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "q15",
+        "query": "What is the emergency satellite phone number?",
+        "expected_abstain": true,
+        "top_doc": "LEG-HR-PORTAL",
+        "retrieval_score": 0.0,
+        "top_posterior": 0.0,
+        "answer": "I do not know based on the provided documents.",
+        "citations": [],
+        "correct": true,
+        "grounded": true
+      },
+      {
+        "id": "q16",
+        "query": "What is the annual gym membership reimbursement?",
+        "expected_abstain": true,
+        "top_doc": "COM-REIMB",
+        "retrieval_score": 0.333617,
+        "top_posterior": 0.327295,
+        "answer": "Finance may request additional receipts during reimbursement review. [COM-REIMB]",
+        "citations": [
+          "COM-REIMB"
+        ],
+        "correct": false,
+        "grounded": true
+      },
+      {
+        "id": "q17",
+        "query": "Which room is the graduate AI lab currently in?",
+        "expected_abstain": true,
+        "top_doc": "OFF-LAB-A",
+        "retrieval_score": 0.752053,
+        "top_posterior": 0.5,
+        "answer": "The graduate AI lab has moved to room B-204. [OFF-LAB-A]",
+        "citations": [
+          "OFF-LAB-A"
+        ],
+        "correct": false,
+        "grounded": true
+      },
+      {
+        "id": "q18",
+        "query": "When are Professor Sen's office hours?",
+        "expected_abstain": true,
+        "top_doc": "OFF-OFFICE-A",
+        "retrieval_score": 0.702454,
+        "top_posterior": 0.5,
+        "answer": "Professor Sen holds office hours on Tuesday at 3 PM. [OFF-OFFICE-A]",
+        "citations": [
+          "OFF-OFFICE-A"
+        ],
+        "correct": false,
+        "grounded": true
+      }
+    ]
+  }
+}

evaluation/__init__.py

@@ -0,0 +1 @@
+# Benchmark package.

evaluation/framework_rag_benchmark.py

@@ -0,0 +1,55 @@
+from __future__ import annotations
+
+import json
+from pathlib import Path
+from typing import Any
+
+from bayesian_langchain import BayesianIndex, BayesianRAG
+from evaluation.rag_benchmark import CLEAN_CASES, CLEAN_DOCS, CONFLICT_CASES, CONFLICT_DOCS
+
+
+def _case_ok(out: dict[str, Any], case: dict[str, Any]) -> bool:
+    if case["abstain"]:
+        return bool(out.get("abstained")) and not out.get("citations")
+    return (
+        not out.get("abstained")
+        and case["doc"] in out.get("citations", [])
+        and str(case["gold"]).lower() in str(out.get("answer", "")).lower()
+    )
+
+
+def _run_suite(docs, cases) -> dict[str, Any]:
+    rag = BayesianRAG(BayesianIndex(docs), k=4)
+    rows = []
+    for case in cases:
+        out = rag.as_runnable().invoke(case["q"])
+        rows.append({
+            "id": case["id"],
+            "ok": _case_ok(out, case),
+            "answer": out["answer"],
+            "citations": out["citations"],
+            "abstained": out["abstained"],
+            "top_doc": out["hits"][0]["id"] if out["hits"] else None,
+        })
+    return {
+        "n_cases": len(rows),
+        "passed": sum(r["ok"] for r in rows),
+        "accuracy": sum(r["ok"] for r in rows) / len(rows),
+        "rows": rows,
+    }
+
+
+def run_framework_benchmark(output_path: str | Path | None = None) -> dict[str, Any]:
+    result = {
+        "framework": "langchain-core",
+        "execution": "BayesianRetriever(BaseRetriever) -> BayesianRAG RunnableLambda -> grounded answer/abstention",
+        "clean": _run_suite(CLEAN_DOCS, CLEAN_CASES),
+        "conflict": _run_suite(CONFLICT_DOCS, CONFLICT_CASES),
+    }
+    if output_path is not None:
+        Path(output_path).write_text(json.dumps(result, indent=2), encoding="utf-8")
+    return result
+
+
+if __name__ == "__main__":
+    print(json.dumps(run_framework_benchmark(), indent=2))

evaluation/rag_benchmark.py

@@ -0,0 +1,206 @@
+from __future__ import annotations
+
+import json
+from pathlib import Path
+from typing import Any
+
+from bayesian_langchain.core import (
+    BayesianIndex,
+    BayesianRetrievalConfig,
+    DocumentRecord,
+    PlainTfidfIndex,
+    ExtractiveGroundedGenerator,
+)
+
+
+def _md(kind: str) -> dict[str, Any]:
+    if kind == "official":
+        return {"reliability_alpha": 18, "reliability_beta": 2, "authority": 0.98, "freshness": 1.0, "tier": kind}
+    if kind == "stale":
+        return {"reliability_alpha": 6, "reliability_beta": 6, "authority": 0.45, "freshness": 0.18, "tier": kind}
+    if kind == "community":
+        return {"reliability_alpha": 7, "reliability_beta": 5, "authority": 0.55, "freshness": 0.65, "tier": kind}
+    if kind == "official_equal":
+        return {"reliability_alpha": 18, "reliability_beta": 2, "authority": 0.98, "freshness": 1.0, "tier": "official"}
+    return {"reliability_alpha": 1, "reliability_beta": 1, "authority": 0.5, "freshness": 0.5, "tier": kind}
+
+
+CONFLICT_DOCS = [
+    DocumentRecord("LEG-HR-PORTAL", "The current HR portal for leave requests and payslips is WorkSphere.", "Current HR portal login guide", "legacy-cache", _md("stale")),
+    DocumentRecord("OFF-HR-PORTAL", "All employee self-service tasks, including leave requests and payslips, are now handled in PeopleHub.", "Employee self-service migration completed", "official-handbook", _md("official")),
+    DocumentRecord("COM-HR-PORTAL", "Some teams used WorkSphere during migration; ask HR if your bookmark no longer works.", "HR portal discussion", "community-wiki", _md("community")),
+    DocumentRecord("LEG-LEAVE", "Employees receive 15 paid leave days per year under the leave policy.", "Paid leave entitlement FAQ", "legacy-cache", _md("stale")),
+    DocumentRecord("OFF-LEAVE", "The 2026 employee benefits policy grants 18 paid leave days per calendar year.", "2026 employee benefits policy", "official-handbook", _md("official")),
+    DocumentRecord("COM-LEAVE", "Leave balances vary by category and should be checked in the HR system.", "Leave balance help", "community-wiki", _md("community")),
+    DocumentRecord("LEG-REMOTE", "The current remote work policy permits three remote days per week with manager approval.", "Current remote work days", "legacy-cache", _md("stale")),
+    DocumentRecord("OFF-REMOTE", "Under the active flexible-work policy, employees may work remotely up to two days per week with manager approval.", "Flexible-work policy revision", "official-handbook", _md("official")),
+    DocumentRecord("COM-REMOTE", "Remote work requires manager approval and may vary during special projects.", "Remote work tips", "community-wiki", _md("community")),
+    DocumentRecord("LEG-PAYSLIP", "Salary slips are published on the tenth working day of each month.", "Payslip publication date", "legacy-cache", _md("stale")),
+    DocumentRecord("OFF-PAYSLIP", "Payroll publishes salary slips by the fifth working day of each month.", "Payroll service standard", "official-handbook", _md("official")),
+    DocumentRecord("COM-PAYSLIP", "Payslip publication can be delayed around holidays.", "Payroll discussion", "community-wiki", _md("community")),
+    DocumentRecord("LEG-VPN", "The supported corporate VPN protocol is OpenVPN.", "Corporate VPN protocol", "legacy-cache", _md("stale")),
+    DocumentRecord("OFF-VPN", "The security baseline now requires WireGuard for the standard corporate VPN profile.", "Network security baseline", "official-it", _md("official")),
+    DocumentRecord("COM-VPN", "OpenVPN profiles may still exist on older laptops.", "VPN migration notes", "community-wiki", _md("community")),
+    DocumentRecord("LEG-BACKUP", "Production database backups are retained for 14 days.", "Database backup retention policy", "legacy-cache", _md("stale")),
+    DocumentRecord("OFF-BACKUP", "Production database backups have a retention period of 30 days.", "Disaster recovery control DR-7", "official-it", _md("official")),
+    DocumentRecord("COM-BACKUP", "Teams should not rely on local snapshots as their only backup.", "Backup practices", "community-wiki", _md("community")),
+    DocumentRecord("LEG-RATE", "The public API rate limit is 60 requests per minute per token.", "Public API rate limit", "legacy-cache", _md("stale")),
+    DocumentRecord("OFF-RATE", "The current gateway quota permits 120 requests per minute per token.", "API gateway operating limits", "official-it", _md("official")),
+    DocumentRecord("COM-RATE", "Burst traffic may still be throttled even below the nominal quota.", "API troubleshooting", "community-wiki", _md("community")),
+    DocumentRecord("LEG-REGION", "The primary deployment region is us-east-1.", "Primary cloud deployment region", "legacy-cache", _md("stale")),
+    DocumentRecord("OFF-REGION", "The production primary region is ap-south-1; disaster recovery remains in a separate region.", "Cloud production topology", "official-it", _md("official")),
+    DocumentRecord("COM-REGION", "Developers may use several regions for non-production experiments.", "Cloud developer notes", "community-wiki", _md("community")),
+    DocumentRecord("LEG-EXPENSE", "The meal expense cap is INR 3,000 per day on domestic travel.", "Domestic meal expense cap", "legacy-cache", _md("stale")),
+    DocumentRecord("OFF-EXPENSE", "For domestic business travel, reimbursable meals are capped at INR 4,500 per day.", "Travel and expense policy 2026", "official-finance", _md("official")),
+    DocumentRecord("COM-EXPENSE", "Itemized receipts are recommended for meal reimbursement.", "Expense claim tips", "community-wiki", _md("community")),
+    DocumentRecord("LEG-REIMB", "Employees must submit reimbursement claims within 30 days of the expense.", "Reimbursement submission deadline", "legacy-cache", _md("stale")),
+    DocumentRecord("OFF-REIMB", "Expense claims must be submitted within 15 days of the transaction date.", "Expense reimbursement standard", "official-finance", _md("official")),
+    DocumentRecord("COM-REIMB", "Finance may request additional receipts during reimbursement review.", "Reimbursement FAQ", "community-wiki", _md("community")),
+    DocumentRecord("LEG-TRAVEL", "All corporate travel should be booked through TravelPro.", "Corporate travel booking portal", "legacy-cache", _md("stale")),
+    DocumentRecord("OFF-TRAVEL", "Approved business travel must be booked through TripDesk.", "Business travel booking procedure", "official-finance", _md("official")),
+    DocumentRecord("COM-TRAVEL", "Direct airline booking can cause reimbursement delays.", "Travel booking discussion", "community-wiki", _md("community")),
+    DocumentRecord("LEG-PROC", "Purchases above INR 25,000 require procurement review.", "Procurement review threshold", "legacy-cache", _md("stale")),
+    DocumentRecord("OFF-PROC", "Procurement review is mandatory for purchases above INR 50,000.", "Procurement control PC-12", "official-finance", _md("official")),
+    DocumentRecord("COM-PROC", "Some categories require procurement review regardless of value.", "Procurement FAQ", "community-wiki", _md("community")),
+    DocumentRecord("OFF-LAB-A", "The graduate AI lab has moved to room B-204.", "AI lab room notice A", "official-campus", _md("official_equal")),
+    DocumentRecord("OFF-LAB-B", "The graduate AI lab has moved to room C-101.", "AI lab room notice B", "official-campus", _md("official_equal")),
+    DocumentRecord("OFF-OFFICE-A", "Professor Sen holds office hours on Tuesday at 3 PM.", "Office hours notice A", "official-campus", _md("official_equal")),
+    DocumentRecord("OFF-OFFICE-B", "Professor Sen holds office hours on Wednesday at 4 PM.", "Office hours notice B", "official-campus", _md("official_equal")),
+]
+
+CONFLICT_CASES = [
+    {"id":"q1","q":"What is the current HR portal for leave requests and payslips?","gold":"PeopleHub","doc":"OFF-HR-PORTAL","abstain":False},
+    {"id":"q2","q":"How many paid leave days do employees get per year under the current policy?","gold":"18 paid leave days","doc":"OFF-LEAVE","abstain":False},
+    {"id":"q3","q":"How many remote work days per week are currently allowed with manager approval?","gold":"two days per week","doc":"OFF-REMOTE","abstain":False},
+    {"id":"q4","q":"By which working day are salary slips published each month?","gold":"fifth working day","doc":"OFF-PAYSLIP","abstain":False},
+    {"id":"q5","q":"Which VPN protocol is required by the current corporate security baseline?","gold":"WireGuard","doc":"OFF-VPN","abstain":False},
+    {"id":"q6","q":"What is the production database backup retention period?","gold":"30 days","doc":"OFF-BACKUP","abstain":False},
+    {"id":"q7","q":"What is the current public API rate limit per minute per token?","gold":"120 requests per minute","doc":"OFF-RATE","abstain":False},
+    {"id":"q8","q":"Which cloud region is primary for production?","gold":"ap-south-1","doc":"OFF-REGION","abstain":False},
+    {"id":"q9","q":"What is the domestic business travel meal expense cap per day?","gold":"INR 4,500 per day","doc":"OFF-EXPENSE","abstain":False},
+    {"id":"q10","q":"Within how many days must expense reimbursement claims be submitted?","gold":"15 days","doc":"OFF-REIMB","abstain":False},
+    {"id":"q11","q":"Which portal must be used to book approved business travel?","gold":"TripDesk","doc":"OFF-TRAVEL","abstain":False},
+    {"id":"q12","q":"Above what amount is procurement review mandatory?","gold":"INR 50,000","doc":"OFF-PROC","abstain":False},
+    {"id":"q13","q":"What is the company dental insurance annual limit?","gold":None,"doc":None,"abstain":True},
+    {"id":"q14","q":"What is the cafeteria breakfast price?","gold":None,"doc":None,"abstain":True},
+    {"id":"q15","q":"What is the emergency satellite phone number?","gold":None,"doc":None,"abstain":True},
+    {"id":"q16","q":"What is the annual gym membership reimbursement?","gold":None,"doc":None,"abstain":True},
+    {"id":"q17","q":"Which room is the graduate AI lab currently in?","gold":None,"doc":None,"abstain":True},
+    {"id":"q18","q":"When are Professor Sen's office hours?","gold":None,"doc":None,"abstain":True},
+]
+
+CLEAN_DOCS = [
+    DocumentRecord("C1", "The observatory opens to visitors at 9 AM every Saturday.", "Observatory hours", "handbook", _md("official")),
+    DocumentRecord("C2", "The chemistry safety briefing is held in Hall 7.", "Chemistry briefing", "handbook", _md("official")),
+    DocumentRecord("C3", "Library books may be borrowed for 21 days.", "Library borrowing", "handbook", _md("official")),
+    DocumentRecord("C4", "The robotics club meets on Thursday evenings.", "Robotics club", "handbook", _md("official")),
+    DocumentRecord("C5", "Visitor parking is located beside Gate 3.", "Visitor parking", "handbook", _md("official")),
+    DocumentRecord("C6", "The campus shuttle begins service at 6:30 AM.", "Campus shuttle", "handbook", _md("official")),
+    DocumentRecord("C7", "The data science seminar carries two academic credits.", "Data science seminar", "handbook", _md("official")),
+    DocumentRecord("C8", "The emergency assembly point is the North Lawn.", "Emergency procedure", "handbook", _md("official")),
+]
+
+CLEAN_CASES = [
+    {"id":"c1","q":"When does the observatory open on Saturday?","gold":"9 AM","doc":"C1","abstain":False},
+    {"id":"c2","q":"Where is the chemistry safety briefing held?","gold":"Hall 7","doc":"C2","abstain":False},
+    {"id":"c3","q":"How long may library books be borrowed?","gold":"21 days","doc":"C3","abstain":False},
+    {"id":"c4","q":"When does the robotics club meet?","gold":"Thursday evenings","doc":"C4","abstain":False},
+    {"id":"c5","q":"Where is visitor parking?","gold":"Gate 3","doc":"C5","abstain":False},
+    {"id":"c6","q":"What time does the campus shuttle begin service?","gold":"6:30 AM","doc":"C6","abstain":False},
+    {"id":"c7","q":"How many credits is the data science seminar worth?","gold":"two academic credits","doc":"C7","abstain":False},
+    {"id":"c8","q":"Where is the emergency assembly point?","gold":"North Lawn","doc":"C8","abstain":False},
+]
+
+
+def _eval(name: str, docs: list[DocumentRecord], cases: list[dict[str, Any]], bayesian: bool) -> tuple[dict[str, Any], list[dict[str, Any]]]:
+    gen = ExtractiveGroundedGenerator()
+    if bayesian:
+        searcher = BayesianIndex(docs, BayesianRetrievalConfig(candidate_pool=10, likelihood_temperature=2.0))
+    else:
+        searcher = PlainTfidfIndex(docs)
+    rows: list[dict[str, Any]] = []
+    rr: list[float] = []
+    r1: list[float] = []
+    r3: list[float] = []
+    briers: list[float] = []
+    doc_map = {d.id: d for d in docs}
+    for c in cases:
+        hits = searcher.search(c["q"], k=4)
+        top = hits[0] if hits else None
+        if bayesian:
+            abstain = searcher.should_abstain(hits)
+        else:
+            abstain = (not hits) or top.retrieval_score < 0.06
+        if abstain:
+            answer, citations = gen.abstain_text, []
+        else:
+            answer, citations = gen.generate(c["q"], hits)
+        if c["abstain"]:
+            correct = answer == gen.abstain_text
+        else:
+            correct = str(c["gold"]).lower() in answer.lower()
+            ids = [h.document.id for h in hits]
+            rank = ids.index(c["doc"]) + 1 if c["doc"] in ids else 0
+            r1.append(1.0 if rank == 1 else 0.0)
+            r3.append(1.0 if 0 < rank <= 3 else 0.0)
+            rr.append(1.0 / rank if rank else 0.0)
+            p = top.posterior if top else 0.0
+            y = 1.0 if top and top.document.id == c["doc"] else 0.0
+            briers.append((p - y) ** 2)
+        grounded = True
+        if citations:
+            sentence = answer.rsplit(" [", 1)[0]
+            grounded = sentence in doc_map[citations[0]].text
+        rows.append({
+            "id": c["id"],
+            "query": c["q"],
+            "expected_abstain": c["abstain"],
+            "top_doc": None if top is None else top.document.id,
+            "retrieval_score": None if top is None else round(top.retrieval_score, 6),
+            "top_posterior": None if top is None else round(top.posterior, 6),
+            "answer": answer,
+            "citations": citations,
+            "correct": correct,
+            "grounded": grounded,
+        })
+    n_answerable = sum(not c["abstain"] for c in cases)
+    n_abstain = sum(c["abstain"] for c in cases)
+    summary = {
+        "system": name,
+        "n_cases": len(cases),
+        "answerable_cases": n_answerable,
+        "abstention_cases": n_abstain,
+        "recall_at_1": sum(r1) / len(r1) if r1 else None,
+        "recall_at_3": sum(r3) / len(r3) if r3 else None,
+        "mrr": sum(rr) / len(rr) if rr else None,
+        "end_to_end_accuracy": sum(r["correct"] for r in rows) / len(rows),
+        "grounding_accuracy": sum(r["grounded"] for r in rows) / len(rows),
+        "abstention_accuracy": (sum(r["correct"] for r, c in zip(rows, cases) if c["abstain"]) / n_abstain) if n_abstain else None,
+        "answer_accuracy_on_answerable": (sum(r["correct"] for r, c in zip(rows, cases) if not c["abstain"]) / n_answerable) if n_answerable else None,
+        "top_posterior_brier": sum(briers) / len(briers) if briers else None,
+    }
+    return summary, rows
+
+
+def run_benchmark(output_path: str | Path | None = None) -> dict[str, Any]:
+    clean_b, clean_b_rows = _eval("bayesian", CLEAN_DOCS, CLEAN_CASES, True)
+    clean_t, clean_t_rows = _eval("plain_tfidf", CLEAN_DOCS, CLEAN_CASES, False)
+    conflict_b, conflict_b_rows = _eval("bayesian", CONFLICT_DOCS, CONFLICT_CASES, True)
+    conflict_t, conflict_t_rows = _eval("plain_tfidf", CONFLICT_DOCS, CONFLICT_CASES, False)
+    result = {
+        "benchmark": "controlled_rag_v2",
+        "methodology": (
+            "Two end-to-end suites. CLEAN checks ordinary retrieval+grounded answer synthesis. "
+            "CONFLICT adds stale authoritative conflicts, missing evidence, and unresolved equal-authority contradictions. "
+            "The generator is extractive and citation-grounded so the evaluation isolates retrieval and uncertainty behavior without an external LLM confound."
+        ),
+        "clean": {"bayesian": clean_b, "baseline": clean_t, "bayesian_cases": clean_b_rows, "baseline_cases": clean_t_rows},
+        "conflict": {"bayesian": conflict_b, "baseline": conflict_t, "bayesian_cases": conflict_b_rows, "baseline_cases": conflict_t_rows},
+    }
+    if output_path is not None:
+        Path(output_path).write_text(json.dumps(result, indent=2), encoding="utf-8")
+    return result
+
+
+if __name__ == "__main__":
+    print(json.dumps(run_benchmark(), indent=2))

examples/quickstart.py

@@ -0,0 +1,7 @@
+from bayesian_langchain import BayesianIndex, BayesianRAG, DocumentRecord
+
+docs = [
+    DocumentRecord("official", "The current HR portal is PeopleHub.", "Current policy", "official", {"reliability_alpha":18,"reliability_beta":2,"authority":0.98,"freshness":1.0}),
+    DocumentRecord("legacy", "The HR portal is WorkSphere.", "Legacy guide", "cache", {"reliability_alpha":6,"reliability_beta":6,"authority":0.45,"freshness":0.18}),
+]
+print(BayesianRAG(BayesianIndex(docs)).invoke("What is the current HR portal?"))

pyproject.toml

@@ -0,0 +1,32 @@
+[build-system]
+requires = ["setuptools>=68", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "bayesian-langchain-rag"
+version = "1.0.0"
+description = "Generalized-Bayesian retrieval and uncertainty-aware RAG integrated with LangChain"
+readme = "README.md"
+requires-python = ">=3.10"
+license = {text = "MIT"}
+authors = [{name = "Angshul Majumdar"}]
+dependencies = [
+  "numpy>=1.24",
+  "scikit-learn>=1.3",
+  "pydantic>=2.7",
+  "langchain-core>=1.4.7,<2.0",
+]
+
+[project.optional-dependencies]
+dev = ["pytest>=8.0"]
+full = ["langchain>=1.4,<2.0"]
+
+[tool.setuptools]
+package-dir = {"" = "src"}
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[tool.pytest.ini_options]
+pythonpath = ["src"]
+testpaths = ["tests"]

requirements.txt

@@ -0,0 +1,4 @@
+numpy>=1.24
+scikit-learn>=1.3
+pydantic>=2.7
+langchain-core>=1.4.7,<2.0

src/bayesian_langchain/__init__.py

@@ -0,0 +1,23 @@
+from .core import (
+    BayesianIndex,
+    BayesianRetrievalConfig,
+    DocumentRecord,
+    RetrievalHit,
+    ExtractiveGroundedGenerator,
+)
+
+__all__ = [
+    "BayesianIndex",
+    "BayesianRetrievalConfig",
+    "DocumentRecord",
+    "RetrievalHit",
+    "ExtractiveGroundedGenerator",
+]
+
+try:
+    from .retriever import BayesianRetriever
+    from .rag import BayesianRAG
+    __all__ += ["BayesianRetriever", "BayesianRAG"]
+except ImportError:
+    # Core retrieval/evaluation remains usable before optional framework deps are installed.
+    pass

src/bayesian_langchain/core.py

@@ -0,0 +1,255 @@
+from __future__ import annotations
+
+from dataclasses import dataclass, field
+from typing import Any, Iterable, Mapping, Sequence
+import math
+import re
+
+import numpy as np
+from sklearn.feature_extraction.text import TfidfVectorizer, ENGLISH_STOP_WORDS
+
+
+def _clip01(x: float, eps: float = 1e-6) -> float:
+    return min(max(float(x), eps), 1.0)
+
+
+def _logsumexp(xs: Sequence[float]) -> float:
+    if not xs:
+        return float('-inf')
+    finite = [x for x in xs if math.isfinite(x)]
+    if not finite:
+        return float('-inf')
+    m = max(finite)
+    return m + math.log(sum(math.exp(x - m) for x in finite))
+
+
+@dataclass(frozen=True)
+class DocumentRecord:
+    id: str
+    text: str
+    title: str = ""
+    source: str = "unknown"
+    metadata: Mapping[str, Any] = field(default_factory=dict)
+
+    @property
+    def retrieval_text(self) -> str:
+        return (self.title + ". " + self.text).strip(". ")
+
+    def source_prior(self) -> float:
+        md = self.metadata
+        a = max(float(md.get("reliability_alpha", 1.0)), 1e-6)
+        b = max(float(md.get("reliability_beta", 1.0)), 1e-6)
+        reliability = a / (a + b)
+        authority = _clip01(md.get("authority", 0.5))
+        freshness = _clip01(md.get("freshness", 0.5))
+        return reliability * authority * freshness
+
+
+@dataclass(frozen=True)
+class RetrievalHit:
+    document: DocumentRecord
+    retrieval_score: float
+    log_prior: float
+    log_likelihood: float
+    log_posterior_unnormalized: float
+    posterior: float
+    query_coverage: float
+    rank: int
+
+    def as_dict(self) -> dict[str, Any]:
+        return {
+            "id": self.document.id,
+            "title": self.document.title,
+            "text": self.document.text,
+            "source": self.document.source,
+            "metadata": dict(self.document.metadata),
+            "retrieval_score": self.retrieval_score,
+            "log_prior": self.log_prior,
+            "log_likelihood": self.log_likelihood,
+            "posterior": self.posterior,
+            "query_coverage": self.query_coverage,
+            "rank": self.rank,
+        }
+
+
+@dataclass(frozen=True)
+class BayesianRetrievalConfig:
+    candidate_pool: int = 12
+    likelihood_temperature: float = 2.0
+    likelihood_floor: float = 0.02
+    reliability_weight: float = 1.0
+    authority_weight: float = 1.0
+    freshness_weight: float = 0.8
+    prior_floor: float = 1e-4
+    min_relevance: float = 0.06
+    min_top_posterior: float = 0.30
+    min_query_coverage: float = 0.30
+    min_dominance_ratio: float = 1.20
+
+
+class BayesianIndex:
+    """Generalized-Bayesian reranker over TF-IDF retrieval candidates.
+
+    Candidate document hypotheses are scored by
+        p(d|q) ∝ p(d) exp(tau * sim(q,d)).
+    p(d) is induced by a Beta source-reliability mean plus authority/freshness
+    metadata. The exponential similarity term is a Gibbs/generalized-Bayes
+    likelihood, not claimed to be a literal language-model likelihood.
+    """
+
+    def __init__(self, documents: Sequence[DocumentRecord], config: BayesianRetrievalConfig | None = None):
+        if not documents:
+            raise ValueError("documents must be non-empty")
+        self.documents = list(documents)
+        self.config = config or BayesianRetrievalConfig()
+        self.vectorizer = TfidfVectorizer(
+            lowercase=True,
+            stop_words="english",
+            ngram_range=(1, 2),
+            sublinear_tf=True,
+            norm="l2",
+        )
+        self.matrix = self.vectorizer.fit_transform([d.retrieval_text for d in self.documents])
+
+    def _log_prior(self, d: DocumentRecord) -> float:
+        md = d.metadata
+        a = max(float(md.get("reliability_alpha", 1.0)), 1e-6)
+        b = max(float(md.get("reliability_beta", 1.0)), 1e-6)
+        rel = _clip01(a / (a + b))
+        auth = _clip01(md.get("authority", 0.5))
+        fresh = _clip01(md.get("freshness", 0.5))
+        cfg = self.config
+        prior = max(
+            cfg.prior_floor,
+            (rel ** cfg.reliability_weight)
+            * (auth ** cfg.authority_weight)
+            * (fresh ** cfg.freshness_weight),
+        )
+        return math.log(prior)
+
+    def lexical_scores(self, query: str) -> np.ndarray:
+        q = self.vectorizer.transform([str(query)])
+        # TF-IDF vectors are L2 normalized, so dot product is cosine similarity.
+        return (self.matrix @ q.T).toarray().ravel()
+
+    @staticmethod
+    def _query_coverage(query: str, text: str) -> float:
+        qterms = {t for t in re.findall(r"[A-Za-z0-9_-]+", query.lower()) if len(t) > 1 and t not in ENGLISH_STOP_WORDS}
+        if not qterms:
+            return 0.0
+        dterms = set(re.findall(r"[A-Za-z0-9_-]+", text.lower()))
+        return len(qterms & dterms) / len(qterms)
+
+    def search(self, query: str, k: int = 4) -> list[RetrievalHit]:
+        if k <= 0:
+            return []
+        scores = self.lexical_scores(query)
+        pool_n = min(max(k, self.config.candidate_pool), len(self.documents))
+        # Stable sort makes benchmark results reproducible under ties.
+        cand = np.argsort(-scores, kind="stable")[:pool_n]
+        logu: list[float] = []
+        rows: list[tuple[int, float, float, float]] = []
+        for i in cand:
+            d = self.documents[int(i)]
+            s = max(float(scores[int(i)]), 0.0)
+            lp = self._log_prior(d)
+            ll = self.config.likelihood_temperature * math.log(s + self.config.likelihood_floor)
+            u = lp + ll
+            rows.append((int(i), s, lp, ll))
+            logu.append(u)
+        lse = _logsumexp(logu)
+        probs = [math.exp(u - lse) if math.isfinite(lse) else 1.0 / len(logu) for u in logu]
+        ranked = sorted(
+            zip(rows, probs),
+            key=lambda rp: (rp[1], rp[0][1]),
+            reverse=True,
+        )
+        hits: list[RetrievalHit] = []
+        for rank, ((i, s, lp, ll), pr) in enumerate(ranked[:k], start=1):
+            hits.append(RetrievalHit(
+                document=self.documents[i],
+                retrieval_score=s,
+                log_prior=lp,
+                log_likelihood=ll,
+                log_posterior_unnormalized=lp + ll,
+                posterior=float(pr),
+                query_coverage=self._query_coverage(query, self.documents[i].retrieval_text),
+                rank=rank,
+            ))
+        return hits
+
+    def should_abstain(self, hits: Sequence[RetrievalHit]) -> bool:
+        if not hits:
+            return True
+        top = hits[0]
+        if (
+            top.retrieval_score < self.config.min_relevance
+            or top.posterior < self.config.min_top_posterior
+            or top.query_coverage < self.config.min_query_coverage
+        ):
+            return True
+        if len(hits) > 1:
+            second = hits[1]
+            dominance = top.posterior / max(second.posterior, 1e-12)
+            if dominance < self.config.min_dominance_ratio:
+                return True
+        return False
+
+
+class ExtractiveGroundedGenerator:
+    """Dependency-free grounded answer synthesizer for tests and offline use.
+
+    It selects the query-relevant sentence from the highest-posterior evidence
+    and returns that evidence sentence verbatim with its document id. It never
+    invents content not present in the retrieved context.
+    """
+
+    abstain_text = "I do not know based on the provided documents."
+
+    @staticmethod
+    def _sentences(text: str) -> list[str]:
+        parts = re.split(r"(?<=[.!?])\s+|\n+", text.strip())
+        return [p.strip() for p in parts if p.strip()]
+
+    def generate(self, question: str, hits: Sequence[RetrievalHit]) -> tuple[str, list[str]]:
+        if not hits:
+            return self.abstain_text, []
+        # The posterior ranking has already selected the evidence hypothesis.
+        top = hits[0]
+        sents = self._sentences(top.document.text)
+        if not sents:
+            return self.abstain_text, []
+        if len(sents) == 1:
+            sentence = sents[0]
+        else:
+            v = TfidfVectorizer(lowercase=True, stop_words="english", ngram_range=(1, 2))
+            mat = v.fit_transform([question] + sents)
+            q = mat[0]
+            sims = (mat[1:] @ q.T).toarray().ravel()
+            sentence = sents[int(np.argmax(sims))]
+        return f"{sentence} [{top.document.id}]", [top.document.id]
+
+
+class PlainTfidfIndex:
+    """Conventional lexical baseline used by the end-to-end RAG benchmark."""
+    def __init__(self, documents: Sequence[DocumentRecord]):
+        self.documents = list(documents)
+        self.vectorizer = TfidfVectorizer(lowercase=True, stop_words="english", ngram_range=(1, 2), sublinear_tf=True, norm="l2")
+        self.matrix = self.vectorizer.fit_transform([d.retrieval_text for d in self.documents])
+
+    @staticmethod
+    def _query_coverage(query: str, text: str) -> float:
+        qterms = {t for t in re.findall(r"[A-Za-z0-9_-]+", query.lower()) if len(t) > 1 and t not in ENGLISH_STOP_WORDS}
+        if not qterms:
+            return 0.0
+        dterms = set(re.findall(r"[A-Za-z0-9_-]+", text.lower()))
+        return len(qterms & dterms) / len(qterms)
+
+    def search(self, query: str, k: int = 4) -> list[RetrievalHit]:
+        q = self.vectorizer.transform([query])
+        scores = (self.matrix @ q.T).toarray().ravel()
+        order = np.argsort(-scores, kind="stable")[:k]
+        # Baseline assigns normalized positive retrieval mass only for reporting.
+        vals = [max(float(scores[int(i)]), 0.0) for i in order]
+        z = sum(vals) or 1.0
+        return [RetrievalHit(self.documents[int(i)], float(scores[int(i)]), 0.0, 0.0, 0.0, vals[j]/z, 0.0, j+1) for j, i in enumerate(order)]

src/bayesian_langchain/rag.py

@@ -0,0 +1,84 @@
+from __future__ import annotations
+
+from typing import Any, Sequence
+from langchain_core.documents import Document
+from langchain_core.runnables import RunnableLambda
+
+from .core import BayesianIndex, DocumentRecord, ExtractiveGroundedGenerator, RetrievalHit
+from .retriever import BayesianRetriever
+
+
+class BayesianRAG:
+    """End-to-end LangChain RAG with explicit posterior uncertainty.
+
+    Retrieval is performed exactly once through ``BaseRetriever.invoke``.  The
+    returned LangChain ``Document`` objects carry the posterior metadata and are
+    converted back to typed hits only for the uncertainty/grounding layer.
+    """
+
+    def __init__(self, index: BayesianIndex, generator: Any | None = None, k: int = 4):
+        self.index = index
+        self.retriever = BayesianRetriever(index=index, k=k)
+        self.generator = generator or ExtractiveGroundedGenerator()
+        self.k = k
+
+    @staticmethod
+    def _documents_to_hits(documents: Sequence[Document]) -> list[RetrievalHit]:
+        hits: list[RetrievalHit] = []
+        for fallback_rank, doc in enumerate(documents, start=1):
+            md = dict(doc.metadata)
+            rec = DocumentRecord(
+                id=str(md.pop("id", f"doc-{fallback_rank}")),
+                text=doc.page_content,
+                title=str(md.pop("title", "")),
+                source=str(md.pop("source", "unknown")),
+                metadata={
+                    k: v
+                    for k, v in md.items()
+                    if k not in {
+                        "retrieval_score", "log_prior", "log_likelihood",
+                        "log_posterior_unnormalized", "posterior",
+                        "query_coverage", "rank",
+                    }
+                },
+            )
+            hits.append(RetrievalHit(
+                document=rec,
+                retrieval_score=float(doc.metadata.get("retrieval_score", 0.0)),
+                log_prior=float(doc.metadata.get("log_prior", 0.0)),
+                log_likelihood=float(doc.metadata.get("log_likelihood", 0.0)),
+                log_posterior_unnormalized=float(doc.metadata.get("log_posterior_unnormalized", 0.0)),
+                posterior=float(doc.metadata.get("posterior", 0.0)),
+                query_coverage=float(doc.metadata.get("query_coverage", 0.0)),
+                rank=int(doc.metadata.get("rank", fallback_rank)),
+            ))
+        return hits
+
+    def invoke(self, question: str) -> dict[str, Any]:
+        documents = self.retriever.invoke(question)
+        hits = self._documents_to_hits(documents)
+        if self.index.should_abstain(hits):
+            answer = ExtractiveGroundedGenerator.abstain_text
+            citations: list[str] = []
+        elif hasattr(self.generator, "generate"):
+            answer, citations = self.generator.generate(question, hits)
+        else:
+            context = "\n".join(f"[{h.document.id}] {h.document.text}" for h in hits)
+            prompt = (
+                "Answer only from the supplied context. Cite document ids in square brackets. "
+                "If unsupported, say you do not know.\n\n"
+                f"Context:\n{context}\n\nQuestion: {question}"
+            )
+            resp = self.generator.invoke(prompt)
+            answer = getattr(resp, "content", str(resp))
+            citations = [h.document.id for h in hits if f"[{h.document.id}]" in answer]
+        return {
+            "question": question,
+            "answer": answer,
+            "citations": citations,
+            "abstained": not citations,
+            "hits": [h.as_dict() for h in hits],
+        }
+
+    def as_runnable(self) -> RunnableLambda:
+        return RunnableLambda(self.invoke)

src/bayesian_langchain/retriever.py

@@ -0,0 +1,44 @@
+from __future__ import annotations
+
+from typing import Any
+from pydantic import ConfigDict
+from langchain_core.documents import Document
+from langchain_core.retrievers import BaseRetriever
+
+from .core import BayesianIndex, RetrievalHit
+
+
+class BayesianRetriever(BaseRetriever):
+    """LangChain-native retriever backed by the generalized Bayesian index.
+
+    The LangChain contract returns ``Document`` objects.  All posterior fields
+    required downstream are copied into document metadata so an end-to-end RAG
+    chain can use the *actual* retriever output without recomputing retrieval.
+    """
+
+    model_config = ConfigDict(arbitrary_types_allowed=True)
+    index: Any
+    k: int = 4
+
+    def search_hits(self, query: str) -> list[RetrievalHit]:
+        return self.index.search(query, k=self.k)
+
+    @staticmethod
+    def _hit_to_document(h: RetrievalHit) -> Document:
+        md = dict(h.document.metadata)
+        md.update({
+            "id": h.document.id,
+            "title": h.document.title,
+            "source": h.document.source,
+            "retrieval_score": h.retrieval_score,
+            "log_prior": h.log_prior,
+            "log_likelihood": h.log_likelihood,
+            "log_posterior_unnormalized": h.log_posterior_unnormalized,
+            "posterior": h.posterior,
+            "query_coverage": h.query_coverage,
+            "rank": h.rank,
+        })
+        return Document(page_content=h.document.text, metadata=md)
+
+    def _get_relevant_documents(self, query: str, *, run_manager: Any = None) -> list[Document]:
+        return [self._hit_to_document(h) for h in self.search_hits(query)]

src/bayesian_langchain/smc.py

@@ -0,0 +1,129 @@
+from __future__ import annotations
+
+from dataclasses import dataclass, field
+from typing import Any, Callable, Generic, Sequence, TypeVar
+import math
+import random
+
+S = TypeVar("S")
+A = TypeVar("A")
+
+
+def normalize_logweights(logw: Sequence[float]) -> list[float]:
+    if not logw:
+        return []
+    finite = [w for w in logw if math.isfinite(w)]
+    if not finite:
+        return [1.0 / len(logw)] * len(logw)
+    m = max(finite)
+    ws = [math.exp(w - m) if math.isfinite(w) else 0.0 for w in logw]
+    z = sum(ws)
+    if not math.isfinite(z) or z <= 0.0:
+        return [1.0 / len(logw)] * len(logw)
+    return [w / z for w in ws]
+
+
+def effective_sample_size(probs: Sequence[float]) -> float:
+    s2 = sum(float(p) ** 2 for p in probs)
+    return 0.0 if s2 <= 0.0 else 1.0 / s2
+
+
+def systematic_resample(probs: Sequence[float], rng: random.Random) -> list[int]:
+    n = len(probs)
+    if n == 0:
+        return []
+    cdf: list[float] = []
+    total = 0.0
+    for p in probs:
+        total += float(p)
+        cdf.append(total)
+    if total <= 0.0 or not math.isfinite(total):
+        return list(range(n))
+    cdf = [x / total for x in cdf]
+    cdf[-1] = 1.0
+    u0 = rng.random() / n
+    out: list[int] = []
+    j = 0
+    for i in range(n):
+        u = u0 + i / n
+        while j < n - 1 and u > cdf[j]:
+            j += 1
+        out.append(j)
+    return out
+
+
+@dataclass(frozen=True)
+class Proposal(Generic[A]):
+    action: A
+    probability: float
+
+
+@dataclass
+class Particle(Generic[S]):
+    state: S
+    logw: float = 0.0
+    history: list[Any] = field(default_factory=list)
+
+
+@dataclass(frozen=True)
+class SMCConfig:
+    n_particles: int = 32
+    n_steps: int = 3
+    ess_resample_fraction: float = 0.5
+    seed: int = 0
+
+
+class ParticleFilter(Generic[S, A]):
+    """Importance-corrected SMC runtime.
+
+    `propose(state, t)` returns a discrete proposal distribution q(a|state).
+    `transition(state, action, t)` produces the next state.
+    `log_target_increment(prev, action, nxt, t)` supplies the incremental
+    log target density. The importance update is log w += log target - log q.
+    Resampling is never performed after the final weighting step, so the final
+    posterior weights are preserved.
+    """
+
+    def __init__(self, config: SMCConfig | None = None):
+        self.config = config or SMCConfig()
+
+    def run(
+        self,
+        initial_state: S,
+        propose: Callable[[S, int], Sequence[Proposal[A]]],
+        transition: Callable[[S, A, int], S],
+        log_target_increment: Callable[[S, A, S, int], float],
+    ) -> list[Particle[S]]:
+        cfg = self.config
+        rng = random.Random(cfg.seed)
+        particles = [Particle(state=initial_state) for _ in range(cfg.n_particles)]
+        for t in range(cfg.n_steps):
+            nxt: list[Particle[S]] = []
+            for p in particles:
+                proposals = [x for x in propose(p.state, t) if x.probability > 0.0]
+                if not proposals:
+                    raise ValueError("proposal distribution is empty")
+                z = sum(x.probability for x in proposals)
+                probs = [x.probability / z for x in proposals]
+                r = rng.random()
+                acc = 0.0
+                j = len(proposals) - 1
+                for i, pr in enumerate(probs):
+                    acc += pr
+                    if r <= acc:
+                        j = i
+                        break
+                selected = proposals[j]
+                q = probs[j]
+                new_state = transition(p.state, selected.action, t)
+                inc = float(log_target_increment(p.state, selected.action, new_state, t))
+                nxt.append(Particle(new_state, p.logw + inc - math.log(q), p.history + [selected.action]))
+            particles = nxt
+            probs_now = normalize_logweights([p.logw for p in particles])
+            if t < cfg.n_steps - 1 and effective_sample_size(probs_now) < cfg.ess_resample_fraction * len(particles):
+                ids = systematic_resample(probs_now, rng)
+                particles = [Particle(particles[i].state, 0.0, list(particles[i].history)) for i in ids]
+        final_probs = normalize_logweights([p.logw for p in particles])
+        for p, pr in zip(particles, final_probs):
+            setattr(p, "posterior", pr)
+        return particles

tests/test_core.py

@@ -0,0 +1,17 @@
+import math
+from bayesian_langchain.core import BayesianIndex, DocumentRecord
+from bayesian_langchain.smc import normalize_logweights
+
+
+def test_all_negative_infinity_weights_become_uniform():
+    p = normalize_logweights([float('-inf'), float('-inf'), float('-inf')])
+    assert p == [1/3, 1/3, 1/3]
+
+
+def test_authoritative_current_document_beats_stale_high_overlap():
+    docs = [
+        DocumentRecord('stale','The current HR portal is WorkSphere.','Current HR portal','cache',{'reliability_alpha':6,'reliability_beta':6,'authority':0.45,'freshness':0.18}),
+        DocumentRecord('official','Leave requests and payslips are now handled in PeopleHub.','Migration complete','official',{'reliability_alpha':18,'reliability_beta':2,'authority':0.98,'freshness':1.0}),
+    ]
+    hits = BayesianIndex(docs).search('What is the current HR portal for leave requests and payslips?', k=2)
+    assert hits[0].document.id == 'official'

tests/test_framework_rag_e2e.py

@@ -0,0 +1,7 @@
+from evaluation.framework_rag_benchmark import run_framework_benchmark
+
+
+def test_real_langchain_framework_rag_suite():
+    result = run_framework_benchmark()
+    assert result["clean"]["passed"] == result["clean"]["n_cases"] == 8
+    assert result["conflict"]["passed"] == result["conflict"]["n_cases"] == 18

tests/test_langchain_integration.py

@@ -0,0 +1,37 @@
+from langchain_core.runnables import RunnableLambda
+
+from bayesian_langchain import BayesianIndex, BayesianRAG, BayesianRetriever, DocumentRecord
+
+
+def _docs():
+    return [
+        DocumentRecord(
+            'd1',
+            'The access code is ORBIT-7.',
+            'Access',
+            'official',
+            {'reliability_alpha':10,'reliability_beta':1,'authority':0.99,'freshness':1.0},
+        )
+    ]
+
+
+def test_real_langchain_retriever_and_runnable():
+    idx = BayesianIndex(_docs())
+    retriever = BayesianRetriever(index=idx, k=1)
+    out = retriever.invoke('What is the access code?')
+    assert out[0].metadata['id'] == 'd1'
+    assert out[0].metadata['posterior'] > 0.0
+    rag = BayesianRAG(idx)
+    ans = rag.as_runnable().invoke('What is the access code?')
+    assert 'ORBIT-7' in ans['answer']
+    assert ans['citations'] == ['d1']
+
+
+def test_langchain_model_runnable_generation_branch():
+    idx = BayesianIndex(_docs())
+    generator = RunnableLambda(lambda prompt: 'The access code is ORBIT-7. [d1]')
+    rag = BayesianRAG(idx, generator=generator, k=1)
+    out = rag.as_runnable().invoke('What is the access code?')
+    assert out['answer'] == 'The access code is ORBIT-7. [d1]'
+    assert out['citations'] == ['d1']
+    assert not out['abstained']

tests/test_rag_e2e.py

@@ -0,0 +1,9 @@
+from evaluation.rag_benchmark import run_benchmark
+
+
+def test_end_to_end_rag_benchmark():
+    x = run_benchmark()
+    assert x['clean']['bayesian']['end_to_end_accuracy'] == 1.0
+    assert x['conflict']['bayesian']['recall_at_1'] == 1.0
+    assert x['conflict']['bayesian']['abstention_accuracy'] == 1.0
+    assert x['conflict']['bayesian']['grounding_accuracy'] == 1.0

tests/test_smc_math.py

@@ -0,0 +1,52 @@
+import math
+
+
+def _imports():
+    try:
+        from bayesian_langchain.smc import ParticleFilter, Proposal, SMCConfig
+    except ImportError:
+        from bayesian_langgraph.smc import ParticleFilter, Proposal, SMCConfig
+    return ParticleFilter, Proposal, SMCConfig
+
+
+def test_final_step_preserves_nonuniform_posterior_weights():
+    ParticleFilter, Proposal, SMCConfig = _imports()
+    pf = ParticleFilter(SMCConfig(n_particles=64, n_steps=1, ess_resample_fraction=0.99, seed=7))
+
+    def propose(state, t):
+        return [Proposal("good", 0.5), Proposal("bad", 0.5)]
+
+    def transition(state, action, t):
+        return action
+
+    def log_target(prev, action, nxt, t):
+        return 0.0 if action == "good" else math.log(1e-5)
+
+    particles = pf.run(None, propose, transition, log_target)
+    probs = [p.posterior for p in particles]
+    assert abs(sum(probs) - 1.0) < 1e-12
+    # If the old final-resampling bug returned, these would all be 1/N.
+    assert max(probs) - min(probs) > 1e-3
+
+
+def test_importance_weight_contains_inverse_proposal_probability():
+    ParticleFilter, Proposal, SMCConfig = _imports()
+    pf = ParticleFilter(SMCConfig(n_particles=256, n_steps=1, seed=11))
+
+    def propose(state, t):
+        return [Proposal("a", 0.9), Proposal("b", 0.1)]
+
+    def transition(state, action, t):
+        return action
+
+    # Equal target mass for both actions. Importance correction should cancel
+    # the 9:1 proposal imbalance in the weighted posterior, up to MC error.
+    def log_target(prev, action, nxt, t):
+        return math.log(0.5)
+
+    particles = pf.run(None, propose, transition, log_target)
+    mass = {"a": 0.0, "b": 0.0}
+    for p in particles:
+        mass[p.state] += p.posterior
+    assert abs(mass["a"] - 0.5) < 0.15
+    assert abs(mass["b"] - 0.5) < 0.15