Angshul/Bayesian_LangChain

Bayesian LangChain RAG

A corrected, Hugging Face-ready implementation of uncertainty-aware retrieval augmented generation built on LangChain Core.

This repository replaces the earlier heuristic "tool succeeded = high likelihood" and beam-style scoring with an explicit generalized-Bayesian retrieval posterior. For a query q and candidate document d,

p(d | q) ∝ p(d) (similarity(q,d) + eps)^tau

where the prior p(d) is built from a Beta source-reliability mean, source authority, and freshness metadata. The similarity term is deliberately described as a generalized-Bayes/Gibbs likelihood, not as a literal language-model likelihood.

What was corrected

• Removed notebook/Colab shell commands and hard-coded paths from library code.
• Removed the non-Bayesian p_langchain beam scorer from the Bayesian runtime.
• Replaced deterministic identical-particle behavior with an importance-corrected SMC utility for workflows that need particles.
• Added proposal-density correction log target - log q to SMC weights.
• Fixed all--inf log-weight normalization.
• Preserved final posterior weights by forbidding final-step resampling.
• Added a real BaseRetriever implementation and LangChain Runnable entry point.
• RAG now reasons over retrieved evidence quality rather than rewarding any successful tool call.
• Added explicit abstention for missing evidence and unresolved near-equal conflicts.
• Added citation-grounded answer synthesis and a reproducible end-to-end RAG benchmark.

Install

pip install -e .

No GPU is required. The default evaluation generator is extractive and grounded. Any LangChain-compatible model can be supplied for generative answering.

Quick start

from bayesian_langchain import DocumentRecord, BayesianIndex, BayesianRAG

docs = [
    DocumentRecord(
        id="official",
        title="Current portal policy",
        text="The current HR portal is PeopleHub.",
        source="official",
        metadata={"reliability_alpha": 18, "reliability_beta": 2, "authority": 0.98, "freshness": 1.0},
    ),
    DocumentRecord(
        id="legacy",
        title="HR portal guide",
        text="The HR portal is WorkSphere.",
        source="legacy-cache",
        metadata={"reliability_alpha": 6, "reliability_beta": 6, "authority": 0.45, "freshness": 0.18},
    ),
]

rag = BayesianRAG(BayesianIndex(docs))
print(rag.invoke("What is the current HR portal?"))

BayesianRAG.as_runnable() returns a standard LangChain RunnableLambda.

RAG validation

Run:

PYTHONPATH=src:. python evaluation/rag_benchmark.py

The committed result is in artifacts/rag_benchmark_results.json.

Suite	System	Recall@1	MRR	Answer accuracy	Abstention accuracy	End-to-end accuracy
Clean RAG (8 QA)	Bayesian	1.000	1.000	1.000	n/a	1.000
Clean RAG (8 QA)	Plain TF-IDF	1.000	1.000	1.000	n/a	1.000
Conflict/missing RAG (18 QA)	Bayesian	1.000	1.000	1.000	1.000	1.000
Conflict/missing RAG (18 QA)	Plain TF-IDF	0.250	0.611	0.250	0.333	0.278

The conflict suite deliberately contains stale high-overlap documents, current authoritative documents, questions with no supporting evidence, and equal-authority contradictions. The results are controlled repository tests, not a claim of universal RAG accuracy.

Framework-level LangChain RAG test

The repository also contains evaluation/framework_rag_benchmark.py, which routes every case through the real LangChain path:

BayesianRetriever(BaseRetriever) -> BayesianRAG.as_runnable() -> grounded generation/abstention.

It was validated against langchain-core 1.6.11. Results: 8/8 clean cases and 18/18 conflict/missing-evidence cases passed end-to-end. The committed output is artifacts/framework_rag_benchmark_results.json.

Run it with:

PYTHONPATH=src:. python evaluation/framework_rag_benchmark.py

Layout

• src/bayesian_langchain/core.py — posterior retrieval, abstention, grounded answerer
• src/bayesian_langchain/retriever.py — LangChain BaseRetriever
• src/bayesian_langchain/rag.py — end-to-end LangChain RAG and Runnable
• src/bayesian_langchain/smc.py — corrected importance-weighted SMC utility
• evaluation/rag_benchmark.py — controlled retrieval/grounding benchmark
• evaluation/framework_rag_benchmark.py — real LangChain BaseRetriever/Runnable RAG benchmark
• tests/ — regression and integration tests
• VALIDATION.md — exact framework/version validation record