README.md

metadata

license: mit
tags:
  - langgraph
  - rag
  - bayesian
  - probabilistic
  - uncertainty
  - retrieval

Bayesian LangGraph RAG

A corrected, Hugging Face-ready Bayesian RAG implementation using a real LangGraph StateGraph.

The original repository used a custom Python loop under the LangGraph name and could erase its final SMC posterior through final-step resampling. This version separates retrieval inference from orchestration and uses the graph:

START -> retrieve -> assess -> (generate | abstain) -> END

The retriever uses the generalized-Bayesian posterior

p(d | q) ∝ p(d) (similarity(q,d) + eps)^tau

with a Beta reliability prior plus authority and freshness metadata.

What was corrected

• Replaced the custom pseudo-LangGraph runtime with an actual StateGraph.
• Removed Colab-only setup, Drive mounts, repository cloning, and import-time execution.
• Fixed all--inf log-weight normalization.
• Fixed final-step resampling so the posterior cannot be reset to uniform.
• Added proposal-density correction to the reusable SMC implementation.
• Removed deterministic identical-particle demos as evidence of inference quality.
• Replaced hard-coded "Bayes wins" benchmark flags with end-to-end document retrieval and grounded answering.
• Added uncertainty-aware abstention for missing evidence and unresolved source conflicts.
• Added reproducible RAG evaluation with a plain TF-IDF baseline.

Install

pip install -e .

No GPU is required. The default generator is extractive and citation-grounded; a model-backed generator can be supplied if desired.

Quick start

from bayesian_langgraph import DocumentRecord, BayesianIndex, BayesianRAGGraph

docs = [
    DocumentRecord(
        id="official",
        title="Current portal policy",
        text="The current HR portal is PeopleHub.",
        source="official",
        metadata={"reliability_alpha": 18, "reliability_beta": 2, "authority": 0.98, "freshness": 1.0},
    ),
    DocumentRecord(
        id="legacy",
        title="HR portal guide",
        text="The HR portal is WorkSphere.",
        source="legacy-cache",
        metadata={"reliability_alpha": 6, "reliability_beta": 6, "authority": 0.45, "freshness": 0.18},
    ),
]

graph_rag = BayesianRAGGraph(BayesianIndex(docs))
print(graph_rag.invoke("What is the current HR portal?"))

RAG validation

Run:

PYTHONPATH=src:. python evaluation/rag_benchmark.py

The committed result is in artifacts/rag_benchmark_results.json.

Suite	System	Recall@1	MRR	Answer accuracy	Abstention accuracy	End-to-end accuracy
Clean RAG (8 QA)	Bayesian	1.000	1.000	1.000	n/a	1.000
Clean RAG (8 QA)	Plain TF-IDF	1.000	1.000	1.000	n/a	1.000
Conflict/missing RAG (18 QA)	Bayesian	1.000	1.000	1.000	1.000	1.000
Conflict/missing RAG (18 QA)	Plain TF-IDF	0.250	0.611	0.250	0.333	0.278

These are controlled end-to-end repository tests, not a claim of universal RAG accuracy.

Framework-level LangGraph RAG test

evaluation/framework_rag_benchmark.py executes every case through the compiled LangGraph graph itself. The validated topology is:

START -> retrieve -> assess -> {generate | abstain} -> END.

It was validated against langgraph 1.2.16 with langchain-core 1.6.11. Results: 8/8 clean cases and 18/18 conflict/missing-evidence cases passed end-to-end. The committed output is artifacts/framework_rag_benchmark_results.json.

Run it with:

PYTHONPATH=src:. python evaluation/framework_rag_benchmark.py

Layout

• src/bayesian_langgraph/core.py — posterior retrieval, abstention, grounded answerer
• src/bayesian_langgraph/graph.py — actual LangGraph StateGraph
• src/bayesian_langgraph/smc.py — corrected importance-weighted SMC utility
• evaluation/rag_benchmark.py — controlled retrieval/grounding benchmark
• evaluation/framework_rag_benchmark.py — compiled StateGraph RAG benchmark
• tests/ — regression and graph-integration tests
• VALIDATION.md — exact framework/version validation record