| --- |
| license: mit |
| tags: |
| - langgraph |
| - rag |
| - bayesian |
| - probabilistic |
| - uncertainty |
| - retrieval |
| --- |
| |
| # Bayesian LangGraph RAG |
|
|
| A corrected, Hugging Face-ready Bayesian RAG implementation using a **real LangGraph `StateGraph`**. |
|
|
| The original repository used a custom Python loop under the LangGraph name and could erase its final SMC posterior through final-step resampling. This version separates retrieval inference from orchestration and uses the graph: |
|
|
| `START -> retrieve -> assess -> (generate | abstain) -> END` |
|
|
| The retriever uses the generalized-Bayesian posterior |
|
|
| `p(d | q) ∝ p(d) (similarity(q,d) + eps)^tau` |
|
|
| with a Beta reliability prior plus authority and freshness metadata. |
|
|
| ## What was corrected |
|
|
| - Replaced the custom pseudo-LangGraph runtime with an actual `StateGraph`. |
| - Removed Colab-only setup, Drive mounts, repository cloning, and import-time execution. |
| - Fixed all-`-inf` log-weight normalization. |
| - Fixed final-step resampling so the posterior cannot be reset to uniform. |
| - Added proposal-density correction to the reusable SMC implementation. |
| - Removed deterministic identical-particle demos as evidence of inference quality. |
| - Replaced hard-coded "Bayes wins" benchmark flags with end-to-end document retrieval and grounded answering. |
| - Added uncertainty-aware abstention for missing evidence and unresolved source conflicts. |
| - Added reproducible RAG evaluation with a plain TF-IDF baseline. |
|
|
| ## Install |
|
|
| ```bash |
| pip install -e . |
| ``` |
|
|
| No GPU is required. The default generator is extractive and citation-grounded; a model-backed generator can be supplied if desired. |
|
|
| ## Quick start |
|
|
| ```python |
| from bayesian_langgraph import DocumentRecord, BayesianIndex, BayesianRAGGraph |
| |
| docs = [ |
| DocumentRecord( |
| id="official", |
| title="Current portal policy", |
| text="The current HR portal is PeopleHub.", |
| source="official", |
| metadata={"reliability_alpha": 18, "reliability_beta": 2, "authority": 0.98, "freshness": 1.0}, |
| ), |
| DocumentRecord( |
| id="legacy", |
| title="HR portal guide", |
| text="The HR portal is WorkSphere.", |
| source="legacy-cache", |
| metadata={"reliability_alpha": 6, "reliability_beta": 6, "authority": 0.45, "freshness": 0.18}, |
| ), |
| ] |
| |
| graph_rag = BayesianRAGGraph(BayesianIndex(docs)) |
| print(graph_rag.invoke("What is the current HR portal?")) |
| ``` |
|
|
| ## RAG validation |
|
|
| Run: |
|
|
| ```bash |
| PYTHONPATH=src:. python evaluation/rag_benchmark.py |
| ``` |
|
|
| The committed result is in `artifacts/rag_benchmark_results.json`. |
|
|
| | Suite | System | Recall@1 | MRR | Answer accuracy | Abstention accuracy | End-to-end accuracy | |
| |---|---|---:|---:|---:|---:|---:| |
| | Clean RAG (8 QA) | Bayesian | 1.000 | 1.000 | 1.000 | n/a | 1.000 | |
| | Clean RAG (8 QA) | Plain TF-IDF | 1.000 | 1.000 | 1.000 | n/a | 1.000 | |
| | Conflict/missing RAG (18 QA) | Bayesian | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | |
| | Conflict/missing RAG (18 QA) | Plain TF-IDF | 0.250 | 0.611 | 0.250 | 0.333 | 0.278 | |
|
|
| These are controlled end-to-end repository tests, not a claim of universal RAG accuracy. |
|
|
| ### Framework-level LangGraph RAG test |
|
|
| `evaluation/framework_rag_benchmark.py` executes every case through the compiled LangGraph graph itself. The validated topology is: |
|
|
| `START -> retrieve -> assess -> {generate | abstain} -> END`. |
|
|
| It was validated against **langgraph 1.2.16** with **langchain-core 1.6.11**. Results: **8/8 clean cases** and **18/18 conflict/missing-evidence cases** passed end-to-end. The committed output is `artifacts/framework_rag_benchmark_results.json`. |
|
|
| Run it with: |
|
|
| ```bash |
| PYTHONPATH=src:. python evaluation/framework_rag_benchmark.py |
| ``` |
|
|
| ## Layout |
|
|
| - `src/bayesian_langgraph/core.py` — posterior retrieval, abstention, grounded answerer |
| - `src/bayesian_langgraph/graph.py` — actual LangGraph `StateGraph` |
| - `src/bayesian_langgraph/smc.py` — corrected importance-weighted SMC utility |
| - `evaluation/rag_benchmark.py` — controlled retrieval/grounding benchmark |
| - `evaluation/framework_rag_benchmark.py` — compiled `StateGraph` RAG benchmark |
| - `tests/` — regression and graph-integration tests |
| - `VALIDATION.md` — exact framework/version validation record |
|
|
