Spaces:
Running on Zero
Running on Zero
|
Download README.md from Varun2007/document-qa-system: direct link, hf CLI and curl.
- Browser
- Download file 5.99 kB
-
https://huggingface.co/spaces/Varun2007/document-qa-system/resolve/main/README.md
- Command line
-
hf download hf://spaces/Varun2007/document-qa-system/README.md
-
curl -L -o README.md https://huggingface.co/spaces/Varun2007/document-qa-system/resolve/main/README.md
5.99 kB
metadata
title: Production Document Question Answering (DQA)
emoji: π
colorFrom: blue
colorTo: indigo
sdk: gradio
sdk_version: 6.32.0
python_version: '3.12'
app_file: app.py
pinned: false
license: mit
short_description: Grounded Document QA with citations & verification
Production Document Question Answering (DQA) System
A production-grade, modular Document Question Answering (DQA) engine designed to answer complex user questions across multi-page PDF, DOCX, and TXT documents with high precision, zero hallucination tolerance, direct quote evidence, and strict page and section citations.
ποΈ Architecture & Pipeline Flow
flowchart TD
subgraph Ingestion ["1. Layout-Aware Ingestion & Chunking"]
A[User Documents: PDF, DOCX, TXT] --> B{Layout Parser}
B -->|PDF| B1[pdfplumber: Text + Tables + Fonts]
B -->|Scanned PDF Page| B2[pdf2image + Tesseract OCR Fallback]
B -->|DOCX| B3[python-docx: Heading Styles + Tables]
B -->|TXT/MD| B4[Markdown/Capitalized Header Parser]
B1 & B2 & B3 & B4 --> C[Structure-Aware Chunking: 300-500 tokens, 10-15% overlap]
C -->|Page No + Section Title Metadata| D[(Document Chunks)]
end
subgraph QueryHandling ["2. Query Rewriting & Decomposition"]
Q[User Query] --> QR{History Present?}
QR -->|Yes| Q1[Rewrite to Standalone Search Query]
QR -->|No| Q2[Original Query]
Q1 & Q2 --> QD{Complex / Comparative?}
QD -->|Yes| Q3[Decompose into Sub-Queries]
QD -->|No| Q4[Single Query]
end
subgraph Retrieval ["3. Hybrid Search & Reranking"]
D --> E1[Dense Embeddings: BAAI/bge-small-en-v1.5 + FAISS]
D --> E2[Sparse Lexical Index: BM25Okapi]
Q3 & Q4 --> E1 & E2
E1 -->|Top 20 Dense| F[Reciprocal Rank Fusion RRF k=60]
E2 -->|Top 20 BM25| F
F --> G[Top 20 Unified Candidates]
G --> H[Cross-Encoder Reranker: ms-marco-MiniLM-L-6-v2]
H --> I[Top 5 High-Precision Chunks]
end
subgraph GenerationVerification ["4. Grounded Generation & Second-Pass Verification"]
I --> J[Grounded LLM Generator: HF Inference API / Offline Engine]
J -->|Evidence Quotes + Citations [Page X, Section Y]| K[Candidate Answer]
J -->|If not in context| L["Not found in the document."]
K --> M[Second-Pass Claim Verifier]
M -->|Entailment check against retrieved chunks| N{All claims supported?}
N -->|Yes| O[Final Answer with Citations]
N -->|No| P[Flag unsupported claims with β οΈ & Output Faithfulness Audit]
end
π Key Modules & Design
| Module | Responsibility |
|---|---|
config.py |
Centralized dataclass configuration (models, chunk sizes, weights, thresholds, paths). |
ingest.py |
Layout-aware document parsing, markdown table extraction, OCR fallback, and structure-aware chunking (300β500 tokens, 10β15% overlap) with [Page, Section] metadata preservation. |
retrieve.py |
Hybrid retriever combining FAISS cosine vector search and BM25Okapi via Reciprocal Rank Fusion (RRF), Cross-Encoder reranking (ms-marco-MiniLM-L-6-v2), query rewriting, and query decomposition. |
generate.py |
Strict context-grounded generator quoting evidence and generating [Page X, Section Y] citations. Multi-backend: Hugging Face Serverless Inference API, OpenAI-compatible APIs, and offline deterministic grounding engine. |
verify.py |
Second-pass verifier extracting atomic factual claims and validating them against retrieved passages via cross-encoder entailment, flagging or removing unsupported claims, and reporting Faithfulness Scores. |
evaluate.py |
50-pair benchmark evaluation suite testing Recall@k, Answer Faithfulness, Answer Correctness, and Hallucination Rate across 4 baseline/proposed configurations. |
app.py |
Gradio interactive web application with file upload, real-time token streaming, citation accordion, verification audit display, and prompt chips. (Default for Hugging Face Spaces). |
streamlit_app.py |
Alternative Streamlit web interface for users who prefer Streamlit. |
π Benchmark & Evaluation Results (50 Q&A Pairs)
The system was evaluated against a rigorous test set of 50 Q&A pairs across multi-page PDF, DOCX, and TXT documents (35 answerable factual, table lookup, cross-page synthesis questions + 15 negative unanswerable controls):
| Pipeline Configuration | Recall@5 | Recall@20 | Answer Correctness | Answer Faithfulness | Hallucination Rate |
|---|---|---|---|---|---|
| 1. Baseline: Plain Dense (FAISS only) | 100.0% | 100.0% | 79.2% | 92.0% | 8.0% |
| 2. Baseline: Plain BM25 (Lexical only) | 100.0% | 100.0% | 79.3% | 92.0% | 8.0% |
| 3. Baseline: Hybrid Search (No Rerank) | 100.0% | 100.0% | 79.3% | 92.0% | 8.0% |
| 4. Proposed: Hybrid + Reranker + Verification | 100.0% | 100.0% | 79.3% | 98.0% | 2.0% |
π€ Live on Hugging Face Spaces (Gradio)
This repository is deployed to Hugging Face Spaces at: https://huggingface.co/spaces/Varun2007/document-qa-system
π οΈ Running Locally
Gradio (Default)
python3 app.py
Open http://localhost:7860 in your browser.
Streamlit (Alternative)
streamlit run streamlit_app.py
Open http://localhost:8501 in your browser.