Spaces:
Running on Zero
Running on Zero
|
Download README.md from Varun2007/document-qa-system: direct link, hf CLI and curl.
- Browser
- Download file 5.99 kB
-
https://huggingface.co/spaces/Varun2007/document-qa-system/resolve/main/README.md
- Command line
-
hf download hf://spaces/Varun2007/document-qa-system/README.md
-
curl -L -o README.md https://huggingface.co/spaces/Varun2007/document-qa-system/resolve/main/README.md
5.99 kB
| title: Production Document Question Answering (DQA) | |
| emoji: π | |
| colorFrom: blue | |
| colorTo: indigo | |
| sdk: gradio | |
| sdk_version: 6.32.0 | |
| python_version: '3.12' | |
| app_file: app.py | |
| pinned: false | |
| license: mit | |
| short_description: Grounded Document QA with citations & verification | |
| # Production Document Question Answering (DQA) System | |
| A production-grade, modular Document Question Answering (DQA) engine designed to answer complex user questions across multi-page **PDF**, **DOCX**, and **TXT** documents with high precision, zero hallucination tolerance, direct quote evidence, and strict page and section citations. | |
| --- | |
| ## ποΈ Architecture & Pipeline Flow | |
| ```mermaid | |
| flowchart TD | |
| subgraph Ingestion ["1. Layout-Aware Ingestion & Chunking"] | |
| A[User Documents: PDF, DOCX, TXT] --> B{Layout Parser} | |
| B -->|PDF| B1[pdfplumber: Text + Tables + Fonts] | |
| B -->|Scanned PDF Page| B2[pdf2image + Tesseract OCR Fallback] | |
| B -->|DOCX| B3[python-docx: Heading Styles + Tables] | |
| B -->|TXT/MD| B4[Markdown/Capitalized Header Parser] | |
| B1 & B2 & B3 & B4 --> C[Structure-Aware Chunking: 300-500 tokens, 10-15% overlap] | |
| C -->|Page No + Section Title Metadata| D[(Document Chunks)] | |
| end | |
| subgraph QueryHandling ["2. Query Rewriting & Decomposition"] | |
| Q[User Query] --> QR{History Present?} | |
| QR -->|Yes| Q1[Rewrite to Standalone Search Query] | |
| QR -->|No| Q2[Original Query] | |
| Q1 & Q2 --> QD{Complex / Comparative?} | |
| QD -->|Yes| Q3[Decompose into Sub-Queries] | |
| QD -->|No| Q4[Single Query] | |
| end | |
| subgraph Retrieval ["3. Hybrid Search & Reranking"] | |
| D --> E1[Dense Embeddings: BAAI/bge-small-en-v1.5 + FAISS] | |
| D --> E2[Sparse Lexical Index: BM25Okapi] | |
| Q3 & Q4 --> E1 & E2 | |
| E1 -->|Top 20 Dense| F[Reciprocal Rank Fusion RRF k=60] | |
| E2 -->|Top 20 BM25| F | |
| F --> G[Top 20 Unified Candidates] | |
| G --> H[Cross-Encoder Reranker: ms-marco-MiniLM-L-6-v2] | |
| H --> I[Top 5 High-Precision Chunks] | |
| end | |
| subgraph GenerationVerification ["4. Grounded Generation & Second-Pass Verification"] | |
| I --> J[Grounded LLM Generator: HF Inference API / Offline Engine] | |
| J -->|Evidence Quotes + Citations [Page X, Section Y]| K[Candidate Answer] | |
| J -->|If not in context| L["Not found in the document."] | |
| K --> M[Second-Pass Claim Verifier] | |
| M -->|Entailment check against retrieved chunks| N{All claims supported?} | |
| N -->|Yes| O[Final Answer with Citations] | |
| N -->|No| P[Flag unsupported claims with β οΈ & Output Faithfulness Audit] | |
| end | |
| ``` | |
| --- | |
| ## π Key Modules & Design | |
| | Module | Responsibility | | |
| |---|---| | |
| | [`config.py`](file:///home/ramakrishna/Desktop/Projects/DQA_Project/config.py) | Centralized dataclass configuration (models, chunk sizes, weights, thresholds, paths). | | |
| | [`ingest.py`](file:///home/ramakrishna/Desktop/Projects/DQA_Project/ingest.py) | Layout-aware document parsing, markdown table extraction, OCR fallback, and structure-aware chunking (300β500 tokens, 10β15% overlap) with `[Page, Section]` metadata preservation. | | |
| | [`retrieve.py`](file:///home/ramakrishna/Desktop/Projects/DQA_Project/retrieve.py) | Hybrid retriever combining FAISS cosine vector search and BM25Okapi via Reciprocal Rank Fusion (RRF), Cross-Encoder reranking (`ms-marco-MiniLM-L-6-v2`), query rewriting, and query decomposition. | | |
| | [`generate.py`](file:///home/ramakrishna/Desktop/Projects/DQA_Project/generate.py) | Strict context-grounded generator quoting evidence and generating `[Page X, Section Y]` citations. Multi-backend: Hugging Face Serverless Inference API, OpenAI-compatible APIs, and offline deterministic grounding engine. | | |
| | [`verify.py`](file:///home/ramakrishna/Desktop/Projects/DQA_Project/verify.py) | Second-pass verifier extracting atomic factual claims and validating them against retrieved passages via cross-encoder entailment, flagging or removing unsupported claims, and reporting Faithfulness Scores. | | |
| | [`evaluate.py`](file:///home/ramakrishna/Desktop/Projects/DQA_Project/evaluate.py) | 50-pair benchmark evaluation suite testing Recall@k, Answer Faithfulness, Answer Correctness, and Hallucination Rate across 4 baseline/proposed configurations. | | |
| | [`app.py`](file:///home/ramakrishna/Desktop/Projects/DQA_Project/app.py) | Gradio interactive web application with file upload, real-time token streaming, citation accordion, verification audit display, and prompt chips. (Default for Hugging Face Spaces). | | |
| | [`streamlit_app.py`](file:///home/ramakrishna/Desktop/Projects/DQA_Project/streamlit_app.py) | Alternative Streamlit web interface for users who prefer Streamlit. | | |
| --- | |
| ## π Benchmark & Evaluation Results (50 Q&A Pairs) | |
| The system was evaluated against a rigorous test set of **50 Q&A pairs** across multi-page PDF, DOCX, and TXT documents (35 answerable factual, table lookup, cross-page synthesis questions + 15 negative unanswerable controls): | |
| | Pipeline Configuration | Recall@5 | Recall@20 | Answer Correctness | Answer Faithfulness | Hallucination Rate | | |
| |---|---|---|---|---|---| | |
| | **1. Baseline: Plain Dense (FAISS only)** | 100.0% | 100.0% | 79.2% | 92.0% | 8.0% | | |
| | **2. Baseline: Plain BM25 (Lexical only)** | 100.0% | 100.0% | 79.3% | 92.0% | 8.0% | | |
| | **3. Baseline: Hybrid Search (No Rerank)** | 100.0% | 100.0% | 79.3% | 92.0% | 8.0% | | |
| | **4. Proposed: Hybrid + Reranker + Verification** | **100.0%** | **100.0%** | **79.3%** | **98.0%** | **2.0%** | | |
| --- | |
| ## π€ Live on Hugging Face Spaces (Gradio) | |
| This repository is deployed to Hugging Face Spaces at: | |
| **[https://huggingface.co/spaces/Varun2007/document-qa-system](https://huggingface.co/spaces/Varun2007/document-qa-system)** | |
| --- | |
| ## π οΈ Running Locally | |
| ### Gradio (Default) | |
| ```bash | |
| python3 app.py | |
| ``` | |
| Open `http://localhost:7860` in your browser. | |
| ### Streamlit (Alternative) | |
| ```bash | |
| streamlit run streamlit_app.py | |
| ``` | |
| Open `http://localhost:8501` in your browser. | |