Spaces:
Paused
Paused
Commit History
-
feat: update prompts for SQL routing, CHUNK-SQL IDs, and schema injection
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>8529300 -
feat: add SQL result formatting and schema summary builder
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>203c044 -
feat: add SQL retriever — validated read-only query execution
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>54bc2bb -
feat: add SQL ingestion — load tabular files into SQLite with schema registry
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>863e8aa -
feat: add SQL ingestion helpers — table naming, type inference, samples
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>9649eb0 -
fix: harden codebase against format injection, empty responses, and unsafe config access
- Add _escape_braces() to all prompt builders to prevent str.format() crash when user documents contain {placeholder} patterns - Fix app_web.py display_query fallback to preserve clarification context - Add empty-response guards to all 3 LLM providers (openai, anthropic, gemini) - Replace all remaining direct dict access with .get() + defaults - Add 2 regression tests for curly-brace format injection - Expand query engine tests with display_query coverage - 62 tests pass across 13 test files Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>9815e49 -
fix: use .get() for all config access + add CLAUDE.md and docs to repo
- src/ingest.py: replace cfg["paths"]["vector_db"] and cfg["paths"]["knowledge_base"] with .get() + defaults - src/llm/__init__.py: replace cfg["llm"]["provider"] and cfg["llm"].get(...) with safe llm_cfg = cfg.get("llm", {}) - src/search/__init__.py: replace r['authors'], r['title'], etc. with .get() + defaults in web results formatter - app_web.py: respect max_clarifications config (was hardcoded to allow 1 round regardless of config) - tests/test_setup.py: add missing assertions for retrieval.top_k and retrieval.max_distance - Add CLAUDE.md, docs/plans/, docs/WORKLOG.md to the repository Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>83087ed -
feat: add query understanding layer for better retrieval
Pre-retrieval LLM step that reformulates user queries into search-optimized forms (expanded keywords, resolved pronouns, domain-aware terms) and optionally asks clarification questions. - New src/query_engine.py with understand_query() + JSON parsing - New prompt template in src/prompts.py - CLI: query understanding + conversation history + clarification flow - Web UI: query understanding + pending_clarification state - Config: query_understanding.enabled/max_history/max_clarifications - 10 new tests (59 total, all passing) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>ebd834c -
feat: retrieve all relevant chunks using distance threshold
Instead of a fixed top_k count, retrieval now uses a distance threshold (max_distance=0.55). Queries up to 50 candidates from ChromaDB, then keeps only chunks with cosine distance below the threshold. This means: - Broad queries with lots of relevant docs get many chunks - Narrow queries with few matches get only the relevant ones - Irrelevant chunks are filtered out regardless of top_k New config field: retrieval.max_distance (default 0.55) top_k now acts as the candidate pool cap (default 50) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>0bb6123 -
fix: increase top_k to 10 and remove last [N] from verification prompt
Two fixes: 1. Increased default top_k from 5 to 10. With top_k=5, short queries like "China in 2024?" missed the relevant page (e.g., page 410) because generic intro pages ranked higher. More chunks = better chance of retrieving the right content. 2. Removed the last literal [N] from the verification prompt template (checklist item 1) to prevent any residual [N] in output. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>ca19369 -
fix: stop LLM from outputting literal [N] in citations
The citation rules used [N] as a placeholder example, but the LLM copied it literally alongside the actual number (e.g., "[N][1]"). Replaced [N] with plain English ("numbered endnote like [1], [2]") and added explicit instruction: "Do NOT output the literal text [N]." Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>a886455 -
fix: show actual file paths in references instead of chunk IDs
The system prompt told the LLM to use "source title or chunk ID" in references, so the LLM copied raw chunk IDs (CHUNK-LOCAL-001) instead of extracting the file name, path, and page number from the context. Now the prompt explicitly says: - Do NOT use chunk IDs in references - Extract actual file name + path from "From:" and "Path:" lines - For web sources, use author/year/title/DOI format - Includes concrete examples of correct formatting Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>31a8b1a -
fix: reduce over-refusal when relevant sources exist
The system prompt used a binary rule: "If the answer is NOT in the context, REFUSE." This caused the LLM to refuse answering even when relevant chunks were retrieved (e.g., CSV rows about China), because the question was broad and the data was sparse. The design doc says "An incomplete answer is always better than a fabricated one" but this guidance was missing from the prompt. Now the prompt distinguishes: - Context has relevant info → provide what you can (even if incomplete) - Context has NO relevant info → refuse This preserves all anti-hallucination guardrails while reducing false refusals. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>a916013 -
Initial commit: RAG Research Chatbot Template
Citation-verified research assistant with 6-layer anti-hallucination stack, 15 file format readers, 3 LLM providers, and dual CLI/Web interfaces. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>0017280
