devank2000 commited on
Commit
b3b0ccd
·
verified ·
1 Parent(s): 80a5ae3

initial commit

Browse files
.gitattributes CHANGED
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ project/assets/chatbot_avatar.png filter=lfs diff=lfs merge=lfs -text
LICENSE ADDED
@@ -0,0 +1,21 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ MIT License
2
+
3
+ Copyright (c) 2025 Giovanni Pasqualino
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
README.md CHANGED
@@ -1,13 +1,1461 @@
1
  ---
2
- title: Agentic Rag
3
- emoji: 🚀
4
  colorFrom: blue
5
- colorTo: red
6
  sdk: gradio
7
- sdk_version: 6.30.0
8
- python_version: '3.12'
9
- app_file: app.py
10
- pinned: false
11
  ---
12
 
13
- Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ title: Document Chat
3
+ emoji: 📚
4
  colorFrom: blue
5
+ colorTo: purple
6
  sdk: gradio
7
+ sdk_version: 6.29.0
8
+ python_version: 3.12.12
9
+ app_file: project/app.py
 
10
  ---
11
 
12
+ **Deployable app:** The Python/Gradio app now uses Groq (`GROQ_API_KEY`) for chat
13
+ and local Hugging Face embeddings for retrieval. Use `uv` to install the runtime
14
+ dependencies in `project/requirements.txt`. The learning notebooks retain their
15
+ Ollama examples. See [deployment instructions](project/README.md#hugging-face-spaces-recommended)
16
+ for Hugging Face Spaces, Render, secrets, and persistent storage.
17
+
18
+ <h1 align="center">Document Chat</h1>
19
+
20
+ <p align="center">
21
+ <strong>Build a modular Agentic RAG system with LangGraph, conversation memory, and human-in-the-loop query clarification</strong>
22
+ </p>
23
+
24
+ <p align="center">
25
+ <a href="#overview">Overview</a> •
26
+ <a href="#how-it-works">How It Works</a> •
27
+ <a href="#llm-provider-configuration">LLM Providers</a> •
28
+ <a href="#implementation">Implementation</a> •
29
+ <a href="#installation--usage">Installation & Usage</a> •
30
+ <a href="#troubleshooting">Troubleshooting</a>
31
+ </p>
32
+
33
+ ## Overview
34
+
35
+ This repository demonstrates how to build an **Agentic RAG (Retrieval-Augmented Generation)** system using LangGraph with minimal code. Most RAG tutorials show basic concepts but lack guidance on building modular, agent-driven systems — this project bridges that gap by providing **both learning materials and an extensible architecture**.
36
+
37
+ ### What's inside
38
+
39
+ | Feature | Description |
40
+ |---|---|
41
+ | 🗂️ **Hierarchical Indexing** | Search small chunks for precision, retrieve large Parent chunks for context |
42
+ | 🧠 **Conversation Memory** | Maintains context across questions for natural dialogue |
43
+ | ❓ **Query Clarification** | Rewrites ambiguous queries or pauses to ask the user for details |
44
+ | 🤖 **Agent Orchestration** | LangGraph coordinates the full retrieval and reasoning workflow |
45
+ | 🔀 **Multi-Agent Map-Reduce** | Decomposes complex queries into parallel sub-queries |
46
+ | ✅ **Self-Correction** | Re-queries automatically if initial results are insufficient |
47
+ | 🗜️ **Context Compression** | Keeps working memory lean across long retrieval loops |
48
+ | 🔍 **Observability** | Track LLM calls, tool usage, and graph execution with Langfuse |
49
+ | 📊 **Evaluation** | Evaluate retrieval and answer quality with RAGAS metrics |
50
+
51
+ ### 🎯 Two Ways to Use This Repo
52
+
53
+ **1️⃣ Learning Path: Interactive Notebook**
54
+
55
+ Step-by-step tutorial perfect for understanding core concepts. Start here if you're new to Agentic RAG or want to experiment quickly.
56
+
57
+ **2️⃣ Building Path: Modular Project**
58
+
59
+ Flexible architecture where each component can be independently adapted — LLM provider, embedding model, PDF converter, and agent workflow. The runnable app uses Groq; the notebooks retain their Ollama examples. Examples are included for Anthropic, OpenAI, and Google.
60
+
61
+ See [Modular Architecture](#modular-architecture) and [Installation & Usage](#installation--usage) to get started.
62
+
63
+ ## How It Works
64
+
65
+ ### Document Preparation: Hierarchical Indexing
66
+
67
+ Before queries can be processed, documents are split twice for optimal retrieval:
68
+
69
+ - **Parent Chunks**: Bounded large sections based on Markdown headers (H1, H2, H3)
70
+ - **Child Chunks**: Small, fixed-size pieces derived from parents
71
+
72
+ This combines the **precision of small chunks** for search with the **contextual richness of large chunks** for answer generation.
73
+
74
+ ---
75
+
76
+ ### Query Processing: Four-Stage Intelligent Workflow
77
+ ```
78
+ User Query → Conversation Summary → Query Rewriting → Query Clarification →
79
+ Parallel Agent Reasoning → Aggregation → Final Response
80
+ ```
81
+
82
+ **Stage 1 — Conversation Understanding:** Maintains a rolling summary and recent conversation history to preserve continuity without indefinitely increasing context size.
83
+
84
+ **Stage 2 — Query Clarification:** Resolves references ("How do I update it?" → "How do I update SQL?"), splits multi-part questions into focused sub-queries, detects unclear inputs, and rewrites queries for optimal retrieval. Pauses for human input when clarification is needed.
85
+
86
+ **Stage 3 — Intelligent Retrieval (Multi-Agent Map-Reduce):** Spawns parallel agent subgraphs — one per sub-query. Each agent searches child chunks, fetches parent chunks for context, self-corrects if results are insufficient, compresses context to avoid redundant fetches, and falls back gracefully if the search budget is exhausted.
87
+
88
+ > **Example:** *"What is JavaScript? What is Python?"* → 2 parallel agents execute simultaneously.
89
+
90
+ **Stage 4 — Response Generation:** Aggregates all agent responses into a single coherent answer.
91
+
92
+ ---
93
+
94
+ ## LLM Provider Configuration
95
+
96
+ This system is provider-agnostic: the runnable app uses Groq by default, and the chat model initialization can be adapted to any LLM provider available in [LangChain](https://python.langchain.com/docs/integrations/chat/). The notebooks retain their Ollama setup. The examples below cover alternative providers.
97
+
98
+ ### Groq (Application Default)
99
+
100
+ Copy `project/.env.example` to `project/.env` and set `GROQ_API_KEY`.
101
+ `GROQ_MODEL` defaults to `llama-3.3-70b-versatile`, with tool calling and
102
+ streaming through `langchain-groq`. Embeddings still run locally; Groq is
103
+ not used as an embedding service.
104
+
105
+ > **Note:** Model names change frequently. Always check the official documentation for the latest available models and their identifiers before deploying.
106
+
107
+ ### Ollama (Local)
108
+
109
+ ```bash
110
+ # Install Ollama from https://ollama.com
111
+ ollama pull granite4.1:8b
112
+ ```
113
+
114
+ ```python
115
+ from langchain_ollama import ChatOllama
116
+
117
+ llm = ChatOllama(model="granite4.1:8b", temperature=0, seed=42)
118
+ ```
119
+ > ⚠️ For reliable tool calling and instruction following, prefer models **8B+**. Smaller models may ignore retrieval instructions or hallucinate. See [Troubleshooting](#troubleshooting).
120
+
121
+ ---
122
+
123
+ ### Cloud Providers
124
+
125
+ <details>
126
+ <summary>Click to expand</summary>
127
+
128
+ **OpenAI GPT:**
129
+ ```bash
130
+ pip install -qU langchain-openai
131
+ ```
132
+ ```python
133
+ from langchain_openai import ChatOpenAI
134
+ import os
135
+
136
+ os.environ["OPENAI_API_KEY"] = "your-api-key-here"
137
+ llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)
138
+ ```
139
+
140
+ **Anthropic Claude:**
141
+ ```bash
142
+ pip install -qU langchain-anthropic
143
+ ```
144
+ ```python
145
+ from langchain_anthropic import ChatAnthropic
146
+ import os
147
+
148
+ os.environ["ANTHROPIC_API_KEY"] = "your-api-key-here"
149
+ llm = ChatAnthropic(model="claude-sonnet-4-5-20250929", temperature=0)
150
+ ```
151
+
152
+ **Google Gemini**
153
+ ```bash
154
+ pip install -qU langchain-google-genai
155
+ ```
156
+ ```python
157
+ import os
158
+ from langchain_google_genai import ChatGoogleGenerativeAI
159
+
160
+ os.environ["GOOGLE_API_KEY"] = "your-api-key-here"
161
+ llm = ChatGoogleGenerativeAI(model="gemini-2.5-flash", temperature=0)
162
+ ```
163
+ </details>
164
+
165
+ ---
166
+
167
+ ## Implementation
168
+
169
+ Additional details, extended explanations, and Langfuse observability are available in the **[notebook](notebooks/agentic_rag.ipynb)** and full project. The companion **[evaluation notebook](notebooks/evaluation.ipynb)** scores the final answers and the actual child/parent tool outputs used by the agent with direct RAGAS metric calls.
170
+
171
+ | Step | Description |
172
+ |------|-------------|
173
+ | 1 | [Initial Setup and Configuration](#step-1-initial-setup-and-configuration) |
174
+ | 2 | [Configure Vector Database](#step-2-configure-vector-database) |
175
+ | 3 | [PDFs to Markdown](#step-3-pdfs-to-markdown) |
176
+ | 4 | [Hierarchical Document Indexing](#step-4-hierarchical-document-indexing) |
177
+ | 5 | [Define Agent Tools](#step-5-define-agent-tools) |
178
+ | 6 | [Define System Prompts](#step-6-define-system-prompts) |
179
+ | 7 | [Define State and Data Models](#step-7-define-state-and-data-models) |
180
+ | 8 | [Agent Configuration](#step-8-agent-configuration) |
181
+ | 9 | [Build Graph Node and Edge Functions](#step-9-build-graph-node-and-edge-functions) |
182
+ | 10 | [Build the LangGraph Graphs](#step-10-build-the-langgraph-graphs) |
183
+ | 11 | [Create Chat Interface](#step-11-create-chat-interface) |
184
+
185
+ ### Step 1: Initial Setup and Configuration
186
+
187
+ Define paths and initialize core components.
188
+
189
+ ```python
190
+ import os
191
+ from pathlib import Path
192
+ from langchain_huggingface import HuggingFaceEmbeddings
193
+ from langchain_qdrant.fastembed_sparse import FastEmbedSparse
194
+ from qdrant_client import QdrantClient
195
+
196
+ DOCS_DIR = "docs" # Directory containing your pdf files
197
+ MARKDOWN_DIR = "markdown_docs" # Directory containing the pdfs converted to markdown
198
+ PARENT_STORE_PATH = "parent_store" # Directory for parent chunk JSON files
199
+ CHILD_COLLECTION = "document_child_chunks"
200
+ DEFAULT_RETRIEVAL_K = 7
201
+ CHILD_CHUNK_SEPARATOR = "\n\n<CHILD_CHUNK_BOUNDARY>\n\n"
202
+
203
+ os.makedirs(DOCS_DIR, exist_ok=True)
204
+ os.makedirs(MARKDOWN_DIR, exist_ok=True)
205
+ os.makedirs(PARENT_STORE_PATH, exist_ok=True)
206
+
207
+ from langchain_ollama import ChatOllama
208
+ llm = ChatOllama(model="granite4.1:8b", temperature=0, seed=42)
209
+
210
+ dense_embeddings = HuggingFaceEmbeddings(model_name="Qwen/Qwen3-Embedding-0.6B")
211
+ sparse_embeddings = FastEmbedSparse(model_name="Qdrant/bm25")
212
+
213
+ client = QdrantClient(path="qdrant_db")
214
+ ```
215
+
216
+ ---
217
+
218
+ ### Step 2: Configure Vector Database
219
+
220
+ Set up Qdrant to store child chunks with hybrid search capabilities.
221
+
222
+ ```python
223
+ from qdrant_client.http import models as qmodels
224
+ from langchain_qdrant import QdrantVectorStore
225
+ from langchain_qdrant.qdrant import RetrievalMode
226
+
227
+ embedding_dimension = len(dense_embeddings.embed_query("test"))
228
+
229
+ def ensure_collection(collection_name):
230
+ if not client.collection_exists(collection_name):
231
+ client.create_collection(
232
+ collection_name=collection_name,
233
+ vectors_config=qmodels.VectorParams(
234
+ size=embedding_dimension,
235
+ distance=qmodels.Distance.COSINE
236
+ ),
237
+ sparse_vectors_config={
238
+ "sparse": qmodels.SparseVectorParams()
239
+ },
240
+ )
241
+ ```
242
+
243
+ ---
244
+
245
+ ### Step 3: PDFs to Markdown
246
+
247
+ Convert the PDFs to Markdown. For more details about other techniques use this companion [notebook](notebooks/pdf_to_markdown.ipynb).
248
+
249
+ ```python
250
+ import os
251
+ import pymupdf.layout
252
+ import pymupdf4llm
253
+ from pathlib import Path
254
+ import glob
255
+
256
+ os.environ["TOKENIZERS_PARALLELISM"] = "false"
257
+
258
+ def pdf_to_markdown(pdf_path, output_dir):
259
+ doc = pymupdf.open(pdf_path)
260
+ md = pymupdf4llm.to_markdown(doc, header=False, footer=False, page_separators=True, ignore_images=True, write_images=False, image_path=None)
261
+ md_cleaned = md.encode('utf-8', errors='surrogatepass').decode('utf-8', errors='ignore')
262
+ output_path = Path(output_dir) / Path(doc.name).stem
263
+ Path(output_path).with_suffix(".md").write_bytes(md_cleaned.encode('utf-8'))
264
+
265
+ def pdfs_to_markdowns(path_pattern, overwrite: bool = False):
266
+ output_dir = Path(MARKDOWN_DIR)
267
+ output_dir.mkdir(parents=True, exist_ok=True)
268
+
269
+ for pdf_path in map(Path, glob.glob(path_pattern)):
270
+ md_path = (output_dir / pdf_path.stem).with_suffix(".md")
271
+ if overwrite or not md_path.exists():
272
+ pdf_to_markdown(pdf_path, output_dir)
273
+
274
+ pdfs_to_markdowns(f"{DOCS_DIR}/*.pdf")
275
+ ```
276
+
277
+ ---
278
+
279
+ ### Step 4: Hierarchical Document Indexing
280
+
281
+ Process documents with the Parent/Child splitting strategy.
282
+ ```python
283
+ import os
284
+ import glob
285
+ import json
286
+ from pathlib import Path
287
+ from langchain_text_splitters import MarkdownHeaderTextSplitter, RecursiveCharacterTextSplitter
288
+ ```
289
+
290
+ <details>
291
+ <summary>Parent & Child chunk processing functions</summary>
292
+
293
+ ```python
294
+ def merge_metadata(target, source, prepend=False):
295
+ for key, value in source.items():
296
+ if key not in target:
297
+ target[key] = value
298
+ else:
299
+ first, second = (value, target[key]) if prepend else (target[key], value)
300
+ values = [
301
+ item.strip()
302
+ for raw in (first, second)
303
+ for item in str(raw).split(" -> ")
304
+ if item.strip()
305
+ ]
306
+ target[key] = " -> ".join(dict.fromkeys(values))
307
+
308
+ def merge_small_parents(chunks, min_size):
309
+ if not chunks:
310
+ return []
311
+
312
+ merged, current = [], None
313
+
314
+ for chunk in chunks:
315
+ if current is None:
316
+ current = chunk
317
+ else:
318
+ current.page_content += "\n\n" + chunk.page_content
319
+ merge_metadata(current.metadata, chunk.metadata)
320
+
321
+ if len(current.page_content) >= min_size:
322
+ merged.append(current)
323
+ current = None
324
+
325
+ if current:
326
+ if merged:
327
+ merged[-1].page_content += "\n\n" + current.page_content
328
+ merge_metadata(merged[-1].metadata, current.metadata)
329
+ else:
330
+ merged.append(current)
331
+
332
+ return merged
333
+
334
+ def split_large_parents(chunks, max_size, overlap):
335
+ split_chunks = []
336
+
337
+ for chunk in chunks:
338
+ if len(chunk.page_content) <= max_size:
339
+ split_chunks.append(chunk)
340
+ else:
341
+ large_splitter = RecursiveCharacterTextSplitter(
342
+ chunk_size=max_size,
343
+ chunk_overlap=overlap
344
+ )
345
+ sub_chunks = large_splitter.split_documents([chunk])
346
+ split_chunks.extend(sub_chunks)
347
+
348
+ return split_chunks
349
+
350
+ def rebalance_pair(first, second, min_size, max_size):
351
+ combined = first.page_content.rstrip() + "\n\n" + second.page_content.lstrip()
352
+ lower = max(1, len(combined) - max_size)
353
+ upper = min(max_size, len(combined) - 1)
354
+ if len(combined) >= 2 * min_size:
355
+ lower = max(lower, min_size)
356
+ upper = min(upper, len(combined) - min_size)
357
+ preferred = min(max(len(combined) // 2, lower), upper)
358
+
359
+ split_at = preferred
360
+ for separator in ("\n\n", "\n", " "):
361
+ before = combined.rfind(separator, lower, preferred + 1)
362
+ after = combined.find(separator, preferred, upper + 1)
363
+ if before >= lower:
364
+ split_at = before
365
+ break
366
+ if after != -1:
367
+ split_at = after
368
+ break
369
+
370
+ left_text = combined[:split_at].rstrip()
371
+ right_text = combined[split_at:].lstrip()
372
+ if len(combined) >= 2 * min_size and (len(left_text) < min_size or len(right_text) < min_size):
373
+ split_at = preferred
374
+ left_text, right_text = combined[:split_at], combined[split_at:]
375
+ if not left_text or not right_text:
376
+ return first, second
377
+
378
+ metadata = dict(first.metadata)
379
+ merge_metadata(metadata, second.metadata)
380
+ first.page_content, first.metadata = left_text, dict(metadata)
381
+ second.page_content, second.metadata = right_text, dict(metadata)
382
+ return first, second
383
+
384
+ def clean_small_chunks(chunks, min_size, max_size):
385
+ cleaned = []
386
+
387
+ for i, chunk in enumerate(chunks):
388
+ if len(chunk.page_content) < min_size:
389
+ if cleaned and len(cleaned[-1].page_content) + 2 + len(chunk.page_content) <= max_size:
390
+ cleaned[-1].page_content += "\n\n" + chunk.page_content
391
+ merge_metadata(cleaned[-1].metadata, chunk.metadata)
392
+ elif i < len(chunks) - 1 and len(chunk.page_content) + 2 + len(chunks[i + 1].page_content) <= max_size:
393
+ chunks[i + 1].page_content = chunk.page_content + "\n\n" + chunks[i + 1].page_content
394
+ merge_metadata(chunks[i + 1].metadata, chunk.metadata, prepend=True)
395
+ else:
396
+ cleaned.append(chunk)
397
+ else:
398
+ cleaned.append(chunk)
399
+
400
+ for i, chunk in enumerate(cleaned):
401
+ if len(chunk.page_content) >= min_size or len(cleaned) == 1:
402
+ continue
403
+ if i < len(cleaned) - 1:
404
+ cleaned[i], cleaned[i + 1] = rebalance_pair(chunk, cleaned[i + 1], min_size, max_size)
405
+ else:
406
+ cleaned[i - 1], cleaned[i] = rebalance_pair(cleaned[i - 1], chunk, min_size, max_size)
407
+
408
+ return cleaned
409
+ ```
410
+
411
+ </details>
412
+
413
+ ```python
414
+ if client.collection_exists(CHILD_COLLECTION):
415
+ client.delete_collection(CHILD_COLLECTION)
416
+ ensure_collection(CHILD_COLLECTION)
417
+ else:
418
+ ensure_collection(CHILD_COLLECTION)
419
+
420
+ child_vector_store = QdrantVectorStore(
421
+ client=client,
422
+ collection_name=CHILD_COLLECTION,
423
+ embedding=dense_embeddings,
424
+ sparse_embedding=sparse_embeddings,
425
+ retrieval_mode=RetrievalMode.HYBRID,
426
+ sparse_vector_name="sparse"
427
+ )
428
+
429
+ def index_documents():
430
+ headers_to_split_on = [("#", "H1"), ("##", "H2"), ("###", "H3")]
431
+ parent_splitter = MarkdownHeaderTextSplitter(headers_to_split_on=headers_to_split_on, strip_headers=False)
432
+ child_chunk_size = 500
433
+ child_chunk_overlap = 100
434
+ min_parent_size = 2000
435
+ max_parent_size = 4000
436
+ if min_parent_size <= 0 or max_parent_size < min_parent_size:
437
+ raise ValueError("Parent chunk sizes must be positive and min_parent_size <= max_parent_size.")
438
+ if not 0 <= child_chunk_overlap < child_chunk_size:
439
+ raise ValueError("child_chunk_overlap must be smaller than child_chunk_size.")
440
+ child_splitter = RecursiveCharacterTextSplitter(
441
+ chunk_size=child_chunk_size,
442
+ chunk_overlap=child_chunk_overlap,
443
+ )
444
+
445
+ all_parent_pairs, all_child_chunks = [], []
446
+ md_files = sorted(glob.glob(os.path.join(MARKDOWN_DIR, "*.md")))
447
+
448
+ if not md_files:
449
+ return
450
+
451
+ for doc_path_str in md_files:
452
+ doc_path = Path(doc_path_str)
453
+ try:
454
+ with open(doc_path, "r", encoding="utf-8") as f:
455
+ md_text = f.read()
456
+ except Exception as e:
457
+ continue
458
+
459
+ parent_chunks = parent_splitter.split_text(md_text)
460
+ merged_parents = merge_small_parents(parent_chunks, min_parent_size)
461
+ split_parents = split_large_parents(merged_parents, max_parent_size, child_chunk_overlap)
462
+ cleaned_parents = clean_small_chunks(split_parents, min_parent_size, max_parent_size)
463
+ if any(len(chunk.page_content) > max_parent_size for chunk in cleaned_parents):
464
+ raise ValueError("Parent chunking produced an oversized chunk.")
465
+
466
+ for i, p_chunk in enumerate(cleaned_parents):
467
+ parent_id = f"{doc_path.stem}_p{i}"
468
+ p_chunk.metadata.update({"source": doc_path.stem + ".pdf", "parent_id": parent_id})
469
+ all_parent_pairs.append((parent_id, p_chunk))
470
+ children = child_splitter.split_documents([p_chunk])
471
+ all_child_chunks.extend(children)
472
+
473
+ if not all_child_chunks:
474
+ return
475
+
476
+ try:
477
+ child_vector_store.add_documents(all_child_chunks)
478
+ except Exception as e:
479
+ return
480
+
481
+ for item in os.listdir(PARENT_STORE_PATH):
482
+ os.remove(os.path.join(PARENT_STORE_PATH, item))
483
+
484
+ for parent_id, doc in all_parent_pairs:
485
+ doc_dict = {"page_content": doc.page_content, "metadata": doc.metadata}
486
+ filepath = os.path.join(PARENT_STORE_PATH, f"{parent_id}.json")
487
+ with open(filepath, "w", encoding="utf-8") as f:
488
+ json.dump(doc_dict, f, ensure_ascii=False, indent=2)
489
+
490
+ index_documents()
491
+ ```
492
+
493
+ ---
494
+
495
+ ### Step 5: Define Agent Tools
496
+
497
+ Create the retrieval tools the agent will use.
498
+
499
+ ```python
500
+ import json
501
+ from typing import List
502
+ from langchain_core.tools import tool
503
+
504
+ RETRIEVAL_SCORE_THRESHOLD = 0.4
505
+
506
+ @tool
507
+ def search_child_chunks(query: str, limit: int = DEFAULT_RETRIEVAL_K) -> str:
508
+ """Search document excerpts for evidence related to the user question.
509
+
510
+ Use this as the first retrieval step. Results include parent IDs, file
511
+ names, and short child-chunk excerpts. If excerpts are relevant but too
512
+ fragmented to answer confidently, call retrieve_parent_chunks with the
513
+ returned parent_id.
514
+
515
+ Args:
516
+ query: Focused search query with concrete keywords from the question.
517
+ limit: Maximum number of child chunks to return.
518
+ """
519
+ try:
520
+ results = child_vector_store.similarity_search(
521
+ query,
522
+ k=limit,
523
+ score_threshold=RETRIEVAL_SCORE_THRESHOLD,
524
+ )
525
+ if not results:
526
+ return "NO_RELEVANT_CHUNKS"
527
+
528
+ return CHILD_CHUNK_SEPARATOR.join([
529
+ f"Parent ID: {doc.metadata.get('parent_id', '')}\n"
530
+ f"File Name: {doc.metadata.get('source', '')}\n"
531
+ f"Content: {doc.page_content.strip()}"
532
+ for doc in results
533
+ ])
534
+
535
+ except Exception as e:
536
+ return f"RETRIEVAL_ERROR: {str(e)}"
537
+
538
+ @tool
539
+ def retrieve_parent_chunks(parent_id: str) -> str:
540
+ """Retrieve the full parent chunk for a relevant child search result.
541
+
542
+ Use this only after search_child_chunks returns a relevant parent_id and
543
+ the child excerpt needs more surrounding context. Do not call this for
544
+ parent IDs already available in compressed context.
545
+
546
+ Args:
547
+ parent_id: Parent chunk ID returned by search_child_chunks.
548
+ """
549
+ file_name = parent_id if parent_id.lower().endswith(".json") else f"{parent_id}.json"
550
+ path = os.path.join(PARENT_STORE_PATH, file_name)
551
+
552
+ if not os.path.exists(path):
553
+ return "NO_PARENT_DOCUMENT"
554
+
555
+ with open(path, "r", encoding="utf-8") as f:
556
+ data = json.load(f)
557
+
558
+ return (
559
+ f"Parent ID: {parent_id}\n"
560
+ f"File Name: {data.get('metadata', {}).get('source', 'unknown')}\n"
561
+ f"Content: {data.get('page_content', '').strip()}"
562
+ )
563
+
564
+ llm_with_tools = llm.bind_tools([search_child_chunks, retrieve_parent_chunks])
565
+ ```
566
+
567
+ ---
568
+
569
+ ### Step 6: Define System Prompts
570
+
571
+ Define the system prompts for conversation summarization, query rewriting, agent orchestration, context compression, fallback response, and answer aggregation.
572
+
573
+ <details>
574
+ <summary>Conversation Summary Prompt</summary>
575
+
576
+ ```python
577
+ def get_conversation_summary_prompt() -> str:
578
+ return """## Role
579
+ You are a compact memory manager for a retrieval-augmented chat assistant.
580
+
581
+ ## Context
582
+ The input contains an existing rolling summary plus older user/assistant messages that will be removed from raw chat history.
583
+
584
+ ## Instructions
585
+ - Merge the existing summary with the new older messages.
586
+ - Preserve context needed for future follow-up questions: topics, user preferences, important facts, unresolved questions, and referenced source file names.
587
+ - Discard greetings, tool calls, tool outputs, formatting chatter, duplicate details, and resolved misunderstandings.
588
+ - Keep the summary compact: 30-70 words unless more detail is essential.
589
+
590
+ ## Output
591
+ Return exactly one merged summary and nothing else.
592
+ Do not include labels such as "Updated summary:", "Previous summary:", or "New messages:".
593
+ Do not include both old and new summaries.
594
+ If there is no meaningful context, return an empty string.
595
+ """
596
+ ```
597
+
598
+ </details>
599
+
600
+ <details>
601
+ <summary>Query Rewrite Prompt</summary>
602
+
603
+ ```python
604
+ def get_rewrite_query_prompt() -> str:
605
+ return """## Role
606
+ You are a query rewriting specialist for document retrieval in a RAG system.
607
+
608
+ ## Instructions
609
+ - Rewrite the current query so it is clear, self-contained, and useful for retrieval.
610
+ - Use the conversation summary and recent conversation only to resolve vague follow-ups that refer to prior context.
611
+ - When an unresolved query and one or more user clarifications are provided, combine all of them into one self-contained retrieval query.
612
+ - If the query is a follow-up, integrate only the minimal context needed to make it self-contained.
613
+ - Preserve product names, file names, versions, acronyms, numbers, and technical terms exactly.
614
+ - If the user asks about a named topic, product, file, acronym, term, or concept, treat the question as clear even if it is new.
615
+ - Standalone named terms, acronyms, or concepts are valid retrieval queries; do not require prior conversation context.
616
+ - Split only truly separate information needs, with a maximum of 3 rewritten questions.
617
+
618
+ ## Clarification Boundary
619
+ Mark the query unclear only when it depends on an unresolved reference such as "it", "that", "this file", or "the previous one".
620
+ Do not mark a query unclear because the topic was not mentioned earlier.
621
+ Do not ask the user whether a new acronym or term is a typo; preserve it and search for it.
622
+
623
+ ## Constraints
624
+ Do not add facts, expand acronyms, invent context, or broaden the user's meaning.
625
+ """
626
+ ```
627
+
628
+ </details>
629
+
630
+ <details>
631
+ <summary>Orchestrator Prompt</summary>
632
+
633
+ ```python
634
+ def get_orchestrator_prompt() -> str:
635
+ return """## Role
636
+ You are a document-grounded research assistant for an agentic RAG system. Your job is to answer using retrieved document evidence, not general knowledge.
637
+
638
+ ## Available Context
639
+ - Current user question
640
+ - Optional compressed context from prior retrieval steps
641
+ - Tools for searching child chunks and loading full parent chunks
642
+
643
+ ## Tool Guidance
644
+ - Search documents before answering unless compressed context already contains enough evidence.
645
+ - Use 'search_child_chunks' for missing or uncovered parts of the question.
646
+ - If searched or retrieved context is not useful, use the tools again with a different, simpler query or a more relevant parent chunk.
647
+ - Continue tool use until the available evidence is enough, tools stop adding useful information, or the operation limit is reached.
648
+ - Do not repeat search queries or parent IDs listed in compressed context.
649
+ - Do not retrieve the same parent ID twice.
650
+
651
+ ## Response Framework
652
+ 1. Check compressed context for already-known evidence and already-used searches or parents.
653
+ 2. Search for missing evidence.
654
+ 3. Retrieve parent chunks only when child excerpts are relevant but too fragmented.
655
+ 4. Answer using the exact terms and scope in the retrieved evidence.
656
+ 5. If evidence is incomplete, state the specific gap.
657
+
658
+ ## Output
659
+ - Start directly with the substantive answer. Do not start with generic headings such as "Answer", "Final answer", or "Response".
660
+ - Provide the direct answer plus the key supporting details from retrieved evidence; avoid one-sentence fragments unless only one fact is available.
661
+ - Do not mention internal tool calls or reasoning.
662
+ - When sources exist, end with a Sources section in exactly this format:
663
+ Sources:
664
+ - filename.ext
665
+ - Put each source filename on its own bullet line. Never write sources inline, such as "Sources: filename.pdf".
666
+ - Do not invent or infer source filenames.
667
+ - Strip descriptions after file names, including text in parentheses.
668
+ """
669
+ ```
670
+
671
+ </details>
672
+
673
+ <details>
674
+ <summary>Fallback Response Prompt</summary>
675
+
676
+ ```python
677
+ def get_fallback_response_prompt() -> str:
678
+ return """## Role
679
+ You are a constrained evidence synthesizer for a retrieval-augmented assistant after the research loop reached its limit.
680
+
681
+ ## Available Context
682
+ - Compressed Research Context from earlier retrieval steps
683
+ - Retrieved Data from current tool outputs
684
+
685
+ ## Instructions
686
+ - Use only explicit facts from the provided context.
687
+ - Start directly with the substantive answer. Do not start with generic headings such as "Answer", "Final answer", or "Response".
688
+ - Prefer current Retrieved Data over compressed context if they conflict.
689
+ - If the answer is incomplete, mention only the missing parts that matter to the user query.
690
+ - Do not describe the retrieval process, limits, or internal reasoning.
691
+ - Be concise: answer in 1-3 short paragraphs or up to 5 bullets unless the user asks for detail.
692
+ - Provide the direct answer plus the key supporting details from retrieved evidence; avoid one-sentence fragments unless only one fact is available.
693
+ - End with a Sources section only when actual source file names are explicitly present in the context.
694
+ - Use exactly this format:
695
+ Sources:
696
+ - filename.ext
697
+ - Put each source filename on its own bullet line. Never write sources inline, such as "Sources: filename.pdf".
698
+ - Include only bare file names with extensions such as .pdf, .docx, .txt, or .md.
699
+ - Do not invent or infer source filenames.
700
+ """
701
+ ```
702
+
703
+ </details>
704
+
705
+ <details>
706
+ <summary>Context Compression Prompt</summary>
707
+
708
+ ```python
709
+ def get_context_compression_prompt() -> str:
710
+ return """## Role
711
+ You are a research context compressor for an agentic RAG system.
712
+
713
+ ## Instructions
714
+ - Keep only facts relevant to answering the user question.
715
+ - Preserve exact names, figures, versions, technical terms, configuration details, and source file names.
716
+ - Remove duplicates, tool chatter, search query wording, parent IDs, chunk IDs, and other internal identifiers.
717
+ - Organize findings by source file. Each source section heading must be the real filename found in retrieved data.
718
+ - Add a Gaps section only for missing information relevant to the question.
719
+ - Target 400-600 words. If there is too much content, keep the most answer-critical facts.
720
+
721
+ ## Output
722
+ Return only Markdown in this structure:
723
+ # Research Context Summary
724
+
725
+ ## Focus
726
+ [Brief technical restatement of the question]
727
+
728
+ ## Structured Findings
729
+ For each source file, add a level-3 heading with its real filename and bullet the directly relevant facts below it.
730
+
731
+ ## Gaps
732
+ - Missing or incomplete aspects
733
+ """
734
+ ```
735
+
736
+ </details>
737
+
738
+ <details>
739
+ <summary>Aggregation Prompt</summary>
740
+
741
+ ```python
742
+ def get_aggregation_prompt() -> str:
743
+ return """## Role
744
+ You are a final-answer synthesizer for a retrieval-augmented assistant.
745
+
746
+ ## Instructions
747
+ - Use only information present in the retrieved answers.
748
+ - Start directly with the substantive answer. Do not start with generic headings such as "Answer", "Final answer", or "Response".
749
+ - Preserve important names, numbers, versions, examples, and definitions.
750
+ - Do not expand acronyms or interpret terms unless the sources do it.
751
+ - If answers conflict, mention the conflict plainly.
752
+ - Be concise: answer in 1-3 short paragraphs or up to 5 bullets unless the user asks for detail.
753
+ - Provide the direct answer plus the key supporting details from retrieved evidence; avoid one-sentence fragments unless only one fact is available.
754
+ - End with a Sources section only when actual source file names are explicitly present in the retrieved answers.
755
+ - Use exactly this format:
756
+ Sources:
757
+ - filename.ext
758
+ - Put each source filename on its own bullet line. Never write sources inline, such as "Sources: filename.pdf".
759
+ - Include only bare file names with extensions such as .pdf, .docx, .txt, or .md.
760
+ - Do not invent or infer source filenames.
761
+ - If no useful information is available, say: "I couldn't find any information to answer your question in the available sources."
762
+ """
763
+ ```
764
+
765
+ </details>
766
+
767
+ ---
768
+
769
+ ### Step 7: Define State and Data Models
770
+
771
+ Create the state structure for conversation tracking and agent execution.
772
+
773
+ ```python
774
+ from langgraph.graph import MessagesState
775
+ from pydantic import BaseModel, Field
776
+ from typing import List, Annotated, Set
777
+ import operator
778
+
779
+ def accumulate_or_reset(existing: List[dict], new: List[dict]) -> List[dict]:
780
+ if new and any(item.get('__reset__') for item in new):
781
+ return []
782
+ return existing + new
783
+
784
+ def set_union(a: Set[str], b: Set[str]) -> Set[str]:
785
+ return a | b
786
+
787
+ def append_unique(existing: List[str], new: List[str]) -> List[str]:
788
+ return list(dict.fromkeys(existing + new))
789
+
790
+ class State(MessagesState):
791
+ questionIsClear: bool = False
792
+ conversation_summary: str = ""
793
+ originalQuery: str = ""
794
+ pendingQuery: str = ""
795
+ pendingClarifications: List[str] = []
796
+ rewrittenQuestions: List[str] = []
797
+ agent_answers: Annotated[List[dict], accumulate_or_reset] = []
798
+
799
+ class AgentState(MessagesState):
800
+ tool_call_count: Annotated[int, operator.add] = 0
801
+ iteration_count: Annotated[int, operator.add] = 0
802
+ question: str = ""
803
+ question_index: int = 0
804
+ context_summary: str = ""
805
+ retrieval_keys: Annotated[Set[str], set_union] = set()
806
+ retrieved_contexts: Annotated[List[str], append_unique] = []
807
+ final_answer: str = ""
808
+ agent_answers: List[dict] = []
809
+
810
+ class QueryAnalysis(BaseModel):
811
+ is_clear: bool = Field(description="Indicates if the user's question is clear and answerable.")
812
+ questions: List[str] = Field(description="List of rewritten, self-contained questions.")
813
+ clarification_needed: str = Field(description="Explanation if the question is unclear.")
814
+ ```
815
+
816
+ ---
817
+
818
+ ### Step 8: Agent Configuration
819
+
820
+ Hard limits on tool calls and iterations prevent infinite loops. Token counting (via `tiktoken`) drives context compression decisions.
821
+
822
+ ```python
823
+ import tiktoken
824
+ from functools import lru_cache
825
+
826
+ MAX_TOOL_CALLS = 8 # Maximum tool calls per agent run
827
+ MAX_ITERATIONS = 10 # Maximum agent loop iterations
828
+ BASE_TOKEN_THRESHOLD = 2000 # Initial token threshold for compression
829
+ TOKEN_GROWTH_FACTOR = 0.9 # Multiplier applied after each compression
830
+
831
+ @lru_cache(maxsize=1)
832
+ def _get_token_encoding():
833
+ try:
834
+ return tiktoken.encoding_for_model("gpt-4")
835
+ except Exception:
836
+ try:
837
+ return tiktoken.get_encoding("cl100k_base")
838
+ except Exception:
839
+ return None
840
+
841
+ def estimate_context_tokens(messages: list) -> int:
842
+ contents = [
843
+ str(msg.content)
844
+ for msg in messages
845
+ if hasattr(msg, "content") and msg.content
846
+ ]
847
+ encoding = _get_token_encoding()
848
+ if encoding is None:
849
+ return sum(max(1, len(content) // 4) for content in contents)
850
+ return sum(len(encoding.encode(content)) for content in contents)
851
+ ```
852
+
853
+ ---
854
+
855
+ ### Step 9: Build Graph Node and Edge Functions
856
+
857
+ Create the processing nodes and edges for the LangGraph workflow.
858
+
859
+ #### Main Graph Nodes & Edges
860
+ ```python
861
+ from langgraph.types import Send, Command
862
+ from langchain_core.messages import HumanMessage, AIMessage, SystemMessage, RemoveMessage, ToolMessage
863
+ from typing import Literal, Set
864
+
865
+ MAIN_HISTORY_MESSAGES_TO_KEEP = 4
866
+ if MAIN_HISTORY_MESSAGES_TO_KEEP < 2:
867
+ raise ValueError("MAIN_HISTORY_MESSAGES_TO_KEEP must be at least 2.")
868
+ PRE_ANSWER_HISTORY_MESSAGES_TO_KEEP = max(MAIN_HISTORY_MESSAGES_TO_KEEP - 1, 0)
869
+
870
+ def _is_plain_conversation_message(msg) -> bool:
871
+ return (
872
+ isinstance(msg, (HumanMessage, AIMessage))
873
+ and not getattr(msg, "tool_calls", None)
874
+ and not getattr(msg, "name", None)
875
+ )
876
+
877
+ def _name_internal_message(message, name):
878
+ """Tag a subgraph-only message so it is not treated as chat history."""
879
+ return message.model_copy(update={"name": name})
880
+
881
+ def _retrieval_contexts(messages) -> list[str]:
882
+ contexts = []
883
+ ignored_prefixes = (
884
+ "NO_RELEVANT_CHUNKS",
885
+ "NO_PARENT_DOCUMENT",
886
+ "RETRIEVAL_ERROR:",
887
+ "PARENT_RETRIEVAL_ERROR:",
888
+ )
889
+ for message in messages:
890
+ if not isinstance(message, ToolMessage):
891
+ continue
892
+ content = str(message.content).strip()
893
+ if content and not content.startswith(ignored_prefixes):
894
+ parts = content.split(CHILD_CHUNK_SEPARATOR) if message.name == "search_child_chunks" else [content]
895
+ contexts.extend(part for part in parts if part)
896
+ return list(dict.fromkeys(contexts))
897
+
898
+ def _format_conversation(messages) -> str:
899
+ lines = []
900
+ for msg in messages:
901
+ role = "User" if isinstance(msg, HumanMessage) else "Assistant"
902
+ lines.append(f"{role}: {msg.content}")
903
+ return "\n".join(lines)
904
+
905
+ def _remove_messages_not_in(messages, keep_ids):
906
+ removals = []
907
+ for msg in messages:
908
+ msg_id = getattr(msg, "id", None)
909
+ if isinstance(msg, SystemMessage) or not msg_id:
910
+ continue
911
+ if msg_id not in keep_ids:
912
+ removals.append(RemoveMessage(id=msg_id))
913
+ return removals
914
+
915
+ def _recent_conversation(messages, pending_query="") -> list:
916
+ """Return recent context before the current user message."""
917
+ plain_messages = [msg for msg in messages if _is_plain_conversation_message(msg)]
918
+ recent_messages = plain_messages[:-1]
919
+
920
+ if pending_query:
921
+ for index in range(len(recent_messages) - 1, -1, -1):
922
+ msg = recent_messages[index]
923
+ if isinstance(msg, HumanMessage) and str(msg.content).strip() == pending_query:
924
+ return recent_messages[:index]
925
+
926
+ return recent_messages
927
+
928
+ def summarize_history(state: State):
929
+ messages = state.get("messages", [])
930
+ updates = {"agent_answers": [{"__reset__": True}]}
931
+
932
+ if not messages:
933
+ return updates
934
+
935
+ plain_messages = [msg for msg in messages if _is_plain_conversation_message(msg)]
936
+ keep_count = PRE_ANSWER_HISTORY_MESSAGES_TO_KEEP
937
+ messages_to_summarize = plain_messages[:-keep_count] if len(plain_messages) > keep_count else []
938
+ keep_ids = {getattr(msg, "id", None) for msg in plain_messages[-keep_count:]}
939
+ keep_ids.discard(None)
940
+
941
+ removals = _remove_messages_not_in(messages, keep_ids)
942
+ if removals:
943
+ updates["messages"] = removals
944
+
945
+ if not messages_to_summarize:
946
+ return updates
947
+
948
+ existing_summary = state.get("conversation_summary", "").strip()
949
+ conversation = "Existing summary:\n"
950
+ conversation += f"{existing_summary or '(none)'}\n\n"
951
+ conversation += "New messages to merge into the summary:\n"
952
+ conversation += _format_conversation(messages_to_summarize)
953
+
954
+ summary_response = llm.invoke([
955
+ SystemMessage(content=get_conversation_summary_prompt()),
956
+ HumanMessage(content=conversation),
957
+ ])
958
+ updates["conversation_summary"] = summary_response.content.strip()
959
+ return updates
960
+
961
+ def rewrite_query(state: State):
962
+ last_message = state["messages"][-1]
963
+ current_query = str(last_message.content).strip()
964
+ conversation_summary = state.get("conversation_summary", "").strip()
965
+ pending_query = state.get("pendingQuery", "").strip()
966
+ pending_clarifications = state.get("pendingClarifications", [])
967
+ recent_messages = _recent_conversation(state["messages"], pending_query)
968
+
969
+ context_parts = []
970
+ if conversation_summary:
971
+ context_parts.append(f"Conversation Summary:\n{conversation_summary}")
972
+ if recent_messages:
973
+ context_parts.append(f"Recent Conversation:\n{_format_conversation(recent_messages)}")
974
+
975
+ if pending_query:
976
+ clarifications = [*pending_clarifications, current_query]
977
+ clarification_text = "\n".join(
978
+ f"{index}. {value}" for index, value in enumerate(clarifications, start=1)
979
+ )
980
+ context_parts.append(
981
+ f"Unresolved User Query:\n{pending_query}\n\n"
982
+ f"User Clarifications:\n{clarification_text}"
983
+ )
984
+ original_query = f"{pending_query}\nClarifications:\n{clarification_text}"
985
+ else:
986
+ clarifications = []
987
+ context_parts.append(f"User Query:\n{current_query}")
988
+ original_query = current_query
989
+
990
+ context_section = "\n\n".join(context_parts)
991
+ llm_with_structure = llm.with_structured_output(QueryAnalysis)
992
+ response = llm_with_structure.invoke([SystemMessage(content=get_rewrite_query_prompt()), HumanMessage(content=context_section)])
993
+ clarification_message_update = (
994
+ [_name_internal_message(last_message, "clarification_response")]
995
+ if pending_query else []
996
+ )
997
+
998
+ if response.questions and response.is_clear:
999
+ return {
1000
+ "questionIsClear": True,
1001
+ "originalQuery": original_query,
1002
+ "pendingQuery": "",
1003
+ "pendingClarifications": [],
1004
+ "rewrittenQuestions": response.questions,
1005
+ "messages": clarification_message_update,
1006
+ }
1007
+
1008
+ clarification = response.clarification_needed if response.clarification_needed and len(response.clarification_needed.strip()) > 10 else "I need more information to understand your question."
1009
+ return {
1010
+ "questionIsClear": False,
1011
+ "originalQuery": "",
1012
+ "pendingQuery": pending_query or current_query,
1013
+ "pendingClarifications": clarifications,
1014
+ "rewrittenQuestions": [],
1015
+ "messages": clarification_message_update + [
1016
+ AIMessage(content=clarification, name="clarification")
1017
+ ],
1018
+ }
1019
+
1020
+ def request_clarification(state: State):
1021
+ return {}
1022
+
1023
+ def route_after_rewrite(state: State) -> Literal["request_clarification", "agent"]:
1024
+ if not state.get("questionIsClear", False):
1025
+ return "request_clarification"
1026
+ else:
1027
+ return [
1028
+ Send("agent", {"question": query, "question_index": idx, "messages": []})
1029
+ for idx, query in enumerate(state["rewrittenQuestions"])
1030
+ ]
1031
+
1032
+ def aggregate_answers(state: State):
1033
+ messages = state.get("messages", [])
1034
+ plain_messages = [msg for msg in messages if _is_plain_conversation_message(msg)]
1035
+ keep_ids = {getattr(msg, "id", None) for msg in plain_messages[-PRE_ANSWER_HISTORY_MESSAGES_TO_KEEP:]}
1036
+ keep_ids.discard(None)
1037
+ removals = _remove_messages_not_in(messages, keep_ids)
1038
+
1039
+ if not state.get("agent_answers"):
1040
+ return {"messages": removals + [AIMessage(content="No answers were generated.")]}
1041
+
1042
+ sorted_answers = sorted(state["agent_answers"], key=lambda x: x["index"])
1043
+
1044
+ formatted_answers = ""
1045
+ for i, ans in enumerate(sorted_answers, start=1):
1046
+ formatted_answers += (f"\nRetrieved response {i}:\n"f"{ans['answer']}\n")
1047
+
1048
+ user_message = HumanMessage(content=f"""Original user question: {state["originalQuery"]}\nRetrieved answers:{formatted_answers}""")
1049
+ synthesis_response = llm.invoke([SystemMessage(content=get_aggregation_prompt()), user_message])
1050
+ return {"messages": removals + [AIMessage(content=synthesis_response.content)]}
1051
+ ```
1052
+
1053
+ ---
1054
+
1055
+ #### Agent Subgraph Nodes & Edges
1056
+ ```python
1057
+ def orchestrator(state: AgentState):
1058
+ context_summary = state.get("context_summary", "").strip()
1059
+ sys_msg = SystemMessage(content=get_orchestrator_prompt())
1060
+ summary_injection = (
1061
+ [HumanMessage(content=f"[COMPRESSED CONTEXT FROM PRIOR RESEARCH]\n\n{context_summary}")]
1062
+ if context_summary else []
1063
+ )
1064
+ if not state.get("messages"):
1065
+ human_msg = HumanMessage(content=state["question"], name="agent_question")
1066
+ force_search = HumanMessage(content="YOU MUST CALL 'search_child_chunks' AS THE FIRST STEP TO ANSWER THIS QUESTION.")
1067
+ response = llm_with_tools.invoke([sys_msg] + summary_injection + [human_msg, force_search])
1068
+ response = _name_internal_message(response, "agent_response")
1069
+ return {"messages": [human_msg, response], "tool_call_count": len(response.tool_calls or []), "iteration_count": 1}
1070
+
1071
+ response = llm_with_tools.invoke([sys_msg] + summary_injection + state["messages"])
1072
+ response = _name_internal_message(response, "agent_response")
1073
+ tool_calls = response.tool_calls if hasattr(response, "tool_calls") else []
1074
+ return {"messages": [response], "tool_call_count": len(tool_calls) if tool_calls else 0, "iteration_count": 1}
1075
+
1076
+ def route_after_orchestrator_call(state: AgentState) -> Literal["tools", "fallback_response", "collect_answer"]:
1077
+ iteration = state.get("iteration_count", 0)
1078
+ tool_count = state.get("tool_call_count", 0)
1079
+
1080
+ last_message = state["messages"][-1]
1081
+ tool_calls = getattr(last_message, "tool_calls", None) or []
1082
+
1083
+ if not tool_calls:
1084
+ return "collect_answer"
1085
+
1086
+ # Accept a final answer at the iteration boundary, but do not execute
1087
+ # tool calls that would exceed the configured research budget.
1088
+ if iteration >= MAX_ITERATIONS or tool_count > MAX_TOOL_CALLS:
1089
+ return "fallback_response"
1090
+
1091
+ return "tools"
1092
+
1093
+ def fallback_response(state: AgentState):
1094
+ seen = set()
1095
+ unique_contents = []
1096
+ for m in state["messages"]:
1097
+ if isinstance(m, ToolMessage) and m.content not in seen:
1098
+ unique_contents.append(m.content)
1099
+ seen.add(m.content)
1100
+
1101
+ context_summary = state.get("context_summary", "").strip()
1102
+
1103
+ context_parts = []
1104
+ if context_summary:
1105
+ context_parts.append(f"## Compressed Research Context (from prior iterations)\n\n{context_summary}")
1106
+ if unique_contents:
1107
+ context_parts.append(
1108
+ "## Retrieved Data (current iteration)\n\n" +
1109
+ "\n\n".join(f"--- DATA SOURCE {i} ---\n{content}" for i, content in enumerate(unique_contents, 1))
1110
+ )
1111
+
1112
+ context_text = "\n\n".join(context_parts) if context_parts else "No data was retrieved from the documents."
1113
+
1114
+ prompt_content = (
1115
+ f"USER QUERY: {state.get('question')}\n\n"
1116
+ f"{context_text}\n\n"
1117
+ f"INSTRUCTION:\nProvide the best possible answer using only the data above."
1118
+ )
1119
+ response = llm.invoke([SystemMessage(content=get_fallback_response_prompt()), HumanMessage(content=prompt_content)])
1120
+ response = _name_internal_message(response, "agent_response")
1121
+ return {"messages": [response]}
1122
+
1123
+ def should_compress_context(state: AgentState) -> Command[Literal["compress_context", "orchestrator"]]:
1124
+ messages = state["messages"]
1125
+
1126
+ new_ids: Set[str] = set()
1127
+ for msg in reversed(messages):
1128
+ if isinstance(msg, AIMessage) and getattr(msg, "tool_calls", None):
1129
+ for tc in msg.tool_calls:
1130
+ if tc["name"] == "retrieve_parent_chunks":
1131
+ raw = tc["args"].get("parent_id") or tc["args"].get("id") or tc["args"].get("ids") or []
1132
+ if isinstance(raw, str):
1133
+ new_ids.add(f"parent::{raw}")
1134
+ else:
1135
+ new_ids.update(f"parent::{r}" for r in raw)
1136
+
1137
+ elif tc["name"] == "search_child_chunks":
1138
+ query = tc["args"].get("query", "")
1139
+ if query:
1140
+ new_ids.add(f"search::{query}")
1141
+ break
1142
+
1143
+ updated_ids = state.get("retrieval_keys", set()) | new_ids
1144
+
1145
+ current_token_messages = estimate_context_tokens(messages)
1146
+ current_token_summary = estimate_context_tokens([HumanMessage(content=state.get("context_summary", ""))])
1147
+ current_tokens = current_token_messages + current_token_summary
1148
+
1149
+ max_allowed = BASE_TOKEN_THRESHOLD + int(current_token_summary * TOKEN_GROWTH_FACTOR)
1150
+
1151
+ goto = "compress_context" if current_tokens > max_allowed else "orchestrator"
1152
+ return Command(
1153
+ update={
1154
+ "retrieval_keys": updated_ids,
1155
+ "retrieved_contexts": _retrieval_contexts(messages),
1156
+ },
1157
+ goto=goto,
1158
+ )
1159
+
1160
+ def compress_context(state: AgentState):
1161
+ messages = state["messages"]
1162
+ existing_summary = state.get("context_summary", "").strip()
1163
+
1164
+ if not messages:
1165
+ return {}
1166
+
1167
+ conversation_text = f"USER QUESTION:\n{state.get('question')}\n\nConversation to compress:\n\n"
1168
+ if existing_summary:
1169
+ conversation_text += f"[PRIOR COMPRESSED CONTEXT]\n{existing_summary}\n\n"
1170
+
1171
+ for msg in messages[1:]:
1172
+ if isinstance(msg, AIMessage):
1173
+ tool_calls_info = ""
1174
+ if getattr(msg, "tool_calls", None):
1175
+ calls = ", ".join(f"{tc['name']}({tc['args']})" for tc in msg.tool_calls)
1176
+ tool_calls_info = f" | Tool calls: {calls}"
1177
+ conversation_text += f"[ASSISTANT{tool_calls_info}]\n{msg.content or '(tool call only)'}\n\n"
1178
+ elif isinstance(msg, ToolMessage):
1179
+ tool_name = getattr(msg, "name", "tool")
1180
+ conversation_text += f"[TOOL RESULT — {tool_name}]\n{msg.content}\n\n"
1181
+
1182
+ summary_response = llm.invoke([SystemMessage(content=get_context_compression_prompt()), HumanMessage(content=conversation_text)])
1183
+ new_summary = summary_response.content
1184
+
1185
+ retrieved_ids: Set[str] = state.get("retrieval_keys", set())
1186
+ if retrieved_ids:
1187
+ parent_ids = sorted(r for r in retrieved_ids if r.startswith("parent::"))
1188
+ search_queries = sorted(r.replace("search::", "") for r in retrieved_ids if r.startswith("search::"))
1189
+
1190
+ block = "\n\n---\n**Already executed (do NOT repeat):**\n"
1191
+ if parent_ids:
1192
+ block += "Parent chunks retrieved:\n" + "\n".join(f"- {p.replace('parent::', '')}" for p in parent_ids) + "\n"
1193
+ if search_queries:
1194
+ block += "Search queries already run:\n" + "\n".join(f"- {q}" for q in search_queries) + "\n"
1195
+ new_summary += block
1196
+
1197
+ return {"context_summary": new_summary, "messages": [RemoveMessage(id=m.id) for m in messages[1:]]}
1198
+
1199
+ def collect_answer(state: AgentState):
1200
+ last_message = state["messages"][-1]
1201
+ is_valid = isinstance(last_message, AIMessage) and last_message.content and not last_message.tool_calls
1202
+ answer = last_message.content if is_valid else "Unable to generate an answer."
1203
+ return {
1204
+ "final_answer": answer,
1205
+ "agent_answers": [{
1206
+ "index": state["question_index"],
1207
+ "question": state["question"],
1208
+ "answer": answer,
1209
+ "contexts": state.get("retrieved_contexts", []),
1210
+ }]
1211
+ }
1212
+ ```
1213
+
1214
+ **Why this architecture?**
1215
+ - **Summarization** maintains conversational context without overwhelming the LLM
1216
+ - **Query rewriting** ensures search queries are precise and unambiguous, using context intelligently
1217
+ - **Human-in-the-loop** catches unclear queries before wasting any retrieval resources
1218
+ - **Parallel execution** via `Send` API spawns independent agent subgraphs for each sub-question simultaneously
1219
+ - **Context compression** keeps the agent's working memory lean across long retrieval loops, preventing redundant fetches
1220
+ - **Fallback response** ensures graceful degradation — the agent always returns something useful even when the budget runs out
1221
+ - **Answer collection & aggregation** extracts clean final answers from agents and aggregates them into a single coherent response
1222
+ ---
1223
+
1224
+ ### Step 10: Build the LangGraph Graphs
1225
+
1226
+ Assemble the complete workflow graph with conversation memory and multi-agent architecture.
1227
+
1228
+ <details>
1229
+ <summary>Architecture Flow Diagram</summary>
1230
+
1231
+ ```mermaid
1232
+ flowchart TD
1233
+ start(["__start__"]) --> summarize_history("summarize_history")
1234
+ summarize_history --> rewrite_query("rewrite_query")
1235
+
1236
+ rewrite_query -.-> request_clarification("request_clarification<br/>───────────────<br/>__interrupt = before")
1237
+ request_clarification --> rewrite_query
1238
+
1239
+ rewrite_query -.-> agent_start
1240
+
1241
+ subgraph agent ["agent"]
1242
+ agent_start(["__start__"]) --> orchestrator("orchestrator")
1243
+ orchestrator -.-> tools("tools")
1244
+ orchestrator -.-> fallback_response("fallback_response")
1245
+ orchestrator -.-> collect_answer("collect_answer")
1246
+
1247
+ tools --> should_compress_context("should_compress_context")
1248
+ should_compress_context -.-> compress_context("compress_context")
1249
+ should_compress_context -.-> orchestrator
1250
+ compress_context --> orchestrator
1251
+
1252
+ fallback_response --> collect_answer
1253
+ end
1254
+
1255
+ collect_answer --> aggregate_answers("aggregate_answers")
1256
+ aggregate_answers --> end_node(["__end__"])
1257
+ ```
1258
+
1259
+ </details>
1260
+
1261
+ ```python
1262
+ from langgraph.graph import START, END, StateGraph
1263
+ from langgraph.prebuilt import ToolNode
1264
+ from langgraph.checkpoint.memory import InMemorySaver
1265
+
1266
+ checkpointer = InMemorySaver()
1267
+
1268
+ agent_builder = StateGraph(AgentState)
1269
+ agent_builder.add_node(orchestrator)
1270
+ agent_builder.add_node("tools", ToolNode([search_child_chunks, retrieve_parent_chunks]))
1271
+ agent_builder.add_node(compress_context)
1272
+ agent_builder.add_node(fallback_response)
1273
+ agent_builder.add_node(should_compress_context)
1274
+ agent_builder.add_node(collect_answer)
1275
+
1276
+ agent_builder.add_edge(START, "orchestrator")
1277
+ agent_builder.add_conditional_edges("orchestrator", route_after_orchestrator_call, {"tools": "tools", "fallback_response": "fallback_response", "collect_answer": "collect_answer"})
1278
+ agent_builder.add_edge("tools", "should_compress_context")
1279
+ agent_builder.add_edge("compress_context", "orchestrator")
1280
+ agent_builder.add_edge("fallback_response", "collect_answer")
1281
+ agent_builder.add_edge("collect_answer", END)
1282
+ agent_subgraph = agent_builder.compile()
1283
+
1284
+ graph_builder = StateGraph(State)
1285
+ graph_builder.add_node(summarize_history)
1286
+ graph_builder.add_node(rewrite_query)
1287
+ graph_builder.add_node(request_clarification)
1288
+ graph_builder.add_node("agent", agent_subgraph)
1289
+ graph_builder.add_node(aggregate_answers)
1290
+
1291
+ graph_builder.add_edge(START, "summarize_history")
1292
+ graph_builder.add_edge("summarize_history", "rewrite_query")
1293
+ graph_builder.add_conditional_edges("rewrite_query", route_after_rewrite)
1294
+ graph_builder.add_edge("request_clarification", "rewrite_query")
1295
+ graph_builder.add_edge(["agent"], "aggregate_answers")
1296
+ graph_builder.add_edge("aggregate_answers", END)
1297
+
1298
+ agent_graph = graph_builder.compile(checkpointer=checkpointer, interrupt_before=["request_clarification"])
1299
+ ```
1300
+
1301
+ **Graph architecture explained:**
1302
+
1303
+ **Agent Subgraph** (processes individual questions):
1304
+ - START → `orchestrator` (invoke LLM with tools)
1305
+ - `orchestrator` → `tools` (if tool calls needed) OR `fallback_response` (if budget exhausted) OR `collect_answer` (if done)
1306
+ - `tools` → `should_compress_context` (check token budget)
1307
+ - `should_compress_context` → `compress_context` (if threshold exceeded) OR `orchestrator` (otherwise)
1308
+ - `compress_context` → `orchestrator` (resume with compressed memory)
1309
+ - `fallback_response` → `collect_answer` (package best-effort answer)
1310
+ - `collect_answer` → END (clean final answer with index)
1311
+
1312
+ **Main Graph** (orchestrates complete workflow):
1313
+ - START → `summarize_history` (roll older chat into summary and keep only recent exchanges)
1314
+ - `summarize_history` → `rewrite_query` (rewrite query with context, check clarity)
1315
+ - `rewrite_query` → `request_clarification` (if unclear) OR spawn parallel `agent` subgraphs via `Send` (if clear)
1316
+ - `request_clarification` → `rewrite_query` (after user provides clarification)
1317
+ - All `agent` subgraphs → `aggregate_answers` (merge all responses)
1318
+ - `aggregate_answers` → END (return final synthesized answer)
1319
+
1320
+ ---
1321
+
1322
+ ### Step 11: Create Chat Interface
1323
+
1324
+ Build a Gradio interface with conversation persistence and human-in-the-loop support. For a complete end-to-end pipeline Gradio interface, including document ingestion, please refer to [project/README.md](./project/README.md).
1325
+
1326
+ > **Note:** The notebook and full project stream the final aggregated answer while showing query analysis and tool activity in separate collapsible blocks. Raw orchestrator, compression, and fallback model output remains internal. The example below is intentionally minimal.
1327
+
1328
+ ```python
1329
+ import gradio as gr
1330
+ import uuid
1331
+
1332
+ def create_thread_id():
1333
+ """Generate a unique thread ID for each conversation"""
1334
+ return {"configurable": {"thread_id": str(uuid.uuid4())}, "recursion_limit": 50}
1335
+
1336
+ def clear_session():
1337
+ """Clear thread for new conversation"""
1338
+ global config
1339
+ agent_graph.checkpointer.delete_thread(config["configurable"]["thread_id"])
1340
+ config = create_thread_id()
1341
+
1342
+ def chat(message, history):
1343
+ current_state = agent_graph.get_state(config)
1344
+
1345
+ if current_state.next:
1346
+ agent_graph.update_state(config,{"messages": [HumanMessage(content=message.strip())]})
1347
+ result = agent_graph.invoke(None, config)
1348
+ else:
1349
+ result = agent_graph.invoke({"messages": [HumanMessage(content=message.strip())]}, config)
1350
+
1351
+ return result['messages'][-1].content
1352
+
1353
+ config = create_thread_id()
1354
+
1355
+ with gr.Blocks() as demo:
1356
+ chatbot = gr.Chatbot()
1357
+ chatbot.clear(clear_session)
1358
+ gr.ChatInterface(fn=chat, chatbot=chatbot)
1359
+
1360
+ demo.launch(theme=gr.themes.Citrus())
1361
+ ```
1362
+
1363
+ **You're done!** You now have a fully functional Agentic RAG system with conversation memory, hierarchical indexing, and human-in-the-loop query clarification.
1364
+
1365
+ ---
1366
+
1367
+ ## Modular Architecture
1368
+
1369
+ The app (`project/` folder) is organized into modular components — each independently swappable without breaking the system.
1370
+
1371
+ ### 📂 Project Structure
1372
+ ```
1373
+ project/
1374
+ ├── app.py # Main Gradio application entry point
1375
+ ├── config.py # Configuration hub (models, chunk sizes, providers)
1376
+ ├── core/ # RAG system orchestration
1377
+ ├── db/ # Vector DB and parent chunk storage
1378
+ ├── rag_agent/ # LangGraph workflow (nodes, edges, prompts, tools)
1379
+ └── ui/ # Gradio interface
1380
+ ```
1381
+
1382
+ Key customization points: LLM provider, embedding model, chunking strategy, agent workflow, and system prompts — all configurable via `config.py` or their respective modules.
1383
+
1384
+ Full documentation in [project/README.md](./project/README.md).
1385
+
1386
+ ## Installation & Usage
1387
+
1388
+ Sample pdf files can be found here: [javascript](https://www.tutorialspoint.com/javascript/javascript_tutorial.pdf), [blockchain](https://blockchain-observatory.ec.europa.eu/document/download/1063effa-59cc-4df4-aeee-d2cf94f69178_en?filename=Blockchain_For_Beginners_A_EUBOF_Guide.pdf), [fortinet](https://www.commoncriteriaportal.org/files/epfiles/Fortinet%20FortiGate_EAL4_ST_V1.5.pdf(320893)_TMP.pdf).
1389
+
1390
+ ### Option 1: Quickstart Notebook (Recommended for Testing)
1391
+
1392
+ **Google Colab:** Upload the project files and dependency manifests to your notebook's working directory. Upload PDFs to `docs/`. Standard hosted Colab does not provide Ollama, so replace the default Ollama model cell with one of the documented cloud-provider examples before running the remaining cells.
1393
+
1394
+ **Local (Jupyter/VSCode):** Create and activate a virtual environment, install notebook dependencies with `uv pip install -r requirements-notebooks.txt`, add your PDFs to `docs/`, then run all cells top to bottom.
1395
+
1396
+ The chat interface will appear at the end.
1397
+
1398
+ ### Option 2: Full Python Project (Recommended for Development)
1399
+
1400
+ #### 1. Install Dependencies
1401
+ ```bash
1402
+ # Run from the project root.
1403
+ # Windows PowerShell: install only the deployable app
1404
+ uv venv --python 3.13 .venv
1405
+ uv pip install --python .venv\Scripts\python.exe --torch-backend cpu -r project\requirements.txt
1406
+ Copy-Item project\.env.example project\.env
1407
+ # Edit project\.env and set GROQ_API_KEY.
1408
+ # For notebook/evaluation dependencies too, install requirements-notebooks.txt instead.
1409
+ ```
1410
+
1411
+ #### 2. Run the Application
1412
+ ```bash
1413
+ uv run --no-project --python .venv\Scripts\python.exe project\app.py
1414
+ ```
1415
+
1416
+ #### 3. Ask Questions
1417
+
1418
+ Open the local URL (e.g., `http://127.0.0.1:7860`) to start chatting.
1419
+
1420
+ ---
1421
+
1422
+ ### Option 3: Docker Deployment
1423
+
1424
+ See [`project/README.md`](./project/README.md#Docker-Deployment) for full Docker instructions and system requirements.
1425
+
1426
+ ### Example Conversations
1427
+
1428
+ **With Conversation Memory:**
1429
+ ```
1430
+ User: "How do I install SQL?"
1431
+ Agent: [Provides installation steps from documentation]
1432
+
1433
+ User: "How do I update it?"
1434
+ Agent: [Understands "it" = SQL, provides update instructions]
1435
+ ```
1436
+
1437
+ **With Query Clarification:**
1438
+ ```
1439
+ User: "Tell me about that thing"
1440
+ Agent: "I need more information. What specific topic are you asking about?"
1441
+
1442
+ User: "The installation process for PostgreSQL"
1443
+ Agent: [Retrieves and answers with specific information]
1444
+ ```
1445
+
1446
+ ---
1447
+
1448
+ ## Troubleshooting
1449
+
1450
+ | Area | Common Problems | Suggested Solutions |
1451
+ |------|----------------|------------------|
1452
+ | **Model Selection** | - Responses ignore instructions<br>- Tools (retrieval/search) used incorrectly<br>- Poor context understanding<br>- Hallucinations or incomplete aggregation | - Use more capable LLMs<br>- Prefer models 8B+ for better reasoning<br>- Consider cloud-based models if local models are limited |
1453
+ | **System Prompt Behavior** | - Model answers without retrieving documents<br>- Query rewriting loses context<br>- Aggregation introduces hallucinations | - Make retrieval explicit in system prompts<br>- Keep query rewriting close to user intent |
1454
+ | **Retrieval Configuration** | - Relevant documents not retrieved<br>- Too much irrelevant information | - Increase retrieved chunks (`k`) or lower similarity thresholds to improve recall<br>- Reduce `k` or increase thresholds to improve precision |
1455
+ | **Chunk Size / Document Splitting** | - Answers lack context or feel fragmented<br>- Retrieval is slow or embedding costs are high | - Increase chunk & parent sizes for more context<br>- Decrease chunk sizes to improve speed and reduce costs |
1456
+ | **Context Compression** | - Agent loses important details after compression<br>- Compressed summaries are too vague | - Tune the compression system prompt<br>- Increase `BASE_TOKEN_THRESHOLD` to delay compression<br>- Increase `TOKEN_GROWTH_FACTOR` |
1457
+ | **Agent Configuration** | - Agent gives up too early <br>- Agent loops too long| - Increase `MAX_TOOL_CALLS` / `MAX_ITERATIONS` for complex queries<br>- Decrease them to speed up simple queries |
1458
+ | **Temperature & Consistency** | - Responses inconsistent or overly creative<br>- Responses too rigid or repetitive | - Set temperature to `0` for factual, consistent output<br>- Slightly increase temperature for summarization or analysis tasks |
1459
+ | **Embedding Model Quality** | - Poor semantic search<br>- Weak performance on domain-specific or multilingual docs | - Use higher-quality or domain-specific embeddings<br>- Re-index all documents after changing embeddings |
1460
+
1461
+ > 💡 **For additional troubleshooting tips** see the [README Troubleshooting](./project/README.md#troubleshooting).
project/.env.example ADDED
@@ -0,0 +1,24 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Groq chat model (required). Never put this key in frontend code.
2
+ # Get a key at https://console.groq.com/keys
3
+ GROQ_API_KEY=
4
+ GROQ_MODEL=llama-3.3-70b-versatile
5
+
6
+ # MiniLM CPU embeddings are the default for HF Gradio Spaces.
7
+ # Changing embeddings requires clearing and re-indexing existing documents.
8
+ # DENSE_MODEL=Qwen/Qwen3-Embedding-0.6B
9
+
10
+ # Optional: point at a persistent disk mount (e.g. /data in Docker).
11
+ # DATA_DIR=/data
12
+
13
+ # Optional: protect the shared document library and Groq quota with a login.
14
+ # Set both values, preferably as hosting secrets.
15
+ # GRADIO_AUTH_USERNAME=
16
+ # GRADIO_AUTH_PASSWORD=
17
+
18
+ # Langfuse Observability (optional)
19
+ # Set LANGFUSE_ENABLED=true and provide your API keys to enable tracing.
20
+ # Get your keys from https://cloud.langfuse.com or your self-hosted instance.
21
+ LANGFUSE_ENABLED=false
22
+ LANGFUSE_PUBLIC_KEY=pk-lf-...
23
+ LANGFUSE_SECRET_KEY=sk-lf-...
24
+ LANGFUSE_BASE_URL=http://localhost:3000
project/README.md ADDED
@@ -0,0 +1,718 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Document Chat Documentation
2
+
3
+ An **Agentic Retrieval-Augmented Generation (RAG)** system built with **LangGraph**, featuring **parent–child chunking**, **hybrid dense + sparse retrieval**, and Groq-powered chat with a Gradio frontend. Embeddings run locally; no Ollama server or GPU is required for the app. The learning notebooks retain their Ollama setup.
4
+
5
+
6
+ ## Table of Contents
7
+
8
+ [Quick Start](#quick-start) | [Hugging Face Deployment](#hugging-face-spaces-recommended) | [Architecture Overview](#architecture-overview) | [Project Structure](#project-structure) | [Configuration Guide](#configuration-guide) | [Common Customizations](#common-customizations) | [Observability](#observability) | [Advanced Topics](#advanced-topics) | [Troubleshooting](#troubleshooting)
9
+
10
+ ---
11
+
12
+ ## Quick Start
13
+
14
+ ### Installation
15
+
16
+ From the repository root, use `uv` (Windows PowerShell):
17
+
18
+ ```powershell
19
+ uv venv --python 3.13 .venv
20
+ uv pip install --python .venv\Scripts\python.exe --torch-backend cpu -r project\requirements.txt
21
+ Copy-Item project\.env.example project\.env
22
+ ```
23
+
24
+ Set `GROQ_API_KEY` in `project/.env`. Get a key from
25
+ https://console.groq.com/keys. Never commit `.env` or expose the key in frontend
26
+ code. The root `requirements.txt` includes only runtime dependencies (as required
27
+ by HF Gradio). Notebook/evaluation tools are in `requirements-notebooks.txt`.
28
+ Local installs and Docker can also use `project/requirements.txt` directly.
29
+
30
+ ### Running the Application
31
+
32
+ Start the Gradio interface locally:
33
+
34
+ ```powershell
35
+ uv run --no-project --python .venv\Scripts\python.exe project\app.py
36
+ ```
37
+
38
+ The application will be available at `http://localhost:7860` (default Gradio port).
39
+
40
+ > Chat memory is isolated per browser session and kept in memory, so it is lost
41
+ > on restart. Documents are a **shared library**, not private per-user storage:
42
+ > anyone with app access can query, upload, or clear them. Use a private Space
43
+ > or set both `GRADIO_AUTH_USERNAME` and `GRADIO_AUTH_PASSWORD` as secrets.
44
+ > This also protects your Groq quota. Run one app process: local Qdrant cannot
45
+ > be opened by multiple workers at the same path.
46
+
47
+ ### Prerequisites
48
+
49
+ - Python 3.11+
50
+ - A Groq API key; outbound access to Groq and Hugging Face model downloads
51
+
52
+ ---
53
+
54
+ ## Architecture Overview
55
+
56
+ This system implements an advanced RAG pipeline with the following key features:
57
+
58
+ - **Parent-Child Chunking**: Documents are split into small child chunks (for precise retrieval) linked to larger parent chunks (for rich context)
59
+ - **Hybrid Search**: Combines dense embeddings and sparse (BM25) retrieval for optimal results
60
+ - **LangGraph Agent**: Orchestrates query rewriting, retrieval, and response generation
61
+ - **Compact Chat Memory**: Rewrites follow-ups using both a rolling summary and bounded recent conversation history
62
+ - **Clarification Memory**: Preserves an unresolved query across human-in-the-loop pauses, combines its clarifications, then discards those temporary replies after the answer
63
+ - **Groq Chat**: Tool calling, structured query analysis, and streamed responses through `ChatGroq`
64
+ - **Vector Storage**: Uses Qdrant for efficient similarity search
65
+
66
+ ### Data Flow
67
+
68
+ ```
69
+ PDF → Markdown Conversion → Parent/Child Chunking → Vector Indexing → Agent Retrieval → LLM Response
70
+ ```
71
+
72
+ ---
73
+
74
+ ## Project Structure
75
+
76
+ ### Entry Point & Configuration
77
+
78
+ | File | Purpose |
79
+ |------|---------|
80
+ | `project/app.py` | Application entry point, launches Gradio UI |
81
+ | `project/config.py` | **Central configuration hub** - edit this for provider/model/chunking changes |
82
+ | `project/utils.py` | PDF conversion and cached context-token estimation with an offline-safe fallback |
83
+ | `project/document_chunker.py` | Parent/child splitting logic with cleaning and merging rules |
84
+ | `Dockerfile` | uv-based CPU runtime for the backend and Gradio frontend |
85
+
86
+ ### Core System
87
+
88
+ | File | Purpose |
89
+ |------|---------|
90
+ | `project/core/rag_system.py` | System bootstrap - creates managers and compiles LangGraph agent |
91
+ | `project/core/document_manager.py` | Document ingestion pipeline (convert, chunk, index) |
92
+ | `project/core/chat_interface.py` | Streams the aggregated answer while separating query analysis and tool activity from internal node output |
93
+ | `project/core/observability.py` | Optional Langfuse tracing — callback handler lifecycle |
94
+
95
+ ### Database Layer
96
+
97
+ | File | Purpose |
98
+ |------|---------|
99
+ | `project/db/vector_db_manager.py` | Qdrant client wrapper with embedding initialization |
100
+ | `project/db/parent_store_manager.py` | File-backed storage for parent chunks |
101
+
102
+ ### RAG Agent (LangGraph)
103
+
104
+ | File | Purpose |
105
+ |------|---------|
106
+ | `project/rag_agent/graph.py` | Graph builder and compilation logic |
107
+ | `project/rag_agent/graph_state.py` | Shared and per-agent state, including rolling memory, pending clarification, and answer accumulation/reset logic |
108
+ | `project/rag_agent/nodes.py` | Node implementations (summarize, rewrite, agent execution, aggregate) |
109
+ | `project/rag_agent/edges.py` | Conditional edge routing logic (e.g., routing based on query clarity) |
110
+ | `project/rag_agent/tools.py` | Retrieval tools (`search_child_chunks`, `retrieve_parent_chunks`) |
111
+ | `project/rag_agent/prompts.py` | System prompts for agent behavior |
112
+ | `project/rag_agent/schemas.py` | Structured output schemas (Pydantic models) |
113
+
114
+ ### User Interface
115
+
116
+ | File | Purpose |
117
+ |------|---------|
118
+ | `project/ui/css.py` | Custom CSS styling for the Gradio interface |
119
+ | `project/ui/gradio_app.py` | Gradio UI implementation with document upload and chat |
120
+
121
+ ---
122
+
123
+ ## Configuration Guide
124
+
125
+ All primary settings are in `project/config.py`. Key parameters:
126
+
127
+ ### Directory Configuration
128
+
129
+ ```python
130
+ MARKDOWN_DIR = "markdown_docs" # Storage for converted PDF → Markdown files
131
+ PARENT_STORE_PATH = "parent_store" # File-backed storage for parent chunks
132
+ QDRANT_DB_PATH = "qdrant_db" # Local Qdrant vector database path
133
+ ```
134
+
135
+ ### Qdrant Configuration
136
+
137
+ ```python
138
+ CHILD_COLLECTION = "document_child_chunks" # Collection name for child chunks
139
+ SPARSE_VECTOR_NAME = "sparse" # Named sparse vector field (BM25)
140
+ ```
141
+
142
+ ### Model Configuration
143
+
144
+ ```python
145
+ # Default: single model configuration
146
+ DENSE_MODEL = "sentence-transformers/all-MiniLM-L6-v2"
147
+ SPARSE_MODEL = "Qdrant/bm25"
148
+ LLM_MODEL = os.environ.get("GROQ_MODEL", "llama-3.3-70b-versatile")
149
+ JUDGE_MODEL = "ministral-3:3b-instruct-2512-q8_0"
150
+ LLM_TEMPERATURE = 0 # 0 = deterministic, 1 = creative
151
+ LLM_SEED = 42
152
+ LLM_TIMEOUT = 60
153
+ LLM_MAX_RETRIES = 2
154
+ ```
155
+
156
+ `GROQ_API_KEY` is required at startup. `DENSE_MODEL` and `DATA_DIR` can also be
157
+ set as environment variables. `JUDGE_MODEL` remains the local Ollama judge for
158
+ the evaluation notebook, not the deployed app. Query analysis explicitly uses
159
+ function calling, so it does not require Groq's model-specific JSON Schema mode.
160
+
161
+ ### Retrieval Configuration
162
+
163
+ ```python
164
+ RETRIEVAL_SCORE_THRESHOLD = 0.4 # Lower = more recall, higher = more precision
165
+ DEFAULT_RETRIEVAL_K = 7 # Default number of child chunks used by retrieval and evaluation
166
+ CHILD_CHUNK_SEPARATOR = "\n\n<CHILD_CHUNK_BOUNDARY>\n\n" # Keeps ranked child results separable for evaluation
167
+ ```
168
+
169
+ ### Agent Configuration
170
+ ```python
171
+ # Hard limits to prevent infinite loops
172
+ MAX_TOOL_CALLS = 8 # Maximum tool calls per agent run
173
+ MAX_ITERATIONS = 10 # Maximum agent loop iterations
174
+ GRAPH_RECURSION_LIMIT = 50 # Maximum number of steps before hitting a stop condition
175
+ MAIN_HISTORY_MESSAGES_TO_KEEP = 4 # Raw messages retained after each answer; minimum 2
176
+
177
+ # Context compression thresholds
178
+ BASE_TOKEN_THRESHOLD = 2000 # Initial token threshold for compression
179
+ TOKEN_GROWTH_FACTOR = 0.9 # Multiplier applied after each compression
180
+ ```
181
+
182
+ ### Terminal Execution Logging
183
+
184
+ ```python
185
+ EXECUTION_LOGGING_ENABLED = False # Print graph steps, state previews, tool calls, and outputs
186
+ EXECUTION_LOG_MAX_CHARS = 1200 # Maximum characters shown for long values
187
+ EXECUTION_LOG_USE_COLOR = True # Use ANSI colors in the terminal
188
+ ```
189
+
190
+ ### Text Splitter Configuration
191
+
192
+ ```python
193
+ CHILD_CHUNK_SIZE = 500 # Size of chunks used for retrieval
194
+ CHILD_CHUNK_OVERLAP = 100 # Overlap between chunks (prevents context loss)
195
+ MIN_PARENT_SIZE = 2000 # Target minimum; very short documents may remain smaller
196
+ MAX_PARENT_SIZE = 4000 # Maximum parent chunk size
197
+
198
+ # Markdown header splitting strategy
199
+ HEADERS_TO_SPLIT_ON = [
200
+ ("#", "H1"),
201
+ ("##", "H2"),
202
+ ("###", "H3")
203
+ ]
204
+ ```
205
+
206
+ ### Langfuse Observability (Optional)
207
+
208
+ ```python
209
+ import os
210
+
211
+ LANGFUSE_ENABLED = os.environ.get("LANGFUSE_ENABLED", "false").lower() == "true"
212
+ LANGFUSE_PUBLIC_KEY = "" # From your Langfuse project settings
213
+ LANGFUSE_SECRET_KEY = "" # From your Langfuse project settings
214
+ LANGFUSE_BASE_URL = "http://localhost:3000" # Langfuse Cloud or self-hosted URL
215
+ ```
216
+
217
+ ---
218
+
219
+ ## Common Customizations
220
+
221
+ ### 1. Switching LLM Provider (Single Provider)
222
+
223
+ > **Performance Note:** LLMs with 8B+ parameters typically offer superior reasoning, context comprehension, and response quality compared to smaller models. This applies to both proprietary and open-source models, as long as they **support native tool/function calling,** which is required for agentic RAG workflows.
224
+
225
+ If you want to switch from Groq to another provider (e.g., Google Gemini), follow these steps:
226
+
227
+ **Step 1:** Install the provider's SDK
228
+
229
+ ```bash
230
+ uv pip install langchain-google-genai
231
+ ```
232
+
233
+ **Step 2:** Set environment variable
234
+
235
+ ```bash
236
+ export GOOGLE_API_KEY="your-google-key"
237
+ ```
238
+
239
+ **Step 3:** Update `project/config.py`
240
+
241
+ ```python
242
+ LLM_MODEL = "gemini-2.5-pro"
243
+ LLM_TEMPERATURE = 0
244
+ LLM_SEED = 42
245
+ ```
246
+
247
+ **Step 4:** Modify `project/core/rag_system.py`
248
+
249
+ Replace:
250
+
251
+ ```python
252
+ llm = ChatGroq(model=config.LLM_MODEL, temperature=config.LLM_TEMPERATURE,
253
+ timeout=config.LLM_TIMEOUT, max_retries=config.LLM_MAX_RETRIES,
254
+ model_kwargs={"seed": config.LLM_SEED})
255
+ ```
256
+
257
+ With:
258
+
259
+ ```python
260
+ from langchain_google_genai import ChatGoogleGenerativeAI
261
+
262
+ llm = ChatGoogleGenerativeAI(model=config.LLM_MODEL, temperature=config.LLM_TEMPERATURE)
263
+ ```
264
+
265
+ ### 2. Optional Multi-Provider Configuration
266
+
267
+ The application ships with a Groq implementation. The following is an optional
268
+ customization example, not the shipped configuration. Remove the Groq-only
269
+ startup key check if implementing a different provider.
270
+
271
+ This approach allows you to maintain multiple provider configurations and switch between them easily.
272
+
273
+ **Step 1:** Install required SDKs
274
+
275
+ ```bash
276
+ uv pip install langchain-openai langchain-anthropic langchain-google-genai
277
+ ```
278
+
279
+ **Step 2:** Set environment variables
280
+
281
+ ```bash
282
+ export OPENAI_API_KEY="your-openai-key"
283
+ export ANTHROPIC_API_KEY="your-anthropic-key"
284
+ export GOOGLE_API_KEY="your-google-key"
285
+ ```
286
+
287
+ **Step 3:** Update `project/config.py` with multi-provider configuration
288
+
289
+ ```python
290
+ # --- Multi-Provider LLM Configuration ---
291
+ LLM_CONFIGS = {
292
+ "ollama": {
293
+ "model": "granite4.1:8b",
294
+ "url":"http://localhost:11434",
295
+ "temperature": 0
296
+ },
297
+ "openai": {
298
+ "model": "gpt-5.2",
299
+ "temperature": 0
300
+ },
301
+ "anthropic": {
302
+ "model": "claude-sonnet-4-6",
303
+ "temperature": 0
304
+ },
305
+ "google": {
306
+ "model": "gemini-2.5-flash",
307
+ "temperature": 0
308
+ }
309
+ }
310
+
311
+ # Switch providers by changing this single line
312
+ ACTIVE_LLM_CONFIG = "ollama"
313
+ ```
314
+
315
+ **Step 4:** Modify `project/core/rag_system.py` in the `initialize()` method
316
+
317
+ Replace the existing LLM initialization with:
318
+
319
+ ```python
320
+ def initialize(self):
321
+ self.vector_db.create_collection(self.collection_name)
322
+ collection = self.vector_db.get_collection(self.collection_name)
323
+
324
+ # Load active configuration
325
+ active_config = config.LLM_CONFIGS[config.ACTIVE_LLM_CONFIG]
326
+ model = active_config["model"]
327
+ temperature = active_config["temperature"]
328
+
329
+ if config.ACTIVE_LLM_CONFIG == "ollama":
330
+ from langchain_ollama import ChatOllama
331
+ llm = ChatOllama(model=model, temperature=temperature, base_url=active_config["url"])
332
+
333
+ elif config.ACTIVE_LLM_CONFIG == "openai":
334
+ from langchain_openai import ChatOpenAI
335
+ llm = ChatOpenAI(model=model, temperature=temperature)
336
+
337
+ elif config.ACTIVE_LLM_CONFIG == "anthropic":
338
+ from langchain_anthropic import ChatAnthropic
339
+ llm = ChatAnthropic(model=model, temperature=temperature)
340
+
341
+ elif config.ACTIVE_LLM_CONFIG == "google":
342
+ from langchain_google_genai import ChatGoogleGenerativeAI
343
+ llm = ChatGoogleGenerativeAI(model=model, temperature=temperature)
344
+
345
+ else:
346
+ raise ValueError(f"Unsupported LLM provider: {config.ACTIVE_LLM_CONFIG}")
347
+
348
+ # Continue with tool and graph initialization
349
+ tools = ToolFactory(collection).create_tools()
350
+ self.agent_graph = create_agent_graph(llm, tools)
351
+ ```
352
+
353
+ **Switching Providers:** Simply change `ACTIVE_LLM_CONFIG` in `config.py`:
354
+
355
+ ```python
356
+ ACTIVE_LLM_CONFIG = "google" # Switch to Gemini Pro
357
+ # ACTIVE_LLM_CONFIG = "anthropic" # Or Claude Sonnet
358
+ # ACTIVE_LLM_CONFIG = "openai" # Or GPT-4o
359
+ ```
360
+
361
+ ---
362
+
363
+ **Provider Reference Table:**
364
+
365
+ | Provider | Environment Variable | Import Statement | Example Models |
366
+ |----------|---------------------|------------------|----------------|
367
+ | Groq | `GROQ_API_KEY` | `from langchain_groq import ChatGroq` | `llama-3.3-70b-versatile` |
368
+ | OpenAI | `OPENAI_API_KEY` | `from langchain_openai import ChatOpenAI` | `gpt-5.2`, `gpt-5-mini` |
369
+ | Anthropic | `ANTHROPIC_API_KEY` | `from langchain_anthropic import ChatAnthropic` | `claude-opus-4-6`, `claude-sonnet-4-6` |
370
+ | Google | `GOOGLE_API_KEY` | `from langchain_google_genai import ChatGoogleGenerativeAI` | `gemini-2.5-pro`, `gemini-2.5-flash` |
371
+ | Ollama | None (local) | `from langchain_ollama import ChatOllama` | `granite4.1:8b`, `llama3.1:8b-instruct-q6_K` |
372
+
373
+ ---
374
+
375
+ ### 3. Changing Embedding Models
376
+
377
+ **Why change?** Trade-offs between speed, cost, and quality.
378
+
379
+ **Step 1:** Update `project/config.py`
380
+
381
+ ```python
382
+ # Default: Faster, smaller CPU model
383
+ DENSE_MODEL = "sentence-transformers/all-MiniLM-L6-v2"
384
+
385
+ # Example: Alternative Sentence Transformers model
386
+ # DENSE_MODEL = "sentence-transformers/all-mpnet-base-v2"
387
+
388
+ # Example: Gemma embeddings (Google's open model)
389
+ # DENSE_MODEL = "google/embeddinggemma-300m"
390
+
391
+ # Optional: Qwen embeddings (Alibaba's multilingual model)
392
+ # DENSE_MODEL = "Qwen/Qwen3-Embedding-0.6B"
393
+
394
+ SPARSE_MODEL = "Qdrant/bm25" # Usually no need to change
395
+ ```
396
+
397
+ **Step 2:** Re-index your documents
398
+
399
+ ⚠️ **Important:** Changing embeddings requires re-indexing. In the educational UI, use **Clear All**, restart if the old collection prevents startup, then upload the documents again.
400
+
401
+ **Implementation Details** (in `project/db/vector_db_manager.py`):
402
+
403
+ ```python
404
+ from langchain_huggingface import HuggingFaceEmbeddings
405
+ from langchain_qdrant import FastEmbedSparse
406
+ import config
407
+
408
+ self.__dense_embeddings = HuggingFaceEmbeddings(
409
+ model_name=config.DENSE_MODEL, model_kwargs={"device": "cpu"}
410
+ )
411
+ self.__sparse_embeddings = FastEmbedSparse(model_name=config.SPARSE_MODEL)
412
+ ```
413
+
414
+ **Popular Embedding Models:**
415
+
416
+ | Model | Context Size | Vector Dimension | Speed | Quality | Use Case |
417
+ |-------|--------------|------------------|-------|---------|----------|
418
+ | all-MiniLM-L6-v2 | 256 tokens | 384 | Fast | Good | Default CPU model for the deployed app |
419
+ | all-mpnet-base-v2 | 512 tokens | 768 | Medium | Excellent | High-accuracy semantic search |
420
+ | bge-large-en-v1.5 | 512 tokens | 1024 | Slow | Best | Production-grade retrieval on GPU |
421
+ | google/embeddinggemma-300m | 2048 tokens | 768 | Fast | Very Good | Lightweight, efficient multilingual retrieval |
422
+ | Qwen/Qwen3-Embedding-0.6B | 32768 tokens | 1024 | Medium | Excellent | Optional multilingual/code retrieval model |
423
+ | Qwen/Qwen3-Embedding-8B | 32768 tokens | 4096 | Very Slow | Excellent / SOTA | Heavy GPU-oriented retrieval experiments |
424
+
425
+ ---
426
+
427
+ ### 4. Adjusting Chunking Strategy
428
+
429
+ **Why adjust?** Balance between retrieval precision and context richness.
430
+
431
+ > Before re-indexing, inspect Markdown output and chunk boundaries to verify the document structure.
432
+
433
+ **Step 1:** Update chunk sizes in `project/config.py`
434
+
435
+ ```python
436
+ # For short, factual queries (e.g., technical documentation)
437
+ CHILD_CHUNK_SIZE = 300
438
+ CHILD_CHUNK_OVERLAP = 60
439
+ MIN_PARENT_SIZE = 1500
440
+ MAX_PARENT_SIZE = 8000
441
+
442
+ # For narrative or contextual queries (e.g., legal documents)
443
+ # CHILD_CHUNK_SIZE = 800
444
+ # CHILD_CHUNK_OVERLAP = 150
445
+ # MIN_PARENT_SIZE = 3000
446
+ # MAX_PARENT_SIZE = 15000
447
+ ```
448
+
449
+ **Step 2 (Optional):** Replace the splitter in `project/document_chunker.py`
450
+
451
+ **Default (Character-based):**
452
+ ```python
453
+ from langchain_text_splitters import RecursiveCharacterTextSplitter
454
+
455
+ self.__child_splitter = RecursiveCharacterTextSplitter(
456
+ chunk_size=config.CHILD_CHUNK_SIZE,
457
+ chunk_overlap=config.CHILD_CHUNK_OVERLAP
458
+ )
459
+ ```
460
+
461
+ **Alternative (Sentence-aware):**
462
+ ```python
463
+ from langchain_text_splitters import SentenceTransformersTokenTextSplitter
464
+
465
+ self.__child_splitter = SentenceTransformersTokenTextSplitter(
466
+ chunk_size=config.CHILD_CHUNK_SIZE,
467
+ chunk_overlap=config.CHILD_CHUNK_OVERLAP
468
+ )
469
+ ```
470
+
471
+ **Step 3:** Re-run ingestion
472
+
473
+ Use **Clear All** in the Gradio interface, then upload the documents again. Existing documents are intentionally skipped to avoid accidental duplicates.
474
+
475
+ **Chunking Guidelines:**
476
+
477
+ > ⚠️ **Disclaimer:** These are empirical guidelines. Optimal sizes depend on:
478
+ > - **Child chunk** → embedding model's context window (e.g. 256 tokens for all-MiniLM-L6-v2, 512 for bge-large-en-v1.5): child size should not exceed it
479
+ > - **Parent chunk** → generative model's context window (e.g. 8K, 32K, 128K tokens): parent must fit within the context sent to the LLM alongside the query
480
+ >
481
+ > Always validate values empirically on your own corpus.
482
+
483
+ The chunker enforces `MAX_PARENT_SIZE`, deduplicates merged header metadata, and rebalances small neighboring chunks when possible. A document shorter than `MIN_PARENT_SIZE` remains a single smaller chunk.
484
+
485
+ | Document Type | Child Size | Parent Size | Reasoning |
486
+ |---------------|-----------|-------------|-----------|
487
+ | Technical Docs | 300-500 | 2000-4000 | Precise lookups, code snippets |
488
+ | Legal Contracts | 600-1000 | 5000-15000 | Context-heavy, definitions |
489
+ | Research Papers | 400-600 | 3000-8000 | Balance of precision and context |
490
+ | FAQs / Knowledge Base | 200-400 | 1500-4000 | Short, focused answers |
491
+
492
+ ---
493
+
494
+ ### 5. Agent Configuration
495
+
496
+ Tune agent behavior in `project/config.py`:
497
+ ```python
498
+ # Hard limits to prevent infinite loops
499
+ MAX_TOOL_CALLS = 8 # Maximum tool calls per agent run
500
+ MAX_ITERATIONS = 10 # Maximum agent loop iterations
501
+ GRAPH_RECURSION_LIMIT = 50 # Maximum number of steps before hitting a stop condition
502
+ MAIN_HISTORY_MESSAGES_TO_KEEP = 4 # Raw messages retained after each answer; minimum 2
503
+
504
+ # Context compression thresholds
505
+ BASE_TOKEN_THRESHOLD = 2000 # Initial token threshold for compression
506
+ TOKEN_GROWTH_FACTOR = 0.9 # Multiplier applied after each compression
507
+ ```
508
+
509
+ | Parameter | Effect |
510
+ |-----------|--------|
511
+ | `MAX_TOOL_CALLS` | Maximum cumulative tool calls requested; a request that would exceed it is not executed |
512
+ | `MAX_ITERATIONS` | Maximum LLM reasoning iterations; a final answer at the boundary is still accepted |
513
+ | `GRAPH_RECURSION_LIMIT` | Increase for complex [graphs](https://docs.langchain.com/oss/python/langgraph/errors/GRAPH_RECURSION_LIMIT) |
514
+ | `BASE_TOKEN_THRESHOLD` | Delay compression by increasing this value |
515
+ | `TOKEN_GROWTH_FACTOR` | Lower values compress more aggressively |
516
+
517
+ ---
518
+
519
+ ## Observability
520
+
521
+ Optional tracing with [Langfuse](https://langfuse.com) captures every LLM call, tool invocation, and graph transition. It is useful for debugging agent behavior, tracking costs, and evaluating retrieval quality.
522
+
523
+ ### Enabling Langfuse
524
+
525
+ 1. Sign up on [Langfuse Cloud](https://cloud.langfuse.com/), create an organization, then create a project and generate API keys from the project settings.
526
+ 2. Set environment variables (or copy `.env.example` to `.env`):
527
+
528
+ ```bash
529
+ export LANGFUSE_ENABLED=true
530
+ export LANGFUSE_PUBLIC_KEY=pk-lf-...
531
+ export LANGFUSE_SECRET_KEY=sk-lf-...
532
+ export LANGFUSE_BASE_URL=https://cloud.langfuse.com # or your self-hosted URL
533
+ ```
534
+
535
+ 3. Run the app normally. Traces appear in your [Langfuse dashboard](https://cloud.langfuse.com/).
536
+
537
+ To disable tracing, set `LANGFUSE_ENABLED=false` or leave the variables unset. The application runs identically either way.
538
+
539
+ For additional details on integrating Langfuse with LangChain or LangGraph, see the official [documentation](https://docs.langchain.com/oss/python/integrations/providers/langfuse).
540
+
541
+ ### What gets traced
542
+
543
+ | Component | Traced operations |
544
+ |-----------|-------------------|
545
+ | Graph nodes | `summarize_history`, `rewrite_query`, `orchestrator`, `compress_context`, `fallback_response`, `aggregate_answers` |
546
+ | Tools | `search_child_chunks`, `retrieve_parent_chunks` (arguments + results) |
547
+ | Structured output | `QueryAnalysis` parsing in the rewrite step |
548
+ | Subgraph fan-out | Parallel agent invocations via `Send()` |
549
+
550
+ ### Hosting options
551
+
552
+ - **Langfuse Cloud** — sign up at [cloud.langfuse.com](https://cloud.langfuse.com) and check the current plan limits there.
553
+ - **Self-hosted** — MIT-licensed, deploy with Docker Compose. See the [official self-hosting docs](https://langfuse.com/self-hosting).
554
+
555
+ For a conceptual tracing guide and platform context, see [`notebooks/observability.ipynb`](../notebooks/observability.ipynb).
556
+
557
+ ---
558
+
559
+ ## Advanced Topics
560
+
561
+ ### Customizing the RAG Agent
562
+
563
+ **Location:** `project/rag_agent/`
564
+
565
+ **Add/Remove Nodes:** Edit `graph.py` and `nodes.py`
566
+
567
+ Example: Adding a fact-checking node
568
+ ```python
569
+ # In nodes.py
570
+ def fact_check_node(state):
571
+ # Your fact-checking logic
572
+ return {"fact_checked": True}
573
+
574
+ # In graph.py
575
+ builder.add_node("fact_check", fact_check_node)
576
+ builder.add_edge("retrieve", "fact_check")
577
+ ```
578
+
579
+ **Modify Conditional Routing:** Edit `edges.py` to change graph flow logic
580
+
581
+ Example from the system - routing based on query clarity:
582
+ ```python
583
+ def route_after_rewrite(state: State) -> Literal["request_clarification", "agent"]:
584
+ """Routes to human input if question unclear, otherwise processes all rewritten queries"""
585
+ if not state.get("questionIsClear", False):
586
+ return "request_clarification"
587
+ else:
588
+ # Fan-out: send each rewritten question to parallel processing
589
+ return [
590
+ Send("agent", {"question": query, "question_index": idx, "messages": []})
591
+ for idx, query in enumerate(state["rewrittenQuestions"])
592
+ ]
593
+ ```
594
+
595
+ This pattern allows the agent to either request clarification from the user or fan-out multiple query variations for parallel retrieval.
596
+
597
+ **Modify Prompts:** Edit `prompts.py` to change agent behavior and response style
598
+
599
+ **Add Custom Tools:** Extend `tools.py` with new retrieval strategies or external integrations
600
+
601
+ ### Replacing Storage Backends
602
+
603
+ **Vector Database:**
604
+ - Default: Local Qdrant
605
+ - Alternatives: Remote Qdrant Cloud, Pinecone, Weaviate
606
+ - Edit: `project/db/vector_db_manager.py`
607
+
608
+ **Parent Store:**
609
+ - Default: JSON file
610
+ - Alternatives: PostgreSQL, MongoDB, S3
611
+ - Edit: `project/db/parent_store_manager.py`
612
+
613
+ ### Extending the UI
614
+
615
+ **Location:** `project/ui/gradio_app.py`
616
+
617
+ Add runtime settings, admin panels, or analytics:
618
+ ```python
619
+ with gr.Accordion("Advanced Settings", open=False):
620
+ provider_dropdown = gr.Dropdown(
621
+ choices=["openai", "anthropic", "google", "ollama"],
622
+ label="LLM Provider"
623
+ )
624
+ ```
625
+
626
+ ### Docker Deployment
627
+
628
+ The root `Dockerfile` installs only runtime dependencies using uv and CPU
629
+ PyTorch. It serves the existing Gradio frontend and backend together; there is
630
+ no separate frontend build or Ollama process. MiniLM reduces memory use,
631
+ but PyTorch and PDF indexing still need more than a 512 MB host. If selecting
632
+ Qwen embeddings instead, allow several GB of RAM; 8 GB is a starting budget.
633
+ Model weights download on first startup, so the initial start takes longer.
634
+
635
+ #### Build and Run
636
+
637
+ ```powershell
638
+ docker build -t agentic-rag .
639
+ docker run --name rag-assistant --env-file project\.env -p 7860:7860 agentic-rag
640
+ ```
641
+
642
+ For persistent documents and indexes:
643
+
644
+ ```powershell
645
+ docker run --name rag-assistant --env-file project\.env -e DATA_DIR=/data -v rag-data:/data -p 7860:7860 agentic-rag
646
+ ```
647
+
648
+ Once running, open `http://localhost:7860`.
649
+
650
+ ### Hugging Face Spaces (Recommended)
651
+
652
+ 1. Create a **Gradio Space** and select **ZeroGPU Free** if offered for your
653
+ account. Do not select Docker. The root README already specifies
654
+ `sdk: gradio`, `app_file: project/app.py`, the Gradio version, and Python
655
+ 3.12.12 (supported by ZeroGPU).
656
+ 2. Upload the root `README.md`, `requirements.txt`, `LICENSE`, and the full
657
+ `project` folder (including `project/requirements.txt`).
658
+ Do not upload `.env`, `.venv`, local documents, or local database files.
659
+ 3. Under **Settings > Variables and secrets**, add `GROQ_API_KEY` as a
660
+ **secret**, optionally `GROQ_MODEL` as a variable, and both authentication
661
+ values as secrets if the Space is public.
662
+ 4. Wait for the build and first model download, then open the **App** tab.
663
+ Upload a small Markdown/PDF file under **Documents**, click **Add Documents**,
664
+ and ask a question under **Chat**.
665
+
666
+ The app runs MiniLM embeddings explicitly on **CPU** and calls Groq remotely.
667
+ It does not use CUDA or request GPU time, so no `@spaces.GPU` decorator is
668
+ needed. Choosing ZeroGPU does not grant a permanently attached GPU or remove
669
+ platform limits. Verify first startup and indexing in the Space logs.
670
+
671
+ The UI is branded **Document Chat**, with a teal-accented header, a compact
672
+ chat panel, and the same **Documents** and **Chat** workflow.
673
+
674
+ Per the [ZeroGPU documentation](https://huggingface.co/docs/hub/spaces-zerogpu),
675
+ free personal accounts in good standing (verified email and older than 30 days)
676
+ can host up to two ZeroGPU Spaces. Availability is account-dependent.
677
+ Gradio Spaces manage dependency installation from root `requirements.txt`;
678
+ uv remains the tool for local installation and the optional Docker build.
679
+
680
+ Documents and indexes use ephemeral local disk and may be lost on
681
+ restart/rebuild. Chat memory is always in memory. Do not use this demo as
682
+ durable storage. `DATA_DIR` can point to an existing writable mount on hosts
683
+ that provide one, but does not provision persistence.
684
+
685
+ #### Render (Alternative)
686
+
687
+ Create a **Docker Web Service** using the root Dockerfile with the repository
688
+ root as the build context. Set `GROQ_API_KEY` as a secret. The app honors
689
+ Render's `PORT` and listens on `0.0.0.0` in Docker. Choose a paid instance with
690
+ enough RAM for the embedding model, and attach a disk at `/data` with
691
+ `DATA_DIR=/data` if documents must survive deploys. Use one instance/worker.
692
+ The app defaults to `DENSE_MODEL=sentence-transformers/all-MiniLM-L6-v2`;
693
+ this trades retrieval quality for lower memory use compared with Qwen.
694
+ Render's free instances sleep and lose local files; do not rely on them for
695
+ this model-heavy app. See [free-service limits](https://render.com/docs/free).
696
+
697
+ #### Vercel
698
+
699
+ Not recommended for the full app: it depends on a long-running Gradio server,
700
+ in-memory LangGraph checkpoints, local Qdrant files, and model downloads.
701
+ Vercel can host a future separate JavaScript frontend, but the current Python
702
+ service should remain on Spaces or Render.
703
+
704
+ ---
705
+
706
+ ## Troubleshooting
707
+
708
+ | Issue | Cause | Solution |
709
+ |-------|-------|----------|
710
+ | "Model not found" error | Incorrect model name for provider | Verify `LLM_MODEL` matches provider's API (e.g., `gpt-4o-mini` not `gpt4-mini`) |
711
+ | Low-quality retrieval results | Poor embedding model or chunk configuration | Re-index with better embeddings or adjust chunk sizes |
712
+ | Vector size mismatch | Existing Qdrant collection was built with a different embedding model | Clear the collection and re-index documents after changing `DENSE_MODEL` |
713
+ | Slow response times | Large embedding model or high `top_k` value | Use smaller embedding models (e.g., all-MiniLM-L6-v2) or reduce `top_k` in retrieval tools |
714
+ | API rate limits exceeded | Too many requests to external provider | Add retry logic with exponential backoff or switch to local Ollama models |
715
+ | Out of memory errors | Large document set or embedding model | Use smaller embeddings, reduce batch size, or enable GPU acceleration |
716
+ | Empty retrieval results | Collection not indexed or wrong collection name | Verify documents are uploaded and `CHILD_COLLECTION` name matches in config |
717
+ | Import errors after provider switch | Missing SDK installation | Install required package: `pip install langchain-{provider}` |
718
+ | Inconsistent answers across runs | High temperature setting | Set `LLM_TEMPERATURE = 0` in config for deterministic responses |
project/app.py ADDED
@@ -0,0 +1,35 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import sys
2
+ import os
3
+ import logging
4
+
5
+ sys.path.insert(0, os.path.dirname(__file__))
6
+
7
+ from dotenv import load_dotenv
8
+ load_dotenv(os.path.join(os.path.dirname(__file__), ".env"))
9
+
10
+ # Suppress OTel "Failed to detach context" warning caused by generator/context interaction.
11
+ # Tracing is unaffected.
12
+ # Known bug: https://github.com/open-telemetry/opentelemetry-python/issues/2606
13
+ class _SuppressOtelDetachWarning(logging.Filter):
14
+ def filter(self, record):
15
+ return "Failed to detach context" not in record.getMessage()
16
+
17
+ logging.getLogger("opentelemetry.context").addFilter(_SuppressOtelDetachWarning())
18
+
19
+ from ui.css import custom_css
20
+ from ui.gradio_app import create_gradio_ui
21
+
22
+ if __name__ == "__main__":
23
+ auth_username = os.environ.get("GRADIO_AUTH_USERNAME", "")
24
+ auth_password = os.environ.get("GRADIO_AUTH_PASSWORD", "")
25
+ if bool(auth_username) != bool(auth_password):
26
+ raise ValueError("Set both GRADIO_AUTH_USERNAME and GRADIO_AUTH_PASSWORD, or neither.")
27
+ print("\n🔨 Creating RAG Assistant...")
28
+ demo = create_gradio_ui()
29
+ print("\n🚀 Launching RAG Assistant...")
30
+ demo.launch(
31
+ css=custom_css,
32
+ server_name=os.environ.get("GRADIO_SERVER_NAME", "0.0.0.0" if os.environ.get("SPACE_ID") else "127.0.0.1"),
33
+ server_port=int(os.environ.get("PORT", os.environ.get("GRADIO_SERVER_PORT", "7860"))),
34
+ auth=(auth_username, auth_password) if auth_username else None,
35
+ )
project/assets/chatbot_avatar.png ADDED

Git LFS Details

  • SHA256: 6f30be24b4548854c29d69ec919639bdca0280b08c778a8ac909c4cbbf152baf
  • Pointer size: 131 Bytes
  • Size of remote file: 116 kB
project/config.py ADDED
@@ -0,0 +1,58 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import os
2
+
3
+ # --- Directory Configuration ---
4
+ _BASE_DIR = os.path.dirname(os.path.dirname(__file__))
5
+ DATA_DIR = os.environ.get("DATA_DIR", _BASE_DIR)
6
+
7
+ MARKDOWN_DIR = os.path.join(DATA_DIR, "markdown_docs")
8
+ PARENT_STORE_PATH = os.path.join(DATA_DIR, "parent_store")
9
+ QDRANT_DB_PATH = os.path.join(DATA_DIR, "qdrant_db")
10
+
11
+ # --- Qdrant Configuration ---
12
+ CHILD_COLLECTION = "document_child_chunks"
13
+ SPARSE_VECTOR_NAME = "sparse"
14
+
15
+ # --- Model Configuration ---
16
+ DENSE_MODEL = os.environ.get("DENSE_MODEL", "sentence-transformers/all-MiniLM-L6-v2")
17
+ SPARSE_MODEL = "Qdrant/bm25"
18
+ LLM_MODEL = os.environ.get("GROQ_MODEL", "llama-3.3-70b-versatile")
19
+ JUDGE_MODEL = "ministral-3:3b-instruct-2512-q8_0"
20
+ LLM_TEMPERATURE = 0
21
+ LLM_SEED = 42
22
+ LLM_TIMEOUT = 60
23
+ LLM_MAX_RETRIES = 2
24
+
25
+ # --- Retrieval Configuration ---
26
+ RETRIEVAL_SCORE_THRESHOLD = 0.4
27
+ DEFAULT_RETRIEVAL_K = 7
28
+ CHILD_CHUNK_SEPARATOR = "\n\n<CHILD_CHUNK_BOUNDARY>\n\n"
29
+
30
+ # --- Agent Configuration ---
31
+ MAX_TOOL_CALLS = 8
32
+ MAX_ITERATIONS = 10
33
+ GRAPH_RECURSION_LIMIT = 50
34
+ MAIN_HISTORY_MESSAGES_TO_KEEP = 4
35
+ BASE_TOKEN_THRESHOLD = 2000
36
+ TOKEN_GROWTH_FACTOR = 0.9
37
+
38
+ # --- Terminal Execution Logging ---
39
+ EXECUTION_LOGGING_ENABLED = False
40
+ EXECUTION_LOG_MAX_CHARS = 1200
41
+ EXECUTION_LOG_USE_COLOR = True
42
+
43
+ # --- Text Splitter Configuration ---
44
+ CHILD_CHUNK_SIZE = 500
45
+ CHILD_CHUNK_OVERLAP = 100
46
+ MIN_PARENT_SIZE = 2000
47
+ MAX_PARENT_SIZE = 4000
48
+ HEADERS_TO_SPLIT_ON = [
49
+ ("#", "H1"),
50
+ ("##", "H2"),
51
+ ("###", "H3")
52
+ ]
53
+
54
+ # --- Langfuse Observability ---
55
+ LANGFUSE_ENABLED = os.environ.get("LANGFUSE_ENABLED", "false").lower() == "true"
56
+ LANGFUSE_PUBLIC_KEY = os.environ.get("LANGFUSE_PUBLIC_KEY", "")
57
+ LANGFUSE_SECRET_KEY = os.environ.get("LANGFUSE_SECRET_KEY", "")
58
+ LANGFUSE_BASE_URL = os.environ.get("LANGFUSE_BASE_URL", "http://localhost:3000")
project/core/__init__.py ADDED
File without changes
project/core/chat_interface.py ADDED
@@ -0,0 +1,169 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import json
2
+ import re
3
+ from langchain_core.messages import HumanMessage, AIMessageChunk, ToolMessage
4
+ from core.execution_logger import log_chat_end, log_chat_start, log_error
5
+
6
+ SYSTEM_NODES = {"summarize_history", "rewrite_query"}
7
+ FINAL_RESPONSE_NODES = {"aggregate_answers"}
8
+
9
+ SYSTEM_NODE_CONFIG = {
10
+ "rewrite_query": {"title": "🔍 Query Analysis & Rewriting"},
11
+ "summarize_history": {"title": "📋 Chat History Summary"},
12
+ }
13
+
14
+ # --- Helpers ---
15
+
16
+ def make_message(content, *, title=None, node=None):
17
+ msg = {"role": "assistant", "content": content}
18
+ if title or node:
19
+ msg["metadata"] = {k: v for k, v in {"title": title, "node": node}.items() if v}
20
+ return msg
21
+
22
+
23
+ def find_msg_idx(messages, node):
24
+ return next(
25
+ (i for i, m in enumerate(messages) if m.get("metadata", {}).get("node") == node),
26
+ None,
27
+ )
28
+
29
+
30
+ def parse_rewrite_json(buffer):
31
+ match = re.search(r"\{.*\}", buffer, re.DOTALL)
32
+ if not match:
33
+ return None
34
+ try:
35
+ return json.loads(match.group())
36
+ except Exception:
37
+ return None
38
+
39
+
40
+ def format_rewrite_content(buffer):
41
+ data = parse_rewrite_json(buffer)
42
+ if not data:
43
+ return "⏳ Analyzing query..."
44
+ if data.get("is_clear"):
45
+ lines = ["✅ **Query is clear**"]
46
+ if data.get("questions"):
47
+ lines += ["\n**Rewritten queries:**"] + [f"- {q}" for q in data["questions"]]
48
+ else:
49
+ lines = ["❓ **Query is unclear**"]
50
+ clarification = data.get("clarification_needed", "")
51
+ if clarification and clarification.strip().lower() != "no":
52
+ lines.append(f"\nClarification needed: *{clarification}*")
53
+ return "\n".join(lines)
54
+
55
+ # --- End of Helpers ---
56
+
57
+ class ChatInterface:
58
+
59
+ def __init__(self, rag_system):
60
+ self.rag_system = rag_system
61
+
62
+ def _handle_system_node(self, content, node, response_messages, system_node_buffer):
63
+ """Update (or create) the collapsible system-node message and surface any clarification."""
64
+ system_node_buffer[node] = system_node_buffer.get(node, "") + content
65
+ buffer = system_node_buffer[node]
66
+ title = SYSTEM_NODE_CONFIG[node]["title"]
67
+ content = format_rewrite_content(buffer) if node == "rewrite_query" else buffer
68
+
69
+ idx = find_msg_idx(response_messages, node)
70
+ if idx is None:
71
+ response_messages.append(make_message(content, title=title, node=node))
72
+ else:
73
+ response_messages[idx]["content"] = content
74
+
75
+ if node == "rewrite_query":
76
+ self._surface_clarification(buffer, response_messages)
77
+
78
+ def _surface_clarification(self, buffer, response_messages):
79
+ """If the query is unclear, add/update a plain clarification message."""
80
+ data = parse_rewrite_json(buffer) or {}
81
+ clarification = data.get("clarification_needed", "")
82
+ if not data.get("is_clear") and clarification.strip().lower() not in ("", "no"):
83
+ cidx = find_msg_idx(response_messages, "clarification")
84
+ if cidx is None:
85
+ response_messages.append(make_message(clarification, node="clarification"))
86
+ else:
87
+ response_messages[cidx]["content"] = clarification
88
+
89
+ def _handle_tool_call(self, chunk, response_messages, active_tool_calls):
90
+ """Register new tool calls as collapsible messages."""
91
+ for tc in chunk.tool_calls:
92
+ if tc.get("id") and tc["id"] not in active_tool_calls:
93
+ response_messages.append(
94
+ make_message(f"Running `{tc['name']}`...", title=f"🛠️ {tc['name']}")
95
+ )
96
+ active_tool_calls[tc["id"]] = len(response_messages) - 1
97
+
98
+ def _handle_tool_result(self, chunk, response_messages, active_tool_calls):
99
+ """Fill in the tool result inside the matching collapsible message."""
100
+ idx = active_tool_calls.get(chunk.tool_call_id)
101
+ if idx is not None:
102
+ preview = str(chunk.content)[:300]
103
+ suffix = "\n..." if len(str(chunk.content)) > 300 else ""
104
+ response_messages[idx]["content"] = f"```\n{preview}{suffix}\n```"
105
+
106
+ def _handle_llm_token(self, chunk, node, response_messages):
107
+ """Append streaming LLM tokens to the last plain assistant message."""
108
+ last = response_messages[-1] if response_messages else None
109
+ if not (last and last.get("role") == "assistant" and "metadata" not in last):
110
+ response_messages.append(make_message(""))
111
+ response_messages[-1]["content"] += chunk.content
112
+
113
+ def chat(self, message, history, thread_id=None):
114
+ """Generator that streams Gradio chat message dicts."""
115
+ if not self.rag_system.agent_graph:
116
+ yield "⚠️ System not initialized!"
117
+ return
118
+
119
+ config = self.rag_system.get_config(thread_id)
120
+ current_state = self.rag_system.agent_graph.get_state(config)
121
+ log_chat_start(message.strip(), config["configurable"]["thread_id"], bool(current_state.next))
122
+
123
+ try:
124
+ if current_state.next:
125
+ self.rag_system.agent_graph.update_state(config, {"messages": [HumanMessage(content=message.strip())]})
126
+ stream_input = None
127
+ else:
128
+ stream_input = {"messages": [HumanMessage(content=message.strip())]}
129
+
130
+ response_messages = []
131
+ active_tool_calls = {}
132
+ system_node_buffer = {}
133
+
134
+ for chunk, metadata in self.rag_system.agent_graph.stream(stream_input, config=config, stream_mode="messages"):
135
+ node = metadata.get("langgraph_node", "")
136
+
137
+ if node in SYSTEM_NODES and isinstance(chunk, AIMessageChunk):
138
+ content = chunk.content
139
+ if node == "rewrite_query" and chunk.tool_call_chunks:
140
+ content = "".join(tc.get("args") or "" for tc in chunk.tool_call_chunks)
141
+ if not content:
142
+ continue
143
+ self._handle_system_node(content, node, response_messages, system_node_buffer)
144
+
145
+ elif hasattr(chunk, "tool_calls") and chunk.tool_calls:
146
+ self._handle_tool_call(chunk, response_messages, active_tool_calls)
147
+
148
+ elif isinstance(chunk, ToolMessage):
149
+ self._handle_tool_result(chunk, response_messages, active_tool_calls)
150
+
151
+ elif isinstance(chunk, AIMessageChunk) and chunk.content and node in FINAL_RESPONSE_NODES:
152
+ self._handle_llm_token(chunk, node, response_messages)
153
+
154
+ else:
155
+ continue
156
+
157
+ yield response_messages
158
+
159
+ final_state = self.rag_system.agent_graph.get_state(config)
160
+ log_chat_end(getattr(final_state, "values", final_state))
161
+
162
+ except Exception as e:
163
+ log_error("chat", e)
164
+ yield f"❌ Error: {str(e)}"
165
+
166
+ def clear_session(self, thread_id=None):
167
+ new_thread = self.rag_system.reset_thread(thread_id)
168
+ self.rag_system.observability.flush()
169
+ return new_thread
project/core/document_manager.py ADDED
@@ -0,0 +1,81 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from pathlib import Path
2
+ import shutil
3
+ import config
4
+ from utils import pdfs_to_markdowns, clear_directory_contents
5
+
6
+ class DocumentManager:
7
+
8
+ def __init__(self, rag_system):
9
+ self.rag_system = rag_system
10
+ self.markdown_dir = Path(config.MARKDOWN_DIR)
11
+ self.markdown_dir.mkdir(parents=True, exist_ok=True)
12
+
13
+ def add_documents(self, document_paths, progress_callback=None):
14
+ if not document_paths:
15
+ return 0, 0
16
+
17
+ document_paths = [document_paths] if isinstance(document_paths, str) else document_paths
18
+ document_paths = [p for p in document_paths if p and Path(p).suffix.lower() in [".pdf", ".md"]]
19
+
20
+ if not document_paths:
21
+ return 0, 0
22
+
23
+ added = 0
24
+ skipped = 0
25
+
26
+ for i, doc_path in enumerate(document_paths):
27
+ if progress_callback:
28
+ progress_callback((i + 1) / len(document_paths), f"Processing {Path(doc_path).name}")
29
+
30
+ source_path = Path(doc_path)
31
+ doc_name = source_path.stem
32
+ md_path = self.markdown_dir / f"{doc_name}.md"
33
+
34
+ if md_path.exists():
35
+ skipped += 1
36
+ continue
37
+
38
+ parent_ids = []
39
+ try:
40
+ if source_path.suffix.lower() == ".md":
41
+ shutil.copy(source_path, md_path)
42
+ else:
43
+ pdfs_to_markdowns(str(source_path), overwrite=False)
44
+ parent_chunks, child_chunks = self.rag_system.chunker.create_chunks_single(
45
+ md_path,
46
+ source_name=source_path.name,
47
+ )
48
+
49
+ if not child_chunks:
50
+ raise ValueError("No child chunks were created.")
51
+
52
+ parent_ids = [parent_id for parent_id, _ in parent_chunks]
53
+ self.rag_system.parent_store.save_many(parent_chunks)
54
+ collection = self.rag_system.vector_db.get_collection(self.rag_system.collection_name)
55
+ collection.add_documents(child_chunks)
56
+
57
+ added += 1
58
+
59
+ except Exception as e:
60
+ self.rag_system.parent_store.delete_many(parent_ids)
61
+ if md_path.exists():
62
+ md_path.unlink()
63
+ print(f"Error processing {doc_path}: {e}")
64
+ skipped += 1
65
+
66
+ return added, skipped
67
+
68
+ def get_markdown_files(self):
69
+ sources = self.rag_system.parent_store.list_sources()
70
+ if sources:
71
+ return sources
72
+ return sorted(p.name for p in self.markdown_dir.glob("*.md"))
73
+
74
+ def clear_all(self):
75
+ self.markdown_dir.mkdir(parents=True, exist_ok=True)
76
+ self.rag_system.vector_db.delete_collection(self.rag_system.collection_name)
77
+
78
+ clear_directory_contents(self.markdown_dir)
79
+ self.rag_system.parent_store.clear_store()
80
+
81
+ self.rag_system.vector_db.create_collection(self.rag_system.collection_name)
project/core/execution_logger.py ADDED
@@ -0,0 +1,213 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from __future__ import annotations
2
+
3
+ from datetime import datetime
4
+ from pprint import pformat
5
+ from typing import Any
6
+
7
+ from langchain_core.messages import AIMessage, HumanMessage, RemoveMessage, SystemMessage, ToolMessage
8
+
9
+ import config
10
+
11
+
12
+ COLORS = {
13
+ "blue": "\033[94m",
14
+ "cyan": "\033[96m",
15
+ "green": "\033[92m",
16
+ "magenta": "\033[95m",
17
+ "red": "\033[91m",
18
+ "yellow": "\033[93m",
19
+ "dim": "\033[2m",
20
+ "reset": "\033[0m",
21
+ }
22
+
23
+
24
+ def _enabled() -> bool:
25
+ return bool(getattr(config, "EXECUTION_LOGGING_ENABLED", True))
26
+
27
+
28
+ def _color(text: str, color: str) -> str:
29
+ if not getattr(config, "EXECUTION_LOG_USE_COLOR", True):
30
+ return text
31
+ return f"{COLORS.get(color, '')}{text}{COLORS['reset']}"
32
+
33
+
34
+ def _truncate(value: Any, max_chars: int | None = None) -> str:
35
+ text = "" if value is None else str(value)
36
+ limit = max_chars or getattr(config, "EXECUTION_LOG_MAX_CHARS", 1200)
37
+ if len(text) <= limit:
38
+ return text
39
+ return f"{text[:limit]}... [truncated {len(text) - limit} chars]"
40
+
41
+
42
+ def _message_role(message: Any) -> str:
43
+ if isinstance(message, HumanMessage):
44
+ return "human"
45
+ if isinstance(message, AIMessage):
46
+ return "ai"
47
+ if isinstance(message, ToolMessage):
48
+ return "tool"
49
+ if isinstance(message, SystemMessage):
50
+ return "system"
51
+ if isinstance(message, RemoveMessage):
52
+ return "remove"
53
+ return message.__class__.__name__
54
+
55
+
56
+ def _message_preview(message: Any) -> dict[str, Any]:
57
+ preview = {
58
+ "type": _message_role(message),
59
+ "id": getattr(message, "id", None),
60
+ }
61
+
62
+ if isinstance(message, RemoveMessage):
63
+ return preview
64
+
65
+ content = getattr(message, "content", "")
66
+ if content:
67
+ preview["content"] = _truncate(content)
68
+
69
+ tool_calls = getattr(message, "tool_calls", None)
70
+ if tool_calls:
71
+ preview["tool_calls"] = [
72
+ {
73
+ "name": call.get("name"),
74
+ "args": call.get("args"),
75
+ "id": call.get("id"),
76
+ }
77
+ for call in tool_calls
78
+ ]
79
+
80
+ tool_name = getattr(message, "name", None)
81
+ if tool_name:
82
+ preview["name"] = tool_name
83
+
84
+ tool_call_id = getattr(message, "tool_call_id", None)
85
+ if tool_call_id:
86
+ preview["tool_call_id"] = tool_call_id
87
+
88
+ return preview
89
+
90
+
91
+ def _messages_preview(messages: list[Any]) -> dict[str, Any]:
92
+ return {
93
+ "count": len(messages),
94
+ "last_messages": [_message_preview(message) for message in messages[-4:]],
95
+ }
96
+
97
+
98
+ def state_preview(state: Any) -> dict[str, Any]:
99
+ if not isinstance(state, dict):
100
+ return {"value": _truncate(state)}
101
+
102
+ preview: dict[str, Any] = {}
103
+
104
+ for key, value in state.items():
105
+ if key == "messages":
106
+ preview[key] = _messages_preview(value or [])
107
+ elif key in {"conversation_summary", "context_summary", "final_answer"}:
108
+ preview[key] = _truncate(value)
109
+ elif key == "agent_answers":
110
+ preview[key] = {
111
+ "count": len(value or []),
112
+ "items": [
113
+ {
114
+ "index": item.get("index"),
115
+ "question": _truncate(item.get("question", ""), 240),
116
+ "answer": _truncate(item.get("answer", ""), 500),
117
+ }
118
+ for item in (value or [])[:3]
119
+ if isinstance(item, dict)
120
+ ],
121
+ }
122
+ elif isinstance(value, set):
123
+ preview[key] = sorted(value)
124
+ else:
125
+ preview[key] = value
126
+
127
+ return preview
128
+
129
+
130
+ def update_preview(update: Any) -> Any:
131
+ if not isinstance(update, dict):
132
+ return _truncate(update)
133
+
134
+ preview: dict[str, Any] = {}
135
+ for key, value in update.items():
136
+ if key == "messages":
137
+ preview[key] = [_message_preview(message) for message in value]
138
+ elif key in {"conversation_summary", "context_summary", "final_answer"}:
139
+ preview[key] = _truncate(value)
140
+ elif key == "agent_answers":
141
+ preview[key] = state_preview({"agent_answers": value})["agent_answers"]
142
+ elif isinstance(value, set):
143
+ preview[key] = sorted(value)
144
+ else:
145
+ preview[key] = value
146
+ return preview
147
+
148
+
149
+ def _print_block(title: str, payload: Any, color: str) -> None:
150
+ if not _enabled():
151
+ return
152
+
153
+ timestamp = datetime.now().strftime("%H:%M:%S")
154
+ print(_color(f"\n[{timestamp}] {title}", color))
155
+ print(_color("-" * 80, "dim"))
156
+ print(pformat(payload, width=120, sort_dicts=False))
157
+
158
+
159
+ def log_chat_start(message: str, thread_id: str, has_pending_interrupt: bool) -> None:
160
+ _print_block(
161
+ "USER QUERY",
162
+ {
163
+ "thread_id": thread_id,
164
+ "pending_interrupt": has_pending_interrupt,
165
+ "message": _truncate(message),
166
+ },
167
+ "blue",
168
+ )
169
+
170
+
171
+ def log_chat_end(state: Any) -> None:
172
+ _print_block("FINAL GRAPH STATE", state_preview(state), "blue")
173
+
174
+
175
+ def log_node_start(name: str, state: Any) -> None:
176
+ _print_block(f"NODE START: {name}", state_preview(state), "cyan")
177
+
178
+
179
+ def log_node_end(name: str, update: Any) -> None:
180
+ _print_block(f"NODE OUTPUT: {name}", update_preview(update), "green")
181
+
182
+
183
+ def log_route(name: str, decision: Any, state: Any | None = None) -> None:
184
+ payload = {"decision": _truncate(decision)}
185
+ if state is not None:
186
+ payload["state"] = state_preview(state)
187
+ _print_block(f"ROUTE: {name}", payload, "yellow")
188
+
189
+
190
+ def log_tool_start(name: str, args: dict[str, Any]) -> None:
191
+ _print_block(f"TOOL START: {name}", args, "magenta")
192
+
193
+
194
+ def log_tool_end(name: str, output: Any) -> None:
195
+ _print_block(f"TOOL OUTPUT: {name}", {"output": _truncate(output)}, "magenta")
196
+
197
+
198
+ def log_error(scope: str, error: Exception) -> None:
199
+ _print_block(f"ERROR: {scope}", {"type": error.__class__.__name__, "message": str(error)}, "red")
200
+
201
+
202
+ def logged_node(name: str, fn):
203
+ def _wrapped(state, *args, **kwargs):
204
+ log_node_start(name, state)
205
+ try:
206
+ result = fn(state, *args, **kwargs)
207
+ except Exception as exc:
208
+ log_error(name, exc)
209
+ raise
210
+ log_node_end(name, result)
211
+ return result
212
+
213
+ return _wrapped
project/core/observability.py ADDED
@@ -0,0 +1,48 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import logging
2
+ import config
3
+
4
+ logger = logging.getLogger(__name__)
5
+
6
+
7
+ class Observability:
8
+
9
+ def __init__(self):
10
+ self._enabled = config.LANGFUSE_ENABLED
11
+ self._handler = None
12
+ self._client = None
13
+
14
+ if not self._enabled:
15
+ return
16
+
17
+ if not config.LANGFUSE_PUBLIC_KEY or not config.LANGFUSE_SECRET_KEY:
18
+ logger.warning("Langfuse enabled but API keys are missing — skipping")
19
+ self._enabled = False
20
+ return
21
+
22
+ try:
23
+ from langfuse import get_client
24
+ from langfuse.langchain import CallbackHandler
25
+
26
+ self._client = get_client()
27
+
28
+ if self._client.auth_check():
29
+ print("Langfuse client is authenticated and ready!")
30
+ else:
31
+ print("Authentication failed. Please check your credentials and host.")
32
+ self._enabled = False
33
+ return
34
+
35
+ self._handler = CallbackHandler()
36
+ except Exception as exc:
37
+ logger.warning("Could not initialize Langfuse: %s", exc)
38
+ self._enabled = False
39
+
40
+ def get_handler(self):
41
+ return self._handler
42
+
43
+ def flush(self):
44
+ if self._client is not None:
45
+ try:
46
+ self._client.flush()
47
+ except Exception:
48
+ pass
project/core/rag_system.py ADDED
@@ -0,0 +1,56 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import uuid
2
+ import os
3
+ from langchain_groq import ChatGroq
4
+ import config
5
+ from db.vector_db_manager import VectorDbManager
6
+ from db.parent_store_manager import ParentStoreManager
7
+ from document_chunker import DocumentChunker
8
+ from rag_agent.tools import ToolFactory
9
+ from rag_agent.graph import create_agent_graph
10
+ from core.observability import Observability
11
+
12
+ class RAGSystem:
13
+
14
+ def __init__(self, collection_name=config.CHILD_COLLECTION):
15
+ if not os.environ.get("GROQ_API_KEY", "").strip():
16
+ raise ValueError("Set GROQ_API_KEY in project/.env or your hosting platform's secrets before starting the app.")
17
+ self.collection_name = collection_name
18
+ self.vector_db = VectorDbManager()
19
+ self.parent_store = ParentStoreManager()
20
+ self.chunker = DocumentChunker()
21
+ self.observability = Observability()
22
+ self.agent_graph = None
23
+ self.thread_id = str(uuid.uuid4())
24
+ self.recursion_limit = config.GRAPH_RECURSION_LIMIT
25
+
26
+ def initialize(self):
27
+ self.vector_db.create_collection(self.collection_name)
28
+ collection = self.vector_db.get_collection(self.collection_name)
29
+
30
+ llm = ChatGroq(
31
+ model=config.LLM_MODEL,
32
+ temperature=config.LLM_TEMPERATURE,
33
+ timeout=config.LLM_TIMEOUT,
34
+ max_retries=config.LLM_MAX_RETRIES,
35
+ model_kwargs={"seed": config.LLM_SEED},
36
+ )
37
+ tools = ToolFactory(collection).create_tools()
38
+ self.agent_graph = create_agent_graph(llm, tools)
39
+
40
+ def get_config(self, thread_id=None):
41
+ cfg = {"configurable": {"thread_id": thread_id or self.thread_id}, "recursion_limit": self.recursion_limit}
42
+ handler = self.observability.get_handler()
43
+ if handler:
44
+ cfg["callbacks"] = [handler]
45
+ return cfg
46
+
47
+ def reset_thread(self, thread_id=None):
48
+ current_thread = thread_id or self.thread_id
49
+ try:
50
+ self.agent_graph.checkpointer.delete_thread(current_thread)
51
+ except Exception as e:
52
+ print(f"Warning: Could not delete thread {current_thread}: {e}")
53
+ new_thread = str(uuid.uuid4())
54
+ if thread_id is None:
55
+ self.thread_id = new_thread
56
+ return new_thread
project/db/__init__.py ADDED
File without changes
project/db/parent_store_manager.py ADDED
@@ -0,0 +1,68 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import re
2
+ import json
3
+ import config
4
+ from utils import clear_directory_contents
5
+ from pathlib import Path
6
+ from typing import List, Dict
7
+
8
+ class ParentStoreManager:
9
+ __store_path: Path
10
+
11
+ def __init__(self, store_path=config.PARENT_STORE_PATH):
12
+ self.__store_path = Path(store_path)
13
+ self.__store_path.mkdir(parents=True, exist_ok=True)
14
+
15
+ def save(self, parent_id: str, content: str, metadata: Dict) -> None:
16
+ file_path = self.__store_path / f"{parent_id}.json"
17
+ file_path.write_text(
18
+ json.dumps({"page_content": content,"metadata": metadata}, ensure_ascii=False, indent=2),
19
+ encoding="utf-8"
20
+ )
21
+
22
+ def save_many(self, parents: List) -> None:
23
+ for parent_id, doc in parents:
24
+ self.save(parent_id, doc.page_content, doc.metadata)
25
+
26
+ def delete_many(self, parent_ids: List[str]) -> None:
27
+ for parent_id in parent_ids:
28
+ file_path = self.__store_path / f"{parent_id}.json"
29
+ if file_path.exists():
30
+ file_path.unlink()
31
+
32
+ def load(self, parent_id: str) -> Dict:
33
+ file_path = self.__store_path / (
34
+ parent_id if parent_id.lower().endswith(".json") else f"{parent_id}.json"
35
+ )
36
+ return json.loads(file_path.read_text(encoding="utf-8"))
37
+
38
+ def load_content(self, parent_id: str) -> Dict:
39
+ data = self.load(parent_id)
40
+ return {
41
+ "content": data["page_content"],
42
+ "parent_id": parent_id,
43
+ "metadata": data["metadata"]
44
+ }
45
+
46
+ @staticmethod
47
+ def _get_sort_key(id_str):
48
+ match = re.search(r'_(?:parent_|p)(\d+)$', id_str)
49
+ return int(match.group(1)) if match else 0
50
+
51
+ def load_content_many(self, parent_ids: List[str]) -> List[Dict]:
52
+ unique_ids = set(parent_ids)
53
+ return [self.load_content(pid) for pid in sorted(unique_ids, key=self._get_sort_key)]
54
+
55
+ def list_sources(self) -> List[str]:
56
+ sources = set()
57
+ for file_path in self.__store_path.glob("*.json"):
58
+ try:
59
+ source = json.loads(file_path.read_text(encoding="utf-8")).get("metadata", {}).get("source")
60
+ if source:
61
+ sources.add(source)
62
+ except (OSError, json.JSONDecodeError):
63
+ continue
64
+ return sorted(sources)
65
+
66
+ def clear_store(self) -> None:
67
+ self.__store_path.mkdir(parents=True, exist_ok=True)
68
+ clear_directory_contents(self.__store_path)
project/db/vector_db_manager.py ADDED
@@ -0,0 +1,73 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import config
2
+ from langchain_huggingface import HuggingFaceEmbeddings
3
+ from langchain_qdrant import QdrantVectorStore, FastEmbedSparse, RetrievalMode
4
+ from qdrant_client import QdrantClient
5
+ from qdrant_client.http import models as qmodels
6
+
7
+ class VectorDbManager:
8
+ __client: QdrantClient
9
+ __dense_embeddings: HuggingFaceEmbeddings
10
+ __sparse_embeddings: FastEmbedSparse
11
+ def __init__(self):
12
+ self.__client = QdrantClient(path=config.QDRANT_DB_PATH)
13
+ self.__dense_embeddings = HuggingFaceEmbeddings(
14
+ model_name=config.DENSE_MODEL,
15
+ model_kwargs={"device": "cpu"},
16
+ )
17
+ self.__sparse_embeddings = FastEmbedSparse(model_name=config.SPARSE_MODEL)
18
+
19
+ def _dense_vector_size(self):
20
+ return len(self.__dense_embeddings.embed_query("test"))
21
+
22
+ @staticmethod
23
+ def _collection_vector_size(collection_info):
24
+ vectors_config = collection_info.config.params.vectors
25
+ if hasattr(vectors_config, "size"):
26
+ return vectors_config.size
27
+ if isinstance(vectors_config, dict) and vectors_config:
28
+ first_vector = next(iter(vectors_config.values()))
29
+ return getattr(first_vector, "size", None)
30
+ return None
31
+
32
+ def create_collection(self, collection_name):
33
+ expected_size = self._dense_vector_size()
34
+ if not self.__client.collection_exists(collection_name):
35
+ print(f"Creating collection: {collection_name}...")
36
+ self.__client.create_collection(
37
+ collection_name=collection_name,
38
+ vectors_config=qmodels.VectorParams(size=expected_size, distance=qmodels.Distance.COSINE),
39
+ sparse_vectors_config={config.SPARSE_VECTOR_NAME: qmodels.SparseVectorParams()},
40
+ )
41
+ print(f"✓ Collection created: {collection_name}")
42
+ else:
43
+ collection_info = self.__client.get_collection(collection_name)
44
+ existing_size = self._collection_vector_size(collection_info)
45
+ if existing_size and existing_size != expected_size:
46
+ raise ValueError(
47
+ f"Qdrant collection '{collection_name}' has dense vector size "
48
+ f"{existing_size}, but '{config.DENSE_MODEL}' produces size "
49
+ f"{expected_size}. Clear and re-index the collection after "
50
+ "changing embedding models."
51
+ )
52
+ print(f"✓ Collection already exists: {collection_name}")
53
+
54
+ def delete_collection(self, collection_name):
55
+ try:
56
+ if self.__client.collection_exists(collection_name):
57
+ print(f"Removing existing Qdrant collection: {collection_name}")
58
+ self.__client.delete_collection(collection_name)
59
+ except Exception as e:
60
+ raise RuntimeError(f"Unable to delete Qdrant collection '{collection_name}'.") from e
61
+
62
+ def get_collection(self, collection_name) -> QdrantVectorStore:
63
+ try:
64
+ return QdrantVectorStore(
65
+ client=self.__client,
66
+ collection_name=collection_name,
67
+ embedding=self.__dense_embeddings,
68
+ sparse_embedding=self.__sparse_embeddings,
69
+ retrieval_mode=RetrievalMode.HYBRID,
70
+ sparse_vector_name=config.SPARSE_VECTOR_NAME
71
+ )
72
+ except Exception as e:
73
+ raise RuntimeError(f"Unable to initialize Qdrant collection '{collection_name}'.") from e
project/document_chunker.py ADDED
@@ -0,0 +1,190 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import os
2
+ import glob
3
+ import config
4
+ from pathlib import Path
5
+ from langchain_text_splitters import MarkdownHeaderTextSplitter, RecursiveCharacterTextSplitter
6
+
7
+ class DocumentChunker:
8
+ def __init__(self):
9
+ if config.MIN_PARENT_SIZE <= 0 or config.MAX_PARENT_SIZE < config.MIN_PARENT_SIZE:
10
+ raise ValueError("Parent chunk sizes must be positive and MIN_PARENT_SIZE <= MAX_PARENT_SIZE.")
11
+ if not 0 <= config.CHILD_CHUNK_OVERLAP < config.CHILD_CHUNK_SIZE:
12
+ raise ValueError("CHILD_CHUNK_OVERLAP must be smaller than CHILD_CHUNK_SIZE.")
13
+ if config.CHILD_CHUNK_OVERLAP >= config.MAX_PARENT_SIZE:
14
+ raise ValueError("CHILD_CHUNK_OVERLAP must be smaller than MAX_PARENT_SIZE.")
15
+
16
+ self.__parent_splitter = MarkdownHeaderTextSplitter(
17
+ headers_to_split_on=config.HEADERS_TO_SPLIT_ON,
18
+ strip_headers=False
19
+ )
20
+ self.__child_splitter = RecursiveCharacterTextSplitter(
21
+ chunk_size=config.CHILD_CHUNK_SIZE,
22
+ chunk_overlap=config.CHILD_CHUNK_OVERLAP
23
+ )
24
+ self.__min_parent_size = config.MIN_PARENT_SIZE
25
+ self.__max_parent_size = config.MAX_PARENT_SIZE
26
+
27
+ @staticmethod
28
+ def __merge_metadata(target, source, prepend=False):
29
+ for key, value in source.items():
30
+ if key not in target:
31
+ target[key] = value
32
+ else:
33
+ first, second = (value, target[key]) if prepend else (target[key], value)
34
+ values = [
35
+ item.strip()
36
+ for raw in (first, second)
37
+ for item in str(raw).split(" -> ")
38
+ if item.strip()
39
+ ]
40
+ target[key] = " -> ".join(dict.fromkeys(values))
41
+
42
+ def create_chunks(self, path_dir=config.MARKDOWN_DIR):
43
+ all_parent_chunks, all_child_chunks = [], []
44
+
45
+ for doc_path_str in sorted(glob.glob(os.path.join(path_dir, "*.md"))):
46
+ doc_path = Path(doc_path_str)
47
+ parent_chunks, child_chunks = self.create_chunks_single(doc_path)
48
+ all_parent_chunks.extend(parent_chunks)
49
+ all_child_chunks.extend(child_chunks)
50
+
51
+ return all_parent_chunks, all_child_chunks
52
+
53
+ def create_chunks_single(self, md_path, source_name=None):
54
+ doc_path = Path(md_path)
55
+ source_name = source_name or f"{doc_path.stem}.pdf"
56
+
57
+ with open(doc_path, "r", encoding="utf-8") as f:
58
+ parent_chunks = self.__parent_splitter.split_text(f.read())
59
+
60
+ merged_parents = self.__merge_small_parents(parent_chunks)
61
+ split_parents = self.__split_large_parents(merged_parents)
62
+ cleaned_parents = self.__clean_small_chunks(split_parents)
63
+ if any(len(chunk.page_content) > self.__max_parent_size for chunk in cleaned_parents):
64
+ raise ValueError("Parent chunking produced a chunk larger than MAX_PARENT_SIZE.")
65
+
66
+ all_parent_chunks, all_child_chunks = [], []
67
+ self.__create_child_chunks(
68
+ all_parent_chunks,
69
+ all_child_chunks,
70
+ cleaned_parents,
71
+ doc_path,
72
+ source_name,
73
+ )
74
+ return all_parent_chunks, all_child_chunks
75
+
76
+ def __merge_small_parents(self, chunks):
77
+ if not chunks:
78
+ return []
79
+
80
+ merged, current = [], None
81
+
82
+ for chunk in chunks:
83
+ if current is None:
84
+ current = chunk
85
+ else:
86
+ current.page_content += "\n\n" + chunk.page_content
87
+ self.__merge_metadata(current.metadata, chunk.metadata)
88
+
89
+ if len(current.page_content) >= self.__min_parent_size:
90
+ merged.append(current)
91
+ current = None
92
+
93
+ if current:
94
+ if merged:
95
+ merged[-1].page_content += "\n\n" + current.page_content
96
+ self.__merge_metadata(merged[-1].metadata, current.metadata)
97
+ else:
98
+ merged.append(current)
99
+
100
+ return merged
101
+
102
+ def __split_large_parents(self, chunks):
103
+ split_chunks = []
104
+
105
+ for chunk in chunks:
106
+ if len(chunk.page_content) <= self.__max_parent_size:
107
+ split_chunks.append(chunk)
108
+ else:
109
+ splitter = RecursiveCharacterTextSplitter(
110
+ chunk_size=self.__max_parent_size,
111
+ chunk_overlap=config.CHILD_CHUNK_OVERLAP
112
+ )
113
+ sub_chunks = splitter.split_documents([chunk])
114
+ split_chunks.extend(sub_chunks)
115
+
116
+ return split_chunks
117
+
118
+ def __rebalance_pair(self, first, second):
119
+ combined = first.page_content.rstrip() + "\n\n" + second.page_content.lstrip()
120
+ lower = max(1, len(combined) - self.__max_parent_size)
121
+ upper = min(self.__max_parent_size, len(combined) - 1)
122
+ if len(combined) >= 2 * self.__min_parent_size:
123
+ lower = max(lower, self.__min_parent_size)
124
+ upper = min(upper, len(combined) - self.__min_parent_size)
125
+ preferred = min(max(len(combined) // 2, lower), upper)
126
+
127
+ split_at = preferred
128
+ for separator in ("\n\n", "\n", " "):
129
+ before = combined.rfind(separator, lower, preferred + 1)
130
+ after = combined.find(separator, preferred, upper + 1)
131
+ if before >= lower:
132
+ split_at = before
133
+ break
134
+ if after != -1:
135
+ split_at = after
136
+ break
137
+
138
+ left_text = combined[:split_at].rstrip()
139
+ right_text = combined[split_at:].lstrip()
140
+ if len(combined) >= 2 * self.__min_parent_size and (
141
+ len(left_text) < self.__min_parent_size
142
+ or len(right_text) < self.__min_parent_size
143
+ ):
144
+ split_at = preferred
145
+ left_text, right_text = combined[:split_at], combined[split_at:]
146
+ if not left_text or not right_text:
147
+ return first, second
148
+
149
+ metadata = dict(first.metadata)
150
+ self.__merge_metadata(metadata, second.metadata)
151
+ first.page_content, first.metadata = left_text, dict(metadata)
152
+ second.page_content, second.metadata = right_text, dict(metadata)
153
+ return first, second
154
+
155
+ def __clean_small_chunks(self, chunks):
156
+ cleaned = []
157
+
158
+ for i, chunk in enumerate(chunks):
159
+ if len(chunk.page_content) < self.__min_parent_size:
160
+ if cleaned and len(cleaned[-1].page_content) + 2 + len(chunk.page_content) <= self.__max_parent_size:
161
+ cleaned[-1].page_content += "\n\n" + chunk.page_content
162
+ self.__merge_metadata(cleaned[-1].metadata, chunk.metadata)
163
+ elif (
164
+ i < len(chunks) - 1
165
+ and len(chunk.page_content) + 2 + len(chunks[i + 1].page_content) <= self.__max_parent_size
166
+ ):
167
+ chunks[i + 1].page_content = chunk.page_content + "\n\n" + chunks[i + 1].page_content
168
+ self.__merge_metadata(chunks[i + 1].metadata, chunk.metadata, prepend=True)
169
+ else:
170
+ cleaned.append(chunk)
171
+ else:
172
+ cleaned.append(chunk)
173
+
174
+ for i, chunk in enumerate(cleaned):
175
+ if len(chunk.page_content) >= self.__min_parent_size or len(cleaned) == 1:
176
+ continue
177
+ if i < len(cleaned) - 1:
178
+ cleaned[i], cleaned[i + 1] = self.__rebalance_pair(chunk, cleaned[i + 1])
179
+ else:
180
+ cleaned[i - 1], cleaned[i] = self.__rebalance_pair(cleaned[i - 1], chunk)
181
+
182
+ return cleaned
183
+
184
+ def __create_child_chunks(self, all_parent_pairs, all_child_chunks, parent_chunks, doc_path, source_name):
185
+ for i, p_chunk in enumerate(parent_chunks):
186
+ parent_id = f"{doc_path.stem}_p{i}"
187
+ p_chunk.metadata.update({"source": source_name, "parent_id": parent_id})
188
+
189
+ all_parent_pairs.append((parent_id, p_chunk))
190
+ all_child_chunks.extend(self.__child_splitter.split_documents([p_chunk]))
project/rag_agent/__init__.py ADDED
File without changes
project/rag_agent/edges.py ADDED
@@ -0,0 +1,40 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from typing import Literal
2
+ from langgraph.types import Send
3
+ from .graph_state import State, AgentState
4
+ from config import MAX_ITERATIONS, MAX_TOOL_CALLS
5
+ from core.execution_logger import log_route
6
+
7
+ def route_after_rewrite(state: State) -> Literal["request_clarification", "agent"]:
8
+ if not state.get("questionIsClear", False):
9
+ decision = "request_clarification"
10
+ else:
11
+ decision = [
12
+ Send("agent", {"question": query, "question_index": idx, "messages": []})
13
+ for idx, query in enumerate(state["rewrittenQuestions"])
14
+ ]
15
+ log_route("after_rewrite", decision, state)
16
+ return decision
17
+
18
+ def route_after_orchestrator_call(state: AgentState) -> Literal["tools", "fallback_response", "collect_answer"]:
19
+ iteration = state.get("iteration_count", 0)
20
+ tool_count = state.get("tool_call_count", 0)
21
+
22
+ last_message = state["messages"][-1]
23
+ tool_calls = getattr(last_message, "tool_calls", None) or []
24
+
25
+ if not tool_calls:
26
+ decision = "collect_answer"
27
+ log_route("after_orchestrator_call", decision, state)
28
+ return decision
29
+
30
+ # The counters already include the current LLM response. Allow a final
31
+ # answer at the iteration boundary, but do not execute tool calls that
32
+ # would exceed the configured research budget.
33
+ if iteration >= MAX_ITERATIONS or tool_count > MAX_TOOL_CALLS:
34
+ decision = "fallback_response"
35
+ log_route("after_orchestrator_call", decision, state)
36
+ return decision
37
+
38
+ decision = "tools"
39
+ log_route("after_orchestrator_call", decision, state)
40
+ return decision
project/rag_agent/graph.py ADDED
@@ -0,0 +1,62 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from langgraph.graph import START, END, StateGraph
2
+ from langgraph.checkpoint.memory import InMemorySaver
3
+ from langgraph.prebuilt import ToolNode
4
+ from functools import partial
5
+
6
+ from .graph_state import State, AgentState
7
+ from core.execution_logger import logged_node
8
+ from .nodes import (
9
+ aggregate_answers,
10
+ collect_answer,
11
+ compress_context,
12
+ fallback_response,
13
+ orchestrator,
14
+ request_clarification,
15
+ rewrite_query,
16
+ should_compress_context,
17
+ summarize_history,
18
+ )
19
+ from .edges import route_after_orchestrator_call, route_after_rewrite
20
+
21
+ def create_agent_graph(llm, tools_list):
22
+ llm_with_tools = llm.bind_tools(tools_list)
23
+ tool_node = ToolNode(tools_list)
24
+
25
+ checkpointer = InMemorySaver()
26
+
27
+ print("Compiling agent graph...")
28
+ agent_builder = StateGraph(AgentState)
29
+ agent_builder.add_node("orchestrator", logged_node("agent.orchestrator", partial(orchestrator, llm_with_tools=llm_with_tools)))
30
+ agent_builder.add_node("tools", tool_node)
31
+ agent_builder.add_node("compress_context", logged_node("agent.compress_context", partial(compress_context, llm=llm)))
32
+ agent_builder.add_node("fallback_response", logged_node("agent.fallback_response", partial(fallback_response, llm=llm)))
33
+ agent_builder.add_node("should_compress_context", logged_node("agent.should_compress_context", should_compress_context))
34
+ agent_builder.add_node("collect_answer", logged_node("agent.collect_answer", collect_answer))
35
+
36
+ agent_builder.add_edge(START, "orchestrator")
37
+ agent_builder.add_conditional_edges("orchestrator", route_after_orchestrator_call, {"tools": "tools", "fallback_response": "fallback_response", "collect_answer": "collect_answer"})
38
+ agent_builder.add_edge("tools", "should_compress_context")
39
+ agent_builder.add_edge("compress_context", "orchestrator")
40
+ agent_builder.add_edge("fallback_response", "collect_answer")
41
+ agent_builder.add_edge("collect_answer", END)
42
+
43
+ agent_subgraph = agent_builder.compile()
44
+
45
+ graph_builder = StateGraph(State)
46
+ graph_builder.add_node("summarize_history", logged_node("main.summarize_history", partial(summarize_history, llm=llm)))
47
+ graph_builder.add_node("rewrite_query", logged_node("main.rewrite_query", partial(rewrite_query, llm=llm)))
48
+ graph_builder.add_node("request_clarification", logged_node("main.request_clarification", request_clarification))
49
+ graph_builder.add_node("agent", agent_subgraph)
50
+ graph_builder.add_node("aggregate_answers", logged_node("main.aggregate_answers", partial(aggregate_answers, llm=llm)))
51
+
52
+ graph_builder.add_edge(START, "summarize_history")
53
+ graph_builder.add_edge("summarize_history", "rewrite_query")
54
+ graph_builder.add_conditional_edges("rewrite_query", route_after_rewrite)
55
+ graph_builder.add_edge("request_clarification", "rewrite_query")
56
+ graph_builder.add_edge(["agent"], "aggregate_answers")
57
+ graph_builder.add_edge("aggregate_answers", END)
58
+
59
+ agent_graph = graph_builder.compile(checkpointer=checkpointer, interrupt_before=["request_clarification"])
60
+
61
+ print("✓ Agent graph compiled successfully.")
62
+ return agent_graph
project/rag_agent/graph_state.py ADDED
@@ -0,0 +1,36 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from typing import List, Annotated, Set
2
+ from langgraph.graph import MessagesState
3
+ import operator
4
+
5
+ def accumulate_or_reset(existing: List[dict], new: List[dict]) -> List[dict]:
6
+ if new and any(item.get('__reset__') for item in new):
7
+ return []
8
+ return existing + new
9
+
10
+ def set_union(a: Set[str], b: Set[str]) -> Set[str]:
11
+ return a | b
12
+
13
+ def append_unique(existing: List[str], new: List[str]) -> List[str]:
14
+ return list(dict.fromkeys(existing + new))
15
+
16
+ class State(MessagesState):
17
+ """State for main agent graph"""
18
+ questionIsClear: bool = False
19
+ conversation_summary: str = ""
20
+ originalQuery: str = ""
21
+ pendingQuery: str = ""
22
+ pendingClarifications: List[str] = []
23
+ rewrittenQuestions: List[str] = []
24
+ agent_answers: Annotated[List[dict], accumulate_or_reset] = []
25
+
26
+ class AgentState(MessagesState):
27
+ """State for individual agent subgraph"""
28
+ question: str = ""
29
+ question_index: int = 0
30
+ context_summary: str = ""
31
+ retrieval_keys: Annotated[Set[str], set_union] = set()
32
+ retrieved_contexts: Annotated[List[str], append_unique] = []
33
+ final_answer: str = ""
34
+ agent_answers: List[dict] = []
35
+ tool_call_count: Annotated[int, operator.add] = 0
36
+ iteration_count: Annotated[int, operator.add] = 0
project/rag_agent/nodes.py ADDED
@@ -0,0 +1,331 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from typing import Literal, Set
2
+ from langchain_core.messages import SystemMessage, HumanMessage, RemoveMessage, AIMessage, ToolMessage
3
+ from langgraph.types import Command
4
+ from .graph_state import State, AgentState
5
+ from .schemas import QueryAnalysis
6
+ from .prompts import *
7
+ from utils import estimate_context_tokens
8
+ from config import BASE_TOKEN_THRESHOLD, CHILD_CHUNK_SEPARATOR, MAIN_HISTORY_MESSAGES_TO_KEEP, TOKEN_GROWTH_FACTOR
9
+
10
+ if MAIN_HISTORY_MESSAGES_TO_KEEP < 2:
11
+ raise ValueError("MAIN_HISTORY_MESSAGES_TO_KEEP must be at least 2.")
12
+
13
+ PRE_ANSWER_HISTORY_MESSAGES_TO_KEEP = max(MAIN_HISTORY_MESSAGES_TO_KEEP - 1, 0)
14
+
15
+ def _is_plain_conversation_message(msg) -> bool:
16
+ return (
17
+ isinstance(msg, (HumanMessage, AIMessage))
18
+ and not getattr(msg, "tool_calls", None)
19
+ and not getattr(msg, "name", None)
20
+ )
21
+
22
+ def _name_internal_message(message, name):
23
+ """Tag a subgraph-only message so it is not treated as chat history."""
24
+ return message.model_copy(update={"name": name})
25
+
26
+ def _retrieval_contexts(messages) -> list[str]:
27
+ contexts = []
28
+ ignored_prefixes = (
29
+ "NO_RELEVANT_CHUNKS",
30
+ "NO_PARENT_DOCUMENT",
31
+ "RETRIEVAL_ERROR:",
32
+ "PARENT_RETRIEVAL_ERROR:",
33
+ )
34
+ for message in messages:
35
+ if not isinstance(message, ToolMessage):
36
+ continue
37
+ content = str(message.content).strip()
38
+ if content and not content.startswith(ignored_prefixes):
39
+ parts = content.split(CHILD_CHUNK_SEPARATOR) if message.name == "search_child_chunks" else [content]
40
+ contexts.extend(part for part in parts if part)
41
+ return list(dict.fromkeys(contexts))
42
+
43
+ def _format_conversation(messages) -> str:
44
+ lines = []
45
+ for msg in messages:
46
+ role = "User" if isinstance(msg, HumanMessage) else "Assistant"
47
+ lines.append(f"{role}: {msg.content}")
48
+ return "\n".join(lines)
49
+
50
+ def _remove_messages_not_in(messages, keep_ids):
51
+ removals = []
52
+ for msg in messages:
53
+ msg_id = getattr(msg, "id", None)
54
+ if isinstance(msg, SystemMessage) or not msg_id:
55
+ continue
56
+ if msg_id not in keep_ids:
57
+ removals.append(RemoveMessage(id=msg_id))
58
+ return removals
59
+
60
+ def _recent_conversation(messages, pending_query="") -> list:
61
+ """Return recent context before the current user message.
62
+
63
+ During clarification, exclude the unresolved query and the assistant's
64
+ clarification request because they are represented explicitly.
65
+ """
66
+ plain_messages = [msg for msg in messages if _is_plain_conversation_message(msg)]
67
+ recent_messages = plain_messages[:-1]
68
+
69
+ if pending_query:
70
+ for index in range(len(recent_messages) - 1, -1, -1):
71
+ msg = recent_messages[index]
72
+ if isinstance(msg, HumanMessage) and str(msg.content).strip() == pending_query:
73
+ return recent_messages[:index]
74
+
75
+ return recent_messages
76
+
77
+ def summarize_history(state: State, llm):
78
+ messages = state.get("messages", [])
79
+ updates = {"agent_answers": [{"__reset__": True}]}
80
+
81
+ if not messages:
82
+ return updates
83
+
84
+ plain_messages = [msg for msg in messages if _is_plain_conversation_message(msg)]
85
+ keep_count = PRE_ANSWER_HISTORY_MESSAGES_TO_KEEP
86
+ messages_to_summarize = plain_messages[:-keep_count] if len(plain_messages) > keep_count else []
87
+ keep_ids = {getattr(msg, "id", None) for msg in plain_messages[-keep_count:]}
88
+ keep_ids.discard(None)
89
+
90
+ removals = _remove_messages_not_in(messages, keep_ids)
91
+ if removals:
92
+ updates["messages"] = removals
93
+
94
+ if not messages_to_summarize:
95
+ return updates
96
+
97
+ existing_summary = state.get("conversation_summary", "").strip()
98
+ conversation = "Existing summary:\n"
99
+ conversation += f"{existing_summary or '(none)'}\n\n"
100
+ conversation += "New messages to merge into the summary:\n"
101
+ conversation += _format_conversation(messages_to_summarize)
102
+
103
+ summary_response = llm.invoke([
104
+ SystemMessage(content=get_conversation_summary_prompt()),
105
+ HumanMessage(content=conversation),
106
+ ])
107
+ updates["conversation_summary"] = summary_response.content.strip()
108
+ return updates
109
+
110
+ def rewrite_query(state: State, llm):
111
+ last_message = state["messages"][-1]
112
+ current_query = str(last_message.content).strip()
113
+ conversation_summary = state.get("conversation_summary", "").strip()
114
+ pending_query = state.get("pendingQuery", "").strip()
115
+ pending_clarifications = state.get("pendingClarifications", [])
116
+ recent_messages = _recent_conversation(state["messages"], pending_query)
117
+
118
+ context_parts = []
119
+ if conversation_summary:
120
+ context_parts.append(f"Conversation Summary:\n{conversation_summary}")
121
+ if recent_messages:
122
+ context_parts.append(f"Recent Conversation:\n{_format_conversation(recent_messages)}")
123
+
124
+ if pending_query:
125
+ clarifications = [*pending_clarifications, current_query]
126
+ clarification_text = "\n".join(
127
+ f"{index}. {value}" for index, value in enumerate(clarifications, start=1)
128
+ )
129
+ context_parts.append(
130
+ f"Unresolved User Query:\n{pending_query}\n\n"
131
+ f"User Clarifications:\n{clarification_text}"
132
+ )
133
+ original_query = f"{pending_query}\nClarifications:\n{clarification_text}"
134
+ else:
135
+ clarifications = []
136
+ context_parts.append(f"User Query:\n{current_query}")
137
+ original_query = current_query
138
+
139
+ context_section = "\n\n".join(context_parts)
140
+ llm_with_structure = llm.with_structured_output(QueryAnalysis, method="function_calling")
141
+ response = llm_with_structure.invoke([SystemMessage(content=get_rewrite_query_prompt()), HumanMessage(content=context_section)])
142
+ clarification_message_update = (
143
+ [_name_internal_message(last_message, "clarification_response")]
144
+ if pending_query else []
145
+ )
146
+
147
+ if response.questions and response.is_clear:
148
+ return {
149
+ "questionIsClear": True,
150
+ "originalQuery": original_query,
151
+ "pendingQuery": "",
152
+ "pendingClarifications": [],
153
+ "rewrittenQuestions": response.questions,
154
+ "messages": clarification_message_update,
155
+ }
156
+
157
+ clarification = response.clarification_needed if response.clarification_needed and len(response.clarification_needed.strip()) > 10 else "I need more information to understand your question."
158
+ return {
159
+ "questionIsClear": False,
160
+ "originalQuery": "",
161
+ "pendingQuery": pending_query or current_query,
162
+ "pendingClarifications": clarifications,
163
+ "rewrittenQuestions": [],
164
+ "messages": clarification_message_update + [
165
+ AIMessage(content=clarification, name="clarification")
166
+ ],
167
+ }
168
+
169
+ def request_clarification(state: State):
170
+ return {}
171
+
172
+ # --- Agent Nodes ---
173
+ def orchestrator(state: AgentState, llm_with_tools):
174
+ context_summary = state.get("context_summary", "").strip()
175
+ sys_msg = SystemMessage(content=get_orchestrator_prompt())
176
+ summary_injection = (
177
+ [HumanMessage(content=f"[COMPRESSED CONTEXT FROM PRIOR RESEARCH]\n\n{context_summary}")]
178
+ if context_summary else []
179
+ )
180
+ if not state.get("messages"):
181
+ human_msg = HumanMessage(content=state["question"], name="agent_question")
182
+ force_search = HumanMessage(content="YOU MUST CALL 'search_child_chunks' AS THE FIRST STEP TO ANSWER THIS QUESTION.")
183
+ response = llm_with_tools.invoke([sys_msg] + summary_injection + [human_msg, force_search])
184
+ response = _name_internal_message(response, "agent_response")
185
+ return {"messages": [human_msg, response], "tool_call_count": len(response.tool_calls or []), "iteration_count": 1}
186
+
187
+ response = llm_with_tools.invoke([sys_msg] + summary_injection + state["messages"])
188
+ response = _name_internal_message(response, "agent_response")
189
+ tool_calls = response.tool_calls if hasattr(response, "tool_calls") else []
190
+ return {"messages": [response], "tool_call_count": len(tool_calls) if tool_calls else 0, "iteration_count": 1}
191
+
192
+ def fallback_response(state: AgentState, llm):
193
+ seen = set()
194
+ unique_contents = []
195
+ for m in state["messages"]:
196
+ if isinstance(m, ToolMessage) and m.content not in seen:
197
+ unique_contents.append(m.content)
198
+ seen.add(m.content)
199
+
200
+ context_summary = state.get("context_summary", "").strip()
201
+
202
+ context_parts = []
203
+ if context_summary:
204
+ context_parts.append(f"## Compressed Research Context (from prior iterations)\n\n{context_summary}")
205
+ if unique_contents:
206
+ context_parts.append(
207
+ "## Retrieved Data (current iteration)\n\n" +
208
+ "\n\n".join(f"--- DATA SOURCE {i} ---\n{content}" for i, content in enumerate(unique_contents, 1))
209
+ )
210
+
211
+ context_text = "\n\n".join(context_parts) if context_parts else "No data was retrieved from the documents."
212
+
213
+ prompt_content = (
214
+ f"USER QUERY: {state.get('question')}\n\n"
215
+ f"{context_text}\n\n"
216
+ f"INSTRUCTION:\nProvide the best possible answer using only the data above."
217
+ )
218
+ response = llm.invoke([SystemMessage(content=get_fallback_response_prompt()), HumanMessage(content=prompt_content)])
219
+ response = _name_internal_message(response, "agent_response")
220
+ return {"messages": [response]}
221
+
222
+ def should_compress_context(state: AgentState) -> Command[Literal["compress_context", "orchestrator"]]:
223
+ messages = state["messages"]
224
+
225
+ new_ids: Set[str] = set()
226
+ for msg in reversed(messages):
227
+ if isinstance(msg, AIMessage) and getattr(msg, "tool_calls", None):
228
+ for tc in msg.tool_calls:
229
+ if tc["name"] == "retrieve_parent_chunks":
230
+ raw = tc["args"].get("parent_id") or tc["args"].get("id") or tc["args"].get("ids") or []
231
+ if isinstance(raw, str):
232
+ new_ids.add(f"parent::{raw}")
233
+ else:
234
+ new_ids.update(f"parent::{r}" for r in raw)
235
+
236
+ elif tc["name"] == "search_child_chunks":
237
+ query = tc["args"].get("query", "")
238
+ if query:
239
+ new_ids.add(f"search::{query}")
240
+ break
241
+
242
+ updated_ids = state.get("retrieval_keys", set()) | new_ids
243
+
244
+ current_token_messages = estimate_context_tokens(messages)
245
+ current_token_summary = estimate_context_tokens([HumanMessage(content=state.get("context_summary", ""))])
246
+ current_tokens = current_token_messages + current_token_summary
247
+
248
+ max_allowed = BASE_TOKEN_THRESHOLD + int(current_token_summary * TOKEN_GROWTH_FACTOR)
249
+
250
+ goto = "compress_context" if current_tokens > max_allowed else "orchestrator"
251
+ return Command(
252
+ update={
253
+ "retrieval_keys": updated_ids,
254
+ "retrieved_contexts": _retrieval_contexts(messages),
255
+ },
256
+ goto=goto,
257
+ )
258
+
259
+ def compress_context(state: AgentState, llm):
260
+ messages = state["messages"]
261
+ existing_summary = state.get("context_summary", "").strip()
262
+
263
+ if not messages:
264
+ return {}
265
+
266
+ conversation_text = f"USER QUESTION:\n{state.get('question')}\n\nConversation to compress:\n\n"
267
+ if existing_summary:
268
+ conversation_text += f"[PRIOR COMPRESSED CONTEXT]\n{existing_summary}\n\n"
269
+
270
+ for msg in messages[1:]:
271
+ if isinstance(msg, AIMessage):
272
+ tool_calls_info = ""
273
+ if getattr(msg, "tool_calls", None):
274
+ calls = ", ".join(f"{tc['name']}({tc['args']})" for tc in msg.tool_calls)
275
+ tool_calls_info = f" | Tool calls: {calls}"
276
+ conversation_text += f"[ASSISTANT{tool_calls_info}]\n{msg.content or '(tool call only)'}\n\n"
277
+ elif isinstance(msg, ToolMessage):
278
+ tool_name = getattr(msg, "name", "tool")
279
+ conversation_text += f"[TOOL RESULT — {tool_name}]\n{msg.content}\n\n"
280
+
281
+ summary_response = llm.invoke([SystemMessage(content=get_context_compression_prompt()), HumanMessage(content=conversation_text)])
282
+ new_summary = summary_response.content
283
+
284
+ retrieved_ids: Set[str] = state.get("retrieval_keys", set())
285
+ if retrieved_ids:
286
+ parent_ids = sorted(r for r in retrieved_ids if r.startswith("parent::"))
287
+ search_queries = sorted(r.replace("search::", "") for r in retrieved_ids if r.startswith("search::"))
288
+
289
+ block = "\n\n---\n**Already executed (do NOT repeat):**\n"
290
+ if parent_ids:
291
+ block += "Parent chunks retrieved:\n" + "\n".join(f"- {p.replace('parent::', '')}" for p in parent_ids) + "\n"
292
+ if search_queries:
293
+ block += "Search queries already run:\n" + "\n".join(f"- {q}" for q in search_queries) + "\n"
294
+ new_summary += block
295
+
296
+ return {"context_summary": new_summary, "messages": [RemoveMessage(id=m.id) for m in messages[1:]]}
297
+
298
+ def collect_answer(state: AgentState):
299
+ last_message = state["messages"][-1]
300
+ is_valid = isinstance(last_message, AIMessage) and last_message.content and not last_message.tool_calls
301
+ answer = last_message.content if is_valid else "Unable to generate an answer."
302
+ return {
303
+ "final_answer": answer,
304
+ "agent_answers": [{
305
+ "index": state["question_index"],
306
+ "question": state["question"],
307
+ "answer": answer,
308
+ "contexts": state.get("retrieved_contexts", []),
309
+ }]
310
+ }
311
+ # --- End of Agent Nodes---
312
+
313
+ def aggregate_answers(state: State, llm):
314
+ messages = state.get("messages", [])
315
+ plain_messages = [msg for msg in messages if _is_plain_conversation_message(msg)]
316
+ keep_ids = {getattr(msg, "id", None) for msg in plain_messages[-PRE_ANSWER_HISTORY_MESSAGES_TO_KEEP:]}
317
+ keep_ids.discard(None)
318
+ removals = _remove_messages_not_in(messages, keep_ids)
319
+
320
+ if not state.get("agent_answers"):
321
+ return {"messages": removals + [AIMessage(content="No answers were generated.")]}
322
+
323
+ sorted_answers = sorted(state["agent_answers"], key=lambda x: x["index"])
324
+
325
+ formatted_answers = ""
326
+ for i, ans in enumerate(sorted_answers, start=1):
327
+ formatted_answers += (f"\nRetrieved response {i}:\n"f"{ans['answer']}\n")
328
+
329
+ user_message = HumanMessage(content=f"""Original user question: {state["originalQuery"]}\nRetrieved answers:{formatted_answers}""")
330
+ synthesis_response = llm.invoke([SystemMessage(content=get_aggregation_prompt()), user_message])
331
+ return {"messages": removals + [AIMessage(content=synthesis_response.content)]}
project/rag_agent/prompts.py ADDED
@@ -0,0 +1,151 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ def get_conversation_summary_prompt() -> str:
2
+ return """## Role
3
+ You are a compact memory manager for a retrieval-augmented chat assistant.
4
+
5
+ ## Context
6
+ The input contains an existing rolling summary plus older user/assistant messages that will be removed from raw chat history.
7
+
8
+ ## Instructions
9
+ - Merge the existing summary with the new older messages.
10
+ - Preserve context needed for future follow-up questions: topics, user preferences, important facts, unresolved questions, and referenced source file names.
11
+ - Discard greetings, tool calls, tool outputs, formatting chatter, duplicate details, and resolved misunderstandings.
12
+ - Keep the summary compact: 30-70 words unless more detail is essential.
13
+
14
+ ## Output
15
+ Return exactly one merged summary and nothing else.
16
+ Do not include labels such as "Updated summary:", "Previous summary:", or "New messages:".
17
+ Do not include both old and new summaries.
18
+ If there is no meaningful context, return an empty string.
19
+ """
20
+
21
+ def get_rewrite_query_prompt() -> str:
22
+ return """## Role
23
+ You are a query rewriting specialist for document retrieval in a RAG system.
24
+
25
+ ## Instructions
26
+ - Rewrite the current query so it is clear, self-contained, and useful for retrieval.
27
+ - Use the conversation summary and recent conversation only to resolve vague follow-ups that refer to prior context.
28
+ - When an unresolved query and one or more user clarifications are provided, combine all of them into one self-contained retrieval query.
29
+ - If the query is a follow-up, integrate only the minimal context needed to make it self-contained.
30
+ - Preserve product names, file names, versions, acronyms, numbers, and technical terms exactly.
31
+ - If the user asks about a named topic, product, file, acronym, term, or concept, treat the question as clear even if it is new.
32
+ - Standalone named terms, acronyms, or concepts are valid retrieval queries; do not require prior conversation context.
33
+ - Split only truly separate information needs, with a maximum of 3 rewritten questions.
34
+
35
+ ## Clarification Boundary
36
+ Mark the query unclear only when it depends on an unresolved reference such as "it", "that", "this file", or "the previous one".
37
+ Do not mark a query unclear because the topic was not mentioned earlier.
38
+ Do not ask the user whether a new acronym or term is a typo; preserve it and search for it.
39
+
40
+ ## Constraints
41
+ Do not add facts, expand acronyms, invent context, or broaden the user's meaning.
42
+ """
43
+
44
+ def get_orchestrator_prompt() -> str:
45
+ return """## Role
46
+ You are a document-grounded research assistant for an agentic RAG system. Your job is to answer using retrieved document evidence, not general knowledge.
47
+
48
+ ## Available Context
49
+ - Current user question
50
+ - Optional compressed context from prior retrieval steps
51
+ - Tools for searching child chunks and loading full parent chunks
52
+
53
+ ## Tool Guidance
54
+ - Search documents before answering unless compressed context already contains enough evidence.
55
+ - Use 'search_child_chunks' for missing or uncovered parts of the question.
56
+ - If searched or retrieved context is not useful, use the tools again with a different, simpler query or a more relevant parent chunk.
57
+ - Continue tool use until the available evidence is enough, tools stop adding useful information, or the operation limit is reached.
58
+ - Do not repeat search queries or parent IDs listed in compressed context.
59
+ - Do not retrieve the same parent ID twice.
60
+
61
+ ## Response Framework
62
+ 1. Check compressed context for already-known evidence and already-used searches or parents.
63
+ 2. Search for missing evidence.
64
+ 3. Retrieve parent chunks only when child excerpts are relevant but too fragmented.
65
+ 4. Answer using the exact terms and scope in the retrieved evidence.
66
+ 5. If evidence is incomplete, state the specific gap.
67
+
68
+ ## Output
69
+ - Start directly with the substantive answer. Do not start with generic headings such as "Answer", "Final answer", or "Response".
70
+ - Provide the direct answer plus the key supporting details from retrieved evidence; avoid one-sentence fragments unless only one fact is available.
71
+ - Do not mention internal tool calls or reasoning.
72
+ - When sources exist, end with a Sources section in exactly this format:
73
+ Sources:
74
+ - filename.ext
75
+ - Put each source filename on its own bullet line. Never write sources inline, such as "Sources: filename.pdf".
76
+ - Do not invent or infer source filenames.
77
+ - Strip descriptions after file names, including text in parentheses.
78
+ """
79
+
80
+ def get_fallback_response_prompt() -> str:
81
+ return """## Role
82
+ You are a constrained evidence synthesizer for a retrieval-augmented assistant after the research loop reached its limit.
83
+
84
+ ## Available Context
85
+ - Compressed Research Context from earlier retrieval steps
86
+ - Retrieved Data from current tool outputs
87
+
88
+ ## Instructions
89
+ - Use only explicit facts from the provided context.
90
+ - Start directly with the substantive answer. Do not start with generic headings such as "Answer", "Final answer", or "Response".
91
+ - Prefer current Retrieved Data over compressed context if they conflict.
92
+ - If the answer is incomplete, mention only the missing parts that matter to the user query.
93
+ - Do not describe the retrieval process, limits, or internal reasoning.
94
+ - Be concise: answer in 1-3 short paragraphs or up to 5 bullets unless the user asks for detail.
95
+ - Provide the direct answer plus the key supporting details from retrieved evidence; avoid one-sentence fragments unless only one fact is available.
96
+ - End with a Sources section only when actual source file names are explicitly present in the context.
97
+ - Use exactly this format:
98
+ Sources:
99
+ - filename.ext
100
+ - Put each source filename on its own bullet line. Never write sources inline, such as "Sources: filename.pdf".
101
+ - Include only bare file names with extensions such as .pdf, .docx, .txt, or .md.
102
+ - Do not invent or infer source filenames.
103
+ """
104
+
105
+ def get_context_compression_prompt() -> str:
106
+ return """## Role
107
+ You are a research context compressor for an agentic RAG system.
108
+
109
+ ## Instructions
110
+ - Keep only facts relevant to answering the user question.
111
+ - Preserve exact names, figures, versions, technical terms, configuration details, and source file names.
112
+ - Remove duplicates, tool chatter, search query wording, parent IDs, chunk IDs, and other internal identifiers.
113
+ - Organize findings by source file. Each source section heading must be the real filename found in retrieved data.
114
+ - Add a Gaps section only for missing information relevant to the question.
115
+ - Target 400-600 words. If there is too much content, keep the most answer-critical facts.
116
+
117
+ ## Output
118
+ Return only Markdown in this structure:
119
+ # Research Context Summary
120
+
121
+ ## Focus
122
+ [Brief technical restatement of the question]
123
+
124
+ ## Structured Findings
125
+ For each source file, add a level-3 heading with its real filename and bullet the directly relevant facts below it.
126
+
127
+ ## Gaps
128
+ - Missing or incomplete aspects
129
+ """
130
+
131
+ def get_aggregation_prompt() -> str:
132
+ return """## Role
133
+ You are a final-answer synthesizer for a retrieval-augmented assistant.
134
+
135
+ ## Instructions
136
+ - Use only information present in the retrieved answers.
137
+ - Start directly with the substantive answer. Do not start with generic headings such as "Answer", "Final answer", or "Response".
138
+ - Preserve important names, numbers, versions, examples, and definitions.
139
+ - Do not expand acronyms or interpret terms unless the sources do it.
140
+ - If answers conflict, mention the conflict plainly.
141
+ - Be concise: answer in 1-3 short paragraphs or up to 5 bullets unless the user asks for detail.
142
+ - Provide the direct answer plus the key supporting details from retrieved evidence; avoid one-sentence fragments unless only one fact is available.
143
+ - End with a Sources section only when actual source file names are explicitly present in the retrieved answers.
144
+ - Use exactly this format:
145
+ Sources:
146
+ - filename.ext
147
+ - Put each source filename on its own bullet line. Never write sources inline, such as "Sources: filename.pdf".
148
+ - Include only bare file names with extensions such as .pdf, .docx, .txt, or .md.
149
+ - Do not invent or infer source filenames.
150
+ - If no useful information is available, say: "I couldn't find any information to answer your question in the available sources."
151
+ """
project/rag_agent/schemas.py ADDED
@@ -0,0 +1,13 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from typing import List
2
+ from pydantic import BaseModel, Field
3
+
4
+ class QueryAnalysis(BaseModel):
5
+ is_clear: bool = Field(
6
+ description="Indicates if the user's question is clear and answerable."
7
+ )
8
+ questions: List[str] = Field(
9
+ description="List of rewritten, self-contained questions."
10
+ )
11
+ clarification_needed: str = Field(
12
+ description="Explanation if the question is unclear."
13
+ )
project/rag_agent/tools.py ADDED
@@ -0,0 +1,88 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from langchain_core.tools import tool
2
+ import config
3
+ from db.parent_store_manager import ParentStoreManager
4
+ from core.execution_logger import log_error, log_tool_end, log_tool_start
5
+
6
+ class ToolFactory:
7
+
8
+ def __init__(self, collection):
9
+ self.collection = collection
10
+ self.parent_store_manager = ParentStoreManager()
11
+
12
+ def _search_child_chunks(self, query: str, limit: int = config.DEFAULT_RETRIEVAL_K) -> str:
13
+ """Search document excerpts for evidence related to the user question.
14
+
15
+ Use this as the first retrieval step. Results include parent IDs, file
16
+ names, and short child-chunk excerpts. If excerpts are relevant but too
17
+ fragmented to answer confidently, call retrieve_parent_chunks with the
18
+ returned parent_id.
19
+
20
+ Args:
21
+ query: Focused search query with concrete keywords from the question.
22
+ limit: Maximum number of child chunks to return.
23
+ """
24
+ log_tool_start("search_child_chunks", {"query": query, "limit": limit})
25
+ try:
26
+ results = self.collection.similarity_search(
27
+ query,
28
+ k=limit,
29
+ score_threshold=config.RETRIEVAL_SCORE_THRESHOLD,
30
+ )
31
+ if not results:
32
+ output = "NO_RELEVANT_CHUNKS"
33
+ log_tool_end("search_child_chunks", output)
34
+ return output
35
+
36
+ output = config.CHILD_CHUNK_SEPARATOR.join([
37
+ f"Parent ID: {doc.metadata.get('parent_id', '')}\n"
38
+ f"File Name: {doc.metadata.get('source', '')}\n"
39
+ f"Content: {doc.page_content.strip()}"
40
+ for doc in results
41
+ ])
42
+ log_tool_end("search_child_chunks", output)
43
+ return output
44
+
45
+ except Exception as e:
46
+ log_error("search_child_chunks", e)
47
+ output = f"RETRIEVAL_ERROR: {str(e)}"
48
+ log_tool_end("search_child_chunks", output)
49
+ return output
50
+
51
+ def _retrieve_parent_chunks(self, parent_id: str) -> str:
52
+ """Retrieve the full parent chunk for a relevant child search result.
53
+
54
+ Use this only after search_child_chunks returns a relevant parent_id and
55
+ the child excerpt needs more surrounding context. Do not call this for
56
+ parent IDs already available in compressed context.
57
+
58
+ Args:
59
+ parent_id: Parent chunk ID returned by search_child_chunks.
60
+ """
61
+ log_tool_start("retrieve_parent_chunks", {"parent_id": parent_id})
62
+ try:
63
+ parent = self.parent_store_manager.load_content(parent_id)
64
+ if not parent:
65
+ output = "NO_PARENT_DOCUMENT"
66
+ log_tool_end("retrieve_parent_chunks", output)
67
+ return output
68
+
69
+ output = (
70
+ f"Parent ID: {parent.get('parent_id', 'n/a')}\n"
71
+ f"File Name: {parent.get('metadata', {}).get('source', 'unknown')}\n"
72
+ f"Content: {parent.get('content', '').strip()}"
73
+ )
74
+ log_tool_end("retrieve_parent_chunks", output)
75
+ return output
76
+
77
+ except Exception as e:
78
+ log_error("retrieve_parent_chunks", e)
79
+ output = f"PARENT_RETRIEVAL_ERROR: {str(e)}"
80
+ log_tool_end("retrieve_parent_chunks", output)
81
+ return output
82
+
83
+ def create_tools(self) -> list:
84
+ """Create and return the list of tools."""
85
+ search_tool = tool("search_child_chunks")(self._search_child_chunks)
86
+ retrieve_tool = tool("retrieve_parent_chunks")(self._retrieve_parent_chunks)
87
+
88
+ return [search_tool, retrieve_tool]
project/requirements.txt ADDED
@@ -0,0 +1,12 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ fastembed==0.8.1
2
+ gradio==6.29.0
3
+ langchain-groq==1.1.3
4
+ langchain-huggingface==1.2.2
5
+ langchain-qdrant==1.1.0
6
+ langchain-text-splitters==1.1.2
7
+ langfuse==4.16.0
8
+ langgraph==1.2.12
9
+ pymupdf4llm==1.28.2
10
+ python-dotenv==1.2.4
11
+ sentence-transformers==6.1.0
12
+ tiktoken==0.14.0
project/ui/__init__.py ADDED
File without changes
project/ui/css.py ADDED
@@ -0,0 +1,294 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ custom_css = """
2
+ /* ============================================
3
+ MAIN CONTAINER
4
+ ============================================ */
5
+ .gradio-container {
6
+ max-width: 1100px !important;
7
+ width: 100% !important;
8
+ margin: 0 auto !important;
9
+ font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', 'Roboto', sans-serif;
10
+ background: #0b1220 !important;
11
+ padding: 20px !important;
12
+ }
13
+
14
+ #app-header {
15
+ background: linear-gradient(120deg, #132a32, #111c30);
16
+ border: 1px solid #294454;
17
+ border-radius: 16px;
18
+ padding: 24px;
19
+ margin-bottom: 18px;
20
+ }
21
+
22
+ #app-header p, #privacy-note p {
23
+ color: #cbd5e1 !important;
24
+ }
25
+
26
+ #privacy-note {
27
+ padding: 12px 4px;
28
+ font-size: 13px;
29
+ }
30
+
31
+ /* ============================================
32
+ TABS
33
+ ============================================ */
34
+ button[role="tab"] {
35
+ color: #a3a3a3 !important;
36
+ border-bottom: 2px solid transparent !important;
37
+ border-radius: 0 !important;
38
+ transition: all 0.2s ease !important;
39
+ background: transparent !important;
40
+ }
41
+
42
+ button[role="tab"]:hover {
43
+ color: #e5e5e5 !important;
44
+ }
45
+
46
+ button[role="tab"][aria-selected="true"] {
47
+ color: #ffffff !important;
48
+ border-bottom: 2px solid #2dd4bf !important;
49
+ border-radius: 0 !important;
50
+ background: transparent !important;
51
+ }
52
+
53
+ .tabs {
54
+ border-bottom: none !important;
55
+ border-radius: 0 !important;
56
+ }
57
+
58
+ .tab-nav {
59
+ border-bottom: 1px solid #3f3f3f !important;
60
+ border-radius: 0 !important;
61
+ }
62
+
63
+ button[role="tab"]::before,
64
+ button[role="tab"]::after,
65
+ .tabs::before,
66
+ .tabs::after,
67
+ .tab-nav::before,
68
+ .tab-nav::after {
69
+ display: none !important;
70
+ content: none !important;
71
+ border-radius: 0 !important;
72
+ }
73
+
74
+ #doc-management-tab {
75
+ max-width: 680px !important;
76
+ margin: 0 auto !important;
77
+ }
78
+
79
+ /* ============================================
80
+ BUTTONS
81
+ ============================================ */
82
+ button {
83
+ border-radius: 8px !important;
84
+ border: none !important;
85
+ font-weight: 500 !important;
86
+ transition: all 0.2s ease !important;
87
+ box-shadow: none !important;
88
+ }
89
+
90
+ .primary {
91
+ background: #0f766e !important;
92
+ color: white !important;
93
+ }
94
+
95
+ .primary:hover {
96
+ background: #115e59 !important;
97
+ transform: translateY(-1px) !important;
98
+ }
99
+
100
+ .stop {
101
+ background: #ef4444 !important;
102
+ color: white !important;
103
+ }
104
+
105
+ .stop:hover {
106
+ background: #dc2626 !important;
107
+ transform: translateY(-1px) !important;
108
+ }
109
+
110
+ /* ============================================
111
+ CHAT INPUT BOX
112
+ ============================================ */
113
+ textarea[placeholder="Type a message..."],
114
+ textarea[data-testid*="textbox"]:not(#file-list-box textarea) {
115
+ background: transparent !important;
116
+ border: none !important;
117
+ box-shadow: none !important;
118
+ }
119
+
120
+ textarea[placeholder="Type a message..."]:focus {
121
+ background: transparent !important;
122
+ border: none !important;
123
+ box-shadow: none !important;
124
+ }
125
+
126
+ .gr-text-input:has(textarea[placeholder="Type a message..."]),
127
+ [class*="chatbot"] + * [data-testid="textbox"],
128
+ form:has(textarea[placeholder="Type a message..."]) > div {
129
+ background: transparent !important;
130
+ border: none !important;
131
+ gap: 12px !important;
132
+ }
133
+
134
+ form:has(textarea[placeholder="Type a message..."]) button,
135
+ [class*="chatbot"] ~ * button[type="submit"] {
136
+ background: transparent !important;
137
+ border: none !important;
138
+ padding: 8px !important;
139
+ }
140
+
141
+ form:has(textarea[placeholder="Type a message..."]) button:hover {
142
+ background: rgba(59, 130, 246, 0.1) !important;
143
+ }
144
+
145
+ form:has(textarea[placeholder="Type a message..."]) {
146
+ gap: 12px !important;
147
+ display: flex !important;
148
+ }
149
+
150
+ /* ============================================
151
+ FILE UPLOAD
152
+ ============================================ */
153
+ .file-preview,
154
+ [data-testid="file-upload"] {
155
+ background: #1a1a1a !important;
156
+ border: 1px solid #3f3f3f !important;
157
+ border-radius: 5px !important;
158
+ color: #ffffff !important;
159
+ min-height: 200px !important;
160
+ }
161
+
162
+ .file-preview:hover,
163
+ [data-testid="file-upload"]:hover {
164
+ border-color: #2dd4bf !important;
165
+ background: #1f1f1f !important;
166
+ }
167
+
168
+ .file-preview *,
169
+ [data-testid="file-upload"] * {
170
+ color: #ffffff !important;
171
+ }
172
+
173
+ .file-preview .label,
174
+ [data-testid="file-upload"] .label {
175
+ display: none !important;
176
+ }
177
+
178
+ /* ============================================
179
+ INPUTS & TEXTAREAS
180
+ ============================================ */
181
+ input,
182
+ textarea {
183
+ background: #1a1a1a !important;
184
+ border: 1px solid #3f3f3f !important;
185
+ border-radius: 10px !important;
186
+ color: #e5e5e5 !important;
187
+ transition: border-color 0.2s ease !important;
188
+ }
189
+
190
+ input:focus,
191
+ textarea:focus {
192
+ border-color: #2dd4bf !important;
193
+ outline: none !important;
194
+ box-shadow: 0 0 0 3px rgba(45, 212, 191, 0.2) !important;
195
+ }
196
+
197
+ textarea[readonly] {
198
+ background: #1a1a1a !important;
199
+ color: #a3a3a3 !important;
200
+ }
201
+
202
+ /* ============================================
203
+ FILE LIST BOX
204
+ ============================================ */
205
+ #file-list-box {
206
+ background: #1a1a1a !important;
207
+ border: 1px solid #3f3f3f !important;
208
+ border-radius: 5px !important;
209
+ padding: 10px !important;
210
+ }
211
+
212
+ #file-list-box textarea {
213
+ background: transparent !important;
214
+ border: none !important;
215
+ color: #e5e5e5 !important;
216
+ padding: 0 !important;
217
+ }
218
+
219
+ /* ============================================
220
+ CHATBOT CONTAINER
221
+ ============================================ */
222
+ .chatbot {
223
+ border-radius: 14px !important;
224
+ background: #1a1a1a !important;
225
+ border: none !important;
226
+ }
227
+
228
+ .chatbot .message-wrap,
229
+ .chatbot > div {
230
+ gap: 8px !important;
231
+ padding: 12px !important;
232
+ }
233
+
234
+ /* ============================================
235
+ MESSAGE BUBBLES
236
+ ============================================ */
237
+ .message {
238
+ border-radius: 10px !important;
239
+ }
240
+
241
+ .message.user {
242
+ background: #0f766e !important;
243
+ color: white !important;
244
+ }
245
+
246
+ .message.bot {
247
+ background: #1f1f1f !important;
248
+ color: #e5e5e5 !important;
249
+ border: 1px solid #3f3f3f !important;
250
+ width: fit-content !important;
251
+ max-width: 90% !important;
252
+ }
253
+
254
+ .message-row img {
255
+ margin: 0px !important;
256
+ }
257
+
258
+ .avatar-container img {
259
+ padding: 0px !important;
260
+ }
261
+
262
+ /* ============================================
263
+ PROGRESS BAR
264
+ ============================================ */
265
+ .progress-bar-wrap {
266
+ border-radius: 10px !important;
267
+ overflow: hidden !important;
268
+ background: #1a1a1a !important;
269
+ }
270
+
271
+ .progress-bar {
272
+ border-radius: 10px !important;
273
+ background: #0f766e !important;
274
+ }
275
+
276
+ /* ============================================
277
+ TYPOGRAPHY
278
+ ============================================ */
279
+ h1, h2, h3, h4, h5, h6 {
280
+ color: #e5e5e5 !important;
281
+ }
282
+
283
+ /* ============================================
284
+ GLOBAL OVERRIDES
285
+ ============================================ */
286
+ button:focus-visible {
287
+ outline: 2px solid #2dd4bf !important;
288
+ outline-offset: 3px;
289
+ }
290
+
291
+ footer {
292
+ visibility: hidden;
293
+ }
294
+ """
project/ui/gradio_app.py ADDED
@@ -0,0 +1,122 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import gradio as gr
2
+ from core.chat_interface import ChatInterface
3
+ from core.document_manager import DocumentManager
4
+ from core.rag_system import RAGSystem
5
+ import uuid
6
+ from threading import Lock
7
+
8
+ def create_gradio_ui():
9
+ rag_system = RAGSystem()
10
+ rag_system.initialize()
11
+
12
+ doc_manager = DocumentManager(rag_system)
13
+ chat_interface = ChatInterface(rag_system)
14
+ rag_lock = Lock()
15
+
16
+ def format_file_list():
17
+ files = doc_manager.get_markdown_files()
18
+ if not files:
19
+ return "📭 No documents available in the knowledge base"
20
+ return "\n".join([f"{f}" for f in files])
21
+
22
+ def upload_handler(files, progress=gr.Progress()):
23
+ if not files:
24
+ return None, format_file_list()
25
+
26
+ with rag_lock:
27
+ added, skipped = doc_manager.add_documents(
28
+ files,
29
+ progress_callback=lambda p, desc: progress(p, desc=desc)
30
+ )
31
+
32
+ gr.Info(f"✅ Added: {added} | Skipped: {skipped}")
33
+ return None, format_file_list()
34
+
35
+ def clear_handler():
36
+ try:
37
+ with rag_lock:
38
+ doc_manager.clear_all()
39
+ gr.Info("🗑️ Removed all documents")
40
+ except Exception as exc:
41
+ gr.Error(f"Unable to clear documents: {exc}")
42
+ return format_file_list()
43
+
44
+ def chat_handler(msg, hist, thread_id):
45
+ with rag_lock:
46
+ yield from chat_interface.chat(msg, hist, thread_id)
47
+
48
+ def clear_chat_handler(thread_id):
49
+ with rag_lock:
50
+ return chat_interface.clear_session(thread_id)
51
+
52
+ with gr.Blocks(title="Document Chat") as demo:
53
+ session_thread = gr.State(value=lambda: str(uuid.uuid4()))
54
+ gr.Markdown(
55
+ "# Document Chat\n"
56
+ "Turn your documents into a conversation. Upload a PDF or Markdown file, "
57
+ "then ask a question.\n\n"
58
+ "**Groq-powered answers** &nbsp; | &nbsp; **Hybrid document search**",
59
+ elem_id="app-header",
60
+ )
61
+
62
+ with gr.Tab("Documents", elem_id="doc-management-tab"):
63
+ gr.Markdown("## Your document library")
64
+ gr.Markdown("Upload PDF or Markdown files to get started. Duplicate filenames are skipped.")
65
+
66
+ files_input = gr.File(
67
+ label="Drop PDF or Markdown files here",
68
+ file_count="multiple",
69
+ file_types=[".pdf", ".md"],
70
+ type="filepath",
71
+ height=200,
72
+ show_label=False
73
+ )
74
+
75
+ add_btn = gr.Button("Add Documents", variant="primary", size="md")
76
+
77
+ gr.Markdown("### Ready to search")
78
+ file_list = gr.Textbox(
79
+ value=format_file_list(),
80
+ interactive=False,
81
+ lines = 7,
82
+ max_lines=10,
83
+ elem_id="file-list-box",
84
+ show_label=False
85
+ )
86
+
87
+ with gr.Row():
88
+ refresh_btn = gr.Button("Refresh", size="md")
89
+ clear_btn = gr.Button("Clear All", variant="stop", size="md")
90
+
91
+ add_btn.click(upload_handler, [files_input], [files_input, file_list], show_progress="corner", concurrency_id="rag")
92
+ refresh_btn.click(format_file_list, None, file_list)
93
+ clear_btn.click(clear_handler, None, file_list, concurrency_id="rag")
94
+
95
+ with gr.Tab("Chat"):
96
+ chatbot = gr.Chatbot(
97
+ height=560,
98
+ placeholder="<strong>What would you like to know?</strong><br>Upload documents first, then ask for a summary, explanation, or comparison.",
99
+ show_label=False,
100
+ layout="bubble",
101
+ elem_id="document-chat",
102
+ )
103
+ chatbot.clear(clear_chat_handler, [session_thread], [session_thread], concurrency_id="rag")
104
+
105
+ gr.ChatInterface(
106
+ fn=chat_handler,
107
+ chatbot=chatbot,
108
+ textbox=gr.Textbox(
109
+ placeholder="Ask a question about your documents...",
110
+ show_label=False,
111
+ elem_id="question-input",
112
+ ),
113
+ additional_inputs=[session_thread],
114
+ concurrency_limit=1,
115
+ )
116
+ gr.Markdown(
117
+ "Documents are shared with everyone who can access this app. "
118
+ "Do not upload private or sensitive files to a public Space.",
119
+ elem_id="privacy-note",
120
+ )
121
+
122
+ return demo.queue(default_concurrency_limit=1)
project/utils.py ADDED
@@ -0,0 +1,62 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import os
2
+ import shutil
3
+ import config
4
+ import pymupdf.layout
5
+ import pymupdf4llm
6
+ from pathlib import Path
7
+ import glob
8
+ import tiktoken
9
+ from functools import lru_cache
10
+
11
+
12
+ def clear_directory_contents(directory: Path) -> None:
13
+ """Delete everything under directory but not the directory itself (safe for Docker volume / bind mount roots)."""
14
+ directory = Path(directory)
15
+ if not directory.is_dir():
16
+ return
17
+ for child in directory.iterdir():
18
+ if child.is_dir():
19
+ shutil.rmtree(child)
20
+ else:
21
+ child.unlink()
22
+
23
+
24
+ os.environ["TOKENIZERS_PARALLELISM"] = "false"
25
+
26
+ def pdf_to_markdown(pdf_path, output_dir):
27
+ doc = pymupdf.open(pdf_path)
28
+ md = pymupdf4llm.to_markdown(doc, header=False, footer=False, page_separators=True, ignore_images=True, write_images=False, image_path=None)
29
+ md_cleaned = md.encode('utf-8', errors='surrogatepass').decode('utf-8', errors='ignore')
30
+ output_path = Path(output_dir) / Path(doc.name).stem
31
+ Path(output_path).with_suffix(".md").write_bytes(md_cleaned.encode('utf-8'))
32
+
33
+ def pdfs_to_markdowns(path_pattern, overwrite: bool = False):
34
+ output_dir = Path(config.MARKDOWN_DIR)
35
+ output_dir.mkdir(parents=True, exist_ok=True)
36
+
37
+ for pdf_path in map(Path, glob.glob(path_pattern)):
38
+ md_path = (output_dir / pdf_path.stem).with_suffix(".md")
39
+ if overwrite or not md_path.exists():
40
+ pdf_to_markdown(pdf_path, output_dir)
41
+
42
+ @lru_cache(maxsize=1)
43
+ def _get_token_encoding():
44
+ try:
45
+ return tiktoken.encoding_for_model("gpt-4")
46
+ except Exception:
47
+ try:
48
+ return tiktoken.get_encoding("cl100k_base")
49
+ except Exception:
50
+ return None
51
+
52
+
53
+ def estimate_context_tokens(messages: list) -> int:
54
+ contents = [
55
+ str(msg.content)
56
+ for msg in messages
57
+ if hasattr(msg, "content") and msg.content
58
+ ]
59
+ encoding = _get_token_encoding()
60
+ if encoding is None:
61
+ return sum(max(1, len(content) // 4) for content in contents)
62
+ return sum(len(encoding.encode(content)) for content in contents)
requirements.txt ADDED
@@ -0,0 +1 @@
 
 
1
+ -r project/requirements.txt