devank2000 commited on
Commit
717b14e
·
verified ·
1 Parent(s): f1ae1d7
README.md CHANGED
@@ -10,8 +10,9 @@ app_file: project/app.py
10
  ---
11
 
12
  **Deployable app:** The Python/Gradio app now uses Groq (`GROQ_API_KEY`) for chat
13
- and local Hugging Face embeddings for retrieval. Use `uv` to install the runtime
14
- dependencies in `project/requirements.txt`. The learning notebooks retain their
 
15
  Ollama examples. See [deployment instructions](project/README.md#hugging-face-spaces-recommended)
16
  for Hugging Face Spaces, Render, secrets, and persistent storage.
17
 
@@ -1402,7 +1403,7 @@ The chat interface will appear at the end.
1402
  # Run from the project root.
1403
  # Windows PowerShell: install only the deployable app
1404
  uv venv --python 3.13 .venv
1405
- uv pip install --python .venv\Scripts\python.exe --torch-backend cpu -r project\requirements.txt
1406
  Copy-Item project\.env.example project\.env
1407
  # Edit project\.env and set GROQ_API_KEY.
1408
  # For notebook/evaluation dependencies too, install requirements-notebooks.txt instead.
 
10
  ---
11
 
12
  **Deployable app:** The Python/Gradio app now uses Groq (`GROQ_API_KEY`) for chat
13
+ and Hugging Face embeddings for retrieval (ZeroGPU on Spaces, CPU locally).
14
+ Use `uv` to install the runtime
15
+ dependencies in the root `requirements.txt`. The learning notebooks retain their
16
  Ollama examples. See [deployment instructions](project/README.md#hugging-face-spaces-recommended)
17
  for Hugging Face Spaces, Render, secrets, and persistent storage.
18
 
 
1403
  # Run from the project root.
1404
  # Windows PowerShell: install only the deployable app
1405
  uv venv --python 3.13 .venv
1406
+ uv pip install --python .venv\Scripts\python.exe --torch-backend cpu -r requirements.txt
1407
  Copy-Item project\.env.example project\.env
1408
  # Edit project\.env and set GROQ_API_KEY.
1409
  # For notebook/evaluation dependencies too, install requirements-notebooks.txt instead.
project/.env.example CHANGED
@@ -3,7 +3,7 @@
3
  GROQ_API_KEY=
4
  GROQ_MODEL=llama-3.3-70b-versatile
5
 
6
- # MiniLM CPU embeddings are the default for HF Gradio Spaces.
7
  # Changing embeddings requires clearing and re-indexing existing documents.
8
  # DENSE_MODEL=Qwen/Qwen3-Embedding-0.6B
9
 
 
3
  GROQ_API_KEY=
4
  GROQ_MODEL=llama-3.3-70b-versatile
5
 
6
+ # MiniLM embeddings use ZeroGPU on HF and CPU locally.
7
  # Changing embeddings requires clearing and re-indexing existing documents.
8
  # DENSE_MODEL=Qwen/Qwen3-Embedding-0.6B
9
 
project/README.md CHANGED
@@ -1,6 +1,6 @@
1
  # Document Chat Documentation
2
 
3
- An **Agentic Retrieval-Augmented Generation (RAG)** system built with **LangGraph**, featuring **parent–child chunking**, **hybrid dense + sparse retrieval**, and Groq-powered chat with a Gradio frontend. Embeddings run locally; no Ollama server or GPU is required for the app. The learning notebooks retain their Ollama setup.
4
 
5
 
6
  ## Table of Contents
@@ -17,15 +17,17 @@ From the repository root, use `uv` (Windows PowerShell):
17
 
18
  ```powershell
19
  uv venv --python 3.13 .venv
20
- uv pip install --python .venv\Scripts\python.exe --torch-backend cpu -r project\requirements.txt
21
  Copy-Item project\.env.example project\.env
22
  ```
23
 
24
  Set `GROQ_API_KEY` in `project/.env`. Get a key from
25
  https://console.groq.com/keys. Never commit `.env` or expose the key in frontend
26
- code. The root `requirements.txt` includes only runtime dependencies (as required
27
- by HF Gradio). Notebook/evaluation tools are in `requirements-notebooks.txt`.
28
- Local installs and Docker can also use `project/requirements.txt` directly.
 
 
29
 
30
  ### Running the Application
31
 
@@ -96,7 +98,8 @@ PDF → Markdown Conversion → Parent/Child Chunking → Vector Indexing → Ag
96
 
97
  | File | Purpose |
98
  |------|---------|
99
- | `project/db/vector_db_manager.py` | Qdrant client wrapper with embedding initialization |
 
100
  | `project/db/parent_store_manager.py` | File-backed storage for parent chunks |
101
 
102
  ### RAG Agent (LangGraph)
@@ -379,7 +382,7 @@ ACTIVE_LLM_CONFIG = "google" # Switch to Gemini Pro
379
  **Step 1:** Update `project/config.py`
380
 
381
  ```python
382
- # Default: Faster, smaller CPU model
383
  DENSE_MODEL = "sentence-transformers/all-MiniLM-L6-v2"
384
 
385
  # Example: Alternative Sentence Transformers model
@@ -405,9 +408,9 @@ from langchain_huggingface import HuggingFaceEmbeddings
405
  from langchain_qdrant import FastEmbedSparse
406
  import config
407
 
408
- self.__dense_embeddings = HuggingFaceEmbeddings(
409
- model_name=config.DENSE_MODEL, model_kwargs={"device": "cpu"}
410
- )
411
  self.__sparse_embeddings = FastEmbedSparse(model_name=config.SPARSE_MODEL)
412
  ```
413
 
@@ -415,7 +418,7 @@ self.__sparse_embeddings = FastEmbedSparse(model_name=config.SPARSE_MODEL)
415
 
416
  | Model | Context Size | Vector Dimension | Speed | Quality | Use Case |
417
  |-------|--------------|------------------|-------|---------|----------|
418
- | all-MiniLM-L6-v2 | 256 tokens | 384 | Fast | Good | Default CPU model for the deployed app |
419
  | all-mpnet-base-v2 | 512 tokens | 768 | Medium | Excellent | High-accuracy semantic search |
420
  | bge-large-en-v1.5 | 512 tokens | 1024 | Slow | Best | Production-grade retrieval on GPU |
421
  | google/embeddinggemma-300m | 2048 tokens | 768 | Fast | Very Good | Lightweight, efficient multilingual retrieval |
@@ -663,10 +666,18 @@ Once running, open `http://localhost:7860`.
663
  Upload a small Markdown/PDF file under **Documents**, click **Add Documents**,
664
  and ask a question under **Chat**.
665
 
666
- The app runs MiniLM embeddings explicitly on **CPU** and calls Groq remotely.
667
- It does not use CUDA or request GPU time, so no `@spaces.GPU` decorator is
668
- needed. Choosing ZeroGPU does not grant a permanently attached GPU or remove
669
- platform limits. Verify first startup and indexing in the Space logs.
 
 
 
 
 
 
 
 
670
 
671
  The UI is branded **Document Chat**, with a teal-accented header, a compact
672
  chat panel, and the same **Documents** and **Chat** workflow.
 
1
  # Document Chat Documentation
2
 
3
+ An **Agentic Retrieval-Augmented Generation (RAG)** system built with **LangGraph**, featuring **parent–child chunking**, **hybrid dense + sparse retrieval**, and Groq-powered chat with a Gradio frontend. Embeddings use ZeroGPU on HF Spaces and CPU locally; no Ollama server is required. The learning notebooks retain their Ollama setup.
4
 
5
 
6
  ## Table of Contents
 
17
 
18
  ```powershell
19
  uv venv --python 3.13 .venv
20
+ uv pip install --python .venv\Scripts\python.exe --torch-backend cpu -r requirements.txt
21
  Copy-Item project\.env.example project\.env
22
  ```
23
 
24
  Set `GROQ_API_KEY` in `project/.env`. Get a key from
25
  https://console.groq.com/keys. Never commit `.env` or expose the key in frontend
26
+ code. The root `requirements.txt` is the standalone runtime dependency manifest.
27
+ HF Gradio mounts only this file during installation, so it must not include
28
+ other local requirement files. Notebook/evaluation tools are in
29
+ `requirements-notebooks.txt`. The optional `project/requirements.txt` redirects
30
+ local installs to the root manifest; Docker installs the root file directly.
31
 
32
  ### Running the Application
33
 
 
98
 
99
  | File | Purpose |
100
  |------|---------|
101
+ | `project/db/vector_db_manager.py` | Qdrant client wrapper with inference-free schema validation |
102
+ | `project/db/dense_embeddings.py` | MiniLM embeddings with ZeroGPU inference on Spaces and CPU inference locally |
103
  | `project/db/parent_store_manager.py` | File-backed storage for parent chunks |
104
 
105
  ### RAG Agent (LangGraph)
 
382
  **Step 1:** Update `project/config.py`
383
 
384
  ```python
385
+ # Default: Compact model (ZeroGPU on HF, CPU locally)
386
  DENSE_MODEL = "sentence-transformers/all-MiniLM-L6-v2"
387
 
388
  # Example: Alternative Sentence Transformers model
 
408
  from langchain_qdrant import FastEmbedSparse
409
  import config
410
 
411
+ from db.dense_embeddings import DenseEmbeddings
412
+
413
+ self.__dense_embeddings = DenseEmbeddings(model_name=config.DENSE_MODEL)
414
  self.__sparse_embeddings = FastEmbedSparse(model_name=config.SPARSE_MODEL)
415
  ```
416
 
 
418
 
419
  | Model | Context Size | Vector Dimension | Speed | Quality | Use Case |
420
  |-------|--------------|------------------|-------|---------|----------|
421
+ | all-MiniLM-L6-v2 | 256 tokens | 384 | Fast | Good | Default compact embedding model |
422
  | all-mpnet-base-v2 | 512 tokens | 768 | Medium | Excellent | High-accuracy semantic search |
423
  | bge-large-en-v1.5 | 512 tokens | 1024 | Slow | Best | Production-grade retrieval on GPU |
424
  | google/embeddinggemma-300m | 2048 tokens | 768 | Fast | Very Good | Lightweight, efficient multilingual retrieval |
 
666
  Upload a small Markdown/PDF file under **Documents**, click **Add Documents**,
667
  and ask a question under **Chat**.
668
 
669
+ The app loads MiniLM on **CUDA** at startup when HF sets `SPACES_ZERO_GPU=true`.
670
+ Dense embedding inference runs inside `@spaces.GPU(duration=30)`, both during
671
+ document indexing and query retrieval. Only texts and embedding results cross
672
+ the worker boundary; local Qdrant writes and Groq calls stay outside GPU tasks.
673
+ Startup obtains the vector dimension from model metadata and validates the
674
+ Qdrant schema without making an inference request.
675
+
676
+ ZeroGPU requires a registered GPU function during startup; a CPU-only app is
677
+ not sufficient on this hardware tier. Embedding requests consume the visitor's
678
+ GPU quota and may queue. Large indexing jobs can require multiple allocations.
679
+ CPU embedding inference is retained for local and non-ZeroGPU deployments.
680
+ Verify first startup and indexing in the Space logs.
681
 
682
  The UI is branded **Document Chat**, with a teal-accented header, a compact
683
  chat panel, and the same **Documents** and **Chat** workflow.
project/db/dense_embeddings.py ADDED
@@ -0,0 +1,34 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import os
2
+ import spaces
3
+ from langchain_core.embeddings import Embeddings
4
+ from sentence_transformers import SentenceTransformer
5
+
6
+
7
+ class DenseEmbeddings(Embeddings):
8
+ def __init__(self, model_name: str):
9
+ is_zero_gpu = os.environ.get("SPACES_ZERO_GPU", "").lower() in {"1", "true"}
10
+ model = SentenceTransformer(
11
+ model_name,
12
+ device="cuda" if is_zero_gpu else "cpu",
13
+ )
14
+ dimension = model.get_sentence_embedding_dimension()
15
+ if not isinstance(dimension, int) or dimension <= 0:
16
+ raise ValueError(f"Cannot determine embedding dimension for '{model_name}'.")
17
+ self.vector_size = dimension
18
+
19
+ # Capture the model in the worker, rather than sending CUDA weights as arguments.
20
+ @spaces.GPU(duration=30)
21
+ def encode(texts: list[str]) -> list[list[float]]:
22
+ return model.encode(
23
+ texts,
24
+ show_progress_bar=False,
25
+ convert_to_numpy=True,
26
+ ).tolist()
27
+
28
+ self._encode = encode
29
+
30
+ def embed_documents(self, texts: list[str]) -> list[list[float]]:
31
+ return self._encode([text.replace("\n", " ") for text in texts])
32
+
33
+ def embed_query(self, text: str) -> list[float]:
34
+ return self.embed_documents([text])[0]
project/db/vector_db_manager.py CHANGED
@@ -1,23 +1,20 @@
1
  import config
2
- from langchain_huggingface import HuggingFaceEmbeddings
3
  from langchain_qdrant import QdrantVectorStore, FastEmbedSparse, RetrievalMode
4
  from qdrant_client import QdrantClient
5
  from qdrant_client.http import models as qmodels
6
 
7
  class VectorDbManager:
8
  __client: QdrantClient
9
- __dense_embeddings: HuggingFaceEmbeddings
10
  __sparse_embeddings: FastEmbedSparse
11
  def __init__(self):
12
  self.__client = QdrantClient(path=config.QDRANT_DB_PATH)
13
- self.__dense_embeddings = HuggingFaceEmbeddings(
14
- model_name=config.DENSE_MODEL,
15
- model_kwargs={"device": "cpu"},
16
- )
17
  self.__sparse_embeddings = FastEmbedSparse(model_name=config.SPARSE_MODEL)
18
 
19
  def _dense_vector_size(self):
20
- return len(self.__dense_embeddings.embed_query("test"))
21
 
22
  @staticmethod
23
  def _collection_vector_size(collection_info):
@@ -41,6 +38,20 @@ class VectorDbManager:
41
  print(f"✓ Collection created: {collection_name}")
42
  else:
43
  collection_info = self.__client.get_collection(collection_name)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
44
  existing_size = self._collection_vector_size(collection_info)
45
  if existing_size and existing_size != expected_size:
46
  raise ValueError(
@@ -67,7 +78,9 @@ class VectorDbManager:
67
  embedding=self.__dense_embeddings,
68
  sparse_embedding=self.__sparse_embeddings,
69
  retrieval_mode=RetrievalMode.HYBRID,
70
- sparse_vector_name=config.SPARSE_VECTOR_NAME
 
 
71
  )
72
  except Exception as e:
73
  raise RuntimeError(f"Unable to initialize Qdrant collection '{collection_name}'.") from e
 
1
  import config
2
+ from db.dense_embeddings import DenseEmbeddings
3
  from langchain_qdrant import QdrantVectorStore, FastEmbedSparse, RetrievalMode
4
  from qdrant_client import QdrantClient
5
  from qdrant_client.http import models as qmodels
6
 
7
  class VectorDbManager:
8
  __client: QdrantClient
9
+ __dense_embeddings: DenseEmbeddings
10
  __sparse_embeddings: FastEmbedSparse
11
  def __init__(self):
12
  self.__client = QdrantClient(path=config.QDRANT_DB_PATH)
13
+ self.__dense_embeddings = DenseEmbeddings(model_name=config.DENSE_MODEL)
 
 
 
14
  self.__sparse_embeddings = FastEmbedSparse(model_name=config.SPARSE_MODEL)
15
 
16
  def _dense_vector_size(self):
17
+ return self.__dense_embeddings.vector_size
18
 
19
  @staticmethod
20
  def _collection_vector_size(collection_info):
 
38
  print(f"✓ Collection created: {collection_name}")
39
  else:
40
  collection_info = self.__client.get_collection(collection_name)
41
+ vectors_config = collection_info.config.params.vectors
42
+ dense_config = (
43
+ vectors_config.get("") if isinstance(vectors_config, dict)
44
+ else vectors_config
45
+ )
46
+ if dense_config is None:
47
+ raise ValueError(f"Qdrant collection '{collection_name}' must have an unnamed dense vector.")
48
+ if dense_config.distance != qmodels.Distance.COSINE:
49
+ raise ValueError(f"Qdrant collection '{collection_name}' must use cosine distance.")
50
+ sparse_config = collection_info.config.params.sparse_vectors or {}
51
+ if config.SPARSE_VECTOR_NAME not in sparse_config:
52
+ raise ValueError(
53
+ f"Qdrant collection '{collection_name}' is missing sparse vector '{config.SPARSE_VECTOR_NAME}'."
54
+ )
55
  existing_size = self._collection_vector_size(collection_info)
56
  if existing_size and existing_size != expected_size:
57
  raise ValueError(
 
78
  embedding=self.__dense_embeddings,
79
  sparse_embedding=self.__sparse_embeddings,
80
  retrieval_mode=RetrievalMode.HYBRID,
81
+ sparse_vector_name=config.SPARSE_VECTOR_NAME,
82
+ # Schema is checked above without requesting GPU inference at startup.
83
+ validate_collection_config=False,
84
  )
85
  except Exception as e:
86
  raise RuntimeError(f"Unable to initialize Qdrant collection '{collection_name}'.") from e
project/requirements.txt CHANGED
@@ -1,12 +1 @@
1
- fastembed==0.8.1
2
- gradio==6.29.0
3
- langchain-groq==1.1.3
4
- langchain-huggingface==1.2.2
5
- langchain-qdrant==1.1.0
6
- langchain-text-splitters==1.1.2
7
- langfuse==4.16.0
8
- langgraph==1.2.12
9
- pymupdf4llm==1.28.2
10
- python-dotenv==1.2.4
11
- sentence-transformers==6.1.0
12
- tiktoken==0.14.0
 
1
+ -r ../requirements.txt
 
 
 
 
 
 
 
 
 
 
 
requirements.txt CHANGED
@@ -9,4 +9,5 @@ langgraph==1.2.12
9
  pymupdf4llm==1.28.2
10
  python-dotenv==1.2.4
11
  sentence-transformers==6.1.0
 
12
  tiktoken==0.14.0
 
9
  pymupdf4llm==1.28.2
10
  python-dotenv==1.2.4
11
  sentence-transformers==6.1.0
12
+ spaces==0.51.3
13
  tiktoken==0.14.0