RAG-Documind-AI / docs /explanations /vectorstore.md
AIenthusSP's picture
feat: implement ChromaDB vector store
1ac24a4
|
Raw History Blame Contribute Delete
11 kB
# πŸ” Deep Dive: ChromaDB Vector Store Architecture
## 🧠 Why Vector Stores Matter in RAG
```
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚ THE RETRIEVAL PROBLEM β”‚
β”‚ β”‚
β”‚ User Query: "What is the refund policy?" β”‚
β”‚ ↓ β”‚
β”‚ Need to find relevant documents from 10,000+ chunks β”‚
β”‚ ↓ β”‚
β”‚ Can't do keyword search (user might say "return" vs "refund") β”‚
β”‚ ↓ β”‚
β”‚ Solution: SEMANTIC SEARCH using vector embeddings β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
```
**Key Insight:** Vector stores enable **semantic similarity** β€” finding documents by meaning, not just keywords.
---
## πŸ“ How Vector Search Works
```
EMBEDDING SPACE (384 dimensions)
"refund policy" ──────────────► [0.23, -0.45, 0.12, ...]
↓
Find nearest neighbors
↓
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚ Stored Vectors: β”‚
β”‚ β”‚
β”‚ "Returns are processed in 7 days" β†’ [0.21, -0.42, ...] β”‚ ← CLOSE!
β”‚ "Contact support for help" β†’ [0.89, 0.12, ...] β”‚ ← FAR
β”‚ "Money back guarantee applies" β†’ [0.24, -0.41, ...] β”‚ ← CLOSE!
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
```
**ChromaDB** is the database that stores these vectors and performs fast similarity search.
---
## πŸ—οΈ VectorStore Class Architecture
```
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚ VectorStore β”‚
β”‚ β”‚
β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚
β”‚ β”‚ embedding_fn β”‚ β”‚ ChromaDB β”‚ β”‚
β”‚ β”‚ (HuggingFace) │───►│ _store β”‚ β”‚
β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚
β”‚ β”‚ β”‚
β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚
β”‚ β–Ό β–Ό β–Ό β”‚
β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚
β”‚ β”‚add_documentsβ”‚ β”‚similarity_ β”‚ β”‚as_ β”‚ β”‚
β”‚ β”‚ β”‚ β”‚search β”‚ β”‚retriever β”‚ β”‚
β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
β”‚
β–Ό
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚ Persistent Storage β”‚
β”‚ (chroma_db/ folder) β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
```
---
## 🎯 Constructor (`__init__`)
```python
def __init__(
self,
embedding_function: Embeddings, # How to convert text β†’ vectors
collection_name: str = "documents", # Namespace for documents
persist_directory: Optional[str] = None, # Where to save
):
```
### Why These Parameters?
| Parameter | Purpose |
|-----------|---------|
| `embedding_function` | **Dependency Injection** β€” allows swapping embedding models |
| `collection_name` | **Multi-tenancy** β€” different projects in same DB |
| `persist_directory` | **Durability** β€” survives application restarts |
### How ChromaDB Initializes:
```python
self._store = Chroma(
collection_name=self.collection_name,
embedding_function=self.embedding_function, # Used automatically
persist_directory=str(self.persist_directory),
)
```
ChromaDB automatically:
1. Creates the collection if it doesn't exist
2. Loads existing data if it does exist
3. Uses the embedding function for all operations
---
## πŸ“₯ The `add_documents` Method
```python
def add_documents(
self,
documents: List[Document],
batch_size: int = 100 # Process 100 at a time
) -> List[str]:
```
### Why Batching?
```
❌ WITHOUT BATCHING:
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚ 10,000 documents β†’ All loaded in memory β†’ OOM ERROR! β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
βœ… WITH BATCHING:
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚ 10,000 documents β”‚
β”‚ ↓ β”‚
β”‚ Batch 1: docs[0:100] β†’ Embed β†’ Store β†’ Free memory β”‚
β”‚ Batch 2: docs[100:200] β†’ Embed β†’ Store β†’ Free memory β”‚
β”‚ ... β”‚
β”‚ Batch 100: docs[9900:10000] β†’ Embed β†’ Store β†’ Done! β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
```
### What Happens Inside:
```python
for i in range(0, len(documents), batch_size):
batch = documents[i:i + batch_size]
# ChromaDB does this internally:
# 1. For each doc: vector = embedding_function.embed(doc.page_content)
# 2. Store (id, vector, metadata, page_content)
ids = self._store.add_documents(batch)
```
---
## πŸ” The `similarity_search` Method
```python
def similarity_search(
self,
query: str,
k: int = None, # Number of results (default: 5 from settings)
filter: Optional[Dict[str, Any]] = None # Metadata filtering
) -> List[Document]:
```
### How It Works:
```
Step 1: Query β†’ Embedding
"What is the refund policy?" β†’ [0.23, -0.45, 0.12, ...]
Step 2: Find K Nearest Neighbors
ChromaDB uses approximate nearest neighbor (ANN) algorithms
Step 3: Return Documents
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚ Document 1: "Refunds are processed within 7 days..." β”‚
β”‚ Document 2: "Our return policy guarantees..." β”‚
β”‚ Document 3: "Money back if not satisfied..." β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
```
### Metadata Filtering:
```python
# Find only from specific source
results = store.similarity_search(
"refund policy",
k=5,
filter={"file_name": "policies.pdf"} # ← Only from this file!
)
```
---
## πŸ“Š The `similarity_search_with_score` Method
```python
def similarity_search_with_score(
self,
query: str,
k: int = None,
filter: Optional[Dict[str, Any]] = None
) -> List[tuple]: # Returns (Document, score) pairs
```
### Why Scores Matter:
```python
results = store.similarity_search_with_score("refund policy", k=3)
# Returns:
[
(Document("Refunds within 7 days..."), 0.92), # ← High confidence
(Document("Return shipping free..."), 0.78), # ← Medium confidence
(Document("Contact us for help..."), 0.31), # ← Low confidence ⚠️
]
```
**Use Cases:**
1. **Threshold filtering** β€” only use if score > 0.7
2. **Confidence display** β€” show users "95% relevant"
3. **Debugging** β€” understand why wrong docs were retrieved
---
## πŸ”— The `as_retriever` Method (LangChain Integration)
```python
@property
def retriever(self):
"""Default retriever with settings from config."""
return self._store.as_retriever(
search_type="similarity",
search_kwargs={"k": settings.top_k_results}
)
```
### Why Retrievers?
LangChain's **LCEL** uses retrievers in chains:
```python
chain = (
{"context": store.retriever, "question": RunnablePassthrough()}
| prompt
| llm
| output_parser
)
answer = chain.invoke("What is the refund policy?")
```
---
## πŸ“ˆ The `get_collection_stats` Method
```python
def get_collection_stats(self) -> Dict[str, Any]:
return {
"collection_name": self.collection_name,
"document_count": collection.count(),
"persist_directory": str(self.persist_directory),
}
```
### Debugging in Production:
```python
stats = store.get_collection_stats()
# {"collection_name": "documents", "document_count": 1547, ...}
```
---
## πŸ“‹ Summary
| Component | Pattern | Purpose |
|-----------|---------|---------|
| `VectorStore` | Wrapper/Facade | Simplify ChromaDB API |
| `embedding_function` | Dependency Injection | Swap embedding models |
| `add_documents()` | Batch Iterator | Memory-efficient ingestion |
| `similarity_search()` | Query Interface | Semantic document retrieval |
| `as_retriever()` | Adapter | LangChain compatibility |
| `create_vector_store()` | Factory | Simplified instantiation |