# πŸ” Deep Dive: ChromaDB Vector Store Architecture ## 🧠 Why Vector Stores Matter in RAG ``` β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ THE RETRIEVAL PROBLEM β”‚ β”‚ β”‚ β”‚ User Query: "What is the refund policy?" β”‚ β”‚ ↓ β”‚ β”‚ Need to find relevant documents from 10,000+ chunks β”‚ β”‚ ↓ β”‚ β”‚ Can't do keyword search (user might say "return" vs "refund") β”‚ β”‚ ↓ β”‚ β”‚ Solution: SEMANTIC SEARCH using vector embeddings β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ ``` **Key Insight:** Vector stores enable **semantic similarity** β€” finding documents by meaning, not just keywords. --- ## πŸ“ How Vector Search Works ``` EMBEDDING SPACE (384 dimensions) "refund policy" ──────────────► [0.23, -0.45, 0.12, ...] ↓ Find nearest neighbors ↓ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ Stored Vectors: β”‚ β”‚ β”‚ β”‚ "Returns are processed in 7 days" β†’ [0.21, -0.42, ...] β”‚ ← CLOSE! β”‚ "Contact support for help" β†’ [0.89, 0.12, ...] β”‚ ← FAR β”‚ "Money back guarantee applies" β†’ [0.24, -0.41, ...] β”‚ ← CLOSE! β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ ``` **ChromaDB** is the database that stores these vectors and performs fast similarity search. --- ## πŸ—οΈ VectorStore Class Architecture ``` β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ VectorStore β”‚ β”‚ β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚ β”‚ embedding_fn β”‚ β”‚ ChromaDB β”‚ β”‚ β”‚ β”‚ (HuggingFace) │───►│ _store β”‚ β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”‚ β”‚ β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚ β–Ό β–Ό β–Ό β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚ β”‚add_documentsβ”‚ β”‚similarity_ β”‚ β”‚as_ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚search β”‚ β”‚retriever β”‚ β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β–Ό β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ Persistent Storage β”‚ β”‚ (chroma_db/ folder) β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ ``` --- ## 🎯 Constructor (`__init__`) ```python def __init__( self, embedding_function: Embeddings, # How to convert text β†’ vectors collection_name: str = "documents", # Namespace for documents persist_directory: Optional[str] = None, # Where to save ): ``` ### Why These Parameters? | Parameter | Purpose | |-----------|---------| | `embedding_function` | **Dependency Injection** β€” allows swapping embedding models | | `collection_name` | **Multi-tenancy** β€” different projects in same DB | | `persist_directory` | **Durability** β€” survives application restarts | ### How ChromaDB Initializes: ```python self._store = Chroma( collection_name=self.collection_name, embedding_function=self.embedding_function, # Used automatically persist_directory=str(self.persist_directory), ) ``` ChromaDB automatically: 1. Creates the collection if it doesn't exist 2. Loads existing data if it does exist 3. Uses the embedding function for all operations --- ## πŸ“₯ The `add_documents` Method ```python def add_documents( self, documents: List[Document], batch_size: int = 100 # Process 100 at a time ) -> List[str]: ``` ### Why Batching? ``` ❌ WITHOUT BATCHING: β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ 10,000 documents β†’ All loaded in memory β†’ OOM ERROR! β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ βœ… WITH BATCHING: β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ 10,000 documents β”‚ β”‚ ↓ β”‚ β”‚ Batch 1: docs[0:100] β†’ Embed β†’ Store β†’ Free memory β”‚ β”‚ Batch 2: docs[100:200] β†’ Embed β†’ Store β†’ Free memory β”‚ β”‚ ... β”‚ β”‚ Batch 100: docs[9900:10000] β†’ Embed β†’ Store β†’ Done! β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ ``` ### What Happens Inside: ```python for i in range(0, len(documents), batch_size): batch = documents[i:i + batch_size] # ChromaDB does this internally: # 1. For each doc: vector = embedding_function.embed(doc.page_content) # 2. Store (id, vector, metadata, page_content) ids = self._store.add_documents(batch) ``` --- ## πŸ” The `similarity_search` Method ```python def similarity_search( self, query: str, k: int = None, # Number of results (default: 5 from settings) filter: Optional[Dict[str, Any]] = None # Metadata filtering ) -> List[Document]: ``` ### How It Works: ``` Step 1: Query β†’ Embedding "What is the refund policy?" β†’ [0.23, -0.45, 0.12, ...] Step 2: Find K Nearest Neighbors ChromaDB uses approximate nearest neighbor (ANN) algorithms Step 3: Return Documents β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ Document 1: "Refunds are processed within 7 days..." β”‚ β”‚ Document 2: "Our return policy guarantees..." β”‚ β”‚ Document 3: "Money back if not satisfied..." β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ ``` ### Metadata Filtering: ```python # Find only from specific source results = store.similarity_search( "refund policy", k=5, filter={"file_name": "policies.pdf"} # ← Only from this file! ) ``` --- ## πŸ“Š The `similarity_search_with_score` Method ```python def similarity_search_with_score( self, query: str, k: int = None, filter: Optional[Dict[str, Any]] = None ) -> List[tuple]: # Returns (Document, score) pairs ``` ### Why Scores Matter: ```python results = store.similarity_search_with_score("refund policy", k=3) # Returns: [ (Document("Refunds within 7 days..."), 0.92), # ← High confidence (Document("Return shipping free..."), 0.78), # ← Medium confidence (Document("Contact us for help..."), 0.31), # ← Low confidence ⚠️ ] ``` **Use Cases:** 1. **Threshold filtering** β€” only use if score > 0.7 2. **Confidence display** β€” show users "95% relevant" 3. **Debugging** β€” understand why wrong docs were retrieved --- ## πŸ”— The `as_retriever` Method (LangChain Integration) ```python @property def retriever(self): """Default retriever with settings from config.""" return self._store.as_retriever( search_type="similarity", search_kwargs={"k": settings.top_k_results} ) ``` ### Why Retrievers? LangChain's **LCEL** uses retrievers in chains: ```python chain = ( {"context": store.retriever, "question": RunnablePassthrough()} | prompt | llm | output_parser ) answer = chain.invoke("What is the refund policy?") ``` --- ## πŸ“ˆ The `get_collection_stats` Method ```python def get_collection_stats(self) -> Dict[str, Any]: return { "collection_name": self.collection_name, "document_count": collection.count(), "persist_directory": str(self.persist_directory), } ``` ### Debugging in Production: ```python stats = store.get_collection_stats() # {"collection_name": "documents", "document_count": 1547, ...} ``` --- ## πŸ“‹ Summary | Component | Pattern | Purpose | |-----------|---------|---------| | `VectorStore` | Wrapper/Facade | Simplify ChromaDB API | | `embedding_function` | Dependency Injection | Swap embedding models | | `add_documents()` | Batch Iterator | Memory-efficient ingestion | | `similarity_search()` | Query Interface | Semantic document retrieval | | `as_retriever()` | Adapter | LangChain compatibility | | `create_vector_store()` | Factory | Simplified instantiation |