RAG-Documind-AI / docs /explanations /vectorstore.md
AIenthusSP's picture
feat: implement ChromaDB vector store
1ac24a4
|
Raw History Blame Contribute Delete
11 kB

πŸ” Deep Dive: ChromaDB Vector Store Architecture

🧠 Why Vector Stores Matter in RAG

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚                    THE RETRIEVAL PROBLEM                         β”‚
β”‚                                                                  β”‚
β”‚  User Query: "What is the refund policy?"                       β”‚
β”‚        ↓                                                         β”‚
β”‚  Need to find relevant documents from 10,000+ chunks            β”‚
β”‚        ↓                                                         β”‚
β”‚  Can't do keyword search (user might say "return" vs "refund")  β”‚
β”‚        ↓                                                         β”‚
β”‚  Solution: SEMANTIC SEARCH using vector embeddings              β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Key Insight: Vector stores enable semantic similarity β€” finding documents by meaning, not just keywords.


πŸ“ How Vector Search Works

                    EMBEDDING SPACE (384 dimensions)
                    
    "refund policy"  ──────────────►  [0.23, -0.45, 0.12, ...]
                                              ↓
                                        Find nearest neighbors
                                              ↓
    β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
    β”‚   Stored Vectors:                                        β”‚
    β”‚                                                          β”‚
    β”‚   "Returns are processed in 7 days" β†’ [0.21, -0.42, ...] β”‚ ← CLOSE!
    β”‚   "Contact support for help"        β†’ [0.89, 0.12, ...]  β”‚ ← FAR
    β”‚   "Money back guarantee applies"    β†’ [0.24, -0.41, ...] β”‚ ← CLOSE!
    β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

ChromaDB is the database that stores these vectors and performs fast similarity search.


πŸ—οΈ VectorStore Class Architecture

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚                        VectorStore                               β”‚
β”‚                                                                  β”‚
β”‚  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”    β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”                    β”‚
β”‚  β”‚ embedding_fn    β”‚    β”‚ ChromaDB         β”‚                    β”‚
β”‚  β”‚ (HuggingFace)   │───►│ _store           β”‚                    β”‚
β”‚  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜    β””β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜                    β”‚
β”‚                                  β”‚                               β”‚
β”‚         β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”     β”‚
β”‚         β–Ό                        β–Ό                        β–Ό     β”‚
β”‚  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”        β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”          β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”  β”‚
β”‚  β”‚add_documentsβ”‚        β”‚similarity_  β”‚          β”‚as_       β”‚  β”‚
β”‚  β”‚             β”‚        β”‚search       β”‚          β”‚retriever β”‚  β”‚
β”‚  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜        β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜          β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜  β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                                  β”‚
                                  β–Ό
                    β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
                    β”‚   Persistent Storage   β”‚
                    β”‚   (chroma_db/ folder)  β”‚
                    β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

🎯 Constructor (__init__)

def __init__(
    self,
    embedding_function: Embeddings,  # How to convert text β†’ vectors
    collection_name: str = "documents",  # Namespace for documents
    persist_directory: Optional[str] = None,  # Where to save
):

Why These Parameters?

Parameter Purpose
embedding_function Dependency Injection β€” allows swapping embedding models
collection_name Multi-tenancy β€” different projects in same DB
persist_directory Durability β€” survives application restarts

How ChromaDB Initializes:

self._store = Chroma(
    collection_name=self.collection_name,
    embedding_function=self.embedding_function,  # Used automatically
    persist_directory=str(self.persist_directory),
)

ChromaDB automatically:

  1. Creates the collection if it doesn't exist
  2. Loads existing data if it does exist
  3. Uses the embedding function for all operations

πŸ“₯ The add_documents Method

def add_documents(
    self,
    documents: List[Document],
    batch_size: int = 100  # Process 100 at a time
) -> List[str]:

Why Batching?

❌ WITHOUT BATCHING:
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚  10,000 documents β†’ All loaded in memory β†’ OOM ERROR!  β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

βœ… WITH BATCHING:
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚  10,000 documents                                        β”‚
β”‚     ↓                                                    β”‚
β”‚  Batch 1: docs[0:100]   β†’ Embed β†’ Store β†’ Free memory   β”‚
β”‚  Batch 2: docs[100:200] β†’ Embed β†’ Store β†’ Free memory   β”‚
β”‚  ...                                                     β”‚
β”‚  Batch 100: docs[9900:10000] β†’ Embed β†’ Store β†’ Done!    β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

What Happens Inside:

for i in range(0, len(documents), batch_size):
    batch = documents[i:i + batch_size]
    
    # ChromaDB does this internally:
    # 1. For each doc: vector = embedding_function.embed(doc.page_content)
    # 2. Store (id, vector, metadata, page_content)
    ids = self._store.add_documents(batch)

πŸ” The similarity_search Method

def similarity_search(
    self,
    query: str,
    k: int = None,  # Number of results (default: 5 from settings)
    filter: Optional[Dict[str, Any]] = None  # Metadata filtering
) -> List[Document]:

How It Works:

Step 1: Query β†’ Embedding
"What is the refund policy?" β†’ [0.23, -0.45, 0.12, ...]

Step 2: Find K Nearest Neighbors
ChromaDB uses approximate nearest neighbor (ANN) algorithms
                    
Step 3: Return Documents
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚ Document 1: "Refunds are processed within 7 days..."  β”‚
β”‚ Document 2: "Our return policy guarantees..."         β”‚
β”‚ Document 3: "Money back if not satisfied..."          β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Metadata Filtering:

# Find only from specific source
results = store.similarity_search(
    "refund policy",
    k=5,
    filter={"file_name": "policies.pdf"}  # ← Only from this file!
)

πŸ“Š The similarity_search_with_score Method

def similarity_search_with_score(
    self,
    query: str,
    k: int = None,
    filter: Optional[Dict[str, Any]] = None
) -> List[tuple]:  # Returns (Document, score) pairs

Why Scores Matter:

results = store.similarity_search_with_score("refund policy", k=3)

# Returns:
[
    (Document("Refunds within 7 days..."), 0.92),   # ← High confidence
    (Document("Return shipping free..."), 0.78),   # ← Medium confidence  
    (Document("Contact us for help..."), 0.31),    # ← Low confidence ⚠️
]

Use Cases:

  1. Threshold filtering β€” only use if score > 0.7
  2. Confidence display β€” show users "95% relevant"
  3. Debugging β€” understand why wrong docs were retrieved

πŸ”— The as_retriever Method (LangChain Integration)

@property
def retriever(self):
    """Default retriever with settings from config."""
    return self._store.as_retriever(
        search_type="similarity",
        search_kwargs={"k": settings.top_k_results}
    )

Why Retrievers?

LangChain's LCEL uses retrievers in chains:

chain = (
    {"context": store.retriever, "question": RunnablePassthrough()}
    | prompt
    | llm
    | output_parser
)

answer = chain.invoke("What is the refund policy?")

πŸ“ˆ The get_collection_stats Method

def get_collection_stats(self) -> Dict[str, Any]:
    return {
        "collection_name": self.collection_name,
        "document_count": collection.count(),
        "persist_directory": str(self.persist_directory),
    }

Debugging in Production:

stats = store.get_collection_stats()
# {"collection_name": "documents", "document_count": 1547, ...}

πŸ“‹ Summary

Component Pattern Purpose
VectorStore Wrapper/Facade Simplify ChromaDB API
embedding_function Dependency Injection Swap embedding models
add_documents() Batch Iterator Memory-efficient ingestion
similarity_search() Query Interface Semantic document retrieval
as_retriever() Adapter LangChain compatibility
create_vector_store() Factory Simplified instantiation