AIenthusSP's picture
feat: add end-to-end testing scripts
7640b67
|
Raw History Blame Contribute Delete
6.44 kB

๐Ÿ” Deep Dive: End-to-End Testing

๐Ÿง  Why E2E Testing for RAG?

โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚                    RAG PIPELINE COMPONENTS                       โ”‚
โ”‚                                                                  โ”‚
โ”‚  Each component can work individually but may fail together:    โ”‚
โ”‚                                                                  โ”‚
โ”‚  [Loader] โ†’ [Chunker] โ†’ [Embeddings] โ†’ [VectorStore]           โ”‚
โ”‚                                              โ†“                   โ”‚
โ”‚                              [Retriever] โ†’ [LLM] โ†’ [Answer]     โ”‚
โ”‚                                                                  โ”‚
โ”‚  E2E tests verify the ENTIRE chain works together               โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜

๐Ÿ“ Test Script Architecture

scripts/
โ”œโ”€โ”€ test_e2e.py    # Comprehensive test suite
โ””โ”€โ”€ demo.py        # Interactive demo

๐ŸŽฏ test_e2e.py - Comprehensive Test Suite

Test Flow:

1. test_document_loading()   # Load sample documents
       โ†“
2. test_chunking()           # Split into chunks
       โ†“
3. test_embeddings()         # Initialize embedding model
       โ†“
4. test_vector_store()       # Store chunks with embeddings
       โ†“
5. test_retrieval()          # Search for relevant docs
       โ†“
6. test_llm()                # Test LLM generation
       โ†“
7. test_rag_pipeline()       # Full RAG query
       โ†“
8. cleanup_test_collection() # Remove test data

Key Test Methods:

test_document_loading()

loader = MultiFormatDocumentLoader()
documents = loader.load_directory(sample_dir)
# Verifies: Files are read, metadata is extracted

test_chunking()

chunker = DocumentChunker(chunk_size=500, chunk_overlap=50)
chunks = chunker.chunk_documents(documents)
# Verifies: Documents are split, stats are correct

test_embeddings()

manager = EmbeddingsManager()
vector = manager.embed_text("What is RAG?")
# Verifies: Model loads, produces 384-dim vectors

test_vector_store()

store = create_vector_store(embeddings, collection_name="test_collection")
ids = store.add_documents(chunks)
# Verifies: Chunks are stored, IDs are returned

test_retrieval()

retriever = Retriever(vector_store=store, default_k=3)
result = retriever.retrieve("What is RAG?")
# Verifies: Relevant docs are returned with scores

test_llm()

manager = LLMManager()
response = manager.generate("Say hello")
# Verifies: API key works, model responds

test_rag_pipeline()

pipeline = create_rag_pipeline(collection_name="test_collection")
result = pipeline.query("What is RAG?", include_sources=True)
# Verifies: Full pipeline works, sources included

๐ŸŽฎ demo.py - Interactive Demo

Usage:

python scripts/demo.py

Flow:

1. Load documents from data/sample_docs/
2. Chunk documents
3. Index in vector store
4. Start interactive Q&A loop
5. User asks questions
6. Pipeline retrieves + generates
7. Display answer with sources

Interactive Loop:

while True:
    question = input("Your question: ")
    result = pipeline.query(question, include_sources=True)
    print(f"Answer: {result['answer']}")
    print(f"Sources: {result['sources']}")

๐Ÿ”ง Running the Tests

Full Test Suite:

# Activate virtual environment
venv\Scripts\activate

# Run E2E tests
python scripts/test_e2e.py

Interactive Demo:

# Activate virtual environment
venv\Scripts\activate

# Run demo
python scripts/demo.py

โœ… Expected Output

============================================================
  ๐Ÿš€ RAG Pipeline End-to-End Test Suite
============================================================

============================================================
  1. Testing Document Loading
============================================================
โœ… Loaded 1 document(s)
   - sample.txt: 1024 chars

============================================================
  2. Testing Document Chunking
============================================================
โœ… Created 3 chunks
   - Average chunk size: 341 chars

============================================================
  3. Testing Embeddings
============================================================
โœ… Embedding model loaded: sentence-transformers/all-MiniLM-L6-v2
   - Vector dimensions: 384

============================================================
  4. Testing Vector Store
============================================================
โœ… Added 3 chunks to vector store
   - Collection: test_collection
   - Document count: 3

============================================================
  5. Testing Retrieval
============================================================
๐Ÿ“ Query: 'What is RAG?'
   Retrieved 3 documents
   Top score: 0.8542

============================================================
  6. Testing LLM
============================================================
โœ… LLM configured: gemini-2.5-flash
   - Test response: Hello, RAG!

============================================================
  7. Testing Complete RAG Pipeline
============================================================
โœ… RAG Pipeline created
๐Ÿ“ Testing query: 'What is RAG and what are its benefits?'
๐Ÿ’ฌ Answer: Based on the documentation, RAG is a technique...

============================================================
  8. Cleanup
============================================================
โœ… Test collection cleaned up

============================================================
  Test Summary
============================================================
โœ… All tests passed!

๐ŸŽ‰ Phase 1 Complete: Core RAG Pipeline is working!

๐Ÿ“‹ Summary

Script Purpose When to Use
test_e2e.py Verify all components After code changes
demo.py Interactive testing Demo to users