# PHASE_1_FOUNDATION.md — Phase 1: Project Foundation ## RAG-Based Technical Documentation Assistant --- ## 1. Phase Goal * **Business Goal**: Establish a robust, standard project layout and execution runtime that ensures clean separation of concerns, structured configuration loading, and error traceability. * **Technical Goal**: Scaffolding the python workspace directory, defining Pydantic settings loading, structured logging, custom domain exceptions, and SQLite database utility. * **Completion Criteria**: A developer can launch the workspace, load configuration properties, and execute a shell command to initialize SQL tables idempotently. --- ## 2. Scope ### Included * Folder structure layout according to `FOLDER_STRUCTURE.md`. * Virtual environment configuration files (`pyproject.toml`, `requirements.txt`, `requirements-dev.txt`). * Environment settings setup utilizing `pydantic-settings` (loading `.env`). * Structured logging configuration (JSON logs wrapper). * Custom system exceptions hierarchy (BaseException, LLMError, DBError, etc.). * SQLite connection utility and DDL schema initialization. ### Excluded * FastAPI endpoint routers. * LangGraph workflow compilation. * Vector database storage setup. * ChromaDB indexing or SentenceTransformer models loading. --- ## 3. Dependencies * Python 3.11 installed on the system. * Visual Studio Build Tools (for any local compilation requirements). --- ## 4. Deliverables * `requirements.txt` * `requirements-dev.txt` * `pyproject.toml` * `app/config.py` * `app/core/logging.py` * `app/core/exceptions.py` * `app/core/database.py` --- ## 5. Sub-Phases ### Phase 1.1: Dependency Scaffolding & Setup * **Goal**: Create environment configuration files and initialize standard directory folders. * **Tasks**: 1. Create directory tree structure (`app/api/routes`, `app/api/schemas`, `app/workflow/nodes`, `app/infrastructure/vector_store`, `app/infrastructure/embeddings`, `app/infrastructure/llm`, `app/infrastructure/document_loader`, `app/repositories`, `app/core`, `app/utils`, `ingestion`, `corpus`, `tests/unit`, `tests/integration`, `tests/api`, `data`, `chroma_db`). 2. Write `requirements.txt` listing all baseline packages. 3. Write `requirements-dev.txt` listing development tools. 4. Write `pyproject.toml` configuration for linters (Ruff/MyPy) and pytest settings. * **Files**: - `requirements.txt` - `requirements-dev.txt` - `pyproject.toml` * **Acceptance Criteria**: Running `pip install` works, directories are properly initialized, and config files match expectations. * **Verification**: Ensure python environment activates and dependencies resolve. --- ### Phase 1.2: Configuration & Logging * **Goal**: Load settings from environment variables and setup JSON logs. * **Tasks**: 1. Implement `app/config.py` using `pydantic-settings`. 2. Implement `app/core/logging.py` with custom formatting for structured logs. 3. Write `.env.example` file. * **Files**: - `app/config.py` - `app/core/logging.py` - `.env.example` * **Acceptance Criteria**: Configurations can be loaded from `.env` or system environment, defaulting correctly, and logger outputs JSON lines. * **Verification**: Check settings outputs using a python test script. --- ### Phase 1.3: Custom Exceptions & SQLite Database Setup * **Goal**: Define error mappings and build SQLite connection factory to setup databases. * **Tasks**: 1. Implement `app/core/exceptions.py` mapping domain errors. 2. Implement `app/core/database.py` containing connection context and tables setup. * **Files**: - `app/core/exceptions.py` - `app/core/database.py` * **Acceptance Criteria**: Database initialization script executes idempotently, creating `documents` and `feedback` tables with indices. * **Verification**: Executing table creation script creates `./data/registry.db` and `./data/feedback.db`. --- ## 6. AI Build Prompt (`AI_BUILD_PROMPT.md`) ```markdown # AI Build Prompt: Phase 1 (Foundation) ## Goal Build the foundation infrastructure for the RAG-Based Technical Documentation Assistant. This includes directory layout, configurations, logging, exceptions, and SQLite storage initialization. ## Files to Create 1. **requirements.txt**: Pin dependencies: - fastapi>=0.111.0 - uvicorn[standard]>=0.30.0 - langgraph>=0.1.0 - langchain>=0.2.0 - langchain-community>=0.2.0 - chromadb>=0.5.0 - sentence-transformers>=3.0.0 - pydantic>=2.7.0 - pydantic-settings>=2.2.0 - python-dotenv>=1.0.0 - groq>=0.9.0 - openai>=1.30.0 - google-genai>=0.1.0 - python-multipart>=0.0.9 - aiofiles>=23.2.0 - httpx>=0.27.0 2. **requirements-dev.txt**: pytest, pytest-asyncio, pytest-cov, ruff, black 3. **pyproject.toml**: Configure Ruff, Pytest (asyncio auto mode), MyPy 4. **app/config.py**: Class Settings inheriting from BaseSettings: - LLM_PROVIDER (str, default "google") - LLM_MODEL (str, default "gemini-2.5-flash") - GEMINI_API_KEY (Optional[str]) - GROQ_API_KEY (Optional[str]) - OPENAI_API_KEY (Optional[str]) - EMBEDDING_MODEL (str, default "sentence-transformers/all-MiniLM-L6-v2") - CHROMA_PERSIST_DIR (str, default "./chroma_db") - TOP_K (int, default 5) - MAX_RETRIES (int, default 2) - CHUNK_SIZE (int, default 512) - CHUNK_OVERLAP (int, default 64) 5. **app/core/logging.py**: Configure structured JSON log output formatting to stdout. 6. **app/core/exceptions.py**: Class hierarchy for custom exceptions: - RAGException (Base) - LLMProviderError - VectorStoreError - DatabaseError - IngestionError 7. **app/core/database.py**: Connection factory for SQLite. Define schema tables: - documents: id (TEXT PK), filename (TEXT), source_url (TEXT), file_hash (TEXT), file_size_bytes (INTEGER), file_type (TEXT), chunk_count (INTEGER), status (TEXT), ingestion_timestamp (TEXT), embedding_model (TEXT), error_message (TEXT). - feedback: id (INTEGER PK), feedback_id (TEXT UNIQUE), query (TEXT), answer (TEXT), rating (TEXT), comment (TEXT), sources_used (TEXT), session_id (TEXT), query_type (TEXT), retry_count (INTEGER), response_time_ms (INTEGER), created_at (TEXT). Write an idempotent function `initialize_db()` that runs on module load/startup. ## Constraints - Do not import any non-existent packages. - Ensure type annotations are strictly followed. - Use native sqlite3 module for simplicity and to avoid bloated ORM dependencies. ## Acceptance Criteria - Run a simple script to verify database creation. - Check that configuration settings parse standard `.env` properties. ``` --- ## 7. Verification Package ### Manual Verification 1. Install all dependencies: ```bash python -m venv .venv .venv\Scripts\activate pip install -r requirements.txt -r requirements-dev.txt ``` 2. Run database initialization check: ```bash python -c "from app.core.database import initialize_db; initialize_db(); print('DB Ready')" ``` ### Expected Results * The script should execute cleanly, print "DB Ready". * `data/registry.db` and `data/feedback.db` (or a single `data/app.db` containing both tables) should be created successfully in the folder structure. ### Failure Conditions * SQLite script crashes with SQL syntax errors. * Config initialization fails because of missing environment variables (ensure optional values are handled correctly). --- ## 8. Review Gates - [ ] All required files created successfully. - [ ] Requirements pinned with exact or minimum versions. - [ ] Idempotent DB initialization logic verified manually. - [ ] No secrets stored directly in `app/config.py`. - [ ] Log output formatting verified. - [ ] Python linter commands (`ruff check .`) return no major issues.