| # Project Context |
|
|
| ## Purpose |
| Build an "Accelerated Book Copilot" Streamlit app that ingests long-form content, delivers layered summaries, enables adjustable-speed listening, answers reader questions with cited passages, and enriches context via targeted web crawling. |
|
|
| ## Tech Stack |
| - Python 3.11+ |
| - Streamlit for the interactive UI |
| - LangChain for orchestration, retrieval, and tool management |
| - Azure OpenAI / OpenAI models for summarization and Q&A |
| - SentenceTransformers or `text-embedding-3-small` embeddings |
| - FAISS (local) or Pinecone (managed) vector store |
| - TTS utilities from FOMO project (gTTS / local models) with optional Azure Speech |
| - Newspaper3k / Requests / BeautifulSoup for focused crawling |
| - Altair for visualization (timeline, reading metrics) |
|
|
| ## Project Conventions |
|
|
| ### Code Style |
| - Pythonic style with type hints, descriptive snake_case naming, and minimal inline comments. |
| - Prefer pure functions where practical; separate UI logic from data/LLM orchestration. |
| - Use `black`/`ruff` compatible formatting (4-space indent, double quotes acceptable but prefer single quotes unless escaping). |
| - Avoid interactive `input()`; rely on Streamlit widgets and session state. |
| |
| ### Architecture Patterns |
| - Layered architecture: ingestion → processing (chunk/embedding/summarization) → experience (UI tabs). |
| - Retrieval-Augmented Generation for Q&A with citation packaging. |
| - Background task pattern for heavy preprocessing (async threads or queued jobs). |
| - Modular service helpers (ingestion, summarization, retrieval, TTS) to keep Streamlit pages thin. |
| |
| ### Testing Strategy |
| - Unit tests for ingestion utilities, chunking, and summarization prompt builders. |
| - Integration smoke tests for the Streamlit app using sample documents. |
| - Mock external APIs (Azure OpenAI, Pinecone) to keep CI deterministic. |
| - Manual regression checklist for multilingual TTS and crawl workflows before demos. |
| |
| ### Git Workflow |
| - Use feature branches named `feature/<topic>` or `fix/<topic>` off `main`. |
| - Conventional commit messages (e.g., `feat: add crawl summarizer`) with co-author trailer when pair hacking. |
| - Pull requests require reviewer sign-off plus local smoke test evidence. |
| - Avoid committing secrets; rely on `.env` templates and local dotenv loading. |
| |
| ## Domain Context |
| - Focused on long-form content acceleration for readers, researchers, and students. |
| - Supports multiple summary granularities (flash, detailed, character/plot focused). |
| - Q&A must cite source passages to maintain trust; responses should match question language when possible. |
| - Listening mode ties TTS playback to highlighted text segments for faster comprehension. |
| - Crawl mode gathers external commentary (reviews, interviews, scholarly notes) to enrich understanding. |
| |
| ## Important Constraints |
| - Must operate within hackathon time constraints; prioritize features that demo well within 3 days. |
| - Respect licensing of ingested books; app assumes user-provided content is permitted. |
| - Ensure OpenAI/Azure usage stays within provided quota; implement caching to minimize token spend. |
| - Crawling restricted to whitelisted, publicly accessible sources to avoid legal issues. |
| - Offline-friendly fallback path (local embeddings/FAISS) required if Pinecone credentials unavailable. |
| |
| ## External Dependencies |
| - Azure OpenAI / OpenAI API keys for LLM and embedding calls. |
| - Pinecone account (optional) for managed vector storage. |
| - Hugging Face / SentenceTransformers models for local embeddings. |
| - Third-party TTS backends (gTTS/local models/Azure Speech) for audio output. |
| - Newspaper3k/Requests for external article retrieval; BeautifulSoup for parsing. |
| |
