Spaces:
Sleeping
Sleeping
|
Download README.md from prithvi55/talk2data: direct link, hf CLI and curl.
- Browser
- Download file 5.86 kB
-
https://huggingface.co/spaces/prithvi55/talk2data/resolve/main/README.md
- Command line
-
hf download hf://spaces/prithvi55/talk2data/README.md
-
curl -L -o README.md https://huggingface.co/spaces/prithvi55/talk2data/resolve/main/README.md
5.86 kB
| license: mit | |
| title: AutoSQL | |
| sdk: streamlit | |
| emoji: π | |
| colorTo: green | |
| pinned: true | |
| short_description: Talk2data | |
| sdk_version: 1.48.1 | |
| colorFrom: indigo | |
| app_file: app.py | |
| thumbnail: >- | |
| https://cdn-uploads.huggingface.co/production/uploads/672c97d92408a6ad52e8d829/G_PWElEW9RVSmsHTsiA9z.png | |
| # Multi-Modal RAG System | |
| ## Overview data | |
| This project is a modular, multi-modal Retrieval-Augmented Generation (RAG) system built with Streamlit. It enables natural language querying of business data, prompt tuning, PDF Q&A, and multi-agent reasoning. The system is designed for maintainability, local/cloud compatibility, and easy deployment (e.g., Hugging Face Spaces). | |
| ## Features | |
| - **SQL Chat Agent:** Chat with an agent that generates and executes SQL queries on a SQLite business database. | |
| - **Prompt Tuning Agent:** Improve user prompts for better SQL generation using LLM. | |
| - **PDF Q&A Agent:** Upload PDFs and ask questions; the agent uses semantic search and LLM to answer. | |
| - **Multi-Agent Q&A:** Complex queries are handled by a multi-agent system using SQL, prompt tuning, and PDF tools. | |
| - **Session History:** All chat interactions are preserved and displayed. | |
| - **Data Upload:** Upload CSV, Excel, or PDF files for querying and analysis. | |
| - **Vector DB Integration:** Prompts and SQL pairs are stored in ChromaDB for semantic search and reuse. | |
| - **Download/Save Results:** Download query results as CSV or save as database views. | |
| ## File Structure | |
| - `app.py`: Main Streamlit app. All core logic, UI, and agent orchestration are here. Utility functions are defined locally for maintainability. | |
| - `db_utils.py`: Database connection utilities (engine, SQLDatabase). | |
| - `llm_utils.py`: LLM and prompt template setup. | |
| - `chroma_utils.py`: ChromaDB client, collection, and embedding model setup. | |
| - `pdf_utils.py`: PDF text extraction utility. | |
| - `requirements.txt`: All required Python packages for local/cloud deployment. | |
| - `business_data.db`: SQLite database file. | |
| - `prompts.jsonl`: Stores prompt/SQL pairs for semantic search and history. | |
| ## How It Works | |
| 1. **Startup:** | |
| - Launches Streamlit UI with custom styling. | |
| - Initializes database, LLM, ChromaDB, and embedding model. | |
| 2. **Chat:** | |
| - User enters a question. | |
| - The agent checks for schema/meta queries, direct table requests, follow-ups, semantic matches, and generates SQL using LLM. | |
| - Results are displayed and saved to history. | |
| 3. **Prompt Tuning:** | |
| - User can improve prompts for better SQL generation. | |
| 4. **PDF Q&A:** | |
| - Upload PDFs and ask questions; agent finds relevant context and answers using LLM. | |
| 5. **Multi-Agent Q&A:** | |
| - Handles complex queries using all available tools. | |
| 6. **Data Upload:** | |
| - Upload CSV/Excel/PDF files for analysis and querying. | |
| 7. **Vector DB:** | |
| - Prompts and SQL pairs are stored in ChromaDB for semantic search and reuse. | |
| ## Setup & Usage | |
| 1. **Install Requirements:** | |
| ```bash | |
| pip install -r requirements.txt | |
| ``` | |
| 2. **Run the App:** | |
| ```bash | |
| streamlit run app.py | |
| ``` | |
| 3. **Upload Data:** | |
| - Use the sidebar to upload CSV, Excel, or PDF files. | |
| 4. **Interact:** | |
| - Use the chat input to ask questions about your data. | |
| - Use tabs for prompt tuning, PDF Q&A, and multi-agent queries. | |
| ## Deployment | |
| - Designed for local use and cloud platforms (e.g., Hugging Face Spaces). | |
| - All file paths are relative for compatibility. | |
| - Modular structure allows easy maintenance and extension. | |
| ## Customization | |
| - To add new agents/tools, define them locally in `app.py` or as utility modules. | |
| - To change the database, update `db_utils.py` and the database file. | |
| - To extend prompt logic, modify the prompt templates in `llm_utils.py` or directly in `app.py`. | |
| ## Architecture | |
| The Multi-Modal RAG System is designed with modularity and extensibility in mind. The architecture consists of the following layers and components: | |
| - **Streamlit UI Layer:** | |
| - Provides the chat interface, sidebar for file uploads, and tabbed navigation for different agent modes. | |
| - Displays session history, query results, and agent responses. | |
| - **Core Logic Layer (`app.py`):** | |
| - Orchestrates all agent interactions, data flow, and UI updates. | |
| - Contains local definitions for utility functions and agent logic for maintainability. | |
| - **Utility Modules:** | |
| - `db_utils.py`: Handles database engine and connection setup. | |
| - `llm_utils.py`: Manages LLM initialization and prompt templates. | |
| - `chroma_utils.py`: Sets up ChromaDB client, collections, and embedding models for semantic search. | |
| - `pdf_utils.py`: Extracts text from uploaded PDF files. | |
| - **Agents and Tools:** | |
| - **SQL Chat Agent:** Generates and executes SQL queries using LLM and database schema. | |
| - **Prompt Tuning Agent:** Improves user prompts for more accurate SQL generation. | |
| - **PDF Q&A Agent:** Uses semantic search and LLM to answer questions about uploaded PDFs. | |
| - **Multi-Agent Q&A:** Combines all tools for complex, context-aware reasoning. | |
| - **Data Flow:** | |
| 1. User submits a query or uploads a file via the UI. | |
| 2. The appropriate agent/tool processes the input, using the database, LLM, or ChromaDB as needed. | |
| 3. Results are displayed in the UI and saved to session history. | |
| 4. Prompts and SQL pairs are stored in ChromaDB and `prompts.jsonl` for future semantic search and reuse. | |
| - **Extensibility:** | |
| - New agents/tools can be added as local functions or utility modules. | |
| - The modular structure allows easy updates for deployment on cloud platforms or local environments. | |
| **Diagram (Textual):** | |
| ``` | |
| User ββ> Streamlit UI ββ> app.py ββ> [db_utils | llm_utils | chroma_utils | pdf_utils] | |
| β β | |
| β ββ> Agents/Tools (SQL, Prompt, PDF, Multi-Agent) | |
| β | |
| ββ> Results, History, Downloads, Views | |
| ``` | |
| ## License | |
| MIT License | |
| ## Author | |
| pb59 |