Upload 10 files

README.md

@@ -1,20 +1,282 @@
----
-title: GenAI Text2SQL Analytics Assistant
-emoji: 🚀
-colorFrom: red
-colorTo: red
-sdk: docker
-app_port: 8501
-tags:
-- streamlit
-pinned: false
-short_description: An advanced Generative AI-powered analytics assistant
-license: mit
----
-
-# Welcome to Streamlit!
-
-Edit `/src/streamlit_app.py` to customize this app to your heart's desire. :heart:
-
-If you have any questions, checkout our [documentation](https://docs.streamlit.io) and [community
-forums](https://discuss.streamlit.io).
+# 🤖 GenAI Text2SQL Analytics Assistant
+
+An advanced Generative AI-powered analytics assistant that converts natural language questions into executable SQL queries and retrieves insights from a live relational database.
+
+Built using LangChain, Groq LLMs, SQLAlchemy, SQLite, and Streamlit.
+
+---
+
+# 🚀 Features
+
+- Natural Language to SQL Conversion
+- Live Database Query Execution
+- AI-Powered Business Analytics
+- SQL Query Transparency
+- Interactive Analytics Dashboard
+- CSV Export Functionality
+- Automatic SQL Cleaning & Validation
+- Multi-table SQL Reasoning
+- Conversational Analytics Experience
+- Real-time Data Insights
+
+---
+
+# 🧠 How It Works
+
+The application does not train a custom AI model.
+
+Instead, it uses a Large Language Model (LLM) to:
+1. Understand the user's business question
+2. Generate optimized SQL queries dynamically
+3. Execute queries on a live SQLite database
+4. Return analytics results and business insights
+
+---
+
+# 🏗️ Architecture
+
+```text
+User Query
+    ↓
+LLM (Groq + LangChain)
+    ↓
+SQL Query Generation
+    ↓
+SQLite Database Execution
+    ↓
+Analytics Results
+    ↓
+AI Business Summary + Visualization
+```
+
+---
+
+# 🛠️ Tech Stack
+
+| Category | Technology |
+|---|---|
+| LLM Provider | Groq |
+| Framework | LangChain |
+| Database | SQLite |
+| ORM | SQLAlchemy |
+| Frontend | Streamlit |
+| Data Generation | Faker |
+| Data Processing | Pandas |
+| Language | Python |
+
+---
+
+# 📂 Project Structure
+
+```bash
+TEXT2SQL-GENAI/
+│
+├── agents/
+│   ├── sql_agent.py
+│   └── langchain_sql_agent.py
+│
+├── database/
+│   ├── create_db.py
+│   ├── seed_data.py
+│   └── ecommerce.db
+│
+├── prompts/
+│   └── sql_prompt.py
+│
+├── utils/
+│   ├── sql_utils.py
+│   └── query_engine.py
+│
+├── app.py
+├── requirements.txt
+├── .env
+└── README.md
+```
+
+---
+
+# 🗄️ Database Schema
+
+The project uses a relational e-commerce database containing:
+
+- Customers
+- Products
+- Employees
+- Orders
+- Order Items
+
+The database was populated with synthetic business data using Faker.
+
+---
+
+# 📊 Example Questions
+
+```text
+Top 5 products by revenue
+```
+
+```text
+Which employee handled the most orders?
+```
+
+```text
+Average order value by city
+```
+
+```text
+Show revenue by product category
+```
+
+```text
+Top customers by spending
+```
+
+---
+
+# 🔥 Advanced Features Implemented
+
+## ✅ Schema-Aware Prompting
+Injected database schema directly into prompts to improve SQL accuracy.
+
+## ✅ SQL Cleaning
+Automatically removes markdown formatting and cleans generated SQL.
+
+## ✅ Error Handling & Retry Logic
+Detects invalid SQL queries and retries with correction prompts.
+
+## ✅ LangChain SQL Agent
+Implemented autonomous SQL reasoning using LangChain SQL Agent.
+
+## ✅ AI Business Summaries
+Generates human-readable business insights from query results.
+
+## ✅ Interactive Analytics Dashboard
+Built using Streamlit with:
+- Data tables
+- SQL visibility
+- CSV export
+- Charts and visualizations
+
+---
+
+# 📈 Sample Analytics Capabilities
+
+- Revenue Analysis
+- Customer Segmentation
+- Product Performance
+- Sales Trends
+- Employee Performance Metrics
+- Business Intelligence Reporting
+
+---
+
+# ⚙️ Installation
+
+## 1️⃣ Clone Repository
+
+```bash
+git clone <your_repo_url>
+cd TEXT2SQL-GENAI
+```
+
+---
+
+## 2️⃣ Create Virtual Environment
+
+### Windows
+
+```bash
+python -m venv venv
+venv\Scripts\activate
+```
+
+---
+
+## 3️⃣ Install Dependencies
+
+```bash
+pip install -r requirements.txt
+```
+
+---
+
+## 4️⃣ Configure Environment Variables
+
+Create `.env`
+
+```env
+GROQ_API_KEY=your_api_key_here
+```
+
+---
+
+# ▶️ Running The Project
+
+## Step 1 — Create Database
+
+```bash
+python database/create_db.py
+```
+
+---
+
+## Step 2 — Seed Database
+
+```bash
+python database/seed_data.py
+```
+
+---
+
+## Step 3 — Run Streamlit App
+
+```bash
+streamlit run app.py
+```
+
+---
+
+# 🌟 Key Learning Outcomes
+
+- Generative AI Application Development
+- Prompt Engineering
+- LangChain Agent Workflows
+- SQL Query Generation
+- Database Integration with LLMs
+- Conversational Analytics Systems
+- Streamlit Dashboard Development
+- AI-powered Business Intelligence
+
+---
+
+# 🎯 Future Improvements
+
+- PostgreSQL Support
+- Authentication System
+- Query History
+- Role-Based Access Control
+- Real-time Streaming Responses
+- Data Upload & Auto Analysis
+- Multi-turn Conversational Memory
+- Advanced Chart Visualizations
+
+---
+
+# 📌 Important Note
+
+This project does not fine-tune or train an LLM.
+
+Instead, it demonstrates:
+- LLM orchestration
+- schema-aware prompting
+- live SQL generation
+- autonomous database querying
+- AI analytics workflows
+
+which closely reflects real-world enterprise GenAI systems.
+
+---
+
+# 👨‍💻 Author
+
+Mohd Faizanullah

agents/langchain_sql_agent.py

@@ -0,0 +1,142 @@
+import os
+import re
+
+from dotenv import load_dotenv
+
+from langchain_groq import ChatGroq
+from langchain_core.messages import HumanMessage, SystemMessage
+
+from utils.query_engine import run_query
+
+
+load_dotenv()
+
+GROQ_API_KEY = os.getenv("GROQ_API_KEY")
+
+
+
+llm = ChatGroq(
+    groq_api_key=GROQ_API_KEY,
+    model_name="llama-3.3-70b-versatile",
+    temperature=0
+)
+
+
+
+SYSTEM_PROMPT = """
+You are an expert SQL assistant.
+
+IMPORTANT:
+1. Generate ONLY SQLite SQL queries.
+2. Do not explain anything.
+3. Use valid SQLite syntax.
+4. Return only executable SQL.
+
+DATABASE TABLES:
+
+customers(
+    customer_id,
+    name,
+    email,
+    city,
+    signup_date
+)
+
+products(
+    product_id,
+    product_name,
+    category,
+    price,
+    stock
+)
+
+employees(
+    employee_id,
+    employee_name,
+    department
+)
+
+orders(
+    order_id,
+    customer_id,
+    employee_id,
+    order_date,
+    total_amount
+)
+
+order_items(
+    order_item_id,
+    order_id,
+    product_id,
+    quantity
+)
+"""
+
+
+
+def clean_sql(query):
+
+    query = query.replace("```sql", "")
+    query = query.replace("```", "")
+
+    return query.strip()
+
+
+
+def generate_sql(question):
+
+    messages = [
+        SystemMessage(content=SYSTEM_PROMPT),
+        HumanMessage(content=question)
+    ]
+
+    response = llm.invoke(messages)
+
+    sql_query = clean_sql(response.content)
+
+    return sql_query
+
+
+
+def generate_summary(question, dataframe):
+
+    summary_prompt = f"""
+    User Question:
+    {question}
+
+    Query Result:
+    {dataframe.head(10).to_string()}
+
+    Generate a short business summary.
+    """
+
+    response = llm.invoke(summary_prompt)
+
+    return response.content
+
+
+
+def ask_agent(question):
+
+    try:
+
+        # Generate SQL
+        sql_query = generate_sql(question)
+
+        # Execute Query
+        result_df = run_query(sql_query)
+
+        # Generate Summary
+        summary = generate_summary(question, result_df)
+
+        return {
+            "sql": sql_query,
+            "data": result_df,
+            "summary": summary
+        }
+
+    except Exception as e:
+
+        return {
+            "error": str(e)
+        }

agents/sql_agent.py

@@ -0,0 +1,111 @@
+import os
+from dotenv import load_dotenv
+
+from langchain_groq import ChatGroq
+from langchain_core.messages import HumanMessage, SystemMessage
+
+from sqlalchemy import create_engine, text
+import pandas as pd
+
+from prompts.sql_prompt import SYSTEM_PROMPT
+from utils.sql_utils import clean_sql_query
+
+
+
+load_dotenv()
+
+GROQ_API_KEY = os.getenv("GROQ_API_KEY")
+
+
+
+llm = ChatGroq(
+    groq_api_key=GROQ_API_KEY,
+    model_name="llama-3.3-70b-versatile"
+)
+
+
+
+engine = create_engine("sqlite:///database/ecommerce.db")
+
+
+
+def generate_sql(user_question):
+
+    messages = [
+        SystemMessage(content=SYSTEM_PROMPT),
+        HumanMessage(content=user_question)
+    ]
+
+    response = llm.invoke(messages)
+
+    sql_query = response.content.strip()
+
+    cleaned_query = clean_sql_query(sql_query)
+
+    return cleaned_query
+
+# =====================================
+# Execute SQL
+# =====================================
+
+def execute_sql(sql_query):
+
+    try:
+        with engine.connect() as connection:
+
+            result = connection.execute(text(sql_query))
+
+            rows = result.fetchall()
+
+            columns = result.keys()
+
+            df = pd.DataFrame(rows, columns=columns)
+
+            return df
+
+    except Exception as e:
+
+        return f"SQL ERROR: {str(e)}"
+
+
+
+if __name__ == "__main__":
+
+    question = input("Ask your question: ")
+
+    # Generate SQL
+    sql_query = generate_sql(question)
+
+    print("\nGenerated SQL:\n")
+    print(sql_query)
+
+    print("\nExecuting Query...\n")
+
+    # Execute SQL
+    result = execute_sql(sql_query)
+
+    # Retry if SQL fails
+    if isinstance(result, str) and "SQL ERROR" in result:
+
+        print("\nRetrying with error feedback...\n")
+
+        retry_prompt = f"""
+        The following SQL query failed:
+
+        {sql_query}
+
+        Error:
+        {result}
+
+        Generate corrected SQLite SQL only.
+        """
+
+        corrected_sql = generate_sql(retry_prompt)
+
+        print("\nCorrected SQL:\n")
+        print(corrected_sql)
+
+        result = execute_sql(corrected_sql)
+
+    print("\nFinal Result:\n")
+    print(result)

app.py

@@ -0,0 +1,100 @@
+import streamlit as st
+import pandas as pd
+
+from agents.langchain_sql_agent import ask_agent
+
+
+
+st.set_page_config(
+    page_title="GenAI Text2SQL Assistant",
+    page_icon="🤖",
+    layout="wide"
+)
+
+
+
+st.title("🤖 GenAI Text2SQL Analytics Assistant")
+
+st.markdown("""
+Query your business database using natural language.
+
+Examples:
+- Top selling products
+- Revenue by category
+- Monthly sales trend
+- Customer spending analysis 
+""")
+
+
+
+question = st.text_input(
+    "Ask your business question:"
+)
+
+
+
+if st.button("Generate Insights"):
+
+    if question:
+
+        with st.spinner("Analyzing database..."):
+
+            response = ask_agent(question)
+
+        # Error Handling
+        if "error" in response:
+
+            st.error(response["error"])
+
+        else:
+
+
+            st.subheader("🧠 Generated SQL")
+
+            st.code(response["sql"], language="sql")
+
+            
+
+            st.subheader("📊 Query Results")
+
+            st.dataframe(
+                response["data"],
+                use_container_width=True
+            )
+
+           
+
+            csv = response["data"].to_csv(index=False)
+
+            st.download_button(
+                label="⬇ Download Results CSV",
+                data=csv,
+                file_name="query_results.csv",
+                mime="text/csv"
+            )
+
+            
+
+            st.subheader("🤖 AI Business Summary")
+
+            st.write(response["summary"])
+
+            
+
+            df = response["data"]
+
+            if len(df.columns) >= 2:
+
+                numeric_cols = df.select_dtypes(
+                    include="number"
+                ).columns
+
+                if len(numeric_cols) > 0:
+
+                    st.subheader("📈 Visualization")
+
+                    chart_col = numeric_cols[0]
+
+                    st.bar_chart(
+                        df.set_index(df.columns[0])[chart_col]
+                    )

database/create_db.py

@@ -0,0 +1,82 @@
+from sqlalchemy import create_engine, Column, Integer, String, Float, ForeignKey, Date
+from sqlalchemy.orm import declarative_base, relationship
+from sqlalchemy.orm import sessionmaker
+
+engine = create_engine("sqlite:///ecommerce.db")
+
+
+Base = declarative_base()
+
+
+class Customer(Base):
+    __tablename__ = "customers"
+
+    customer_id = Column(Integer, primary_key=True)
+    name = Column(String, nullable=False)
+    email = Column(String, unique=True, nullable=False)
+    city = Column(String)
+    signup_date = Column(Date)
+
+    orders = relationship("Order", back_populates="customer")
+
+
+
+class Product(Base):
+    __tablename__ = "products"
+
+    product_id = Column(Integer, primary_key=True)
+    product_name = Column(String, nullable=False)
+    category = Column(String)
+    price = Column(Float)
+    stock = Column(Integer)
+
+    order_items = relationship("OrderItem", back_populates="product")
+
+
+class Employee(Base):
+    __tablename__ = "employees"
+
+    employee_id = Column(Integer, primary_key=True)
+    employee_name = Column(String, nullable=False)
+    department = Column(String)
+
+    orders = relationship("Order", back_populates="employee")
+
+
+
+class Order(Base):
+    __tablename__ = "orders"
+
+    order_id = Column(Integer, primary_key=True)
+
+    customer_id = Column(Integer, ForeignKey("customers.customer_id"))
+    employee_id = Column(Integer, ForeignKey("employees.employee_id"))
+
+    order_date = Column(Date)
+    total_amount = Column(Float)
+
+    customer = relationship("Customer", back_populates="orders")
+    employee = relationship("Employee", back_populates="orders")
+
+    order_items = relationship("OrderItem", back_populates="order")
+
+
+
+class OrderItem(Base):
+    __tablename__ = "order_items"
+
+    order_item_id = Column(Integer, primary_key=True)
+
+    order_id = Column(Integer, ForeignKey("orders.order_id"))
+    product_id = Column(Integer, ForeignKey("products.product_id"))
+
+    quantity = Column(Integer)
+
+    order = relationship("Order", back_populates="order_items")
+    product = relationship("Product", back_populates="order_items")
+
+
+
+Base.metadata.create_all(engine)
+
+print(" Database and tables created successfully!")

database/seed_db.py

@@ -0,0 +1,141 @@
+import random
+from faker import Faker
+from datetime import datetime, timedelta
+
+from sqlalchemy import create_engine
+from sqlalchemy.orm import sessionmaker
+
+from create_db import (
+    Base,
+    Customer,
+    Product,
+    Employee,
+    Order,
+    OrderItem
+)
+
+
+
+fake = Faker()
+
+engine = create_engine("sqlite:///ecommerce.db")
+Session = sessionmaker(bind=engine)
+
+session = Session()
+
+
+
+customers = []
+
+for _ in range(100):
+    customer = Customer(
+        name=fake.name(),
+        email=fake.unique.email(),
+        city=fake.city(),
+        signup_date=fake.date_between(start_date="-2y", end_date="today")
+    )
+
+    customers.append(customer)
+
+session.add_all(customers)
+session.commit()
+
+print(" Customers inserted")
+
+
+
+categories = [
+    "Electronics",
+    "Clothing",
+    "Books",
+    "Home",
+    "Sports"
+]
+
+products = []
+
+for _ in range(50):
+    product = Product(
+        product_name=fake.word().capitalize() + " Product",
+        category=random.choice(categories),
+        price=round(random.uniform(10, 1000), 2),
+        stock=random.randint(10, 500)
+    )
+
+    products.append(product)
+
+session.add_all(products)
+session.commit()
+
+print(" Products inserted")
+
+
+departments = [
+    "Sales",
+    "Support",
+    "Operations",
+    "HR"
+]
+
+employees = []
+
+for _ in range(10):
+    employee = Employee(
+        employee_name=fake.name(),
+        department=random.choice(departments)
+    )
+
+    employees.append(employee)
+
+session.add_all(employees)
+session.commit()
+
+print(" Employees inserted")
+
+
+
+orders = []
+
+for _ in range(500):
+
+    customer = random.choice(customers)
+    employee = random.choice(employees)
+
+    order = Order(
+        customer_id=customer.customer_id,
+        employee_id=employee.employee_id,
+        order_date=fake.date_between(start_date="-1y", end_date="today"),
+        total_amount=0
+    )
+
+    session.add(order)
+    session.flush()
+
+    total_amount = 0
+
+    # Each order gets 1-5 products
+    for _ in range(random.randint(1, 5)):
+
+        product = random.choice(products)
+
+        quantity = random.randint(1, 5)
+
+        order_item = OrderItem(
+            order_id=order.order_id,
+            product_id=product.product_id,
+            quantity=quantity
+        )
+
+        session.add(order_item)
+
+        total_amount += product.price * quantity
+
+    order.total_amount = round(total_amount, 2)
+
+    orders.append(order)
+
+session.commit()
+
+print(" Orders and order items inserted")
+
+print("\n🎉 Database successfully populated with synthetic data!")

prompts/sql_prompt.py

@@ -0,0 +1,67 @@
+SYSTEM_PROMPT = """
+You are an expert SQL assistant.
+
+Your task:
+1. Convert natural language questions into valid SQLite SQL queries.
+2. Only generate SQL.
+3. Do NOT explain anything.
+4. Use only the provided schema.
+
+DATABASE SCHEMA:
+
+CREATE TABLE customers (
+    customer_id INTEGER PRIMARY KEY,
+    name TEXT,
+    email TEXT,
+    city TEXT,
+    signup_date DATE
+);
+
+CREATE TABLE products (
+    product_id INTEGER PRIMARY KEY,
+    product_name TEXT,
+    category TEXT,
+    price FLOAT,
+    stock INTEGER
+);
+
+CREATE TABLE employees (
+    employee_id INTEGER PRIMARY KEY,
+    employee_name TEXT,
+    department TEXT
+);
+
+CREATE TABLE orders (
+    order_id INTEGER PRIMARY KEY,
+    customer_id INTEGER,
+    employee_id INTEGER,
+    order_date DATE,
+    total_amount FLOAT
+);
+
+CREATE TABLE order_items (
+    order_item_id INTEGER PRIMARY KEY,
+    order_id INTEGER,
+    product_id INTEGER,
+    quantity INTEGER
+);
+
+FEW SHOT EXAMPLES:
+
+Q: Show all customers from Hyderabad
+SQL:
+SELECT * FROM customers
+WHERE city = 'Hyderabad';
+
+Q: Top 5 products by price
+SQL:
+SELECT product_name, price
+FROM products
+ORDER BY price DESC
+LIMIT 5;
+
+Q: Total revenue generated
+SQL:
+SELECT SUM(total_amount) AS total_revenue
+FROM orders;
+"""

utils/query_engine.py

@@ -0,0 +1,28 @@
+import pandas as pd
+from sqlalchemy import create_engine, text
+
+
+
+engine = create_engine("sqlite:///database/ecommerce.db")
+
+
+
+def run_query(sql_query):
+
+    try:
+
+        with engine.connect() as connection:
+
+            result = connection.execute(text(sql_query))
+
+            rows = result.fetchall()
+
+            columns = result.keys()
+
+            df = pd.DataFrame(rows, columns=columns)
+
+            return df
+
+    except Exception as e:
+
+        return str(e)

utils/sql_utils.py

@@ -0,0 +1,12 @@
+import re
+
+def clean_sql_query(query):
+
+    # Remove markdown SQL formatting
+    query = query.replace("```sql", "")
+    query = query.replace("```", "")
+
+    # Remove extra spaces/newlines
+    query = query.strip()
+
+    return query