README.md

metadata

title: AssistantBot
emoji: 😻
colorFrom: red
colorTo: pink
sdk: gradio
sdk_version: 6.14.0
python_version: '3.13'
app_file: app.py
pinned: false

Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference

Ask Julia: AI Assistant

A lightweight, CPU-optimized Retrieval-Augmented Generation (RAG) assistant designed specifically for the diffusion.fyi knowledge base.

This project allows users to ask complex questions about flow matching, diffusion models, and custom research notes (like the "Julia Constant"), receiving accurate, mathematically formatted answers in real-time.

Key Features

• CPU-Optimized Inference: Powered by the Qwen-2.5-1.5B-Instruct-GGUF model via llama-cpp-python. It is heavily optimized to run on standard CPUs (like a free Hugging Face Space) without needing an expensive GPU.
• Custom Knowledge Retrieval (RAG): Uses a local FAISS vector database to read and retrieve specific context from the knowledge_base directory before answering.
• Native Math Rendering: The frontend fully supports LaTeX. Inline math ($x$) and block equations ($$y=x$$) are parsed and beautifully rendered using MathJax.
• Responsive UI: Features a sleek, mobile-friendly slide-out sidebar that doesn't obstruct the reading material.
• Strict Hallucination Control: The system prompt is engineered with strict low-temperature (0.1) constraints to ensure the bot only answers based on the provided notes and refuses to guess unknown concepts.

Tech Stack

• Backend: FastAPI, Python 3.11
• AI/LLM: llama-cpp-python, Hugging Face Hub (Qwen/Qwen2.5-1.5B-Instruct-GGUF)
• RAG Engine: FAISS (CPU), sentence-transformers (BAAI/bge-small-en-v1.5)
• Frontend: Vanilla HTML, CSS (Flexbox), JavaScript, MathJax, Marked.js
• Deployment: Docker (Hugging Face Spaces compatible)

Repository Structure

├── app.py                 # Main FastAPI application and API routes
├── llm_cpu.py             # LLM initialization and strict prompt formatting logic
├── rag.py                 # Vector DB initialization and context retrieval
├── index.html             # The frontend user interface (UI)
├── Dockerfile             # Custom Dockerfile for non-root CPU cloud deployment
├── requirements.txt       # Python dependencies
└── knowledge_base/        # Markdown files containing the core research notes
    ├── faiss_index.bin    # Pre-computed vector embeddings
    └── faiss_metadata...  # Metadata linking vectors to text chunks