Murali Chandran commited on
Commit
736560d
Β·
1 Parent(s): bf4478b

updated:-

Browse files

README
requirements

Files changed (2) hide show
  1. README.md +35 -0
  2. requirements.txt +6 -6
README.md CHANGED
@@ -11,3 +11,38 @@ pinned: false
11
  ---
12
 
13
  Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
11
  ---
12
 
13
  Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
14
+
15
+
16
+ # Ask Julia: AI Assistant
17
+
18
+ A lightweight, CPU-optimized Retrieval-Augmented Generation (RAG) assistant designed specifically for the `diffusion.fyi` knowledge base.
19
+
20
+ This project allows users to ask complex questions about flow matching, diffusion models, and custom research notes (like the "Julia Constant"), receiving accurate, mathematically formatted answers in real-time.
21
+
22
+ ## Key Features
23
+
24
+ * **CPU-Optimized Inference:** Powered by the `Qwen-2.5-1.5B-Instruct-GGUF` model via `llama-cpp-python`. It is heavily optimized to run on standard CPUs (like a free Hugging Face Space) without needing an expensive GPU.
25
+ * **Custom Knowledge Retrieval (RAG):** Uses a local FAISS vector database to read and retrieve specific context from the `knowledge_base` directory before answering.
26
+ * **Native Math Rendering:** The frontend fully supports LaTeX. Inline math (`$x$`) and block equations (`$$y=x$$`) are parsed and beautifully rendered using MathJax.
27
+ * **Responsive UI:** Features a sleek, mobile-friendly slide-out sidebar that doesn't obstruct the reading material.
28
+ * **Strict Hallucination Control:** The system prompt is engineered with strict low-temperature (`0.1`) constraints to ensure the bot *only* answers based on the provided notes and refuses to guess unknown concepts.
29
+
30
+ ## Tech Stack
31
+
32
+ * **Backend:** FastAPI, Python 3.11
33
+ * **AI/LLM:** `llama-cpp-python`, Hugging Face Hub (`Qwen/Qwen2.5-1.5B-Instruct-GGUF`)
34
+ * **RAG Engine:** FAISS (CPU), `sentence-transformers` (`BAAI/bge-small-en-v1.5`)
35
+ * **Frontend:** Vanilla HTML, CSS (Flexbox), JavaScript, MathJax, Marked.js
36
+ * **Deployment:** Docker (Hugging Face Spaces compatible)
37
+
38
+ ## Repository Structure
39
+ ```text
40
+ β”œβ”€β”€ app.py # Main FastAPI application and API routes
41
+ β”œβ”€β”€ llm_cpu.py # LLM initialization and strict prompt formatting logic
42
+ β”œβ”€β”€ rag.py # Vector DB initialization and context retrieval
43
+ β”œβ”€β”€ index.html # The frontend user interface (UI)
44
+ β”œβ”€β”€ Dockerfile # Custom Dockerfile for non-root CPU cloud deployment
45
+ β”œβ”€β”€ requirements.txt # Python dependencies
46
+ └── knowledge_base/ # Markdown files containing the core research notes
47
+ β”œβ”€β”€ faiss_index.bin # Pre-computed vector embeddings
48
+ └── faiss_metadata... # Metadata linking vectors to text chunks
requirements.txt CHANGED
@@ -7,10 +7,10 @@ sentence-transformers
7
  numpy
8
 
9
  # CPU Related -- helped for quantisation
10
- #llama-cpp-python
11
- #huggingface-hub
12
 
13
- # GPU Related
14
- transformers
15
- accelerate
16
- bitsandbytes
 
7
  numpy
8
 
9
  # CPU Related -- helped for quantisation
10
+ llama-cpp-python
11
+ huggingface-hub
12
 
13
+ # # GPU Related
14
+ # transformers
15
+ # accelerate
16
+ # bitsandbytes