Spaces:
Sleeping
Sleeping
Murali Chandran commited on
Commit Β·
736560d
1
Parent(s): bf4478b
updated:-
Browse filesREADME
requirements
- README.md +35 -0
- requirements.txt +6 -6
README.md
CHANGED
|
@@ -11,3 +11,38 @@ pinned: false
|
|
| 11 |
---
|
| 12 |
|
| 13 |
Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 11 |
---
|
| 12 |
|
| 13 |
Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
|
| 14 |
+
|
| 15 |
+
|
| 16 |
+
# Ask Julia: AI Assistant
|
| 17 |
+
|
| 18 |
+
A lightweight, CPU-optimized Retrieval-Augmented Generation (RAG) assistant designed specifically for the `diffusion.fyi` knowledge base.
|
| 19 |
+
|
| 20 |
+
This project allows users to ask complex questions about flow matching, diffusion models, and custom research notes (like the "Julia Constant"), receiving accurate, mathematically formatted answers in real-time.
|
| 21 |
+
|
| 22 |
+
## Key Features
|
| 23 |
+
|
| 24 |
+
* **CPU-Optimized Inference:** Powered by the `Qwen-2.5-1.5B-Instruct-GGUF` model via `llama-cpp-python`. It is heavily optimized to run on standard CPUs (like a free Hugging Face Space) without needing an expensive GPU.
|
| 25 |
+
* **Custom Knowledge Retrieval (RAG):** Uses a local FAISS vector database to read and retrieve specific context from the `knowledge_base` directory before answering.
|
| 26 |
+
* **Native Math Rendering:** The frontend fully supports LaTeX. Inline math (`$x$`) and block equations (`$$y=x$$`) are parsed and beautifully rendered using MathJax.
|
| 27 |
+
* **Responsive UI:** Features a sleek, mobile-friendly slide-out sidebar that doesn't obstruct the reading material.
|
| 28 |
+
* **Strict Hallucination Control:** The system prompt is engineered with strict low-temperature (`0.1`) constraints to ensure the bot *only* answers based on the provided notes and refuses to guess unknown concepts.
|
| 29 |
+
|
| 30 |
+
## Tech Stack
|
| 31 |
+
|
| 32 |
+
* **Backend:** FastAPI, Python 3.11
|
| 33 |
+
* **AI/LLM:** `llama-cpp-python`, Hugging Face Hub (`Qwen/Qwen2.5-1.5B-Instruct-GGUF`)
|
| 34 |
+
* **RAG Engine:** FAISS (CPU), `sentence-transformers` (`BAAI/bge-small-en-v1.5`)
|
| 35 |
+
* **Frontend:** Vanilla HTML, CSS (Flexbox), JavaScript, MathJax, Marked.js
|
| 36 |
+
* **Deployment:** Docker (Hugging Face Spaces compatible)
|
| 37 |
+
|
| 38 |
+
## Repository Structure
|
| 39 |
+
```text
|
| 40 |
+
βββ app.py # Main FastAPI application and API routes
|
| 41 |
+
βββ llm_cpu.py # LLM initialization and strict prompt formatting logic
|
| 42 |
+
βββ rag.py # Vector DB initialization and context retrieval
|
| 43 |
+
βββ index.html # The frontend user interface (UI)
|
| 44 |
+
βββ Dockerfile # Custom Dockerfile for non-root CPU cloud deployment
|
| 45 |
+
βββ requirements.txt # Python dependencies
|
| 46 |
+
βββ knowledge_base/ # Markdown files containing the core research notes
|
| 47 |
+
βββ faiss_index.bin # Pre-computed vector embeddings
|
| 48 |
+
βββ faiss_metadata... # Metadata linking vectors to text chunks
|
requirements.txt
CHANGED
|
@@ -7,10 +7,10 @@ sentence-transformers
|
|
| 7 |
numpy
|
| 8 |
|
| 9 |
# CPU Related -- helped for quantisation
|
| 10 |
-
|
| 11 |
-
|
| 12 |
|
| 13 |
-
# GPU Related
|
| 14 |
-
transformers
|
| 15 |
-
accelerate
|
| 16 |
-
bitsandbytes
|
|
|
|
| 7 |
numpy
|
| 8 |
|
| 9 |
# CPU Related -- helped for quantisation
|
| 10 |
+
llama-cpp-python
|
| 11 |
+
huggingface-hub
|
| 12 |
|
| 13 |
+
# # GPU Related
|
| 14 |
+
# transformers
|
| 15 |
+
# accelerate
|
| 16 |
+
# bitsandbytes
|