capcut8128 commited on
Commit
e219bfd
·
verified ·
1 Parent(s): 22b64ec

Upload 3 files

Browse files
Files changed (3) hide show
  1. .gitignore +6 -0
  2. README.md +37 -13
  3. requirements.txt +8 -0
.gitignore ADDED
@@ -0,0 +1,6 @@
 
 
 
 
 
 
 
1
+ venv
2
+ __pycache__
3
+ data
4
+ qdrant_data
5
+ .env
6
+ .pytest_cache
README.md CHANGED
@@ -1,13 +1,37 @@
1
- ---
2
- title: Docs Qa Backend
3
- emoji: 🚀
4
- colorFrom: yellow
5
- colorTo: green
6
- sdk: gradio
7
- sdk_version: 6.30.0
8
- python_version: '3.12'
9
- app_file: app.py
10
- pinned: false
11
- ---
12
-
13
- Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # RAG Backend (v0.3)
2
+
3
+ ## Setup
4
+ ```bash
5
+ python -m venv venv
6
+ venv\Scripts\Activate.ps1 # Mac/Linux: source venv/bin/activate
7
+ pip install -r requirements.txt
8
+ copy .env.example .env # add your free GROQ_API_KEY (https://console.groq.com/keys)
9
+ uvicorn app.main:app --reload
10
+ ```
11
+ Docs UI: http://127.0.0.1:8000/docs
12
+
13
+ First start downloads two small models once (embedding ~90 MB, re-ranker ~90 MB). Data is stored in `./data`.
14
+ Upgrading from v0.2? Delete the old `qdrant_data` folder and upload your files again (chunking changed).
15
+
16
+ ## What it does
17
+ 1. **Upload** (`POST /documents`) returns immediately; parsing, chunking and embedding run in the background with progress (`GET /documents`). Duplicate files are detected by hash. Pages with no text (scans) are reported.
18
+ 2. **Understands the question** before searching: finds page numbers ("page 20", "20 no ke page"), document names, and rewrites the question into a clean English search query (Groq) so Hinglish and typos work.
19
+ 3. **Page questions** ("what is on page 20?") read that page directly instead of guessing by similarity.
20
+ 4. **Other questions** use hybrid search: vector search + BM25 keywords, fused with RRF, then a cross-encoder re-ranker. With several documents, each is searched separately so one large PDF cannot crowd out the others.
21
+ 5. **Answers** come from Groq using only the retrieved context, with numbered citations.
22
+
23
+ ## Endpoints
24
+ - `GET /health`
25
+ - `GET /documents`, `POST /documents`, `DELETE /documents/{doc_id}`
26
+ - `POST /ask` with `{"question": "...", "doc_ids": ["..."] | null, "history": [{"role": "user", "content": "..."}]}`
27
+
28
+ ## Tests (no models or API key needed)
29
+ ```bash
30
+ pip install pytest httpx reportlab
31
+ python -m pytest -q
32
+ ```
33
+
34
+ ## Notes
35
+ - Page numbers are the position in the PDF file (1 = first page), which can differ from the number printed on the page.
36
+ - Scanned PDFs (images) have no text and need OCR first.
37
+ - Settings you can change in `.env`: `LLM_MODEL`, `USE_RERANKER`, `USE_QUERY_REWRITE` (see `app/config.py` for chunk size, top_k, etc.).
requirements.txt ADDED
@@ -0,0 +1,8 @@
 
 
 
 
 
 
 
 
 
1
+ fastapi==0.115.*
2
+ uvicorn[standard]==0.32.*
3
+ python-multipart==0.0.*
4
+ pydantic-settings==2.*
5
+ pypdf==5.*
6
+ sentence-transformers==3.*
7
+ qdrant-client==1.12.*
8
+ groq>=0.13