Spaces:
Sleeping
Sleeping
Commit History
-
Add hybrid retrieval (TF-IDF + embeddings), OCR, A/B eval, and answer trust/UX
Aissugur Kurganbekov +1: Claude Opus 4.8Smarter retrieval: - Hybrid dense+lexical retrieval: fuse TF-IDF with fastembed vectors via Reciprocal Rank Fusion (retrieval.hybrid_rank), wired into answering. The abstention gate still runs on TF-IDF so grounding stays conservative. Graceful fallback to pure TF-IDF when fastembed/the model is unavailable. - Store chunk vectors on Chunk.embedding (new column); idempotent ensure_schema() migration adds it to existing SQLite DBs on startup. - OCR fallback for scanned/image PDFs (PyMuPDF + pytesseract), graceful when the Tesseract binary is absent; tesseract-ocr added to the Docker image. - A/B eval harness (app.ab_eval / eval_cli --ab) comparing TF-IDF vs hybrid. Answer trust / UX: - answer_question now returns answer metadata (confidence, question_type, top_score, term coverage, chunk counts, latency); AskResponse.metadata and Citation.match_text added. - Frontend: confidence meter, exact source-span highlighting, and a printable one-page evidence report. Verified: 4…f65a6b3