|
Download docs/roadmap.md from WickTech/lumen-rag: direct link, hf CLI and curl.
- Browser
- Download file 2.62 kB
-
https://huggingface.co/spaces/WickTech/lumen-rag/resolve/main/docs/roadmap.md
- Command line
-
hf download hf://spaces/WickTech/lumen-rag/docs/roadmap.md
-
curl -L -o roadmap.md https://huggingface.co/spaces/WickTech/lumen-rag/resolve/main/docs/roadmap.md
2.62 kB
A newer version of the Gradio SDK is available: 6.32.0
Roadmap
What is done and what is next. The reasoning and status behind it is in RAG_GAP_ANALYSIS.md. Back to the README.
πΊοΈ Roadmap
- BM25 hybrid search β sparse + dense fusion via Reciprocal Rank Fusion;
vector | bm25 | hybridmode per query - PDF / DOCX / HTML ingestion β document loaders with clean text extraction
- Streaming answers β server-sent events on
POST /query/stream - Eval regression guard β
scripts/check_eval.pyin CI; build fails if recall drops - PR eval report β sticky PR comment with metric deltas on the v2 eval set; fails on a drop over 0.01
- pgvector adapter with namespaces behind a
Storeinterface (Qdrant / Pinecone still open) - Qdrant / Pinecone adapters β swap the in-memory store for a production vector DB without touching engine callers
- Cross-encoder reranker β second-stage BGE reranking via fastembed (
LUMEN_RERANKER) - Layout-aware ingestion β Docling backend for PDF/DOCX/PPTX/HTML (
LUMEN_DOCLING=1) - Answer-quality eval β DeepEval faithfulness / relevancy metrics (
lumen eval-answers) - Claim-level diagnostics β
lumen eval-answers --claimsattributes wrong answers to retrieval vs generation, offline - Multi-tenant namespaces β
LUMEN_NAMESPACEisolates collections per user/team (pgvector backend) - Async FastAPI endpoints β non-blocking model calls on a pooled async client, blocking work in worker threads (async DB and embedder clients are still open)
- Real embedder benchmark β local bge-small (fastembed) next to the hashing baseline; OpenAI row pending
- Abstention β confidence-gated refusal, calibrated on held-out questions, measured end to end
- Answerability check beyond similarity β an extractive SQuAD 2.0 reader refuses near misses that similarity lets through
- Query decomposition for multi-hop questions: measured, no gain with a real embedder (see benchmarks); kept opt-in
- LLM query rewriting for implicit multi-hop questions (needs an API key with credit)
- Contextual chunk headers β offline title + section headers before embedding and BM25; helps keyword search, small effect on dense (see benchmarks)
- LLM-written chunk context β Anthropic's full version, needs an API key
- Public benchmark β BEIR SciFact alongside the in-repo sets
- Fine-tuned embedding adapter β LoRA adapter trained on domain Q&A pairs to improve recall