Spaces:
Runtime error
Runtime error
Commit History
-
feat(pipeline): RAG quality gate, live coverage, streaming API, recursion limit
sourabh guptaRAG quality gate (rag.py + constants.py): - COVERAGE_RELEVANCE_FLOOR = -5.0: reranker scores are absolute/cross-query-comparable so a floor correctly gates garbage retrieval (APAC → top score -6.85 → no coverage) - LLM sentinel detection: if LLM emits the no-coverage string, drop sources rather than returning contradictory "No coverage found... Sources: NFL Fumbles" Live coverage (coverage.py): - count_coverage(): hybrid retrieve + rerank → count chunks above the relevance floor Reuses the RAG pipeline so coverage = "what RAG would consider relevant" API streaming (chat.py): - agent.invoke → agent.stream(stream_mode='updates') via shared trace.log_step - ChatResponse gains sources: list[str] collected structurally from rag_search results - recursion_limit 10 → 50 Scripts: - demo_queries.py: recursion_limit 10 → 50 - build_graph.py, ingest.py: minor pipeline improvements - build_gartner_seeds.py, tag_topics.py: topic seeding tooling Docs: MVP_IMPLEMENTATION_PLAN.md addedc24f0b6
-
feat(p2): implement data ingestion pipeline with dense and BM25 sparse vectors
sourabh gupta- environment.yaml, packages/vantage_core/pyproject.toml: Add fastembed (BM25) and sentence-transformers - packages/vantage_core/models.py: Core Document and Chunk models - packages/vantage_core/loader.py: Stream HuffPost JSONL generator skipping noise rows - packages/vantage_core/chunker.py: RecursiveCharacterTextSplitter chunking safety net - packages/vantage_core/adapters/embedder_st.py: STEmbedder for all-MiniLM-L6-v2 dense embeddings - packages/vantage_core/adapters/vector_qdrant.py: Hybrid dual-vector (dense + BM25 sparse) upserts with Qdrant Vector Repo - scripts/ingest.py: Orchestrate ingestion batch flushes (sample 2k docs default) - scripts/peek.py: Verification helper scrolling payloads6985a83