Spaces:
Runtime error
Runtime error
Commit History
-
feat: update API routers, dashboard, graph panel, and gap table UI
sourabh gupta477675d
-
refactor(agent): semantic topic index, label-free gaps, decluttered prompt
sourabh gupta- Replace Neo4j fulltext fuzzy filter in top_topics/top_gaps with caller-supplied name list; TopicIndex does a single Qdrant dense search against pre-ingested vantage_topics collection instead of re-embedding 708 topics on every cold start - Add scripts/ingest_topics.py (one-shot: embed seeds/topics.csv → vantage_topics Qdrant collection, dense-only, idempotent upsert) - Expose QdrantVectorRepo.client so TopicIndex can share the connection - Remove DEMAND/COVERAGE/VELOCITY_HIGH_THRESHOLD constants and _label() from gap.py; GapResult.label kept for graph write compat but no longer populated; tool output is a plain numeric table with no thresholds - Rewrite system prompt: ~30 lines vs 110, removes quadrant taxonomy, makes estimate_gap-first routing and KB-miss fallback explicit - 4/4 tests passcf55da0 -
feat(pipeline): RAG quality gate, live coverage, streaming API, recursion limit
sourabh guptaRAG quality gate (rag.py + constants.py): - COVERAGE_RELEVANCE_FLOOR = -5.0: reranker scores are absolute/cross-query-comparable so a floor correctly gates garbage retrieval (APAC → top score -6.85 → no coverage) - LLM sentinel detection: if LLM emits the no-coverage string, drop sources rather than returning contradictory "No coverage found... Sources: NFL Fumbles" Live coverage (coverage.py): - count_coverage(): hybrid retrieve + rerank → count chunks above the relevance floor Reuses the RAG pipeline so coverage = "what RAG would consider relevant" API streaming (chat.py): - agent.invoke → agent.stream(stream_mode='updates') via shared trace.log_step - ChatResponse gains sources: list[str] collected structurally from rag_search results - recursion_limit 10 → 50 Scripts: - demo_queries.py: recursion_limit 10 → 50 - build_graph.py, ingest.py: minor pipeline improvements - build_gartner_seeds.py, tag_topics.py: topic seeding tooling Docs: MVP_IMPLEMENTATION_PLAN.md addedc24f0b6 -
fix(agent): rank gaps by score, fuzzy topic filter, stream in web API
sourabh guptaThree issues surfaced from query logs against live data: 1. gap_analysis returned 0 rows. top_gaps gated on quadrant='priority_gap', but no topic qualifies (all 316 are 'watch', 3 'well_covered' — none reach the demand>=0.5 threshold for a priority label). Dropped the quadrant gate: now MATCH all topics with a gap_score, ORDER BY gap_score DESC. The score already encodes demand×(1−coverage); label is descriptive, not a filter. Fixes both gap_analysis and the /api/gaps endpoint. 2. trending_topics(filter='AI healthcare') returned 'drainage', 'mountains', 'airports' — because 'ai' substring-matched them. Replaced CONTAINS-word filtering with a Neo4j full-text index + fuzzy Lucene query (tokenised, so 'ai' no longer matches 'drainage'; fuzzy ~ tolerates typos: 'imigration' → immigration). New _fulltext_query() + ensure_fulltext_index() (asserted on every build()). Note: the vocabulary has no AI/healthcare/tariff topics, so those filters correctly return …bb5095f -
feat(observability): INFO-level logging across all agent components
sourabh guptaEvery layer now logs by default (LOG_LEVEL=INFO): - agent_cli: invoke→stream with per-step [TOOL CALL]/[THINK]/[TOOL RESULT] logs — the ReAct chain is now fully visible per turn with elapsed time - tools: all five tools log their return value (row count, top result, answer_chars) in addition to entry args - llm_openai: logs sys_chars/user_chars, token usage (prompt+completion+total), and answer preview (150 chars) - rag: logs source titles, context_chars before LLM call, raw vs stripped char counts and answer preview - retrieval: logs qdrant_hits + top-3 scores/titles after search; reranked count + top/bottom scores after cross-encoder - gap: logs per-topic demand/coverage/gap/label inline; logs top-1 result after sort - signals_pytrends: replaces all print() with log; INFO per batch (anchor_mean, keywords), INFO per keyword (volume, velocity), WARNING on 429b8196ed -
refactor(pipeline): metadata-driven coverage, per-topic scroll, pure gap fn
sourabh gupta- gap.py: pure compute_gaps(signals, coverage_counts) — no embedder/repo - models.py: Chunk.topics field (preserves tag_topics.py payload) - vector_qdrant.py: scroll_by_topic(topic) with payload index filter - ports.py: drop count_above_threshold - constants.py: anchor weather → calculator - build_graph.py: TOPICS/SAMPLE env vars, per-topic scroll, full pipeline - gap_report.py: metadata coverage, no similarity search - tests: updated to pure compute_gaps signature3c2c4aa -
feat(kg): COVERED_BY edges, wipe script, scroll_all, docs_for_topic
sourabh gupta- graph_neo4j: add wipe(), link_coverage(), docs_for_topic() - vector_qdrant: add scroll_all() via Qdrant scroll API - build_graph: wire scroll + link_coverage after gap build, print sample - wipe_graph: new standalone script for destructive resetcf1f646 -
feat(topics): mine 667 GDELT WB-taxonomy topics + orgs for 2026-06-14
sourabh guptaSources: World Bank Topical Taxonomy themes (WB_* codes) + V2Organizations from gdelt-bq.gdeltv2.gkg_partitioned. Label derived from GDELT-standardized code format (WB_<id>_<LABEL>). Deduped and appended to seeds/topics.csv.c52d3b5
-
feat(kb): Solid Gartner KB — cross-industry Wikidata seed, Wikidata-native relationships, 7-day signal cache
sourabh gupta1. Rewrite data/seed_topics_wikidata.py — cross-industry SPARQL (disciplines/tech/industry/vendors) with sitelinks quality gate; fetch P279 BROADER_THAN + P452 OPERATES_IN in second POST query 2. Refactor graph_neo4j.py — remove spaCy NER entirely, add seed_from_file(), scope DETACH DELETE to Topics only, update queries to use OPERATES_IN 3. Update scripts/build_graph.py — new build() signature, wire seed_from_file() after every build 4. Add scripts/update_topics_csv.py — rebuild topics.csv from Wikidata seed labels (vendor cat excluded) 5. Rebuild packages/vantage_core/seeds/topics.csv — 448 cross-industry Wikidata topic keywords replacing 66 noisy manual terms 6. Change SIGNAL_TTL_HOURS 6 → 168 in constants.py — 7-day stable cache for case study runs Result: 449 topics + 252 entities + 236 BROADER_THAN + 34 OPERATES_IN in Neo4j; 438 signals fetcheda82d0a6 -
feat(agent): implement comprehensive logging and fix recursion loop
sourabh gupta- Create 'vantage_core/log.py' logging utility with timed/timeit helpers. - Instrument adapters, domain layer, agent tools, scripts, and API. - Fix topic-blindness in gap_analysis tool and top_gaps Neo4j query. - Enforce recursion_limit=10 on LangGraph agent invocations. - Add behavioral STOP CONDITION to agent system prompt.1743c95 -
feat(api): implement FastAPI read-side layer with 4 endpoints and OpenAPI contract
sourabh gupta- apps/api/main.py: FastAPI app with lifespan agent singleton and CORS middleware - apps/api/routers/chat.py: POST /api/chat — async agent invocation with thread_id sessions - apps/api/routers/gaps.py: GET /api/gaps — ranked GapScore results from Neo4j - apps/api/routers/graph.py: GET /api/graph/{topic} — entity lookup from KG - apps/api/routers/signals.py: GET /api/signals — sorted external momentum signals - scripts/export_openapi.py: static offline OpenAPI YAML export (no server required) - contracts/openapi.yaml: generated API contract (SSOT for P10 TS client) - start.sh / stop.sh: local uvicorn lifecycle scripts - Makefile: add api, stop-api, openapi targets; fix python3.11 executable6930676 -
feat(agent): revert weight bar overrides, restore original illustrative queries, and print responses first
sourabh gupta9ea0251 -
feat(agent): implement end-to-end client demand sensing demo script
sourabh gupta668371c -
feat(agent): implement production-ready LangGraph ReAct agent
sourabh gupta- docs/adr/0001-agent-architecture.md: Add ADR detailing prebuilt agent trade-offs - packages/vantage_core/adapters/graph_neo4j.py: Add case-insensitive entities_for_topic search - packages/vantage_core/agent/tools.py: Implement tools (RAG, signals, gaps, graph lookup) and gap_log logging - packages/vantage_core/agent/graph.py: Build ChatOpenAI + tools compiled LangGraph agent with MemorySaver - scripts/agent_cli.py: Add interactive REPL and Mermaid chart exporter39afcd3 -
feat(graph): implement lean Neo4j Knowledge Graph build with spaCy NER
sourabh gupta- packages/vantage_core/adapters/__init__.py: Export Neo4jGraphRepo - packages/vantage_core/adapters/graph_neo4j.py: Create lean GraphRepo storing Topics, Entities, and RELATED_TO edges - scripts/build_graph.py: Add CLI to build the Neo4j graph end-to-endcc01889 -
feat(gap): implement ultra-lean GapScore engine and reporting
sourabh gupta- packages/vantage_core/models.py: Add GapResult model and update file header - packages/vantage_core/ports.py: Declare count_above_threshold signature in VectorRepo protocol - packages/vantage_core/adapters/vector_qdrant.py: Implement count_above_threshold with threshold-filtered query_points - packages/vantage_core/domain/gap.py: Create lean compute_gaps function mapping densities to percentiles - scripts/gap_report.py: Add CLI to compute and format top-20 gap reportd365397
-
feat(signals): implement external trends signals with cache-aside TTL
sourabh gupta- packages/vantage_core/models.py: Add SignalRecord domain schema - packages/vantage_core/adapters/signals_pytrends.py: Implement fetch_signals using robust 90-day z-score daily interest on 20 curated keywords - scripts/refresh_signals.py: Implement CLI script printing sorted momentum percentiles - .gitignore: Ignore transient signals.json cache file26d7a99 -
feat(rag): implement cited RAG answer generation
sourabh gupta- packages/vantage_core/models.py: Add RAGAnswer domain model - packages/vantage_core/ports.py: Add LLM protocol for structural chat completions - packages/vantage_core/adapters/llm_openai.py: Implement OpenAILLM adapter targeting gpt-4o-mini - packages/vantage_core/domain/rag.py: Implement retrieve-prompt-generate loop with post-generation citation stripping - scripts/ask.py: Implement CLI utility for verification of precise, cited answersba8ab56 -
feat(retrieval): implement hybrid search with cross-encoder reranking
sourabh gupta- packages/vantage_core/models.py: Add ScoredChunk domain composition model - packages/vantage_core/ports.py: Abstract Embedder, VectorRepo, and Reranker protocols - packages/vantage_core/adapters/vector_qdrant.py: Implement hybrid Qdrant search with RRF fusion - packages/vantage_core/adapters/reranker_cross_encoder.py: Implement CrossEncoderReranker using ms-marco-MiniLM-L-6-v2 - packages/vantage_core/domain/retrieval.py: Orchestrate modular dense/BM25 retrieval + reranking - scripts/search.py: CLI for verification testing on 5 diverse topicsdcc99a5 -
feat(p2): implement data ingestion pipeline with dense and BM25 sparse vectors
sourabh gupta- environment.yaml, packages/vantage_core/pyproject.toml: Add fastembed (BM25) and sentence-transformers - packages/vantage_core/models.py: Core Document and Chunk models - packages/vantage_core/loader.py: Stream HuffPost JSONL generator skipping noise rows - packages/vantage_core/chunker.py: RecursiveCharacterTextSplitter chunking safety net - packages/vantage_core/adapters/embedder_st.py: STEmbedder for all-MiniLM-L6-v2 dense embeddings - packages/vantage_core/adapters/vector_qdrant.py: Hybrid dual-vector (dense + BM25 sparse) upserts with Qdrant Vector Repo - scripts/ingest.py: Orchestrate ingestion batch flushes (sample 2k docs default) - scripts/peek.py: Verification helper scrolling payloads6985a83 -
feat(p2): add download_data.sh + gitignore data/
sourabh gupta- scripts/download_data.sh: idempotent Kaggle CLI download with DATA_URL override - .gitignore: exclude data/ (reproducible via script, not committed) - Inspected 209,527 rows: p99=379 chars, only 0.13% exceed 512-char chunk size → Chunking decision: RecursiveCharacterTextSplitter as safety net only; 99.87% of docs produce exactly 1 chunk (no semantic chunker needed)6e0e5a7 -
feat(phase-1): bootstrap project scaffold and connectivity layer
sourabh gupta1. Add vantage_core package (config, constants, pyproject.toml, domain + adapters stubs) 2. Add root environment.yaml for conda Python 3.11 environment 3. Add .env.example with all nine required environment variable keys 4. Add Makefile with install, healthcheck, seed, eval, and clean targets 5. Add scripts/healthcheck.py — four-service connectivity gate (Qdrant, Neo4j, OpenAI, Langfuse) 6. Add folder skeleton for apps/, contracts/, and eval/ with .gitkeep72fa63d