Spaces:
Runtime error
Runtime error
Commit History
-
added config.py
sourabh guptab884c24 -
feat(agent): rag_search returns structured JSON, agent synthesizes cited report
sourabh gupta1. tools.py: bypass internal RAG LLM — rag_search now calls retrieve() directly and returns a JSON array of {citation, title, body, category, date} 2. tools.py: add retrieve import; cross-encoder (score >= 0.0) is the only gate 3. graph.py: add synthesis instructions — agent builds prose report with inline [N] citations 4. graph.py: add two concrete examples showing correct cited report format7b74d82 -
Revert "fix(agent): pass topic keyword not meta-request to rag_search"
sourabh guptaThis reverts commit c0c324e51d45f40303bf2ae98152ec7731686a73.e7bdd0e -
fix(agent): pass topic keyword not meta-request to rag_search
sourabh guptaRoot cause: agent was calling rag_search('summarize our research on social media') instead of rag_search('social media'). Cross-encoder scores meta-questions near -1 because no chunk says 'summarize our research' — they talk ABOUT the topic. Bad query -> low rerank score -> no-coverage gate -> agent falls to GPT memory. Add concrete CORRECT/WRONG examples to rag_search tool description so the LLM strips meta-framing before calling.c0c324e -
fix(agent): forbid multi-call rag_search loop per article
sourabh gupta1. Add CRITICAL guard to Rule 4: rag_search must be called exactly once 2. Explicitly forbid calling rag_search with individual article titles 3. Clarify that one call returns a synthesized answer across all articles15a1cc6 -
fix(agent,web): fix citation routing and sources rendering
sourabh gupta1. graph.py: add 'summarize what we have on X' to Rule 4 rag_search triggers 2. graph.py: remove ambiguous Rule 6 that incorrectly routed summarization to coverage_lookup 3. tools.py: tighten coverage_lookup docstring to exclude substance/summarization queries 4. types.gen.ts: add missing sources field to ChatResponse schema 5. MessageBubble.tsx: add sources field to Message interface + render source citations block 6. ChatPanel.tsx: map res.sources from API response to message object0456e7e -
fix: replace BM25 coverage counting with reranker-based pipeline
sourabh gupta- count_coverage now uses retrieve+rerank (same as rag_search) instead of BM25 keyword search, fixing the contradiction where rag_search says 'no coverage' but estimate_gap reports 100 articles - Raise COVERAGE_RELEVANCE_FLOOR from -5.0 to 0.0 (cross-encoder's natural decision boundary) — ensures rag_search and count_coverage agree on what 'covered' means - Remove dead COVERAGE_BM25_LIMIT constant - Pass _embedder and _reranker to count_coverage from tools.py25d9d16 -
refactor(coverage): replace hybrid+rerank count with BM25-only keyword search
sourabh gupta1. Add count_by_keyword() to QdrantVectorRepo — sparse BM25 query, no ML inference 2. Add count_by_keyword to VectorRepo port (SSOT) 3. Replace count_coverage() implementation: drop embedder+reranker, use BM25 count 4. Add COVERAGE_BM25_LIMIT=100 to constants.py 5. Simplify estimate_gap call: count_coverage(topic, _repo) only Mirrors tag_topics.py token-overlap logic — eliminates false positives from dense semantic drift (e.g. 'Crowd Solving' no longer counts as CrowdStrike coverage)4122295 -
feat(agent): surface momentum_percentile, label, coverage in trending_topics
sourabh gupta1. Add t.label, t.coverage_count to top_topics() RETURN clause in graph_neo4j.py 2. Expand trending_topics table: Momentum %ile, Label, Coverage columns in tools.py1fe74dc
-
chore: stop tracking .devin agent config
sourabh guptaRemove the local .devin/ rules and skills from version control and gitignore the directory. Files stay on disk; they're no longer tracked.13d7677 -
docs(readme): richer architecture diagrams + Windows disclaimer
sourabh guptaReplace the single ASCII box with a two-plane Mermaid architecture (serving vs ingestion), a request-lifecycle note, and a hexagonal layering table. Add a Windows section: the bash/Unix-only ingestion pipeline can't run natively on Windows; Windows users run the Docker image and consume the dashboard + assistant against already-populated databases.a29cad4 -
feat(agent): remove scope guardrail from system prompt
sourabh guptaDrop the "## Guardrails" section that forced a fixed refusal on off-topic questions. The agent returns to intent-based routing and will answer real-world-topic queries again instead of declining them.d0fdbe8 -
feat: update API routers, dashboard, graph panel, and gap table UI
sourabh gupta477675d -
feat(agent): generic system prompt, 10s estimate_gap cap, scope guardrail
sourabh guptaRewrite SYSTEM_PROMPT to route by query intent instead of the four hardcoded demo queries (removed verbatim few-shot examples), so custom queries route correctly. Cap the live Google Trends fetch in estimate_gap at 10s via a worker thread, so a rate-limited (429) call can no longer stall a turn for 60-240s; on timeout it returns coverage-only with demand unavailable. Add a Guardrails section: refuse off-topic questions with a fixed "Hi, I can only help you with market research." reply.836e471 -
docs: comprehensive README with run order; mine_gdelt_topics required
sourabh gupta11cb3a7 -
add: Docker deployment for HF Spaces demo
sourabh guptaSingle-container image (FastAPI + React static files, baked ML models). Serves API and frontend on port 7860. Secrets injected at runtime.3da409a -
fix(signals): guard zero baseline only on denominator
sourabh gupta(recent - baseline) / (baseline or 1) instead of substituting 1 into baseline itself. Zero-series now yields velocity=0.0 instead of -1.0, preventing demand from being double-zeroed on unresolvable B2B phrases. All non-zero scenarios produce identical velocity values.c20adad
-
refactor(web): simplify footer and typing indicator UI
sourabh gupta- Remove "Prepared for Principal Data Scientist Case Study" from footer - Remove "Reasoning in LangGraph..." badge from typing indicator - Arrange typing indicator label and dots on single line349e6d3 -
fix(topic_index): use query_points not search (client API)
sourabh guptaThe installed qdrant-client exposes query_points, not the removed search() method — matches the call style already used in vector_qdrant. Verified end-to-end: ingest 707 topics, nearest() returns relevant hits.537c6dd -
refactor(agent): semantic topic index, label-free gaps, decluttered prompt
sourabh gupta- Replace Neo4j fulltext fuzzy filter in top_topics/top_gaps with caller-supplied name list; TopicIndex does a single Qdrant dense search against pre-ingested vantage_topics collection instead of re-embedding 708 topics on every cold start - Add scripts/ingest_topics.py (one-shot: embed seeds/topics.csv → vantage_topics Qdrant collection, dense-only, idempotent upsert) - Expose QdrantVectorRepo.client so TopicIndex can share the connection - Remove DEMAND/COVERAGE/VELOCITY_HIGH_THRESHOLD constants and _label() from gap.py; GapResult.label kept for graph write compat but no longer populated; tool output is a plain numeric table with no thresholds - Rewrite system prompt: ~30 lines vs 110, removes quadrant taxonomy, makes estimate_gap-first routing and KB-miss fallback explicit - 4/4 tests passcf55da0 -
Add Langfuse observability via LangChain CallbackHandler
sourabh guptaWire LangfuseHandler into every agent.stream() call so all LLM calls, tool invocations, latency, and token usage are traced automatically. Add rag.retrieve and rag.generate sub-spans in the RAG pipeline using the v3 start_as_current_observation() context manager. Session grouping via langfuse_session_id in config metadata. Bump langfuse dep to >=3.0 and align env var name to LANGFUSE_BASE_URL.b7884a3 -
feat(pipeline): RAG quality gate, live coverage, streaming API, recursion limit
sourabh guptaRAG quality gate (rag.py + constants.py): - COVERAGE_RELEVANCE_FLOOR = -5.0: reranker scores are absolute/cross-query-comparable so a floor correctly gates garbage retrieval (APAC → top score -6.85 → no coverage) - LLM sentinel detection: if LLM emits the no-coverage string, drop sources rather than returning contradictory "No coverage found... Sources: NFL Fumbles" Live coverage (coverage.py): - count_coverage(): hybrid retrieve + rerank → count chunks above the relevance floor Reuses the RAG pipeline so coverage = "what RAG would consider relevant" API streaming (chat.py): - agent.invoke → agent.stream(stream_mode='updates') via shared trace.log_step - ChatResponse gains sources: list[str] collected structurally from rag_search results - recursion_limit 10 → 50 Scripts: - demo_queries.py: recursion_limit 10 → 50 - build_graph.py, ingest.py: minor pipeline improvements - build_gartner_seeds.py, tag_topics.py: topic seeding tooling Docs: MVP_IMPLEMENTATION_PLAN.md addedc24f0b6 -
fix(agent): prompt routing + determinism guard + gap table
sourabh gupta- estimate_gap is now THE primary tool for any specific named topic. Prompt instructs the agent to extract a clean 1-3 word term from the question (never a sentence/date/quarter) before calling it. Examples embedded. - Routing section added: specific topic → estimate_gap; overall landscape → gap_analysis/trending_topics. Closes the path where AI-healthcare got trending_topics → 0 → gave up without ever calling estimate_gap. - Determinism guard in prompt: gap_analysis, trending_topics, coverage_lookup, graph_lookup and estimate_gap are deterministic — call each at most once per question, report and stop even if the result seems thin. - gap_analysis returns a markdown table (GapScore/Topic/Label/Demand/Coverage/Entities). - Topics need not be Gartner-domain-relevant — handle anything.b992818 -
fix(agent): rank gaps by score, fuzzy topic filter, stream in web API
sourabh guptaThree issues surfaced from query logs against live data: 1. gap_analysis returned 0 rows. top_gaps gated on quadrant='priority_gap', but no topic qualifies (all 316 are 'watch', 3 'well_covered' — none reach the demand>=0.5 threshold for a priority label). Dropped the quadrant gate: now MATCH all topics with a gap_score, ORDER BY gap_score DESC. The score already encodes demand×(1−coverage); label is descriptive, not a filter. Fixes both gap_analysis and the /api/gaps endpoint. 2. trending_topics(filter='AI healthcare') returned 'drainage', 'mountains', 'airports' — because 'ai' substring-matched them. Replaced CONTAINS-word filtering with a Neo4j full-text index + fuzzy Lucene query (tokenised, so 'ai' no longer matches 'drainage'; fuzzy ~ tolerates typos: 'imigration' → immigration). New _fulltext_query() + ensure_fulltext_index() (asserted on every build()). Note: the vocabulary has no AI/healthcare/tariff topics, so those filters correctly return …bb5095f -
feat(observability): INFO-level logging across all agent components
sourabh guptaEvery layer now logs by default (LOG_LEVEL=INFO): - agent_cli: invoke→stream with per-step [TOOL CALL]/[THINK]/[TOOL RESULT] logs — the ReAct chain is now fully visible per turn with elapsed time - tools: all five tools log their return value (row count, top result, answer_chars) in addition to entry args - llm_openai: logs sys_chars/user_chars, token usage (prompt+completion+total), and answer preview (150 chars) - rag: logs source titles, context_chars before LLM call, raw vs stripped char counts and answer preview - retrieval: logs qdrant_hits + top-3 scores/titles after search; reranked count + top/bottom scores after cross-encoder - gap: logs per-topic demand/coverage/gap/label inline; logs top-1 result after sort - signals_pytrends: replaces all print() with log; INFO per batch (anchor_mean, keywords), INFO per keyword (volume, velocity), WARNING on 429b8196ed -
feat(agent): fix broken tools, add coverage_lookup, rewrite system prompt
sourabh gupta- trending_topics was calling fetch_signals() with no args after Phase 1 changed the signature to require keywords — TypeError at runtime. Fixed by reading precomputed demand/velocity from Neo4j Topic nodes instead of calling Google Trends live (fast, demo-safe, deterministic). - gap_analysis now surfaces label, demand, and coverage_count so the agent can report 'emerging_gap, 0 articles' instead of bare percentiles. - New coverage_lookup tool wraps the existing docs_for_topic() method to let the agent name which library articles cover a topic (or prove zero). - build() upsert now stores t.volume alongside demand/velocity so the trending query has the full signal set available. - top_topics() added to Neo4jGraphRepo: reads Topic nodes, supports filter= (domain token-match) and sort= (demand or velocity). - SYSTEM_PROMPT rewritten: ReAct loop explained, correct Gap v2 formula (demand × (1−coverage)), label taxonomy, tool signatures, non-negotiable rules including the '…70ff794 -
refactor(pipeline): metadata-driven coverage, per-topic scroll, pure gap fn
sourabh gupta- gap.py: pure compute_gaps(signals, coverage_counts) — no embedder/repo - models.py: Chunk.topics field (preserves tag_topics.py payload) - vector_qdrant.py: scroll_by_topic(topic) with payload index filter - ports.py: drop count_above_threshold - constants.py: anchor weather → calculator - build_graph.py: TOPICS/SAMPLE env vars, per-topic scroll, full pipeline - gap_report.py: metadata coverage, no similarity search - tests: updated to pure compute_gaps signature3c2c4aa -
feat(kg): COVERED_BY edges, wipe script, scroll_all, docs_for_topic
sourabh gupta- graph_neo4j: add wipe(), link_coverage(), docs_for_topic() - vector_qdrant: add scroll_all() via Qdrant scroll API - build_graph: wire scroll + link_coverage after gap build, print sample - wipe_graph: new standalone script for destructive resetcf1f646 -
feat(topics): mine 667 GDELT WB-taxonomy topics + orgs for 2026-06-14
sourabh guptaSources: World Bank Topical Taxonomy themes (WB_* codes) + V2Organizations from gdelt-bq.gdeltv2.gkg_partitioned. Label derived from GDELT-standardized code format (WB_<id>_<LABEL>). Deduped and appended to seeds/topics.csv.c52d3b5 -
refactor(signals): 180-day weekly horizon, negative velocity, no cache
sourabh gupta- Switch timeframe 3-m → 6-m (daily → weekly, ~26 pts, less noise) - Recent window: tail(14d) → tail(2w) ≈ 15 days at weekly granularity - Baseline: head(76d) → head(24w) for a cleaner long-run reference - Remove max(0, growth) floor — shrinking topics now score lower naturally - Drop all caching, fallback, and logging — fetch_signals(keywords) is pure - Add file header (objective / does / doesn't) matching project conventionf4ba966 -
feat(gap): Phase 1 — Gap v2 (Demand×(1−Coverage)) + anchor-normalized signals
sourabh guptaReplace momentum-percentile formula with Demand = Volume × (1 + Velocity), fix pytrends cross-batch comparability via stable anchor keyword, add label classification (emerging_gap / evergreen_gap / well_covered / watch), and persist new fields (demand, velocity, label, coverage_count) on Neo4j Topic nodes.c49a258
-
perf: block huggingface network checks on startup
sourabh gupta- Set local_files_only=True in STEmbedder and CrossEncoderReranker - Fix NameError in graph.py (SYSTEM_PROMPT reference) - Reduced model load time from multi-second network polling to <500ms local load80030b2 -
feat(kb): Solid Gartner KB — cross-industry Wikidata seed, Wikidata-native relationships, 7-day signal cache
sourabh gupta1. Rewrite data/seed_topics_wikidata.py — cross-industry SPARQL (disciplines/tech/industry/vendors) with sitelinks quality gate; fetch P279 BROADER_THAN + P452 OPERATES_IN in second POST query 2. Refactor graph_neo4j.py — remove spaCy NER entirely, add seed_from_file(), scope DETACH DELETE to Topics only, update queries to use OPERATES_IN 3. Update scripts/build_graph.py — new build() signature, wire seed_from_file() after every build 4. Add scripts/update_topics_csv.py — rebuild topics.csv from Wikidata seed labels (vendor cat excluded) 5. Rebuild packages/vantage_core/seeds/topics.csv — 448 cross-industry Wikidata topic keywords replacing 66 noisy manual terms 6. Change SIGNAL_TTL_HOURS 6 → 168 in constants.py — 7-day stable cache for case study runs Result: 449 topics + 252 entities + 236 BROADER_THAN + 34 OPERATES_IN in Neo4j; 438 signals fetcheda82d0a6 -
feat(web): Render agent markdown responses with customized tailwind classes
sourabh gupta1. Install `react-markdown` and `remark-gfm` dependencies in `apps/web` 2. Update `MessageBubble.tsx` to render agent messages as Markdown with custom overrides for headings, lists, strong text, and code blocksd72e9c0 -
feat(agent): implement comprehensive logging and fix recursion loop
sourabh gupta- Create 'vantage_core/log.py' logging utility with timed/timeit helpers. - Instrument adapters, domain layer, agent tools, scripts, and API. - Fix topic-blindness in gap_analysis tool and top_gaps Neo4j query. - Enforce recursion_limit=10 on LangGraph agent invocations. - Add behavioral STOP CONDITION to agent system prompt.1743c95 -
docs(web): Trim welcome message and update illustrative examples
sourabh gupta1. Reduced initial assistant welcome message to a concise one-liner 2. Updated SUGGESTIONS array with new industry-specific research queriescd4d912 -
feat(web): implement live interactive knowledge graph using react-force-graph
sourabh gupta- apps/web/src/features/graph/GraphPanel.tsx: Implements topic selector dropdown, measurements for container resize width, loading spinners, and interactive ForceGraph2D canvas representing Neo4j nodes and connections - apps/web/src/App.tsx: Mounts GraphPanel dynamically under the active graph tab851e4b7 -
feat(ops): add all-in-one start and stop orchestrator scripts for local development
sourabh gupta- start.sh: Rewritten to gracefully terminate stale processes, start FastAPI (via direct py311 interpreter path to avoid buffering), poll /api/gaps up to 45 seconds to accommodate local reranker weights loading, and spin up Vite dev server - stop.sh: Expanded to gracefully terminate both API and Vite frontend server processes and clean up tracking files - Makefile: Added start/stop orchestrations and mapped api/stop-api targets - .gitignore: Excluded PID tracking files (.api.pid, .web.pid, *.pid)032f23b -
feat(web): implement live multi-turn chat assistant with real-time tool badges
sourabh gupta- apps/web/src/features/chat/MessageBubble.tsx: Renders bubbles dynamically with distinct alignment and color schemas. Maps active tools as coral badges - apps/web/src/features/chat/ChatPanel.tsx: Implements message list tracking, input form, optimistic state rendering, auto-scrolling, thread_id preservation, suggestions, and inline loader with animation delay - apps/web/src/App.tsx: Mounts ChatPanel dynamically under the active chat tab4fc0cbc -
feat(web): wire live opportunity score dashboard using recharts and api facade
sourabh gupta- apps/web/src/lib/api.ts: Creates a single fetch facade for all 4 endpoints typed to types.gen.ts - apps/web/src/features/dashboard/GapBars.tsx: Renders horizontal Recharts BarChart representing Opportunity Score Ranking - apps/web/src/features/dashboard/DashboardPanel.tsx: Implements a container that fetches live gaps + signals, handles loading/error states, and coordinates layouts - apps/web/src/App.tsx: Mounts DashboardPanel dynamically under the active dashboard tabe769f38 -
style(web): sanitize codebase comments and labels to use clean and generic terms
sourabh gupta75e25b4 -
feat(web): scaffold Phase 10.0 React client application with Gartner brand design
sourabh gupta- apps/web: Scaffold React/TS/Vite/Tailwind project inside flat layout - tailwind.config.ts: Wires corporate Gartner-inspired palette (navy, azure, teal, coral) - index.html: Adds Google Inter fonts - src/App.tsx: Builds high-fidelity 3-tab layout shell (Chat, Graph, Dashboard) - postcss.config.cjs: Renamed to cjs for ESM compatibility with standard package config - npm run gen: Generates TypeScript schemas from contracts/openapi.yaml - Verified successful production compilation via tsc and vite builda5d0f2e -
feat(api): implement FastAPI read-side layer with 4 endpoints and OpenAPI contract
sourabh gupta- apps/api/main.py: FastAPI app with lifespan agent singleton and CORS middleware - apps/api/routers/chat.py: POST /api/chat — async agent invocation with thread_id sessions - apps/api/routers/gaps.py: GET /api/gaps — ranked GapScore results from Neo4j - apps/api/routers/graph.py: GET /api/graph/{topic} — entity lookup from KG - apps/api/routers/signals.py: GET /api/signals — sorted external momentum signals - scripts/export_openapi.py: static offline OpenAPI YAML export (no server required) - contracts/openapi.yaml: generated API contract (SSOT for P10 TS client) - start.sh / stop.sh: local uvicorn lifecycle scripts - Makefile: add api, stop-api, openapi targets; fix python3.11 executable6930676 -
feat(agent): revert weight bar overrides, restore original illustrative queries, and print responses first
sourabh gupta9ea0251 -
feat(agent): implement end-to-end client demand sensing demo script
sourabh gupta668371c -
feat(agent): implement production-ready LangGraph ReAct agent
sourabh gupta- docs/adr/0001-agent-architecture.md: Add ADR detailing prebuilt agent trade-offs - packages/vantage_core/adapters/graph_neo4j.py: Add case-insensitive entities_for_topic search - packages/vantage_core/agent/tools.py: Implement tools (RAG, signals, gaps, graph lookup) and gap_log logging - packages/vantage_core/agent/graph.py: Build ChatOpenAI + tools compiled LangGraph agent with MemorySaver - scripts/agent_cli.py: Add interactive REPL and Mermaid chart exporter39afcd3 -
feat(graph): implement lean Neo4j Knowledge Graph build with spaCy NER
sourabh gupta- packages/vantage_core/adapters/__init__.py: Export Neo4jGraphRepo - packages/vantage_core/adapters/graph_neo4j.py: Create lean GraphRepo storing Topics, Entities, and RELATED_TO edges - scripts/build_graph.py: Add CLI to build the Neo4j graph end-to-endcc01889 -
feat(gap): implement ultra-lean GapScore engine and reporting
sourabh gupta- packages/vantage_core/models.py: Add GapResult model and update file header - packages/vantage_core/ports.py: Declare count_above_threshold signature in VectorRepo protocol - packages/vantage_core/adapters/vector_qdrant.py: Implement count_above_threshold with threshold-filtered query_points - packages/vantage_core/domain/gap.py: Create lean compute_gaps function mapping densities to percentiles - scripts/gap_report.py: Add CLI to compute and format top-20 gap reportd365397
-
feat(signals): implement csv-driven topic seeding and cache relocation
sourabh gupta- packages/vantage_core/seeds/topics.csv: Add git-tracked list of 90+ internal/external topics - packages/vantage_core/topic_seeds.py: Create extensible load_topics() loader as the SSOT - packages/vantage_core/adapters/signals_pytrends.py: Integrate load_topics() and add 3.0s delay to protect Google API - packages/vantage_core/constants.py: Relocate SIGNAL_CACHE_FILE to data/signals.json - .gitignore: Clean up root signals.json ignore0118989 -
feat(signals): implement external trends signals with cache-aside TTL
sourabh gupta- packages/vantage_core/models.py: Add SignalRecord domain schema - packages/vantage_core/adapters/signals_pytrends.py: Implement fetch_signals using robust 90-day z-score daily interest on 20 curated keywords - scripts/refresh_signals.py: Implement CLI script printing sorted momentum percentiles - .gitignore: Ignore transient signals.json cache file26d7a99