# Polyhedra - Product Requirements Document v2.1 ## Document Control | Field | Value | |-------|-------| | **Project Name** | Polyhedra | | **Version** | 2.1.0 | | **Type** | Hybrid MCP Server (Tools + Limited Agent) | | **Status** | Draft | | **Created** | November 30, 2025 | | **Last Updated** | November 30, 2025 | | **Author** | Product Team | | **Stakeholders** | Engineering, Research Community | ### Revision History | Version | Date | Author | Changes | |---------|------|--------|----------| | 2.1.0 | 2025-11-30 | Product Team | Added literature review generation capability | | 2.0.0 | 2024-11-26 | Product Team | BREAKING: Removed internal LLM calls, simplified to pure MCP tool server | | 1.x | - | Product Team | Legacy versions with built-in LLM agents | --- ## Executive Summary ### Purpose Polyhedra is a **hybrid MCP server** that augments IDE-based AI assistants with academic research capabilities. Version 2.1 introduces **literature review generation** as the first built-in agent capability, while maintaining the pure tool server approach for other features. ### Key Capabilities - **Paper Discovery**: Search and retrieve academic papers via Semantic Scholar API - **Citation Management**: Automated BibTeX generation and reference management - **RAG Retrieval**: Semantic search across indexed papers for relevant citations - **Context Management**: Read/write project files for research workflow - **Project Scaffolding**: Initialize standardized research project structures - **πŸ†• Literature Review Generation**: AI-powered synthesis of research papers into structured reviews ### Target Audience - ML/AI researchers using modern IDEs with AI assistants - PhD students requiring streamlined research workflows - Academic professionals seeking IDE-integrated paper management ### Success Criteria - Sub-2-second paper search response time - 100% valid BibTeX generation - Literature review quality: Coherent, well-structured, properly cited - Support for 4+ major IDEs (Cursor, Copilot, Windsurf, VS Code) - Setup time under 10 minutes (increased due to LLM config) --- ## 1. Product Overview ### 1.1 Vision Polyhedra is a **hybrid MCP server** that provides both: 1. **Pure tools** for data access (search, citations, files) 2. **Limited agent capabilities** for complex synthesis tasks (literature review) This design balances simplicity with capabilityβ€”keeping most features as passive tools while adding AI where synthesis is truly needed. ### 1.2 Architecture Philosophy (v2.1) **Two Deployment Modes**: #### Mode 1: Pure MCP Server (Default) Traditional tool server that IDE's AI can call directly. ``` β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ IDE (Cursor / Copilot) β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚ β”‚ IDE's LLM (Claude/GPT) β”‚ β”‚ β”‚ β”‚ β€’ Intent understanding β”‚ β”‚ β”‚ β”‚ β€’ Paper writing β”‚ β”‚ β”‚ β”‚ β€’ Hypothesis generation β”‚ β”‚ β”‚ β”‚ β€’ Method design β”‚ β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”‚ β”‚ MCP Protocol β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚ β”‚ Polyhedra MCP Server v2.1 β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ Pure Tools (no LLM): β”‚ β”‚ β”‚ β”‚ β€’ Paper search (Semantic Scholar API) β”‚ β”‚ β”‚ β”‚ β€’ Context management (project files) β”‚ β”‚ β”‚ β”‚ β€’ RAG retrieval (relevant papers) β”‚ β”‚ β”‚ β”‚ β€’ Citation management (BibTeX) β”‚ β”‚ β”‚ β”‚ β€’ File read/write β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ πŸ†• Agent Capabilities (with LLM): β”‚ β”‚ β”‚ β”‚ β€’ Literature review generation β”‚ β”‚ β”‚ β”‚ └─> Calls LLM for synthesis β”‚ β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ ``` #### Mode 2: Custom Agent (BMAD-style) πŸ†• Packaged as an autonomous agent within the IDE. ``` β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ IDE (Cursor) β”‚ β”‚ β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚ β”‚ Polyhedra Research Agent (Custom Agent) β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚ β”‚ β”‚ β”‚ Agent Orchestration Layer β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β€’ User intent understanding β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β€’ Task planning & execution β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β€’ Multi-step workflow coordination β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β€’ Error handling & recovery β”‚ β”‚ β”‚ β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚ β”‚ β”‚ β”‚ Agent's LLM β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ (Claude/GPT via IDE) β”‚ β”‚ β”‚ β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”‚ β”‚ β”‚ β”‚ Internal Tool Calls β”‚ β”‚ β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚ β”‚ β”‚ β”‚ Polyhedra Tools β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β€’ search_papers β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β€’ generate_literature_review β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β€’ query_similar_papers β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β€’ add_citation β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β€’ save_file β”‚ β”‚ β”‚ β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”‚ β”‚ β”‚ User: "@research efficient transformers" β”‚ β”‚ Agent: [Autonomously executes: search β†’ index β†’ review] β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ ``` ### 1.3 What Polyhedra Does vs What IDE Does | Capability | Polyhedra (MCP) | IDE (LLM) | |------------|-----------------|-----------| | Search academic papers | βœ“ | | | Fetch paper metadata & BibTeX | βœ“ | | | Read project files | βœ“ | | | Write/save files | βœ“ | | | RAG retrieval | βœ“ | | | Manage citations | βœ“ | | | **πŸ†• Generate literature review** | **βœ“ (built-in)** | | | Understand user intent | | βœ“ | | Generate hypotheses | | βœ“ | | Write paper sections | | βœ“ | | Decide workflow | | βœ“ | ### 1.4 Design Rationale: Why Add Literature Review? **Why literature review needs built-in LLM**: 1. **Synthesis complexity**: Requires reading 20-100 papers and synthesizing themes 2. **Structured output**: Needs taxonomy, gaps analysis, citation weaving 3. **Context window limits**: IDE's LLM may hit token limits with 50+ papers 4. **Specialized prompts**: Literature review has domain-specific patterns **Why other features stay as tools**: - Search: Data retrieval only - Citations: Deterministic operations - Files: Simple I/O - RAG: Vector similarity computation **Hybrid approach benefits**: - Keep simplicity where possible - Add intelligence where necessary - Users can still override (regenerate with IDE's LLM if preferred) ### 1.4.1 Custom Agent Mode (BMAD-style) πŸ†• **What is Custom Agent Mode?** Polyhedra can be deployed as a **self-contained research agent** within IDEs (like Cursor's Custom Agents or GitHub Copilot Agents), similar to how BMAD (Build-Measure-Analyze-Debug) works. **Key Differences from Pure MCP**: | Aspect | Pure MCP Mode | Custom Agent Mode | |--------|---------------|-------------------| | **User Interaction** | User explicitly calls tools | User gives high-level command | | **Orchestration** | IDE's LLM decides | Agent decides internally | | **Workflow** | Step-by-step manual | Autonomous multi-step | | **Example** | "search papers X" β†’ "generate review" | "@research X" β†’ [auto: search+index+review] | | **Control** | Full user control | Agent autonomy with checkpoints | **Custom Agent Capabilities**: 1. **Natural Language Commands**: ``` User: "@research efficient vision transformers" Agent: [Understands intent] β”œβ”€ Step 1: Search papers (50 results) β”œβ”€ Step 2: Index for RAG β”œβ”€ Step 3: Generate literature review └─ Step 4: Save to literature/review.md βœ“ Complete! Review saved with 47 citations. ``` 2. **Multi-step Workflows**: - Literature survey workflow - Paper comparison workflow - Citation finding workflow - Research gap analysis workflow 3. **Proactive Suggestions**: ``` Agent: "I found 47 papers. Would you like me to: 1. Generate a full review 2. Show top 10 most cited papers first 3. Focus on a specific sub-topic" ``` 4. **Error Recovery**: ``` Agent: "Failed to fetch 3 papers due to API timeout. Continuing with 44 papers. Retry failed papers? [Y/n]" ``` **Architecture: Custom Agent Implementation**: ```python # polyhedra/agent/research_agent.py class PolyhedraResearchAgent: """ Custom Agent for autonomous research workflows. Inspired by BMAD's autonomous task execution. """ def __init__(self, llm_client, tools): self.llm = llm_client self.tools = tools self.workflows = self._load_workflows() async def handle_command(self, command: str) -> str: """ Main entry point for custom agent. Supported commands: - @research - @compare - @find-gaps - @cite """ # Parse intent intent = await self._understand_intent(command) # Select workflow workflow = self.workflows.get(intent["type"]) if not workflow: return "Unknown command. Try: @research " # Execute workflow result = await workflow.execute(intent["params"]) return result.summary async def _understand_intent(self, command: str) -> dict: """Use LLM to understand user intent""" prompt = f""" Analyze this research command and extract intent: Command: {command} Available workflows: - research: Comprehensive literature survey - compare: Side-by-side paper comparison - find_gaps: Research gap identification - cite: Find relevant citations for writing Return JSON: {{ "type": "workflow_name", "params": {{ "topic": "...", "focus": "...", "depth": "standard" }} }} """ response = await self.llm.complete(prompt) return json.loads(response) def _load_workflows(self): """Load pre-defined multi-step workflows""" return { "research": LiteratureSurveyWorkflow(self.tools), "compare": PaperComparisonWorkflow(self.tools), "find_gaps": GapAnalysisWorkflow(self.tools), "cite": CitationFindingWorkflow(self.tools) } class LiteratureSurveyWorkflow: """ Multi-step workflow for literature survey. Similar to BMAD's build-measure-analyze cycle. """ def __init__(self, tools): self.tools = tools self.steps = [ SearchPapersStep(), IndexPapersStep(), GenerateReviewStep(), IdentifyGapsStep(), SaveOutputsStep() ] async def execute(self, params: dict) -> WorkflowResult: """Execute all steps with error handling""" context = {"params": params, "artifacts": {}} for step in self.steps: try: # Execute step result = await step.run(context, self.tools) # Update context context["artifacts"][step.name] = result # User checkpoint (optional) if step.requires_approval: approved = await self._request_approval(step, result) if not approved: return WorkflowResult.cancelled(step.name) except Exception as e: # Error recovery recovered = await self._handle_error(step, e, context) if not recovered: return WorkflowResult.failed(step.name, str(e)) return WorkflowResult.success(context["artifacts"]) async def _request_approval(self, step, result): """Request user approval at checkpoint""" # Implementation depends on IDE's interaction model pass async def _handle_error(self, step, error, context): """Attempt to recover from error""" # Retry logic, fallbacks, etc. pass class SearchPapersStep: """Step 1: Search papers""" name = "search_papers" requires_approval = False async def run(self, context, tools): topic = context["params"]["topic"] limit = context["params"].get("limit", 50) papers = await tools.search_papers( query=topic, limit=limit ) return { "papers": papers, "count": len(papers), "message": f"Found {len(papers)} papers on '{topic}'" } class GenerateReviewStep: """Step 3: Generate literature review""" name = "generate_review" requires_approval = True # User can preview papers before review async def run(self, context, tools): papers = context["artifacts"]["search_papers"]["papers"] review = await tools.generate_literature_review( papers=papers, focus=context["params"].get("focus"), depth=context["params"].get("depth", "standard") ) return { "review_text": review["review"], "word_count": review["metadata"]["word_count"], "gaps": review["metadata"]["research_gaps"], "cost": review["cost"]["total_usd"], "message": f"Generated {review['metadata']['word_count']}-word review" } ``` **IDE Integration (Cursor Example)**: ```json // .cursor/agents/polyhedra.json { "name": "Polyhedra Research Agent", "description": "Autonomous research assistant for academic literature", "trigger": "@research", "capabilities": [ "literature_survey", "paper_comparison", "gap_analysis", "citation_finding" ], "tools": [ "search_papers", "get_paper", "generate_literature_review", "query_similar_papers", "add_citation", "save_file" ], "system_prompt": "You are an expert research assistant...", "autonomy_level": "supervised", // Request approval at checkpoints "error_handling": "recover" // Attempt recovery before failing } ``` **Usage Examples**: ``` # Example 1: Simple research survey User: @research efficient vision transformers Agent: Starting literature survey on "efficient vision transformers"... [Step 1/5] Searching papers... βœ“ Found 47 papers from Semantic Scholar [Step 2/5] Indexing papers for RAG... βœ“ Indexed 47 papers (embeddings cached) [Step 3/5] Generating literature review... ⚠️ This will cost ~$0.15. Continue? [Y/n] User: Y Agent: βœ“ Generated 4,523-word review βœ“ Identified 5 research gaps βœ“ Added 47 citations to references.bib βœ“ Saved to literature/review.md πŸ“„ Preview: # Literature Review: Efficient Vision Transformers ## Overview Vision Transformers have demonstrated remarkable... [Read full review] [Regenerate with focus] [Export] # Example 2: Focused research with constraints User: @research sparse attention mechanisms --years 2023-2024 --focus "scaling beyond 1B parameters" --depth brief Agent: Customizing search: β€’ Time range: 2023-2024 β€’ Focus: scaling beyond 1B parameters β€’ Depth: brief (2-3 pages) [Executing workflow...] βœ“ Found 23 papers (filtered by year) βœ“ Generated brief review (2,100 words) βœ“ Cost: $0.08 Key finding: Only 3 papers address scaling >1B β†’ This could be a good research gap to explore! # Example 3: Compare two papers User: @compare arxiv:2304.12345 arxiv:2305.67890 Agent: Comparing two papers: 1. "EfficientViT" (Liu et al., 2023) 2. "MobileViT v2" (Mehta et al., 2023) [Fetching papers...] βœ“ Generated comparison table: | Aspect | EfficientViT | MobileViT v2 | |--------|--------------|--------------| | Method | Sparse attention | Hybrid CNN-ViT | | Params | 32M | 18M | | FLOPs | 2.1G | 1.8G | | Accuracy | 82.3% | 81.7% | [Full comparison] [Add to review] [Export] # Example 4: Find citations while writing User: @cite "methods for reducing transformer memory footprint" Agent: Finding relevant papers for: "methods for reducing transformer memory footprint" [Querying indexed papers...] Top 5 relevant citations: 1. [@vaswani2017attention] - Original transformer (baseline) 2. [@kitaev2020reformer] - Reversible layers 3. [@dao2022flashattention] - Memory-efficient attention 4. [@liu2023efficientvit] - Architecture optimization 5. [@child2019generating] - Sparse transformers [Insert citations] [See abstracts] [Add to bib] ``` **Benefits of Custom Agent Mode**: 1. **Reduced cognitive load**: User gives intent, agent handles details 2. **Consistent workflows**: Pre-defined best practices 3. **Error resilience**: Automatic retry and recovery 4. **Cost awareness**: Warns before expensive operations 5. **Contextual help**: Suggests next steps based on current state **Comparison with BMAD**: | Feature | BMAD | Polyhedra Agent | |---------|------|-----------------| | **Domain** | Software debugging | Research literature | | **Workflow** | Build β†’ Measure β†’ Analyze β†’ Debug | Search β†’ Index β†’ Review β†’ Analyze | | **Autonomy** | Fully autonomous | Supervised (checkpoints) | | **Tools** | Compiler, debugger, profiler | Search, RAG, LLM synthesis | | **Output** | Bug fixes | Literature reviews, gaps | | **Iteration** | Until bug fixed | Until research question answered | ### 1.5 Target Users - ML/AI researchers using Cursor, Copilot, or Windsurf - PhD students who want AI-assisted literature synthesis - Anyone who wants academic paper tools in their IDE ### 1.6 Tech Stack ```yaml language: Python 3.11+ mcp_framework: mcp (official SDK) http_client: httpx data_validation: Pydantic bibtex_parser: bibtexparser embeddings: sentence-transformers vector_store: numpy (simple cosine similarity) package_manager: uv # πŸ†• New in v2.1 llm_providers: - anthropic (Claude) - openai (GPT-4) - configurable via environment # πŸ†• Custom Agent Mode agent_framework: - workflow_engine: custom (inspired by BMAD) - state_management: in-memory with persistence - error_recovery: retry with exponential backoff - checkpoints: user approval at key steps supported_ides: - Cursor (with Custom Agents support) - GitHub Copilot (with Agents support) - Windsurf - VS Code (with MCP extension) deployment_modes: - mode_1: Pure MCP Server (tool calls) - mode_2: Custom Agent (autonomous workflows) ``` ### 1.7 Deployment Modes Comparison | Aspect | Mode 1: Pure MCP | Mode 2: Custom Agent | |--------|------------------|----------------------| | **Best For** | Power users, flexible integration | Beginners, streamlined workflows | | **User Control** | Full (every step) | Partial (checkpoints) | | **Setup Complexity** | Low (just install server) | Medium (agent config + server) | | **Learning Curve** | Steep (must know all tools) | Gentle (natural language commands) | | **Flexibility** | High (any combination) | Medium (pre-defined workflows) | | **Error Handling** | Manual | Automatic retry | | **Example Use** | "search_papers X" β†’ "index" β†’ "generate_review" | "@research X" | | **Cost Predictability** | User-controlled | Agent-estimated (with warnings) | **Recommendation**: - Start with **Mode 2** (Custom Agent) for ease of use - Switch to **Mode 1** (Pure MCP) for advanced customization --- ## 2. Multi-Agent Research System πŸ†• ### 2.1 Overview Polyhedra v2.1 introduces a **multi-agent research system** that covers the complete academic research lifecycle. Inspired by BMAD's autonomous approach, each specialized agent handles a distinct phase of research, from literature review to paper publication. ### 2.2 Research Lifecycle Coverage ``` Complete Research Pipeline: β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ Research Lifecycle β”‚ β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ β”‚ 1. Literature Review β†’ LiteratureAgent β”‚ β”‚ 2. Hypothesis Generation β†’ HypothesisAgent β”‚ β”‚ 3. Method Design β†’ MethodAgent β”‚ β”‚ 4. Experiment Planning β†’ ExperimentAgent β”‚ β”‚ 5. Result Analysis β†’ AnalysisAgent β”‚ β”‚ 6. Paper Writing β†’ WritingAgent β”‚ β”‚ 7. Review & Revision β†’ ReviewAgent β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ ``` ### 2.3 Multi-Agent Architecture ``` β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ PolyhedraAgentOrchestrator β”‚ β”‚ (Coordinator / Research Project Manager) β”‚ β”‚ β”‚ β”‚ β€’ Understands user's research goal β”‚ β”‚ β€’ Plans which agents to invoke β”‚ β”‚ β€’ Manages state across research stages β”‚ β”‚ β€’ Coordinates agent handoffs β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚ LiteratureAgent β”‚ β”‚ HypothesisAgentβ”‚ β”‚ MethodAgent β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β€’ Search papers β”‚ β”‚ β€’ Identify gapsβ”‚ β”‚ β€’ Design β”‚ β”‚ β”‚ β€’ Generate review β”‚ β”‚ β€’ Generate β”‚ β”‚ experimentsβ”‚ β”‚ β”‚ β€’ Identify gaps β”‚ β”‚ hypotheses β”‚ β”‚ β€’ Plan β”‚ β”‚ β”‚ β€’ Save artifacts β”‚ β”‚ β€’ Evaluate β”‚ β”‚ methodologyβ”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ novelty β”‚ β”‚ β€’ Resource β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ estimation β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚ ExperimentAgent β”‚ β”‚ AnalysisAgentβ”‚ β”‚ WritingAgent β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β€’ Setup code β”‚ β”‚ β€’ Parse β”‚ β”‚ β€’ Draft β”‚ β”‚ β”‚ β€’ Run experiments β”‚ β”‚ results β”‚ β”‚ sections β”‚ β”‚ β”‚ β€’ Monitor progressβ”‚ β”‚ β€’ Generate β”‚ β”‚ β€’ Cite papersβ”‚ β”‚ β”‚ β€’ Handle failures β”‚ β”‚ plots β”‚ β”‚ β€’ Format β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β€’ Statisticalβ”‚ β”‚ references β”‚ β”‚ β”‚ tests β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚ ReviewAgent β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β€’ Self-review β”‚ β”‚ β”‚ β€’ Quality checks β”‚ β”‚ β”‚ β€’ Suggest edits β”‚ β”‚ β”‚ β€’ Check citations β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ ``` ### 2.4 Key Concepts **Agent**: Specialized autonomous component for one research phase - Has specific domain expertise (literature, experiments, writing) - Can invoke tools and make decisions - Produces structured artifacts **Orchestrator**: Meta-agent that coordinates specialized agents - Understands overall research goals - Plans agent execution sequence - Manages handoffs and dependencies **Checkpoint**: User approval point before critical/expensive operations **Artifact**: Structured output from each agent (review.md, hypotheses.json, paper.tex) **State Management**: Track progress across the entire research lifecycle ### 2.5 Specialized Agents #### Agent 1: LiteratureAgent ```yaml name: LiteratureAgent trigger: "@literature " or "@research " phase: Literature Review (Phase 1) description: Search, analyze, and synthesize academic papers capabilities: - Search academic databases (Semantic Scholar) - Index papers for semantic search - Generate structured literature reviews - Identify research gaps - Compare papers - Find citations for claims workflows: - literature_survey: Comprehensive review generation - paper_comparison: Side-by-side analysis - gap_analysis: Identify underexplored areas - citation_finding: Find relevant citations outputs: - literature/review.md (structured review) - literature/gaps.md (research opportunities) - literature/papers.json (paper metadata) - references.bib (all citations) typical_duration: 3-5 minutes for 50 papers typical_cost: $0.15-0.30 ``` #### Agent 2: HypothesisAgent ```yaml name: HypothesisAgent trigger: "@hypothesize" or "@generate-ideas" phase: Hypothesis Generation (Phase 2) description: Generate and evaluate research hypotheses capabilities: - Generate hypotheses from research gaps - Evaluate novelty (check against existing work) - Assess feasibility (resources, timeline) - Rank hypotheses by potential impact - Suggest experimental validation approaches inputs: - literature/gaps.md - literature/papers.json - User constraints (budget, timeline, resources) workflows: - hypothesis_generation: Create 5-10 candidate hypotheses - novelty_check: Verify each hypothesis is novel - feasibility_analysis: Estimate resources needed - ranking: Sort by impact Γ— feasibility outputs: - ideas/hypotheses.md (ranked list) - ideas/novelty-analysis.md (why each is novel) - ideas/feasibility.json (resource estimates) typical_duration: 5-10 minutes typical_cost: $0.20-0.50 ``` #### Agent 3: MethodAgent ```yaml name: MethodAgent trigger: "@design-method" or "@plan-experiments" phase: Method Design (Phase 3) description: Design experimental methodology capabilities: - Design experimental setup - Select appropriate baselines - Choose datasets and metrics - Plan ablation studies - Estimate computational requirements - Generate code scaffolding inputs: - ideas/hypotheses.md (selected hypothesis) - User constraints (available hardware, time) workflows: - method_design: Create detailed methodology - baseline_selection: Choose comparison methods - dataset_selection: Pick appropriate datasets - metric_selection: Define evaluation metrics - resource_planning: Estimate GPU hours, cost outputs: - method/design.md (detailed methodology) - method/baselines.md (comparison methods) - method/experiments.yaml (experiment configs) - method/code_scaffold/ (initial code structure) typical_duration: 10-15 minutes typical_cost: $0.30-0.60 ``` #### Agent 4: ExperimentAgent ```yaml name: ExperimentAgent trigger: "@run-experiments" or "@execute" phase: Experiment Execution (Phase 4) description: Setup, run, and monitor experiments capabilities: - Generate experiment code from methodology - Setup training pipelines - Monitor experiment progress - Handle failures (auto-restart, checkpointing) - Log metrics to Wandb/TensorBoard - Alert on anomalies (NaN loss, divergence) inputs: - method/experiments.yaml - method/code_scaffold/ workflows: - code_generation: Generate full training code - environment_setup: Install dependencies, prepare data - experiment_execution: Run experiments with monitoring - checkpoint_management: Save/restore experiment state outputs: - experiments/runs/ (logs, checkpoints, configs) - experiments/results.json (final metrics) - experiments/logs.txt (execution logs) typical_duration: Hours to days (actual experiment runtime) typical_cost: Varies (depends on compute requirements) agent_role: Setup and monitoring, not compute itself ``` #### Agent 5: AnalysisAgent ```yaml name: AnalysisAgent trigger: "@analyze-results" phase: Result Analysis (Phase 5) description: Analyze experimental results and generate insights capabilities: - Parse experiment logs and metrics - Generate plots and visualizations - Perform statistical significance tests - Compare against baselines - Identify interesting patterns - Suggest follow-up experiments inputs: - experiments/results.json - method/baselines.md workflows: - metric_parsing: Extract results from logs - visualization: Generate plots (loss curves, accuracy tables) - statistical_analysis: T-tests, confidence intervals - comparison: Compare against baselines - insight_generation: Identify key findings outputs: - results/analysis.md (narrative analysis) - results/figures/ (all plots) - results/tables.tex (LaTeX tables) - results/statistics.json (significance tests) typical_duration: 5-10 minutes typical_cost: $0.15-0.30 ``` #### Agent 6: WritingAgent ```yaml name: WritingAgent trigger: "@write-paper" or "@draft
" phase: Paper Writing (Phase 6) description: Draft paper sections with proper citations capabilities: - Draft paper sections (intro, related work, method, etc.) - Cite papers from literature review - Generate figure captions - Format references (BibTeX) - Check for citation completeness - Suggest paper structure inputs: - literature/review.md - ideas/hypotheses.md - method/design.md - results/analysis.md - references.bib workflows: - abstract_writing: Concise summary (250 words) - introduction_writing: Motivation and contributions - related_work_writing: Compare with existing work - method_writing: Describe methodology - results_writing: Present findings - conclusion_writing: Summary and future work outputs: - paper/abstract.tex - paper/introduction.tex - paper/related_work.tex - paper/method.tex - paper/experiments.tex - paper/conclusion.tex typical_duration: 3-5 minutes per section typical_cost: $0.10-0.20 per section ``` #### Agent 7: ReviewAgent ```yaml name: ReviewAgent trigger: "@review-paper" or "@check-quality" phase: Review & Revision (Phase 7) description: Self-review and quality checking capabilities: - Check citation completeness (all claims cited) - Verify figure/table references - Check grammar and clarity - Assess novelty claims - Suggest improvements - Check against venue guidelines inputs: - paper/*.tex (all paper sections) - references.bib workflows: - citation_check: Ensure all claims are cited - reference_check: Verify figure/table numbers - clarity_check: Identify unclear sentences - novelty_check: Verify contribution claims - guideline_check: Match venue requirements (page limit, format) outputs: - review/checklist.md (quality checklist) - review/suggestions.md (improvement suggestions) - review/missing_citations.md (uncited claims) typical_duration: 5-8 minutes typical_cost: $0.20-0.40 ``` ### 2.6 Agent Orchestration #### Orchestrator Design ```python # polyhedra/orchestrator/research_orchestrator.py class PolyhedraResearchOrchestrator: """ Meta-agent that coordinates specialized agents across research lifecycle. Manages state, handoffs, and dependencies between agents. """ RESEARCH_PHASES = [ "literature_review", "hypothesis_generation", "method_design", "experiment_execution", "result_analysis", "paper_writing", "review_revision" ] def __init__(self, project_root: Path): self.project = project_root self.state = ResearchState.load(project_root) # Initialize all specialized agents self.agents = { "literature": LiteratureAgent(project_root), "hypothesis": HypothesisAgent(project_root), "method": MethodAgent(project_root), "experiment": ExperimentAgent(project_root), "analysis": AnalysisAgent(project_root), "writing": WritingAgent(project_root), "review": ReviewAgent(project_root) } async def handle_command(self, command: str) -> str: """ Main entry point for research commands. Examples: - "@research transformers" β†’ LiteratureAgent - "@hypothesize" β†’ HypothesisAgent - "@write-paper" β†’ WritingAgent - "@full-pipeline" β†’ All agents in sequence """ # Parse intent intent = await self._parse_intent(command) # Route to appropriate agent or pipeline if intent["type"] == "single_agent": agent = self.agents[intent["agent"]] result = await agent.execute(intent["params"]) return self._format_result(result) elif intent["type"] == "pipeline": # Execute multiple agents in sequence result = await self._execute_pipeline(intent["phases"]) return self._format_pipeline_result(result) async def _execute_pipeline(self, phases: list[str]) -> dict: """ Execute multiple research phases in sequence. Each agent produces artifacts consumed by next agent. """ results = {} for phase in phases: # Get agent for this phase agent = self._get_agent_for_phase(phase) # Check prerequisites if not self._check_prerequisites(phase): return { "success": False, "phase_failed": phase, "error": f"Prerequisites not met for {phase}" } # Execute agent try: result = await agent.execute_for_pipeline(self.state) results[phase] = result # Update state with artifacts self.state.update(phase, result.artifacts) self.state.save() except Exception as e: return { "success": False, "phase_failed": phase, "error": str(e), "completed_phases": list(results.keys()) } return { "success": True, "results": results } def _check_prerequisites(self, phase: str) -> bool: """Check if prerequisites for a phase are met""" prereqs = { "hypothesis_generation": ["literature_review"], "method_design": ["hypothesis_generation"], "experiment_execution": ["method_design"], "result_analysis": ["experiment_execution"], "paper_writing": ["literature_review", "result_analysis"], "review_revision": ["paper_writing"] } if phase not in prereqs: return True for prereq_phase in prereqs[phase]: if not self.state.is_completed(prereq_phase): return False return True class ResearchState: """ Tracks progress across research lifecycle. Persists to .polyhedra/state.json """ def __init__(self, project_root: Path): self.root = project_root self.phases = { "literature_review": {"status": "not_started"}, "hypothesis_generation": {"status": "not_started"}, "method_design": {"status": "not_started"}, "experiment_execution": {"status": "not_started"}, "result_analysis": {"status": "not_started"}, "paper_writing": {"status": "not_started"}, "review_revision": {"status": "not_started"} } self.artifacts = {} self.metadata = { "created": datetime.now().isoformat(), "last_updated": datetime.now().isoformat() } def update(self, phase: str, artifacts: dict): """Update state after phase completion""" self.phases[phase]["status"] = "completed" self.phases[phase]["completed_at"] = datetime.now().isoformat() self.artifacts[phase] = artifacts self.metadata["last_updated"] = datetime.now().isoformat() def is_completed(self, phase: str) -> bool: """Check if a phase is completed""" return self.phases[phase]["status"] == "completed" def save(self): """Persist state to disk""" state_file = self.root / ".polyhedra" / "state.json" state_file.parent.mkdir(parents=True, exist_ok=True) state_file.write_text(json.dumps({ "phases": self.phases, "artifacts": self.artifacts, "metadata": self.metadata }, indent=2)) @classmethod def load(cls, project_root: Path): """Load state from disk""" state_file = project_root / ".polyhedra" / "state.json" if not state_file.exists(): return cls(project_root) data = json.loads(state_file.read_text()) state = cls(project_root) state.phases = data["phases"] state.artifacts = data["artifacts"] state.metadata = data["metadata"] return state ``` ### 2.7 Complete Research Pipeline **End-to-End Command**: ``` User: @full-research "efficient vision transformers" Orchestrator: Planning complete research pipeline... Phases to execute: 1. Literature Review (3-5 min, ~$0.20) 2. Hypothesis Generation (5-10 min, ~$0.40) 3. Method Design (10-15 min, ~$0.50) 4. Paper Writing (20-30 min, ~$1.00) 5. Review & Polish (5-8 min, ~$0.30) Total estimated time: 45-70 minutes Total estimated cost: $2.40 (Experiment execution excluded - requires compute) Proceed? [Y/n] User: Y Orchestrator: [Phase 1/5] Literature Review β”œβ”€ Searching papers... βœ“ (47 papers found) β”œβ”€ Indexing for RAG... βœ“ β”œβ”€ Generating review... βœ“ (4,200 words) └─ Identifying gaps... βœ“ (6 gaps found) Phase 1 complete. Artifacts: β€’ literature/review.md β€’ literature/gaps.md β€’ references.bib (47 citations) [Phase 2/5] Hypothesis Generation β”œβ”€ Analyzing gaps... βœ“ β”œβ”€ Generating hypotheses... βœ“ (8 candidates) β”œβ”€ Novelty checking... βœ“ └─ Feasibility analysis... βœ“ Top hypothesis (ranked by impact Γ— feasibility): "Structured sparse attention patterns can scale to 10B parameters with <5% accuracy degradation" ⚠️ Checkpoint: Select hypothesis to proceed? 1. Use top-ranked hypothesis (recommended) 2. Review all 8 hypotheses first 3. Generate more hypotheses User: 1 Orchestrator: [Phase 3/5] Method Design β”œβ”€ Designing experiments... βœ“ β”œβ”€ Selecting baselines... βœ“ (Dense ViT, EfficientViT) β”œβ”€ Choosing datasets... βœ“ (ImageNet-1K) β”œβ”€ Planning metrics... βœ“ (Acc, FLOPs, Latency) └─ Generating code scaffold... βœ“ Method design complete. Artifacts: β€’ method/design.md β€’ method/experiments.yaml β€’ method/code_scaffold/ ⚠️ Phase 4 (Experiments) requires manual execution or external compute. Run experiments, then: $ @analyze-results Skip to Phase 5 (Paper Writing)? [Y/n] User: Y Orchestrator: [Phase 5/5] Paper Writing β”œβ”€ Drafting abstract... βœ“ β”œβ”€ Writing introduction... βœ“ β”œβ”€ Writing related work... βœ“ (cited 23 papers) β”œβ”€ Writing method... βœ“ β”œβ”€ Writing experiments... βœ“ (placeholder results) └─ Writing conclusion... βœ“ Paper draft complete. Artifacts: β€’ paper/main.tex (full paper) β€’ paper/figures/ (placeholders) βœ“ Research pipeline complete! Next steps: 1. Run experiments (method/experiments.yaml) 2. Update results in paper/experiments.tex 3. Run quality check: @review-paper ``` ### 2.8 Agent Architecture ```python # polyhedra/agent/__init__.py from .research_agent import PolyhedraResearchAgent from .workflows import ( LiteratureSurveyWorkflow, PaperComparisonWorkflow, GapAnalysisWorkflow, CitationFindingWorkflow ) __all__ = [ "PolyhedraResearchAgent", "LiteratureSurveyWorkflow", "PaperComparisonWorkflow", "GapAnalysisWorkflow", "CitationFindingWorkflow" ] ``` ```python # polyhedra/agent/research_agent.py from typing import Dict, Any, Optional import asyncio from dataclasses import dataclass @dataclass class WorkflowResult: """Result of workflow execution""" success: bool artifacts: Dict[str, Any] error: Optional[str] = None step_failed: Optional[str] = None @classmethod def success(cls, artifacts: Dict[str, Any]): return cls(success=True, artifacts=artifacts) @classmethod def failed(cls, step: str, error: str): return cls(success=False, artifacts={}, step_failed=step, error=error) @classmethod def cancelled(cls, step: str): return cls(success=False, artifacts={}, step_failed=step, error="User cancelled") class PolyhedraResearchAgent: """ Autonomous research agent with multi-step workflow execution. Inspired by BMAD's autonomous debugging approach: - Understands high-level user intent - Decomposes into actionable steps - Executes with error recovery - Provides progress feedback """ SUPPORTED_COMMANDS = { "research": "Comprehensive literature survey", "compare": "Side-by-side paper comparison", "find-gaps": "Research gap identification", "cite": "Find relevant citations" } def __init__(self, llm_client, polyhedra_tools, config: Optional[Dict] = None): """ Args: llm_client: IDE's LLM client (Claude/GPT) polyhedra_tools: All Polyhedra MCP tools config: Agent configuration (checkpoints, retries, etc.) """ self.llm = llm_client self.tools = polyhedra_tools self.config = config or self._default_config() # Load workflows self.workflows = { "research": LiteratureSurveyWorkflow(self.tools, self.llm), "compare": PaperComparisonWorkflow(self.tools, self.llm), "find_gaps": GapAnalysisWorkflow(self.tools, self.llm), "cite": CitationFindingWorkflow(self.tools, self.llm) } async def handle_command(self, command: str) -> str: """ Main entry point for agent commands. Example commands: - "@research efficient transformers" - "@compare arxiv:2304.12345 arxiv:2305.67890" - "@find-gaps sparse attention" - "@cite methods for reducing memory in transformers" Returns: Human-readable summary of execution """ # Parse command try: intent = await self._parse_intent(command) except ValueError as e: return self._format_error(str(e)) # Get workflow workflow = self.workflows.get(intent["command"]) if not workflow: return self._format_unknown_command(intent["command"]) # Execute workflow try: result = await workflow.execute(intent["params"]) if result.success: return self._format_success(workflow, result) else: return self._format_failure(workflow, result) except Exception as e: return self._format_error(f"Workflow failed: {str(e)}") async def _parse_intent(self, command: str) -> Dict[str, Any]: """ Parse natural language command into structured intent. Uses LLM to extract: - Command type (research, compare, etc.) - Parameters (topic, papers, options) - Preferences (depth, focus, etc.) """ prompt = f""" Parse this research command: Command: {command} Available commands: {self._format_available_commands()} Extract: 1. Command type (one of: {', '.join(self.SUPPORTED_COMMANDS.keys())}) 2. Primary parameter (topic, paper IDs, query) 3. Options (year range, depth, focus, etc.) Return JSON: {{ "command": "research", "params": {{ "topic": "extracted topic", "year_start": 2020, "year_end": 2024, "depth": "standard", "focus": null }} }} If command is unclear or unsupported, return: {{ "error": "explanation of what's wrong" }} """ response = await self.llm.complete(prompt) intent = json.loads(response) if "error" in intent: raise ValueError(intent["error"]) return intent def _format_available_commands(self) -> str: """Format supported commands for display""" lines = [] for cmd, desc in self.SUPPORTED_COMMANDS.items(): lines.append(f" @{cmd}: {desc}") return "\n".join(lines) def _format_success(self, workflow, result: WorkflowResult) -> str: """Format successful execution message""" artifacts = result.artifacts # Workflow-specific formatting if isinstance(workflow, LiteratureSurveyWorkflow): return f""" βœ“ Literature survey complete! πŸ“„ Review: {artifacts['review_path']} β€’ {artifacts['word_count']} words β€’ {artifacts['paper_count']} papers cited β€’ {len(artifacts['gaps'])} research gaps identified πŸ’° Cost: ${artifacts['cost']:.2f} πŸ“‹ Next steps: β€’ Review the gaps in literature/gaps.md β€’ Refine specific sections if needed β€’ Start hypothesis generation with @hypothesize """ elif isinstance(workflow, PaperComparisonWorkflow): return f""" βœ“ Paper comparison complete! πŸ“Š Comparison saved to: {artifacts['comparison_path']} Key differences: {self._format_comparison_summary(artifacts['comparison'])} [View full comparison] [Add to review] [Compare more papers] """ # Generic success message return f"βœ“ Workflow '{workflow.name}' completed successfully" def _format_failure(self, workflow, result: WorkflowResult) -> str: """Format failure message with recovery suggestions""" return f""" βœ— Workflow failed at step: {result.step_failed} Error: {result.error} What you can try: {self._suggest_recovery(workflow, result)} [Retry] [Skip this step] [Start over] """ def _format_error(self, error: str) -> str: """Format generic error message""" return f""" ❌ Error: {error} Available commands: {self._format_available_commands()} Example: @research efficient vision transformers """ def _default_config(self) -> Dict: """Default agent configuration""" return { "max_retries": 3, "retry_delay": 2.0, "enable_checkpoints": True, "auto_save": True, "verbose": True } class BaseWorkflow: """Base class for all workflows""" def __init__(self, tools, llm): self.tools = tools self.llm = llm self.steps = [] self.checkpoints = [] async def execute(self, params: Dict[str, Any]) -> WorkflowResult: """Execute workflow with error handling and checkpoints""" context = { "params": params, "artifacts": {}, "state": "running" } for step in self.steps: # Check if this is a checkpoint if step.name in self.checkpoints: approved = await self._checkpoint(step, context) if not approved: return WorkflowResult.cancelled(step.name) # Execute step try: result = await self._execute_step_with_retry(step, context) context["artifacts"][step.name] = result except Exception as e: # Attempt recovery recovered = await self._recover_from_error(step, e, context) if not recovered: return WorkflowResult.failed(step.name, str(e)) return WorkflowResult.success(context["artifacts"]) async def _execute_step_with_retry(self, step, context, max_retries=3): """Execute step with automatic retry on transient failures""" for attempt in range(max_retries): try: return await step.run(context, self.tools) except (TimeoutError, ConnectionError) as e: if attempt < max_retries - 1: await asyncio.sleep(2 ** attempt) # Exponential backoff continue raise async def _checkpoint(self, step, context) -> bool: """ Pause for user approval. Implementation depends on IDE's interaction model. """ # In practice, this would show a UI dialog or inline message # For now, return True (auto-approve) return True async def _recover_from_error(self, step, error, context) -> bool: """Attempt to recover from error""" # Example recovery strategies: if "rate limit" in str(error).lower(): await asyncio.sleep(60) # Wait for rate limit reset return True if "not found" in str(error).lower(): # Try alternative source return await self._try_alternative(step, context) return False ``` ### 2.5 IDE Integration **Cursor Configuration**: ```json // .cursor/agents/polyhedra.json { "name": "Polyhedra Research", "version": "2.1.0", "description": "Autonomous research assistant for academic literature", "activation": { "triggers": ["@research", "@compare", "@find-gaps", "@cite"], "file_patterns": ["*.md", "*.tex", "*.bib"] }, "capabilities": { "autonomous_execution": true, "multi_step_workflows": true, "error_recovery": true, "cost_estimation": true }, "configuration": { "llm_provider": "anthropic", "enable_checkpoints": true, "max_cost_per_workflow": 1.00, "auto_save_artifacts": true }, "workflows": [ { "name": "literature_survey", "description": "Comprehensive literature review", "estimated_time": "2-5 minutes", "estimated_cost": "$0.10-0.30" }, { "name": "paper_comparison", "description": "Compare two papers side-by-side", "estimated_time": "30-60 seconds", "estimated_cost": "$0.05-0.10" } ] } ``` **System Prompt for Agent**: ``` You are Polyhedra, an expert research assistant specializing in academic literature. Your capabilities: β€’ Search and retrieve papers from Semantic Scholar β€’ Generate comprehensive literature reviews β€’ Identify research gaps and opportunities β€’ Find relevant citations for specific claims β€’ Compare papers side-by-side Your interaction style: β€’ Proactive: Suggest next steps and alternatives β€’ Transparent: Explain what you're doing and why β€’ Cost-conscious: Warn before expensive operations β€’ Helpful: Recover gracefully from errors When given a command like "@research X": 1. Understand the research topic X 2. Plan the workflow (search β†’ index β†’ review) 3. Estimate cost and time 4. Request approval for expensive steps 5. Execute with progress updates 6. Summarize results with actionable next steps Always prioritize the researcher's time and budget. ``` ### 2.6 Example Session ``` [User opens research project in Cursor] User: @research efficient vision transformers for mobile devices Agent: πŸ” Starting literature survey... Topic: Efficient vision transformers for mobile devices Planning workflow: 1. Search papers (Semantic Scholar) 2. Filter for mobile/edge deployment focus 3. Index papers for RAG 4. Generate literature review 5. Identify research gaps Estimated time: 3-4 minutes Estimated cost: $0.15-0.20 Proceed? [Y/n] User: y Agent: [1/5] Searching papers... βœ“ Found 52 papers on "efficient vision transformers" βœ“ Filtered to 31 papers mentioning mobile/edge deployment [2/5] Analyzing paper relevance... βœ“ Top papers by citations: β€’ MobileViT (Mehta et al., 2022) - 890 citations β€’ EfficientViT (Liu et al., 2023) - 245 citations β€’ EdgeViT (Pan et al., 2022) - 156 citations [3/5] Indexing papers... βœ“ Generated embeddings for 31 papers βœ“ Cached in .poly/embeddings/ [4/5] Generating literature review... πŸ“Š Review will include: β€’ 3 main categories (architecture, training, deployment) β€’ Critical analysis of trade-offs β€’ 5-7 research gaps This step will cost ~$0.14. Continue? [Y/n] User: y Agent: βœ“ Generated literature review (4,234 words) [5/5] Finalizing outputs... βœ“ Saved to literature/review.md βœ“ Added 31 citations to references.bib βœ“ Identified 6 research gaps β†’ literature/gaps.md Total cost: $0.16 | Time: 3m 42s πŸ“„ Review Preview: # Literature Review: Efficient Vision Transformers for Mobile ## Overview This review synthesizes 31 recent papers (2022-2024) on efficient vision transformers targeting mobile and edge deployment... [Read full review] πŸ”¬ Top Research Gaps: 1. **Real-time performance on mobile GPUs** Only 3 papers benchmark on actual mobile hardware. Most use FLOPs as proxy metric. 2. **Energy consumption analysis** Critical for battery-powered devices, but rarely measured. [View all gaps] πŸ’‘ Suggested next steps: β€’ Generate hypotheses based on gaps (@hypothesize) β€’ Compare top 3 approaches (@compare) β€’ Find related work on energy efficiency (@cite "energy efficient transformers") User: @compare EfficientViT MobileViT Agent: πŸ”¬ Comparing papers... Paper 1: EfficientViT (Liu et al., 2023) Paper 2: MobileViT (Mehta et al., 2022) [Fetching full metadata...] βœ“ Generated comparison: | Metric | EfficientViT | MobileViT | |--------|--------------|-----------| | Parameters | 32M | 18M | | FLOPs | 2.1G | 1.8G | | ImageNet Acc | 82.3% | 81.7% | | Mobile latency | 12ms (A100) | 8ms (iPhone 13) | | Method | Sparse attention | Hybrid CNN-ViT | | Memory | 180MB | 95MB | Key insights: β€’ MobileViT is more memory-efficient (2x smaller) β€’ EfficientViT has slightly better accuracy (+0.6%) β€’ MobileViT benchmarked on real mobile device βœ“ β€’ EfficientViT only on server GPU βœ— [Save comparison] [Add to review] [Export table] User: save comparison Agent: βœ“ Saved to literature/comparisons/efficientvit-vs-mobilevit.md βœ“ Added to references.bib You can now cite this comparison in your paper: ```latex \cite{liu2023efficient,mehta2022mobile} ``` ``` --- ## 3. New Feature: Literature Review Generation ### 2.1 Overview **Tool Name**: `generate_literature_review` **Purpose**: Automatically synthesize multiple papers into a structured literature review with: - Thematic organization - Gap identification - Proper citations - Critical analysis ### 2.2 User Workflow ``` User: "Generate a literature review on efficient transformers" Step 1: Polyhedra searches papers β”œβ”€ Calls search_papers("efficient transformers", limit=50) β”œβ”€ Auto-indexes papers └─ Returns 47 papers found Step 2: User confirms or refines β”œβ”€ Option A: Proceed with all 47 papers β”œβ”€ Option B: Filter by year/venue └─ Option C: Add specific papers Step 3: Polyhedra generates review β”œβ”€ Sends papers to LLM with specialized prompt β”œβ”€ Generates structured review: β”‚ β”œβ”€ Abstract/Overview β”‚ β”œβ”€ Taxonomy of approaches β”‚ β”œβ”€ Key papers per category β”‚ β”œβ”€ Research gaps β”‚ └─ Future directions β”œβ”€ Auto-adds citations to references.bib └─ Saves to literature/review.md Step 4: User can refine β”œβ”€ Regenerate specific sections β”œβ”€ Add focus on specific topics └─ Adjust structure ``` ### 2.3 Tool Specification ```yaml name: generate_literature_review description: | Generate a structured literature review from a collection of papers. This tool calls an LLM to synthesize papers into a coherent review with thematic organization, gap analysis, and proper citations. input_schema: type: object properties: papers_file: type: string default: "literature/papers.json" description: Path to papers JSON (from search_papers) focus: type: string description: Optional focus area (e.g., "sparse attention mechanisms") structure: type: string enum: ["thematic", "chronological", "methodological"] default: "thematic" description: Organization structure for the review depth: type: string enum: ["brief", "standard", "comprehensive"] default: "standard" description: | - brief: 2-3 pages, high-level overview - standard: 5-8 pages, detailed analysis - comprehensive: 10-15 pages, deep dive include_gaps: type: boolean default: true description: Whether to identify research gaps output_path: type: string default: "literature/review.md" description: Where to save the generated review llm_model: type: string default: "claude-3-5-sonnet-20241022" description: LLM model to use for generation required: [] output_schema: type: object properties: review: type: string description: The generated literature review (markdown) metadata: type: object properties: paper_count: type: integer word_count: type: integer sections: type: array items: type: string research_gaps: type: array items: type: object properties: gap: type: string description: type: string citations_added: type: integer cost: type: object properties: input_tokens: type: integer output_tokens: type: integer total_usd: type: number saved_to: type: string description: File path where review was saved ``` ### 2.4 LLM Prompt Design ```python # Core prompt for literature review generation LITERATURE_REVIEW_PROMPT = """ You are an expert academic researcher tasked with writing a literature review. INPUT: {paper_count} academic papers on the topic: "{topic}" PAPERS DATA: {papers_json} TASK: Write a comprehensive literature review with the following structure: 1. **Overview** (1 paragraph) - Summarize the research area - State the scope of this review - Preview main findings 2. **Taxonomy of Approaches** (main section) - Organize papers into 3-5 thematic categories - For each category: * Define the approach * List 3-5 key papers with citations * Compare methods * Note trends and developments 3. **Critical Analysis** (1-2 paragraphs per category) - Strengths and limitations of each approach - Comparative insights - Identify what works and what doesn't 4. **Research Gaps** (bullet list) - Identify 3-7 gaps in current literature - For each gap: * What is missing? * Why does it matter? * Potential research directions 5. **Conclusion** (1 paragraph) - Synthesize main insights - Highlight most promising directions REQUIREMENTS: - Use markdown format - Cite papers as [Author et al., Year] or [@bibtex_key] - Be critical and analytical, not just descriptive - Focus on {focus_area} if specified - Target length: {target_length} words - Structure: {structure_type} OUTPUT FORMAT: Return ONLY the markdown content, no preamble. """ # Prompt for gap identification GAP_IDENTIFICATION_PROMPT = """ Based on the literature review you just generated, identify research gaps. For each gap, provide: 1. Gap title (concise) 2. Description (2-3 sentences) 3. Why it matters 4. Potential approaches to address it Return as JSON array: [ { "title": "Gap title", "description": "...", "significance": "...", "potential_approaches": ["approach1", "approach2"] } ] """ ``` ### 2.5 Implementation Architecture ```python # services/literature_review_service.py from anthropic import Anthropic from openai import OpenAI import json from pathlib import Path from typing import Literal class LiteratureReviewService: def __init__( self, llm_provider: Literal["anthropic", "openai"] = "anthropic", api_key: str = None ): self.provider = llm_provider if llm_provider == "anthropic": self.client = Anthropic(api_key=api_key) self.default_model = "claude-3-5-sonnet-20241022" else: self.client = OpenAI(api_key=api_key) self.default_model = "gpt-4-turbo-preview" async def generate_review( self, papers: list[dict], focus: str = None, structure: str = "thematic", depth: str = "standard", include_gaps: bool = True, model: str = None ) -> dict: """ Generate literature review from papers Returns: { "review": "markdown content", "metadata": {...}, "cost": {...} } """ model = model or self.default_model # Prepare papers data papers_summary = self._prepare_papers_summary(papers) # Determine target length length_map = { "brief": "1500-2000", "standard": "3000-5000", "comprehensive": "6000-10000" } target_length = length_map[depth] # Build prompt prompt = LITERATURE_REVIEW_PROMPT.format( paper_count=len(papers), topic=focus or "the research area", papers_json=papers_summary, focus_area=focus or "general research trends", target_length=target_length, structure_type=structure ) # Call LLM if self.provider == "anthropic": response = self.client.messages.create( model=model, max_tokens=8192, messages=[{"role": "user", "content": prompt}] ) review_text = response.content[0].text usage = { "input_tokens": response.usage.input_tokens, "output_tokens": response.usage.output_tokens, "total_usd": self._calculate_cost_anthropic( response.usage.input_tokens, response.usage.output_tokens, model ) } else: # OpenAI response = self.client.chat.completions.create( model=model, messages=[{"role": "user", "content": prompt}], max_tokens=8192 ) review_text = response.choices[0].message.content usage = { "input_tokens": response.usage.prompt_tokens, "output_tokens": response.usage.completion_tokens, "total_usd": self._calculate_cost_openai( response.usage.prompt_tokens, response.usage.completion_tokens, model ) } # Extract metadata word_count = len(review_text.split()) sections = self._extract_sections(review_text) # Identify gaps if requested gaps = [] if include_gaps: gaps = await self._identify_gaps(review_text, model) return { "review": review_text, "metadata": { "paper_count": len(papers), "word_count": word_count, "sections": sections, "research_gaps": gaps, "citations_added": self._count_citations(review_text) }, "cost": usage } def _prepare_papers_summary(self, papers: list[dict]) -> str: """Convert papers to structured summary for LLM""" summaries = [] for p in papers: summaries.append(f""" Title: {p['title']} Authors: {', '.join(p['authors'])} Year: {p['year']} Venue: {p.get('venue', 'N/A')} Citations: {p['citation_count']} BibTeX Key: {p['bibtex_key']} Abstract: {p['abstract']} --- """) return "\n".join(summaries) async def _identify_gaps(self, review: str, model: str) -> list[dict]: """Extract research gaps from review""" prompt = GAP_IDENTIFICATION_PROMPT if self.provider == "anthropic": response = self.client.messages.create( model=model, max_tokens=2048, messages=[ {"role": "user", "content": review}, {"role": "assistant", "content": "I've written the review."}, {"role": "user", "content": prompt} ] ) gaps_json = response.content[0].text else: response = self.client.chat.completions.create( model=model, messages=[ {"role": "user", "content": review}, {"role": "assistant", "content": "I've written the review."}, {"role": "user", "content": prompt} ], max_tokens=2048 ) gaps_json = response.choices[0].message.content # Parse JSON response try: gaps = json.loads(gaps_json) return gaps except: return [] def _extract_sections(self, review: str) -> list[str]: """Extract section headings from markdown""" import re headings = re.findall(r'^#{1,3}\s+(.+)$', review, re.MULTILINE) return headings def _count_citations(self, review: str) -> int: """Count citation references in text""" import re citations = re.findall(r'\[@\w+\d+\]|\[.+?\s+et al\.,\s+\d{4}\]', review) return len(citations) def _calculate_cost_anthropic(self, input_tokens: int, output_tokens: int, model: str) -> float: """Calculate cost for Anthropic models""" prices = { "claude-3-5-sonnet-20241022": {"input": 3.0, "output": 15.0}, # per 1M tokens "claude-3-opus-20240229": {"input": 15.0, "output": 75.0} } price = prices.get(model, prices["claude-3-5-sonnet-20241022"]) cost = (input_tokens * price["input"] + output_tokens * price["output"]) / 1_000_000 return round(cost, 4) def _calculate_cost_openai(self, input_tokens: int, output_tokens: int, model: str) -> float: """Calculate cost for OpenAI models""" prices = { "gpt-4-turbo-preview": {"input": 10.0, "output": 30.0}, "gpt-4": {"input": 30.0, "output": 60.0} } price = prices.get(model, prices["gpt-4-turbo-preview"]) cost = (input_tokens * price["input"] + output_tokens * price["output"]) / 1_000_000 return round(cost, 4) ``` ### 2.6 MCP Tool Implementation ```python # In server.py from polyhedra.services.literature_review_service import LiteratureReviewService # Initialize service _review_service: LiteratureReviewService = None def _init_review_service(): global _review_service import os provider = os.getenv("POLYHEDRA_LLM_PROVIDER", "anthropic") api_key = os.getenv("ANTHROPIC_API_KEY") if provider == "anthropic" else os.getenv("OPENAI_API_KEY") _review_service = LiteratureReviewService( llm_provider=provider, api_key=api_key ) _init_review_service() @server.list_tools() async def list_tools() -> list[Tool]: return [ # ... existing tools ... Tool( name="generate_literature_review", description="Generate a structured literature review from papers using AI synthesis", inputSchema={ "type": "object", "properties": { "papers_file": { "type": "string", "default": "literature/papers.json", "description": "Path to papers JSON file" }, "focus": { "type": "string", "description": "Optional focus area for the review" }, "structure": { "type": "string", "enum": ["thematic", "chronological", "methodological"], "default": "thematic" }, "depth": { "type": "string", "enum": ["brief", "standard", "comprehensive"], "default": "standard" }, "include_gaps": { "type": "boolean", "default": True }, "output_path": { "type": "string", "default": "literature/review.md" }, "llm_model": { "type": "string", "description": "LLM model to use" } } } ) ] @server.call_tool() async def call_tool(name: str, arguments: dict) -> list[TextContent]: # ... existing tools ... elif name == "generate_literature_review": # Load papers papers_file = _project_root / arguments.get("papers_file", "literature/papers.json") if not papers_file.exists(): return [TextContent( type="text", text=json.dumps({ "error": "Papers file not found. Run search_papers first." }) )] papers = json.loads(papers_file.read_text()) # Generate review result = await _review_service.generate_review( papers=papers, focus=arguments.get("focus"), structure=arguments.get("structure", "thematic"), depth=arguments.get("depth", "standard"), include_gaps=arguments.get("include_gaps", True), model=arguments.get("llm_model") ) # Save to file output_path = _project_root / arguments.get("output_path", "literature/review.md") output_path.parent.mkdir(parents=True, exist_ok=True) output_path.write_text(result["review"]) # Add citations to references.bib citations_added = 0 for paper in papers: key, added = _citation_manager.add_entry(paper["bibtex_entry"]) if added: citations_added += 1 return [TextContent( type="text", text=json.dumps({ "success": True, "saved_to": str(output_path), "metadata": result["metadata"], "cost": result["cost"], "citations_added": citations_added }, indent=2) )] ``` ### 2.7 Configuration **Environment Variables**: ```bash # .env file POLYHEDRA_LLM_PROVIDER=anthropic # or "openai" ANTHROPIC_API_KEY=sk-ant-... OPENAI_API_KEY=sk-... ``` **IDE Configuration**: ```json // .cursor/mcp.json or .vscode/settings.json { "mcpServers": { "polyhedra": { "command": "python", "args": ["-m", "polyhedra.server"], "env": { "POLYHEDRA_LLM_PROVIDER": "anthropic", "ANTHROPIC_API_KEY": "sk-ant-..." } } } } ``` --- ## 3. Updated System Architecture ### 3.1 Layered Architecture (v2.1) ``` β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ MCP Tool Layer β”‚ β”‚ Pure Tools + Agent Tool β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ Service Layer β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚ β”‚ Papers β”‚ β”‚ Context β”‚ β”‚ Citationsβ”‚ β”‚ β”‚ β”‚ Service β”‚ β”‚ Service β”‚ β”‚ Service β”‚ β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚ β”‚ RAG β”‚ β”‚ πŸ†• Literature β”‚ β”‚ β”‚ β”‚ Service β”‚ β”‚ Review Service β”‚ β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ External Services β”‚ β”‚ β€’ Semantic Scholar API β”‚ β”‚ β€’ πŸ†• Anthropic/OpenAI API β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ ``` ### 3.2 Updated Directory Structure ``` polyhedra/ β”œβ”€β”€ pyproject.toml β”œβ”€β”€ README.md β”œβ”€β”€ .env.example # πŸ†• LLM API keys β”œβ”€β”€ src/ β”‚ └── polyhedra/ β”‚ β”œβ”€β”€ __init__.py β”‚ β”œβ”€β”€ server.py β”‚ β”œβ”€β”€ tools/ β”‚ β”‚ β”œβ”€β”€ __init__.py β”‚ β”‚ β”œβ”€β”€ papers.py β”‚ β”‚ β”œβ”€β”€ context.py β”‚ β”‚ β”œβ”€β”€ retrieval.py β”‚ β”‚ β”œβ”€β”€ citations.py β”‚ β”‚ β”œβ”€β”€ files.py β”‚ β”‚ └── review.py # πŸ†• Literature review tool β”‚ β”œβ”€β”€ services/ β”‚ β”‚ β”œβ”€β”€ __init__.py β”‚ β”‚ β”œβ”€β”€ semantic_scholar.py β”‚ β”‚ β”œβ”€β”€ context_manager.py β”‚ β”‚ β”œβ”€β”€ citation_manager.py β”‚ β”‚ β”œβ”€β”€ rag_service.py β”‚ β”‚ └── literature_review_service.py # πŸ†• β”‚ β”œβ”€β”€ schemas/ β”‚ β”‚ β”œβ”€β”€ __init__.py β”‚ β”‚ β”œβ”€β”€ paper.py β”‚ β”‚ └── project.py β”‚ └── prompts/ # πŸ†• Prompt templates β”‚ β”œβ”€β”€ __init__.py β”‚ β”œβ”€β”€ literature_review.py β”‚ └── gap_analysis.py └── tests/ β”œβ”€β”€ test_tools/ β”‚ └── test_review.py # πŸ†• └── test_services/ └── test_literature_review_service.py # πŸ†• ``` --- ## 4. Updated Functional Requirements ### FR-006: LiteratureAgent - Autonomous Literature Review (NEW) πŸ†• **Priority**: P1 (High - Week 4) **Status**: Not Started **Agent**: Phase 1 of Research Pipeline **Description** Autonomous agent that searches academic literature, synthesizes findings, and identifies research gaps. First agent in the multi-agent research pipeline. **User Story** "As a researcher starting a new project, I want an AI agent to read my research direction, search relevant papers, organize them by themes, and generate a well-structured literature review with proper citations, so I can quickly understand the state of the art." **Capabilities** 1. **Intent understanding**: Parse natural language research topics 2. **Multi-query search**: Generate and execute 3-5 complementary search queries 3. **Paper selection**: Rank papers by relevance, select top 10-20 4. **Theme clustering**: Group papers into coherent themes (Methods, Applications, Gaps) 5. **Synthesis generation**: Write 500-1500 word literature review 6. **Citation integration**: Properly cite all referenced papers in BibTeX format 7. **Gap identification**: Highlight under-explored areas and suggest research directions **Triggers** - Direct command: `@literature-review "multimodal transformers for vision-language"` - Pipeline mode: First phase in `@full-research` **Input Parameters** ```python { "research_topic": str, # Natural language description "max_papers": int = 20, # Papers to include "depth": str = "standard" # "brief" (500w), "standard" (1000w), "comprehensive" (2000w) } ``` **Output Artifacts** (consumed by HypothesisAgent) ```json { "review_text": "markdown content with citations", "papers": [ {"paper_id": "arxiv:2203.12345", "relevance_score": 0.95, "theme": "Architecture"} ], "themes": ["Architectures", "Pre-training Methods", "Benchmarks"], "identified_gaps": [ "Few studies on low-resource languages", "Limited benchmarks for video understanding" ], "suggested_directions": [ "Investigate synthetic data for low-resource VL", "Extend temporal modeling for video tasks" ], "metadata": { "paper_count": 18, "citation_count": 18, "generation_time_sec": 125, "cost_usd": 0.25 } } ``` **Checkpoint System** ``` User: @literature-review "multimodal transformers for vision-language" Agent: [Step 1/4] Searching papers... βœ“ Generated 4 search queries βœ“ Found 45 papers on Semantic Scholar β†’ Top 20 selected by relevance [Checkpoint] Do you want to: 1. Continue with these 20 papers βœ“ 2. Add more search queries 3. Manually select specific papers User: 1 [Step 2/4] Clustering papers into themes... βœ“ Identified 4 themes: Architectures (8), Pre-training (5), Benchmarks (4), Applications (3) [Step 3/4] Generating literature review (est. 2-3 min)... βœ“ Generated 1200-word review βœ“ Added 18 citations to references.bib [Step 4/4] Identifying research gaps... βœ“ Found 3 under-explored areas [Output] Saved to: literature/multimodal_transformers_review.md Next: Run @hypothesize to generate research hypotheses from these gaps ``` **Cost & Performance** - LLM calls: 3-5 (synthesis, clustering, gap analysis) - Total tokens: ~15K input + 5K output - Cost: $0.15-0.30 per review - Time: 3-5 minutes **Dependencies** - Semantic Scholar API (FR-001) - Citation manager (FR-002) - LLM client (Anthropic/OpenAI) **Edge Cases** - Handle papers without abstracts (use title + introduction text) - Handle duplicate papers (deduplicate by DOI/arxiv ID) - Handle API rate limits (exponential backoff) - Handle < 5 papers found (adjust depth, warn user) --- ### FR-007: HypothesisAgent - Research Hypothesis Generation (NEW) πŸ†• **Priority**: P1 (High - Week 5) **Status**: Not Started **Agent**: Phase 2 of Research Pipeline **Description** Generates testable research hypotheses based on literature gaps. Performs novelty checking and feasibility analysis. **User Story** "Given the gaps I've identified in existing work, I want an AI agent to propose 3-5 testable hypotheses with novelty scores and feasibility assessments, so I can select the most promising research direction." **Capabilities** 1. **Gap analysis**: Process identified gaps from LiteratureAgent 2. **Hypothesis generation**: Propose 3-5 testable hypotheses per gap 3. **Novelty checking**: Search for similar existing work, compute novelty score 4. **Feasibility assessment**: Evaluate required resources (data, compute, time) 5. **Impact prediction**: Estimate potential contribution to the field 6. **Interactive selection**: Help user choose best hypothesis **Triggers** - Direct command: `@hypothesize` - Pipeline mode: Second phase in `@full-research` (after LiteratureAgent) **Input** (from LiteratureAgent) ```json { "literature_review": {...}, "identified_gaps": ["Gap 1", "Gap 2", "Gap 3"], "papers": [...] } ``` **Output Artifacts** (consumed by MethodAgent) ```json { "hypotheses": [ { "id": "H1", "statement": "Fine-tuning with synthetic data improves low-resource VL tasks", "novelty_score": 0.85, "novelty_rationale": "Only 2 papers explore synthetic data for VL, none for low-resource", "feasibility": "High", "required_resources": { "data": "Public datasets + Stable Diffusion API", "compute": "2x A100 GPUs", "time_weeks": 3 }, "expected_impact": "Address major gap in low-resource languages", "risks": ["Synthetic data distribution mismatch"] } ], "selected_hypothesis": {...}, "selection_rationale": "Selected H1 because highest novelty (0.85) + feasible timeline (3 weeks)" } ``` **Checkpoint System** ``` Agent: [Step 1/3] Analyzing 3 identified gaps... βœ“ Gap 1: Low-resource languages (8 relevant papers) βœ“ Gap 2: Video understanding (12 relevant papers) βœ“ Gap 3: Efficiency (15 relevant papers) [Step 2/3] Generating hypotheses (est. 5 min)... βœ“ Generated 5 hypotheses βœ“ Novelty scores computed (range: 0.65-0.92) [Checkpoint] Top 3 hypotheses: 1. [Novelty: 0.92, Feasibility: High] Synthetic data for low-resource VL 2. [Novelty: 0.85, Feasibility: Medium] Temporal attention for video 3. [Novelty: 0.78, Feasibility: High] Efficient cross-modal fusion Which hypothesis should we develop? (1-3, or 'all' to compare): User: 1 [Step 3/3] Assessing feasibility... βœ“ Feasibility: High βœ“ Resources: 2 GPUs, 3 weeks, $500 compute βœ“ Expected impact: High (addresses major gap in low-resource languages) βœ“ Risks identified: 1 (synthetic data distribution mismatch) Next: Run @design-method to create experimental setup ``` **Cost & Performance** - LLM calls: 5-8 (generation, novelty checking, feasibility analysis) - Total tokens: ~20K input + 8K output - Cost: $0.20-0.50 per run - Time: 5-10 minutes **Dependencies** - LiteratureAgent output (FR-006) - Semantic Scholar API (for novelty checking) - LLM client **Edge Cases** - Handle no clear gaps (ask user for manual direction) - Handle all hypotheses low novelty (< 0.5) - warn and suggest broader search - Handle infeasible resources (> user's compute budget) - suggest scaled-down version --- ### FR-008: MethodAgent - Experimental Method Design (NEW) πŸ†• **Priority**: P1 (High - Week 5-6) **Status**: Not Started **Agent**: Phase 3 of Research Pipeline **Description** Designs detailed experimental methods to test hypotheses. Selects baselines, metrics, datasets, and creates implementation plan. **User Story** "Given a research hypothesis, I want an AI agent to design a complete experimental setup with baselines, metrics, datasets, and implementation plan, so I can start experiments quickly." **Capabilities** 1. **Method design**: Propose experimental approach 2. **Baseline selection**: Identify 3-5 relevant baselines from literature 3. **Metric definition**: Choose appropriate evaluation metrics 4. **Dataset recommendation**: Suggest suitable datasets (public/synthetic) 5. **Resource estimation**: Compute hours, time, budget 6. **Risk assessment**: Identify potential failure points and mitigations **Triggers** - Direct command: `@design-method` - Pipeline mode: Third phase in `@full-research` (after HypothesisAgent) **Input** (from HypothesisAgent) ```json { "selected_hypothesis": {...}, "feasibility_assessment": {...}, "required_resources": {...} } ``` **Output Artifacts** (consumed by ExperimentAgent) ```json { "method_description": "markdown text (500-1000 words)", "baselines": [ {"name": "CLIP", "paper_id": "...", "reason": "Standard baseline", "code_url": "..."}, {"name": "ALIGN", "paper_id": "...", "reason": "SOTA comparison", "code_url": "..."} ], "metrics": [ {"name": "Accuracy", "justification": "Standard for classification"}, {"name": "F1", "justification": "Handle class imbalance"} ], "datasets": [ {"name": "COCO", "split": "val", "size": "5K images"}, {"name": "Flickr30K", "split": "test", "size": "1K images"} ], "implementation_plan": { "steps": [ "1. Prepare data (Week 1)", "2. Implement model (Week 1-2)", "3. Train baselines + ours (Week 2-3)", "4. Evaluate (Week 3)" ], "estimated_time_weeks": 3, "compute_resources": "2x A100 GPUs", "estimated_cost_usd": 500 }, "risks": [ { "risk": "Data leakage risk", "mitigation": "Use strict train/test split, no test set touching during dev" } ] } ``` **Checkpoint System** ``` Agent: [Step 1/4] Analyzing hypothesis... βœ“ Hypothesis: Synthetic data for low-resource VL βœ“ Task type: Vision-Language Classification [Step 2/4] Searching for baselines (est. 3 min)... βœ“ Found 12 relevant papers with code βœ“ Selected 4 baselines: CLIP, ALIGN, LiT, FLAVA [Checkpoint] Proposed method: 1. Generate synthetic image-text pairs using Stable Diffusion + LLM 2. Fine-tune CLIP on synthetic + real data (mix ratio 50:50) 3. Evaluate on 3 low-resource language benchmarks Baselines: - CLIP (zero-shot) - lower bound - CLIP fine-tuned on real data only - ablation - ALIGN - larger model comparison Datasets: - XM3600 (35 languages) - main benchmark - COCO-CN (Chinese) - language-specific - Multi30K-Hindi (Hindi) - language-specific Metrics: - Accuracy (primary) - F1 (handle class imbalance) - Inference time (efficiency) Does this setup make sense? (y/n/modify): User: y [Step 3/4] Estimating resources... βœ“ Compute: 2x A100 GPUs (24GB VRAM each) βœ“ Time: 3 weeks (1 week data gen, 2 weeks training) βœ“ Budget: ~$500 compute cost (Lambda Labs) [Step 4/4] Identifying risks... ⚠ Risk 1: Synthetic data quality may not match real data β†’ Mitigation: Use diverse prompts, human verification sampling ⚠ Risk 2: Overfitting to synthetic distribution β†’ Mitigation: Mix 50:50 with real data, monitor val loss Next: Run @setup-experiment to generate code scaffolding ``` **Cost & Performance** - LLM calls: 6-10 (method design, baseline search, risk analysis) - Total tokens: ~25K input + 10K output - Cost: $0.30-0.60 per run - Time: 10-15 minutes **Dependencies** - HypothesisAgent output (FR-007) - Semantic Scholar API (for baseline paper search) - LLM client **Edge Cases** - Handle no public datasets available (suggest synthetic data generation) - Handle baselines without code (suggest reimplementation) - Handle resource exceeds budget (suggest scaled-down method) --- ### FR-009: ExperimentAgent - Experiment Setup & Monitoring (NEW) πŸ†• **Priority**: P2 (Medium - Week 7) **Status**: Not Started **Agent**: Phase 4 of Research Pipeline **Description** Generates experiment code scaffolding, config files, and monitoring scripts. Does NOT run experiments (user's compute). **User Story** "Given an experimental design, I want an AI agent to generate training scripts, config files, and monitoring dashboards so I can start experiments immediately." **Capabilities** 1. **Code generation**: PyTorch/TensorFlow training scripts 2. **Config creation**: YAML configs for hyperparameters 3. **Data pipeline**: Dataset loaders and preprocessing code 4. **Monitoring setup**: Wandb/TensorBoard integration 5. **Baseline implementation**: Generate baseline comparison code 6. **Checkpointing**: Auto-save best models, resume training **Triggers** - Direct command: `@setup-experiment` - Pipeline mode: Fourth phase in `@full-research` (after MethodAgent) **Input** (from MethodAgent) ```json { "method_description": "...", "baselines": [...], "datasets": [...], "implementation_plan": {...} } ``` **Output Artifacts** (used by researcher + AnalysisAgent later) ```json { "code_files": { "train.py": "# PyTorch training script (250 lines)", "model.py": "# Model architecture (180 lines)", "data.py": "# Dataset loaders (120 lines)", "config.yaml": "# Hyperparameters", "README.md": "# Setup and run instructions" }, "setup_instructions": "markdown text", "monitoring": { "wandb_project": "synthetic-vlm", "dashboard_url": "https://wandb.ai/user/synthetic-vlm", "tracked_metrics": ["train_loss", "val_acc", "val_f1"] }, "expected_outputs": { "checkpoints": "checkpoints/*.pth", "logs": "logs/*.json", "results": "results.csv" } } ``` **Checkpoint System** ``` Agent: [Step 1/3] Generating code scaffolding... βœ“ Created train.py (250 lines) - main training loop βœ“ Created model.py (180 lines) - CLIP + custom head βœ“ Created data.py (120 lines) - XM3600, COCO-CN, Multi30K loaders βœ“ Created config.yaml - hyperparameters (LR, batch size, epochs) [Step 2/3] Setting up monitoring... βœ“ Wandb integration added βœ“ Metrics: train_loss, val_acc, val_f1, inference_time βœ“ Checkpointing: save top-3 models by val_acc [Checkpoint] Generated files: - experiments/synthetic_vlm/train.py - experiments/synthetic_vlm/model.py - experiments/synthetic_vlm/data.py - experiments/synthetic_vlm/config.yaml - experiments/synthetic_vlm/README.md Ready to review code? (y/n): User: y [Agent shows code preview in IDE] [Step 3/3] Creating monitoring dashboard... βœ“ Wandb project created: synthetic-vlm βœ“ Dashboard URL: https://wandb.ai/user/synthetic-vlm [Instructions] To start training: cd experiments/synthetic_vlm pip install -r requirements.txt python train.py --config config.yaml Expected runtime: 2 weeks Expected outputs: results.csv, checkpoints/, logs/ Next: After training completes, run @analyze-results experiments/synthetic_vlm/results.csv ``` **Cost & Performance** - LLM calls: 4-6 (code generation, config creation) - Total tokens: ~15K input + 12K output (code is verbose) - Cost: $0.20-0.40 per run - Time: 5-8 minutes (setup only, NOT training time) **Dependencies** - MethodAgent output (FR-008) - LLM client (for code generation) **Edge Cases** - Handle unsupported frameworks (only PyTorch/TensorFlow) - Handle custom architectures (generate template, ask user to fill) - Handle missing dependencies (auto-generate requirements.txt) **Note**: This agent does NOT run experiments. User runs experiments on their own compute. Agent only prepares code and monitoring infrastructure. --- ### FR-010: AnalysisAgent - Result Analysis & Visualization (NEW) πŸ†• **Priority**: P2 (Medium - Week 7-8) **Status**: Not Started **Agent**: Phase 5 of Research Pipeline **Description** Analyzes experimental results, generates publication-quality plots, performs statistical tests, and summarizes findings. **User Story** "After my experiments finish, I want an AI agent to parse result logs, generate publication-quality plots, run significance tests, and summarize findings in plain language." **Capabilities** 1. **Result parsing**: Read CSV/JSON/TensorBoard logs 2. **Plot generation**: Training curves, comparison bar charts, ablation studies 3. **Statistical tests**: T-tests, confidence intervals, significance testing 4. **Table generation**: LaTeX tables for paper 5. **Finding summarization**: Interpret results in plain language 6. **Failure diagnosis**: Identify why experiments failed (if applicable) **Triggers** - Direct command: `@analyze-results ` - Pipeline mode: Fifth phase in `@full-research` (after user completes experiments) **Input** (from ExperimentAgent + user's training runs) ```json { "log_files": ["results.csv", "logs/train.json"], "checkpoint_dir": "checkpoints/", "baseline_results": {...} } ``` **Output Artifacts** (consumed by WritingAgent) ```json { "summary": "Proposed method achieved 78.5% accuracy (+5.2% vs CLIP baseline, p<0.01)", "plots": [ "figures/training_curve.pdf", "figures/baseline_comparison.pdf", "figures/ablation_study.pdf" ], "tables": { "results_table.tex": "\\begin{table}...", "ablation_table.tex": "\\begin{table}..." }, "statistical_tests": { "ours_vs_clip": {"t_statistic": 3.45, "p_value": 0.008, "significant": true}, "ours_vs_align": {"t_statistic": 1.52, "p_value": 0.12, "significant": false} }, "insights": [ "Synthetic data most effective for low-resource languages (+7.2% on Hindi)", "Performance drops for fine-grained tasks (only +2.1% on COCO)", "Inference time comparable to CLIP baseline (45ms vs 43ms)" ], "recommendations": [ "Investigate why fine-grained tasks benefit less", "Try higher synthetic:real ratios for low-resource languages" ] } ``` **Checkpoint System** ``` User: @analyze-results experiments/synthetic_vlm/results.csv Agent: [Step 1/4] Parsing results... βœ“ Found 3 runs: Ours, CLIP, ALIGN βœ“ Metrics: Accuracy, F1, Inference Time βœ“ Datasets: XM3600 (35 languages), COCO-CN, Multi30K-Hindi [Step 2/4] Generating plots (est. 2 min)... βœ“ Training curve: figures/training_curve.pdf βœ“ Baseline comparison: figures/comparison.pdf βœ“ Per-language breakdown: figures/per_language.pdf [Step 3/4] Running statistical tests... βœ“ Ours vs CLIP: +5.2% (t=3.45, p=0.008, **significant**) βœ“ Ours vs ALIGN: +1.8% (t=1.52, p=0.12, not significant) [Checkpoint] Summary: - Proposed method: 78.5% Β± 1.2% - CLIP baseline: 73.3% Β± 1.5% - ALIGN baseline: 76.7% Β± 1.8% - Statistical significance: Yes (vs CLIP), No (vs ALIGN) Generate LaTeX tables? (y/n): User: y [Step 4/4] Generating LaTeX tables... βœ“ Main results table: tables/results.tex βœ“ Ablation study table: tables/ablation.tex Key insights: 1. Synthetic data provides +5.2% improvement over CLIP 2. Most effective for low-resource languages (+7.2% on Hindi) 3. Performance comparable to larger model ALIGN (76.7% vs 78.5%) 4. Inference time unchanged (45ms vs 43ms) Recommendations: - Investigate why fine-grained tasks benefit less - Try higher synthetic:real ratios for low-resource languages Next: Run @write-paper method to draft the methodology section ``` **Cost & Performance** - LLM calls: 3-5 (interpretation, insight generation) - Total tokens: ~10K input + 5K output - Cost: $0.15-0.30 per run - Time: 5-10 minutes **Dependencies** - ExperimentAgent output (FR-009) - Matplotlib/Seaborn (for plotting) - SciPy (for statistical tests) - LLM client (for insight generation) **Edge Cases** - Handle missing baselines (skip comparison, warn user) - Handle failed experiments (diagnose failure, suggest fixes) - Handle incomplete logs (work with partial data) --- ### FR-011: WritingAgent - Academic Paper Drafting (NEW) πŸ†• **Priority**: P1 (High - Week 6) **Status**: Not Started **Agent**: Phase 6 of Research Pipeline **Description** Drafts paper sections with proper academic style, citations, and LaTeX formatting. **User Story** "Given my experimental results and literature review, I want an AI agent to draft paper sections (introduction, related work, method, results) in academic style with proper citations." **Capabilities** 1. **Section drafting**: Introduction, Related Work, Method, Results, Discussion, Conclusion 2. **Citation integration**: Automatically cite papers from references.bib 3. **Academic style**: Formal language, appropriate hedging, clear argumentation 4. **Figure references**: Cross-reference generated plots and tables 5. **Logical flow**: Ensure coherent narrative across sections 6. **LaTeX formatting**: Output ready for Overleaf/LaTeX compilation **Triggers** - Direct command: `@write-paper
` (e.g., `@write-paper introduction`) - Pipeline mode: Sixth phase in `@full-research` (after AnalysisAgent) **Input** (from previous agents) ```json { "literature_review": {...}, # From LiteratureAgent "hypothesis": {...}, # From HypothesisAgent "method": {...}, # From MethodAgent "results": {...}, # From AnalysisAgent "references_bib": "path/to/references.bib" } ``` **Output Artifacts** (consumed by ReviewAgent) ```json { "sections": { "introduction.tex": "\\section{Introduction}...", "related_work.tex": "\\section{Related Work}...", "method.tex": "\\section{Method}...", "results.tex": "\\section{Results}...", "discussion.tex": "\\section{Discussion}...", "conclusion.tex": "\\section{Conclusion}..." }, "word_counts": { "introduction": 650, "related_work": 1200, "method": 1800, "results": 900, "discussion": 600, "conclusion": 300 }, "citation_count": 24, "figure_references": ["fig:training_curve", "fig:comparison", "tab:results"] } ``` **Checkpoint System** ``` User: @write-paper introduction Agent: [Step 1/3] Analyzing context... βœ“ Literature review: 18 papers βœ“ Research gap: Low-resource VL βœ“ Hypothesis: Synthetic data improves performance βœ“ Key result: +5.2% improvement (p<0.01) [Step 2/3] Drafting introduction (est. 3 min)... βœ“ Generated 650 words βœ“ Added 8 citations βœ“ Structure: Context β†’ Gap β†’ Contribution β†’ Outline [Checkpoint] Preview: \section{Introduction} Vision-language models have achieved remarkable success on high-resource languages \cite{radford2021learning}, but performance degrades significantly for low-resource languages due to limited training data \cite{aggarwal2023multilingual}... [... 650 words total ...] The main contributions of this work are: \begin{itemize} \item A method to generate high-quality synthetic training data for low-resource vision-language tasks \item Empirical validation showing 5.2\% accuracy improvement over CLIP baseline (p<0.01) \item Analysis of effectiveness across 35 languages, with strongest gains on low-resource languages \end{itemize} Accept this draft? (y/n/revise): User: y [Step 3/3] Saving... βœ“ Saved to: paper/introduction.tex βœ“ Word count: 650 βœ“ Citations: 8 (all in references.bib) Next sections to draft: - @write-paper related-work (est. 5 min, 1200 words) - @write-paper method (est. 4 min, 1800 words) - @write-paper results (est. 3 min, 900 words) Or: @write-paper all (draft all sections, est. 15 min) ``` **Cost & Performance per Section** - Introduction: $0.10-0.15 (650 words, 3-5 min) - Related Work: $0.15-0.25 (1200 words, 5-7 min) - Method: $0.20-0.30 (1800 words, 4-6 min) - Results: $0.10-0.15 (900 words, 3-5 min) - Discussion: $0.10-0.15 (600 words, 3-4 min) - Conclusion: $0.05-0.10 (300 words, 2-3 min) - **Total for full paper**: $0.70-1.10, 20-30 minutes **Dependencies** - LiteratureAgent output (FR-006) - HypothesisAgent output (FR-007) - MethodAgent output (FR-008) - AnalysisAgent output (FR-010) - references.bib (FR-002) - LLM client **Edge Cases** - Handle missing citations (add placeholder, warn user) - Handle overly long sections (exceed word limit) - auto-trim or warn - Handle inconsistent results (method says X, results show Y) - flag for review --- ### FR-012: ReviewAgent - Quality Check & Compliance (NEW) πŸ†• **Priority**: P2 (Medium - Week 6) **Status**: Not Started **Agent**: Phase 7 of Research Pipeline (Final) **Description** Reviews drafted paper for quality, citation completeness, and conference guideline compliance. **User Story** "Before submitting my paper, I want an AI agent to check for missing citations, style issues, guideline compliance, and suggest improvements." **Capabilities** 1. **Citation check**: Ensure all claims have proper citations 2. **Style review**: Check academic writing quality (clarity, conciseness) 3. **Guideline compliance**: Verify page limit, formatting, required sections 4. **Consistency check**: Cross-reference method description and results 5. **Improvement suggestions**: Highlight weak arguments or unclear statements 6. **Final checklist**: Submission readiness score (0-10) **Triggers** - Direct command: `@review-paper` - Pipeline mode: Seventh phase in `@full-research` (after WritingAgent) **Input** (from WritingAgent) ```json { "paper_sections": {...}, "figures": ["figures/..."], "tables": ["tables/..."], "references_bib": "path/to/references.bib", "venue": "NeurIPS" # Optional: check venue-specific guidelines } ``` **Output** (final artifacts) ```json { "review_report": { "overall_score": 8.5, "readiness": "90%", "sections_reviewed": 6, "issues_found": 5, "issues_fixed": 2 }, "issues": [ { "severity": "high", "location": "introduction.tex:45", "issue": "Claim without citation: 'multimodal models are widely used'", "suggestion": "Add citation to support this claim (e.g., \\cite{radford2021learning})" }, { "severity": "medium", "location": "method.tex:120", "issue": "Passive voice overuse (65% of sentences)", "suggestion": "Rewrite some sentences in active voice for clarity" }, { "severity": "low", "location": "results.tex:80", "issue": "Repetition: 'propose' used 8 times", "suggestion": "Use synonyms: suggest, introduce, present" } ], "guideline_compliance": { "page_limit": {"status": "pass", "actual": 8, "limit": 8}, "sections": {"status": "pass", "missing": []}, "citations": {"status": "pass", "count": 24, "formatted": true}, "figures": {"status": "pass", "format": "PDF", "resolution": "300dpi"} }, "suggested_revisions": "markdown text with actionable items" } ``` **Checkpoint System** ``` User: @review-paper Agent: [Step 1/5] Checking citations (est. 3 min)... βœ“ Scanned 6 sections (4500 words) βœ“ Found 24 citations ⚠ Found 3 claims without citations: 1. introduction.tex:45 - "multimodal models are widely used" 2. method.tex:230 - "synthetic data is effective" 3. results.tex:180 - "our approach is efficient" [Step 2/5] Checking style... βœ“ Academic tone: Good βœ“ Clarity: Good (Flesch reading ease: 45) ⚠ Passive voice: 65% (acceptable for methods, but high overall) ⚠ Repetition: "propose" used 8 times (suggest synonyms) [Step 3/5] Checking guideline compliance (NeurIPS)... βœ“ Page limit: 8/8 pages (perfect) βœ“ Sections: All required sections present βœ“ Figure format: All PDFs, 300dpi βœ“ References: BibTeX formatted correctly βœ“ Anonymization: No author names in text [Step 4/5] Checking consistency... βœ“ Method matches results (all baselines evaluated) βœ“ Figures referenced in text (3/3) ⚠ Table 2 mentioned in results.tex but not found in tables/ [Step 5/5] Generating improvement suggestions... [Checkpoint] Review Summary: Overall Score: 8.5/10 (Ready with minor revisions) Submission Readiness: 90% High Priority Issues (3): 1. Add citation for claim in introduction.tex:45 2. Add citation for claim in method.tex:230 3. Add citation for claim in results.tex:180 Medium Priority Issues (2): 1. Reduce repetition of "propose" (8 occurrences) 2. Add missing Table 2 or remove reference Low Priority Issues: - Consider rewriting some passive voice sentences Guideline Compliance: βœ“ All checks passed (NeurIPS 2024) Apply automatic fixes? (y/n/manual): User: y [Applying fixes...] βœ“ Added citation placeholders for 3 uncited claims βœ“ Replaced "propose" with synonyms where appropriate βœ“ Removed Table 2 reference (table doesn't exist) Updated score: 9.2/10 Submission readiness: 98% Remaining manual tasks: - Fill in citation placeholders with specific papers - Final proofread for typos - Upload to conference submission system Paper ready for submission! πŸŽ‰ ``` **Cost & Performance** - LLM calls: 5-8 (citation check, style review, suggestions) - Total tokens: ~20K input + 8K output - Cost: $0.20-0.40 per review - Time: 5-8 minutes **Dependencies** - WritingAgent output (FR-011) - references.bib (FR-002) - LaTeX parsing library (for citation extraction) - LLM client (for style and improvement suggestions) **Edge Cases** - Handle non-LaTeX formats (warn: only LaTeX supported) - Handle unknown venue (skip venue-specific checks) - Handle very long papers (> 20 pages) - may timeout, suggest section-by-section review --- ## 5. Updated Development Tasks ### Phase 1: Core Services (Week 1) ```yaml # Same as v2.0 tasks: - T-001 to T-005 (unchanged) ``` ### Phase 2: MCP Server + Basic Tools (Week 2) ```yaml # Same as v2.0 tasks: - T-006 to T-008 (unchanged) ``` ### Phase 3: Literature Review Feature (Week 3) ```yaml tasks: - id: T-012 name: Literature review service description: | - LLM client initialization (Anthropic/OpenAI) - Prompt templates - Review generation logic - Gap identification - Cost calculation estimate: 6h depends_on: [T-002, T-003] - id: T-013 name: MCP tool integration description: | - Add generate_literature_review tool - Environment variable handling - Error handling for API failures - Token limit management estimate: 3h depends_on: [T-012, T-006] - id: T-014 name: Prompt engineering description: | - Test prompts with sample papers - Optimize for quality and coherence - Handle edge cases (few papers, missing abstracts) - Validate citation format estimate: 4h depends_on: [T-012] - id: T-015 name: Integration testing description: | - End-to-end test: search β†’ review - Test with different paper counts (5, 20, 50, 100) - Test different structures (thematic, chronological) - Validate output quality manually estimate: 3h depends_on: [T-013, T-014] ``` ### Phase 4: Multi-Agent System - Foundation (Week 4) πŸ†• ```yaml tasks: - id: T-019 name: Agent orchestration framework description: | - PolyhedraResearchOrchestrator (meta-agent) - ResearchState (lifecycle tracking) - Intent parsing with LLM - Agent registry and routing - Checkpoint system - Pipeline execution engine estimate: 12h depends_on: [T-006] - id: T-020 name: BaseAgent class description: | - Abstract base class for all agents - Common capabilities (LLM access, tools, state) - Workflow execution framework - Error recovery patterns - Artifact management estimate: 6h depends_on: [T-019] - id: T-021 name: LiteratureAgent (Phase 1) description: | - Search and index papers - Generate literature reviews - Identify research gaps - Paper comparison - Citation finding estimate: 8h depends_on: [T-020, T-012] - id: T-022 name: IDE integration description: | - Cursor custom agent configuration - Command triggers (@research, @hypothesize, etc.) - Progress display in IDE - Checkpoint UI interactions estimate: 4h depends_on: [T-021] ``` ### Phase 5: Multi-Agent System - Core Agents (Weeks 5-6) πŸ†• ```yaml tasks: - id: T-024 name: HypothesisAgent (Phase 2) description: | - Generate hypotheses from gaps - Novelty checking against literature - Feasibility analysis - Ranking by impact Γ— feasibility - Interactive hypothesis selection estimate: 10h depends_on: [T-020] - id: T-025 name: MethodAgent (Phase 3) description: | - Experimental design generation - Baseline selection - Dataset recommendation - Metrics definition - Resource estimation (GPU hours, cost) - Code scaffolding generation estimate: 12h depends_on: [T-020] - id: T-026 name: WritingAgent (Phase 6) description: | - Paper section drafting (abstract, intro, method, etc.) - Citation integration from references.bib - Figure/table caption generation - LaTeX formatting - Section coherence checking estimate: 10h depends_on: [T-020, T-021] - id: T-027 name: ReviewAgent (Phase 7) description: | - Citation completeness check - Figure/table reference validation - Grammar and clarity check - Novelty claim verification - Venue guideline compliance estimate: 8h depends_on: [T-026] ``` ### Phase 6: Advanced Agents (Weeks 7-8) πŸ†• ```yaml tasks: - id: T-028 name: ExperimentAgent (Phase 4) description: | - Training code generation from methodology - Experiment setup automation - Progress monitoring integration (Wandb) - Checkpoint management - Failure detection and recovery - NOTE: Does not run compute, only setup/monitor estimate: 12h depends_on: [T-025] - id: T-029 name: AnalysisAgent (Phase 5) description: | - Parse experiment logs and results - Generate plots (matplotlib/seaborn) - Statistical significance testing - Baseline comparison - Insight generation - LaTeX table generation estimate: 10h depends_on: [T-028] - id: T-030 name: Full pipeline integration description: | - End-to-end pipeline testing - Agent handoff validation - State persistence across phases - Pipeline resume after interruption - @full-research command estimate: 8h depends_on: [T-024, T-025, T-026, T-027, T-028, T-029] ``` ### Phase 7: Polish & Documentation (Week 9) ```yaml tasks: - id: T-016 name: Documentation description: | - Update README with literature review feature - API key setup instructions - Example workflows - Cost estimation guide estimate: 2h depends_on: [T-015] - id: T-017 name: Configuration templates description: | - .env.example file - IDE config with LLM keys - Troubleshooting guide estimate: 2h depends_on: [T-016] - id: T-018 name: Quality assurance description: | - Manual review of 5 generated reviews - Check citation accuracy - Verify gap identification quality - Cost benchmarking estimate: 3h depends_on: [T-015] ``` **Total Development Timeline**: - Mode 1 only (Pure MCP): 3 weeks - Mode 1 + LiteratureAgent: 4 weeks - **Full Multi-Agent System: 9 weeks** πŸ†• **Incremental Delivery**: - Week 4: LiteratureAgent (usable for literature review) - Week 6: + HypothesisAgent + MethodAgent + WritingAgent (70% functionality) - Week 8: + ExperimentAgent + AnalysisAgent + ReviewAgent (100% functionality) - Week 9: Polish & documentation --- ## 6. Cost Analysis ### Literature Review Generation Costs **Assumptions**: - Average paper: 300 words (title + abstract) - 50 papers = 15,000 words β‰ˆ 20,000 tokens - Review output: 3,000-5,000 words β‰ˆ 6,000 tokens **Cost per Review** (Claude 3.5 Sonnet): - Input: 20,000 tokens Γ— $3/1M = $0.06 - Output: 6,000 tokens Γ— $15/1M = $0.09 - **Total: ~$0.15 per review** **Cost per Review** (GPT-4 Turbo): - Input: 20,000 tokens Γ— $10/1M = $0.20 - Output: 6,000 tokens Γ— $30/1M = $0.18 - **Total: ~$0.38 per review** **Recommendation**: Default to Claude 3.5 Sonnet for cost-effectiveness. --- ## 7. Success Metrics (Updated) ### Performance Metrics | Metric | Target | v2.0 | v2.1 | |--------|--------|------|------| | Paper Search | < 2 seconds | βœ“ | βœ“ | | Local Operations | < 100ms | βœ“ | βœ“ | | RAG Indexing | < 5 sec / 100 papers | βœ“ | βœ“ | | RAG Query | < 500ms | βœ“ | βœ“ | | **πŸ†• Literature Review** | **< 60 seconds** | - | **βœ“** | ### Quality Metrics (NEW) | Metric | Target | Measurement | |--------|--------|-------------| | Review Coherence | 4/5 avg rating | Manual evaluation by researchers | | Citation Accuracy | 100% valid format | Automated validation | | Gap Relevance | 80% useful | Survey of users | | Cost per Review | < $0.20 | Actual API usage | ### Adoption Metrics | Metric | v2.0 Target | v2.1 Target | |--------|-------------|-------------| | Active Users (3 months) | 100+ | 150+ | | Papers Searched | 10,000+ | 15,000+ | | Projects Initialized | 50+ | 75+ | | **πŸ†• Reviews Generated** | - | **500+** | --- ## 8. Migration from v2.0 ### For Existing Users **What's New**: 1. One new tool: `generate_literature_review` 2. Requires LLM API key (Anthropic or OpenAI) 3. All existing tools work exactly the same **Migration Steps**: 1. Update Polyhedra: `pip install --upgrade polyhedra` 2. Add API key to environment: `export ANTHROPIC_API_KEY=...` 3. Update IDE config with env vars 4. (Optional) Try new tool: `generate_literature_review` **Backward Compatibility**: - βœ… All v2.0 tools unchanged - βœ… No breaking changes - βœ… New tool is optional (skipped if no API key) --- ## 9. Roadmap & Evolution Path ### v2.1 (Current) - Multi-Agent Research System **Timeline**: 9 weeks **Status**: In planning **Agents**: - βœ… LiteratureAgent (Phase 1: Literature Review) - βœ… HypothesisAgent (Phase 2: Hypothesis Generation) - βœ… MethodAgent (Phase 3: Method Design) - βœ… ExperimentAgent (Phase 4: Experiment Setup) - βœ… AnalysisAgent (Phase 5: Result Analysis) - βœ… WritingAgent (Phase 6: Paper Writing) - βœ… ReviewAgent (Phase 7: Review & Polish) **Capabilities**: - Complete research lifecycle coverage - Agent orchestration and handoffs - State management across phases - Checkpoint-based user control - End-to-end pipeline (@full-research) ### v2.2 (Q1 2026) - Enhanced Intelligence **Timeline**: +4 weeks **New Features**: - **Adaptive Agents**: Learn from user preferences - Remember which papers user finds relevant - Adapt writing style to match user's papers - Adjust checkpoint frequency based on user trust - **Multi-modal Support**: - Generate figures from data - OCR for reading paper figures - Code-to-diagram conversion - **Advanced Analysis**: - Citation network visualization - Research trend prediction - Conference acceptance probability estimation - **Collaboration**: - Team mode (shared state) - Multi-author paper writing - Reviewer feedback integration ### v2.3 (Q2 2026) - Production Ready **Timeline**: +3 weeks **Enhancements**: - **Robustness**: - 99% uptime for critical agents - Comprehensive error recovery - Data backup and versioning - **Performance**: - 2x faster literature review (parallel processing) - 50% cost reduction (prompt optimization) - Caching and incremental updates - **Integration**: - Overleaf sync - Zotero import/export - Wandb/TensorBoard integration - arXiv submission automation ### v3.0 (Q3 2026) - Full Research Assistant **Timeline**: +8 weeks **Major Upgrades**: - **Confidence-Driven Autonomy** (ResearchKit approach): - Self-assess output quality - Request review only when uncertain - Explain reasoning for all decisions - **User Profile Learning**: - Research interests and expertise - Writing style and preferences - Resource constraints (budget, timeline) - Publication strategy - **Proactive Assistance**: - Suggest research directions - Alert on relevant new papers - Recommend collaborators - Track paper deadlines - **Advanced Workflows**: - Multi-paper comparative studies - Meta-analysis generation - Literature survey papers - Replication studies ### Evolution Comparison | Version | Agents | Autonomy | Lifecycle Coverage | Development | |---------|--------|----------|-------------------|-------------| | v2.0 | 0 (Pure tools) | Manual | 20% (literature only) | 3 weeks | | v2.1 | 7 specialized | Supervised | 100% (full pipeline) | 9 weeks | | v2.2 | 7 + enhancements | Adaptive | 100% + collaboration | +4 weeks | | v2.3 | 7 + production | Reliable | 100% + integrations | +3 weeks | | v3.0 | 10+ (specialized) | Confident | 100% + proactive | +8 weeks | ### Research Lifecycle Coverage Evolution ``` v2.0: [β–ˆβ–ˆβ–ˆβ–ˆβ–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘] 20% Literature review only v2.1: [β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ] 100% Complete research pipeline: Literature β†’ Hypothesis β†’ Method β†’ Experiments β†’ Analysis β†’ Writing β†’ Review v2.2: [β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ] 100% + Collaboration + Team features, multi-modal support v2.3: [β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ] 100% + Integrations + External tools (Overleaf, Zotero, arXiv) v3.0: [β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ] 100% + Proactive + Confidence-driven, learning, suggestions ``` ### Comparison with ResearchKit v2.0 | Feature | ResearchKit v2.0 | Polyhedra v2.1 | Polyhedra v3.0 | |---------|------------------|----------------|----------------| | **Agents** | 6 agents | 7 agents | 10+ agents | | **Philosophy** | Collaborative | Supervised | Confident | | **Interaction** | Progressive clarification | Checkpoint-based | Adaptive | | **Learning** | User profiles | None yet | Full learning | | **Branching** | Built-in | Not yet | Planned v3.0 | | **Transparency** | Explainable decisions | Basic | Full reasoning | | **Timeline** | 8 months (complex) | 9 weeks (focused) | +6 months | **Strategic Positioning**: - **v2.1**: Faster to market, covers full lifecycle - **v3.0**: Matches ResearchKit's sophistication - **Approach**: Incremental delivery vs big-bang release --- ## 10. Open Questions 1. **Chunking Strategy**: How to handle >100 papers exceeding context window? - Option A: Summarize papers first, then synthesize summaries - Option B: Generate review in multiple passes (per category) - Decision: TBD based on testing 2. **Review Quality Validation**: How to ensure consistent quality? - Approach: Manual review of first 50 generated reviews - Collect user feedback via survey - Iterate on prompts based on feedback 3. **Cost Management**: Should we add a token budget limit? - Concern: Users accidentally generating expensive reviews - Proposal: Add `max_cost` parameter (default $1.00) - Warn user before generation if estimate exceeds budget 4. **Multi-language Support**: Should reviews support non-English papers? - Current: English only - Future: Detect paper language, generate review in same language --- ## Appendix A: Example Usage ### Scenario: Generate Literature Review ```bash # Step 1: Search papers $ # In IDE, call: search_papers("efficient vision transformers", limit=50) # β†’ Saves to literature/papers.json # Step 2: Generate review $ # In IDE, call: generate_literature_review() # β†’ Generates review # β†’ Saves to literature/review.md # β†’ Returns: { "success": true, "saved_to": "literature/review.md", "metadata": { "paper_count": 47, "word_count": 4523, "sections": [ "Overview", "Taxonomy of Approaches", "Architecture-based Methods", "Training-based Methods", "Deployment-focused Methods", "Critical Analysis", "Research Gaps", "Conclusion" ], "research_gaps": [ { "title": "Scaling sparse attention beyond 1B parameters", "description": "Current work on sparse attention...", "significance": "..." }, ... ], "citations_added": 47 }, "cost": { "input_tokens": 18234, "output_tokens": 5890, "total_usd": 0.14 } } # Step 3: Review the output $ # Open literature/review.md in editor # β†’ Edit if needed # β†’ All citations already in references.bib ``` ### Generated Review Sample ```markdown # Literature Review: Efficient Vision Transformers ## Overview Vision Transformers (ViTs) have demonstrated remarkable performance across various computer vision tasks, but their computational demands remain a significant barrier to practical deployment. This review synthesizes 47 recent papers (2022-2024) exploring efficiency improvements for ViTs, organizing approaches into three main categories: architecture modifications, training optimizations, and deployment techniques. ## Taxonomy of Approaches ### 1. Architecture-based Methods Architecture-based approaches modify the ViT structure to reduce computational complexity while maintaining accuracy. **Sparse Attention Mechanisms** focus on reducing the quadratic complexity of self-attention. Notable work includes EfficientViT [@liu2023efficient], which introduces a memory-efficient attention design achieving 3Γ— speedup with <1% accuracy drop. Similarly, Sparse ViT [@chen2023sparse] demonstrates that structured sparsity patterns can scale to larger models effectively. **Hierarchical Designs** adopt pyramid structures similar to CNNs. MobileViT [@mehta2022mobile] combines convolutions with transformers for mobile deployment, while Swin Transformer [@liu2021swin] uses shifted windows to limit attention scope, achieving linear complexity. [... continues with detailed analysis ...] ## Research Gaps Based on this review, we identify five critical gaps: 1. **Scaling Sparse Attention Beyond 1B Parameters** - Current work demonstrates sparse attention up to 1B parameters - Unclear if benefits persist at 10B+ scale - Potential approaches: Hybrid sparse-dense architectures 2. **Unified Efficiency Metrics** - Papers report FLOPs, latency, memory separately - Need standardized benchmarking framework - Potential: Community-wide efficiency benchmark suite [... more gaps ...] ## Conclusion The field of efficient Vision Transformers has made substantial progress, with sparse attention and hierarchical designs showing particular promise. However, significant gaps remain, especially in scaling efficiency techniques to larger models and establishing standardized evaluation protocols. Future work should prioritize real-world deployment scenarios and cross-technique combinations. ``` --- ## Appendix B: Comparison Table | Feature | v2.0 | v2.1 | |---------|------|------| | Paper Search | βœ“ | βœ“ | | Citation Management | βœ“ | βœ“ | | RAG Retrieval | βœ“ | βœ“ | | File Management | βœ“ | βœ“ | | **Literature Review** | **IDE's LLM** | **βœ“ Built-in** | | Hypothesis Generation | IDE's LLM | IDE's LLM | | Method Design | IDE's LLM | IDE's LLM | | Paper Writing | IDE's LLM | IDE's LLM | | **Requires API Key** | **No** | **Yes (Anthropic/OpenAI)** | | **Setup Time** | **< 5 min** | **< 10 min** | | **Development Time** | **3 weeks** | **4 weeks** | --- ## Appendix C: Feature Matrix ### Functional Requirements Summary | ID | Requirement | Priority | Status | Mode 1 (MCP) | Mode 2 (Agent) | |----|-------------|----------|--------|--------------|----------------| | FR-001 | Paper Search | P0 | Not Started | βœ“ Manual call | βœ“ Auto in workflows | | FR-002 | RAG Retrieval | P0 | Not Started | βœ“ Manual call | βœ“ Auto in workflows | | FR-003 | Citation Management | P0 | Not Started | βœ“ Manual call | βœ“ Auto in workflows | | FR-004 | Context Management | P0 | Not Started | βœ“ Manual call | βœ“ Auto in workflows | | FR-005 | Project Initialization | P1 | Not Started | βœ“ Manual call | βœ“ Auto in workflows | | FR-006 | Literature Review | P1 | Not Started | βœ“ Manual call | βœ“ Auto in workflows | | FR-007 | Custom Agent | P1 | Not Started | βœ— Not available | βœ“ Core feature | | FR-008 | Workflow System | P1 | Not Started | βœ— Not available | βœ“ Core feature | ### Development Effort Comparison | Aspect | Mode 1 Only | Mode 1 + Mode 2 | |--------|-------------|-----------------| | **Development Time** | 4 weeks | 5 weeks | | **Codebase Size** | ~3,000 LOC | ~5,000 LOC | | **Test Coverage** | Tools + Services | + Agent + Workflows | | **Complexity** | Low | Medium | | **Maintenance** | Minimal | Moderate | ### User Experience Comparison | Scenario | Mode 1: Pure MCP | Mode 2: Custom Agent | |----------|------------------|----------------------| | **Literature Survey** | 5 manual steps | 1 command: @research | | **Paper Comparison** | 3 manual steps | 1 command: @compare | | **Find Citations** | 2 manual steps | 1 command: @cite | | **Error Handling** | Manual retry | Auto retry + recovery | | **Cost Awareness** | Check manually | Auto warnings | | **Learning Curve** | Steep | Gentle | --- **Document Status**: Draft - Ready for Team Review **Next Steps**: 1. Validate literature review prompt with sample papers 2. Prototype LiteratureReviewService 3. Prototype Custom Agent framework (PolyhedraResearchAgent) 4. Test workflows with real research tasks 5. Cost analysis with real API usage 6. User testing with PhD students (both modes)