docs: add memory, planning, and collaboration design spec

Covers pre-integration refactoring (dead code removal, history rendering,
part name extraction, config) plus CrewAI Memory recall/remember,
planning on Crews, and collaboration via allow_delegation.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

docs/superpowers/specs/2026-04-12-memory-planning-collab-design.md

@@ -0,0 +1,297 @@
+# CrewAI Memory, Planning & Collaboration — Design Spec
+
+**Date:** 2026-04-12
+**Scope:** Pre-integration refactoring + adding CrewAI memory (cross-turn recall), planning (step-by-step coordination), and collaboration (agent delegation) to the existing AgentDispatchFlow.
+
+## Problem
+
+1. Agents have no memory — each turn is stateless. `_build_agent_context()` dumps raw chat history as text into every agent's prompt, scaling linearly with conversation length. Agents repeat prior advice because they can't semantically recall relevant context.
+
+2. No planning — agents respond independently without coordinated step-by-step plans per turn.
+
+3. No collaboration — agents can't ask each other questions mid-turn (`allow_delegation=False`).
+
+4. Dead code in `agents/prompts.py` — 4 functions orphaned after Flow refactor. Part name derivation duplicated in two files.
+
+## Decision
+
+**Phase 1 (this spec):** Refactor dead code, remove raw history, add config, then integrate CrewAI Memory + Planning + Collaboration into the existing AgentDispatchFlow.
+
+## Phase 1A: Pre-Integration Refactoring
+
+### 1. Clean up `agents/prompts.py`
+
+Remove dead functions:
+- `build_orchestrator_system_prompt()` — not called in CrewAI flow
+- `build_chat_messages()` — not called in CrewAI flow
+- `parse_orchestrator_response()` — not called in CrewAI flow
+- `route_by_keywords()` — duplicated by `agent_flow.route_agents()`
+
+Also remove module-level constants only used by dead functions:
+- `_ROUTING_KEYWORDS`
+- `CAD_TRIGGER_KEYWORDS`
+
+Keep only `parse_mentions()` (used by `server/routes.py` and `server/mcp.py`).
+
+### 2. Remove raw history from `_build_agent_context()`
+
+In `agents/crew_orchestrator.py`, change `_build_agent_context()` to render only:
+- DesignState spec (or approved plan)
+- User's latest message
+
+Remove the `## Recent conversation` block that renders `history[-max_history:]` as text. Memory recall will replace this.
+
+The `history` parameter stays in the function signature (used by `extract_decisions()` downstream) but is no longer rendered into agent context.
+
+### 3. Extract part name derivation
+
+Create `core/utils.py` with:
+
+```python
+def derive_part_name(text: str, max_chars: int = 40) -> str:
+    """Derive a filesystem-safe part name from text."""
+    name = text[:max_chars].strip().replace(" ", "_").lower()
+    name = "".join(c for c in name if c.isalnum() or c == "_")
+    return name or "part"
+```
+
+Replace duplicated logic in:
+- `agents/crew_orchestrator.py:208-209`
+- `agents/orchestrator.py:100-104`
+
+### 4. Add memory/planning/collaboration config
+
+Add to `config.yaml`:
+
+```yaml
+memory:
+  enabled: true
+  embedder_provider: google-generativeai
+  embedder_model: gemini-embedding-001
+  recency_weight: 0.4
+  semantic_weight: 0.4
+  importance_weight: 0.2
+  recency_half_life_days: 1
+  recall_limit: 5
+  recall_depth: shallow
+
+crew:
+  planning: true
+  collaboration: true
+```
+
+Add to `config/settings.py`:
+
+```python
+class MemoryConfig(BaseModel):
+    enabled: bool = True
+    embedder_provider: str = "google-generativeai"
+    embedder_model: str = "gemini-embedding-001"
+    recency_weight: float = 0.4
+    semantic_weight: float = 0.4
+    importance_weight: float = 0.2
+    recency_half_life_days: float = 1.0
+    recall_limit: int = 5
+    recall_depth: str = "shallow"
+
+class CrewConfig(BaseModel):
+    planning: bool = True
+    collaboration: bool = True
+```
+
+## Phase 1B: Memory Integration
+
+### Memory lifecycle
+
+`CrewOrchestrator.__init__()` creates a `Memory` instance if `settings.memory.enabled`:
+
+```python
+from crewai import Memory
+
+self._memory = Memory(
+    storage=str(self.output_dir / ".memory"),
+    embedder={
+        "provider": settings.memory.embedder_provider,
+        "config": {"model_name": settings.memory.embedder_model},
+    },
+    recency_weight=settings.memory.recency_weight,
+    semantic_weight=settings.memory.semantic_weight,
+    importance_weight=settings.memory.importance_weight,
+    recency_half_life_days=settings.memory.recency_half_life_days,
+) if settings.memory.enabled else None
+```
+
+### Flow integration
+
+`AgentFlowState` gets a new optional field to carry the memory reference:
+
+```python
+class AgentFlowState(BaseModel):
+    ...
+    # Memory reference (not serialized, set by orchestrator)
+    class Config:
+        arbitrary_types_allowed = True
+```
+
+The memory instance is passed as an instance attribute on the Flow, not via state (Memory isn't serializable). The orchestrator sets it before kickoff:
+
+```python
+flow = AgentDispatchFlow(initial_state=AgentFlowState(...))
+flow._memory = self._memory  # Instance attr, not state field
+flow.kickoff()
+```
+
+### Recall — before each agent
+
+In `_build_crew_agent()`, replace the raw history context with recalled memories:
+
+```python
+def _recall_for_agent(self, agent_id: str) -> str:
+    if self._memory is None:
+        return ""
+    matches = self._memory.recall(
+        self.state.message,
+        scope=f"/agent/{agent_id}",
+        limit=settings.memory.recall_limit,
+        depth=settings.memory.recall_depth,
+    )
+    if not matches:
+        return ""
+    lines = [f"- {m.record.content}" for m in matches]
+    return "## Relevant context from prior turns\n" + "\n".join(lines)
+```
+
+Injected into task description:
+
+```python
+memories = self._recall_for_agent(agent_id)
+task_description = (
+    f"{self.state.context}\n\n"   # DesignState + user message
+    f"{memories}\n\n"              # Recalled memories
+    f"As the {agent_def.role}, ..."
+)
+```
+
+### Remember — after each agent
+
+In `_run_advisor_crew()`, after parsing responses:
+
+```python
+for i, agent_id in enumerate(advisor_ids):
+    ...
+    if raw.strip():
+        responses.append(AgentResponse.from_agent(agent_id, raw.strip()))
+        self._remember_response(agent_id, raw.strip())
+```
+
+In `_run_cad_step()` and `_run_cam_step()`, after setting response:
+
+```python
+self._remember_response("cad", raw_output)
+self._remember_response("cam", raw_output)
+```
+
+Helper:
+
+```python
+def _remember_response(self, agent_id: str, content: str):
+    if self._memory is None:
+        return
+    self._memory.remember(content, scope=f"/agent/{agent_id}")
+```
+
+### Agent scopes
+
+```
+/agent/design        — Design Agent memories
+/agent/engineering   — Engineering Agent memories
+/agent/cnc           — CNC Agent memories
+/agent/cad           — CAD Coder memories
+/agent/cam           — CAM Agent memories
+```
+
+## Phase 1C: Planning Integration
+
+Enable planning on all Crews. The AgentPlanner generates step-by-step instructions injected into task descriptions before execution.
+
+In `_run_advisor_crew()`:
+
+```python
+crew = Crew(
+    agents=crew_agents,
+    tasks=crew_tasks,
+    process=Process.sequential,
+    planning=settings.crew.planning,
+    planning_llm=self._build_llm(),
+    verbose=False,
+)
+```
+
+In `_run_single_agent_crew()`:
+
+```python
+crew = Crew(
+    agents=[crew_agent],
+    tasks=[task],
+    process=Process.sequential,
+    planning=settings.crew.planning,
+    planning_llm=self._build_llm(),
+    verbose=False,
+)
+```
+
+## Phase 1D: Collaboration Integration
+
+Enable delegation on advisory agents so they can ask each other questions within the advisor Crew. CAD and CAM keep `allow_delegation=False` (single-agent Crews).
+
+In `_build_crew_agent()`:
+
+```python
+crew_agent = Agent(
+    ...
+    allow_delegation=settings.crew.collaboration and agent_id in ADVISOR_IDS,
+    ...
+)
+```
+
+## What Changes
+
+| File | Change |
+|------|--------|
+| `agents/prompts.py` | Remove 4 dead functions + dead constants. Keep `parse_mentions()` only. |
+| `agents/crew_orchestrator.py` | Remove history rendering from `_build_agent_context()`. Use `derive_part_name()`. Create `Memory` in `__init__()`. Pass memory to Flow. |
+| `agents/agent_flow.py` | Add `_memory` attr. Add `_recall_for_agent()`, `_remember_response()`. Update `_build_crew_agent()` for delegation. Update Crew creation for planning. Inject memories into task descriptions. |
+| `agents/orchestrator.py` | Use `derive_part_name()` in `_execute_cad_code()`. |
+| `core/utils.py` | **NEW** — `derive_part_name()` |
+| `config.yaml` | Add `memory` and `crew` sections. |
+| `config/settings.py` | Add `MemoryConfig`, `CrewConfig` models. |
+
+## What Does NOT Change
+
+| File | Reason |
+|------|--------|
+| `agents/base.py` | Interface unchanged |
+| `agents/design_state.py` | Domain model, not orchestration |
+| `agents/tools.py` | Tools unchanged |
+| `agents/gap_analyzer.py` | Runs after Flow |
+| `agents/definitions.py` | Agent metadata unchanged |
+| `server/routes.py` | API contract unchanged |
+| `web/index.html` | Frontend unchanged |
+
+## Testing
+
+**New tests:**
+- `TestDerivePartName` — empty, special chars, truncation
+- `TestRecallForAgent` — returns formatted memories, empty when no memory
+- `TestRememberResponse` — stores with correct scope, no-op when memory is None
+- `TestPlanningEnabled` — Crew created with `planning=True`
+- `TestCollaborationEnabled` — advisors have `allow_delegation=True`, generators have `False`
+
+**Modified tests:**
+- `test_crew_orchestrator.py` — update any tests relying on raw history in context
+- `test_agent_flow.py` — update mocks for memory/planning/collaboration
+
+**Existing tests:**
+- Routing tests unchanged
+- Gap analysis tests unchanged
+- Design state tests unchanged