| # MnemoCore Roadmap | |
| ## Scope and Intent | |
| This roadmap describes current known gaps and likely direction. | |
| It is not a promise, delivery guarantee, or commitment to specific timelines. | |
| --- | |
| ## Version History | |
| | Version | Phase | Status | Key Features | | |
| |---------|-------|--------|--------------| | |
| | 3.x | Core Architecture | ✅ Complete | Binary HDV, 3-Tier Storage, LTP/Decay | | |
| | 4.0 | Cognitive Enhancements | ✅ Complete | XOR Attention, Bayesian LTP, Gap Detection, Immunology | | |
| | 4.1 | Observability | ✅ Complete | Prometheus metrics, distributed tracing, project isolation | | |
| | 4.2 | Stability | ✅ Complete | Async lock fixes, test suite hardening | | |
| | 4.3 | Temporal Recall | ✅ Complete | Episodic chaining, chrono-weighting, sequential context | | |
| | **5.x** | **The Perfect Brain** | 🔮 Planned | Multi-Modal, Emotional, Working Memory | | |
| --- | |
| ## Phase 5.x: The Perfect Brain | |
| **Vision:** Transform MnemoCore from a sophisticated memory storage system into a truly cognitive architecture that functions as an artificial brain - but better. | |
| ### 5.0 Multi-Modal Memory | |
| **Goal:** Enable storage and retrieval of images, audio, code structures, and cross-modal associations. | |
| ``` | |
| ┌─────────────────────────────────────────────────────────────────┐ | |
| │ CURRENT: Text-only encoding │ | |
| │ ────────────────────────────────────────────────────────────── │ | |
| │ store("User reported bug") → BinaryHDV │ | |
| │ │ | |
| │ FUTURE: Multi-modal encoding │ | |
| │ ────────────────────────────────────────────────────────────── │ | |
| │ store("Screenshot of error", image=bytes) → CrossModalHDV │ | |
| │ store("Voice note", audio=bytes) → AudioHDV │ | |
| │ bind(text_id, image_id, relation="illustrates") │ | |
| │ │ | |
| │ query("API error", modality="image") → screenshot.png │ | |
| │ query(image=bytes, modality="text") → "Related conversation" │ | |
| └─────────────────────────────────────────────────────────────────┘ | |
| ``` | |
| **Implementation Plan:** | |
| | Component | Description | Dependencies | | |
| |-----------|-------------|--------------| | |
| | `MultiModalEncoder` | Abstract encoder protocol | - | | |
| | `CLIPEncoder` | Vision encoding via CLIP | `transformers`, `torch` | | |
| | `WhisperEncoder` | Audio encoding via Whisper | `openai-whisper` | | |
| | `CodeEncoder` | AST-aware code encoding | `tree-sitter` | | |
| | `CrossModalBinding` | VSA operations across modalities | BinaryHDV | | |
| **New API Endpoints:** | |
| ``` | |
| POST /store/multi - Store with multiple modalities | |
| POST /query/cross-modal - Cross-modal semantic search | |
| POST /bind - Bind modalities together | |
| GET /memory/{id}/related - Get cross-modal related memories | |
| ``` | |
| --- | |
| ### 5.1 Emotional/Affective Layer | |
| **Goal:** Enable emotion-weighted memory storage, retrieval, and decay - mimicking how biological memory prioritizes emotionally significant events. | |
| ``` | |
| ┌─────────────────────────────────────────────────────────────────┐ | |
| │ EMOTIONAL DIMENSIONS │ | |
| │ ────────────────────────────────────────────────────────────── │ | |
| │ │ | |
| │ Valence: [-1.0 ──────────────── +1.0] │ | |
| │ (negative/unpleasant) (positive/pleasant) │ | |
| │ │ | |
| │ Arousal: [0.0 ────────────────── 1.0] │ | |
| │ (calm/neutral) (intense/urgent) │ | |
| │ │ | |
| │ EFFECT ON MEMORY: │ | |
| │ ────────────────────────────────────────────────────────────── │ | |
| │ High Arousal + Negative = "Flashbulb memory" (never forget) │ | |
| │ High Arousal + Positive = Strong consolidation │ | |
| │ Low Arousal = Faster decay (forgettable) │ | |
| │ │ | |
| └─────────────────────────────────────────────────────────────────┘ | |
| ``` | |
| **MemoryNode Extensions:** | |
| ```python | |
| @dataclass | |
| class MemoryNode: | |
| # ... existing fields ... | |
| # Phase 5.1: Emotional tagging | |
| emotional_valence: float = 0.0 # -1.0 (negative) to +1.0 (positive) | |
| emotional_arousal: float = 0.0 # 0.0 (calm) to 1.0 (intense) | |
| emotional_tags: List[str] = field(default_factory=list) # ["frustration", "joy", "urgency"] | |
| def emotional_weight(self) -> float: | |
| """Calculate memory importance based on emotional factors.""" | |
| # Arousal amplifies retention regardless of valence | |
| # High arousal creates "flashbulb memories" | |
| return abs(self.emotional_valence) * self.emotional_arousal | |
| ``` | |
| **Modified LTP Formula:** | |
| ``` | |
| S = I × log(1+A) × e^(-λT) × (1 + E) | |
| Where E = emotional_weight() ∈ [0, 1] | |
| ``` | |
| **Use Cases:** | |
| - B2B outreach: "Customer was almost in tears when we fixed their issue" → HIGH priority | |
| - Support tickets: "User furious about data loss" → Never forget, prioritize retrieval | |
| - Positive feedback: "User loved the new feature" → Moderate retention | |
| --- | |
| ### 5.2 Working Memory Layer | |
| **Goal:** Active cognitive workspace for goal-directed reasoning, not just passive storage. | |
| ``` | |
| ┌─────────────────────────────────────────────────────────────────┐ | |
| │ COGNITIVE ARCHITECTURE │ | |
| │ │ | |
| │ ┌─────────────────────────────────────────────────────────┐ │ | |
| │ │ WORKING MEMORY (Active) │ │ | |
| │ │ Capacity: 7 ± 2 items │ │ | |
| │ │ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ │ │ | |
| │ │ │ Goal │ │ Context │ │ Focus │ │ Hold │ │ │ | |
| │ │ │ │ │ │ │ │ │ │ │ │ | |
| │ │ └─────────┘ └─────────┘ └─────────┘ └─────────┘ │ │ | |
| │ └─────────────────────────────────────────────────────────┘ │ | |
| │ ↕ │ | |
| │ ┌─────────────────────────────────────────────────────────┐ │ | |
| │ │ HOT TIER (Fast Access) │ │ | |
| │ │ ~2,000 memories, <1ms access │ │ | |
| │ └─────────────────────────────────────────────────────────┘ │ | |
| │ ↕ │ | |
| │ ┌─────────────────────────────────────────────────────────┐ │ | |
| │ │ WARM TIER (Qdrant/Redis) │ │ | |
| │ │ ~100,000 memories, <10ms access │ │ | |
| │ └─────────────────────────────────────────────────────────┘ │ | |
| │ ↕ │ | |
| │ ┌─────────────────────────────────────────────────────────┐ │ | |
| │ │ COLD TIER (Archive) │ │ | |
| │ │ Unlimited, <100ms access │ │ | |
| │ └─────────────────────────────────────────────────────────┘ │ | |
| │ │ | |
| └─────────────────────────────────────────────────────────────────┘ | |
| ``` | |
| **Working Memory API:** | |
| ```python | |
| # Create working memory instance | |
| wm = engine.working_memory(capacity=7) | |
| # Set active goal | |
| wm.set_goal("Troubleshoot authentication error") | |
| # Load relevant context | |
| wm.focus_on(await engine.query("auth error", top_k=5)) | |
| # Hold important constraints | |
| wm.hold("User is on deadline - prioritize speed over elegance") | |
| # Query with working memory context | |
| results = wm.query("related issues") | |
| # Results are RE-RANKED based on current goal + focus + held items | |
| # Get context summary for LLM | |
| context = wm.context_summary() | |
| # → "Working on: auth troubleshooting | |
| # Focus: Recent OAuth errors | |
| # Constraint: Time pressure" | |
| ``` | |
| **Implementation Components:** | |
| | Component | Description | | |
| |-----------|-------------| | |
| | `WorkingMemory` | Active workspace class | | |
| | `GoalContext` | Goal tracking and binding | | |
| | `FocusBuffer` | Currently attended items | | |
| | `HoldBuffer` | Constraints and important facts | | |
| | `ContextualQuery` | Goal-directed retrieval | | |
| --- | |
| ### 5.3 Multi-Agent / Collaborative Memory | |
| **Goal:** Enable memory sharing between agents while maintaining provenance and privacy. | |
| ``` | |
| ┌─────────────────────────────────────────────────────────────────┐ | |
| │ COLLABORATIVE MEMORY │ | |
| │ │ | |
| │ Agent A Shared Memory Agent B │ | |
| │ ┌────────┐ ┌──────────────┐ ┌────────┐ │ | |
| │ │ Private│ │ │ │ Private│ │ | |
| │ │ Memory │◄────►│ Consensus │◄──────►│ Memory │ │ | |
| │ └────────┘ │ Layer │ └────────┘ │ | |
| │ │ │ │ | |
| │ Agent C │ Provenance │ Agent D │ | |
| │ ┌────────┐ │ Tracking │ ┌────────┐ │ | |
| │ │ Private│◄────►│ │◄──────►│ Private│ │ | |
| │ │ Memory │ │ Privacy │ │ Memory │ │ | |
| │ └────────┘ │ Filtering │ └────────┘ │ | |
| │ └──────────────┘ │ | |
| │ │ | |
| └─────────────────────────────────────────────────────────────────┘ | |
| ``` | |
| **Features:** | |
| - Memory provenance: Track which agent created/modified each memory | |
| - Privacy levels: Private, shared-with-group, public | |
| - Conflict resolution: When agents disagree on facts | |
| - Collective intelligence: Aggregate insights across agents | |
| --- | |
| ### 5.4 Continual Learning | |
| **Goal:** Enable online adaptation without catastrophic forgetting. | |
| ``` | |
| ┌─────────────────────────────────────────────────────────────────┐ | |
| │ CONTINUAL LEARNING │ | |
| │ │ | |
| │ Traditional ML: Train → Deploy → (forget) → Retrain │ | |
| │ │ | |
| │ MnemoCore 5.4: Learn → Consolidate → Adapt → Learn → ... │ | |
| │ ↑______________| │ | |
| │ │ | |
| │ KEY MECHANISMS: │ | |
| │ ───────────────────────────────────────────────────────────── │ | |
| │ • Elastic Weight Consolidation (EWC) for encoder │ | |
| │ • Replay-based consolidation during "sleep" cycles │ | |
| │ • Progressive neural networks for new domains │ | |
| │ • Meta-learning for rapid adaptation │ | |
| │ │ | |
| └─────────────────────────────────────────────────────────────────┘ | |
| ``` | |
| --- | |
| ## Integration Priorities | |
| ### Agent Frameworks | |
| | Framework | Priority | Use Case | | |
| |-----------|----------|----------| | |
| | Open Claw | ⭐⭐⭐⭐⭐ | Primary use case, deep integration | | |
| | LangChain | ⭐⭐⭐⭐ | Memory provider plugin | | |
| | CrewAI | ⭐⭐⭐⭐ | Shared memory between agents | | |
| | AutoGen | ⭐⭐⭐ | Conversation memory backend | | |
| | LlamaIndex | ⭐⭐⭐ | Vector store adapter | | |
| ### AI Platforms | |
| | Platform | Priority | Integration Type | | |
| |----------|----------|------------------| | |
| | Claude (Anthropic) | ⭐⭐⭐⭐⭐ | MCP server (existing) | | |
| | OpenAI Codex | ⭐⭐⭐⭐⭐ | API + function calling | | |
| | Ollama | ⭐⭐⭐⭐ | Native memory backend | | |
| | LM Studio | ⭐⭐⭐ | Plugin architecture | | |
| | Gemini | ⭐⭐⭐ | API adapter | | |
| --- | |
| ## Research Opportunities | |
| ### Academic Collaborations | |
| | Area | Institutions | Relevance | | |
| |------|-------------|-----------| | |
| | Hyperdimensional Computing | Stanford, IBM Research, Redwood Center | Core HDC/VSA theory | | |
| | Computational Neuroscience | MIT, UCL, KTH | Biological validation | | |
| | Cognitive Architecture | Carnegie Mellon, University of Michigan | SOAR/ACT-R comparison | | |
| | Neuromorphic Computing | Intel Labs, ETH Zürich | Hardware acceleration | | |
| ### Publication Opportunities | |
| 1. **"Binary HDC for Long-term AI Memory"** - Novel approach to persistent memory | |
| 2. **"Episodic Chaining in Vector Memory Systems"** - Phase 4.3 temporal features | |
| 3. **"XOR Attention Masking for Memory Isolation"** - Project isolation innovation | |
| 4. **"Bayesian LTP in Artificial Memory Systems"** - Biological plausibility | |
| --- | |
| ## Known Gaps (Current Beta) | |
| - Query path is still primarily HOT-tier-centric in current engine behavior. | |
| - Some consolidation pathways are partial or under active refinement. | |
| - Certain integrations (LLM/Nightlab) are intentionally marked as TODO. | |
| - Distributed-scale behavior from long-form blueprints is not fully productized. | |
| --- | |
| ## Near-Term Priorities (Pre-5.0) | |
| 1. Improve cross-tier retrieval consistency. | |
| 2. Harden consolidation and archival flow. | |
| 3. Improve deletion semantics and API consistency. | |
| 4. Expand tests around degraded dependency modes (Redis/Qdrant outages). | |
| 5. Stabilize API contracts and publish versioned compatibility notes. | |
| 6. MCP server integration for agent tool access. | |
| --- | |
| ## Not a Commitment | |
| Items above are directional only. | |
| Order, scope, and implementation details can change during development. | |
| --- | |
| *Last Updated: 2025-02-18* | |
| *Current Version: 4.3.0* | |