--- title: MnemoCore emoji: 🧠 colorFrom: indigo colorTo: purple sdk: docker pinned: false license: mit language: - en tags: - memory - vector-database - cognitive-architecture - agents - hyperdimensional-computing - vsa --- # MnemoCore ### Infrastructure for Persistent Cognitive Memory > *"Memory is not a container. It is a living process β€” a holographic continuum where every fragment contains the whole."*

--- ## Quick Install **Install v3.0.0 after publication:** ```bash pip install mnemocore==3.0.0 ``` **Install from source (development):** ```bash git clone https://github.com/RobinALG87/MnemoCore-Persistent-Cognitive-Ai-Memory.git cd MnemoCore-Persistent-Cognitive-Ai-Memory python -m venv .venv && .\.venv\Scripts\activate # Windows # source .venv/bin/activate # Linux / macOS pip install -e . # editable install pip install -e ".[dev]" # + pytest, mypy, black, etc. ``` > **For the REST server, set your API key before starting:** > ```bash > # Windows PowerShell > $env:HAIM_API_KEY = "your-secure-key" > # Linux / macOS > # export HAIM_API_KEY="your-secure-key" > ``` > Then start the legacy v2 API locally: `mnemocore`. `mnemocore` starts the legacy v2 REST server; `mnemocore-cli` is the legacy Click CLI. Neither is the v3 AgentMemory persistence boundary. For v3, compose `HybridMemoryRuntime` over an explicit scope or deploy `create_v3_app` with a credential-to-scope authorizer. AgentMemory itself is local-first and does not require a running Redis or Qdrant service. Full legacy setup, Docker, and configuration details are in [Installation](#installation) below. --- ## MnemoCore 3.0.0 beta MnemoCore is persistent memory and context infrastructure for agents. Version 3.0.0 beta releases an **AgentMemory-first** runtime: local-first, exact-scope SQLite memory with timeline/history support, deterministic context compilation, and rebuildable projections. `HybridMemoryRuntime` adds deterministic lexical and BinaryHDV retrieval without adding a second persistence path. The v3 contract is explicit: - Every operation requires a complete `MemoryScope`; cross-scope fallback is forbidden. - AgentMemory is the only durable source of truth for v3. - Cognitive effects are validated plans, not autonomous direct writes. - The v3 HTTP application is composed with a scope authorizer and fails closed when one is absent. - v2 HAIM, REST, MCP, and CLI paths remain compatibility-only and are not the v3 persistence layer. Version 3.0 retains the established dependency set for installation compatibility. A dependency-minimal AgentMemory distribution remains a future major-release migration. ### Legacy cognitive capabilities (v2 compatibility only) The following HAIM description is retained as a reference for existing v2 deployments. It is not a v3 persistence or deployment contract. Traditional vector stores retrieve. MnemoCore **thinks**. It is built on the mathematical framework of **Binary Hyperdimensional Computing (HDC)** and **Vector Symbolic Architectures (VSA)**, principles rooted in Pentti Kanerva's landmark 2009 theory of cognitive computing. Every memory is encoded as a **16,384-dimensional binary holographic vector** β€” a format that is simultaneously compact (2,048 bytes), noise-tolerant (Hamming geometry), and algebraically rich (XOR binding, majority bundling, circular permutation). At its core lives the **Holographic Active Inference Memory (HAIM) Engine** β€” a system that does not merely answer queries, but: - **Evaluates** the epistemic novelty of every incoming memory before deciding to store it - **Dreams** β€” strengthening synaptic connections between related memories during idle cycles - **Reasons by analogy** β€” if `king:man :: ?:woman`, the VSA soul computes `queen` - **Self-organizes** into tiered storage based on biologically-inspired Long-Term Potentiation (LTP) - **Scales** from a single process to distributed nodes targeting 1B+ memories Phase 4.x introduces cognitive enhancements including contextual masking, reliability feedback loops, semantic consolidation, gap detection/filling, temporal recall (episodic chaining + chrono-weighted query), a Subconscious Daemon with LLM-powered dream synthesis, and a full dependency-injection container pattern for clean modularity. Phase 5 transforms MnemoCore into a true cognitive architecture with Working Memory, Episodic/Semantic/Procedural stores, Meta Memory, and a Self-Improvement Worker β€” all orchestrated by an 11-phase Pulse loop. Phase 6 adds four research-backed services: **StrategyBank** (reasoning bank with Bayesian confidence), **KnowledgeGraph** (spreading activation + community detection), **MemoryScheduler** (OS-level priority queue with interrupts), and **SAMEP** (multi-agent memory exchange with HMAC integrity). --- ## Table of Contents - [Architecture](#architecture) - [Core Technology](#core-technology-binary-hdv--vsa) - [The Memory Lifecycle](#the-memory-lifecycle) - [Tiered Storage](#tiered-storage-hotwarmcold) - [Phase 4.0 Cognitive Enhancements](#phase-40-cognitive-enhancements) - [Phase 4.4–4.5 Subconscious Daemon & LLM Integration](#phase-4445-subconscious-daemon--llm-integration) - [API Reference](#api-reference) - [Python Library Usage](#python-library-usage) - [Installation](#installation) - [Configuration](#configuration) - [MCP Server Integration](#mcp-server-integration) - [Observability](#observability) - [Roadmap](#roadmap) - [Contributing](#contributing) --- ## Architecture The diagram below documents the legacy v2 architecture. New v3 deployments use the scoped AgentMemory and HybridMemoryRuntime composition described above. ``` β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ MnemoCore Stack β”‚ β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ β”‚ β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚ β”‚ REST API (FastAPI / Async) β”‚ β”‚ β”‚ β”‚ /store /query /feedback /insights/gaps /stats β”‚ β”‚ β”‚ β”‚ Rate Limiting Β· API Key Auth Β· Prometheus Metrics β”‚ β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”‚ β”‚ β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚ β”‚ HAIM Engine β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚ β”‚ β”‚ β”‚ Text Encoder β”‚ β”‚ EIG / Epist β”‚ β”‚ Subconsc. β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ (tokenβ†’HDV) β”‚ β”‚ Drive β”‚ β”‚ Dream Loop β”‚ β”‚ β”‚ β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚ β”‚ β”‚ β”‚ Binary HDV Core (VSA) β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ XOR bind Β· majority_bundle Β· permute Β· Hamming β”‚ β”‚ β”‚ β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”‚ β”‚ β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚ β”‚ Tier Manager β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ πŸ”₯ HOT 🌑 WARM ❄️ COLD β”‚ β”‚ β”‚ β”‚ In-Memory Redis / mmap Qdrant / Disk / S3 β”‚ β”‚ β”‚ β”‚ ≀2,000 nodes ≀100,000 nodes ∞ nodes β”‚ β”‚ β”‚ β”‚ <1ms <10ms <100ms β”‚ β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”‚ β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚ β”‚ Conceptual Layer ("The Soul") β”‚ β”‚ β”‚ β”‚ ConceptualMemory Β· Analogy Engine Β· Symbol Algebra β”‚ β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ ``` ### Component Overview | Component | File | Responsibility | |-----------|------|----------------| | **HAIM Engine** | `src/mnemocore/core/engine.py` | Central cognitive coordinator β€” store, query, dream, delete | | **BinaryHDV** | `src/mnemocore/core/binary_hdv.py` | 16384-dim binary vector math (XOR, Hamming, bundle, permute) | | **TextEncoder** | `src/mnemocore/core/binary_hdv.py` | Tokenβ†’HDV pipeline with positional permutation binding | | **MemoryNode** | `src/mnemocore/core/node.py` | Memory unit with LTP, epistemic values, tier state | | **TierManager** | `src/mnemocore/core/tier_manager.py` | HOT/WARM/COLD orchestration with LTP-driven eviction | | **SynapticConnection** | `src/mnemocore/core/synapse.py` | Hebbian synapse with strength, decay, and fire tracking | | **SynapseIndex** | `src/mnemocore/core/synapse_index.py` | Fast synapse lookup index for associative spreading | | **ConceptualMemory** | `src/mnemocore/core/holographic.py` | VSA soul for analogy and cross-domain symbolic reasoning | | **AsyncRedisStorage** | `src/mnemocore/core/async_storage.py` | Async Redis backend (WARM tier + pub/sub) | | **BayesianLTP** | `src/mnemocore/core/bayesian_ltp.py` | Bayesian reliability scoring on top of LTP strength | | **SemanticConsolidation** | `src/mnemocore/core/semantic_consolidation.py` | Memory deduplication via majority-bundle prototyping | | **ConsolidationWorker** | `src/mnemocore/core/consolidation_worker.py` | Async worker scheduling nightly consolidation | | **GapDetector** | `src/mnemocore/core/gap_detector.py` | Temporal co-occurrence analysis for knowledge gaps | | **GapFiller** | `src/mnemocore/core/gap_filler.py` | Bridge detected gaps via synapse creation | | **Immunology** | `src/mnemocore/core/immunology.py` | Auto-associative attractor cleanup for vector drift | | **Attention** | `src/mnemocore/core/attention.py` | XOR context masking / project isolation | | **BatchOps** | `src/mnemocore/core/batch_ops.py` | Vectorized bulk store / query operations | | **HNSWIndex** | `src/mnemocore/core/hnsw_index.py` | In-process HNSW approximate nearest-neighbour index | | **QdrantStore** | `src/mnemocore/core/qdrant_store.py` | Async Qdrant COLD tier backend | | **RecursiveSynthesizer** | `src/mnemocore/core/recursive_synthesizer.py` | Deep concept synthesis via iterative VSA composition | | **RippleContext** | `src/mnemocore/core/ripple_context.py` | Cascading context propagation across synaptic graph | | **SubconsciousAI** | `src/mnemocore/core/subconscious_ai.py` | LLM-guided dream synthesis worker | | **SubconsciousDaemon** | `src/mnemocore/subconscious/daemon.py` | Background process orchestrating dream/consolidation cycles | | **LLMIntegration** | `src/mnemocore/llm_integration.py` | Agent-facing LLM connector (OpenAI / Anthropic compatible) | | **Container** | `src/mnemocore/core/container.py` | Dependency-injection wiring for all core components | | **GoalTree** | `src/mnemocore/meta/goal_tree.py` | Hierarchical goal / task tracking for meta-cognition | | **LearningJournal** | `src/mnemocore/meta/learning_journal.py` | Persistent log of what the agent has learned over time | | **API** | `src/mnemocore/api/main.py` | Legacy v2 HAIM REST interface; v3 uses `create_v3_app` with scope authorization | | **MCP Server** | `src/mnemocore/mcp/server.py` | Legacy v2 HAIM REST adapter; v3 uses scoped AgentMemory integration helpers | --- ## Core Technology: Binary HDV & VSA MnemoCore's mathematical foundation is **Hyperdimensional Computing** β€” a computing paradigm that encodes information in very high-dimensional binary vectors (HDVs), enabling noise-tolerant, distributed, and algebraically composable representations. ### The Vector Space Every piece of information β€” a word, a sentence, a concept, a goal β€” is encoded as a **16,384-dimensional binary vector**: ``` Dimension D = 16,384 bits = 2,048 bytes per vector Storage: packed as numpy uint8 arrays Similarity: Hamming distance (popcount of XOR result) Random pair: ~50% similarity (orthogonality by probability) ``` At this dimensionality, two random vectors will differ in ~50% of bits. This near-orthogonality is the foundation of the system's expressive power β€” related concepts cluster together while unrelated ones remain maximally distant. ### VSA Algebra Four primitive operations make the entire system work: #### Binding β€” XOR `βŠ•` Creates an association between two concepts. Crucially, the result is **dissimilar to both inputs** (appears as noise), making it a true compositional operation. ```python # Bind content to its context bound = content_vec.xor_bind(context_vec) # content βŠ• context # Self-inverse: unbind by re-binding recovered = bound.xor_bind(context_vec) # β‰ˆ content (XOR cancels) ``` Key mathematical properties: - **Self-inverse**: `A βŠ• A = 0` (XOR cancels itself) - **Commutative**: `A βŠ• B = B βŠ• A` - **Distance-preserving**: `hamming(AβŠ•C, BβŠ•C) = hamming(A, B)` #### Bundling β€” Majority Vote Creates a **prototype** that is similar to all inputs. This is how multiple memories combine into a concept. ```python from mnemocore.core.binary_hdv import majority_bundle # Create semantic prototype from related memories concept = majority_bundle([vec_a, vec_b, vec_c, vec_d]) # similar to all inputs ``` #### Permutation β€” Circular Shift Encodes **sequence and roles** without separate positional embeddings. ```python # Positional encoding: token at position i positioned = token_vec.permute(shift=i) # circular bit-shift # Encode "hello world" with order information hello_positioned = encoder.get_token_vector("hello").permute(0) world_positioned = encoder.get_token_vector("world").permute(1) sentence_vec = majority_bundle([hello_positioned, world_positioned]) ``` #### Similarity β€” Hamming Distance Fast comparison using vectorized popcount over XOR results: ```python # Normalized similarity: 1.0 = identical, 0.5 = unrelated sim = vec_a.similarity(vec_b) # 1.0 - hamming(a, b) / D # Batch nearest-neighbor search (no Python loops) distances = batch_hamming_distance(query, database_matrix) ``` ### Text Encoding Pipeline The `TextEncoder` converts natural language to HDVs using a token-position binding scheme: ``` "Python TypeError" β†’ token_hdv("python") βŠ• permute(0) = positioned_0 token_hdv("typeerror") βŠ• permute(1) = positioned_1 majority_bundle([positioned_0, positioned_1]) = final_hdv ``` Token vectors are **deterministic** β€” seeded via SHAKE-256 hash β€” meaning the same word always produces the same base vector, enabling cross-session consistency without a vocabulary file. --- ## The Memory Lifecycle Every memory passes through a defined lifecycle from ingestion to long-term storage: ``` Incoming Content β”‚ β–Ό β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ TextEncoder β”‚ β†’ 16,384-dim binary HDV β””β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”˜ β”‚ β–Ό β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ Context Binding β”‚ β†’ XOR bind with goal_context if present β”‚ (XOR) β”‚ bound_vec = content βŠ• context β””β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β–Ό β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ EIG Evaluation β”‚ β†’ Epistemic Information Gain β”‚ (Novelty Check) β”‚ eig = normalized_distance(vec, context_vec) β””β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ tag "epistemic_high" if eig > threshold β”‚ β–Ό β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ MemoryNode β”‚ β†’ id, hdv, content, metadata β”‚ Creation β”‚ ltp_strength = I Γ— log(1+A) Γ— e^(-Ξ»T) β””β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β–Ό β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ HOT Tier β”‚ β†’ In-memory dict (max 2000 nodes) β”‚ (RAM) β”‚ LTP eviction: low-LTP nodes β†’ WARM β””β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ (background) β–Ό β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ Subconscious β”‚ β†’ Dream cycle fires β”‚ Dream Loop β”‚ Query similar memories β””β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ Strengthen synapses (Hebbian) β”‚ β–Ό β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ WARM Tier β”‚ β†’ Redis-backed persistence β”‚ (Redis/mmap) β”‚ async dual-write + pub/sub events β””β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ (scheduled, nightly) β–Ό β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ COLD Tier β”‚ β†’ Qdrant / Disk / S3 β”‚ (Archival) β”‚ ANN search, long-term persistence β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ ``` ### Long-Term Potentiation (LTP) Memories are not equal. Importance is computed dynamically using a biologically-inspired LTP formula: ``` S = I Γ— log(1 + A) Γ— e^(-Ξ» Γ— T) Where: S = LTP strength (determines tier placement) I = Importance (derived from epistemic + pragmatic value) A = Access count (frequency of retrieval) Ξ» = Decay lambda (configurable, default ~0.01) T = Age in days ``` Memories with high LTP remain in HOT tier. Those that decay are automatically promoted to WARM, then COLD β€” mirroring how biological memory consolidates from working memory to long-term storage. ### Synaptic Connections Memories are linked by `SynapticConnection` objects that implement Hebbian learning: *"neurons that fire together, wire together."* Every time two memories are co-retrieved (via the background dream loop or explicit binding), their synaptic strength increases. During query time, synaptic spreading amplifies scores of connected memories even when they do not directly match the query vector β€” enabling **associative recall**. ```python # Explicit synapse creation engine.bind_memories(id_a, id_b, success=True) # Associative spreading: query top seeds spread activation to neighbors # neighbor_score += seed_score Γ— synapse_strength Γ— 0.3 ``` --- ## Tiered Storage: HOT / WARM / COLD | Tier | Backend | Capacity | Latency | Eviction Trigger | |------|---------|----------|---------|------------------| | πŸ”₯ **HOT** | Python dict (RAM) | 2,000 nodes | < 1ms | LTP < threshold | | 🌑 **WARM** | Redis + mmap | 100,000 nodes | < 10ms | Age + low access | | ❄️ **COLD** | Qdrant / Disk / S3 | Unlimited | < 100ms | Manual / scheduled | Promotion is automatic: accessing a WARM or COLD memory re-promotes it to HOT based on recalculated LTP. Eviction is LRU-weighted by LTP strength β€” the most biologically active memories always stay hot. --- ## Phase 4.0 Cognitive Enhancements MnemoCore Phase 4.0 introduces five architectural enhancements that elevate the system from **data retrieval** to **cognitive reasoning**. --- ### 1. Contextual Query Masking *(XOR Attention)* **Problem**: Large multi-project deployments suffer from cross-context interference. A query for `"Python error handling"` returns memories from all projects equally, diluting precision. **Solution**: Bidirectional XOR context binding β€” apply the same context vector at both **storage** and **query** time: ``` Store: bound_vec = content βŠ• context_vec Query: masked_query = query βŠ• context_vec Result: (content βŠ• C) Β· (query βŠ• C) β‰ˆ content Β· query (context cancels, cross-project noise is suppressed) ``` ```python # Store memories in a project context engine.store("API rate limiting logic", goal_id="ProjectAlpha") engine.store("Garden watering schedule", goal_id="HomeProject") # Query with context mask β€” only ProjectAlpha memories surface results = engine.query("API logic", top_k=5, context="ProjectAlpha") ``` **Expected impact**: +50–80% query precision (P@5) in multi-project deployments. --- ### 2. Reliability Feedback Loop *(Self-Correcting Memory)* **Problem**: Wrong or outdated memories persist with the same retrieval weight as correct ones. The system has no mechanism to learn from its own mistakes. **Solution**: Bayesian reliability scoring with real-world outcome feedback: ``` reliability = (successes + 1) / (successes + failures + 2) # Laplace smoothing LTP_enhanced = I Γ— log(1+A) Γ— e^(-Ξ»T) Γ— reliability ``` ```python # After using a retrieved memory: engine.provide_feedback(memory_id, outcome=True) # Worked β†’ boost reliability engine.provide_feedback(memory_id, outcome=False) # Failed β†’ reduce reliability # System auto-tags consistently wrong memories as "unreliable" # and verified memories (>5 successes, >0.8 score) as "verified" ``` The system converges toward **high-confidence knowledge** β€” memories that have demonstrably worked in practice rank above theoretically similar but unproven ones. --- ### 3. Semantic Memory Consolidation *(Dream-Phase Synthesis)* **Problem**: Episodic memory grows without bound. 1,000 memories about `"Python TypeError"` are semantically equivalent but consume 2MB of vector space and slow down linear scan queries. **Solution**: Nightly `ConsolidationWorker` clusters similar WARM tier memories and replaces them with a **semantic anchor** β€” a majority-bundled prototype: ``` BEFORE consolidation: mem_001: "Python TypeError in line 45" (2KB vector) mem_002: "TypeError calling function" (2KB vector) ... Γ—100 similar memories (200KB total) AFTER consolidation: anchor_001: "Semantic pattern: python typeerror function" metadata: {source_count: 100, confidence: 0.94} hdv: majority_bundle([mem_001.hdv, ..., mem_100.hdv]) (2KB) ``` ```python # Manual trigger (runs automatically at 3 AM) stats = engine.trigger_consolidation() # β†’ {"abstractions_created": 12, "memories_consolidated": 847} # Via API (admin endpoint) POST /admin/consolidate ``` **Expected impact**: 70–90% memory footprint reduction, 10x query speedup at scale. --- ### 4. Auto-Associative Cleanup Loop *(Vector Immunology)* **Problem**: Holographic vectors degrade over time through repeated XOR operations, noise accumulation, and long-term storage drift. After months of operation, retrieved vectors become "blurry" and similarity scores fall. **Solution**: Iterative attractor dynamics β€” when a retrieved vector appears noisy, snap it to the nearest stable concept in a **codebook** of high-confidence prototypes: ``` noisy_vec β†’ find K nearest in codebook β†’ majority_bundle(K neighbors) β†’ check convergence (Hamming distance < 5%) β†’ iterate until converged or max iterations reached ``` ```python # Cleanup runs automatically on retrieval when noise > 15% node = engine.get_memory(memory_id, auto_cleanup=True) # node.metadata["cleaned"] = True (if cleanup was triggered) # node.metadata["cleanup_iterations"] = 3 # Codebook is auto-populated from most-accessed, high-reliability memories ``` **Expected impact**: Maintain >95% similarity fidelity even after years of operation. --- ### 5. Knowledge Gap Detection *(Proactive Curiosity)* **Problem**: The system is entirely reactive β€” it answers queries but never identifies what it *doesn't know*. True cognitive autonomy requires self-directed learning. **Solution**: Temporal co-occurrence analysis β€” detect concepts that are frequently accessed **close in time** but have **no synaptic connection**, flagging them as knowledge gaps: ```python # Automatically runs hourly gaps = engine.detect_knowledge_gaps(time_window_seconds=300) # Returns structured insight: # [ # { # "concept_a": "Python asyncio event loop", # "concept_b": "FastAPI dependency injection", # "suggested_query": "How does asyncio relate to FastAPI dependency injection?", # "co_occurrence_count": 4 # } # ] # Query endpoint GET /insights/gaps?lookback_hours=24 # Fill gap manually (or via LLM agent) POST /insights/fill-gap {"concept_a_id": "mem_xxx", "concept_b_id": "mem_yyy", "explanation": "FastAPI uses asyncio's event loop internally..."} ``` The system becomes capable of **saying what it doesn't understand** and requesting clarification β€” the first step toward genuine cognitive autonomy. --- ## Phase 4.4–4.5: Subconscious Daemon & LLM Integration ### Subconscious Daemon *(Autonomous Background Mind)* Phase 4.4 introduced `SubconsciousAI` β€” a worker that fires during idle cycles and calls an external LLM to generate **synthetic dream memories**: structured insights derived by reasoning over existing memory clusters, rather than through direct observation. Phase 4.5 hardened this into a full `SubconsciousDaemon` β€” an independently managed asyncio process that orchestrates dream cycles, consolidation scheduling, and subconscious queue processing: ```python # The daemon is started automatically when the API starts up. # It coordinates: # - Dream synthesis: SubconsciousAI β†’ LLM β†’ synthetic insights stored back # - Consolidation scheduling: ConsolidationWorker fired on a configurable interval # - Subconscious queue: novelty detection from Redis pub/sub stream ``` Configure in `config.yaml`: ```yaml haim: subconscious_ai: enabled: true api_url: "https://api.openai.com/v1/chat/completions" # or Anthropic model: "gpt-4o-mini" # api_key: set via SUBCONSCIOUS_AI_API_KEY env var dream_interval_seconds: 300 batch_size: 5 ``` ### Dependency Injection Container All major services (TierManager, AsyncRedisStorage, QdrantStore, SubconsciousAI, etc.) are now wired through `src/mnemocore/core/container.py`. This eliminates global singleton state and makes every subsystem testable in isolation: ```python from mnemocore.core.container import build_container container = build_container(config) engine = container.engine() tier_mgr = container.tier_manager() ``` ### LLM Agent Integration `src/mnemocore/llm_integration.py` provides a high-level interface for attaching MnemoCore to any OpenAI/Anthropic-style LLM agent loop: ```python from mnemocore.llm_integration import MnemoCoreAgent agent = MnemoCoreAgent(engine) # Store agent observations agent.observe("User prefers concise answers over verbose ones") # Recall relevant context before a response context = agent.recall("user preference", top_k=3) ``` --- ## API Reference (v2 compatibility) This section documents the legacy HAIM REST service (`mnemocore.api.main:app`). It remains available for v2 clients but its global JSONL/tiering lifecycle is not AgentMemory-backed and must not be presented as v3 persistence. For v3, use `create_v3_app(sqlite_path, scope_authorizer=...)`; the authorizer must approve the caller's complete requested scope before each operation. ### Authentication All endpoints require an API key via the `X-API-Key` header: ```bash export HAIM_API_KEY="your-secure-key" curl -H "X-API-Key: $HAIM_API_KEY" ... ``` ### Endpoints #### `POST /store` Store a new memory with optional context binding. ```json Request: { "content": "FastAPI uses Pydantic v2 for request validation.", "metadata": {"source": "docs", "tags": ["python", "fastapi"]}, "context": "ProjectAlpha", "agent_id": "agent-001", "ttl": 3600 } Response: { "ok": true, "memory_id": "mem_1739821234567", "message": "Stored memory: mem_1739821234567" } ``` #### `POST /query` Query memories by semantic similarity with optional context masking. ```json Request: { "query": "How does FastAPI handle request validation?", "top_k": 5, "context": "ProjectAlpha" } Response: { "ok": true, "query": "How does FastAPI handle request validation?", "results": [ { "id": "mem_1739821234567", "content": "FastAPI uses Pydantic v2 for request validation.", "score": 0.8923, "metadata": {"source": "docs"}, "tier": "hot" } ] } ``` #### `POST /feedback` Report outcome of a retrieved memory (Phase 4.0 reliability loop). ```json Request: { "memory_id": "mem_1739821234567", "outcome": true, "comment": "This solution worked perfectly." } Response: { "ok": true, "memory_id": "mem_1739821234567", "reliability_score": 0.714, "success_count": 4, "failure_count": 1 } ``` #### `GET /memory/{memory_id}` Retrieve a specific memory with full metadata. ```json Response: { "id": "mem_1739821234567", "content": "...", "metadata": {...}, "created_at": "2026-02-17T20:00:00Z", "ltp_strength": 1.847, "epistemic_value": 0.73, "reliability_score": 0.714, "tier": "hot" } ``` #### `DELETE /memory/{memory_id}` Delete memory from all tiers and clean up synapses. #### `POST /concept` Define a symbolic concept for analogical reasoning. ```json {"name": "king", "attributes": {"gender": "man", "role": "ruler", "domain": "royalty"}} ``` #### `POST /analogy` Solve analogies using VSA algebra: `source:value :: target:?` ```json Request: {"source_concept": "king", "source_value": "man", "target_concept": "queen"} Response: {"results": [{"value": "woman", "score": 0.934}]} ``` #### `GET /insights/gaps` Detect knowledge gaps from recent temporal co-activity (Phase 4.0). ```json Response: { "gaps_detected": 3, "knowledge_gaps": [ { "concept_a": "asyncio event loop", "concept_b": "FastAPI middleware", "suggested_query": "How does event loop relate to middleware?", "co_occurrence_count": 5 } ] } ``` #### `POST /admin/consolidate` Trigger manual semantic consolidation (normally runs automatically at 3 AM). #### `GET /stats` Engine statistics - tiers, synapse count, consolidation state. Requires `X-API-Key` by default. #### `GET /health` Public liveness and dependency diagnostics. A local-only runtime can report `degraded` when Redis is unavailable while remaining live. #### `GET /ready` Readiness of the initialized local runtime. Redis is reported diagnostically but is not required for the single-node prototype to become ready. #### `GET /metrics/` Prometheus metrics endpoint on the API listener (port 8100). The trailing slash is canonical. --- ## Python Library Usage For durable SQLite-backed memory with exact agent scopes, async/sync APIs, bitemporal recall, supersession, and evidence receipts, see the [Persistent Agent Memory Quickstart](docs/AGENT_MEMORY_QUICKSTART.md). Retrieval in this foundation is SQLite FTS5 lexical search only; it is not semantic or embedding retrieval. ### v3 scoped runtime ```python from mnemocore.agent_memory import AgentMemory, MemoryScope from mnemocore.hybrid import HybridMemoryRuntime import asyncio async def main(): scope = MemoryScope(tenant_id="local", user_id="demo", agent_id="assistant") async with await AgentMemory.open("memory.db", scope=scope) as memory: runtime = HybridMemoryRuntime(memory, scope=scope) await memory.remember("User prefers concise answers") results = await runtime.recall(scope, "concise", limit=3) print(results) asyncio.run(main()) ``` In v3, `LiteEngine` and the entire unscoped `Memory(...)` facade are removed and fail with a migration error; use `AgentMemory` with an explicit `MemoryScope`. Every runtime operation uses the complete scope tuple; there is no broader scope fallback. `HAIMEngine` remains available only for v2 compatibility: its global JSONL/tiering lifecycle is not AgentMemory-backed and must not be used as v3 persistence. Use `HybridMemoryRuntime` (or `create_v3_app` for HTTP) for v3. For a temporary bridge of its legacy `store`/`query`/`delete_memory` shape, use the deprecated `HAIMEngineAdapter` with an already-open, scope-bound `AgentMemory` client. The hybrid runtime also has an explicit synchronous facade for synchronous applications. Do not call it from an already-running event loop; use the async runtime there. See the [v3 hybrid-runtime guide](docs/V3_HYBRID_RUNTIME.md) for retrieval metadata, cognitive-plan safeguards, API composition, and the staged remaining work. ### Advanced / Low-level (v2 compatibility only) `HAIMEngine` below is retained for existing v2 integrations. It is deprecated for new work, does not enforce `MemoryScope`, and is not the v3 persistence layer. New v3 code must use the scoped runtime shown above. ```python from mnemocore.core.engine import HAIMEngine engine = HAIMEngine(persist_path="./data/memory.jsonl") # Store memories engine.store("Python generators are lazy iterators", metadata={"topic": "python"}) engine.store("Use 'yield' to create generator functions", metadata={"topic": "python"}) engine.store("Redis XADD appends to a stream", goal_id="infrastructure") # Query (global) results = engine.query("How do Python generators work?", top_k=3) for mem_id, score in results: mem = engine.get_memory(mem_id) print(f"[{score:.3f}] {mem.content}") # Query with context masking results = engine.query("data streams", top_k=5, context="infrastructure") engine.close() ``` ### Analogical Reasoning ```python # Define concepts engine.define_concept("king", {"gender": "man", "role": "ruler"}) engine.define_concept("queen", {"gender": "woman", "role": "ruler"}) engine.define_concept("man", {"gender": "man"}) # VSA analogy: king:man :: ?:woman β†’ queen result = engine.reason_by_analogy( src="king", val="man", tgt="woman" ) print(result) # [("queen", 0.934), ...] ``` ### Working with the Binary HDV Layer Directly ```python from mnemocore.core.binary_hdv import BinaryHDV, TextEncoder, majority_bundle encoder = TextEncoder(dimension=16384) # Encode text python_vec = encoder.encode("Python programming") fastapi_vec = encoder.encode("FastAPI framework") error_vec = encoder.encode("error handling") # Bind concept to role python_in_fastapi = python_vec.xor_bind(fastapi_vec) # Bundle multiple concepts into prototype web_dev_prototype = majority_bundle([python_vec, fastapi_vec, error_vec]) # Similarity print(python_vec.similarity(web_dev_prototype)) # High (part of bundle) print(python_vec.similarity(error_vec)) # ~0.5 (unrelated) # Batch nearest-neighbor search from mnemocore.core.binary_hdv import batch_hamming_distance import numpy as np database = np.stack([v.data for v in [python_vec, fastapi_vec, error_vec]]) distances = batch_hamming_distance(python_vec, database) ``` ### Reliability Feedback Loop ```python mem_id = engine.store("Always use asyncio.Lock() in async code, not threading.Lock()") results = engine.query("async locking") # It works β€” report success engine.provide_feedback(mem_id, outcome=True, comment="Solved deadlock issue") # Over time, high-reliability memories get 'verified' tag # and are ranked above unproven ones in future queries ``` ### Semantic Consolidation ```python stats = engine.trigger_consolidation() print(f"Created {stats['abstractions_created']} semantic anchors") print(f"Consolidated {stats['memories_consolidated']} episodic memories") # Automatic: runs every night at 3 AM via background asyncio task ``` --- ## Installation ### Prerequisites - **Python 3.10+** - Redis and Qdrant are optional runtime services for legacy tiering and vector integrations. They are not required for local AgentMemory. - The current 2.x wheel retains its compatibility dependencies. It does not connect to Redis or Qdrant merely by importing `mnemocore.agent_memory`. ### Quick Start (AgentMemory β€” stable local-first path) ```bash # 1. Install. No Redis/Qdrant service is needed for AgentMemory. pip install -e . ``` ```python # 2. Use AgentMemory with an explicit scope and local SQLite store. from pathlib import Path from mnemocore.agent_memory import MemoryScope, SyncAgentMemory scope = MemoryScope(tenant_id="local", user_id="robin", agent_id="assistant") memory = SyncAgentMemory.open(Path("agent-memory.sqlite3"), scope=scope) memory.remember("User prefers concise answers") print(memory.recall("concise")) ``` See "Full server setup (optional)" below for HAIM, Redis, Docker, and the REST API. ### Full server setup (optional, for Redis-backed etc.) # To include dev tools (pytest, mypy, black, etc.): pip install -e ".[dev]" # Start Redis docker run -d -p 6379:6379 redis:7.2-alpine # Set API key (never hardcode β€” use env var or .env file) # Windows PowerShell: $env:HAIM_API_KEY = "your-secure-key-here" # Linux / macOS: # export HAIM_API_KEY="your-secure-key-here" # Start the API locally # PowerShell; use export MNEMOCORE_HOST=127.0.0.1 on Unix $env:MNEMOCORE_HOST = "127.0.0.1" mnemocore The API is now live at `http://localhost:8100`. Visit `http://localhost:8100/docs` for the interactive Swagger UI. > Do not bind MnemoCore directly to a public IP unless it is behind a trusted reverse proxy, TLS, and a strong `HAIM_API_KEY`. Diagnostics (`/stats`, `/rate-limits`, `/metrics`) require `X-API-Key` by default. ### Using the .env file Copy the provided template and fill in your values β€” the API and docker-compose both pick it up automatically: ```bash cp .env.example .env # Edit .env and set HAIM_API_KEY, REDIS_PASSWORD, QDRANT_API_KEY, etc. ``` > **Note:** `.env` is listed in `.gitignore` and must never be committed. Only `.env.example` (with placeholder values) belongs in version control. ### Full Stack with Docker Compose ```bash # Requires .env with HAIM_API_KEY, REDIS_PASSWORD, and QDRANT_API_KEY set. docker compose config docker compose up -d ``` This starts MnemoCore, Redis 7.2, and Qdrant. Only the API is published, on `127.0.0.1:8100`; metrics are served from `/metrics/` on the same port. ### With Qdrant (Phase 4.x Scale) ```bash # Start Qdrant alongside Redis docker run -d -p 6333:6333 qdrant/qdrant # Enable in config.yaml qdrant: enabled: true host: localhost port: 6333 ``` --- ## Configuration All configuration lives in `config.yaml`. Sensitive values can be overridden with environment variables β€” the config loader looks for `HAIM_`-prefixed vars and also honours per-service overrides like `HAIM_API_KEY`, `REDIS_PASSWORD`, `QDRANT_API_KEY`, `HAIM_CORS_ORIGINS`, and `SUBCONSCIOUS_AI_API_KEY`. ```yaml haim: version: "5.0" dimensionality: 16384 # Binary vector dimensions (must be multiple of 64) encoding: mode: "binary" # "binary" (recommended) or "float" (legacy, deprecated) token_method: "bundle" # "bundle" (XOR+permute) or "hash" tiers: hot: max_memories: 2000 # Max nodes in RAM ltp_threshold_min: 0.7 # Evict below this LTP strength eviction_policy: "lru" warm: max_memories: 100000 # Max nodes in Redis/mmap ltp_threshold_min: 0.3 cold: storage_backend: "filesystem" # "filesystem" or "s3" compression: "gzip" ltp: initial_importance: 0.5 decay_lambda: 0.01 # Higher = faster forgetting permanence_threshold: 0.95 # LTP above this is immune to decay half_life_days: 30.0 hysteresis: promote_delta: 0.15 # LTP must exceed threshold by this much to promote demote_delta: 0.10 redis: url: "redis://localhost:6379/0" stream_key: "haim:subconscious" max_connections: 10 socket_timeout: 5 # password: set via REDIS_PASSWORD env var qdrant: url: "http://localhost:6333" collection_hot: "haim_hot" collection_warm: "haim_warm" enabled: false # api_key: set via QDRANT_API_KEY env var security: # api_key: set via HAIM_API_KEY env var β€” never hardcode here cors_origins: ["http://localhost:3000"] public_health_enabled: true public_stats_enabled: false public_rate_limits_enabled: false public_metrics_enabled: false require_api_key_for_export: true subconscious_ai: enabled: false api_url: "https://api.openai.com/v1/chat/completions" model: "gpt-4o-mini" dream_interval_seconds: 300 batch_size: 5 # api_key: set via SUBCONSCIOUS_AI_API_KEY env var observability: metrics_port: 8100 log_level: "INFO" structured_logging: true paths: data_dir: "./data" memory_file: "./data/memory.jsonl" codebook_file: "./data/codebook.json" concepts_file: "./data/concepts.json" synapses_file: "./data/synapses.json" warm_mmap_dir: "./data/warm_tier" cold_archive_dir: "./data/cold_archive" mcp: enabled: false transport: "stdio" host: "127.0.0.1" port: 8110 api_base_url: "http://localhost:8100" ``` ### Security Note MnemoCore requires an explicit API key. There is no default fallback key in production builds. ```bash # Generate a cryptographically secure key: python -c "import secrets; print(secrets.token_urlsafe(32))" # Set it (never commit this value): export HAIM_API_KEY="" ``` Persistent webhooks store only an opaque `secret_ref`; the signing secret is resolved when a delivery is sent. Inline signing secrets, legacy plaintext records, and all custom headers are rejected for persistent webhook managers. Writes are serialized and atomically replaced, and malformed persistence loads fail closed without partial in-memory state. Memory-only webhook managers retain the legacy inline-secret/header behavior for compatibility. AgentMemory also exposes a physical SQLite erase operation with exact-scope ownership checks, optional whole-supersession-component cascade, dependent-row cleanup, database integrity checks, and a content-free `ErasureReceipt`. All database users must cooperate with the store's sidecar lock; raw SQLite clients are outside the safety contract during erasure. Backups, derived external artifacts, power-loss recovery, and exhaustive failure injection remain separate production gates. --- ## MCP Server Integration (v2 compatibility) MnemoCore exposes a legacy **Model Context Protocol (MCP)** server for existing v2 integrations. It proxies the legacy HAIM REST API and is not backed by v3 AgentMemory truth. New scoped integrations should use the AgentMemory MCP helpers in `mnemocore.integrations.mcp` (or build on `HybridMemoryRuntime`). ### Setup ```bash # Start the legacy v2 API first uvicorn mnemocore.api.main:app --host 0.0.0.0 --port 8100 # Configure MCP in config.yaml haim: mcp: enabled: true transport: "stdio" # or "sse" for streaming # Run MCP server python -m mnemocore.mcp.server ``` ### Claude Desktop Configuration Add to your Claude Desktop `config.json`: ```json { "mcpServers": { "mnemocore": { "command": "python", "args": ["-m", "mnemocore.mcp.server"], "env": { "HAIM_API_KEY": "your-key", "HAIM_BASE_URL": "http://localhost:8100" } } } } ``` Once connected, the agent can: - `store_memory(content, context)` β€” persist learned information - `query_memory(query, context, top_k)` β€” recall relevant memories - `provide_feedback(memory_id, outcome)` β€” signal what worked - `get_knowledge_gaps()` β€” surface what it doesn't understand --- ## Observability MnemoCore ships with built-in Prometheus metrics and structured logging. ### Prometheus Metrics Available at canonical `GET /metrics/` on the API listener, port 8100: | Metric | Description | |--------|-------------| | `haim_api_request_count` | Total requests by endpoint and status | | `haim_api_request_latency_seconds` | Request latency histogram | | `haim_storage_operation_count` | Store/query/delete operations | | `haim_hot_tier_size` | Current HOT tier memory count | | `haim_synapse_count` | Active synaptic connections | ### Grafana Dashboard A sample Grafana dashboard config is available at `grafana-dashboard.json` in the repository root. Import it directly into Grafana via **Dashboards β†’ Import β†’ Upload JSON file**. ### Structured Logging All components use structured Python logging with contextual fields: ``` 2026-02-17 20:00:00 INFO Stored memory mem_1739821234567 (EIG: 0.7823) 2026-02-17 20:00:01 INFO Memory mem_1739821234567 reliability updated: 0.714 (4βœ“ / 1βœ—) 2026-02-17 03:00:00 INFO Consolidation complete: abstractions_created=12, consolidated=847 2026-02-17 04:00:00 INFO Knowledge gap detected: asyncio ↔ FastAPI middleware (5 co-occurrences) ``` --- ## Testing ```bash # Run full test suite pytest # Run with coverage pytest --cov=src --cov-report=html # Run specific feature tests pytest tests/test_xor_attention.py # Contextual masking pytest tests/test_stability.py # Reliability/Bayesian stability pytest tests/test_consolidation.py # Semantic consolidation pytest tests/test_engine_cleanup.py # Cleanup and decay pytest tests/test_phase43_regressions.py # Phase 4.3 regression guardrails pytest tests/test_tier_manager.py # Tier demotion / promotion logic pytest tests/test_dream_loop.py # Subconscious dream loop pytest tests/test_subconscious_ai_worker.py # LLM-powered dream worker (if offline: uses mocks) pytest tests/test_recursive_synthesizer.py # Deep concept synthesis pytest tests/test_batch_ops.py # Bulk ingestion operations pytest tests/test_mcp_server.py # MCP server adapter # End-to-end flow pytest tests/test_e2e_flow.py -v ``` --- ## Roadmap ### v3.0.0 beta: released surface - [x] Exact-scope AgentMemory persistence, history, timeline, and context compilation - [x] Hybrid lexical/BinaryHDV retrieval with versioned deterministic scoring - [x] Rebuildable tier and graph projections - [x] Validated, atomic cognitive plan application with provenance - [x] Scoped HTTP composition that fails closed without authorization - [x] Explicit migration away from LiteEngine and implicit `Memory(...)` ### v2 compatibility inventory - [x] Binary HDV core (XOR bind / bundle / permute / Hamming) - [x] Three-tier HOT/WARM/COLD memory lifecycle - [x] Async API + MCP integration - [x] XOR attention masking + Bayesian reliability updates - [x] Semantic consolidation, immunology cleanup, and gap detection/filling - [x] Temporal recall: episodic chaining + chrono-weighted query - [x] Phase 4.4 β€” Subconscious AI Worker (LLM-powered dream synthesis) - [x] Phase 4.5 β€” Subconscious Daemon, persistence hardening, tier-manager demotion race fix - [x] Dependency-injection Container pattern (replaces singleton) - [x] HNSW in-process index for hot-tier ANN search - [x] Batch operations for bulk ingestion - [x] Meta-cognition layer: GoalTree + LearningJournal - [x] Phase 5 β€” Cognitive Architecture: Working Memory, Episodic/Semantic/Procedural stores, Meta Memory, Self-Improvement Worker, 7-phase Pulse loop - [x] Phase 5 Hardening β€” 136 additional tests, config organization - [x] Phase 6 β€” Research Services: StrategyBank, KnowledgeGraph, MemoryScheduler, SAMEP, 4 new Pulse phases - [x] 1291+ tests passing ### Next Steps - [ ] Production hardening: distributed HOT-tier, CUDA kernels, chaos engineering - [ ] Self-improvement Phase 1: enable writes with human-in-the-loop approval - [ ] Multi-modal memory: image/audio embedding via CLIP/Whisper - [ ] Additional first-party integration adapters - [ ] Extended observability standardization (`mnemocore_*` metric prefix) --- ## Contributing MnemoCore is an active research project. Contributions are welcome β€” especially: - **Performance**: CUDA kernels, FAISS integration, async refactoring - **Algorithms**: Better clustering for consolidation, improved EIG formulas - **Integrations**: New storage backends, LLM connectors - **Tests**: Coverage for edge cases, property-based testing ### Process ```bash # Fork and clone git checkout -b feature/your-feature-name # Make changes, ensure tests pass pytest # Commit with semantic message git commit -m "feat(consolidation): add LLM-powered prototype labeling" # Open PR β€” describe the what, why, and performance impact ``` Please follow the implementation patterns established in `docs/ARCHITECTURE.md` and `docs/ROADMAP.md` for architectural guidance, and review `CHANGELOG.md` to understand what has already landed. --- ## Documentation | Document | Description | |----------|-------------| | [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) | System architecture with Mermaid diagrams | | [docs/API.md](docs/API.md) | Complete REST API reference (41 endpoints) | | [docs/CLI.md](docs/CLI.md) | Command-line interface reference | | [docs/CONFIGURATION.md](docs/CONFIGURATION.md) | All configuration options (41 dataclasses) | | [docs/DEPLOYMENT.md](docs/DEPLOYMENT.md) | Docker, Compose, and Kubernetes deployment | | [docs/EVENTS.md](docs/EVENTS.md) | Event system and webhook delivery | | [docs/FORGETTING.md](docs/FORGETTING.md) | SM-2 spaced repetition and decay system | | [docs/LLM_INTEGRATION.md](docs/LLM_INTEGRATION.md) | Multi-provider LLM integration | | [docs/GLOSSARY.md](docs/GLOSSARY.md) | Terms and abbreviations | | [docs/SUBCONSCIOUS_AI.md](docs/SUBCONSCIOUS_AI.md) | Subconscious worker and dream pipeline | | [docs/RECONSTRUCTIVE_MEMORY.md](docs/RECONSTRUCTIVE_MEMORY.md) | Reconstructive recall module | | [docs/EMBEDDING_REGISTRY.md](docs/EMBEDDING_REGISTRY.md) | Embedding model versioning | | [CHANGELOG.md](CHANGELOG.md) | Release history | | [CONTRIBUTING.md](CONTRIBUTING.md) | Contribution guidelines | --- ## License MIT License β€” see [LICENSE](LICENSE) for details. --- ## Contact **Robin Granberg** πŸ“§ robin@veristatesystems.com ---

Building the cognitive substrate for the next generation of autonomous AI.