| --- |
| license: apache-2.0 |
| language: |
| - en |
| tags: |
| - holographic-reduced-representations |
| - vector-symbolic-architecture |
| - hyperdimensional-computing |
| - fhrr |
| - associative-memory |
| - educational |
| - research |
| - memory |
| - sequence-modeling |
| - binding |
| library_name: holomind |
| pipeline_tag: feature-extraction |
| --- |
| |
| # HoloMind — one holographic substrate, several memories |
|
|
| A single FHRR (Fourier Holographic Reduced Representation) substrate with |
| typed regions. Facts, episodes, multimodal associations, and sequences live |
| in the same role/value codebook. Retrieval is algebraic unbinding plus |
| nearest-neighbour classification. Writes are additive; eviction is |
| subtractive; capacity is measurable from the trace magnitude. |
|
|
| This is an **educational micro-model**, not a trained neural network. It |
| has no weights, no gradients, and no training loop. Its purpose is to expose |
| the primitive operations that a persistent LLM memory could be built from, |
| and to report where those operations hold and where they break. |
|
|
| Companion to *Twelve experiments probing the holographic substrate*. |
|
|
| --- |
|
|
| ## Results |
|
|
| All numbers below are from `holomind.py` at `D = 2048`, seed 0, on CPU. |
|
|
| ### [1] Facts |
|
|
| ``` |
| capital of france -> paris (sim=0.181) |
| capital of japan -> tokyo (sim=0.187) |
| ``` |
|
|
| ### [2] Multi-hop |
|
|
| ``` |
| france --capital_of--> paris --in_country--> france (sim=0.167) |
| ``` |
|
|
| ### [3] Episodes (directed sequence) |
|
|
| ``` |
| walk from 'wake': ['wake', 'coffee', 'commute', 'work', 'lunch', 'work2'] |
| walk from 'lunch': ['lunch', 'work2', 'commute2', 'dinner'] |
| ``` |
|
|
| ### [4] Multimodal (one concept, several addresses) |
|
|
| ``` |
| img_0 -> cat (sim=0.145) |
| aud_1 -> dog (sim=0.132) |
| ``` |
|
|
| ### [5] Cross-region |
|
|
| ``` |
| capitals('france') -> paris (sim=0.214) |
| facts('paris', in_country) -> france (sim=0.167) |
| continents('france') -> europe (sim=0.282) |
| ``` |
|
|
| ### [6] Capacity — one-stage vs two-stage read at the same load |
|
|
| | N | one-stage | two-stage | \|trace\| | |
| |---|-----------|-----------|-----------| |
| | 10 | 1.00 | 0.90 | 3.2 | |
| | 20 | 1.00 | 1.00 | 4.5 | |
| | 40 | 1.00 | 0.55 | 6.3 | |
| | 60 | 0.95 | 0.47 | 8.0 | |
| | 80 | 0.95 | 0.26 | 8.9 | |
| | 120 | 0.66 | 0.23 | 11.3 | |
|
|
| **Finding.** Retrieval accuracy at load K depends on the number of successive |
| unbindings in the query path, not on K alone. One unbind tolerates K ≈ 40–80 |
| at D = 2048. Each additional unbind in the chain roughly halves the clean-load |
| ceiling. This is not a degradation of the substrate — it is the noise floor |
| compounding through successive algebraic operations. |
|
|
| ### [7] Eviction (reversible additive memory) |
|
|
| ``` |
| before evict: key=e_2 -> e_2 (sim=0.224) |
| after evict: key=e_2 -> e_3 (sim=0.020) [expected noise] |
| after evict: key=e_0 -> e_0 (sim=0.232) [unaffected] |
| ``` |
|
|
| ### [8] Confidence — \|trace\| vs. correctness |
|
|
| | region | n | \|trace\| | hit | verdict | |
| |--------|---|-----------|-----|---------| |
| | load1_10 | 10 | 3.2 | 1.00 | confident | |
| | load1_40 | 40 | 6.3 | 1.00 | confident | |
| | load1_80 | 80 | 8.9 | 0.95 | confident | |
| | load1_120 | 120 | 11.3 | 0.66 | confident | |
|
|
| **Finding.** \|trace\| rises monotonically with load while hit rate falls. |
| A threshold on \|trace\| is a cheap scalar confidence estimate. In this run the |
| verdict classifier is deliberately crude (thresholds at 0.20 and 0.25 of D); |
| the signal shape — monotone trace growth against monotone accuracy decline — |
| is the publishable part. |
|
|
| --- |
|
|
| ## What the substrate is |
|
|
| - **Dimensionality.** Each item is a complex vector of length D = 2048. |
| - **Binding.** `bind(a, b) = ifft(fft(a) · fft(b))`. Symmetric. |
| - **Directed binding.** `bind_dir(a, b) = ifft(fft(a) · roll(fft(b), 1))`. |
| Non-commutative. Removes the 2-cycle failure that symmetric binding |
| produces on sequences. |
| - **Item regions.** `S = Σᵢ bind(itemᵢ, itemᵢ)`. Query: unbind a partial key, |
| read fields by unbinding their roles. |
| - **Transition regions.** `S = Σᵢ bind_dir(sᵢ, sᵢ₊₁)`. Walk by unbinding the |
| current state. |
| - **Roles and values.** Named vectors generated from a stable hash. Shared |
| across all regions, which is what makes a retrieved vector a valid key |
| elsewhere. |
|
|
| ## What the substrate is not |
|
|
| - Not a trained model. No weights, no gradients, no optimization. |
| - Not a replacement for attention or KV cache. It is a substrate those |
| mechanisms could be built on. |
| - Not production. Python loops, no batching, D = 2048 fixed in the demo. |
| Batch the FFT path with `einsum` before using it at scale. |
|
|
| ## Failure modes and fixes |
|
|
| | Experiment | Failure | Fix | |
| |------------|---------|-----| |
| | Analogy (A:B::C:?) | Raw `bind(A,B)` has no recoverable relation. | Explicit relation role: `bind(rel, bind(A, B))`. Same class of bug: the operation must match the structure. | |
| | Nested binding | `bind(text, bind(color, tag))` is unaddressable. | Role-prefix every level: `bind(role_text, bind(role_color, ...))`. Equivalent to flat role summation. | |
| | Sequence retrieval | Symmetric binding produces a 2-cycle: `[0,1,2,3,2,3,...]`. | Directed binding via frequency roll. `bind_dir` is not commutative, so incoming transitions do not cancel coherently. | |
|
|
| The three failures are the same class: the operation did not match the |
| structure of the content. |
|
|
| ## Novel claims |
|
|
| 1. **One-stage vs two-stage capacity gap** at fixed D and fixed K. |
| 2. **Directed binding via frequency roll** as a fix for the commutative |
| binding 2-cycle, with a working benchmark. |
| 3. **\|trace\| as a confidence signal** tied to measured accuracy. |
| 4. **Free role orthogonality at constant D** — adding modalities does not |
| cost retrieval capacity. |
|
|
| ## Usage |
|
|
| No dependencies beyond NumPy. No file I/O. No external data. |
|
|
| ```bash |
| python holomind.py |
| ``` |
|
|
| Expected runtime: ~4 s on a laptop. |
|
|
| ## Files |
|
|
| - `holomind.py` — substrate and demo. Runnable as-is. |
| - `README.md` — this file. |
|
|
| ## Citation |
|
|
| If you use this in academic work: |
|
|
| ```bibtex |
| @misc{holomind2026, |
| title = {HoloMind: one holographic substrate, several memories}, |
| author = {zeechimp}, |
| year = {2026}, |
| note = {Educational micro-model. FHRR substrate with item and transition regions.} |
| } |
| ``` |
|
|
| ## License |
|
|
| Apache 2.0 |
