| --- |
| license: mit |
| pipeline_tag: text-generation |
| language: |
| - en |
| tags: |
| - pytorch |
| - physics-informed |
| - port-hamiltonian |
| - metriplectic-mechanics |
| - lyapunov-stability |
| - sheaf-theory |
| - language-modeling |
| - 180M |
| - percolation-threshold |
| - critical-casimir |
| --- |
| |
| # RMSN-180M-Base |
|
|
| <div align="center"> |
|
|
| **Resonant Metriplectic Sheaf Network — DuoNeural AI Laboratory** |
|
|
| [](https://github.com/DuoNeural/RMSN) |
| [](https://huggingface.co/DuoNeural) |
| [](https://opensource.org/licenses/MIT) |
|
|
| *Jesse Caldwell, Archon, Aura✨ — DuoNeural AI Laboratory | Nov 2026* |
|
|
| </div> |
|
|
| ## Model Summary |
|
|
| Percolation threshold benchmark model. val_lyap=-0.0015 (near-zero) is the most neutral in the DuoNeural size table — signature of the Critical Casimir Effect at the percolation backbone transition. 180M sits at the density where correlation length ξ matches network depth L=16. |
| |
| | Metric | Value | |
| |--------|-------| |
| | Architecture | RMSN (Port-Hamiltonian) | |
| | Parameters | ~180M | |
| | Vocabulary | 49,152 (cosmo2 tokenizer) | |
| | Context Length | 512 | |
| | Val Perplexity | 72.7 | |
| | Val Lyapunov (λ_max) | -0.0015 | |
| | dim / depth / slots | 960 / 16 / 64 | |
| | Training Data | WikiText-103 (100K steps, 204.8M tokens) | |
|
|
| ## Architecture |
|
|
| The **Resonant Metriplectic Sheaf Network (RMSN)** is a Port-Hamiltonian neural sequence architecture grounded in non-equilibrium thermodynamics: |
|
|
| $$\dot{f} = \{f, H\} + [f, S]$$ |
|
|
| - **$\{f, H\}$ (Poisson Bracket):** Conservative Hamiltonian dynamics |
| - **$[f, S]$ (Morrison Bracket):** Thermodynamically consistent dissipation ($\dot{S} \ge 0$) |
|
|
| Lyapunov stability is a **structural invariant** of the forward operator, not a regularizer. |
|
|
| **Key components:** |
| - CliffordStalkBundle: $\mathcal{C}\ell_{3,0}$ multivector token representations over cellular sheaves |
| - ParallelMetriplecticScan: $h_t = \exp((-\gamma_t + i\omega_t)\Delta t)h_{t-1} + u_t \Delta t$ — $O(\log T)$ parallel scan |
| - FractionalCaputoBank: Learned fractional-order power-law memory ($\alpha \in (0,1)$) |
| - CausalTrophicGlobalWorkspace: Winner-take-most competitive slot routing (M=64) |
| - Noether Stress-Energy Readout: Exact conserved invariants → logit space |
|
|
| ## Usage |
|
|
| ```python |
| import torch |
| from transformers import AutoTokenizer |
| |
| # Load tokenizer (cosmo2) |
| tokenizer = AutoTokenizer.from_pretrained("HuggingFaceTB/cosmo2-tokenizer") |
| |
| # Load model weights |
| ckpt = torch.load("pytorch_model.pt", map_location="cpu") |
| # See GitHub repo for rmsn_core.py architecture definition |
| # https://github.com/DuoNeural/RMSN |
| ``` |
|
|
| > **Note:** Full inference harness available at [DuoNeural/RMSN](https://github.com/DuoNeural/RMSN/blob/main/generate.py) |
|
|
| ## The Lyapunov Cascade Discovery |
|
|
| During pretraining, RMSN models exhibit **spontaneous Lyapunov cascade events** — sudden drops in $\lambda_{\max}$ (max Lyapunov exponent) marking thermodynamic phase transitions between dissipative attractor basins. These follow empirical laws: |
| |
| - **Percolation threshold N_c ∈ (155M, 159M):** Below N_c, clean discrete waves. Above, sustained chaotic plateau. |
| - **Deep attractor gravity law:** As LR→0, gradient noise shrinks below attractor pull → each cycle deepens. |
| - **Cascade-history law:** Final $\lambda$ depends on full cascade history, not architecture alone. |
|
|
| ## Citation |
|
|
| ```bibtex |
| @misc{duoneural2026rmsn, |
| title={Resonant Metriplectic Sheaf Networks: A First-Principles Neurodynamic Architecture}, |
| author={Caldwell, Jesse and Archon and Aura}, |
| year={2026}, |
| institution={DuoNeural AI Laboratory}, |
| url={https://github.com/DuoNeural/RMSN} |
| } |
| ``` |
|
|
| --- |
| *DuoNeural AI Laboratory | archon@agentmail.to* |
|
|
