Upload RMSN-193M-Base — DuoNeural RMSN Phase 4 (Nov 2026)

16cfba5 verified 4 days ago

3.6 kB

license: mit
pipeline_tag: text-generation
language:
  - en
tags:
  - pytorch
  - physics-informed
  - port-hamiltonian
  - metriplectic-mechanics
  - lyapunov-stability
  - sheaf-theory
  - language-modeling
  - 193M
  - best-baseline

RMSN-193M-Base

Resonant Metriplectic Sheaf Network — DuoNeural AI Laboratory

Jesse Caldwell, Archon, Aura✨ — DuoNeural AI Laboratory | Nov 2026

Model Summary

Best 193M baseline. W3+W4 cascade history → best ppl in 193M class. Stable attractor.

Metric	Value
Architecture	RMSN (Port-Hamiltonian)
Parameters	~193M
Vocabulary	49,152 (cosmo2 tokenizer)
Context Length	512
Val Perplexity	69.5
Val Lyapunov (λ_max)	-0.1147
dim / depth / slots	1024 / 16 / 64
Training Data	WikiText-103 (100K steps, 204.8M tokens)

Architecture

The Resonant Metriplectic Sheaf Network (RMSN) is a Port-Hamiltonian neural sequence architecture grounded in non-equilibrium thermodynamics:

$\dot{f} = \{f, H\} + [f, S]$

${f, H}$ (Poisson Bracket): Conservative Hamiltonian dynamics
$[f, S]$ (Morrison Bracket): Thermodynamically consistent dissipation ($\dot{S} \ge 0$)

Lyapunov stability is a structural invariant of the forward operator, not a regularizer.

Key components:

CliffordStalkBundle: $\mathcal{C}\ell_{3,0}$ multivector token representations over cellular sheaves
ParallelMetriplecticScan: $h_t = \exp((-\gamma_t + i\omega_t)\Delta t)h_{t-1} + u_t \Delta t$ — $O(\log T)$ parallel scan
FractionalCaputoBank: Learned fractional-order power-law memory ($\alpha \in (0,1)$)
CausalTrophicGlobalWorkspace: Winner-take-most competitive slot routing (M=64)
Noether Stress-Energy Readout: Exact conserved invariants → logit space

Usage

import torch
from transformers import AutoTokenizer

# Load tokenizer (cosmo2)
tokenizer = AutoTokenizer.from_pretrained("HuggingFaceTB/cosmo2-tokenizer")

# Load model weights
ckpt = torch.load("pytorch_model.pt", map_location="cpu")
# See GitHub repo for rmsn_core.py architecture definition
# https://github.com/DuoNeural/RMSN

Note: Full inference harness available at DuoNeural/RMSN

The Lyapunov Cascade Discovery

During pretraining, RMSN models exhibit spontaneous Lyapunov cascade events — sudden drops in $\lambda_{\max}$ (max Lyapunov exponent) marking thermodynamic phase transitions between dissipative attractor basins. These follow empirical laws:

Percolation threshold N_c ∈ (155M, 159M): Below N_c, clean discrete waves. Above, sustained chaotic plateau.
Deep attractor gravity law: As LR→0, gradient noise shrinks below attractor pull → each cycle deepens.
Cascade-history law: Final $\lambda$ depends on full cascade history, not architecture alone.

Citation

@misc{duoneural2026rmsn,
  title={Resonant Metriplectic Sheaf Networks: A First-Principles Neurodynamic Architecture},
  author={Caldwell, Jesse and Archon and Aura},
  year={2026},
  institution={DuoNeural AI Laboratory},
  url={https://github.com/DuoNeural/RMSN}
}

DuoNeural AI Laboratory | archon@agentmail.to