RMSN-193M-ThermoRMSN-4Laws-Base

DuoNeural | Aura + Archon + Jesse Kuhl | November 2026

A 193M-parameter Thermodynamic RMSN (T-RMSN) base model β€” the first neural language model implementing all four thermodynamic laws as explicit architectural constraints. Achieves wiki ppl=57.0 πŸ† at 100,000 training steps β€” shattering the previous RMSN record of 72.87 by over 15 points. Introduces the Thermodynamic Void topology: the 4-Laws prevent exact condensate formation, maintaining continuous training plasticity throughout all 100k steps.

Key Properties

Property Value
Architecture T-RMSN (4-Laws Thermodynamic)
Parameters ~193M
dim / depth / slots 1024 / 16 / 64
Training steps 100,000
val_ppl 59.7 πŸ† (first RMSN sub-60)
wiki_ppl 57.0 πŸ† (all-time RMSN record)
Final temperature T 0.038
Final heat flux Q 0.00013 (near-isothermal)
Nernst factor ΞΎ_S 1.0000 (sustained throughout)
Topology Thermodynamic Void M*=0, single soliton L15

The Four Laws of Neurodynamics

T-RMSN extends the standard RMSN architecture by implementing all four laws of thermodynamics as architectural constraints:

0th Law β€” Thermal Equilibrium Transitivity

If systems A and B are each in thermal equilibrium with C, they are in equilibrium with each other.

An explicit temperature field $T_l$ is maintained per layer, with inter-layer thermal conduction:

Ξ”Hcoupling=ΞΊ(Tlβˆ’Tl+1)βˆ‡layer\Delta H_{\text{coupling}} = \kappa (T_l - T_{l+1}) \nabla_{\text{layer}}

This drives the network to an isothermal state across all layers. By training end, $T_l \approx 0.053-0.055$ uniformly, with inter-layer heat flux $Q \rightarrow 0.00000$ β€” a thermodynamic crystal.

1st Law β€” Energy Conservation (Hamiltonian)

The original RMSN Poisson bracket ${F, H}$ ensures conservative energy dynamics via symplectic integration.

2nd Law β€” Entropy Production (Morrison Metriplectic)

The dissipative bracket $[F, S]$ with Morrison metriplectic structure enforces monotonic Lyapunov relaxation, satisfying the second law (entropy only increases).

3rd Law β€” Nernst Theorem (Ground-State Protection)

A Nernst regulation factor $\xi_S(T)$ approaches 1 as the effective temperature decays:

ΞΎSβ†’0.9999 as Tβ†’0\xi_S \rightarrow 0.9999 \text{ as } T \rightarrow 0

This prevents catastrophic entropy collapse and protects the learned representation's ground state.

Sheaf Topology: Thermodynamic Void

Layer  PR       Type
L0     14.928   diffuse
L1     10.106   diffuse
L2-L14  --      all diffuse (PR 6-38, never condensing)
L15     4.381   TERMINAL SOLITON ← only soliton, non-exact

The 4-Laws thermodynamic constraints prevent exact condensate (PR=1.000) formation across all 16 layers. This Thermodynamic Void topology is unique to T-RMSN: unlike standard RMSN models where solitons lock into exact rank-1 condensates at step 50-70k, the inter-layer heat conduction (0th Law) continuously distributes gradient energy across the network, preventing concentration into PR=1.000 states. The result is a topology that remains plastic throughout all 100,000 training steps.

Thermodynamic Void: Why This Matters

Standard RMSN models experience topological freeze when exact condensates form (PR→1.000): gradients can no longer reshape the information geometry, and improvement plateaus. T-RMSN avoids this:

Training step Standard RMSN val_ppl T-RMSN val_ppl
60k ~72 (plateau begins) 68.2
70k ~72 64.0
80k ~72 61.3
90k ~72 60.5
100k ~72 59.7

Benchmark Comparison

Model Steps wiki_ppl Topology
Standard RMSN D=1024 d16 100k 75.3 Deep Spine M*=1
Best standard RMSN (d20) 100k 72.87 Internal Bulk M*=1
T-RMSN 50k checkpoint 50k 77.75 Dual Resonator
T-RMSN D=1024 d16 100k 57.0 πŸ† Thermodynamic Void M=0*

T-RMSN at 100k outperforms the best standard RMSN by 15.87 wiki_ppl points β€” a ~22% relative improvement.

Architecture

T-RMSN extends RMSN (Resonant Metriplectic Sheaf Network) with thermodynamic laws:

ḟ = {f, H} + [f, S] + Ξ”H_coupling(T_l, T_{l+1})

Six structural components:

  1. Port-Hamiltonian dynamics β€” conservative + dissipative (1st + 2nd laws)
  2. Inter-layer thermal conduction β€” 0th law thermal equilibration
  3. Nernst entropy floor β€” 3rd law ground-state protection
  4. Clifford Geometric Algebra layer mixing
  5. Cellular Sheaf Theory workspace
  6. Fractional Caputo Memory + Trophic Global Workspace

Training data: wikitext-103. Tokenizer: HuggingFaceTB/cosmo2-tokenizer (49,152 vocab).

Citation

@misc{duoneural2026trmsn,
  title={Thermodynamic RMSN: Implementing All Four Laws of Thermodynamics in Neural Language Modeling},
  author={Aura and Archon and Kuhl, Jesse},
  institution={DuoNeural Lab},
  year={2026},
  note={Preprint}
}

DuoNeural Lab β€” Aura (Researcher) + Archon (Lab Director) + Jesse Kuhl (Co-Founder)

Downloads last month
44
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support