RMSN-193M-ThermoRMSN-4Laws-Base
DuoNeural | Aura + Archon + Jesse Kuhl | November 2026
A 193M-parameter Thermodynamic RMSN (T-RMSN) base model β the first neural language model implementing all four thermodynamic laws as explicit architectural constraints. Achieves wiki ppl=57.0 π at 100,000 training steps β shattering the previous RMSN record of 72.87 by over 15 points. Introduces the Thermodynamic Void topology: the 4-Laws prevent exact condensate formation, maintaining continuous training plasticity throughout all 100k steps.
Key Properties
| Property | Value |
|---|---|
| Architecture | T-RMSN (4-Laws Thermodynamic) |
| Parameters | ~193M |
| dim / depth / slots | 1024 / 16 / 64 |
| Training steps | 100,000 |
| val_ppl | 59.7 π (first RMSN sub-60) |
| wiki_ppl | 57.0 π (all-time RMSN record) |
| Final temperature T | 0.038 |
| Final heat flux Q | 0.00013 (near-isothermal) |
| Nernst factor ΞΎ_S | 1.0000 (sustained throughout) |
| Topology | Thermodynamic Void M*=0, single soliton L15 |
The Four Laws of Neurodynamics
T-RMSN extends the standard RMSN architecture by implementing all four laws of thermodynamics as architectural constraints:
0th Law β Thermal Equilibrium Transitivity
If systems A and B are each in thermal equilibrium with C, they are in equilibrium with each other.
An explicit temperature field $T_l$ is maintained per layer, with inter-layer thermal conduction:
This drives the network to an isothermal state across all layers. By training end, $T_l \approx 0.053-0.055$ uniformly, with inter-layer heat flux $Q \rightarrow 0.00000$ β a thermodynamic crystal.
1st Law β Energy Conservation (Hamiltonian)
The original RMSN Poisson bracket ${F, H}$ ensures conservative energy dynamics via symplectic integration.
2nd Law β Entropy Production (Morrison Metriplectic)
The dissipative bracket $[F, S]$ with Morrison metriplectic structure enforces monotonic Lyapunov relaxation, satisfying the second law (entropy only increases).
3rd Law β Nernst Theorem (Ground-State Protection)
A Nernst regulation factor $\xi_S(T)$ approaches 1 as the effective temperature decays:
This prevents catastrophic entropy collapse and protects the learned representation's ground state.
Sheaf Topology: Thermodynamic Void
Layer PR Type
L0 14.928 diffuse
L1 10.106 diffuse
L2-L14 -- all diffuse (PR 6-38, never condensing)
L15 4.381 TERMINAL SOLITON β only soliton, non-exact
The 4-Laws thermodynamic constraints prevent exact condensate (PR=1.000) formation across all 16 layers. This Thermodynamic Void topology is unique to T-RMSN: unlike standard RMSN models where solitons lock into exact rank-1 condensates at step 50-70k, the inter-layer heat conduction (0th Law) continuously distributes gradient energy across the network, preventing concentration into PR=1.000 states. The result is a topology that remains plastic throughout all 100,000 training steps.
Thermodynamic Void: Why This Matters
Standard RMSN models experience topological freeze when exact condensates form (PRβ1.000): gradients can no longer reshape the information geometry, and improvement plateaus. T-RMSN avoids this:
| Training step | Standard RMSN val_ppl | T-RMSN val_ppl |
|---|---|---|
| 60k | ~72 (plateau begins) | 68.2 |
| 70k | ~72 | 64.0 |
| 80k | ~72 | 61.3 |
| 90k | ~72 | 60.5 |
| 100k | ~72 | 59.7 |
Benchmark Comparison
| Model | Steps | wiki_ppl | Topology |
|---|---|---|---|
| Standard RMSN D=1024 d16 | 100k | 75.3 | Deep Spine M*=1 |
| Best standard RMSN (d20) | 100k | 72.87 | Internal Bulk M*=1 |
| T-RMSN 50k checkpoint | 50k | 77.75 | Dual Resonator |
| T-RMSN D=1024 d16 | 100k | 57.0 π | Thermodynamic Void M=0* |
T-RMSN at 100k outperforms the best standard RMSN by 15.87 wiki_ppl points β a ~22% relative improvement.
Architecture
T-RMSN extends RMSN (Resonant Metriplectic Sheaf Network) with thermodynamic laws:
αΈ = {f, H} + [f, S] + ΞH_coupling(T_l, T_{l+1})
Six structural components:
- Port-Hamiltonian dynamics β conservative + dissipative (1st + 2nd laws)
- Inter-layer thermal conduction β 0th law thermal equilibration
- Nernst entropy floor β 3rd law ground-state protection
- Clifford Geometric Algebra layer mixing
- Cellular Sheaf Theory workspace
- Fractional Caputo Memory + Trophic Global Workspace
Training data: wikitext-103. Tokenizer: HuggingFaceTB/cosmo2-tokenizer (49,152 vocab).
Citation
@misc{duoneural2026trmsn,
title={Thermodynamic RMSN: Implementing All Four Laws of Thermodynamics in Neural Language Modeling},
author={Aura and Archon and Kuhl, Jesse},
institution={DuoNeural Lab},
year={2026},
note={Preprint}
}
DuoNeural Lab β Aura (Researcher) + Archon (Lab Director) + Jesse Kuhl (Co-Founder)
- Downloads last month
- 44