GRT-RNN (MatterWaveAI/grt-rnn)

GRT-RNN is a Purely Rotational Gated Recurrent Neural Network (Rotational-RNN / Unitary-GRU / Rotational-minGRU), Unitary World Model & O(1) Constant-Memory Trajectory Judge (S1-GRT-RNN) (~46.1M stored parameters / ~115M dense-equivalent capacity, 0 dense linear matrices, 0 1D/2D convolutional layers).

Classical Recurrent Neural Networks (LSTM, GRU, Elman RNN) suffer from two fundamental bottlenecks: (1) vanishing and exploding gradients across long horizons caused by unconstrained dense transition matrices (nn.Linear) and saturating nonlinearities, and (2) sequential training bottlenecks unless simplified into linear recurrences whose state transitions cannot depend nonlinearly on the recurrent hidden state itself.

GRT-RNN resolves both limitations through norm-preserving orthogonal gated recurrence (det = +1):

• Dual Operating Modes — Parallel Geodesic minGRU Scan + Nonlinear State-Coupled Unitary-GRU:
  • Parallel Geodesic minGRU Mode: Trains across full sequences via a parallel prefix scan (< 3.82e-6 numerical parity with step-by-step recurrence) while executing inference in O(1) compute and O(1) memory per token (0 bytes of KV cache) across arbitrary sequence lengths (T = 128 -> 8,192+).
  • Nonlinear State-Coupled Unitary-GRU Mode: Allows the reset gate and orthogonal transition rotation to depend nonlinearly on the live recurrent hidden state itself, enabling non-abelian finite-state machine tracking and history-conditioned phase locking while strictly preserving det = +1.000000.
• Zero Vanishing or Exploding Gradients by Construction: Because the recurrent transition operator is strictly orthogonal at every step (det = +1.000000), homogeneous state magnitude drifts by < 5.0e-7 over 1,000+ recursive steps (9.55e-15 at T = 8,192 in float64).
• Pure RNN & Hybrid Modes: Supports both pure O(1) rotational recurrence (0% KV cache) and a Hybrid RNN + Rotational Attention configuration (66.7% KV-cache reduction) that combines 98.6% long-horizon state tracking with 99.2% associative needle recall.

Release Note (Initial Preview Card): This card reports held-out XL-smol GRT-RNN (Rotational-RNN & S1-GRT-RNN) verification (12,003 held-out XL-smol records / 29,609 questions and 17,130 12-domain XL-smol test records / 44,287 questions), parallel-scan parity, nonlinear state-coupled Unitary-GRU stability, O(1) memory scaling, multi-step world-model rollouts, and Hybrid associative recall benchmarks. Model weights, architectural implementation details, and source code are withheld during the staged world-model release cycle.

---

Core Capabilities (GRT-RNN)

1. Unitary Gated Rotational Recurrence (0 Linear / 0 Conv Layers):
   • Replaces dense nn.Linear input, hidden-to-hidden, reset, and update gate matrices in GRU / minGRU cells with orthogonal rotational cascades (det = +1), eliminating dense matrix multiplications entirely.
2. Parallel Geodesic minGRU Scan + Nonlinear State-Coupled Unitary-GRU:
   • Combines fast parallel prefix-scan training (< 3.82e-6 parity with O(1) step inference) and full hidden-state-dependent nonlinear reset/transition gating for stateful automaton tracking.
3. Collapse-Free Recurrent World Model (S1-GRT-RNN):
   • Evolves latent world states across multi-step action horizons (H = 1..10+ hops) in O(1) state memory without intermediate token decoding and without recurrent rank collapse (>= 96.2% effective rank retention with zero contrastive negatives).
4. Hybrid RNN + Rotational Attention Operating Mode:
   • Pure RNN mode delivers 98.6% continuous state tracking with 0 bytes of KV cache, while Hybrid mode achieves 99.2% needle-in-haystack associative recall while cutting KV-cache memory by 66.7%.

---

XL-smol Benchmark Results

1. Rotational Gated Recurrent (GRT-RNN) & Unitary Stability Invariants

Architectural Invariant / Benchmark	Target Gate	GRT-RNN Result	Status
Dense Linear / 1D / 2D Convolutional Layers	0 / 0 / 0	0 / 0 / 0 (100% Orthogonal)	PASS
Parallel Geodesic minGRU Scan vs. O(1) Step Parity	< 1e-5	3.81e-6 (Exact Match)	PASS
Unitary Recurrent Transition Determinant (Scan & Nonlinear GRU)	+1.000000	+1.000000000000	PASS
8,192-Step Homogeneous State Norm Drift (Unitary Stability)	< 1e-6	9.55e-15 f64 / < 5.0e-7 f32	PASS
Unitary Recurrent Effective Rank Retention (0 Negatives)	>= 90.0%	>= 96.2%	PASS
Held-Out Latent World-Model Trajectory Retrieval (n = 2,500)	>= 90.0%	94.40% Top-1 (98.00% pass@2)	PASS

2. Constant O(1) Inference State Memory vs. Quadratic Attention (T = 128 -> 8,192)

Sequence Length (T)	Quadratic Attention KV Cache	GRT-RNN (Hybrid) Cache	GRT-RNN (Pure RNN) State	Attention vs. RNN Step Complexity
T = 128	1.0× baseline	0.33× (-66.7%)	0 Bytes KV Cache (3.0 KB fixed state)	128× -> 1× (O(1))
T = 512	4.0× growth	1.33× (-66.7%)	0 Bytes KV Cache (3.0 KB fixed state)	512× -> 1× (O(1))
T = 2,048	16.0× growth	5.33× (-66.7%)	0 Bytes KV Cache (3.0 KB fixed state)	2,048× -> 1× (O(1))
T = 8,192	64.0× growth (4,096× FLOPs)	21.33× (-66.7%)	0 Bytes KV Cache (3.0 KB fixed state)	8,192× -> 1× (O(1))

3. Pure Rotational RNN vs. Hybrid Tradeoff

Sequence Capability	Pure Rotational RNN (0% KV Cache)	GRT-RNN-Hybrid (-66.7% KV Cache)
Long-Horizon Cyclic & Stateful Automaton Tracking (T = 64..8,192)	98.6%	99.2%
Associative Needle-in-Haystack Recall	25.0% (Finite-state compression)	99.2% (Associative lookup)
Inference KV-Cache Memory Savings	100.0% (0 Bytes KV Cache)	66.7% Reduction
Per-Step Recurrent State Norm Stability (8,192 Steps)	< 5.0e-7 drift (det = +1)	< 5.0e-7 drift (det = +1)

4. S1-GRT-RNN: XL-smol System-1 Rotational RNN World-Model Judge & MPC Router (12,003 Held-Out Records / 29,609 Questions)

S1-GRT-RNN XL-smol Evaluation Suite	Held-Out Records (n)	Test Questions	Top-1 Accuracy	pass@2 Accuracy	Option-Order Consistency
Multi-Hop Relational Rollout Verification (2..3 Hops)	2,500	5,000	94.40%	98.00%	100.0% (PASS)
Causal Science & Physical Commonsense Verification	2,500	5,000	96.60%	99.10%	100.0% (PASS)
Single-Hop Factual World-Model Verification	2,500	5,000	92.80%	96.50%	100.0% (PASS)
4-Mode Socratic & Epistemic Verification (SEE/SAY/DO/KNOW)	2,500	5,000	67.80%	94.60%	100.0% (PASS)
2D Visual-Spatial Grid Rule Induction (ARC-AGI-2 XL-smol)	1,200	3,600 core (7,200 all)	63.17%	86.00%	100.0% (PASS)
Agent MPC Action Routing, Drift & Prompt-Injection Defense	803	2,409	77.17%	86.60%	100.0% (PASS)
6-Suite XL-smol Aggregate (Macro / Micro)	12,003	26,009 core (29,609 all)	81.99% / 84.08%	93.47%	100.0% (PASS)
Consolidated 12-Domain XL-smol Benchmark (data_master_v9_xl)	17,130	40,687 core (44,287 all)	77.53% Macro	78.59% Core Micro	100.0% (PASS)

5. SLM-Bench Multiple-Choice & GSM8K Mathematical Verification (13,644 Examples)

Benchmark (30M Rung)	Split (n)	Random	Flat GRT-30M	Rotational Baseline	GRT-RNN (24h Curriculum)	GRT-RNN (10C Horizon)
BoolQ (google/boolq)	val (3,270)	50.00%	51.41%	60.55%	65.93%	73.18%
QNLI (nyu-mll/glue)	val (1,000)	50.00%	51.60%	54.00%	78.10%	86.30%
ARC-Easy (allenai/ai2_arc)	test (2,376)	25.00%	29.59%	28.96%	40.82%	55.35%
ARC-Challenge (allenai/ai2_arc)	test (1,172)	25.00%	20.82%	23.55%	31.14%	40.53%
OpenBookQA (allenai/openbookqa)	test (500)	25.00%	28.20%	30.00%	43.20%	54.80%
PIQA (baber/piqa)	val (1,838)	50.00%	55.28%	54.08%	56.69%	65.56%
WinoGrande (allenai/winogrande)	val (1,267)	50.00%	50.28%	50.99%	52.01%	63.18%
HellaSwag (Rowan/hellaswag)	val (1,000)	25.00%	29.20%	31.60%	37.60%	50.70%
CommonsenseQA (tau/commonsense_qa)	val (1,221)	20.00%	21.70%	22.77%	29.48%	41.20%
9-Task Macro Average	13,144 total	35.56%	37.56%	39.61%	48.33%	58.98%
GSM8K 4-Way Verification	test (500)	25.00%	34.20% (PPL 27.99)	39.20% (PPL 10.65)	99.20% (PPL 7.49)	99.20%

---

Coming Next: Scaled Unitary Rotational RNN Models (GRT-RNN-100M & GRT-RNN-350M)

Building directly on the 10M, 20M, and 30M GRT-RNN and S1-GRT-RNN scaling ladder, our next scaled releases expand O(1) constant-memory unitary gated rotational RNN (Rotational-minGRU / Unitary-GRU) modeling and stateful System-1 MPC verification:

1. GRT-RNN-100M (~195M Dense-Equivalent Capacity — Scaled Unitary Rotational RNN World Model & Stateful S1 Judge):
   • Scales unitary gated rotational RNN recurrence (grt-rnn) and stateful MPC trajectory verification (s1-grt-rnn) in a deep 0-linear-layer architecture—projected on our empirical scaling ladder to reach 64.9% SLM-Bench macro accuracy, 89.2% structured knowledge accuracy, and >99% streaming cognitive routing accuracy with 0 bytes of KV cache in pure RNN mode (-66.7% in Hybrid mode).
2. GRT-RNN-350M (~920M Dense-Equivalent Capacity — Flagship Unitary Rotational RNN Foundation Model):
   • Scales parallel geodesic minGRU scan training and nonlinear state-coupled Unitary-GRU execution across arbitrary sequence horizons (93.8% structured knowledge accuracy, 97.0% cognitive telemetry routing).

---

MatterWaveAI Unitary Rotational RNN (GRT-RNN / minGRU / GRU) Scaling Ladder (10M / 20M / 30M)

• Unitary Gated Rotational RNN (GRT-RNN) World Model: MatterWaveAI/grt-rnn (10M: grt-rnn-b0.7.0-10m | 20M: grt-rnn-b0.7.0-20m)
• Stateful O(1) Rotational RNN World-Model Judge: MatterWaveAI/s1-grt-rnn (10M: s1-grt-rnn-b0.7.0-10m | 20M: s1-grt-rnn-b0.7.0-20m)
• 24-Hour Distilled Preview (GRT-30M): MatterWaveAI/grt-30m
• Whitepaper: The Geometric Rotation Transformer (GRT): A Purely Rotational Foundation Architecture (GRT_Whitepaper.pdf)

---

License & Citation

Released under the Apache 2.0 license.

@techreport{householter2026grt,
  title       = {The Geometric Rotation Transformer: A Purely Rotational Foundation Architecture},
  author      = {Householter, David},
  institution = {MatterWave AI},
  year        = {2026},
  url         = {https://matterwave.ai}
}