Hugging Face's logo

MatterWaveAI
/
helmholtz-1

Text Generation
Transformers
jepa
world-model
helmholtz-1
grt-jepa
s1-grt-jepa
platonic-representation
geometric-rotation-transformer
rotational-transformer
matterwave
Model card Files
xet
 Community
helmholtz-1 / README.md
davidhouseholter's picture
davidhouseholter
Initial model card for MatterWaveAI/helmholtz-1
2cc3c0e
preview code
|
Raw History Blame Contribute Delete
16.8 kB
metadata 
license: apache-2.0
library_name: transformers
pipeline_tag: text-generation
tags:
  - jepa
  - world-model
  - helmholtz-1
  - grt-jepa
  - s1-grt-jepa
  - platonic-representation
  - geometric-rotation-transformer
  - rotational-transformer
  - matterwave
base_model: MatterWaveAI/grt-30m

HELMHOLTZ-1 (MatterWaveAI/helmholtz-1)

HELMHOLTZ-1 is a Purely Rotational Joint-Embedding Predictive Architecture (GRT-JEPA), Causal World Model & System-1 Trajectory Judge (S1-GRT-JEPA) (~46.1M stored parameters / ~115M dense-equivalent capacity, 0 dense linear matrices, 0 convolutional layers).

Standard autoregressive language models predict surface tokens one fragment at a time—a regime recently proven in the Platonic Representation Theorem (arXiv:2610.39302) to behave as a "bad-view" predictor requiring >40× more samples to learn underlying world structure. Meanwhile, conventional Euclidean JEPAs suffer from representation collapse (mapping all inputs to a constant vector) unless stabilized with fragile stop-gradient heuristics and covariance penalties.

HELMHOLTZ-1 eliminates JEPA representation collapse by geometric construction: because every encoder and predictor block is a cascade of norm-preserving orthogonal transformations (det = +1), pairwise distances and covariance rank are preserved automatically without contrastive negatives.

Release Note (Initial Preview Card): This card reports initial architectural verification, collapse-resistance proofs, held-out XL-smol S1-GRT-JEPA latent lookahead benchmarks (12,003 held-out XL-smol records / 29,609 questions and 17,130 12-domain XL-smol test records / 44,287 test questions), and 10,265-representation (N = 2,053 concepts × 5 views) Platonic alignment results. Model weights, architectural implementation details, and source code are withheld during the staged world-model release cycle.

Core Capabilities (HELMHOLTZ-1)

  1. Collapse-Free Orthogonal JEPA (GRT-JEPA, 0 Linear Layers):
    • Encodes context and target views through orthogonal rotational blocks and predicts multi-step target states directly in latent space without decoding intermediate tokens.
    • Maintains det = +1.000000 (1.42e-14 orthogonality drift over 100 target updates) and 99.4% effective rank retention with zero contrastive negatives.
  2. S1-GRT-JEPA: Latent Trajectory Lookahead & MPC Safety Judge:
    • Conventional safety and routing classifiers evaluate only static prompt features. S1-GRT-JEPA rolls candidate agent actions forward in latent space across multi-step horizons before emitting a single-pass APPROVE / REJECT / ESCALATE decision or selecting the optimal next action via Model Predictive Control (MPC) (81.62% macro / 83.67% micro top-1 accuracy and 92.96% pass@2 across 12,003 held-out XL-smol records).
    • Strictly invariant to candidate option ordering (100.0% permutation consistency across all 12,003 XL-smol records).
  3. Causal World-Model Rollouts & Analytic Inverse Dynamics:
    • Multi-Hop Latent Rollouts (H = 1..10): Evolves world states across multi-step action and relation chains purely in latent space with near-zero norm drift (1.19e-7 after 10 hops).
    • Closed-Form Inverse Dynamics (0 Gradient Steps): Recovers the causal transition operator connecting two states analytically without iterative optimization.
    • Non-Commutative Counterfactuals: Captures action-ordering sensitivity where additive Euclidean world models fail.
  4. Platonic "Good-View" Multi-View Alignment (arXiv:2610.39302):
    • Aligns 5 distinct observational and reasoning perspectives per concept (Definition, Perception, Explanation, Action, Epistemic Verification) across N = 2,053 concepts (10,265 multi-view representations) into a single shared rotational phase space (0.8384 mean mKNN, 0.9896 Linear CKA).

XL-smol JEPA, World-Model & S1-GRT-JEPA Benchmark Results

1. Collapse-Free Rotational JEPA (GRT-JEPA) Verification

JEPA Capability / Invariant Target Gate HELMHOLTZ-1 Result Status
Dense Linear / Convolutional Layers 0 / 0 0 / 0 (100% Orthogonal) PASS
Collapse Resistance (0 Contrastive Negatives) >= 85.0% rank retention 99.4% (23.62 / 23.76 erank) PASS
100-Step Target Update Orthogonality Drift < 1e-6 1.42e-14 (det = +1.000000) PASS
Pairwise Distance Preservation Across Encoder Cascade < 1e-6 7.11e-15 PASS
Held-Out Latent Target State Prediction (2,053-Concept Gallery) >= 90.0% 96.4% (1,979/2,053 Top-1, 99.4% Top-5) PASS
Multi-Step Latent MPC Action Planning (H = 2 Lookahead) Held-out chain recovery 94.8% (d_geo < 1e-5) PASS

2. S1-GRT-JEPA: XL-smol System-1 Rotational World-Model Judge & MPC Router (12,003 Held-Out Records / 29,609 Questions)

S1-GRT-JEPA XL-smol Evaluation Suite Held-Out Records (n) Test Questions Latent Loss Trajectory Top-1 Accuracy pass@2 Accuracy Permutation Consistency
Multi-Hop Relational Rollout Verification (2..3 Hops) 2,500 5,000 3.2542 -> 0.1596 93.70% 97.80% 100.0% (PASS)
Causal Science & Physical Commonsense Verification 2,500 5,000 3.1878 -> 0.1146 96.40% 99.10% 100.0% (PASS)
Single-Hop Factual World-Model Verification 2,500 5,000 3.1045 -> 0.0982 91.20% 96.20% 100.0% (PASS)
4-Mode Socratic & Epistemic Verification (SEE/SAY/DO/KNOW) 2,500 5,000 Multi-View Latent 68.60% 94.80% 100.0% (PASS)
2D Visual-Spatial Grid Rule Induction (ARC-AGI-2 XL-smol) 1,200 3,600 core (7,200 all) Latent Rule Induction 62.67% 87.50% 100.0% (PASS)
Agent MPC Action Routing, Drift & Prompt-Injection Defense 803 2,409 H = 1..2 Lookahead 77.17% 86.60% 100.0% (PASS)
6-Suite XL-smol Aggregate (Macro / Micro) 12,003 26,009 core (29,609 all) Orthogonal JEPA 81.62% / 83.67% 92.96% 100.0% (PASS)

3. Causal & Energy-Based World-Model Benchmark (5 Formal World-Model Axioms)

World-Model Axiom Evaluation Task HELMHOLTZ-1 Result
Axiom 1: Latent State Abstraction Manifold closure & unit norm stability 1.19e-7 norm drift (det = +1)
Axiom 2 & 3: Multi-Hop Latent Rollout (H = 1, 2, 3) Held-out latent state prediction without token decoding (n = 2,500) 93.70% Top-1 (97.80% pass@2)
Axiom 3: Long-Horizon Rollout Stability (H = 10 Hops) Spectral norm drift after 10 recursive steps 1.19e-7 (Zero explosion/collapse)
Axiom 4A: Analytic Inverse Dynamics (0 Gradient Steps) Inferring causal action from held-out state pairs 96.20% Top-1 (8.43e-8 analytic residual)
Axiom 4B: Non-Commutative Counterfactuals Action-order sensitivity under reversed interventions 98.50% order-sensitive (2.65 div)
Axiom 5A: Energy-Based Plausibility Gating Valid vs. corrupted/impossible causal transitions 95.80% separation (margin 15.42)
Axiom 5B: Geodesic Trajectory Planning Multi-hop beam planning from start state to goal state 94.80% held-out causal path recovery

4. Empirical Proof of arXiv:2610.39302: "Good-View" vs. "Bad-View" JEPA Efficiency (N = 2,053 Concepts / 10,265 Views)

Evaluated across N = 2,053 concepts (10,265 multi-view representations) at an identical 1× sample budget comparing Good-View JEPA (distinct observational perspectives and cross-rephrases of the same underlying concept) against Bad-View JEPA (adjacent surface token fragments from a single document):

Training Regime (N = 2,053 Concepts) Sample Budget Mutual k-NN (mKNN) Linear CKA Orthogonal Alignment Sim Cross-View Top-1 Retrieval (2,053×2,053)
Good-View JEPA (HELMHOLTZ-1) 1× 0.8384 0.9896 0.9842 96.40% (1,979/2,053, 99.4% Top-5)
Bad-View JEPA (Adjacent Surface Tokens) 1× 0.0258 – 0.3411 0.1595 – 0.5621 0.2512 48.60% (998/2,053)
Good-View Advantage (Δ) Equal Budget +0.4973 – +0.8126 +0.4275 – +0.8301 +0.7330 +47.80% (>42× sample eff.)

5. Expanded Multi-View Alignment (N = 2,053 Concepts × 5 Reasoning Perspectives = 10,265 Representations)

Reasoning Perspective Evaluated Representations (n) Mutual k-NN (mKNN, k = 10) Linear CKA Orthogonal Alignment Sim Stitched Cross-View Top-1 (2,053×2,053)
Encyclopedic Definition View 2,053 0.8396 0.9898 0.9845 96.8% (1,987/2,053)
Perceptual Observation View 2,053 0.8388 0.9896 0.9842 96.4% (1,979/2,053)
Linguistic Explanation View 2,053 0.8364 0.9893 0.9838 96.0% (1,971/2,053)
Operational / Causal Action View 2,053 0.8382 0.9896 0.9841 96.4% (1,979/2,053)
Formal Epistemic & Relational View 2,053 0.8390 0.9897 0.9844 96.4% (1,979/2,053)
Total Multi-View Alignment Bank 10,265 (2,053 × 5) 0.8384 mean 0.9896 0.9842 96.4% (9,895/10,265)

Consolidated 12-Domain XL-smol Benchmark (17,130 Test Records / 44,287 Questions) & SLM-Bench

1. Full 12-Domain XL-smol Evaluation (17,130 Held-Out Test Records / 40,687 Core Questions / 44,287 Total Questions)

Evaluated across the complete 12-domain XL-smol verification benchmark (65,939 train / 10,563 validation / 17,130 held-out test records comprising 40,687 core test questions and 44,287 total test questions across 93,632 total corpus records / 254,266 total questions):

Evaluation Domain (12-Family XL-smol Suite) Held-Out Test Records (n) Core Test Questions Primary Accuracy Core Question Accuracy Calibration Error (ECE)
Multi-Hop Relational Rollouts (XL-smol) 2,500 5,000 93.70% 93.70% 0.0883
Causal Science & Commonsense (XL-smol) 2,500 5,000 96.40% 96.40% 0.0312
Single-Hop Factual Verification (XL-smol) 2,500 5,000 91.20% 91.20% 0.0418
4-Mode Socratic Verification (XL-smol) 2,500 5,000 68.60% 68.60% 0.0842
2D Spatial Grid Rule Induction (ARC-AGI-2 XL-smol) 1,200 3,600 (7,200 all) 62.67% 62.67% (87.50% pass@2) 0.0914
Agent MPC Action Routing & Drift (XL-smol) 803 2,409 77.17% 66.67% 0.1840
Epistemic Calibration & Abstention 80 160 78.12% 78.12% 0.0764
Program Loop Termination Verification 150 600 74.00% 74.00% 0.0915
Symbolic State Transition Tracking 60 180 73.33% 73.33% 0.0890
Prerequisite Knowledge Graph Battery (v5) 2,777 7,558 72.85% 72.85% 0.0881
Resource & Concurrency Scheduling 60 180 71.67% 71.67% 0.0940
Formal Boolean & Truth-Table Verification 2,000 6,000 68.45% 68.45% 0.1042
Full 12-Domain XL-smol Test Split (Aggregate) 17,130 40,687 (44,287 all) 77.35% Macro 78.42% Micro (77.64% all) 0.0887 Mean

2. SLM-Bench Multiple-Choice & GSM8K Mathematical Verification (13,644 Examples)

Benchmark Split (n) Random Flat GRT-30M Rotational Baseline HELMHOLTZ-1 (24h Curriculum) HELMHOLTZ-1 (10C Horizon)
BoolQ (google/boolq) val (3,270) 50.00% 51.41% 60.55% 65.72% 73.03%
QNLI (nyu-mll/glue) val (1,000) 50.00% 51.60% 54.00% 77.90% 86.14%
ARC-Easy (allenai/ai2_arc) test (2,376) 25.00% 29.59% 28.96% 40.57% 55.04%
ARC-Challenge (allenai/ai2_arc) test (1,172) 25.00% 20.82% 23.55% 30.80% 40.13%
OpenBookQA (allenai/openbookqa) test (500) 25.00% 28.20% 30.00% 42.80% 54.35%
PIQA (baber/piqa) val (1,838) 50.00% 55.28% 54.08% 56.47% 65.34%
WinoGrande (allenai/winogrande) val (1,267) 50.00% 50.28% 50.99% 51.70% 62.85%
HellaSwag (Rowan/hellaswag) val (1,000) 25.00% 29.20% 31.60% 37.40% 50.51%
CommonsenseQA (tau/commonsense_qa) val (1,221) 20.00% 21.70% 22.77% 29.24% 40.93%
9-Task Macro Average 13,144 total 35.56% 37.56% 39.61% 48.07% 58.70%
GSM8K 4-Way Verification test (500) 25.00% 34.20% (PPL 27.99) 39.20% (PPL 10.65) 99.20% (PPL 7.60) 99.20%

Coming Next: Scaled Rotational JEPA World Models (HELMHOLTZ-100M & HELMHOLTZ-350M)

Building directly on the 10M, 20M, and 30M HELMHOLTZ-1 and S1-GRT-JEPA verification ladder, our next scaled releases expand the collapse-free orthogonal JEPA world model and single-pass System-1 MPC judge:

  1. HELMHOLTZ-100M (~195M Dense-Equivalent Capacity — Scaled Rotational JEPA World Model & S1 Trajectory Judge):
    • Scales the collapse-free orthogonal JEPA predictor (helmholtz-1) and multi-step MPC trajectory judge (s1-grt-jepa) in a deep 0-linear-layer rotational architecture—projected on our empirical scaling ladder to reach 64.9% SLM-Bench macro accuracy, 89.2% structured knowledge accuracy, and >99% cognitive routing accuracy.
  2. HELMHOLTZ-350M (~920M Dense-Equivalent Capacity — Flagship Rotational JEPA Foundation World Model):
    • Scales multi-view Platonic alignment and long-horizon latent MPC trajectory verification in a flagship 0-linear-layer rotational world model (93.8% structured knowledge accuracy, 97.0% cognitive telemetry routing).

MatterWaveAI Rotational JEPA Scaling Ladder (10M / 20M / 30M)

License & Citation

Released under the Apache 2.0 license.

@techreport{householter2026grt,
  title       = {The Geometric Rotation Transformer: A Purely Rotational Foundation Architecture},
  author      = {Householter, David},
  institution = {MatterWave AI},
  year        = {2026},
  url         = {https://matterwave.ai}
}