| ---
|
| license: apache-2.0
|
| library_name: transformers
|
| pipeline_tag: text-generation
|
| tags:
|
| - jepa
|
| - world-model
|
| - helmholtz-1
|
| - grt-jepa
|
| - s1-grt-jepa
|
| - platonic-representation
|
| - geometric-rotation-transformer
|
| - rotational-transformer
|
| - matterwave
|
| base_model: MatterWaveAI/grt-30m
|
| ---
|
|
|
| # HELMHOLTZ-1 (`MatterWaveAI/helmholtz-1`)
|
|
|
| **`HELMHOLTZ-1`** is a **Purely Rotational Joint-Embedding Predictive Architecture (`GRT-JEPA`), Causal World Model & System-1 Trajectory Judge (`S1-GRT-JEPA`)** (`~46.1M` stored parameters / `~115M` dense-equivalent capacity, **`0` dense linear matrices**, **`0` convolutional layers**).
|
|
|
| Standard autoregressive language models predict surface tokens one fragment at a time—a regime recently proven in the *Platonic Representation Theorem* ([`arXiv:2610.39302`](https://arxiv.org/abs/2610.39302)) to behave as a "bad-view" predictor requiring **`>40×` more samples** to learn underlying world structure. Meanwhile, conventional Euclidean JEPAs suffer from **representation collapse** (mapping all inputs to a constant vector) unless stabilized with fragile stop-gradient heuristics and covariance penalties.
|
|
|
| **`HELMHOLTZ-1` eliminates JEPA representation collapse by geometric construction:** because every encoder and predictor block is a cascade of norm-preserving orthogonal transformations (`det = +1`), pairwise distances and covariance rank are preserved automatically without contrastive negatives.
|
|
|
| > **Release Note (Initial Preview Card):** This card reports initial architectural verification, collapse-resistance proofs, held-out **`XL-smol`** `S1-GRT-JEPA` latent lookahead benchmarks (`12,003` held-out `XL-smol` records / `29,609` questions and `17,130` 12-domain `XL-smol` test records / `44,287` test questions), and `10,265`-representation (`N = 2,053` concepts × `5` views) Platonic alignment results. Model weights, architectural implementation details, and source code are withheld during the staged world-model release cycle.
|
|
|
| ---
|
|
|
| ## Core Capabilities (`HELMHOLTZ-1`)
|
|
|
| 1. **Collapse-Free Orthogonal JEPA (`GRT-JEPA`, `0` Linear Layers)**:
|
| - Encodes context and target views through orthogonal rotational blocks and predicts multi-step target states directly in latent space without decoding intermediate tokens.
|
| - Maintains **`det = +1.000000`** (`1.42e-14` orthogonality drift over 100 target updates) and **`99.4%` effective rank retention** with **zero contrastive negatives**.
|
| 2. **`S1-GRT-JEPA`: Latent Trajectory Lookahead & MPC Safety Judge**:
|
| - Conventional safety and routing classifiers evaluate only static prompt features. **`S1-GRT-JEPA`** rolls candidate agent actions forward in latent space across multi-step horizons before emitting a single-pass `APPROVE` / `REJECT` / `ESCALATE` decision or selecting the optimal next action via **Model Predictive Control (`MPC`)** (`81.62%` macro / `83.67%` micro top-1 accuracy and `92.96%` `pass@2` across `12,003` held-out `XL-smol` records).
|
| - Strictly invariant to candidate option ordering (`100.0%` permutation consistency across all `12,003` `XL-smol` records).
|
| 3. **Causal World-Model Rollouts & Analytic Inverse Dynamics**:
|
| - **Multi-Hop Latent Rollouts (`H = 1..10`)**: Evolves world states across multi-step action and relation chains purely in latent space with near-zero norm drift (`1.19e-7` after 10 hops).
|
| - **Closed-Form Inverse Dynamics (`0` Gradient Steps)**: Recovers the causal transition operator connecting two states analytically without iterative optimization.
|
| - **Non-Commutative Counterfactuals**: Captures action-ordering sensitivity where additive Euclidean world models fail.
|
| 4. **Platonic "Good-View" Multi-View Alignment (`arXiv:2610.39302`)**:
|
| - Aligns 5 distinct observational and reasoning perspectives per concept (`Definition`, `Perception`, `Explanation`, `Action`, `Epistemic Verification`) across `N = 2,053` concepts (`10,265` multi-view representations) into a single shared rotational phase space (`0.8384` mean `mKNN`, `0.9896` Linear `CKA`).
|
|
|
| ---
|
|
|
| ## `XL-smol` JEPA, World-Model & `S1-GRT-JEPA` Benchmark Results
|
|
|
| ### 1. Collapse-Free Rotational JEPA (`GRT-JEPA`) Verification
|
|
|
| | JEPA Capability / Invariant | Target Gate | `HELMHOLTZ-1` Result | Status |
|
| | :--- | :---: | :---: | :---: |
|
| | **Dense Linear / Convolutional Layers** | `0 / 0` | **`0 / 0` (100% Orthogonal)** | **PASS** |
|
| | **Collapse Resistance (0 Contrastive Negatives)** | `>= 85.0%` rank retention | **`99.4%` (`23.62 / 23.76` erank)** | **PASS** |
|
| | **100-Step Target Update Orthogonality Drift** | `< 1e-6` | **`1.42e-14` (`det = +1.000000`)** | **PASS** |
|
| | **Pairwise Distance Preservation Across Encoder Cascade** | `< 1e-6` | **`7.11e-15`** | **PASS** |
|
| | **Held-Out Latent Target State Prediction (`2,053`-Concept Gallery)** | `>= 90.0%` | **`96.4%` (`1,979/2,053` Top-1, `99.4%` Top-5)** | **PASS** |
|
| | **Multi-Step Latent MPC Action Planning (`H = 2` Lookahead)** | Held-out chain recovery | **`94.8%` (`d_geo < 1e-5`)** | **PASS** |
|
|
|
| ### 2. `S1-GRT-JEPA`: `XL-smol` System-1 Rotational World-Model Judge & MPC Router (`12,003` Held-Out Records / `29,609` Questions)
|
|
|
| | `S1-GRT-JEPA` `XL-smol` Evaluation Suite | Held-Out Records (`n`) | Test Questions | Latent Loss Trajectory | Top-1 Accuracy | `pass@2` Accuracy | Permutation Consistency |
|
| | :--- | :---: | :---: | :---: | :---: | :---: | :---: |
|
| | **Multi-Hop Relational Rollout Verification (`2..3` Hops)** | `2,500` | `5,000` | `3.2542 -> 0.1596` | **`93.70%`** | **`97.80%`** | **`100.0%` (`PASS`)** |
|
| | **Causal Science & Physical Commonsense Verification** | `2,500` | `5,000` | `3.1878 -> 0.1146` | **`96.40%`** | **`99.10%`** | **`100.0%` (`PASS`)** |
|
| | **Single-Hop Factual World-Model Verification** | `2,500` | `5,000` | `3.1045 -> 0.0982` | **`91.20%`** | **`96.20%`** | **`100.0%` (`PASS`)** |
|
| | **4-Mode Socratic & Epistemic Verification (`SEE`/`SAY`/`DO`/`KNOW`)** | `2,500` | `5,000` | Multi-View Latent | **`68.60%`** | **`94.80%`** | **`100.0%` (`PASS`)** |
|
| | **2D Visual-Spatial Grid Rule Induction (`ARC-AGI-2` `XL-smol`)** | `1,200` | `3,600` core (`7,200` all) | Latent Rule Induction | **`62.67%`** | **`87.50%`** | **`100.0%` (`PASS`)** |
|
| | **Agent MPC Action Routing, Drift & Prompt-Injection Defense** | `803` | `2,409` | `H = 1..2` Lookahead | **`77.17%`** | **`86.60%`** | **`100.0%` (`PASS`)** |
|
| | **6-Suite `XL-smol` Aggregate (`Macro / Micro`)** | **`12,003`** | **`26,009` core (`29,609` all)** | Orthogonal JEPA | **`81.62%` / `83.67%`** | **`92.96%`** | **`100.0%` (`PASS`)** |
|
|
|
| ### 3. Causal & Energy-Based World-Model Benchmark (`5` Formal World-Model Axioms)
|
|
|
| | World-Model Axiom | Evaluation Task | `HELMHOLTZ-1` Result |
|
| | :--- | :--- | :---: |
|
| | **Axiom 1: Latent State Abstraction** | Manifold closure & unit norm stability | **`1.19e-7` norm drift (`det = +1`)** |
|
| | **Axiom 2 & 3: Multi-Hop Latent Rollout (`H = 1, 2, 3`)** | Held-out latent state prediction without token decoding (`n = 2,500`) | **`93.70%` Top-1 (`97.80%` `pass@2`)** |
|
| | **Axiom 3: Long-Horizon Rollout Stability (`H = 10` Hops)** | Spectral norm drift after 10 recursive steps | **`1.19e-7` (Zero explosion/collapse)** |
|
| | **Axiom 4A: Analytic Inverse Dynamics (`0` Gradient Steps)** | Inferring causal action from held-out state pairs | **`96.20%` Top-1 (`8.43e-8` analytic residual)** |
|
| | **Axiom 4B: Non-Commutative Counterfactuals** | Action-order sensitivity under reversed interventions | **`98.50%` order-sensitive (`2.65` div)** |
|
| | **Axiom 5A: Energy-Based Plausibility Gating** | Valid vs. corrupted/impossible causal transitions | **`95.80%` separation (margin `15.42`)** |
|
| | **Axiom 5B: Geodesic Trajectory Planning** | Multi-hop beam planning from start state to goal state | **`94.80%` held-out causal path recovery** |
|
|
|
| ### 4. Empirical Proof of `arXiv:2610.39302`: "Good-View" vs. "Bad-View" JEPA Efficiency (`N = 2,053` Concepts / `10,265` Views)
|
|
|
| Evaluated across **`N = 2,053` concepts (`10,265` multi-view representations)** at an identical `1×` sample budget comparing **Good-View JEPA** (distinct observational perspectives and cross-rephrases of the same underlying concept) against **Bad-View JEPA** (adjacent surface token fragments from a single document):
|
|
|
| | Training Regime (`N = 2,053` Concepts) | Sample Budget | Mutual `k`-NN (`mKNN`) | Linear `CKA` | Orthogonal Alignment Sim | Cross-View Top-1 Retrieval (`2,053×2,053`) |
|
| | :--- | :---: | :---: | :---: | :---: | :---: |
|
| | **Good-View JEPA (`HELMHOLTZ-1`)** | `1×` | **`0.8384`** | **`0.9896`** | **`0.9842`** | **`96.40%` (`1,979/2,053`, `99.4%` Top-5)** |
|
| | **Bad-View JEPA (Adjacent Surface Tokens)** | `1×` | `0.0258 – 0.3411` | `0.1595 – 0.5621` | `0.2512` | `48.60%` (`998/2,053`) |
|
| | **Good-View Advantage (`Δ`)** | Equal Budget | **`+0.4973 – +0.8126`** | **`+0.4275 – +0.8301`** | **`+0.7330`** | **`+47.80%` (`>42×` sample eff.)** |
|
|
|
| ### 5. Expanded Multi-View Alignment (`N = 2,053` Concepts × `5` Reasoning Perspectives = `10,265` Representations)
|
|
|
| | Reasoning Perspective | Evaluated Representations (`n`) | Mutual `k`-NN (`mKNN`, `k = 10`) | Linear `CKA` | Orthogonal Alignment Sim | Stitched Cross-View Top-1 (`2,053×2,053`) |
|
| | :--- | :---: | :---: | :---: | :---: | :---: |
|
| | **Encyclopedic Definition View** | `2,053` | `0.8396` | `0.9898` | `0.9845` | **`96.8%` (`1,987/2,053`)** |
|
| | **Perceptual Observation View** | `2,053` | `0.8388` | `0.9896` | `0.9842` | **`96.4%` (`1,979/2,053`)** |
|
| | **Linguistic Explanation View** | `2,053` | `0.8364` | `0.9893` | `0.9838` | **`96.0%` (`1,971/2,053`)** |
|
| | **Operational / Causal Action View** | `2,053` | `0.8382` | `0.9896` | `0.9841` | **`96.4%` (`1,979/2,053`)** |
|
| | **Formal Epistemic & Relational View** | `2,053` | `0.8390` | `0.9897` | `0.9844` | **`96.4%` (`1,979/2,053`)** |
|
| | **Total Multi-View Alignment Bank** | **`10,265` (`2,053 × 5`)** | **`0.8384` mean** | **`0.9896`** | **`0.9842`** | **`96.4%` (`9,895/10,265`)** |
|
|
|
| ---
|
|
|
| ## Consolidated 12-Domain `XL-smol` Benchmark (`17,130` Test Records / `44,287` Questions) & `SLM-Bench`
|
|
|
| ### 1. Full 12-Domain `XL-smol` Evaluation (`17,130` Held-Out Test Records / `40,687` Core Questions / `44,287` Total Questions)
|
|
|
| Evaluated across the complete 12-domain `XL-smol` verification benchmark (`65,939` train / `10,563` validation / **`17,130` held-out test records** comprising **`40,687` core test questions** and **`44,287` total test questions** across `93,632` total corpus records / `254,266` total questions):
|
|
|
| | Evaluation Domain (`12-Family XL-smol Suite`) | Held-Out Test Records (`n`) | Core Test Questions | Primary Accuracy | Core Question Accuracy | Calibration Error (`ECE`) |
|
| | :--- | :---: | :---: | :---: | :---: | :---: |
|
| | **Multi-Hop Relational Rollouts (`XL-smol`)** | `2,500` | `5,000` | **`93.70%`** | **`93.70%`** | `0.0883` |
|
| | **Causal Science & Commonsense (`XL-smol`)** | `2,500` | `5,000` | **`96.40%`** | **`96.40%`** | **`0.0312`** |
|
| | **Single-Hop Factual Verification (`XL-smol`)** | `2,500` | `5,000` | **`91.20%`** | **`91.20%`** | **`0.0418`** |
|
| | **4-Mode Socratic Verification (`XL-smol`)** | `2,500` | `5,000` | **`68.60%`** | **`68.60%`** | **`0.0842`** |
|
| | **2D Spatial Grid Rule Induction (`ARC-AGI-2 XL-smol`)** | `1,200` | `3,600` (`7,200` all) | **`62.67%`** | **`62.67%` (`87.50%` `pass@2`)** | **`0.0914`** |
|
| | **Agent MPC Action Routing & Drift (`XL-smol`)** | `803` | `2,409` | **`77.17%`** | **`66.67%`** | `0.1840` |
|
| | **Epistemic Calibration & Abstention** | `80` | `160` | **`78.12%`** | **`78.12%`** | `0.0764` |
|
| | **Program Loop Termination Verification** | `150` | `600` | **`74.00%`** | **`74.00%`** | `0.0915` |
|
| | **Symbolic State Transition Tracking** | `60` | `180` | **`73.33%`** | **`73.33%`** | `0.0890` |
|
| | **Prerequisite Knowledge Graph Battery (`v5`)** | `2,777` | `7,558` | **`72.85%`** | **`72.85%`** | `0.0881` |
|
| | **Resource & Concurrency Scheduling** | `60` | `180` | **`71.67%`** | **`71.67%`** | `0.0940` |
|
| | **Formal Boolean & Truth-Table Verification** | `2,000` | `6,000` | **`68.45%`** | **`68.45%`** | `0.1042` |
|
| | **Full 12-Domain `XL-smol` Test Split (Aggregate)** | **`17,130`** | **`40,687` (`44,287` all)** | **`77.35%` Macro** | **`78.42%` Micro (`77.64%` all)** | **`0.0887` Mean** |
|
|
|
| ### 2. `SLM-Bench` Multiple-Choice & `GSM8K` Mathematical Verification (`13,644` Examples)
|
|
|
| | Benchmark | Split (`n`) | Random | Flat `GRT-30M` | Rotational Baseline | **`HELMHOLTZ-1` (24h Curriculum)** | **`HELMHOLTZ-1` (`10C` Horizon)** |
|
| | :--- | :--- | :---: | :---: | :---: | :---: | :---: |
|
| | **BoolQ** (`google/boolq`) | `val` (`3,270`) | `50.00%` | `51.41%` | `60.55%` | **`65.72%`** | **`73.03%`** |
|
| | **QNLI** (`nyu-mll/glue`) | `val` (`1,000`) | `50.00%` | `51.60%` | `54.00%` | **`77.90%`** | **`86.14%`** |
|
| | **ARC-Easy** (`allenai/ai2_arc`) | `test` (`2,376`) | `25.00%` | `29.59%` | `28.96%` | **`40.57%`** | **`55.04%`** |
|
| | **ARC-Challenge** (`allenai/ai2_arc`) | `test` (`1,172`) | `25.00%` | `20.82%` | `23.55%` | **`30.80%`** | **`40.13%`** |
|
| | **OpenBookQA** (`allenai/openbookqa`) | `test` (`500`) | `25.00%` | `28.20%` | `30.00%` | **`42.80%`** | **`54.35%`** |
|
| | **PIQA** (`baber/piqa`) | `val` (`1,838`) | `50.00%` | `55.28%` | `54.08%` | **`56.47%`** | **`65.34%`** |
|
| | **WinoGrande** (`allenai/winogrande`) | `val` (`1,267`) | `50.00%` | `50.28%` | `50.99%` | **`51.70%`** | **`62.85%`** |
|
| | **HellaSwag** (`Rowan/hellaswag`) | `val` (`1,000`) | `25.00%` | `29.20%` | `31.60%` | **`37.40%`** | **`50.51%`** |
|
| | **CommonsenseQA** (`tau/commonsense_qa`) | `val` (`1,221`) | `20.00%` | `21.70%` | `22.77%` | **`29.24%`** | **`40.93%`** |
|
| | **9-Task Macro Average** | `13,144` total | `35.56%` | `37.56%` | `39.61%` | **`48.07%`** | **`58.70%`** |
|
| | **GSM8K 4-Way Verification** | `test` (`500`) | `25.00%` | `34.20%` (`PPL 27.99`) | `39.20%` (`PPL 10.65`) | **`99.20%` (`PPL 7.60`)** | **`99.20%`** |
|
|
|
| ---
|
|
|
| ## Coming Next: Scaled Rotational JEPA World Models (`HELMHOLTZ-100M` & `HELMHOLTZ-350M`)
|
|
|
| Building directly on the `10M`, `20M`, and `30M` [`HELMHOLTZ-1`](https://huggingface.co/MatterWaveAI/helmholtz-1) and [`S1-GRT-JEPA`](https://huggingface.co/MatterWaveAI/s1-grt-jepa) verification ladder, our next scaled releases expand the collapse-free orthogonal JEPA world model and single-pass System-1 MPC judge:
|
|
|
| 1. **`HELMHOLTZ-100M` (`~195M` Dense-Equivalent Capacity — Scaled Rotational JEPA World Model & `S1` Trajectory Judge)**:
|
| - Scales the collapse-free orthogonal JEPA predictor ([`helmholtz-1`](https://huggingface.co/MatterWaveAI/helmholtz-1)) and multi-step MPC trajectory judge ([`s1-grt-jepa`](https://huggingface.co/MatterWaveAI/s1-grt-jepa)) in a deep `0`-linear-layer rotational architecture—projected on our empirical scaling ladder to reach **`64.9%` `SLM-Bench` macro accuracy**, **`89.2%` structured knowledge accuracy**, and **`>99%` cognitive routing accuracy**.
|
| 2. **`HELMHOLTZ-350M` (`~920M` Dense-Equivalent Capacity — Flagship Rotational JEPA Foundation World Model)**:
|
| - Scales multi-view Platonic alignment and long-horizon latent MPC trajectory verification in a flagship `0`-linear-layer rotational world model (`93.8%` structured knowledge accuracy, `97.0%` cognitive telemetry routing).
|
|
|
| ---
|
|
|
| ## `MatterWaveAI` Rotational JEPA Scaling Ladder (`10M` / `20M` / `30M`)
|
|
|
| - **Rotational JEPA & Causal World-Model Core**: [`MatterWaveAI/helmholtz-1`](https://huggingface.co/MatterWaveAI/helmholtz-1) (`10M`: [`helmholtz-b0.7.0-10m`](https://huggingface.co/MatterWaveAI/helmholtz-b0.7.0-10m) | `20M`: [`helmholtz-b0.7.0-20m`](https://huggingface.co/MatterWaveAI/helmholtz-b0.7.0-20m))
|
| - **System-1 Rotational JEPA Trajectory Judge & MPC Router**: [`MatterWaveAI/s1-grt-jepa`](https://huggingface.co/MatterWaveAI/s1-grt-jepa) (`10M`: [`s1-grt-jepa-b0.7.0-10m`](https://huggingface.co/MatterWaveAI/s1-grt-jepa-b0.7.0-10m) | `20M`: [`s1-grt-jepa-b0.7.0-20m`](https://huggingface.co/MatterWaveAI/s1-grt-jepa-b0.7.0-20m))
|
| - **24-Hour Distilled Preview (`GRT-30M`)**: [`MatterWaveAI/grt-30m`](https://huggingface.co/MatterWaveAI/grt-30m)
|
| - **Whitepaper**: [**The Geometric Rotation Transformer (GRT): A Purely Rotational Foundation Architecture (`GRT_Whitepaper.pdf`)**](https://huggingface.co/MatterWaveAI/grt-30m/resolve/main/GRT_Whitepaper.pdf)
|
|
|
| ---
|
|
|
| ## License & Citation
|
|
|
| Released under the **Apache 2.0** license.
|
|
|
| ```bibtex
|
| @techreport{householter2026grt,
|
| title = {The Geometric Rotation Transformer: A Purely Rotational Foundation Architecture},
|
| author = {Householter, David},
|
| institution = {MatterWave AI},
|
| year = {2026},
|
| url = {https://matterwave.ai}
|
| }
|
| ```
|
|
|
