---
title: README
emoji: 🌀
colorFrom: indigo
colorTo: blue
sdk: static
pinned: false
---
_**Purely Rotational Foundation Models & Geometric Optimization**_
**MatterWave AI** researches and builds **[Geometric Rotation Transformers (GRT)](https://huggingface.co/MatterWaveAI/grt-30m/resolve/main/GRT_Whitepaper.pdf)** — a foundation architecture that replaces dense linear weight matrices (`nn.Linear`) in attention (`Q, K, V, O`) and feed-forward (`expand, contract`) blocks with cascades of content-modulated 2×2 Givens rotations on `SO(D)` interleaved with butterfly permutations.
## Build and Scale with Geometric Rotation Transformers
Because every transform in the GRT residual path is an orthogonal rotation cascade, hidden-state norms are preserved by construction rather than by post-hoc rescaling. GRT checkpoints execute deterministically with bit-for-bit greedy generation parity across **PyTorch** (CUDA, ROCm, Intel XPU), **vLLM**, and **llama.cpp** (CPU AVX2, Vulkan, and ROCm HIP).
Starting from our pretrained **GRT-10M** and **GRT-20M** NeurIPS submission checkpoints (trained with first-generation **PairAdam**) and our 24-hour Qwen 3 distilled **GRT-30M** base checkpoint, replacing the optimizer with **HuygensSymplecticAdam** and training across a multi-round curriculum delivers substantial improvements across the 10-task **SLM-Bench** evaluation suite (13,144 held-out multiple-choice examples across BoolQ, ARC-Easy, ARC-Challenge, OpenBookQA, PIQA, WinoGrande, HellaSwag, QNLI, and CommonsenseQA, plus 500 held-out GSM8K mathematical reasoning problems):
* **[GRT-10M-Curriculum (8.7M total / 2.2M backbone)](https://huggingface.co/MatterWaveAI/grt-10m-curriculum)** — Lifts 9-task multiple-choice accuracy from **37.61%** (PairAdam NeurIPS Baseline) to **41.71%** (Single-Pass) to **45.74%** (Multi-Round Curriculum, **+8.13%**), while GSM8K 4-way verification reaches **96.60%** (perplexity reduced **5.77×** from 145.85 to **25.28**).
* **[GRT-20M-Curriculum (16.7M total / 10.3M backbone)](https://huggingface.co/MatterWaveAI/grt-20m-curriculum)** — Lifts 9-task multiple-choice accuracy from **38.63%** (PairAdam NeurIPS Baseline) to **45.03%** (Single-Pass) to **48.17%** (Multi-Round Curriculum, **+9.54%**), while GSM8K 4-way verification reaches **98.20%** (perplexity reduced **7.86×** from 60.12 to **7.65**).
* **[GRT-30M-Curriculum (29.3M total / 9.74M backbone)](https://huggingface.co/MatterWaveAI/grt-30m-curriculum)** — Lifts 9-task multiple-choice accuracy on the 24-hour Qwen 3 distilled base from **39.61%** (PairAdam Baseline) to **43.86%** (Single-Pass) to **48.07%** (Multi-Round Curriculum, **+8.46%**), while GSM8K 4-way verification reaches **99.20%** (perplexity reduced **3.68×** from 27.99 to **7.60**).
## System-1 Cognitive Telemetry & Multimodal Specialists
Beyond base causal language modeling, the **[GRT-30M](https://huggingface.co/MatterWaveAI/grt-30m)** backbone powers specialized reasoning routers, retrieval encoders, and unified multimodal checkpoints:
* **[Open-S1-GRT-30M](https://huggingface.co/MatterWaveAI/open-s1-grt-30m)** — 5-head System-1 cognitive telemetry router (action routing, escalation, loop guard, specification support, and verification gating) reaching **95.83%** 5-family macro accuracy (**92.00%** `MC_SHUFFLE_V2` transfer with `HuygensSymplecticAdam`; **ECE = 0.0095** with `SymplecticAdam`).
* **[GRT Architecture Whitepaper (PDF)](https://huggingface.co/MatterWaveAI/grt-30m/resolve/main/GRT_Whitepaper.pdf)** — Full mathematical formulation, kernel design, and multi-backend verification report.
## Explore the MatterWave AI Model Suite
Browse our open-weights repositories by category: