GRT-30M-VL (`MatterWaveAI/grt-30m-vl`)

GRT-30M-VL is a 100% Native Rotational Vision-Language Model (34.26M total parameters, zero nn.Linear dense weight matrices across both the vision encoder and language backbone) built on top of MatterWaveAI/grt-30m.

Architecture Summary

Component	Module	Parameters	Geometry
Patch Stem	`GRTPolarPatchEmbedding`	`20,992`	`14x14` unfold → `GRTPearlNorm` → 11-stage Givens butterfly cascade → `256`-dim (`128` planar pairs)
2D Positional Encoding	`GRTRotaryEmbedding2D`	`128`	2D axial rotary encoding (`64` row pairs + `64` column pairs) over the `16x16` patch grid
Vision Backbone	6x `GRTDecoderLayer` (`is_causal=False`)	`4,877,138`	Bidirectional multi-head rotational attention (`n_qkv_rotations=11`) + Clifford mixer + `21`-stage rotational FFN
Spatial Merger	`GRTSpatialRotationalMerger`	`56,000`	`2x2` spatial pixel-unshuffle (`256` → `64` tokens) + `21`-stage Givens expand/RadialGELU/contract projector
Vision Tower Total	`GRTVisionModel` (`model.visual`)	`4,954,258`	0 `nn.Linear` layers (`224x224` RGB → `64` visual planar-pair tokens of dim `256`)
Language Backbone	`GRTModel` + `GRTCosineHead`	`29,305,498`	12-layer causal rotational language model (`MatterWaveAI/grt-30m`, tied polar embeddings & cosine head)
Total (`GRT-30M-VL`)	`GRTVLForConditionalGeneration`	`34,259,756`	100% Rotational Multimodal Transformer

Quickstart (`transformers`)

from PIL import Image
import torch
from transformers import AutoModelForImageTextToText, AutoProcessor

model_id = "MatterWaveAI/grt-30m-vl"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(model_id, dtype=torch.float32)

image = Image.new("RGB", (224, 224), color=(225, 45, 45))
inputs = processor(text="<|image_pad|>Describe this image:", images=image, return_tensors="pt")

out_ids = model.generate(**inputs, max_new_tokens=20, do_sample=False)
print(processor.batch_decode(out_ids, skip_special_tokens=True)[0])

Alignment & Evaluation Summary

Held-out Visual Captioning Perplexity: 25,190.18 (unaligned) → 1.2242 (PolarNaturalAdam 200-step VL alignment; 1.2246 w/ HuygensAdam, 1.2619 w/ PairAdam)
Held-out Caption Token Top-1 Accuracy: 1.91% → 93.08% (HuygensAdam; 92.73% w/ PolarNaturalAdam, 92.62% w/ SymplecticAdam)
Image-Conditioned Contrast Margin (log P(caption | correct_img) - log P(caption | wrong_img)): -0.1635 → +1.6284 nats/token (+19.13 nats/sequence, 100.0% pairwise image preference accuracy across all 8 optimizers)
Held-out Visual Attribute Accuracy:
- Color recognition: 100.0% (PairAdam; 95.8% w/ PolarNaturalAdam)
- Spatial position recognition: 100.0% (PolarNaturalAdam, SymplecticAdam; 95.8% w/ PairAdam)
- Shape recognition: 54.2% (GyricAdam, HuygensSymplecticAdam; 50.0% w/ HessianPairAdam, CyclideAdam)
- Exact caption match: 41.7% (PolarNaturalAdam, PairAdam, CyclideAdam, SymplecticAdam)

8-Way Classical Geometric Optimizer Ablation (`Spark ROCm gfx1151`)

We benchmarked all 8 classical geometric, symplectic, gyroscopic, and synchronization optimizers from transformers.models.grt.optimization_grt on MatterWaveAI/grt-30m-vl across a controlled 200-step multimodal alignment protocol (100 Stage-1 steps aligning the 896,003-parameter 2D Givens vision stem + spatial rotational merger at lr = 4.0e-3 + 100 Stage-2 steps aligning the top-2 multimodal language decoder blocks blocks[10:12] + final_norm at lr = 3.5e-3, batch_size = 48, max_angular_step = 0.08, seed = 42):

Rank	Optimizer (`optimizer_type`)	Stage-1 Loss	Final Train Loss	Held-Out Val Loss	Held-Out Val PPL	Caption Token Acc	Cross-Image Logit Diff (nats/tok)	Color Acc	Shape Acc	Position Acc	Exact Match	Time (s)
1	`PolarNaturalAdam` (`polar_natural_adam`)	`2.9295`	`0.1395`	`0.2023`	`1.2242`	`92.73%`	`1.6031`	`95.8%`	`41.7%`	`100.0%`	`41.7%`	`64.75s`
2	`HuygensAdam` (`huygens_adam`)	`2.9123`	`0.1509`	`0.2026`	`1.2246`	`93.08%`	`1.6284`	`83.3%`	`45.8%`	`91.7%`	`37.5%`	`81.59s`
3	`GyricAdam` (`gyric_adam`)	`2.9060`	`0.1267`	`0.2116`	`1.2357`	`92.30%`	`1.4828`	`87.5%`	`54.2%`	`79.2%`	`33.3%`	`80.83s`
4	`HessianPairAdam` (`hessian_pair_adam`)	`2.9412`	`0.1853`	`0.2267`	`1.2545`	`92.06%`	`1.3347`	`87.5%`	`50.0%`	`83.3%`	`37.5%`	`55.45s`
5	`SymplecticAdam` (`symplectic_adam`)	`2.9590`	`0.1537`	`0.2299`	`1.2585`	`92.62%`	`1.4001`	`83.3%`	`41.7%`	`100.0%`	`41.7%`	`82.80s`
6	`HuygensSymplecticAdam` (`huygens_symplectic_adam`)	`2.9507`	`0.1735`	`0.2319`	`1.2610`	`91.91%`	`1.3576`	`87.5%`	`54.2%`	`29.2%`	`12.5%`	`83.02s`
7	`PairAdam` (`pair_adam`)	`2.9584`	`0.1596`	`0.2326`	`1.2619`	`91.60%`	`1.3513`	`100.0%`	`41.7%`	`95.8%`	`41.7%`	`55.47s`
8	`CyclideAdam` (`cyclide_adam`)	`2.9265`	`0.1650`	`0.2505`	`1.2847`	`91.03%`	`1.4793`	`91.7%`	`50.0%`	`91.7%`	`41.7%`	`81.74s`

Key Geometric Findings on Rotational Vision-Language Alignment

Riemannian Polar Metric (PolarNaturalAdam) Excels at 2D Patch-to-Language Phase Alignment: By preconditioning angular updates with the inverse polar metric r_p^(-2α) across GRTPolarPatchEmbedding (11 butterfly stages) and GRTSpatialRotationalMerger (21 expand/contract stages), PolarNaturalAdam achieves #1 held-out VL validation loss (0.2023, val_ppl = 1.2242), 100.0% spatial position accuracy, 95.8% color accuracy, and 41.7% exact caption match (up 3.3× from the 170-step baseline).
Huygens Circular Synchronization Splay Maximizes Cross-Image Contrast & Token Accuracy: HuygensAdam achieves #1 caption token top-1 accuracy (93.08%) and #1 cross-image log-probability contrast margin (+1.6284 nats/token, +19.13 nats/sequence over mismatched images), demonstrating that desynchronizing redundant butterfly stages in GRTVisionModel sharpens visual conditioning.
Gyroscopic Precession (GyricAdam) Achieves Lowest Training Loss & Highest Geometric Shape Recognition: GyricAdam reaches the lowest Stage-1 vision loss (2.9060), lowest Stage-2 LM training loss (0.1267), and #1 geometric shape accuracy (54.2%, tied with HuygensSymplecticAdam).

Complete `MatterWaveAI/grt-30m*` Research Suite on Hugging Face

Base Causal LM: MatterWaveAI/grt-30m
Chat Fine-Tune: MatterWaveAI/grt-30m-chat
4-Expert Mixture-of-Rotations (MoR): MatterWaveAI/grt-30m-mor
Circular-Pooled Sentence Embedding: MatterWaveAI/grt-30m-embedding
Cross-Encoder Reranker: MatterWaveAI/grt-30m-reranker
Vision-Language (GRTVL): MatterWaveAI/grt-30m-vl
Canonical Tokenizer & Polar Embedding Bank: MatterWaveAI/grt-30m-tokenizer
Rotational Text-to-Speech (GRTTTS): MatterWaveAI/grt-30m-tts
Multimodal Vision-Audio-Language-Speech (GRTOmni): MatterWaveAI/grt-30m-omni

Whitepaper: The Geometric Rotation Transformer (GRT): A Purely Rotational Foundation Architecture (GRT_Whitepaper.pdf)

License

Apache 2.0

Citation

@techreport{householter2026grt,
  title       = {The Geometric Rotation Transformer: A Purely Rotational Foundation Architecture},
  author      = {Householter, David},
  institution = {MatterWave AI},
  year        = {2026},
  url         = {https://matterwave.ai}
}

Downloads last month: 176

Safetensors

Model size

35.1M params

Tensor type

I64

F32

Model tree for MatterWaveAI/grt-30m-vl

Base model

MatterWaveAI/grt-30m

Finetuned

(14)

this model

GRT-30M-VL (MatterWaveAI/grt-30m-vl)