GRT-30M-Chat (MatterWaveAI/grt-30m-chat)

GRT-30M-Chat is the instruction and multi-turn conversational fine-tune of MatterWaveAI/grt-30m (29,305,498 parameters, zero nn.Linear layers), formatted with the standard ChatML (<|im_start|> / <|im_end|>) template.

SFT Alignment Metrics

• Held-out Conversational Perplexity: 182.79 (base) → 1.34 (HessianPairAdam 200-step Chat SFT; 1.37 at 100-step PairAdam)
• Held-out Assistant Loss: 5.2083 → 0.2904 (HessianPairAdam; 0.2932 w/ CyclideAdam, 0.2971 w/ PairAdam)
• Held-out Assistant Token Top-1 Accuracy: 24.69% → 100.00% (HuygensAdam; 99.88% w/ HessianPairAdam, CyclideAdam, PairAdam, HuygensSymplecticAdam)
• Held-out Multi-Turn ChatML Validation Loss: 5.3480 → 1.6645 (HuygensAdam, val_ppl = 5.28)

---

8-Way Classical Geometric Optimizer Ablation (Spark ROCm gfx1151)

We benchmarked all 8 classical geometric, symplectic, gyroscopic, and synchronization optimizers from transformers.models.grt.optimization_grt on MatterWaveAI/grt-30m-chat (200 ChatML SFT steps on blocks[9:12] + final_norm + <|im_end|> polar phase calibration from an identical cached blocks[:9] prefix state, 2,435,209 trainable parameters, batch_size = 36, lr = 3.5e-3, max_angular_step = 0.08, seed = 42):

Rank	Optimizer (optimizer_type)	Final Train Loss	Held-Out Val Loss	Held-Out Val PPL	Assistant Token Acc	Assistant Response Margin (nats/tok)	Multi-Turn Val Loss	Multi-Turn Token Acc	Time (s)
1	HessianPairAdam (hessian_pair_adam)	0.9501	0.2904	1.3370	99.88%	1.0754	1.7855	80.26%	49.51s
2	CyclideAdam (cyclide_adam)	0.9506	0.2932	1.3406	99.88%	0.9446	1.9591	74.87%	50.19s
3	PairAdam (pair_adam)	0.9505	0.2971	1.3460	99.88%	1.0982	2.0593	73.85%	48.57s
4	HuygensAdam (huygens_adam)	0.9498	0.3107	1.3644	100.00%	1.0257	1.6645	81.54%	67.79s
5	HuygensSymplecticAdam (huygens_symplectic_adam)	0.9502	0.3361	1.3995	99.88%	1.0957	1.7592	80.51%	65.69s
6	SymplecticAdam (symplectic_adam)	0.9502	0.3441	1.4108	99.38%	1.1464	1.9487	75.13%	50.95s
7	GyricAdam (gyric_adam)	0.9503	0.3476	1.4157	99.75%	1.0779	1.8009	77.18%	70.50s
8	PolarNaturalAdam (polar_natural_adam)	0.9521	0.3883	1.4745	99.13%	0.9159	1.8642	76.92%	49.40s

Key Geometric Findings on ChatML Instruction Tuning

• Cauchy–Riemann Toroidal Curvature (HessianPairAdam) Achieves Lowest Conversational Validation Loss: Using Theorem 2.1's exact scale-angle curvature identity c_{θ_p} = -∑_{i∈{0,1}} (softplus(s_{i,p}) / σ(s_{i,p})) ∇_{s_{i,p}} L without extra forward/backward passes, HessianPairAdam achieves #1 held-out ChatML validation loss (0.2904) and perplexity (1.3370), followed closely by CyclideAdam (0.2932 / 1.3406 PPL) and PairAdam (0.2971 / 1.3460 PPL).
• Huygens Circular Synchronization Splay Excels at Multi-Turn Generalization: HuygensAdam achieves 100.00% held-out assistant token top-1 accuracy, the lowest final training loss (0.9498), and #1 multi-turn ChatML validation loss (1.6645, 81.54% multi-turn token accuracy) by preventing rotational butterfly stages from over-synchronizing onto single-turn prompt templates.
• Symplectic Non-Squeezing Maximizes Target-vs-Distractor Contrast Margin: SymplecticAdam achieves the highest assistant response log-probability margin (1.1464 nats/token, +34.60 nats/sequence over cross-domain distractors) via its Gromov–Liouville canonical area floor.

---

Quickstart (transformers)

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "MatterWaveAI/grt-30m-chat"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.float32)

messages = [
    {"role": "user", "content": "What is GRT-30M?"}
]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=False)

out_ids = model.generate(**inputs, max_new_tokens=64, do_sample=False)
print(tokenizer.decode(out_ids[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))

---

Complete MatterWaveAI/grt-30m* Research Suite on Hugging Face

1. Base Causal LM: MatterWaveAI/grt-30m
2. Chat Fine-Tune: MatterWaveAI/grt-30m-chat
3. 4-Expert Mixture-of-Rotations (MoR): MatterWaveAI/grt-30m-mor
4. Circular-Pooled Sentence Embedding: MatterWaveAI/grt-30m-embedding
5. Cross-Encoder Reranker: MatterWaveAI/grt-30m-reranker
6. Vision-Language (GRTVL): MatterWaveAI/grt-30m-vl
7. Canonical Tokenizer & Polar Embedding Bank: MatterWaveAI/grt-30m-tokenizer
8. Rotational Text-to-Speech (GRTTTS): MatterWaveAI/grt-30m-tts
9. Multimodal Vision-Audio-Language-Speech (GRTOmni): MatterWaveAI/grt-30m-omni

• Whitepaper: The Geometric Rotation Transformer (GRT): A Purely Rotational Foundation Architecture (GRT_Whitepaper.pdf)

---

License

Apache 2.0

---

Citation

@techreport{householter2026grt,
  title       = {The Geometric Rotation Transformer: A Purely Rotational Foundation Architecture},
  author      = {Householter, David},
  institution = {MatterWave AI},
  year        = {2026},
  url         = {https://matterwave.ai}
}