---
license: apache-2.0
library_name: transformers
pipeline_tag: text-generation
tags:
  - grt
  - chat
  - conversational
  - rotational-transformer
  - givens-rotations
  - clifford-algebra
  - matterwave
base_model: MatterWaveAI/grt-30m
---

# GRT-30M-Chat (`MatterWaveAI/grt-30m-chat`)

**GRT-30M-Chat** is the instruction and multi-turn conversational fine-tune of [`MatterWaveAI/grt-30m`](https://huggingface.co/MatterWaveAI/grt-30m) (`29,305,498` parameters, **zero `nn.Linear` layers**), formatted with the standard **ChatML** (`<|im_start|>` / `<|im_end|>`) template.

## SFT Alignment Metrics

- **Held-out Conversational Perplexity**: `182.79` (base) → **`1.34`** (`HessianPairAdam` 200-step Chat SFT; `1.37` at 100-step `PairAdam`)
- **Held-out Assistant Loss**: `5.2083` → **`0.2904`** (`HessianPairAdam`; `0.2932` w/ `CyclideAdam`, `0.2971` w/ `PairAdam`)
- **Held-out Assistant Token Top-1 Accuracy**: `24.69%` → **`100.00%`** (`HuygensAdam`; `99.88%` w/ `HessianPairAdam`, `CyclideAdam`, `PairAdam`, `HuygensSymplecticAdam`)
- **Held-out Multi-Turn ChatML Validation Loss**: `5.3480` → **`1.6645`** (`HuygensAdam`, `val_ppl = 5.28`)

---

## 8-Way Classical Geometric Optimizer Ablation (`Spark ROCm gfx1151`)

We benchmarked all 8 classical geometric, symplectic, gyroscopic, and synchronization optimizers from `transformers.models.grt.optimization_grt` on `MatterWaveAI/grt-30m-chat` (`200` ChatML SFT steps on `blocks[9:12]` + `final_norm` + `<|im_end|>` polar phase calibration from an identical cached `blocks[:9]` prefix state, `2,435,209` trainable parameters, `batch_size = 36`, `lr = 3.5e-3`, `max_angular_step = 0.08`, `seed = 42`):

| Rank | Optimizer (`optimizer_type`) | Final Train Loss | Held-Out Val Loss | Held-Out Val PPL | Assistant Token Acc | Assistant Response Margin (nats/tok) | Multi-Turn Val Loss | Multi-Turn Token Acc | Time (s) |
| :---: | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| **1** | **`HessianPairAdam` (`hessian_pair_adam`)** | `0.9501` | **`0.2904`** | **`1.3370`** | `99.88%` | `1.0754` | `1.7855` | `80.26%` | `49.51s` |
| **2** | **`CyclideAdam` (`cyclide_adam`)** | `0.9506` | `0.2932` | `1.3406` | `99.88%` | `0.9446` | `1.9591` | `74.87%` | `50.19s` |
| **3** | **`PairAdam` (`pair_adam`)** | `0.9505` | `0.2971` | `1.3460` | `99.88%` | `1.0982` | `2.0593` | `73.85%` | **`48.57s`** |
| **4** | **`HuygensAdam` (`huygens_adam`)** | **`0.9498`** | `0.3107` | `1.3644` | **`100.00%`** | `1.0257` | **`1.6645`** | **`81.54%`** | `67.79s` |
| **5** | **`HuygensSymplecticAdam` (`huygens_symplectic_adam`)** | `0.9502` | `0.3361` | `1.3995` | `99.88%` | `1.0957` | `1.7592` | `80.51%` | `65.69s` |
| **6** | **`SymplecticAdam` (`symplectic_adam`)** | `0.9502` | `0.3441` | `1.4108` | `99.38%` | **`1.1464`** | `1.9487` | `75.13%` | `50.95s` |
| **7** | **`GyricAdam` (`gyric_adam`)** | `0.9503` | `0.3476` | `1.4157` | `99.75%` | `1.0779` | `1.8009` | `77.18%` | `70.50s` |
| **8** | **`PolarNaturalAdam` (`polar_natural_adam`)** | `0.9521` | `0.3883` | `1.4745` | `99.13%` | `0.9159` | `1.8642` | `76.92%` | `49.40s` |

### Key Geometric Findings on ChatML Instruction Tuning
- **Cauchy–Riemann Toroidal Curvature (`HessianPairAdam`) Achieves Lowest Conversational Validation Loss**: Using Theorem 2.1's exact scale-angle curvature identity `c_{θ_p} = -∑_{i∈{0,1}} (softplus(s_{i,p}) / σ(s_{i,p})) ∇_{s_{i,p}} L` without extra forward/backward passes, `HessianPairAdam` achieves **#1 held-out ChatML validation loss (`0.2904`) and perplexity (`1.3370`)**, followed closely by `CyclideAdam` (`0.2932` / `1.3406` PPL) and `PairAdam` (`0.2971` / `1.3460` PPL).
- **Huygens Circular Synchronization Splay Excels at Multi-Turn Generalization**: `HuygensAdam` achieves **100.00% held-out assistant token top-1 accuracy**, the lowest final training loss (`0.9498`), and **#1 multi-turn ChatML validation loss (`1.6645`, `81.54%` multi-turn token accuracy)** by preventing rotational butterfly stages from over-synchronizing onto single-turn prompt templates.
- **Symplectic Non-Squeezing Maximizes Target-vs-Distractor Contrast Margin**: `SymplecticAdam` achieves the **highest assistant response log-probability margin (`1.1464` nats/token, `+34.60` nats/sequence over cross-domain distractors)** via its Gromov–Liouville canonical area floor.

---

## Quickstart (`transformers`)

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "MatterWaveAI/grt-30m-chat"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.float32)

messages = [
    {"role": "user", "content": "What is GRT-30M?"}
]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=False)

out_ids = model.generate(**inputs, max_new_tokens=64, do_sample=False)
print(tokenizer.decode(out_ids[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))
```

---

## Complete `MatterWaveAI/grt-30m*` Research Suite on Hugging Face

1. **Base Causal LM**: [`MatterWaveAI/grt-30m`](https://huggingface.co/MatterWaveAI/grt-30m)
2. **Chat Fine-Tune**: [`MatterWaveAI/grt-30m-chat`](https://huggingface.co/MatterWaveAI/grt-30m-chat)
3. **4-Expert Mixture-of-Rotations (`MoR`)**: [`MatterWaveAI/grt-30m-mor`](https://huggingface.co/MatterWaveAI/grt-30m-mor)
4. **Circular-Pooled Sentence Embedding**: [`MatterWaveAI/grt-30m-embedding`](https://huggingface.co/MatterWaveAI/grt-30m-embedding)
5. **Cross-Encoder Reranker**: [`MatterWaveAI/grt-30m-reranker`](https://huggingface.co/MatterWaveAI/grt-30m-reranker)
6. **Vision-Language (`GRTVL`)**: [`MatterWaveAI/grt-30m-vl`](https://huggingface.co/MatterWaveAI/grt-30m-vl)
7. **Canonical Tokenizer & Polar Embedding Bank**: [`MatterWaveAI/grt-30m-tokenizer`](https://huggingface.co/MatterWaveAI/grt-30m-tokenizer)
8. **Rotational Text-to-Speech (`GRTTTS`)**: [`MatterWaveAI/grt-30m-tts`](https://huggingface.co/MatterWaveAI/grt-30m-tts)
9. **Multimodal Vision-Audio-Language-Speech (`GRTOmni`)**: [`MatterWaveAI/grt-30m-omni`](https://huggingface.co/MatterWaveAI/grt-30m-omni)

- **Whitepaper**: [**The Geometric Rotation Transformer (GRT): A Purely Rotational Foundation Architecture (`GRT_Whitepaper.pdf`)**](https://huggingface.co/MatterWaveAI/grt-30m/resolve/main/GRT_Whitepaper.pdf)

---

## License

Apache 2.0

---

## Citation

```bibtex
@techreport{householter2026grt,
  title       = {The Geometric Rotation Transformer: A Purely Rotational Foundation Architecture},
  author      = {Householter, David},
  institution = {MatterWave AI},
  year        = {2026},
  url         = {https://matterwave.ai}
}
```