Instructions to use MatterWaveAI/grt-10m-curriculum with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use MatterWaveAI/grt-10m-curriculum with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="MatterWaveAI/grt-10m-curriculum")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("MatterWaveAI/grt-10m-curriculum", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use MatterWaveAI/grt-10m-curriculum with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "MatterWaveAI/grt-10m-curriculum"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "MatterWaveAI/grt-10m-curriculum",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/MatterWaveAI/grt-10m-curriculum

SGLang

How to use MatterWaveAI/grt-10m-curriculum with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "MatterWaveAI/grt-10m-curriculum" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "MatterWaveAI/grt-10m-curriculum",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "MatterWaveAI/grt-10m-curriculum" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "MatterWaveAI/grt-10m-curriculum",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use MatterWaveAI/grt-10m-curriculum with Docker Model Runner:
```
docker model run hf.co/MatterWaveAI/grt-10m-curriculum
```

GRT-10M-Curriculum: Multi-Round Symplectic Curriculum (`MatterWaveAI/grt-10m-curriculum`)

Multi-Round Symplectic Curriculum Release: Starting from the exact same 8.7M-parameter (2.2M non-embedding backbone across 6 rotational blocks) checkpoint as MatterWaveAI/grt-10m, MatterWaveAI/grt-10m-curriculum applies multi-round curriculum optimization with HuygensSymplecticAdam. Across the 9-task SLM-Bench multiple-choice suite (13,144 held-out examples), macro-average accuracy rises from 37.61% (PairAdam NeurIPS Submission) → 41.71% (Single-Pass HuygensSymplecticAdam) → 45.74% (Multi-Round Curriculum, +8.13%), while GSM8K 4-way numerical verification reaches 96.60% (PPL = 25.28, down 5.77× from 145.85).

Model Details


Total parameters	8,685,283 (8.7M)
Non-embedding parameters	2,235,283 (2.2M)
d_model	256
n_blocks	6
n_heads	4
n_qkv_rotations	5
ffn_n_rotations	9
Vocabulary	50,000 (BPE)
Max sequence length	1,024
Precision	float32

The embedding dominates the parameter count: 6.45M of 8.7M parameters are the token embedding (50,000 tokens x 128 phase angles, plus a log-magnitude per token).

Training


Base Pretraining Data	Nemotron
Base Pretraining Steps	336,000
Base Pretraining Optimizer	PairAdam (`MatterWaveAI/grt-10m-base`)
Base Peak LR	1e-3, cosine decay to 10%
Base Training Loss	2.144
Task Adaptation Optimizer	`HuygensSymplecticAdam` (`SymplecticAdam` two-stage curriculum)

Metric	Value
Validation loss	4.033
Validation perplexity	56.4
Validation accuracy	0.400

Benchmarks: NeurIPS Submission (`PairAdam`) → Single-Pass `HuygensSymplecticAdam` → Multi-Round Curriculum (`grt-10m-curriculum`)

Evaluated across the 10-task SLM-Bench small-language-model suite (training/slm_bench.py / grt-slm-bench) starting from the exact same 8.7M-parameter (2.2M non-embedding) checkpoint:

1. Multiple-Choice Accuracy (`9` Core `SLM-Bench` Tasks, `13,144` Held-Out Examples)

Task (`SLM-Bench`)	Split (`n`)	Random	Zero-Shot	NeurIPS Baseline (`PairAdam`)	Single-Pass (`HuygensSymplectic`)	Multi-Round Curriculum	Total Gain (Curriculum vs. `PairAdam`)
BoolQ (`google/boolq`)	`val` (`3,270`)	`50.00%`	`62.14%`	`61.13%`	`62.17%`	`64.59%`	`+3.46%` (`+2.42%` vs. Single-Pass)
ARC-Easy (`allenai/ai2_arc`)	`test` (`2,376`)	`25.00%`	`26.94%`	`26.60%`	`32.28%`	`33.16%`	`+6.56%` (`+0.88%` vs. Single-Pass)
ARC-Challenge (`allenai/ai2_arc`)	`test` (`1,172`)	`25.00%`	`22.53%`	`24.91%`	`27.13%`	`30.03%`	`+5.12%` (`+2.90%` vs. Single-Pass)
OpenBookQA (`allenai/openbookqa`)	`test` (`500`)	`25.00%`	`30.00%`	`22.60%`	`35.20%`	`40.00%`	`+17.40%` (`+4.80%` vs. Single-Pass)
PIQA (`baber/piqa`)	`val` (`1,838`)	`50.00%`	`52.18%`	`50.82%`	`52.18%`	`53.37%`	`+2.55%` (`+1.19%` vs. Single-Pass)
WinoGrande (`allenai/winogrande`)	`val` (`1,267`)	`50.00%`	`50.51%`	`50.75%`	`51.85%`	`52.33%`	`+1.58%` (`+0.48%` vs. Single-Pass)
HellaSwag (`Rowan/hellaswag`)	`val` (`1,000`)	`25.00%`	`25.60%`	`29.10%`	`30.80%`	`36.80%`	`+7.70%` (`+6.00%` vs. Single-Pass)
QNLI (`nyu-mll/glue`)	`val` (`1,000`)	`50.00%`	`52.70%`	`49.70%`	`60.90%`	`76.20%`	`+26.50%` (`+15.30%` vs. Single-Pass)
CommonsenseQA (`tau/commonsense_qa`)	`val` (`1,221`)	`20.00%`	`20.15%`	`22.85%`	`22.85%`	`25.14%`	`+2.29%` (`+2.29%` vs. Single-Pass)
9-Task Multiple-Choice Macro Average	`13,144` total	`35.56%`	`38.08%`	`37.61%`	`41.71%`	`45.74%`	`+8.13%` (`+4.03%` vs. Single-Pass)

2. Mathematical Reasoning Held-Out Perplexity & Verification (`GSM8K`, `test` split, `n = 500`)

Metric (`SLM-Bench`)	Zero-Shot	NeurIPS Baseline (`PairAdam`)	Single-Pass (`HuygensSymplectic`)	Multi-Round Curriculum	Total Improvement (Curriculum vs. Zero-Shot / `PairAdam`)
GSM8K Held-Out Perplexity (`PPL` ↓)	`145.85`	`37.27`	`30.09`	`25.28`	`5.77×` lower vs. Zero-Shot (`-32.2%` vs. `PairAdam`)
GSM8K 4-Way Numerical Verification (`Acc` ↑)	`22.00%`	`21.80%`	`84.60%`	`96.60%`	`+74.80%` vs. `PairAdam` (`+12.00%` vs. Single-Pass)

Overview of Multi-Round Curriculum Optimization

While single-pass task adaptation with HuygensSymplecticAdam already improves substantially over first-generation PairAdam, a compact 2.2M-parameter rotational backbone (6 Givens butterfly blocks on SO(D)) benefits further from multi-round curriculum training:

Norm-Preserving Rotational Geometry (SO(D)): Unlike standard dense transformers (nn.Linear) whose unconstrained weight matrices overfit small-scale benchmarks within 1–2 epochs, GRT-10M parameterizes its projections as cascades of norm-preserving 2×2 Givens rotations. Because orthogonal transformations preserve vector norms (||R(theta) x||_2 = ||x||_2), the backbone can be trained across multiple progressive curriculum stages without norm explosion or catastrophic forgetting.
Symplectic & Wavefront-Coupled Angular Updates (HuygensSymplecticAdam): SymplecticAdam and HuygensSymplecticAdam track angular velocity and cross-stage phase coherence across the Givens cascades, allowing deeper rotational blocks to align smoothly over multi-stage schedules where first-order PairAdam stalls.
Progressive Multi-Round Curriculum: Starting from the base pretrained checkpoint, the model is adapted across successive curriculum stages with cosine learning-rate scheduling, steadily refining angular phase alignment across all 6 blocks before consolidating into the published model.safetensors checkpoint.

Usage

pip install geometric-rotation-transformer

from grt import GRTForCausalLM, RotationalTokenizerBPE

model = GRTForCausalLM.from_pretrained("MatterWaveAI/grt-10m-curriculum")
tokenizer = RotationalTokenizerBPE.from_pretrained("MatterWaveAI/grt-10m-curriculum")

input_ids = tokenizer("The geometry of language is", return_tensors="pt").input_ids
output = model.generate(input_ids, max_new_tokens=50, temperature=0.7)
print(tokenizer.decode(output[0], skip_special_tokens=True))

Importing grt registers the architecture with the transformers Auto classes:

import grt  # registers model_type "grt"
from transformers import AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained("MatterWaveAI/grt-10m-curriculum")

The package is pure PyTorch and runs on CPU. Optimized CUDA kernels are optional and load automatically when installed.

trust_remote_code=True is not supported for this repository. The model code is distributed as a pip package rather than as Hub remote code, because the implementation spans a package with absolute imports that a flat Hub snapshot cannot resolve. Install geometric-rotation-transformer first; import grt then registers model_type: "grt" so the Auto classes resolve these weights.

Architecture

GRT replaces the standard Q/K/V projections and FFN weight matrices with cascades of learned 2x2 Givens rotations in a butterfly pattern. Each stage is modulated by data-dependent Clifford features:

feat  = [S0, V1, V2, B12]           # scalar energy, direction, cross-pair bivector
theta = angles + tanh(modulation)   # base angle + content-dependent nudge
r0    = x0*cos(theta) - x1*sin(theta)
r1    = x0*sin(theta) + x1*cos(theta)

Components:

RotationalEmbedding — tokens stored as phase angles on the unit hypersphere
RotationalAttention — Q/K/V via Givens cascades
RotationalFFN — feed-forward via meta-coupled rotation stages
PearlNorm — pair-radius normalization preserving per-pair phase
RadialGELU — activation gating pair magnitudes, preserving direction
PairAdam — AdamW with factored second moments: 1D params (angles, scales) use standard diagonal moments, 2D params (coupling matrices) use a row factor [m,1] and col factor [1,n] reconstructed as an outer product, at O(m+n) instead of O(mn). Optionally clamps per-step angular updates.

Because every projection is a rotation, the transform is norm-preserving by construction rather than by normalization.

Intended Use & Limitations

Intended for research into structured/rotational architectures. Not intended for production use or deployment.

Known limitations:

English only, trained on a single corpus (Nemotron), so it inherits that corpus's coverage gaps and biases.
No instruction tuning. This is a base model trained on next-token prediction.
At 8.7M parameters, generated text is frequently incoherent. Do not expect instruction-following, factual reliability, or coherent long-form output.
No safety tuning, RLHF, or content filtering of any kind.
Context is capped at 1,024 tokens.

Citation

@article{grt2026,
  title={Geometric Rotation Transformer: Structured SO(n) Rotations for Efficient Sequence Modeling},
  author={MatterWave AI},
  year={2026},
  url={https://github.com/matterwaveai/grt}
}

License

Apache 2.0

Downloads last month: 171

Safetensors

Model size

8.8M params

Tensor type

I64

F32

MatterWaveAI
/

grt-10m-curriculum

GRT-10M-Curriculum: Multi-Round Symplectic Curriculum (`MatterWaveAI/grt-10m-curriculum`)

Model Details

Training

Benchmarks: NeurIPS Submission (`PairAdam`) → Single-Pass `HuygensSymplecticAdam` → Multi-Round Curriculum (`grt-10m-curriculum`)

1. Multiple-Choice Accuracy (`9` Core `SLM-Bench` Tasks, `13,144` Held-Out Examples)

2. Mathematical Reasoning Held-Out Perplexity & Verification (`GSM8K`, `test` split, `n = 500`)

Overview of Multi-Round Curriculum Optimization

Usage

Architecture

Intended Use & Limitations

Citation

License

Dataset used to train MatterWaveAI/grt-10m-curriculum

GRT-10M-Curriculum: Multi-Round Symplectic Curriculum (MatterWaveAI/grt-10m-curriculum)

Model Details

Training

Benchmarks: NeurIPS Submission (PairAdam) → Single-Pass HuygensSymplecticAdam → Multi-Round Curriculum (grt-10m-curriculum)

1. Multiple-Choice Accuracy (9 Core SLM-Bench Tasks, 13,144 Held-Out Examples)

2. Mathematical Reasoning Held-Out Perplexity & Verification (GSM8K, test split, n = 500)

Overview of Multi-Round Curriculum Optimization

Usage

Architecture

Intended Use & Limitations

Citation

License

Dataset used to train MatterWaveAI/grt-10m-curriculum

GRT-10M-Curriculum: Multi-Round Symplectic Curriculum (`MatterWaveAI/grt-10m-curriculum`)

Benchmarks: NeurIPS Submission (`PairAdam`) → Single-Pass `HuygensSymplecticAdam` → Multi-Round Curriculum (`grt-10m-curriculum`)

1. Multiple-Choice Accuracy (`9` Core `SLM-Bench` Tasks, `13,144` Held-Out Examples)

2. Mathematical Reasoning Held-Out Perplexity & Verification (`GSM8K`, `test` split, `n = 500`)