GRT-30M-PRM (MatterWaveAI/grt-30m-prm)

GRT-30M-PRM is the 100% rotational Process Reward Model (PRM) and scalar RLHF Verifier initialized from MatterWaveAI/grt-30m (GRTForRewardModeling, 29,306,021 parameters, zero nn.Linear layers, zero nn.Conv1d layers).

It equips the 12-layer Givens Rotational Transformer backbone with dual zero-nn.Linear rotational heads (GRTCosineClassificationHead):

• reward_head: Evaluates circular-pooled planar-pair sequence representations (GRTForSequenceClassification._circular_pool) to emit a sequence-level scalar reward (logits of shape [B, 1]).
• step_head: Evaluates L2-normalized token/step hidden states across reasoning traces to emit fine-grained per-token / per-step verification scores (step_logits of shape [B, L, 1]).

Process Verification & Preference Alignment Metrics

• Held-Out Pairwise Preference Ranking Accuracy: 37.50% (untrained head baseline) → 100.00% (across all 8 classical geometric optimizers)
• Held-Out Step-Level Verification Accuracy: 31.42% → 98.07% (HuygensSymplecticAdam; 97.55% w/ PolarNaturalAdam, 97.21% w/ CyclideAdam)
• Held-Out Step-Level Verification F1: 98.59% (HuygensSymplecticAdam; 98.23% w/ PolarNaturalAdam, 97.99% w/ CyclideAdam)
• Held-Out Total Verification Loss (Bradley-Terry + Step BCE): 15.4491 → 0.0382 (HuygensSymplecticAdam, w/ 0.0000 held-out Bradley-Terry preference loss)
• Positive-vs-Flawed Sequence Reward Margin: -0.2095 → +25.1196 (CyclideAdam) / +24.4569 (HuygensSymplecticAdam)

---

8-Way Classical Geometric Optimizer Ablation

We benchmarked all 8 classical geometric, Riemannian, conic, symplectic, gyroscopic, and synchronization optimizers from transformers.models.grt.optimization_grt on MatterWaveAI/grt-30m-prm (200 steps of joint Bradley-Terry pairwise preference margin + token/step-level binary cross-entropy process verification on blocks[11:12] + final_norm + reward_head + step_head, 812,423 trainable parameters, batch_size = 8 pairs, lr = 4.5e-3, max_angular_step = 0.08, seed = 42):

Rank	Optimizer (optimizer_type)	Final Train Loss	Held-Out Total Loss	Held-Out BT Loss	Preference Acc (%)	Step Verification Acc (%)	Step F1 (%)	Reward Margin	Wall Time (s)
1	HuygensSymplecticAdam (huygens_symplectic_adam)	0.0464	0.0382	0.0000	100.00%	98.07%	98.59%	+24.4569	320.77s
2	PolarNaturalAdam (polar_natural_adam)	0.0503	0.0533	0.0003	100.00%	97.55%	98.23%	+23.7497	301.44s
3	CyclideAdam (cyclide_adam)	0.0714	0.0588	0.0000	100.00%	97.21%	97.99%	+25.1196	303.28s
4	SymplecticAdam (symplectic_adam)	0.1077	0.0729	0.0003	100.00%	96.76%	97.65%	+21.5043	310.61s
5	PairAdam (pair_adam)	0.1109	0.1139	0.0074	100.00%	95.86%	97.06%	+17.8117	257.80s
6	HessianPairAdam (hessian_pair_adam)	0.1688	0.1637	0.0003	100.00%	92.83%	94.66%	+15.9064	261.16s
7	HuygensAdam (huygens_adam)	0.0911	0.1795	0.0265	100.00%	95.34%	96.63%	+13.5915	289.98s
8	GyricAdam (gyric_adam)	0.2236	0.2213	0.0057	100.00%	89.13%	91.58%	+14.9081	315.54s

Key Geometric Findings on Process Reward Modeling

• Two-Stage Huygens → Symplectic Curriculum (HuygensSymplecticAdam) Achieves #1 Step Verification & Lowest Loss: Annealing circular synchronization splay torque (splay_anneal_steps = 100) into a Gromov–Liouville symplectic non-squeezing floor enables HuygensSymplecticAdam to separate valid derivation steps from mid-trace mathematical/algorithmic errors with 98.07% step accuracy (98.59% F1) and 0.0382 total validation loss.
• Riemannian Polar Metric (PolarNaturalAdam) & Dupin Cyclide Conic Metric (CyclideAdam) Excel at Sharp Step Boundaries: PolarNaturalAdam (97.55% step acc, 0.0533 loss) and CyclideAdam (97.21% step acc, +25.1196 sequence reward margin) leverage radial-scale preconditioning on reward_head and step_head to amplify the scalar reward separation between valid and flawed multi-step traces.

---

Quickstart (transformers)

import torch
from transformers import AutoTokenizer
from transformers.models.grt.modeling_grt import GRTForRewardModeling

model_id = "MatterWaveAI/grt-30m-prm"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = GRTForRewardModeling.from_pretrained(model_id, dtype=torch.float32)

trace = (
    "<|im_start|>user\nProblem: Solve for x: 4x - 9 = 19.<|im_end|>\n"
    "<|im_start|>assistant\n"
    "Step 1: Start with the linear equation 4x - 9 = 19.\n"
    "Step 2: Add 9 to both sides to isolate the 4x term: 4x = 28.\n"
    "Step 3: Divide both sides by 4: x = 28 / 4 = 7.\n"
    "Step 4: Verify: 4(7) - 9 = 28 - 9 = 19. Answer: x = 7.<|im_end|>"
)
inputs = tokenizer(trace, return_tensors="pt", add_special_tokens=False)
with torch.no_grad():
    out = model(**inputs)
    seq_reward = out.logits.item()
    step_probs = torch.sigmoid(out.step_logits[0, :, 0])
print(f"Sequence Reward: {seq_reward:.4f} | Mean Step Verification Confidence: {step_probs.mean().item():.4f}")

---

License

Apache 2.0