---
license: apache-2.0
library_name: transformers
pipeline_tag: text-classification
tags:
  - grt
  - reward-model
  - process-reward-model
  - prm
  - rlhf-verifier
  - rotational-transformer
  - givens-rotations
  - clifford-algebra
  - matterwave
base_model: MatterWaveAI/grt-30m
---

# GRT-30M-PRM (`MatterWaveAI/grt-30m-prm`)

**GRT-30M-PRM** is the **100% rotational Process Reward Model (PRM) and scalar RLHF Verifier** initialized from [`MatterWaveAI/grt-30m`](https://huggingface.co/MatterWaveAI/grt-30m) (`GRTForRewardModeling`, `29,306,021` parameters, **zero `nn.Linear` layers**, **zero `nn.Conv1d` layers**).

It equips the 12-layer Givens Rotational Transformer backbone with dual zero-`nn.Linear` rotational heads (`GRTCosineClassificationHead`):
- **`reward_head`**: Evaluates circular-pooled planar-pair sequence representations (`GRTForSequenceClassification._circular_pool`) to emit a sequence-level scalar reward (`logits` of shape `[B, 1]`).
- **`step_head`**: Evaluates `L2`-normalized token/step hidden states across reasoning traces to emit fine-grained per-token / per-step verification scores (`step_logits` of shape `[B, L, 1]`).

## Process Verification & Preference Alignment Metrics

- **Held-Out Pairwise Preference Ranking Accuracy**: `37.50%` (untrained head baseline) → **`100.00%`** (across all 8 classical geometric optimizers)
- **Held-Out Step-Level Verification Accuracy**: `31.42%` → **`98.07%`** (`HuygensSymplecticAdam`; `97.55%` w/ `PolarNaturalAdam`, `97.21%` w/ `CyclideAdam`)
- **Held-Out Step-Level Verification F1**: **`98.59%`** (`HuygensSymplecticAdam`; `98.23%` w/ `PolarNaturalAdam`, `97.99%` w/ `CyclideAdam`)
- **Held-Out Total Verification Loss (Bradley-Terry + Step BCE)**: `15.4491` → **`0.0382`** (`HuygensSymplecticAdam`, w/ `0.0000` held-out Bradley-Terry preference loss)
- **Positive-vs-Flawed Sequence Reward Margin**: `-0.2095` → **`+25.1196`** (`CyclideAdam`) / **`+24.4569`** (`HuygensSymplecticAdam`)

---

## 8-Way Classical Geometric Optimizer Ablation

We benchmarked all 8 classical geometric, Riemannian, conic, symplectic, gyroscopic, and synchronization optimizers from `transformers.models.grt.optimization_grt` on `MatterWaveAI/grt-30m-prm` (`200` steps of joint Bradley-Terry pairwise preference margin + token/step-level binary cross-entropy process verification on `blocks[11:12]` + `final_norm` + `reward_head` + `step_head`, `812,423` trainable parameters, `batch_size = 8` pairs, `lr = 4.5e-3`, `max_angular_step = 0.08`, `seed = 42`):

| Rank | Optimizer (`optimizer_type`) | Final Train Loss | Held-Out Total Loss | Held-Out BT Loss | Preference Acc (%) | Step Verification Acc (%) | Step F1 (%) | Reward Margin | Wall Time (s) |
| :---: | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| **1** | **`HuygensSymplecticAdam` (`huygens_symplectic_adam`)** | **`0.0464`** | **`0.0382`** | **`0.0000`** | **`100.00%`** | **`98.07%`** | **`98.59%`** | `+24.4569` | `320.77s` |
| **2** | **`PolarNaturalAdam` (`polar_natural_adam`)** | `0.0503` | `0.0533` | `0.0003` | **`100.00%`** | `97.55%` | `98.23%` | `+23.7497` | `301.44s` |
| **3** | **`CyclideAdam` (`cyclide_adam`)** | `0.0714` | `0.0588` | **`0.0000`** | **`100.00%`** | `97.21%` | `97.99%` | **`+25.1196`** | `303.28s` |
| **4** | **`SymplecticAdam` (`symplectic_adam`)** | `0.1077` | `0.0729` | `0.0003` | **`100.00%`** | `96.76%` | `97.65%` | `+21.5043` | `310.61s` |
| **5** | **`PairAdam` (`pair_adam`)** | `0.1109` | `0.1139` | `0.0074` | **`100.00%`** | `95.86%` | `97.06%` | `+17.8117` | **`257.80s`** |
| **6** | **`HessianPairAdam` (`hessian_pair_adam`)** | `0.1688` | `0.1637` | `0.0003` | **`100.00%`** | `92.83%` | `94.66%` | `+15.9064` | `261.16s` |
| **7** | **`HuygensAdam` (`huygens_adam`)** | `0.0911` | `0.1795` | `0.0265` | **`100.00%`** | `95.34%` | `96.63%` | `+13.5915` | `289.98s` |
| **8** | **`GyricAdam` (`gyric_adam`)** | `0.2236` | `0.2213` | `0.0057` | **`100.00%`** | `89.13%` | `91.58%` | `+14.9081` | `315.54s` |

### Key Geometric Findings on Process Reward Modeling
- **Two-Stage Huygens → Symplectic Curriculum (`HuygensSymplecticAdam`) Achieves #1 Step Verification & Lowest Loss**: Annealing circular synchronization splay torque (`splay_anneal_steps = 100`) into a Gromov–Liouville symplectic non-squeezing floor enables `HuygensSymplecticAdam` to separate valid derivation steps from mid-trace mathematical/algorithmic errors with **`98.07%` step accuracy (`98.59%` F1)** and **`0.0382` total validation loss**.
- **Riemannian Polar Metric (`PolarNaturalAdam`) & Dupin Cyclide Conic Metric (`CyclideAdam`) Excel at Sharp Step Boundaries**: `PolarNaturalAdam` (`97.55%` step acc, `0.0533` loss) and `CyclideAdam` (`97.21%` step acc, **`+25.1196` sequence reward margin**) leverage radial-scale preconditioning on `reward_head` and `step_head` to amplify the scalar reward separation between valid and flawed multi-step traces.

---

## Quickstart (`transformers`)

```python
import torch
from transformers import AutoTokenizer
from transformers.models.grt.modeling_grt import GRTForRewardModeling

model_id = "MatterWaveAI/grt-30m-prm"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = GRTForRewardModeling.from_pretrained(model_id, dtype=torch.float32)

trace = (
    "<|im_start|>user\nProblem: Solve for x: 4x - 9 = 19.<|im_end|>\n"
    "<|im_start|>assistant\n"
    "Step 1: Start with the linear equation 4x - 9 = 19.\n"
    "Step 2: Add 9 to both sides to isolate the 4x term: 4x = 28.\n"
    "Step 3: Divide both sides by 4: x = 28 / 4 = 7.\n"
    "Step 4: Verify: 4(7) - 9 = 28 - 9 = 19. Answer: x = 7.<|im_end|>"
)
inputs = tokenizer(trace, return_tensors="pt", add_special_tokens=False)
with torch.no_grad():
    out = model(**inputs)
    seq_reward = out.logits.item()
    step_probs = torch.sigmoid(out.step_logits[0, :, 0])
print(f"Sequence Reward: {seq_reward:.4f} | Mean Step Verification Confidence: {step_probs.mean().item():.4f}")
```

---

## License

Apache 2.0