# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModelForQuestionAnswering
model = AutoModelForQuestionAnswering.from_pretrained("MatterWaveAI/grt-30m-qa", device_map="auto")
GRT-30M-QA (MatterWaveAI/grt-30m-qa)
GRT-30M-QA (GRTForQuestionAnswering) is a 100% Native Rotational Extractive Question Answering Model (29,306,020 parameters, zero nn.Linear layers, zero nn.Conv1d layers) built on MatterWaveAI/grt-30m.
GRTForQuestionAnswering replaces traditional dense span projection layers (nn.Linear(hidden_size, 2)) with a 2-channel GRTCosineClassificationHead (qa_outputs) parameterized by planar-pair phase_angles ([2, 128]) and softplus radial pair_scales ([2, 128]), emitting start_logits and end_logits across context token positions.
8-Way Classical Geometric Optimizer Ablation (GRTForQuestionAnswering)
All 8 classical geometric optimizers in transformers.models.grt.optimization_grt were benchmarked under an identical 200-step adaptation protocol (seed = 42, batch_size = 24, backbone_lr = 1.8e-4) unfreezing the top rotational transformer block blocks[-1] (811,651 parameters), final_norm (128 parameters), and qa_outputs (523 parameters) for 812,430 total trainable parameters (0 nn.Linear):
| Rank | Optimizer (optimization_grt.py) |
Final Train Loss | Val Span Loss | Exact Match (EM %) |
Span Token F1 (%) |
Start Acc (%) |
End Acc (%) |
Wall Time (200 Steps) |
|---|---|---|---|---|---|---|---|---|
| 1 | polar_natural_adam (PolarNaturalAdam (Riemannian Polar Metric + Cauchy-Riemann Hessian)) |
0.1385 |
0.6512 |
75.00% |
83.57% |
83.75% |
76.25% |
219.08s |
| 2 | cyclide_adam (CyclideAdam (Dupin Cyclide Conic Metric + Deviatoric Twist)) |
0.1382 |
0.6520 |
75.00% |
83.57% |
83.75% |
76.25% |
225.19s |
| 3 | pair_adam (PairAdam (Factored Toroidal Baseline)) |
0.1383 |
0.6532 |
75.00% |
83.57% |
83.75% |
76.25% |
207.23s |
| 4 | symplectic_adam (SymplecticAdam (Gromov-Liouville Non-Squeezing Floor + Casimir Cavity)) |
0.1391 |
0.6537 |
75.00% |
83.57% |
83.75% |
76.25% |
261.38s |
| 5 | huygens_symplectic_adam (HuygensSymplecticAdam (Two-Stage Huygens -> Symplectic Curriculum)) |
0.1391 |
0.6537 |
75.00% |
83.57% |
83.75% |
76.25% |
146.70s |
| 6 | gyric_adam (GyricAdam (Poinsot-Lande Gyroscopic Projection + Thomas Precession)) |
0.1393 |
0.6541 |
75.00% |
83.57% |
83.75% |
76.25% |
221.44s |
| 7 | huygens_adam (HuygensAdam (Circular Order Parameter R_p Splay Torque)) |
0.1378 |
0.6557 |
75.00% |
83.57% |
83.75% |
76.25% |
152.14s |
| 8 | hessian_pair_adam (HessianPairAdam (Sophia-Torus Cauchy-Riemann Curvature)) |
0.1387 |
0.6506 |
73.75% |
83.39% |
83.75% |
75.00% |
232.96s |
| — | Step-0 Initialized Rotational Head (Before Adaptation) | — | 8.9793 |
0.00% |
0.45% |
3.75% |
0.00% |
— |
Quickstart (transformers)
import torch
from transformers import AutoTokenizer, GRTForQuestionAnswering
model_id = "MatterWaveAI/grt-30m-qa"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = GRTForQuestionAnswering.from_pretrained(model_id)
prompt = (
"Question: Who serves as the chief architect at MatterWave AI?\n"
"Context: While Elena Rostova directs European Space Agency in Central Zurich, "
"the chief architect at MatterWave AI leading Givens Rotational Transformer is David Householter."
)
inputs = tokenizer(prompt, return_tensors="pt")
with torch.no_grad():
outputs = model(**inputs)
s = int(outputs.start_logits.argmax(dim=-1).item())
e = int(outputs.end_logits.argmax(dim=-1).item())
print("Extracted Answer:", tokenizer.decode(inputs.input_ids[0][s : e + 1]).strip())
License
Apache 2.0
- Downloads last month
- 21
Model tree for MatterWaveAI/grt-30m-qa
Base model
MatterWaveAI/grt-30m
# Use a pipeline as a high-level helper # Warning: Pipeline type "question-answering" is no longer supported in transformers v5. # You must load the model directly (see below) or downgrade to v4.x with: # pip install "transformers<5.0.0" from transformers import pipeline pipe = pipeline("question-answering", model="MatterWaveAI/grt-30m-qa")