Release MatterWaveAI/grt-30m-qa (100% rotational head + 8-way geometric optimizer ablation)
92454d2 verified | license: apache-2.0 | |
| library_name: transformers | |
| pipeline_tag: question-answering | |
| tags: | |
| - question-answering | |
| - extractive-qa | |
| - squad | |
| - grt | |
| - rotational-transformer | |
| - givens-rotations | |
| - clifford-algebra | |
| - matterwave | |
| base_model: MatterWaveAI/grt-30m | |
| # GRT-30M-QA (`MatterWaveAI/grt-30m-qa`) | |
| **GRT-30M-QA** (`GRTForQuestionAnswering`) is a **100% Native Rotational Extractive Question Answering Model** (`29,306,020` parameters, **zero `nn.Linear` layers, zero `nn.Conv1d` layers**) built on `MatterWaveAI/grt-30m`. | |
| `GRTForQuestionAnswering` replaces traditional dense span projection layers (`nn.Linear(hidden_size, 2)`) with a 2-channel `GRTCosineClassificationHead` (`qa_outputs`) parameterized by planar-pair `phase_angles` (`[2, 128]`) and softplus radial `pair_scales` (`[2, 128]`), emitting `start_logits` and `end_logits` across context token positions. | |
| --- | |
| ## 8-Way Classical Geometric Optimizer Ablation (`GRTForQuestionAnswering`) | |
| All 8 classical geometric optimizers in `transformers.models.grt.optimization_grt` were benchmarked under an identical 200-step adaptation protocol (`seed = 42`, `batch_size = 24`, `backbone_lr = 1.8e-4`) unfreezing the top rotational transformer block `blocks[-1]` (`811,651` parameters), `final_norm` (`128` parameters), and `qa_outputs` (`523` parameters) for `812,430` total trainable parameters (`0` `nn.Linear`): | |
| | Rank | Optimizer (`optimization_grt.py`) | Final Train Loss | Val Span Loss | Exact Match (`EM %`) | Span Token F1 (`%`) | Start Acc (`%`) | End Acc (`%`) | Wall Time (`200` Steps) | | |
| | :---: | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | | |
| | **1** | **`polar_natural_adam`** (PolarNaturalAdam (Riemannian Polar Metric + Cauchy-Riemann Hessian)) | `0.1385` | `0.6512` | **`75.00%`** | **`83.57%`** | `83.75%` | `76.25%` | `219.08s` | | |
| | **2** | **`cyclide_adam`** (CyclideAdam (Dupin Cyclide Conic Metric + Deviatoric Twist)) | `0.1382` | `0.6520` | **`75.00%`** | **`83.57%`** | `83.75%` | `76.25%` | `225.19s` | | |
| | **3** | **`pair_adam`** (PairAdam (Factored Toroidal Baseline)) | `0.1383` | `0.6532` | **`75.00%`** | **`83.57%`** | `83.75%` | `76.25%` | `207.23s` | | |
| | **4** | **`symplectic_adam`** (SymplecticAdam (Gromov-Liouville Non-Squeezing Floor + Casimir Cavity)) | `0.1391` | `0.6537` | **`75.00%`** | **`83.57%`** | `83.75%` | `76.25%` | `261.38s` | | |
| | **5** | **`huygens_symplectic_adam`** (HuygensSymplecticAdam (Two-Stage Huygens -> Symplectic Curriculum)) | `0.1391` | `0.6537` | **`75.00%`** | **`83.57%`** | `83.75%` | `76.25%` | `146.70s` | | |
| | **6** | **`gyric_adam`** (GyricAdam (Poinsot-Lande Gyroscopic Projection + Thomas Precession)) | `0.1393` | `0.6541` | **`75.00%`** | **`83.57%`** | `83.75%` | `76.25%` | `221.44s` | | |
| | **7** | **`huygens_adam`** (HuygensAdam (Circular Order Parameter R_p Splay Torque)) | `0.1378` | `0.6557` | **`75.00%`** | **`83.57%`** | `83.75%` | `76.25%` | `152.14s` | | |
| | **8** | **`hessian_pair_adam`** (HessianPairAdam (Sophia-Torus Cauchy-Riemann Curvature)) | `0.1387` | `0.6506` | **`73.75%`** | **`83.39%`** | `83.75%` | `75.00%` | `232.96s` | | |
| | — | **Step-0 Initialized Rotational Head (Before Adaptation)** | — | `8.9793` | `0.00%` | `0.45%` | `3.75%` | `0.00%` | — | | |
| --- | |
| ## Quickstart (`transformers`) | |
| ```python | |
| import torch | |
| from transformers import AutoTokenizer, GRTForQuestionAnswering | |
| model_id = "MatterWaveAI/grt-30m-qa" | |
| tokenizer = AutoTokenizer.from_pretrained(model_id) | |
| model = GRTForQuestionAnswering.from_pretrained(model_id) | |
| prompt = ( | |
| "Question: Who serves as the chief architect at MatterWave AI?\n" | |
| "Context: While Elena Rostova directs European Space Agency in Central Zurich, " | |
| "the chief architect at MatterWave AI leading Givens Rotational Transformer is David Householter." | |
| ) | |
| inputs = tokenizer(prompt, return_tensors="pt") | |
| with torch.no_grad(): | |
| outputs = model(**inputs) | |
| s = int(outputs.start_logits.argmax(dim=-1).item()) | |
| e = int(outputs.end_logits.argmax(dim=-1).item()) | |
| print("Extracted Answer:", tokenizer.decode(inputs.input_ids[0][s : e + 1]).strip()) | |
| ``` | |
| --- | |
| ## License | |
| Apache 2.0 | |