Spaces:
Running
Running
docs: remove confusing 20m-curriculum link on HuygensSymplecticAdam
Browse files
README.md
CHANGED
|
@@ -34,7 +34,7 @@ _**Purely Rotational Foundation Models & Geometric Optimization**_
|
|
| 34 |
|
| 35 |
Because every transform in the GRT residual path is an orthogonal rotation cascade, hidden-state norms are preserved by construction rather than by post-hoc rescaling. GRT checkpoints execute deterministically with bit-for-bit greedy generation parity across **PyTorch** (CUDA, ROCm, Intel XPU), **vLLM**, and **llama.cpp** (CPU AVX2, Vulkan, and ROCm HIP).
|
| 36 |
|
| 37 |
-
Starting from the exact same pretrained checkpoints built for our NeurIPS submission with first-generation **PairAdam**, replacing the optimizer with **
|
| 38 |
|
| 39 |
* **[GRT-10M-Curriculum (8.7M total / 2.2M backbone)](https://huggingface.co/MatterWaveAI/grt-10m-curriculum)** — Lifts 9-task multiple-choice accuracy from **37.61%** (PairAdam) to **41.71%** (Single-Pass) to **44.57%** (Multi-Round Curriculum, **+6.96%**), while GSM8K 4-way verification reaches **95.80%** (perplexity reduced **5.77×** from 145.85 to **25.28**).
|
| 40 |
* **[GRT-20M-Curriculum (16.7M total / 10.3M backbone)](https://huggingface.co/MatterWaveAI/grt-20m-curriculum)** — Lifts 9-task multiple-choice accuracy from **38.63%** (PairAdam) to **45.03%** (Single-Pass) to **47.36%** (Multi-Round Curriculum, **+8.73%**), while GSM8K 4-way verification reaches **97.60%** (perplexity reduced **7.86×** from 60.12 to **7.65**).
|
|
|
|
| 34 |
|
| 35 |
Because every transform in the GRT residual path is an orthogonal rotation cascade, hidden-state norms are preserved by construction rather than by post-hoc rescaling. GRT checkpoints execute deterministically with bit-for-bit greedy generation parity across **PyTorch** (CUDA, ROCm, Intel XPU), **vLLM**, and **llama.cpp** (CPU AVX2, Vulkan, and ROCm HIP).
|
| 36 |
|
| 37 |
+
Starting from the exact same pretrained checkpoints built for our NeurIPS submission with first-generation **PairAdam**, replacing the optimizer with **HuygensSymplecticAdam** and training across a multi-round curriculum delivers substantial improvements across the 10-task **SLM-Bench** evaluation suite (13,144 held-out multiple-choice examples across BoolQ, ARC-Easy, ARC-Challenge, OpenBookQA, PIQA, WinoGrande, HellaSwag, QNLI, and CommonsenseQA, plus 500 held-out GSM8K mathematical reasoning problems):
|
| 38 |
|
| 39 |
* **[GRT-10M-Curriculum (8.7M total / 2.2M backbone)](https://huggingface.co/MatterWaveAI/grt-10m-curriculum)** — Lifts 9-task multiple-choice accuracy from **37.61%** (PairAdam) to **41.71%** (Single-Pass) to **44.57%** (Multi-Round Curriculum, **+6.96%**), while GSM8K 4-way verification reaches **95.80%** (perplexity reduced **5.77×** from 145.85 to **25.28**).
|
| 40 |
* **[GRT-20M-Curriculum (16.7M total / 10.3M backbone)](https://huggingface.co/MatterWaveAI/grt-20m-curriculum)** — Lifts 9-task multiple-choice accuracy from **38.63%** (PairAdam) to **45.03%** (Single-Pass) to **47.36%** (Multi-Round Curriculum, **+8.73%**), while GSM8K 4-way verification reaches **97.60%** (perplexity reduced **7.86×** from 60.12 to **7.65**).
|