Blame for README.md

---

license: apache-2.0
library_name: safetensors
tags:
  - arc-agi
  - abstract-reasoning
  - geometric-deep-learning
  - world-model
  - test-time-training
  - rotational-transformer
---


# GRT-ARC-v2 (`Smol`, `Large`, `XL`): Abstract Visual Reasoning from 114 KB to 1.49 MB with a Pure-Rotational 2D Topological World Model

**Team / Organization:** MatterWave AI  
**Model Family:** Geometric Rotation Transformer Scaling Ladder (`GRT-ARC-v2-Smol`, `GRT-ARC-v2-Large`, `GRT-ARC-v2-XL`)  
**Architectural Class:** 100% Pure-Rotational 2D Grid Transducer + Compositional Topological Solver (`0 nn.Linear`, `0 nn.Conv1d`, `0 nn.Conv2d`)  
**Model Footprints:**  
- **`GRT-ARC-v2-Smol`**: `26,950` parameters (**`114 KB`** `.safetensors`)  
- **`GRT-ARC-v2-Large`**: `139,906` parameters (**`572 KB`** `.safetensors`)  
- **`GRT-ARC-v2-XL`**: `369,578` parameters (**`1.49 MB`** `.safetensors`)  
**Inference Regime:** Fully Offline (`Internet = Off`), Single-CPU / Single-GPU Test-Time Geometric Adaptation (`~24–36s` across all `400` official evaluation tasks)

---

## 1. Overview: Rethinking `ARC-AGI` from First Principles

The **Abstraction and Reasoning Corpus (`ARC-AGI`)** was designed to measure something current foundation models struggle to do: **induce a novel transformation rule from just 2 to 5 visual examples** and generalize it to an unseen test canvas.

Today, the dominant approach on `ARC-AGI` is to take a massive 1D autoregressive language model (`7B` to `70B+` parameters, `14 GB` to `140 GB` of weights), flatten 2D grids into text strings, and sample thousands of candidate completions at test time. While brute-force scale can memorize surface patterns, dense 1D sequence models face three structural bottlenecks on abstract grid reasoning:

1. **Loss of 2D Spatial Continuity:** Flattening a 2D canvas into a 1D token stream breaks vertical, diagonal, and cyclic spatial relationships across rows.
2. **Topological Blindness:** Standard dense linear layers and local convolutions do not natively represent global topological properties—such as **inside vs. outside**, **closed loops vs. open contours**, **containment**, or **connected-component hierarchy**—without deep, parameter-heavy stacks.
3. **Symmetry Fragility:** In `ARC-AGI`, a single underlying rule remains valid across 2D rotations, reflections, canvas rescalings, and color relabelings. Standard Euclidean weight matrices must re-learn these symmetries from scratch through massive data augmentation.

With the **`GRT-ARC-v2` Scaling Ladder (`Smol` → `Large` → `XL`)**, we explore the opposite extreme of the compute-and-parameter spectrum:
> **Can a family of compact (`114 KB` to `1.49 MB`) models built with zero dense linear layers (`0 nn.Linear`) and zero convolutions (`0 nn.Conv2d`) induce and compose abstract 2D transformations directly in rotational phase space?**

---

## 2. System Architecture & Core Formulation

```mermaid

flowchart LR

    subgraph Context["Few-Shot Demonstration Context (2–5 Pairs)"]

        Demos["Demonstration Pairs\n{(Input_i, Output_i)}"] --> Enc["2D Pure-Rotational\nTopological Encoder"]

        Enc --> Rule["Geometric Rule Induction\n& Compositional DSL Beam"]

    end



    subgraph Adaptation["Test-Time Geometric Adaptation"]

        Rule --> TTT["Leave-One-Out Test-Time\nRotational Adaptation"]

        TestIn["Unseen Test Canvas"] --> TTT

        TTT --> Vote["2D Symmetry Consensus\n& 100% Demo Verification"]

        Vote --> Out["Top-2 Predictions\n(attempt_1, attempt_2)"]

    end

```

Rather than relying on dense matrix multiplications (`y = Wx + b`), `GRT-ARC-v2` is built around four geometric design principles with a simple, norm-preserving mathematical formulation:

### Pillar I — Pure-Rotational 2D Grid Representation (`0 nn.Linear`, `0 nn.Conv2d`)
Every layer in `GRT-ARC-v2` preserves feature norm and operates strictly through structured `2 × 2` planar rotations across `d_model / 2` paired channels. A grid cell at 2D coordinates `(r, c)` with discrete color symbol `k ∈ {0, ..., 9}` is embedded directly onto the unit circle as a phase vector `z(r, c) = (u, v)` and transformed via block-diagonal Givens rotations `R(θ)`:

$$
R(\theta)(u, v) = \Big( u \cos\theta - v \sin\theta, \quad u \sin\theta + v \cos\theta \Big), \qquad z'(r, c) = R\Big(\theta + \omega(r, c)\Big) z(r, c)
$$

Because `‖R(θ) · z‖₂ = ‖z‖₂` for any angle `θ`, signal magnitude never explodes or vanishes across deep stacks, and 2D spatial translations `(Δr, Δc)` reduce to simple angle addition `ω(r + Δr, c + Δc) = ω(r, c) + α·Δr + β·Δc` rather than dense learned position matrices.

### Pillar II — Native 2D Topological & Enclosure Awareness
Many `ARC-AGI` puzzles hinge on whether a region is **enclosed inside a boundary**, whether a contour forms a **closed loop**, or how many **interior cavities** an object contains. Prior to rotational message passing, `GRT-ARC-v2` computes a parameter-free 2D topological feature field `Φ(r, c)` from the grid's connected components:

$$
\Phi(r, c) = \Big( \operatorname{Enclosed}(r, c), \quad \beta_1(C), \quad \operatorname{Area}(C), \quad d(r, c) \Big)

$$



where `Enclosed(r, c) ∈ {0, 1}` is determined by flood-filling background cells reachable from the canvas border (any background cell not reachable from the border is topologically enclosed inside a closed loop), and `β₁(C)` counts the number of distinct interior cavities inside connected component `C`. These topological invariants directly modulate the local rotation angles `θ(r, c)`.



### Pillar III — Few-Shot Rule Induction as an Angular Transition Operator

Instead of treating demonstration pairs `(X^(i), Y^(i))` as a flat 1D prompt, `GRT-ARC-v2` encodes each demonstration input `X^(i)` and output `Y^(i)` into rotational states and extracts their **angular transition difference** `Δθ^(i) = arg Z(Y^(i)) − arg Z(X^(i))`. Across `M` demonstration pairs, the task rule operator `Δθ` is computed via the circular mean on the unit circle:



$$

\Delta\theta = \operatorname{atan2}\left( \frac{1}{M}\sum_{i=1}^{M} \sin\big(\Delta\theta^{(i)}\big), \quad \frac{1}{M}\sum_{i=1}^{M} \cos\big(\Delta\theta^{(i)}\big) \right)

$$



In parallel, our compositional topological solver searches 1-hop, 2-hop, and 3-hop symbolic-topological programs `P` and enforces a strict **100% demonstration verification gate** before any candidate rule is selected:



$$

\operatorname{Verify}(\mathcal{P}) = 1 \quad \Longleftrightarrow \quad \mathcal{P}\big(X^{(i)}\big) = Y^{(i)} \quad \forall i \in [1, M]

$$



### Pillar IV — Test-Time Geometric Adaptation & Symmetry Consensus

When a puzzle requires continuous neural adaptation at test time, `GRT-ARC-v2` constructs **leave-one-out (`LOO`) validation episodes** from the `M` training demonstrations—holding out demonstration `(X^(j), Y^(j))` as a pseudo-test target while conditioning on the remaining `M − 1` pairs—and minimizes a composite geometric loss with respect to the model's rotation angles `θ`:



$$

\mathcal{L}(\theta) = \mathcal{L}_{\text{CE}}\big(\hat{Y}^{(j)}, Y^{(j)}\big) + \lambda \Big( 1 - \cos\big(\Delta\theta^{(j)} - \Delta\theta^{(-j)}\big) \Big) + \gamma \Big| \operatorname{shape}\big(\hat{Y}^{(j)}\big) - \operatorname{shape}\big(Y^{(j)}\big) \Big|
$$

Finally, predictions are evaluated across all `8` dihedral (`D4`) canvas orientations (`4` rotations × `2` reflections), inverse-transformed back into the canonical grid frame, and ranked by demonstration consistency to emit `attempt_1` and `attempt_2` (`pass@2`).

---

## 3. Empirical Results & 3-Model Scaling Ladder (`Smol` → `Large` → `XL`)

### 3.1 Parameter, Footprint & Official 400-Task `ARC-AGI` Scaling Comparison

| Model / Baseline | Linear / Conv Layers | `(d_model, L, H, R)` | Parameter Count | Checkpoint Size | Official 400-Task `pass@1` | Official 400-Task `pass@2` | 400-Task Runtime |
| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| Standard `7B` Dense LLM (Zero-Shot) | Hundreds | `(4096, 32, 32, -)` | `~7,000,000,000` | `~14.0 GB` | `~9% – 14%` | `~12% – 16%` | Multi-GPU Hours |
| **`GRT-ARC-v2-Smol` (Ours)** | **`0`** | `(256, 6, 8, 4)` | **`26,950`** | **`114 KB`** | **`35 / 400` (`8.75%`)** | **`35 / 400` (`8.75%`)** | **`21.0s`** |
| **`GRT-ARC-v2-Large` (Ours)** | **`0`** | `(512, 12, 16, 8)` | **`139,906`** | **`572 KB`** | **`76 / 400` (`19.00%`)** | **`77 / 400` (`19.25%`)** | **`23.8s`** |
| **`GRT-ARC-v2-XL` (Ours)** | **`0`** | `(768, 16, 24, 12)` | **`369,578`** | **`1.49 MB`** | **`119 / 400` (`29.75%`)** | **`121 / 400` (`30.25%`)** | **`20.6s`** |

Even our largest model (**`GRT-ARC-v2-XL`**, `369.6K` parameters, `1.49 MB`) is **19,000× smaller** than a `7B` dense language model while solving **`121 / 400` (`30.25%`)** of the official evaluation split, and **`GRT-ARC-v2-Smol`** (`114 KB`) is **250,000× smaller**—allowing the entire model family and solver to execute across all `400` official evaluation tasks in **`~21 seconds`** on a single CPU or GPU.

---

### 3.2 Training Data Provenance, Procedural Synthetic Generation & 6-Stage Curriculum

**1. Data Sources & Strict Zero Evaluation Contamination:**
Training relies exclusively on two strictly isolated sources:
- **On-the-Fly Procedural Synthetic Episodes (Stages 1–6):** Unlimited few-shot tasks generated dynamically in memory during training—never from static pre-rendered datasets.
- **Official Public `ARC-AGI` Training Split (`400` Tasks, Stages 5–6):** The `400` public training puzzles from `ARC-AGI/data/training`.
- **Zero Evaluation Leakage:** Zero tasks from the `400`-task official evaluation split (`ARC-AGI/data/evaluation`) are ever seen during pretraining, curriculum consolidation, or hyperparameter tuning.

**2. How We Synthetically Generate Few-Shot Episodes (5-Tier Procedural Generator):**
Rather than training on single grid pairs, our procedural generator synthesizes complete few-shot episodes `E = {(X_1, Y_1), ..., (X_M, Y_M), (X_test, Y_test)}` (`M = 3` demos + `1` query pair per episode) on random canvas dimensions `(H, W) ∈ [5×5, 20×20]`. All pairs within an episode share a single sampled latent program `T`, while their object layouts, positions, and canvas sizes vary independently across demos:
- **Tier 1 — Atomic Dihedral & Spatial Geometry:** Procedurally places multi-color connected sprites and L-ominoes on a black background and applies a sampled dihedral group operation `g ∈ D4` (rotations `90°/180°/270°`, horizontal/vertical/diagonal flips), periodic translations, or tight foreground bounding-box extraction.
- **Tier 2 — Topological Containment & Closed-Loop Cavities:** Generates closed rectangular and polygonal loops of random dimensions alongside unclosed distractor segments, then applies flood-fill reachability from the canvas border so the target rule fills only strictly enclosed interior cavities (`is_enclosed(r, c) == 1`) with a target color.
- **Tier 3 — Relational Object Ranking, Counting & Gravity:** Synthesizes `2–5` disjoint connected components with distinct guaranteed area ranks (`Area(C_1) < Area(C_2) < ...`) or free foreground particles above a barrier line; the ground-truth rule recolors/isolates the largest (`argmax area`) or smallest (`argmin area`) object, or simulates directional gravity until collision.
- **Tier 4 — Multi-Hop Compositional Programs (`2–3` Hops):** Samples composed operators `T(X) = (T_2 ∘ T_1)(X)` that chain a topological or relational filter (such as enclosed cavity fill or largest-object extraction) with a geometric `D4` rotation and a deterministic color permutation.
- **Tier 5 — Conditional Multi-Object Topology:** Constructs multi-object scenes mixing hollow frames (`hole_count >= 1`) and solid shapes (`hole_count == 0`), where the transformation rule branches conditionally on both topological genus and relative object area.

**3. Dynamic Group-Equivariant Augmentation (`D4 × S9`):**
Every synthetic and `ARC-AGI` training episode is dynamically augmented on the fly with:
1. **Dihedral Orientation Sampling (`D4`, `8` views):** A random rotation/reflection `g ∈ D4` applied across every input and output grid in the episode.
2. **Foreground Color Permutation (`S9`, `9! = 362,880` bijections):** A random bijection over the `9` non-black colors `{1, ..., 9}` (keeping background `0` fixed) applied consistently across the episode so the model learns structural and topological rules rather than memorizing specific color IDs.

**4. 6-Stage Curriculum Progression:**

| Curriculum Stage | Data Mix & Abstraction Focus | Final Total Loss | Task-Consistency Loss | Canvas Shape Error | Training Cell Accuracy |
| :---: | :--- | :---: | :---: | :---: | :---: |
| **Stage 1** | **Tier 1 Synthetic (`100%`):** Atomic Spatial & Dihedral Symmetries | `0.4671` | `0.0019` | `0.0000` | **`91.4%`** |
| **Stage 2** | **Tiers 1–2 Synthetic:** Topological Containment, Loops & Enclosure | `0.6162` | `0.0022` | `0.0000` | **`85.6%`** |
| **Stage 3** | **Tiers 1–3 Synthetic:** Relational Area Ranking, Counting & Gravity | `0.4730` | `0.0010` | `0.0000` | **`91.0%`** |
| **Stage 4** | **Tiers 1–4 Synthetic:** Multi-Hop Compositional Rules (`2–3` Hops) | `0.7795` | `0.0074` | `0.0000` | **`82.4%`** |
| **Stage 5** | **Tier 5 Synthetic (`70%`) + `ARC-AGI` Train Split (`30%`):** Conditional Topology | `0.9525` | `0.0670` | `0.0000` | **`72.9%`** |
| **Stage 6** | **All Tiers 1–5 (`50%`) + `ARC-AGI` Train Split (`50%`):** Unified Consolidation | `0.7497` | `0.0384` | `0.0000` | **`78.3%`** |

---

### 3.3 Held-Out 5-Tier Benchmark & Official `ARC-AGI` Evaluation Results

| Evaluation Benchmark | Unseen Tasks | Exact Grid `pass@1` | Exact Grid `pass@2` | Mean Cell Accuracy | Leave-One-Out TTT Loss (`Before → After`) |
| :--- | :---: | :---: | :---: | :---: | :---: |
| **Held-Out Tiers 1–5** (Procedural Generalization) | `40` | **`100.0%` (`40/40`)** | **`100.0%` (`40/40`)** | **`100.0%`** | **`0.6806 → 0.4537` (`-33.3%`)** |
| **Official `ARC-AGI` Eval Subset (Neural Angular TTT)** | `24` | **`58.33%` (`14/24`)** | **`62.50%` (`15/24`)** | **`82.02%`** | **`1.9817 → 1.2488` (`-37.0%`)** |
| **Full 400-Task `ARC-AGI` Eval (`GRT-ARC-v2-Smol`, `114 KB`)** | `400` | **`35 / 400` (`8.75%`)** | **`35 / 400` (`8.75%`)** | `74.8%` | Zero-leak `100%` demo-verified |
| **Full 400-Task `ARC-AGI` Eval (`GRT-ARC-v2-Large`, `572 KB`)** | `400` | **`76 / 400` (`19.00%`)** | **`77 / 400` (`19.25%`)** | `79.6%` | Zero-leak `100%` demo-verified (`79` verified) |
| **Full 400-Task `ARC-AGI` Eval (`GRT-ARC-v2-XL`, `1.49 MB`)** | `400` | **`119 / 400` (`29.75%`)** | **`121 / 400` (`30.25%`)** | `84.2%` | Zero-leak `100%` demo-verified (`122` verified) |

---

### 3.4 Taxonomy of the `121` Solved Official `ARC-AGI` Evaluation Puzzles

Across the `Smol` → `Large` → `XL` scaling ladder, `GRT-ARC-v2-XL` solves **`121 / 400` (`30.25%`)** official `ARC-AGI` evaluation tasks (`119 / 400` = `29.75%` `pass@1`, with `122 / 400` 100%-demo-verified rules and `99.2%` verification precision) under strict zero test-set leakage across nine reasoning families:

1. **Multi-Panel Relational Boolean Algebra, `2×2` Quadrant Overlays & Parity Selection (`22` tasks — including `0c9aba6e`, `195ba7dc`, `19bb5feb`, `281123b4`, `31d5ba1a`, `34b99a2b`, `3d31c5b3`, `506d28a5`, `5d2a5c43`, `66f2d22f`, `6a11f6da`, `8ba14f53`, `93b4f4b3`, `c658a4bd`, `d19f7514`, `d47aa2ff`, `e133d23d`, `e345f17b`, `e99362f0`, `ea9794b1`):** Detecting horizontal/vertical/cross separator bars or equal quadrant splits and executing cell-wise logical compositions (`XOR`, `NOR`, `OR`, equality/difference maps), depth-ordered `2×2` quadrant overlays, or panel-consensus summaries.
2. **Dihedral (`D4`), Periodic Wallpaper & Rotational Inpainting (`17` tasks — including `1d0a4b61`, `67b4a34d`, `7c9b52a0`, `903d1b4a`, `929ab4e9`, `981571dc`, `9ddd00f0`, `af22c60d`, `c663677b`, `ca8f78db`, `ce039d91`, `e66aafb8`, `e95e3d8e`, `f4081712`, `f823c43c`):** Reconstructing occluded rectangular masks across dihedral symmetry planes, `4`-way rotational canvas completions, and 2D translational wallpaper lattices.
3. **Kronecker Self-Tiling, Fractal Substitutions & Learned Dihedral Block Assemblies (`21` tasks — including `00576224`, `0692e18c`, `0c786b71`, `1990f7a8`, `3ee1011a`, `48131b3c`, `5b6cbef5`, `60c09cac`, `7953d61e`, `833dafe3`, `8e2edd66`, `a59b95c0`, `bc4146bd`, `c48954c1`, `ccd554ac`, `ed98d772`, `f0afb749`):** Inducing self-Kronecker products, selective foreground-masked fractals, concentric frame scales, and multi-block dihedral reflection tilings (`2×2`, `3×3`, and `1×5`).
4. **Topological Enclosure, Betti-1 Hole Counting & Loop Halos (`14` tasks — including `00dbd492`, `09c534e7`, `0a2355a6`, `12eac192`, `18419cfa`, `2072aba6`, `37d3e8b2`, `42918530`, `84f2aca1`, `ae58858e`, `af24b4cc`, `e0fb7511`, `e7639916`):** Measuring interior cavity area, counting enclosed Betti-1 holes per connected component, stamping template patterns inside hollow frames, and coloring inner/outer topological boundary halos.
5. **Topological Invariant Subgrid, Marker-Crop & Frame Extraction (`14` tasks — including `0a1d4ef5`, `0bb8deee`, `1a6449f1`, `358ba94e`, `58743b76`, `73182012`, `7d18a6fb`, `8597cfd7`, `9a4bb226`, `b7999b51`, `cd3c21df`, `d56f2372`):** Cropping to the unique-area, unique-color, maximum-diversity, or axially symmetric connected component, extracting regions bounded by corner markers, or ranking vertical/horizontal bars.
6. **Ray Casting, Crosshairs, Diagonal Extensions & Orthogonal Span Bridges (`9` tasks — including `070dd51e`, `0b17323b`, `15663ba9`, `15696249`, `21f83797`, `45bbe264`, `54db823b`, `aa18de87`):** Casting ordered horizontal/vertical/diagonal rays, extrapolating arithmetic step sequences, synthesizing crosshair intersections, and filling spans between aligned anchors.
7. **Shape-Invariance Matching, Cellular Neighborhoods & Legend Swaps (`10` tasks — including `0becf7df`, `1e81d6f9`, `2546ccf6`, `2a5f8217`, `45737921`, `4cd1b7b2`, `4ff4c9da`, `575b1a71`, `845d6e51`, `c7d4e6ad`):** Matching uncolored shapes against colored reference legends up to dihedral rotation, completing quadrant mirrors, and applying deterministic `3×3` local cellular rules.
8. **Directional Gravity, Self-Width Translation & Barrier Sliding (`5` tasks — including `2685904e`, `27a77e38`, `5ffb2104`, `64a7c07e`, `9c56f360`):** Sliding rigid components or foreground particles until collision, and translating components by their own bounding-box width/height.
9. **Multi-Hop Compositional Beam Search & Block Condensation (`9` tasks — including `3b4c2228`, `5783df64`, `68b67ca3`, `6ea4a07e`, `7039b2d7`, `be03b35f`, `ce8d95cc`, `e1baa8a4`, `e57337a4`):** Chaining quadrant extraction with dihedral rotations (`quad_tl → d4_1`), deduplicating consecutive rows/columns, and condensing `2×2` and `3×3` blocks into macro-grids.

---

## 4. Key Takeaways & Future Directions

1. **A Clear Scaling Law in Pure-Rotational Space (`114 KB` → `572 KB` → `1.49 MB`):** Scaling `GRT-ARC-v2` from `Smol` (`26.9K` params) to `Large` (`139.9K` params) and `XL` (`369.6K` params) while expanding the compositional topological DSL more than triples zero-leak accuracy on the official 400-task evaluation split from **`8.75%` (`35/400`)** to **`19.25%` (`77/400`)** and **`30.25%` (`121/400`)**, while executing all `400` tasks in just **`20.6 seconds`**.
2. **Fast Angular Test-Time Adaptation:** Adapting only rotational phase angles on leave-one-out demonstration episodes reduces held-out demonstration loss by **`33%–42%`** in just `4–12` optimization steps without overfitting or collapsing on `2–5` examples.
3. **Scaling the Pure-Rotational World-Model Ladder:** `GRT-ARC-v2` (`Smol`, `Large`, `XL`) demonstrates that `0 nn.Linear` / `0 nn.Conv2d` rotational manifolds paired with compositional topological induction achieve strong abstract visual reasoning at a tiny fraction of standard LLM compute and memory.

---

## 5. Model Checkpoints (`.safetensors`) & Verification Snippet

This repository provides the three pure-rotational `.safetensors` checkpoints (`0 nn.Linear`, `0 nn.Conv2d`) across the `GRT-ARC-v2` scaling ladder:

| Checkpoint File | `(d_model, L, H, R)` | Parameter Count | File Size | Official 400-Task `ARC-AGI` `pass@1` | Official 400-Task `ARC-AGI` `pass@2` |
| :--- | :---: | :---: | :---: | :---: | :---: |
| `grt_arc_v2_smol.safetensors` | `(256, 6, 8, 4)` | `26,950` | `114 KB` | `35 / 400` (`8.75%`) | `35 / 400` (`8.75%`) |
| `grt_arc_v2_large.safetensors` | `(512, 12, 16, 8)` | `139,906` | `572 KB` | `76 / 400` (`19.00%`) | `77 / 400` (`19.25%`) |
| `grt_arc_v2_xl.safetensors` | `(768, 16, 24, 12)` | `369,578` | `1.49 MB` | `119 / 400` (`29.75%`) | `121 / 400` (`30.25%`) |

### Inspecting Checkpoint Tensors & Parameter Counts

```python

from safetensors.torch import load_file



for name in ["grt_arc_v2_smol.safetensors", "grt_arc_v2_large.safetensors", "grt_arc_v2_xl.safetensors"]:

    sd = load_file(name)

    n_params = sum(t.numel() for t in sd.values())

    n_rot, half_d = sd["blocks.0.q_angles"].shape

    n_blocks = max(int(k.split(".")[1]) for k in sd if k.startswith("blocks.")) + 1

    print(f"{name:28s} | params={n_params:7,d} | d_model={half_d*2} | blocks={n_blocks:2d} | rotations={n_rot:2d} | Linear/Conv2d=0")

```