---
license: apache-2.0
library_name: transformers
pipeline_tag: text-generation
tags:
  - grt
  - code
  - python
  - tool-calling
  - function-calling
  - rotational-transformer
  - givens-rotations
  - clifford-algebra
  - matterwave
base_model: MatterWaveAI/grt-30m
---

# GRT-30M-Code (`MatterWaveAI/grt-30m-code`)

**GRT-30M-Code** is the **Python algorithmic code synthesis and structured JSON tool-calling specialist** fine-tuned from [`MatterWaveAI/grt-30m`](https://huggingface.co/MatterWaveAI/grt-30m) (`GRTForCausalLM`, `29,305,498` parameters, **zero `nn.Linear` layers**, **zero `nn.Conv1d` layers**).

It specializes the 12-layer Givens Rotational Transformer backbone for:
1. **Type-Annotated Python Algorithmic Synthesis**: Generating clean, syntactically valid (`ast.parse`) Python functions across search, graph/array dynamic programming, number theory, string parsing, and linear algebra.
2. **Structured `<tool_call>` JSON Invocation**: Emitting schema-compliant `{"name": ..., "arguments": {...}}` JSON tool invocations conditioned on developer instructions.

## Code & Tool-Calling Alignment Metrics

- **Held-Out Code & Tool-Call Validation Perplexity**: `54.93` (`val_loss = 4.0061` base) → **`1.7632`** (`val_loss = 0.5672` w/ `HuygensAdam`; `1.8253` w/ `PairAdam`, `1.8619` w/ `GyricAdam`)
- **Held-Out Python Code Token Top-1 Accuracy**: `46.80%` → **`99.07%`** (`HuygensAdam`; `98.93%` w/ `CyclideAdam`, `98.53%` w/ `GyricAdam`)
- **Held-Out Python AST Syntax Validity (`ast.parse`)**: `0.00%` → **`91.67%`** (`HuygensAdam`; `83.33%` w/ `CyclideAdam`)
- **Held-Out Structured Tool-Call Token Top-1 Accuracy**: `36.44%` → **`90.69%`** (`HuygensAdam` & `PairAdam`; `89.50%` w/ `CyclideAdam`)
- **Held-Out JSON Tool-Call Schema Match Rate**: `0.00%` → **`83.33%`** (`HuygensAdam` & `PairAdam`; `75.00%` w/ `CyclideAdam`, `GyricAdam`, `HuygensSymplecticAdam`)
- **Code Signature/Docstring Conditioning Margin (`ΔL`)**: `0.8998` → **`1.6597` nats/tok** (`PairAdam`) / **`1.5766` nats/tok** (`SymplecticAdam`)

---

## 8-Way Classical Geometric Optimizer Ablation

We benchmarked all 8 classical geometric, Riemannian, conic, symplectic, gyroscopic, and synchronization optimizers from `transformers.models.grt.optimization_grt` on `MatterWaveAI/grt-30m-code` (`200` SFT steps on `blocks[11:12]` + `final_norm` + `<|im_end|>` polar phase calibration, `811,907` trainable parameters, `batch_size = 16`, `lr = 6.5e-3`, `max_angular_step = 0.12`, `seed = 42`):

| Rank | Optimizer (`optimizer_type`) | Final Train Loss | Val Loss | Val PPL | Code Token Acc (%) | Tool-Call Token Acc (%) | JSON Schema Match (%) | Python AST Valid (%) | Conditioning Margin (`ΔL`) | Wall Time (s) |
| :---: | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| **1** | **`HuygensAdam` (`huygens_adam`)** | **`1.0882`** | **`0.5672`** | **`1.7632`** | **`99.07%`** | **`90.69%`** | **`83.33%`** | **`91.67%`** | `1.5152` | `138.63s` |
| **2** | **`PairAdam` (`pair_adam`)** | `1.0994` | `0.6018` | `1.8253` | `97.60%` | **`90.69%`** | **`83.33%`** | `50.00%` | **`1.6597`** | `278.80s` |
| **3** | **`GyricAdam` (`gyric_adam`)** | `1.1168` | `0.6216` | `1.8619` | `98.53%` | `87.92%` | `75.00%` | `50.00%` | `1.5745` | `138.83s` |
| **4** | **`CyclideAdam` (`cyclide_adam`)** | `1.1564` | `0.6400` | `1.8965` | `98.93%` | `89.50%` | `75.00%` | `83.33%` | `1.5727` | `141.92s` |
| **5** | **`HuygensSymplecticAdam` (`huygens_symplectic_adam`)** | `1.1819` | `0.6537` | `1.9227` | `98.27%` | `89.31%` | `75.00%` | `66.67%` | `1.4057` | `122.82s` |
| **6** | **`HessianPairAdam` (`hessian_pair_adam`)** | `1.2171` | `0.7292` | `2.0734` | `97.87%` | `87.33%` | `66.67%` | `58.33%` | `1.4236` | `117.82s` |
| **7** | **`PolarNaturalAdam` (`polar_natural_adam`)** | `1.2463` | `0.7891` | `2.2015` | `96.93%` | `86.34%` | `66.67%` | `50.00%` | `1.4294` | **`108.18s`** |
| **8** | **`SymplecticAdam` (`symplectic_adam`)** | `1.2764` | `0.8445` | `2.3268` | `97.60%` | `81.39%` | `50.00%` | `50.00%` | `1.5766` | `138.10s` |

### Key Geometric Findings on Python Code & Structured Tool Calling
- **Huygens Circular Synchronization Splay (`HuygensAdam`) Sweeps Code & Tool-Call Generation**: By preventing butterfly rotation stages from collapsing onto repetitive indentation/delimiter tokens, `HuygensAdam` achieves **#1 validation loss (`0.5672` / `1.7632` PPL)**, **#1 Python code token accuracy (`99.07%`)**, **#1 Python AST syntax validity (`91.67%`)**, and **#1 tool-call token accuracy (`90.69%`) & JSON schema match rate (`83.33%`)**.
- **Dupin Cyclide Conic Metric (`CyclideAdam`) Excels at Syntactic Delimiter Balance**: `CyclideAdam` achieves **`98.93%` Python code token accuracy** and **`83.33%` Python AST syntax validity** (#2 overall on AST validity).

---

## Quickstart (`transformers`)

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "MatterWaveAI/grt-30m-code"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.float32)

prompt = (
    "<|im_start|>system\n"
    "You are GRT-30M-Code, a Python algorithmic coding and structured JSON tool-calling specialist "
    "built by MatterWave AI on the 100% rotational Givens Rotational Transformer architecture.<|im_end|>\n"
    "<|im_start|>user\n"
    "Write a Python function `gcd(a: int, b: int) -> int` using Euclid's iterative algorithm.<|im_end|>\n"
    "<|im_start|>assistant\n"
)
inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=False)
out_ids = model.generate(**inputs, max_new_tokens=48, do_sample=False)
print(tokenizer.decode(out_ids[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))
```

---

## License

Apache 2.0