Fix model card formatting (replace raw LaTeX with clean Unicode and code spans)

5115ebe verified 1 day ago

5.78 kB

license: apache-2.0
library_name: transformers
pipeline_tag: text-generation
tags:
  - grt
  - code
  - python
  - tool-calling
  - function-calling
  - rotational-transformer
  - givens-rotations
  - clifford-algebra
  - matterwave
base_model: MatterWaveAI/grt-30m

GRT-30M-Code (`MatterWaveAI/grt-30m-code`)

GRT-30M-Code is the Python algorithmic code synthesis and structured JSON tool-calling specialist fine-tuned from MatterWaveAI/grt-30m (GRTForCausalLM, 29,305,498 parameters, zero nn.Linear layers, zero nn.Conv1d layers).

It specializes the 12-layer Givens Rotational Transformer backbone for:

Type-Annotated Python Algorithmic Synthesis: Generating clean, syntactically valid (ast.parse) Python functions across search, graph/array dynamic programming, number theory, string parsing, and linear algebra.
Structured <tool_call> JSON Invocation: Emitting schema-compliant {"name": ..., "arguments": {...}} JSON tool invocations conditioned on developer instructions.

Code & Tool-Calling Alignment Metrics

Held-Out Code & Tool-Call Validation Perplexity: 54.93 (val_loss = 4.0061 base) → 1.7632 (val_loss = 0.5672 w/ HuygensAdam; 1.8253 w/ PairAdam, 1.8619 w/ GyricAdam)
Held-Out Python Code Token Top-1 Accuracy: 46.80% → 99.07% (HuygensAdam; 98.93% w/ CyclideAdam, 98.53% w/ GyricAdam)
Held-Out Python AST Syntax Validity (ast.parse): 0.00% → 91.67% (HuygensAdam; 83.33% w/ CyclideAdam)
Held-Out Structured Tool-Call Token Top-1 Accuracy: 36.44% → 90.69% (HuygensAdam & PairAdam; 89.50% w/ CyclideAdam)
Held-Out JSON Tool-Call Schema Match Rate: 0.00% → 83.33% (HuygensAdam & PairAdam; 75.00% w/ CyclideAdam, GyricAdam, HuygensSymplecticAdam)
Code Signature/Docstring Conditioning Margin (ΔL): 0.8998 → 1.6597 nats/tok (PairAdam) / 1.5766 nats/tok (SymplecticAdam)

8-Way Classical Geometric Optimizer Ablation

We benchmarked all 8 classical geometric, Riemannian, conic, symplectic, gyroscopic, and synchronization optimizers from transformers.models.grt.optimization_grt on MatterWaveAI/grt-30m-code (200 SFT steps on blocks[11:12] + final_norm + <|im_end|> polar phase calibration, 811,907 trainable parameters, batch_size = 16, lr = 6.5e-3, max_angular_step = 0.12, seed = 42):

Rank	Optimizer (`optimizer_type`)	Final Train Loss	Val Loss	Val PPL	Code Token Acc (%)	Tool-Call Token Acc (%)	JSON Schema Match (%)	Python AST Valid (%)	Conditioning Margin (`ΔL`)	Wall Time (s)
1	`HuygensAdam` (`huygens_adam`)	`1.0882`	`0.5672`	`1.7632`	`99.07%`	`90.69%`	`83.33%`	`91.67%`	`1.5152`	`138.63s`
2	`PairAdam` (`pair_adam`)	`1.0994`	`0.6018`	`1.8253`	`97.60%`	`90.69%`	`83.33%`	`50.00%`	`1.6597`	`278.80s`
3	`GyricAdam` (`gyric_adam`)	`1.1168`	`0.6216`	`1.8619`	`98.53%`	`87.92%`	`75.00%`	`50.00%`	`1.5745`	`138.83s`
4	`CyclideAdam` (`cyclide_adam`)	`1.1564`	`0.6400`	`1.8965`	`98.93%`	`89.50%`	`75.00%`	`83.33%`	`1.5727`	`141.92s`
5	`HuygensSymplecticAdam` (`huygens_symplectic_adam`)	`1.1819`	`0.6537`	`1.9227`	`98.27%`	`89.31%`	`75.00%`	`66.67%`	`1.4057`	`122.82s`
6	`HessianPairAdam` (`hessian_pair_adam`)	`1.2171`	`0.7292`	`2.0734`	`97.87%`	`87.33%`	`66.67%`	`58.33%`	`1.4236`	`117.82s`
7	`PolarNaturalAdam` (`polar_natural_adam`)	`1.2463`	`0.7891`	`2.2015`	`96.93%`	`86.34%`	`66.67%`	`50.00%`	`1.4294`	`108.18s`
8	`SymplecticAdam` (`symplectic_adam`)	`1.2764`	`0.8445`	`2.3268`	`97.60%`	`81.39%`	`50.00%`	`50.00%`	`1.5766`	`138.10s`

Key Geometric Findings on Python Code & Structured Tool Calling

Huygens Circular Synchronization Splay (HuygensAdam) Sweeps Code & Tool-Call Generation: By preventing butterfly rotation stages from collapsing onto repetitive indentation/delimiter tokens, HuygensAdam achieves #1 validation loss (0.5672 / 1.7632 PPL), #1 Python code token accuracy (99.07%), #1 Python AST syntax validity (91.67%), and #1 tool-call token accuracy (90.69%) & JSON schema match rate (83.33%).
Dupin Cyclide Conic Metric (CyclideAdam) Excels at Syntactic Delimiter Balance: CyclideAdam achieves 98.93% Python code token accuracy and 83.33% Python AST syntax validity (#2 overall on AST validity).

Quickstart (`transformers`)

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "MatterWaveAI/grt-30m-code"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.float32)

prompt = (
    "<|im_start|>system\n"
    "You are GRT-30M-Code, a Python algorithmic coding and structured JSON tool-calling specialist "
    "built by MatterWave AI on the 100% rotational Givens Rotational Transformer architecture.<|im_end|>\n"
    "<|im_start|>user\n"
    "Write a Python function `gcd(a: int, b: int) -> int` using Euclid's iterative algorithm.<|im_end|>\n"
    "<|im_start|>assistant\n"
)
inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=False)
out_ids = model.generate(**inputs, max_new_tokens=48, do_sample=False)
print(tokenizer.decode(out_ids[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))

License

Apache 2.0

GRT-30M-Code (MatterWaveAI/grt-30m-code)