Hugging Face's logo

MatterWaveAI
/
grt-30m-code

Text Generation
Transformers
Safetensors
grt
code
python
tool-calling
function-calling
rotational-transformer
givens-rotations
clifford-algebra
matterwave
Model card Files
xet
 Community
davidhouseholter commited on
Commit
6f6a759
·
verified ·
1 Parent(s): 54f19c6

Release MatterWaveAI/grt-30m-code (zero nn.Linear, 8-way classical geometric optimizer ablation)

Browse files
Files changed (11) hide show
  1. .gitattributes +2 -0
  2. GRT_Whitepaper.pdf +3 -0
  3. README.md +85 -0
  4. SHA256SUMS +9 -0
  5. config.json +43 -0
  6. eval_results.json +147 -0
  7. generation_config.json +10 -0
  8. model.safetensors +3 -0
  9. optimizer_ablation_results.json +140 -0
  10. tokenizer.json +3 -0
  11. tokenizer_config.json +30 -0
.gitattributes CHANGED
@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ GRT_Whitepaper.pdf filter=lfs diff=lfs merge=lfs -text
37
+ tokenizer.json filter=lfs diff=lfs merge=lfs -text
GRT_Whitepaper.pdf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:79219257df919cc598a75f7ae0968988ed0d33ffc1777469f9a9da39a558819d
3
+ size 1034984
README.md ADDED
@@ -0,0 +1,85 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ library_name: transformers
4
+ pipeline_tag: text-generation
5
+ tags:
6
+ - grt
7
+ - code
8
+ - python
9
+ - tool-calling
10
+ - function-calling
11
+ - rotational-transformer
12
+ - givens-rotations
13
+ - clifford-algebra
14
+ - matterwave
15
+ base_model: MatterWaveAI/grt-30m
16
+ ---
17
+
18
+ # GRT-30M-Code (`MatterWaveAI/grt-30m-code`)
19
+
20
+ **GRT-30M-Code** is the **Python algorithmic code synthesis and structured JSON tool-calling specialist** fine-tuned from [`MatterWaveAI/grt-30m`](https://huggingface.co/MatterWaveAI/grt-30m) (`GRTForCausalLM`, `29,305,498` parameters, **zero `nn.Linear` layers**, **zero `nn.Conv1d` layers**).
21
+
22
+ It specializes the 12-layer Givens Rotational Transformer backbone for:
23
+ 1. **Type-Annotated Python Algorithmic Synthesis**: Generating clean, syntactically valid (`ast.parse`) Python functions across search, graph/array dynamic programming, number theory, string parsing, and linear algebra.
24
+ 2. **Structured `<tool_call>` JSON Invocation**: Emitting schema-compliant `{"name": ..., "arguments": {...}}` JSON tool invocations conditioned on developer instructions.
25
+
26
+ ## Code & Tool-Calling Alignment Metrics
27
+
28
+ - **Held-Out Code & Tool-Call Validation Perplexity**: `54.93` (`val_loss = 4.0061` base) $\rightarrow$ **`1.7632`** (`val_loss = 0.5672` w/ `HuygensAdam`; `1.8253` w/ `PairAdam`, `1.8619` w/ `GyricAdam`)
29
+ - **Held-Out Python Code Token Top-1 Accuracy**: `46.80%` $\rightarrow$ **`99.07%`** (`HuygensAdam`; `98.93%` w/ `CyclideAdam`, `98.53%` w/ `GyricAdam`)
30
+ - **Held-Out Python AST Syntax Validity (`ast.parse`)**: `0.00%` $\rightarrow$ **`91.67%`** (`HuygensAdam`; `83.33%` w/ `CyclideAdam`)
31
+ - **Held-Out Structured Tool-Call Token Top-1 Accuracy**: `36.44%` $\rightarrow$ **`90.69%`** (`HuygensAdam` & `PairAdam`; `89.50%` w/ `CyclideAdam`)
32
+ - **Held-Out JSON Tool-Call Schema Match Rate**: `0.00%` $\rightarrow$ **`83.33%`** (`HuygensAdam` & `PairAdam`; `75.00%` w/ `CyclideAdam`, `GyricAdam`, `HuygensSymplecticAdam`)
33
+ - **Code Signature/Docstring Conditioning Margin ($\Delta\mathcal{L}$)**: `0.8998` $\rightarrow$ **`1.6597` nats/tok** (`PairAdam`) / **`1.5766` nats/tok** (`SymplecticAdam`)
34
+
35
+ ---
36
+
37
+ ## 8-Way Classical Geometric Optimizer Ablation
38
+
39
+ We benchmarked all 8 classical geometric, Riemannian, conic, symplectic, gyroscopic, and synchronization optimizers from `transformers.models.grt.optimization_grt` on `MatterWaveAI/grt-30m-code` (`200` SFT steps on `blocks[11:12]` + `final_norm` + `<|im_end|>` polar phase calibration, `811,907` trainable parameters, `batch_size = 16`, `lr = 6.5e-3`, `max_angular_step = 0.12`, `seed = 42`):
40
+
41
+ | Rank | Optimizer (`optimizer_type`) | Final Train Loss | Val Loss | Val PPL | Code Token Acc (%) | Tool-Call Token Acc (%) | JSON Schema Match (%) | Python AST Valid (%) | Conditioning Margin ($\Delta\mathcal{L}$) | Wall Time (s) |
42
+ | :---: | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
43
+ | **1** | **`HuygensAdam` (`huygens_adam`)** | **`1.0882`** | **`0.5672`** | **`1.7632`** | **`99.07%`** | **`90.69%`** | **`83.33%`** | **`91.67%`** | `1.5152` | `138.63s` |
44
+ | **2** | **`PairAdam` (`pair_adam`)** | `1.0994` | `0.6018` | `1.8253` | `97.60%` | **`90.69%`** | **`83.33%`** | `50.00%` | **`1.6597`** | `278.80s` |
45
+ | **3** | **`GyricAdam` (`gyric_adam`)** | `1.1168` | `0.6216` | `1.8619` | `98.53%` | `87.92%` | `75.00%` | `50.00%` | `1.5745` | `138.83s` |
46
+ | **4** | **`CyclideAdam` (`cyclide_adam`)** | `1.1564` | `0.6400` | `1.8965` | `98.93%` | `89.50%` | `75.00%` | `83.33%` | `1.5727` | `141.92s` |
47
+ | **5** | **`HuygensSymplecticAdam` (`huygens_symplectic_adam`)** | `1.1819` | `0.6537` | `1.9227` | `98.27%` | `89.31%` | `75.00%` | `66.67%` | `1.4057` | `122.82s` |
48
+ | **6** | **`HessianPairAdam` (`hessian_pair_adam`)** | `1.2171` | `0.7292` | `2.0734` | `97.87%` | `87.33%` | `66.67%` | `58.33%` | `1.4236` | `117.82s` |
49
+ | **7** | **`PolarNaturalAdam` (`polar_natural_adam`)** | `1.2463` | `0.7891` | `2.2015` | `96.93%` | `86.34%` | `66.67%` | `50.00%` | `1.4294` | **`108.18s`** |
50
+ | **8** | **`SymplecticAdam` (`symplectic_adam`)** | `1.2764` | `0.8445` | `2.3268` | `97.60%` | `81.39%` | `50.00%` | `50.00%` | `1.5766` | `138.10s` |
51
+
52
+ ### Key Geometric Findings on Python Code & Structured Tool Calling
53
+ - **Huygens Circular Synchronization Splay (`HuygensAdam`) Sweeps Code & Tool-Call Generation**: By preventing butterfly rotation stages from collapsing onto repetitive indentation/delimiter tokens, `HuygensAdam` achieves **#1 validation loss (`0.5672` / `1.7632` PPL)**, **#1 Python code token accuracy (`99.07%`)**, **#1 Python AST syntax validity (`91.67%`)**, and **#1 tool-call token accuracy (`90.69%`) & JSON schema match rate (`83.33%`)**.
54
+ - **Dupin Cyclide Conic Metric (`CyclideAdam`) Excels at Syntactic Delimiter Balance**: `CyclideAdam` achieves **`98.93%` Python code token accuracy** and **`83.33%` Python AST syntax validity** (#2 overall on AST validity).
55
+
56
+ ---
57
+
58
+ ## Quickstart (`transformers`)
59
+
60
+ ```python
61
+ import torch
62
+ from transformers import AutoModelForCausalLM, AutoTokenizer
63
+
64
+ model_id = "MatterWaveAI/grt-30m-code"
65
+ tokenizer = AutoTokenizer.from_pretrained(model_id)
66
+ model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.float32)
67
+
68
+ prompt = (
69
+ "<|im_start|>system\n"
70
+ "You are GRT-30M-Code, a Python algorithmic coding and structured JSON tool-calling specialist "
71
+ "built by MatterWave AI on the 100% rotational Givens Rotational Transformer architecture.<|im_end|>\n"
72
+ "<|im_start|>user\n"
73
+ "Write a Python function `gcd(a: int, b: int) -> int` using Euclid's iterative algorithm.<|im_end|>\n"
74
+ "<|im_start|>assistant\n"
75
+ )
76
+ inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=False)
77
+ out_ids = model.generate(**inputs, max_new_tokens=48, do_sample=False)
78
+ print(tokenizer.decode(out_ids[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))
79
+ ```
80
+
81
+ ---
82
+
83
+ ## License
84
+
85
+ Apache 2.0
SHA256SUMS ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ 79219257df919cc598a75f7ae0968988ed0d33ffc1777469f9a9da39a558819d GRT_Whitepaper.pdf
2
+ 6ed57d1c779a1faa14c5c662213d3ea7708905f33ccae3be3369eb17a3ae54cb README.md
3
+ 0d5e59b6feb435bacfdb3ffc4f7017d7d08c7a8054c65f6fb9e69f120e2b6f7d config.json
4
+ 386e2d8e35dfd941f5642a1148751e822c12157f43413a7a694b11e7837a007b eval_results.json
5
+ 8855005a7319830582feb0ec2bde29cad5b163c3a1682ed3eb69c7d0d59e53cb generation_config.json
6
+ 11ac67b25baec1f95808d8093ecc27ba4e0b3742318ca60d971448cc46d714d1 model.safetensors
7
+ 5cbf4aa8575f35c43e5c4f3822427b233bfa5c52fd1db896389f7d3252723b1b optimizer_ablation_results.json
8
+ be75606093db2094d7cd20f3c2f385c212750648bd6ea4fb2bf507a6a4c55506 tokenizer.json
9
+ 5f007d04324afa93876e2a484190e868136c69dfe0e1a29a1063eae583df9d32 tokenizer_config.json
config.json ADDED
@@ -0,0 +1,43 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "angle_init_scale": 0.1,
3
+ "architectures": [
4
+ "GRTForCausalLM"
5
+ ],
6
+ "attention_dropout": 0.0,
7
+ "bos_token_id": 151643,
8
+ "coupling_init_mode": "diagonal",
9
+ "coupling_init_scale": 0.01,
10
+ "dtype": "float32",
11
+ "eos_token_id": 151645,
12
+ "ffn_n_experts": 0,
13
+ "ffn_n_rotations": 21,
14
+ "hidden_size": 256,
15
+ "initializer_range": 0.02,
16
+ "intermediate_size": 1024,
17
+ "learnable_residual_gates": true,
18
+ "learnable_rope": true,
19
+ "log_magnitude_init": 0.0,
20
+ "max_position_embeddings": 2048,
21
+ "model_type": "grt",
22
+ "mor_delta_scale": 0.5,
23
+ "mor_mode": "direct",
24
+ "mor_router_mode": "rotational",
25
+ "mor_split_routers": false,
26
+ "mor_top_k": null,
27
+ "n_mixer_rotations": 1,
28
+ "n_qkv_rotations": 11,
29
+ "num_attention_heads": 4,
30
+ "num_hidden_layers": 12,
31
+ "output_scale_init": 30.0,
32
+ "pad_token_id": 151643,
33
+ "peri_ln": false,
34
+ "phase_init_scale": 0.1,
35
+ "pos_freq_base": 10000.0,
36
+ "residual_gate_mode": "bounded_depth",
37
+ "residual_norm_mode": "none",
38
+ "rope_placement": "stream",
39
+ "tie_word_embeddings": true,
40
+ "transformers_version": "5.23.0.dev0",
41
+ "use_cache": true,
42
+ "vocab_size": 151669
43
+ }
eval_results.json ADDED
@@ -0,0 +1,147 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "model_id": "MatterWaveAI/grt-30m-code",
3
+ "architecture": "GRTForCausalLM",
4
+ "pipeline_tag": "text-generation",
5
+ "parameters": 29305498,
6
+ "nn_linear_layers": 0,
7
+ "nn_conv1d_layers": 0,
8
+ "best_optimizer": "huygens_adam",
9
+ "before_training": {
10
+ "val_loss": 4.0061,
11
+ "val_ppl": 54.9316,
12
+ "corrupted_prompt_val_loss": 4.9059,
13
+ "code_conditioning_margin": 0.8998,
14
+ "code_token_acc": 46.8,
15
+ "tool_call_token_acc": 36.44,
16
+ "json_schema_match_rate": 0.0,
17
+ "python_ast_syntax_validity": 0.0
18
+ },
19
+ "best_checkpoint_metrics": {
20
+ "optimizer_type": "huygens_adam",
21
+ "variant": "HuygensAdam (Circular Order Parameter R_p Splay Torque)",
22
+ "final_train_loss": 1.0882,
23
+ "val_loss": 0.5672,
24
+ "val_ppl": 1.7632,
25
+ "code_token_acc": 99.07,
26
+ "tool_call_token_acc": 90.69,
27
+ "json_schema_match_rate": 83.33,
28
+ "python_ast_syntax_validity": 91.67,
29
+ "code_conditioning_margin": 1.5152,
30
+ "corrupted_prompt_val_loss": 2.0824,
31
+ "wall_time_s": 138.63
32
+ },
33
+ "optimizer_ablation_summary": {
34
+ "pair_adam": {
35
+ "optimizer_type": "pair_adam",
36
+ "variant": "PairAdam (Factored Toroidal Baseline)",
37
+ "final_train_loss": 1.0994,
38
+ "val_loss": 0.6018,
39
+ "val_ppl": 1.8253,
40
+ "code_token_acc": 97.6,
41
+ "tool_call_token_acc": 90.69,
42
+ "json_schema_match_rate": 83.33,
43
+ "python_ast_syntax_validity": 50.0,
44
+ "code_conditioning_margin": 1.6597,
45
+ "corrupted_prompt_val_loss": 2.2614,
46
+ "wall_time_s": 278.8
47
+ },
48
+ "hessian_pair_adam": {
49
+ "optimizer_type": "hessian_pair_adam",
50
+ "variant": "HessianPairAdam (Sophia-Torus Cauchy-Riemann Curvature)",
51
+ "final_train_loss": 1.2171,
52
+ "val_loss": 0.7292,
53
+ "val_ppl": 2.0734,
54
+ "code_token_acc": 97.87,
55
+ "tool_call_token_acc": 87.33,
56
+ "json_schema_match_rate": 66.67,
57
+ "python_ast_syntax_validity": 58.33,
58
+ "code_conditioning_margin": 1.4236,
59
+ "corrupted_prompt_val_loss": 2.1528,
60
+ "wall_time_s": 117.82
61
+ },
62
+ "polar_natural_adam": {
63
+ "optimizer_type": "polar_natural_adam",
64
+ "variant": "PolarNaturalAdam (Riemannian Polar Metric + Cauchy-Riemann Hessian)",
65
+ "final_train_loss": 1.2463,
66
+ "val_loss": 0.7891,
67
+ "val_ppl": 2.2015,
68
+ "code_token_acc": 96.93,
69
+ "tool_call_token_acc": 86.34,
70
+ "json_schema_match_rate": 66.67,
71
+ "python_ast_syntax_validity": 50.0,
72
+ "code_conditioning_margin": 1.4294,
73
+ "corrupted_prompt_val_loss": 2.2185,
74
+ "wall_time_s": 108.18
75
+ },
76
+ "cyclide_adam": {
77
+ "optimizer_type": "cyclide_adam",
78
+ "variant": "CyclideAdam (Dupin Cyclide Conic Metric + Deviatoric Twist)",
79
+ "final_train_loss": 1.1564,
80
+ "val_loss": 0.64,
81
+ "val_ppl": 1.8965,
82
+ "code_token_acc": 98.93,
83
+ "tool_call_token_acc": 89.5,
84
+ "json_schema_match_rate": 75.0,
85
+ "python_ast_syntax_validity": 83.33,
86
+ "code_conditioning_margin": 1.5727,
87
+ "corrupted_prompt_val_loss": 2.2127,
88
+ "wall_time_s": 141.92
89
+ },
90
+ "symplectic_adam": {
91
+ "optimizer_type": "symplectic_adam",
92
+ "variant": "SymplecticAdam (Gromov-Liouville Non-Squeezing Floor + Casimir Cavity)",
93
+ "final_train_loss": 1.2764,
94
+ "val_loss": 0.8445,
95
+ "val_ppl": 2.3268,
96
+ "code_token_acc": 97.6,
97
+ "tool_call_token_acc": 81.39,
98
+ "json_schema_match_rate": 50.0,
99
+ "python_ast_syntax_validity": 50.0,
100
+ "code_conditioning_margin": 1.5766,
101
+ "corrupted_prompt_val_loss": 2.421,
102
+ "wall_time_s": 138.1
103
+ },
104
+ "gyric_adam": {
105
+ "optimizer_type": "gyric_adam",
106
+ "variant": "GyricAdam (Poinsot-Lande Gyroscopic Projection + Thomas Precession)",
107
+ "final_train_loss": 1.1168,
108
+ "val_loss": 0.6216,
109
+ "val_ppl": 1.8619,
110
+ "code_token_acc": 98.53,
111
+ "tool_call_token_acc": 87.92,
112
+ "json_schema_match_rate": 75.0,
113
+ "python_ast_syntax_validity": 50.0,
114
+ "code_conditioning_margin": 1.5745,
115
+ "corrupted_prompt_val_loss": 2.1961,
116
+ "wall_time_s": 138.83
117
+ },
118
+ "huygens_adam": {
119
+ "optimizer_type": "huygens_adam",
120
+ "variant": "HuygensAdam (Circular Order Parameter R_p Splay Torque)",
121
+ "final_train_loss": 1.0882,
122
+ "val_loss": 0.5672,
123
+ "val_ppl": 1.7632,
124
+ "code_token_acc": 99.07,
125
+ "tool_call_token_acc": 90.69,
126
+ "json_schema_match_rate": 83.33,
127
+ "python_ast_syntax_validity": 91.67,
128
+ "code_conditioning_margin": 1.5152,
129
+ "corrupted_prompt_val_loss": 2.0824,
130
+ "wall_time_s": 138.63
131
+ },
132
+ "huygens_symplectic_adam": {
133
+ "optimizer_type": "huygens_symplectic_adam",
134
+ "variant": "HuygensSymplecticAdam (Two-Stage Huygens -> Symplectic Curriculum)",
135
+ "final_train_loss": 1.1819,
136
+ "val_loss": 0.6537,
137
+ "val_ppl": 1.9227,
138
+ "code_token_acc": 98.27,
139
+ "tool_call_token_acc": 89.31,
140
+ "json_schema_match_rate": 75.0,
141
+ "python_ast_syntax_validity": 66.67,
142
+ "code_conditioning_margin": 1.4057,
143
+ "corrupted_prompt_val_loss": 2.0594,
144
+ "wall_time_s": 122.82
145
+ }
146
+ }
147
+ }
generation_config.json ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_from_model_config": true,
3
+ "bos_token_id": 151643,
4
+ "eos_token_id": 151643,
5
+ "output_attentions": false,
6
+ "output_hidden_states": false,
7
+ "pad_token_id": 151643,
8
+ "transformers_version": "5.23.0.dev0",
9
+ "use_cache": true
10
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:11ac67b25baec1f95808d8093ecc27ba4e0b3742318ca60d971448cc46d714d1
3
+ size 121573808
optimizer_ablation_results.json ADDED
@@ -0,0 +1,140 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "model_id": "MatterWaveAI/grt-30m-code",
3
+ "architecture": "GRTForCausalLM",
4
+ "pipeline_tag": "text-generation",
5
+ "base_model": "MatterWaveAI/grt-30m",
6
+ "parameters": 29305498,
7
+ "trainable_parameters": 811907,
8
+ "trainable_scope": "blocks[11:12] + final_norm + <|im_end|> phase calibration",
9
+ "nn_linear_layers": 0,
10
+ "nn_conv1d_layers": 0,
11
+ "steps": 200,
12
+ "batch_size": 16,
13
+ "learning_rate": 0.0035,
14
+ "seed": 42,
15
+ "best_optimizer": "huygens_adam",
16
+ "before_training": {
17
+ "val_loss": 4.0061,
18
+ "val_ppl": 54.9316,
19
+ "corrupted_prompt_val_loss": 4.9059,
20
+ "code_conditioning_margin": 0.8998,
21
+ "code_token_acc": 46.8,
22
+ "tool_call_token_acc": 36.44,
23
+ "json_schema_match_rate": 0.0,
24
+ "python_ast_syntax_validity": 0.0
25
+ },
26
+ "optimizers": {
27
+ "pair_adam": {
28
+ "optimizer_type": "pair_adam",
29
+ "variant": "PairAdam (Factored Toroidal Baseline)",
30
+ "final_train_loss": 1.0994,
31
+ "val_loss": 0.6018,
32
+ "val_ppl": 1.8253,
33
+ "code_token_acc": 97.6,
34
+ "tool_call_token_acc": 90.69,
35
+ "json_schema_match_rate": 83.33,
36
+ "python_ast_syntax_validity": 50.0,
37
+ "code_conditioning_margin": 1.6597,
38
+ "corrupted_prompt_val_loss": 2.2614,
39
+ "wall_time_s": 278.8
40
+ },
41
+ "hessian_pair_adam": {
42
+ "optimizer_type": "hessian_pair_adam",
43
+ "variant": "HessianPairAdam (Sophia-Torus Cauchy-Riemann Curvature)",
44
+ "final_train_loss": 1.2171,
45
+ "val_loss": 0.7292,
46
+ "val_ppl": 2.0734,
47
+ "code_token_acc": 97.87,
48
+ "tool_call_token_acc": 87.33,
49
+ "json_schema_match_rate": 66.67,
50
+ "python_ast_syntax_validity": 58.33,
51
+ "code_conditioning_margin": 1.4236,
52
+ "corrupted_prompt_val_loss": 2.1528,
53
+ "wall_time_s": 117.82
54
+ },
55
+ "polar_natural_adam": {
56
+ "optimizer_type": "polar_natural_adam",
57
+ "variant": "PolarNaturalAdam (Riemannian Polar Metric + Cauchy-Riemann Hessian)",
58
+ "final_train_loss": 1.2463,
59
+ "val_loss": 0.7891,
60
+ "val_ppl": 2.2015,
61
+ "code_token_acc": 96.93,
62
+ "tool_call_token_acc": 86.34,
63
+ "json_schema_match_rate": 66.67,
64
+ "python_ast_syntax_validity": 50.0,
65
+ "code_conditioning_margin": 1.4294,
66
+ "corrupted_prompt_val_loss": 2.2185,
67
+ "wall_time_s": 108.18
68
+ },
69
+ "cyclide_adam": {
70
+ "optimizer_type": "cyclide_adam",
71
+ "variant": "CyclideAdam (Dupin Cyclide Conic Metric + Deviatoric Twist)",
72
+ "final_train_loss": 1.1564,
73
+ "val_loss": 0.64,
74
+ "val_ppl": 1.8965,
75
+ "code_token_acc": 98.93,
76
+ "tool_call_token_acc": 89.5,
77
+ "json_schema_match_rate": 75.0,
78
+ "python_ast_syntax_validity": 83.33,
79
+ "code_conditioning_margin": 1.5727,
80
+ "corrupted_prompt_val_loss": 2.2127,
81
+ "wall_time_s": 141.92
82
+ },
83
+ "symplectic_adam": {
84
+ "optimizer_type": "symplectic_adam",
85
+ "variant": "SymplecticAdam (Gromov-Liouville Non-Squeezing Floor + Casimir Cavity)",
86
+ "final_train_loss": 1.2764,
87
+ "val_loss": 0.8445,
88
+ "val_ppl": 2.3268,
89
+ "code_token_acc": 97.6,
90
+ "tool_call_token_acc": 81.39,
91
+ "json_schema_match_rate": 50.0,
92
+ "python_ast_syntax_validity": 50.0,
93
+ "code_conditioning_margin": 1.5766,
94
+ "corrupted_prompt_val_loss": 2.421,
95
+ "wall_time_s": 138.1
96
+ },
97
+ "gyric_adam": {
98
+ "optimizer_type": "gyric_adam",
99
+ "variant": "GyricAdam (Poinsot-Lande Gyroscopic Projection + Thomas Precession)",
100
+ "final_train_loss": 1.1168,
101
+ "val_loss": 0.6216,
102
+ "val_ppl": 1.8619,
103
+ "code_token_acc": 98.53,
104
+ "tool_call_token_acc": 87.92,
105
+ "json_schema_match_rate": 75.0,
106
+ "python_ast_syntax_validity": 50.0,
107
+ "code_conditioning_margin": 1.5745,
108
+ "corrupted_prompt_val_loss": 2.1961,
109
+ "wall_time_s": 138.83
110
+ },
111
+ "huygens_adam": {
112
+ "optimizer_type": "huygens_adam",
113
+ "variant": "HuygensAdam (Circular Order Parameter R_p Splay Torque)",
114
+ "final_train_loss": 1.0882,
115
+ "val_loss": 0.5672,
116
+ "val_ppl": 1.7632,
117
+ "code_token_acc": 99.07,
118
+ "tool_call_token_acc": 90.69,
119
+ "json_schema_match_rate": 83.33,
120
+ "python_ast_syntax_validity": 91.67,
121
+ "code_conditioning_margin": 1.5152,
122
+ "corrupted_prompt_val_loss": 2.0824,
123
+ "wall_time_s": 138.63
124
+ },
125
+ "huygens_symplectic_adam": {
126
+ "optimizer_type": "huygens_symplectic_adam",
127
+ "variant": "HuygensSymplecticAdam (Two-Stage Huygens -> Symplectic Curriculum)",
128
+ "final_train_loss": 1.1819,
129
+ "val_loss": 0.6537,
130
+ "val_ppl": 1.9227,
131
+ "code_token_acc": 98.27,
132
+ "tool_call_token_acc": 89.31,
133
+ "json_schema_match_rate": 75.0,
134
+ "python_ast_syntax_validity": 66.67,
135
+ "code_conditioning_margin": 1.4057,
136
+ "corrupted_prompt_val_loss": 2.0594,
137
+ "wall_time_s": 122.82
138
+ }
139
+ }
140
+ }
tokenizer.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:be75606093db2094d7cd20f3c2f385c212750648bd6ea4fb2bf507a6a4c55506
3
+ size 11422650
tokenizer_config.json ADDED
@@ -0,0 +1,30 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_prefix_space": false,
3
+ "backend": "tokenizers",
4
+ "bos_token": null,
5
+ "clean_up_tokenization_spaces": false,
6
+ "eos_token": "<|endoftext|>",
7
+ "errors": "replace",
8
+ "extra_special_tokens": [
9
+ "<|im_start|>",
10
+ "<|im_end|>",
11
+ "<|object_ref_start|>",
12
+ "<|object_ref_end|>",
13
+ "<|box_start|>",
14
+ "<|box_end|>",
15
+ "<|quad_start|>",
16
+ "<|quad_end|>",
17
+ "<|vision_start|>",
18
+ "<|vision_end|>",
19
+ "<|vision_pad|>",
20
+ "<|image_pad|>",
21
+ "<|video_pad|>"
22
+ ],
23
+ "is_local": true,
24
+ "local_files_only": false,
25
+ "model_max_length": 131072,
26
+ "pad_token": "<|endoftext|>",
27
+ "split_special_tokens": false,
28
+ "tokenizer_class": "Qwen2Tokenizer",
29
+ "unk_token": null
30
+ }