codeShare commited on
Commit
c6e7a83
·
verified ·
1 Parent(s): b296de3

Upload Gemma 4 E2B LoRA adapter

Browse files
.gitattributes CHANGED
@@ -34,3 +34,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
  gemma4_preencoded_dataset/processor/tokenizer.json filter=lfs diff=lfs merge=lfs -text
 
 
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
  gemma4_preencoded_dataset/processor/tokenizer.json filter=lfs diff=lfs merge=lfs -text
37
+ gemma4_lora/processor/tokenizer.json filter=lfs diff=lfs merge=lfs -text
gemma4_lora/README.md ADDED
@@ -0,0 +1,207 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: coder3101/gemma-4-E2B-it-heretic
3
+ library_name: peft
4
+ pipeline_tag: text-generation
5
+ tags:
6
+ - base_model:adapter:coder3101/gemma-4-E2B-it-heretic
7
+ - lora
8
+ - transformers
9
+ ---
10
+
11
+ # Model Card for Model ID
12
+
13
+ <!-- Provide a quick summary of what the model is/does. -->
14
+
15
+
16
+
17
+ ## Model Details
18
+
19
+ ### Model Description
20
+
21
+ <!-- Provide a longer summary of what this model is. -->
22
+
23
+
24
+
25
+ - **Developed by:** [More Information Needed]
26
+ - **Funded by [optional]:** [More Information Needed]
27
+ - **Shared by [optional]:** [More Information Needed]
28
+ - **Model type:** [More Information Needed]
29
+ - **Language(s) (NLP):** [More Information Needed]
30
+ - **License:** [More Information Needed]
31
+ - **Finetuned from model [optional]:** [More Information Needed]
32
+
33
+ ### Model Sources [optional]
34
+
35
+ <!-- Provide the basic links for the model. -->
36
+
37
+ - **Repository:** [More Information Needed]
38
+ - **Paper [optional]:** [More Information Needed]
39
+ - **Demo [optional]:** [More Information Needed]
40
+
41
+ ## Uses
42
+
43
+ <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
44
+
45
+ ### Direct Use
46
+
47
+ <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
48
+
49
+ [More Information Needed]
50
+
51
+ ### Downstream Use [optional]
52
+
53
+ <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
54
+
55
+ [More Information Needed]
56
+
57
+ ### Out-of-Scope Use
58
+
59
+ <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
60
+
61
+ [More Information Needed]
62
+
63
+ ## Bias, Risks, and Limitations
64
+
65
+ <!-- This section is meant to convey both technical and sociotechnical limitations. -->
66
+
67
+ [More Information Needed]
68
+
69
+ ### Recommendations
70
+
71
+ <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
72
+
73
+ Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
74
+
75
+ ## How to Get Started with the Model
76
+
77
+ Use the code below to get started with the model.
78
+
79
+ [More Information Needed]
80
+
81
+ ## Training Details
82
+
83
+ ### Training Data
84
+
85
+ <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
86
+
87
+ [More Information Needed]
88
+
89
+ ### Training Procedure
90
+
91
+ <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
92
+
93
+ #### Preprocessing [optional]
94
+
95
+ [More Information Needed]
96
+
97
+
98
+ #### Training Hyperparameters
99
+
100
+ - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
101
+
102
+ #### Speeds, Sizes, Times [optional]
103
+
104
+ <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
105
+
106
+ [More Information Needed]
107
+
108
+ ## Evaluation
109
+
110
+ <!-- This section describes the evaluation protocols and provides the results. -->
111
+
112
+ ### Testing Data, Factors & Metrics
113
+
114
+ #### Testing Data
115
+
116
+ <!-- This should link to a Dataset Card if possible. -->
117
+
118
+ [More Information Needed]
119
+
120
+ #### Factors
121
+
122
+ <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
123
+
124
+ [More Information Needed]
125
+
126
+ #### Metrics
127
+
128
+ <!-- These are the evaluation metrics being used, ideally with a description of why. -->
129
+
130
+ [More Information Needed]
131
+
132
+ ### Results
133
+
134
+ [More Information Needed]
135
+
136
+ #### Summary
137
+
138
+
139
+
140
+ ## Model Examination [optional]
141
+
142
+ <!-- Relevant interpretability work for the model goes here -->
143
+
144
+ [More Information Needed]
145
+
146
+ ## Environmental Impact
147
+
148
+ <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
149
+
150
+ Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
151
+
152
+ - **Hardware Type:** [More Information Needed]
153
+ - **Hours used:** [More Information Needed]
154
+ - **Cloud Provider:** [More Information Needed]
155
+ - **Compute Region:** [More Information Needed]
156
+ - **Carbon Emitted:** [More Information Needed]
157
+
158
+ ## Technical Specifications [optional]
159
+
160
+ ### Model Architecture and Objective
161
+
162
+ [More Information Needed]
163
+
164
+ ### Compute Infrastructure
165
+
166
+ [More Information Needed]
167
+
168
+ #### Hardware
169
+
170
+ [More Information Needed]
171
+
172
+ #### Software
173
+
174
+ [More Information Needed]
175
+
176
+ ## Citation [optional]
177
+
178
+ <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
179
+
180
+ **BibTeX:**
181
+
182
+ [More Information Needed]
183
+
184
+ **APA:**
185
+
186
+ [More Information Needed]
187
+
188
+ ## Glossary [optional]
189
+
190
+ <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
191
+
192
+ [More Information Needed]
193
+
194
+ ## More Information [optional]
195
+
196
+ [More Information Needed]
197
+
198
+ ## Model Card Authors [optional]
199
+
200
+ [More Information Needed]
201
+
202
+ ## Model Card Contact
203
+
204
+ [More Information Needed]
205
+ ### Framework versions
206
+
207
+ - PEFT 0.21.2
gemma4_lora/adapter_config.json ADDED
@@ -0,0 +1,258 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alora_invocation_tokens": null,
3
+ "alpha_pattern": {},
4
+ "arrow_config": null,
5
+ "auto_mapping": null,
6
+ "base_model_name_or_path": "coder3101/gemma-4-E2B-it-heretic",
7
+ "bias": "none",
8
+ "corda_config": null,
9
+ "ensure_weight_tying": false,
10
+ "eva_config": null,
11
+ "exclude_modules": null,
12
+ "fan_in_fan_out": false,
13
+ "inference_mode": true,
14
+ "init_lora_weights": true,
15
+ "kasa_config": null,
16
+ "layer_replication": null,
17
+ "layers_pattern": null,
18
+ "layers_to_transform": null,
19
+ "loftq_config": {},
20
+ "lora_alpha": 16,
21
+ "lora_bias": false,
22
+ "lora_dropout": 0.05,
23
+ "lora_ga_config": null,
24
+ "megatron_config": null,
25
+ "megatron_core": "megatron.core",
26
+ "modules_to_save": null,
27
+ "monteclora_config": null,
28
+ "peft_type": "LORA",
29
+ "peft_version": "0.21.2",
30
+ "qalora_group_size": 16,
31
+ "r": 16,
32
+ "rank_pattern": {},
33
+ "revision": null,
34
+ "target_modules": [
35
+ "linear",
36
+ "language_model.layers.3.self_attn.k_proj",
37
+ "26.mlp.down_proj",
38
+ "language_model.layers.15.self_attn.q_proj",
39
+ "18.mlp.down_proj",
40
+ "18.mlp.up_proj",
41
+ "language_model.layers.7.self_attn.o_proj",
42
+ "language_model.layers.0.mlp.up_proj",
43
+ "language_model.layers.15.mlp.up_proj",
44
+ "language_model.layers.7.self_attn.q_proj",
45
+ "language_model.layers.1.mlp.gate_proj",
46
+ "language_model.layers.9.mlp.gate_proj",
47
+ "language_model.layers.8.mlp.down_proj",
48
+ "29.self_attn.q_proj",
49
+ "language_model.layers.5.self_attn.o_proj",
50
+ "language_model.layers.3.self_attn.q_proj",
51
+ "language_model.layers.5.mlp.gate_proj",
52
+ "25.self_attn.o_proj",
53
+ "output_proj",
54
+ "26.self_attn.q_proj",
55
+ "16.self_attn.q_proj",
56
+ "language_model.layers.6.self_attn.k_proj",
57
+ "33.mlp.down_proj",
58
+ "language_model.layers.5.mlp.down_proj",
59
+ "language_model.layers.0.self_attn.o_proj",
60
+ "language_model.layers.14.self_attn.o_proj",
61
+ "language_model.layers.1.self_attn.o_proj",
62
+ "30.mlp.up_proj",
63
+ "21.mlp.up_proj",
64
+ "25.mlp.gate_proj",
65
+ "34.mlp.down_proj",
66
+ "31.mlp.down_proj",
67
+ "language_model.layers.8.mlp.up_proj",
68
+ "21.self_attn.o_proj",
69
+ "32.mlp.up_proj",
70
+ "language_model.layers.4.self_attn.q_proj",
71
+ "language_model.layers.13.mlp.gate_proj",
72
+ "32.mlp.gate_proj",
73
+ "18.self_attn.q_proj",
74
+ "language_model.layers.7.self_attn.v_proj",
75
+ "language_model.layers.0.self_attn.q_proj",
76
+ "23.mlp.gate_proj",
77
+ "language_model.layers.10.self_attn.q_proj",
78
+ "language_model.layers.6.mlp.gate_proj",
79
+ "language_model.layers.10.self_attn.v_proj",
80
+ "per_layer_projection",
81
+ "language_model.layers.0.mlp.down_proj",
82
+ "19.mlp.down_proj",
83
+ "language_model.layers.6.mlp.down_proj",
84
+ "20.mlp.gate_proj",
85
+ "language_model.layers.12.self_attn.q_proj",
86
+ "22.mlp.up_proj",
87
+ "16.self_attn.o_proj",
88
+ "24.self_attn.o_proj",
89
+ "language_model.layers.13.mlp.down_proj",
90
+ "language_model.layers.12.mlp.gate_proj",
91
+ "28.self_attn.o_proj",
92
+ "language_model.layers.0.self_attn.k_proj",
93
+ "language_model.layers.4.self_attn.o_proj",
94
+ "27.mlp.up_proj",
95
+ "32.self_attn.o_proj",
96
+ "20.self_attn.q_proj",
97
+ "language_model.layers.11.mlp.up_proj",
98
+ "language_model.layers.11.self_attn.v_proj",
99
+ "23.mlp.up_proj",
100
+ "language_model.layers.9.self_attn.v_proj",
101
+ "input_proj",
102
+ "19.self_attn.o_proj",
103
+ "language_model.layers.13.self_attn.q_proj",
104
+ "19.mlp.up_proj",
105
+ "language_model.layers.6.self_attn.q_proj",
106
+ "language_model.layers.8.mlp.gate_proj",
107
+ "33.self_attn.o_proj",
108
+ "27.mlp.down_proj",
109
+ "language_model.layers.11.self_attn.k_proj",
110
+ "language_model.layers.10.self_attn.k_proj",
111
+ "language_model.layers.9.self_attn.q_proj",
112
+ "language_model.layers.1.self_attn.k_proj",
113
+ "21.mlp.gate_proj",
114
+ "language_model.layers.11.self_attn.o_proj",
115
+ "language_model.layers.12.self_attn.v_proj",
116
+ "20.self_attn.o_proj",
117
+ "language_model.layers.8.self_attn.q_proj",
118
+ "language_model.layers.6.self_attn.o_proj",
119
+ "language_model.layers.5.self_attn.v_proj",
120
+ "30.self_attn.q_proj",
121
+ "language_model.layers.13.self_attn.k_proj",
122
+ "31.self_attn.o_proj",
123
+ "input_proj_linear",
124
+ "language_model.layers.9.self_attn.o_proj",
125
+ "language_model.layers.13.self_attn.o_proj",
126
+ "26.mlp.up_proj",
127
+ "22.mlp.down_proj",
128
+ "language_model.layers.2.mlp.gate_proj",
129
+ "language_model.layers.15.self_attn.o_proj",
130
+ "language_model.layers.10.mlp.gate_proj",
131
+ "language_model.layers.2.self_attn.k_proj",
132
+ "16.mlp.down_proj",
133
+ "30.mlp.gate_proj",
134
+ "language_model.layers.12.mlp.up_proj",
135
+ "relative_k_proj",
136
+ "34.self_attn.q_proj",
137
+ "21.mlp.down_proj",
138
+ "language_model.layers.4.mlp.up_proj",
139
+ "language_model.layers.12.self_attn.o_proj",
140
+ "29.mlp.down_proj",
141
+ "29.mlp.up_proj",
142
+ "29.self_attn.o_proj",
143
+ "language_model.layers.7.self_attn.k_proj",
144
+ "language_model.layers.2.self_attn.o_proj",
145
+ "language_model.layers.1.mlp.up_proj",
146
+ "language_model.layers.9.mlp.down_proj",
147
+ "26.mlp.gate_proj",
148
+ "per_layer_model_projection",
149
+ "language_model.layers.2.self_attn.v_proj",
150
+ "language_model.layers.12.self_attn.k_proj",
151
+ "language_model.layers.5.mlp.up_proj",
152
+ "language_model.layers.11.self_attn.q_proj",
153
+ "language_model.layers.1.self_attn.v_proj",
154
+ "language_model.layers.11.mlp.gate_proj",
155
+ "language_model.layers.5.self_attn.q_proj",
156
+ "16.mlp.up_proj",
157
+ "language_model.layers.14.mlp.up_proj",
158
+ "language_model.layers.4.mlp.gate_proj",
159
+ "language_model.layers.11.mlp.down_proj",
160
+ "language_model.layers.4.self_attn.v_proj",
161
+ "24.mlp.down_proj",
162
+ "language_model.layers.15.mlp.gate_proj",
163
+ "33.mlp.up_proj",
164
+ "27.self_attn.q_proj",
165
+ "language_model.layers.10.mlp.up_proj",
166
+ "19.mlp.gate_proj",
167
+ "language_model.layers.14.self_attn.q_proj",
168
+ "language_model.layers.15.mlp.down_proj",
169
+ "language_model.layers.7.mlp.down_proj",
170
+ "18.mlp.gate_proj",
171
+ "25.mlp.down_proj",
172
+ "31.mlp.gate_proj",
173
+ "language_model.layers.7.mlp.gate_proj",
174
+ "language_model.layers.2.mlp.up_proj",
175
+ "34.self_attn.o_proj",
176
+ "language_model.layers.3.mlp.up_proj",
177
+ "language_model.layers.3.mlp.down_proj",
178
+ "language_model.layers.10.mlp.down_proj",
179
+ "33.self_attn.q_proj",
180
+ "language_model.layers.7.mlp.up_proj",
181
+ "23.mlp.down_proj",
182
+ "language_model.layers.9.self_attn.k_proj",
183
+ "language_model.layers.12.mlp.down_proj",
184
+ "22.self_attn.q_proj",
185
+ "28.mlp.gate_proj",
186
+ "30.mlp.down_proj",
187
+ "language_model.layers.14.self_attn.v_proj",
188
+ "23.self_attn.q_proj",
189
+ "language_model.layers.2.self_attn.q_proj",
190
+ "language_model.layers.14.mlp.down_proj",
191
+ "language_model.layers.13.mlp.up_proj",
192
+ "24.mlp.up_proj",
193
+ "language_model.layers.14.self_attn.k_proj",
194
+ "31.mlp.up_proj",
195
+ "17.mlp.up_proj",
196
+ "27.mlp.gate_proj",
197
+ "language_model.layers.5.self_attn.k_proj",
198
+ "28.mlp.down_proj",
199
+ "18.self_attn.o_proj",
200
+ "22.self_attn.o_proj",
201
+ "34.mlp.gate_proj",
202
+ "32.mlp.down_proj",
203
+ "language_model.layers.1.mlp.down_proj",
204
+ "language_model.layers.1.self_attn.q_proj",
205
+ "25.self_attn.q_proj",
206
+ "language_model.layers.3.self_attn.o_proj",
207
+ "22.mlp.gate_proj",
208
+ "29.mlp.gate_proj",
209
+ "28.mlp.up_proj",
210
+ "23.self_attn.o_proj",
211
+ "language_model.layers.6.self_attn.v_proj",
212
+ "language_model.layers.3.self_attn.v_proj",
213
+ "language_model.layers.4.mlp.down_proj",
214
+ "language_model.layers.8.self_attn.o_proj",
215
+ "language_model.layers.0.self_attn.v_proj",
216
+ "19.self_attn.q_proj",
217
+ "24.mlp.gate_proj",
218
+ "31.self_attn.q_proj",
219
+ "20.mlp.down_proj",
220
+ "language_model.layers.13.self_attn.v_proj",
221
+ "32.self_attn.q_proj",
222
+ "20.mlp.up_proj",
223
+ "17.self_attn.q_proj",
224
+ "34.mlp.up_proj",
225
+ "language_model.layers.14.mlp.gate_proj",
226
+ "28.self_attn.q_proj",
227
+ "17.mlp.gate_proj",
228
+ "26.self_attn.o_proj",
229
+ "27.self_attn.o_proj",
230
+ "16.mlp.gate_proj",
231
+ "17.self_attn.o_proj",
232
+ "language_model.layers.9.mlp.up_proj",
233
+ "24.self_attn.q_proj",
234
+ "language_model.layers.4.self_attn.k_proj",
235
+ "embedding_projection",
236
+ "language_model.layers.10.self_attn.o_proj",
237
+ "per_layer_input_gate",
238
+ "33.mlp.gate_proj",
239
+ "language_model.layers.6.mlp.up_proj",
240
+ "language_model.layers.2.mlp.down_proj",
241
+ "language_model.layers.8.self_attn.v_proj",
242
+ "17.mlp.down_proj",
243
+ "30.self_attn.o_proj",
244
+ "language_model.layers.0.mlp.gate_proj",
245
+ "language_model.layers.8.self_attn.k_proj",
246
+ "25.mlp.up_proj",
247
+ "language_model.layers.3.mlp.gate_proj",
248
+ "21.self_attn.q_proj"
249
+ ],
250
+ "target_parameters": null,
251
+ "task_type": "CAUSAL_LM",
252
+ "trainable_token_indices": null,
253
+ "use_bdlora": null,
254
+ "use_dora": false,
255
+ "use_qalora": false,
256
+ "use_rslora": false,
257
+ "velora_config": null
258
+ }
gemma4_lora/adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7346110d4763bdd4a12cd709a24bf8399f93fa577d87638e00f6f0f8635dd718
3
+ size 151844848
gemma4_lora/gradient_health_audit.json ADDED
@@ -0,0 +1,143 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "audit_version": 1,
3
+ "audit_timestamp_unix": 1793918763.25352,
4
+ "base_model": "coder3101/gemma-4-E2B-it-heretic",
5
+ "adapter_directory": "/kaggle/working/gemma4_lora/adapter",
6
+ "gradient_health_passed": true,
7
+ "numerical_recovery_passed": true,
8
+ "adapter_weights_finite": true,
9
+ "adapter_export_eligible": true,
10
+ "gradient_checks": 120,
11
+ "skipped_optimizer_steps": 0,
12
+ "skipped_step_indices": [],
13
+ "nonfinite_gradient_steps": [],
14
+ "nonfinite_gradient_tensors": 0,
15
+ "gradient_norm_statistics": {
16
+ "count": 120,
17
+ "minimum": 0.01549019716854724,
18
+ "median": 0.2999997905589369,
19
+ "maximum": 0.2999999967057757,
20
+ "mean": 0.23312759692201965
21
+ },
22
+ "grad_scaler_initial": 2048.0,
23
+ "grad_scaler_final": 2048.0,
24
+ "grad_scaler_minimum": 512.0,
25
+ "loss_history_entries": 121,
26
+ "adapter_tensor_count": 1050,
27
+ "adapter_parameter_count": 37920768,
28
+ "nonfinite_adapter_tensors": [],
29
+ "passed_checks": 22,
30
+ "failed_checks": 0,
31
+ "checks": [
32
+ {
33
+ "name": "Gradient monitoring executed",
34
+ "passed": true,
35
+ "detail": "120 optimizer attempts were monitored."
36
+ },
37
+ {
38
+ "name": "Gradient norm history length",
39
+ "passed": true,
40
+ "detail": "120 norms for 120 checks."
41
+ },
42
+ {
43
+ "name": "Gradient tensor-count history length",
44
+ "passed": true,
45
+ "detail": "120 entries for 120 checks."
46
+ },
47
+ {
48
+ "name": "GradScaler history length",
49
+ "passed": true,
50
+ "detail": "120 entries for 120 checks."
51
+ },
52
+ {
53
+ "name": "Skipped optimizer update limit",
54
+ "passed": true,
55
+ "detail": "0 skipped; maximum allowed is 2."
56
+ },
57
+ {
58
+ "name": "Skipped update indices are unique",
59
+ "passed": true,
60
+ "detail": "Indices: []"
61
+ },
62
+ {
63
+ "name": "Skipped update indices are in range",
64
+ "passed": true,
65
+ "detail": "Valid range: 0 to 119."
66
+ },
67
+ {
68
+ "name": "Non-finite gradient indices are unique",
69
+ "passed": true,
70
+ "detail": "Indices: []"
71
+ },
72
+ {
73
+ "name": "Non-finite gradient indices are in range",
74
+ "passed": true,
75
+ "detail": "Valid range: 0 to 119."
76
+ },
77
+ {
78
+ "name": "Overflow attempts match skipped optimizer attempts",
79
+ "passed": true,
80
+ "detail": "Skipped=[]; non-finite=[]."
81
+ },
82
+ {
83
+ "name": "Non-finite tensor count is consistent",
84
+ "passed": true,
85
+ "detail": "0 non-finite gradient tensors."
86
+ },
87
+ {
88
+ "name": "No optimizer attempt lacked all gradients",
89
+ "passed": true,
90
+ "detail": "No-gradient attempts: []."
91
+ },
92
+ {
93
+ "name": "No consecutive overflow attempts",
94
+ "passed": true,
95
+ "detail": "Overflow indices: []."
96
+ },
97
+ {
98
+ "name": "Final overflow has recovery window",
99
+ "passed": true,
100
+ "detail": "No overflow in the final two attempts."
101
+ },
102
+ {
103
+ "name": "Missing gradient norms match overflow attempts",
104
+ "passed": true,
105
+ "detail": "Missing norms=[]; overflow attempts=[]."
106
+ },
107
+ {
108
+ "name": "All recorded finite gradient norms are valid",
109
+ "passed": true,
110
+ "detail": "120 finite norms; invalid indices=[]."
111
+ },
112
+ {
113
+ "name": "GradScaler history is finite and positive",
114
+ "passed": true,
115
+ "detail": "120 recorded values."
116
+ },
117
+ {
118
+ "name": "Initial GradScaler value is valid",
119
+ "passed": true,
120
+ "detail": "Initial scale: 2048.0."
121
+ },
122
+ {
123
+ "name": "Final GradScaler value is valid",
124
+ "passed": true,
125
+ "detail": "Final scale: 2048.0."
126
+ },
127
+ {
128
+ "name": "Final GradScaler meets minimum",
129
+ "passed": true,
130
+ "detail": "Final=2048.0; minimum=512.0."
131
+ },
132
+ {
133
+ "name": "Recorded loss history contains valid values",
134
+ "passed": true,
135
+ "detail": "120 finite loss values; invalid entries=[]."
136
+ },
137
+ {
138
+ "name": "Adapter weights contain only finite values",
139
+ "passed": true,
140
+ "detail": "1050 tensors; 37,920,768 parameters; 0 non-finite tensors."
141
+ }
142
+ ]
143
+ }
gemma4_lora/gradient_health_report.json ADDED
@@ -0,0 +1,1223 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "gradient_checks": 120,
3
+ "skipped_optimizer_steps": 0,
4
+ "skipped_step_indices": [],
5
+ "nonfinite_gradient_steps": [],
6
+ "nonfinite_gradient_tensors": 0,
7
+ "no_gradient_steps": [],
8
+ "gradient_norms": [
9
+ 0.2999999967057757,
10
+ 0.29999993283753906,
11
+ 0.2999999882627466,
12
+ 0.29999995549701025,
13
+ 0.2999999056871044,
14
+ 0.29999997674311024,
15
+ 0.29999996019639635,
16
+ 0.2999999331868226,
17
+ 0.2999999190714327,
18
+ 0.29999994453961754,
19
+ 0.29999992767652045,
20
+ 0.2999999171052898,
21
+ 0.299999891168433,
22
+ 0.2999999387562522,
23
+ 0.2999998559474201,
24
+ 0.2999998651608199,
25
+ 0.29999989888361955,
26
+ 0.299999860837114,
27
+ 0.2999998637005364,
28
+ 0.2999999487365302,
29
+ 0.29999991210927746,
30
+ 0.29999983302762295,
31
+ 0.29999986467178447,
32
+ 0.2999999246373012,
33
+ 0.2999998710247092,
34
+ 0.29999993885001613,
35
+ 0.2999998562013249,
36
+ 0.2999997898603423,
37
+ 0.2999998988001732,
38
+ 0.2999997968720858,
39
+ 0.29999981960907923,
40
+ 0.29999979953123646,
41
+ 0.29999993062910396,
42
+ 0.2999998143369396,
43
+ 0.29999988013220563,
44
+ 0.29999991627050243,
45
+ 0.2999998549485208,
46
+ 0.29999983698781824,
47
+ 0.29999988739171235,
48
+ 0.29999984926746825,
49
+ 0.29999978238735703,
50
+ 0.29999989584211356,
51
+ 0.29999983784788814,
52
+ 0.2999997514863681,
53
+ 0.2999997904095978,
54
+ 0.29999987607479484,
55
+ 0.29999983808389075,
56
+ 0.29999982376193984,
57
+ 0.299999843238001,
58
+ 0.2999997380879359,
59
+ 0.29999984200200724,
60
+ 0.2999998571695013,
61
+ 0.29999993043955364,
62
+ 0.2999998548770266,
63
+ 0.29999998275241463,
64
+ 0.2999998385367787,
65
+ 0.29999985670904267,
66
+ 0.2999998650251881,
67
+ 0.29999990770273105,
68
+ 0.2999997907082759,
69
+ 0.29999975991044014,
70
+ 0.2999998386237497,
71
+ 0.29999976701075326,
72
+ 0.2999996384655255,
73
+ 0.2999998653212779,
74
+ 0.29999942298684046,
75
+ 0.29999983928156604,
76
+ 0.2999998335974349,
77
+ 0.2999997630936651,
78
+ 0.29999957808098776,
79
+ 0.2999993324745417,
80
+ 0.29999964148031316,
81
+ 0.2999993150217098,
82
+ 0.2999994021040341,
83
+ 0.29999970079639704,
84
+ 0.2999996212821266,
85
+ 0.29999923384458965,
86
+ 0.2400580492521267,
87
+ 0.2999991859568465,
88
+ 0.29999952391317286,
89
+ 0.29999943933173767,
90
+ 0.23904489591558,
91
+ 0.22082136702898114,
92
+ 0.2999996496006251,
93
+ 0.29999960097840744,
94
+ 0.06084565240294252,
95
+ 0.29999985714989785,
96
+ 0.27001140019986175,
97
+ 0.24196433419393892,
98
+ 0.05825369805852839,
99
+ 0.2999991469929516,
100
+ 0.15705863171793738,
101
+ 0.03828652322329344,
102
+ 0.023200811595227796,
103
+ 0.04129434956180859,
104
+ 0.049138123368485935,
105
+ 0.04487694375693962,
106
+ 0.224603229033112,
107
+ 0.03229273522786361,
108
+ 0.032599237798871604,
109
+ 0.022936590735817202,
110
+ 0.02703242390409454,
111
+ 0.04120622897715978,
112
+ 0.08993883373234765,
113
+ 0.017092950887663435,
114
+ 0.0368967124461106,
115
+ 0.032362015474763244,
116
+ 0.020569942602627754,
117
+ 0.033200306693477066,
118
+ 0.030664452587451013,
119
+ 0.01549019716854724,
120
+ 0.06880451374310592,
121
+ 0.1791207763483352,
122
+ 0.035754383235644914,
123
+ 0.0195178739292786,
124
+ 0.021198802830342003,
125
+ 0.0159683130342503,
126
+ 0.03289778612980872,
127
+ 0.03374332580161473,
128
+ 0.026582601743105054
129
+ ],
130
+ "gradient_tensor_counts": [
131
+ 780,
132
+ 780,
133
+ 780,
134
+ 780,
135
+ 780,
136
+ 780,
137
+ 780,
138
+ 780,
139
+ 780,
140
+ 780,
141
+ 780,
142
+ 780,
143
+ 780,
144
+ 780,
145
+ 780,
146
+ 780,
147
+ 780,
148
+ 780,
149
+ 780,
150
+ 780,
151
+ 780,
152
+ 780,
153
+ 780,
154
+ 780,
155
+ 780,
156
+ 780,
157
+ 780,
158
+ 780,
159
+ 780,
160
+ 780,
161
+ 780,
162
+ 780,
163
+ 780,
164
+ 780,
165
+ 780,
166
+ 780,
167
+ 780,
168
+ 780,
169
+ 780,
170
+ 780,
171
+ 780,
172
+ 780,
173
+ 780,
174
+ 780,
175
+ 780,
176
+ 780,
177
+ 780,
178
+ 780,
179
+ 780,
180
+ 780,
181
+ 780,
182
+ 780,
183
+ 780,
184
+ 780,
185
+ 780,
186
+ 780,
187
+ 780,
188
+ 780,
189
+ 780,
190
+ 780,
191
+ 780,
192
+ 780,
193
+ 780,
194
+ 780,
195
+ 780,
196
+ 780,
197
+ 780,
198
+ 780,
199
+ 780,
200
+ 780,
201
+ 780,
202
+ 780,
203
+ 780,
204
+ 780,
205
+ 780,
206
+ 780,
207
+ 780,
208
+ 780,
209
+ 780,
210
+ 780,
211
+ 780,
212
+ 780,
213
+ 780,
214
+ 780,
215
+ 780,
216
+ 780,
217
+ 780,
218
+ 780,
219
+ 780,
220
+ 780,
221
+ 780,
222
+ 780,
223
+ 780,
224
+ 780,
225
+ 780,
226
+ 780,
227
+ 780,
228
+ 780,
229
+ 780,
230
+ 780,
231
+ 780,
232
+ 780,
233
+ 780,
234
+ 780,
235
+ 780,
236
+ 780,
237
+ 780,
238
+ 780,
239
+ 780,
240
+ 780,
241
+ 780,
242
+ 780,
243
+ 780,
244
+ 780,
245
+ 780,
246
+ 780,
247
+ 780,
248
+ 780,
249
+ 780,
250
+ 780
251
+ ],
252
+ "grad_scaler_values": [
253
+ 2048.0,
254
+ 2048.0,
255
+ 2048.0,
256
+ 2048.0,
257
+ 2048.0,
258
+ 2048.0,
259
+ 2048.0,
260
+ 2048.0,
261
+ 2048.0,
262
+ 2048.0,
263
+ 2048.0,
264
+ 2048.0,
265
+ 2048.0,
266
+ 2048.0,
267
+ 2048.0,
268
+ 2048.0,
269
+ 2048.0,
270
+ 2048.0,
271
+ 2048.0,
272
+ 2048.0,
273
+ 2048.0,
274
+ 2048.0,
275
+ 2048.0,
276
+ 2048.0,
277
+ 2048.0,
278
+ 2048.0,
279
+ 2048.0,
280
+ 2048.0,
281
+ 2048.0,
282
+ 2048.0,
283
+ 2048.0,
284
+ 2048.0,
285
+ 2048.0,
286
+ 2048.0,
287
+ 2048.0,
288
+ 2048.0,
289
+ 2048.0,
290
+ 2048.0,
291
+ 2048.0,
292
+ 2048.0,
293
+ 2048.0,
294
+ 2048.0,
295
+ 2048.0,
296
+ 2048.0,
297
+ 2048.0,
298
+ 2048.0,
299
+ 2048.0,
300
+ 2048.0,
301
+ 2048.0,
302
+ 2048.0,
303
+ 2048.0,
304
+ 2048.0,
305
+ 2048.0,
306
+ 2048.0,
307
+ 2048.0,
308
+ 2048.0,
309
+ 2048.0,
310
+ 2048.0,
311
+ 2048.0,
312
+ 2048.0,
313
+ 2048.0,
314
+ 2048.0,
315
+ 2048.0,
316
+ 2048.0,
317
+ 2048.0,
318
+ 2048.0,
319
+ 2048.0,
320
+ 2048.0,
321
+ 2048.0,
322
+ 2048.0,
323
+ 2048.0,
324
+ 2048.0,
325
+ 2048.0,
326
+ 2048.0,
327
+ 2048.0,
328
+ 2048.0,
329
+ 2048.0,
330
+ 2048.0,
331
+ 2048.0,
332
+ 2048.0,
333
+ 2048.0,
334
+ 2048.0,
335
+ 2048.0,
336
+ 2048.0,
337
+ 2048.0,
338
+ 2048.0,
339
+ 2048.0,
340
+ 2048.0,
341
+ 2048.0,
342
+ 2048.0,
343
+ 2048.0,
344
+ 2048.0,
345
+ 2048.0,
346
+ 2048.0,
347
+ 2048.0,
348
+ 2048.0,
349
+ 2048.0,
350
+ 2048.0,
351
+ 2048.0,
352
+ 2048.0,
353
+ 2048.0,
354
+ 2048.0,
355
+ 2048.0,
356
+ 2048.0,
357
+ 2048.0,
358
+ 2048.0,
359
+ 2048.0,
360
+ 2048.0,
361
+ 2048.0,
362
+ 2048.0,
363
+ 2048.0,
364
+ 2048.0,
365
+ 2048.0,
366
+ 2048.0,
367
+ 2048.0,
368
+ 2048.0,
369
+ 2048.0,
370
+ 2048.0,
371
+ 2048.0,
372
+ 2048.0
373
+ ],
374
+ "loss_history": [
375
+ {
376
+ "global_step": 1,
377
+ "epoch": 0.25,
378
+ "loss": 3.0028076171875,
379
+ "learning_rate": 0.0,
380
+ "grad_norm": 5.735446453094482
381
+ },
382
+ {
383
+ "global_step": 2,
384
+ "epoch": 0.5,
385
+ "loss": 2.5738320350646973,
386
+ "learning_rate": 3.3333333333333335e-05,
387
+ "grad_norm": 4.246273517608643
388
+ },
389
+ {
390
+ "global_step": 3,
391
+ "epoch": 0.75,
392
+ "loss": 3.9848618507385254,
393
+ "learning_rate": 6.666666666666667e-05,
394
+ "grad_norm": 7.595849990844727
395
+ },
396
+ {
397
+ "global_step": 4,
398
+ "epoch": 1.0,
399
+ "loss": 2.5158143043518066,
400
+ "learning_rate": 0.0001,
401
+ "grad_norm": 4.926611423492432
402
+ },
403
+ {
404
+ "global_step": 5,
405
+ "epoch": 1.25,
406
+ "loss": 2.6317954063415527,
407
+ "learning_rate": 9.998197638354428e-05,
408
+ "grad_norm": 3.7285847663879395
409
+ },
410
+ {
411
+ "global_step": 6,
412
+ "epoch": 1.5,
413
+ "loss": 3.391207218170166,
414
+ "learning_rate": 9.992791852820709e-05,
415
+ "grad_norm": 6.960104465484619
416
+ },
417
+ {
418
+ "global_step": 7,
419
+ "epoch": 1.75,
420
+ "loss": 1.7886579036712646,
421
+ "learning_rate": 9.983786540671051e-05,
422
+ "grad_norm": 3.496877431869507
423
+ },
424
+ {
425
+ "global_step": 8,
426
+ "epoch": 2.0,
427
+ "loss": 1.9815987348556519,
428
+ "learning_rate": 9.97118819423714e-05,
429
+ "grad_norm": 2.860832691192627
430
+ },
431
+ {
432
+ "global_step": 9,
433
+ "epoch": 2.25,
434
+ "loss": 1.775098443031311,
435
+ "learning_rate": 9.955005896229543e-05,
436
+ "grad_norm": 3.4511520862579346
437
+ },
438
+ {
439
+ "global_step": 10,
440
+ "epoch": 2.5,
441
+ "loss": 1.6050281524658203,
442
+ "learning_rate": 9.935251313189564e-05,
443
+ "grad_norm": 3.639047384262085
444
+ },
445
+ {
446
+ "global_step": 11,
447
+ "epoch": 2.75,
448
+ "loss": 2.5201776027679443,
449
+ "learning_rate": 9.911938687078324e-05,
450
+ "grad_norm": 3.662109613418579
451
+ },
452
+ {
453
+ "global_step": 12,
454
+ "epoch": 3.0,
455
+ "loss": 1.7487728595733643,
456
+ "learning_rate": 9.885084825009086e-05,
457
+ "grad_norm": 2.736379384994507
458
+ },
459
+ {
460
+ "global_step": 13,
461
+ "epoch": 3.25,
462
+ "loss": 1.685441255569458,
463
+ "learning_rate": 9.85470908713026e-05,
464
+ "grad_norm": 2.4754011631011963
465
+ },
466
+ {
467
+ "global_step": 14,
468
+ "epoch": 3.5,
469
+ "loss": 2.0612027645111084,
470
+ "learning_rate": 9.820833372667812e-05,
471
+ "grad_norm": 2.888793468475342
472
+ },
473
+ {
474
+ "global_step": 15,
475
+ "epoch": 3.75,
476
+ "loss": 1.2462215423583984,
477
+ "learning_rate": 9.783482104137127e-05,
478
+ "grad_norm": 2.124307632446289
479
+ },
480
+ {
481
+ "global_step": 16,
482
+ "epoch": 4.0,
483
+ "loss": 1.5945758819580078,
484
+ "learning_rate": 9.742682209735727e-05,
485
+ "grad_norm": 2.0276503562927246
486
+ },
487
+ {
488
+ "global_step": 17,
489
+ "epoch": 4.25,
490
+ "loss": 1.8966870307922363,
491
+ "learning_rate": 9.698463103929542e-05,
492
+ "grad_norm": 2.2649166584014893
493
+ },
494
+ {
495
+ "global_step": 18,
496
+ "epoch": 4.5,
497
+ "loss": 1.220649242401123,
498
+ "learning_rate": 9.650856666246693e-05,
499
+ "grad_norm": 1.7242375612258911
500
+ },
501
+ {
502
+ "global_step": 19,
503
+ "epoch": 4.75,
504
+ "loss": 1.4429657459259033,
505
+ "learning_rate": 9.599897218294122e-05,
506
+ "grad_norm": 1.889949083328247
507
+ },
508
+ {
509
+ "global_step": 20,
510
+ "epoch": 5.0,
511
+ "loss": 1.791039228439331,
512
+ "learning_rate": 9.54562149901362e-05,
513
+ "grad_norm": 3.159863233566284
514
+ },
515
+ {
516
+ "global_step": 21,
517
+ "epoch": 5.25,
518
+ "loss": 1.2491681575775146,
519
+ "learning_rate": 9.48806863819507e-05,
520
+ "grad_norm": 2.510749340057373
521
+ },
522
+ {
523
+ "global_step": 22,
524
+ "epoch": 5.5,
525
+ "loss": 0.9549908638000488,
526
+ "learning_rate": 9.42728012826605e-05,
527
+ "grad_norm": 1.8189880847930908
528
+ },
529
+ {
530
+ "global_step": 23,
531
+ "epoch": 5.75,
532
+ "loss": 1.671364188194275,
533
+ "learning_rate": 9.363299794378073e-05,
534
+ "grad_norm": 1.8584535121917725
535
+ },
536
+ {
537
+ "global_step": 24,
538
+ "epoch": 6.0,
539
+ "loss": 1.1979198455810547,
540
+ "learning_rate": 9.296173762811085e-05,
541
+ "grad_norm": 2.8647730350494385
542
+ },
543
+ {
544
+ "global_step": 25,
545
+ "epoch": 6.25,
546
+ "loss": 1.0778791904449463,
547
+ "learning_rate": 9.225950427718975e-05,
548
+ "grad_norm": 2.2521347999572754
549
+ },
550
+ {
551
+ "global_step": 26,
552
+ "epoch": 6.5,
553
+ "loss": 0.9626718759536743,
554
+ "learning_rate": 9.152680416240059e-05,
555
+ "grad_norm": 2.9740376472473145
556
+ },
557
+ {
558
+ "global_step": 27,
559
+ "epoch": 6.75,
560
+ "loss": 1.4524807929992676,
561
+ "learning_rate": 9.076416551997721e-05,
562
+ "grad_norm": 1.8301929235458374
563
+ },
564
+ {
565
+ "global_step": 28,
566
+ "epoch": 7.0,
567
+ "loss": 0.9822371006011963,
568
+ "learning_rate": 8.997213817017507e-05,
569
+ "grad_norm": 1.515589952468872
570
+ },
571
+ {
572
+ "global_step": 29,
573
+ "epoch": 7.25,
574
+ "loss": 1.3678052425384521,
575
+ "learning_rate": 8.915129312088112e-05,
576
+ "grad_norm": 2.429225444793701
577
+ },
578
+ {
579
+ "global_step": 30,
580
+ "epoch": 7.5,
581
+ "loss": 0.7263110876083374,
582
+ "learning_rate": 8.83022221559489e-05,
583
+ "grad_norm": 1.299315333366394
584
+ },
585
+ {
586
+ "global_step": 31,
587
+ "epoch": 7.75,
588
+ "loss": 0.8736010789871216,
589
+ "learning_rate": 8.742553740855506e-05,
590
+ "grad_norm": 1.5025458335876465
591
+ },
592
+ {
593
+ "global_step": 32,
594
+ "epoch": 8.0,
595
+ "loss": 0.9786587357521057,
596
+ "learning_rate": 8.652187091988517e-05,
597
+ "grad_norm": 1.3604384660720825
598
+ },
599
+ {
600
+ "global_step": 33,
601
+ "epoch": 8.25,
602
+ "loss": 0.5913520455360413,
603
+ "learning_rate": 8.559187418346703e-05,
604
+ "grad_norm": 3.1408023834228516
605
+ },
606
+ {
607
+ "global_step": 34,
608
+ "epoch": 8.5,
609
+ "loss": 0.8007135391235352,
610
+ "learning_rate": 8.463621767547998e-05,
611
+ "grad_norm": 1.5192880630493164
612
+ },
613
+ {
614
+ "global_step": 35,
615
+ "epoch": 8.75,
616
+ "loss": 0.8744857311248779,
617
+ "learning_rate": 8.365559037137852e-05,
618
+ "grad_norm": 2.1710562705993652
619
+ },
620
+ {
621
+ "global_step": 36,
622
+ "epoch": 9.0,
623
+ "loss": 0.8190163373947144,
624
+ "learning_rate": 8.265069924917925e-05,
625
+ "grad_norm": 3.7557790279388428
626
+ },
627
+ {
628
+ "global_step": 37,
629
+ "epoch": 9.25,
630
+ "loss": 0.7986066341400146,
631
+ "learning_rate": 8.162226877976887e-05,
632
+ "grad_norm": 1.8751171827316284
633
+ },
634
+ {
635
+ "global_step": 38,
636
+ "epoch": 9.5,
637
+ "loss": 0.5472699403762817,
638
+ "learning_rate": 8.057104040460062e-05,
639
+ "grad_norm": 1.435860514640808
640
+ },
641
+ {
642
+ "global_step": 39,
643
+ "epoch": 9.75,
644
+ "loss": 0.728087306022644,
645
+ "learning_rate": 7.949777200115616e-05,
646
+ "grad_norm": 2.4945225715637207
647
+ },
648
+ {
649
+ "global_step": 40,
650
+ "epoch": 10.0,
651
+ "loss": 0.7109437584877014,
652
+ "learning_rate": 7.840323733655778e-05,
653
+ "grad_norm": 1.7653728723526
654
+ },
655
+ {
656
+ "global_step": 41,
657
+ "epoch": 10.25,
658
+ "loss": 0.4470646381378174,
659
+ "learning_rate": 7.728822550972523e-05,
660
+ "grad_norm": 1.2734816074371338
661
+ },
662
+ {
663
+ "global_step": 42,
664
+ "epoch": 10.5,
665
+ "loss": 0.7584569454193115,
666
+ "learning_rate": 7.615354038247888e-05,
667
+ "grad_norm": 2.3084428310394287
668
+ },
669
+ {
670
+ "global_step": 43,
671
+ "epoch": 10.75,
672
+ "loss": 0.46312659978866577,
673
+ "learning_rate": 7.500000000000001e-05,
674
+ "grad_norm": 1.7591501474380493
675
+ },
676
+ {
677
+ "global_step": 44,
678
+ "epoch": 11.0,
679
+ "loss": 0.45575660467147827,
680
+ "learning_rate": 7.38284360010654e-05,
681
+ "grad_norm": 1.154929518699646
682
+ },
683
+ {
684
+ "global_step": 45,
685
+ "epoch": 11.25,
686
+ "loss": 0.4252345860004425,
687
+ "learning_rate": 7.263969301848188e-05,
688
+ "grad_norm": 1.0464447736740112
689
+ },
690
+ {
691
+ "global_step": 46,
692
+ "epoch": 11.5,
693
+ "loss": 0.5287308692932129,
694
+ "learning_rate": 7.143462807015271e-05,
695
+ "grad_norm": 2.1150853633880615
696
+ },
697
+ {
698
+ "global_step": 47,
699
+ "epoch": 11.75,
700
+ "loss": 0.4766055643558502,
701
+ "learning_rate": 7.021410994121525e-05,
702
+ "grad_norm": 2.03830885887146
703
+ },
704
+ {
705
+ "global_step": 48,
706
+ "epoch": 12.0,
707
+ "loss": 0.23271411657333374,
708
+ "learning_rate": 6.897901855769483e-05,
709
+ "grad_norm": 1.4084392786026
710
+ },
711
+ {
712
+ "global_step": 49,
713
+ "epoch": 12.25,
714
+ "loss": 0.35807982087135315,
715
+ "learning_rate": 6.773024435212678e-05,
716
+ "grad_norm": 1.8897265195846558
717
+ },
718
+ {
719
+ "global_step": 50,
720
+ "epoch": 12.5,
721
+ "loss": 0.18392693996429443,
722
+ "learning_rate": 6.646868762160399e-05,
723
+ "grad_norm": 1.135053038597107
724
+ },
725
+ {
726
+ "global_step": 51,
727
+ "epoch": 12.75,
728
+ "loss": 0.4107872247695923,
729
+ "learning_rate": 6.519525787871235e-05,
730
+ "grad_norm": 1.7325575351715088
731
+ },
732
+ {
733
+ "global_step": 52,
734
+ "epoch": 13.0,
735
+ "loss": 0.295409232378006,
736
+ "learning_rate": 6.391087319582264e-05,
737
+ "grad_norm": 1.7553775310516357
738
+ },
739
+ {
740
+ "global_step": 53,
741
+ "epoch": 13.25,
742
+ "loss": 0.24841588735580444,
743
+ "learning_rate": 6.261645954321109e-05,
744
+ "grad_norm": 2.4505655765533447
745
+ },
746
+ {
747
+ "global_step": 54,
748
+ "epoch": 13.5,
749
+ "loss": 0.28180259466171265,
750
+ "learning_rate": 6.131295012148612e-05,
751
+ "grad_norm": 1.7769156694412231
752
+ },
753
+ {
754
+ "global_step": 55,
755
+ "epoch": 13.75,
756
+ "loss": 0.21721918880939484,
757
+ "learning_rate": 6.0001284688802226e-05,
758
+ "grad_norm": 8.174845695495605
759
+ },
760
+ {
761
+ "global_step": 56,
762
+ "epoch": 14.0,
763
+ "loss": 0.18195383250713348,
764
+ "learning_rate": 5.868240888334653e-05,
765
+ "grad_norm": 1.8216534852981567
766
+ },
767
+ {
768
+ "global_step": 57,
769
+ "epoch": 14.25,
770
+ "loss": 0.15293434262275696,
771
+ "learning_rate": 5.735727354158581e-05,
772
+ "grad_norm": 1.528199315071106
773
+ },
774
+ {
775
+ "global_step": 58,
776
+ "epoch": 14.5,
777
+ "loss": 0.18003931641578674,
778
+ "learning_rate": 5.602683401276615e-05,
779
+ "grad_norm": 1.7730704545974731
780
+ },
781
+ {
782
+ "global_step": 59,
783
+ "epoch": 14.75,
784
+ "loss": 0.16167467832565308,
785
+ "learning_rate": 5.469204947015897e-05,
786
+ "grad_norm": 2.5438311100006104
787
+ },
788
+ {
789
+ "global_step": 60,
790
+ "epoch": 15.0,
791
+ "loss": 0.06871137768030167,
792
+ "learning_rate": 5.335388221955012e-05,
793
+ "grad_norm": 1.244670033454895
794
+ },
795
+ {
796
+ "global_step": 61,
797
+ "epoch": 15.25,
798
+ "loss": 0.08273965865373611,
799
+ "learning_rate": 5.201329700547076e-05,
800
+ "grad_norm": 1.158942699432373
801
+ },
802
+ {
803
+ "global_step": 62,
804
+ "epoch": 15.5,
805
+ "loss": 0.1550861895084381,
806
+ "learning_rate": 5.0671260315669875e-05,
807
+ "grad_norm": 1.5921404361724854
808
+ },
809
+ {
810
+ "global_step": 63,
811
+ "epoch": 15.75,
812
+ "loss": 0.07742718607187271,
813
+ "learning_rate": 4.9328739684330137e-05,
814
+ "grad_norm": 1.2498779296875
815
+ },
816
+ {
817
+ "global_step": 64,
818
+ "epoch": 16.0,
819
+ "loss": 0.060180872678756714,
820
+ "learning_rate": 4.798670299452926e-05,
821
+ "grad_norm": 0.8160451650619507
822
+ },
823
+ {
824
+ "global_step": 65,
825
+ "epoch": 16.25,
826
+ "loss": 0.11217842251062393,
827
+ "learning_rate": 4.6646117780449876e-05,
828
+ "grad_norm": 2.351928472518921
829
+ },
830
+ {
831
+ "global_step": 66,
832
+ "epoch": 16.5,
833
+ "loss": 0.029149621725082397,
834
+ "learning_rate": 4.530795052984104e-05,
835
+ "grad_norm": 0.5133053064346313
836
+ },
837
+ {
838
+ "global_step": 67,
839
+ "epoch": 16.75,
840
+ "loss": 0.057295918464660645,
841
+ "learning_rate": 4.397316598723385e-05,
842
+ "grad_norm": 1.4209790229797363
843
+ },
844
+ {
845
+ "global_step": 68,
846
+ "epoch": 17.0,
847
+ "loss": 0.05391449108719826,
848
+ "learning_rate": 4.264272645841419e-05,
849
+ "grad_norm": 1.5596890449523926
850
+ },
851
+ {
852
+ "global_step": 69,
853
+ "epoch": 17.25,
854
+ "loss": 0.07454235851764679,
855
+ "learning_rate": 4.131759111665349e-05,
856
+ "grad_norm": 1.2337915897369385
857
+ },
858
+ {
859
+ "global_step": 70,
860
+ "epoch": 17.5,
861
+ "loss": 0.03401903435587883,
862
+ "learning_rate": 3.9998715311197785e-05,
863
+ "grad_norm": 0.718992292881012
864
+ },
865
+ {
866
+ "global_step": 71,
867
+ "epoch": 17.75,
868
+ "loss": 0.016408829018473625,
869
+ "learning_rate": 3.86870498785139e-05,
870
+ "grad_norm": 0.44305819272994995
871
+ },
872
+ {
873
+ "global_step": 72,
874
+ "epoch": 18.0,
875
+ "loss": 0.03439335897564888,
876
+ "learning_rate": 3.738354045678891e-05,
877
+ "grad_norm": 0.8151929974555969
878
+ },
879
+ {
880
+ "global_step": 73,
881
+ "epoch": 18.25,
882
+ "loss": 0.01341971568763256,
883
+ "learning_rate": 3.608912680417737e-05,
884
+ "grad_norm": 0.44037967920303345
885
+ },
886
+ {
887
+ "global_step": 74,
888
+ "epoch": 18.5,
889
+ "loss": 0.015053657814860344,
890
+ "learning_rate": 3.480474212128766e-05,
891
+ "grad_norm": 0.505387008190155
892
+ },
893
+ {
894
+ "global_step": 75,
895
+ "epoch": 18.75,
896
+ "loss": 0.020888736471533775,
897
+ "learning_rate": 3.3531312378396026e-05,
898
+ "grad_norm": 0.9699237942695618
899
+ },
900
+ {
901
+ "global_step": 76,
902
+ "epoch": 19.0,
903
+ "loss": 0.015053548850119114,
904
+ "learning_rate": 3.226975564787322e-05,
905
+ "grad_norm": 0.7767230868339539
906
+ },
907
+ {
908
+ "global_step": 77,
909
+ "epoch": 19.25,
910
+ "loss": 0.005396610591560602,
911
+ "learning_rate": 3.1020981442305184e-05,
912
+ "grad_norm": 0.40644827485084534
913
+ },
914
+ {
915
+ "global_step": 78,
916
+ "epoch": 19.5,
917
+ "loss": 0.004981599748134613,
918
+ "learning_rate": 2.978589005878476e-05,
919
+ "grad_norm": 0.2400580644607544
920
+ },
921
+ {
922
+ "global_step": 79,
923
+ "epoch": 19.75,
924
+ "loss": 0.004586038179695606,
925
+ "learning_rate": 2.8565371929847284e-05,
926
+ "grad_norm": 0.35911986231803894
927
+ },
928
+ {
929
+ "global_step": 80,
930
+ "epoch": 20.0,
931
+ "loss": 0.013846193440258503,
932
+ "learning_rate": 2.7360306981518146e-05,
933
+ "grad_norm": 0.644167959690094
934
+ },
935
+ {
936
+ "global_step": 81,
937
+ "epoch": 20.25,
938
+ "loss": 0.009072684682905674,
939
+ "learning_rate": 2.6171563998934605e-05,
940
+ "grad_norm": 0.559910237789154
941
+ },
942
+ {
943
+ "global_step": 82,
944
+ "epoch": 20.5,
945
+ "loss": 0.0030265708919614553,
946
+ "learning_rate": 2.500000000000001e-05,
947
+ "grad_norm": 0.2390448898077011
948
+ },
949
+ {
950
+ "global_step": 83,
951
+ "epoch": 20.75,
952
+ "loss": 0.0019349269568920135,
953
+ "learning_rate": 2.3846459617521128e-05,
954
+ "grad_norm": 0.220821350812912
955
+ },
956
+ {
957
+ "global_step": 84,
958
+ "epoch": 21.0,
959
+ "loss": 0.008226916193962097,
960
+ "learning_rate": 2.2711774490274766e-05,
961
+ "grad_norm": 0.8569097518920898
962
+ },
963
+ {
964
+ "global_step": 85,
965
+ "epoch": 21.25,
966
+ "loss": 0.007080879062414169,
967
+ "learning_rate": 2.1596762663442218e-05,
968
+ "grad_norm": 0.7534610629081726
969
+ },
970
+ {
971
+ "global_step": 86,
972
+ "epoch": 21.5,
973
+ "loss": 0.0010789937805384398,
974
+ "learning_rate": 2.050222799884387e-05,
975
+ "grad_norm": 0.06084565073251724
976
+ },
977
+ {
978
+ "global_step": 87,
979
+ "epoch": 21.75,
980
+ "loss": 0.0016643002163618803,
981
+ "learning_rate": 1.942895959539939e-05,
982
+ "grad_norm": 1.5307955741882324
983
+ },
984
+ {
985
+ "global_step": 88,
986
+ "epoch": 22.0,
987
+ "loss": 0.002636656863614917,
988
+ "learning_rate": 1.837773122023114e-05,
989
+ "grad_norm": 0.27001139521598816
990
+ },
991
+ {
992
+ "global_step": 89,
993
+ "epoch": 22.25,
994
+ "loss": 0.0008208074723370373,
995
+ "learning_rate": 1.734930075082076e-05,
996
+ "grad_norm": 0.24196434020996094
997
+ },
998
+ {
999
+ "global_step": 90,
1000
+ "epoch": 22.5,
1001
+ "loss": 0.001030398067086935,
1002
+ "learning_rate": 1.6344409628621484e-05,
1003
+ "grad_norm": 0.0582536980509758
1004
+ },
1005
+ {
1006
+ "global_step": 91,
1007
+ "epoch": 22.75,
1008
+ "loss": 0.001805206760764122,
1009
+ "learning_rate": 1.536378232452003e-05,
1010
+ "grad_norm": 0.3588670790195465
1011
+ },
1012
+ {
1013
+ "global_step": 92,
1014
+ "epoch": 23.0,
1015
+ "loss": 0.0013909051194787025,
1016
+ "learning_rate": 1.440812581653298e-05,
1017
+ "grad_norm": 0.15705862641334534
1018
+ },
1019
+ {
1020
+ "global_step": 93,
1021
+ "epoch": 23.25,
1022
+ "loss": 0.0007190693868324161,
1023
+ "learning_rate": 1.3478129080114848e-05,
1024
+ "grad_norm": 0.03828652575612068
1025
+ },
1026
+ {
1027
+ "global_step": 94,
1028
+ "epoch": 23.5,
1029
+ "loss": 0.00025069594266824424,
1030
+ "learning_rate": 1.257446259144494e-05,
1031
+ "grad_norm": 0.023200809955596924
1032
+ },
1033
+ {
1034
+ "global_step": 95,
1035
+ "epoch": 23.75,
1036
+ "loss": 0.0011277927551418543,
1037
+ "learning_rate": 1.1697777844051105e-05,
1038
+ "grad_norm": 0.04129434749484062
1039
+ },
1040
+ {
1041
+ "global_step": 96,
1042
+ "epoch": 24.0,
1043
+ "loss": 0.0009472159435972571,
1044
+ "learning_rate": 1.0848706879118892e-05,
1045
+ "grad_norm": 0.04913812503218651
1046
+ },
1047
+ {
1048
+ "global_step": 97,
1049
+ "epoch": 24.25,
1050
+ "loss": 0.0007696544635109603,
1051
+ "learning_rate": 1.0027861829824952e-05,
1052
+ "grad_norm": 0.04487694054841995
1053
+ },
1054
+ {
1055
+ "global_step": 98,
1056
+ "epoch": 24.5,
1057
+ "loss": 0.0009453248931095004,
1058
+ "learning_rate": 9.235834480022787e-06,
1059
+ "grad_norm": 0.22460322082042694
1060
+ },
1061
+ {
1062
+ "global_step": 99,
1063
+ "epoch": 24.75,
1064
+ "loss": 0.000631998700555414,
1065
+ "learning_rate": 8.473195837599418e-06,
1066
+ "grad_norm": 0.03229273483157158
1067
+ },
1068
+ {
1069
+ "global_step": 100,
1070
+ "epoch": 25.0,
1071
+ "loss": 0.0006901748711243272,
1072
+ "learning_rate": 7.740495722810271e-06,
1073
+ "grad_norm": 0.03259923681616783
1074
+ },
1075
+ {
1076
+ "global_step": 101,
1077
+ "epoch": 25.25,
1078
+ "loss": 0.0003148592368233949,
1079
+ "learning_rate": 7.038262371889159e-06,
1080
+ "grad_norm": 0.022936590015888214
1081
+ },
1082
+ {
1083
+ "global_step": 102,
1084
+ "epoch": 25.5,
1085
+ "loss": 0.0004651989438571036,
1086
+ "learning_rate": 6.367002056219284e-06,
1087
+ "grad_norm": 0.02703242376446724
1088
+ },
1089
+ {
1090
+ "global_step": 103,
1091
+ "epoch": 25.75,
1092
+ "loss": 0.0010305196046829224,
1093
+ "learning_rate": 5.727198717339511e-06,
1094
+ "grad_norm": 0.041206229478120804
1095
+ },
1096
+ {
1097
+ "global_step": 104,
1098
+ "epoch": 26.0,
1099
+ "loss": 0.000634815136436373,
1100
+ "learning_rate": 5.1193136180493095e-06,
1101
+ "grad_norm": 0.08993884176015854
1102
+ },
1103
+ {
1104
+ "global_step": 105,
1105
+ "epoch": 26.25,
1106
+ "loss": 0.00019032439740840346,
1107
+ "learning_rate": 4.54378500986381e-06,
1108
+ "grad_norm": 0.017092950642108917
1109
+ },
1110
+ {
1111
+ "global_step": 106,
1112
+ "epoch": 26.5,
1113
+ "loss": 0.0008260362083092332,
1114
+ "learning_rate": 4.001027817058789e-06,
1115
+ "grad_norm": 0.036896709352731705
1116
+ },
1117
+ {
1118
+ "global_step": 107,
1119
+ "epoch": 26.75,
1120
+ "loss": 0.0005347628612071276,
1121
+ "learning_rate": 3.4914333375330898e-06,
1122
+ "grad_norm": 0.032362014055252075
1123
+ },
1124
+ {
1125
+ "global_step": 108,
1126
+ "epoch": 27.0,
1127
+ "loss": 0.0005628281505778432,
1128
+ "learning_rate": 3.0153689607045845e-06,
1129
+ "grad_norm": 0.020569942891597748
1130
+ },
1131
+ {
1132
+ "global_step": 109,
1133
+ "epoch": 27.25,
1134
+ "loss": 0.0006590321427211165,
1135
+ "learning_rate": 2.573177902642726e-06,
1136
+ "grad_norm": 0.033200304955244064
1137
+ },
1138
+ {
1139
+ "global_step": 110,
1140
+ "epoch": 27.5,
1141
+ "loss": 0.0005788324633613229,
1142
+ "learning_rate": 2.1651789586287442e-06,
1143
+ "grad_norm": 0.03066444955766201
1144
+ },
1145
+ {
1146
+ "global_step": 111,
1147
+ "epoch": 27.75,
1148
+ "loss": 0.0002574412210378796,
1149
+ "learning_rate": 1.7916662733218847e-06,
1150
+ "grad_norm": 0.015490197576582432
1151
+ },
1152
+ {
1153
+ "global_step": 112,
1154
+ "epoch": 28.0,
1155
+ "loss": 0.0007605571881867945,
1156
+ "learning_rate": 1.4529091286973995e-06,
1157
+ "grad_norm": 0.06880451738834381
1158
+ },
1159
+ {
1160
+ "global_step": 113,
1161
+ "epoch": 28.25,
1162
+ "loss": 0.0004236309032421559,
1163
+ "learning_rate": 1.1491517499091498e-06,
1164
+ "grad_norm": 0.17912079393863678
1165
+ },
1166
+ {
1167
+ "global_step": 114,
1168
+ "epoch": 28.5,
1169
+ "loss": 0.0008066121954470873,
1170
+ "learning_rate": 8.806131292167618e-07,
1171
+ "grad_norm": 0.035754382610321045
1172
+ },
1173
+ {
1174
+ "global_step": 115,
1175
+ "epoch": 28.75,
1176
+ "loss": 0.000279993808362633,
1177
+ "learning_rate": 6.474868681043578e-07,
1178
+ "grad_norm": 0.019517874345183372
1179
+ },
1180
+ {
1181
+ "global_step": 116,
1182
+ "epoch": 29.0,
1183
+ "loss": 0.0005598465213552117,
1184
+ "learning_rate": 4.4994103770457653e-07,
1185
+ "grad_norm": 0.021198801696300507
1186
+ },
1187
+ {
1188
+ "global_step": 117,
1189
+ "epoch": 29.25,
1190
+ "loss": 0.000262636283878237,
1191
+ "learning_rate": 2.8811805762860576e-07,
1192
+ "grad_norm": 0.015968313440680504
1193
+ },
1194
+ {
1195
+ "global_step": 118,
1196
+ "epoch": 29.5,
1197
+ "loss": 0.000800407025963068,
1198
+ "learning_rate": 1.6213459328950352e-07,
1199
+ "grad_norm": 0.03289778530597687
1200
+ },
1201
+ {
1202
+ "global_step": 119,
1203
+ "epoch": 29.75,
1204
+ "loss": 0.0005571640795096755,
1205
+ "learning_rate": 7.208147179291192e-08,
1206
+ "grad_norm": 0.033743325620889664
1207
+ },
1208
+ {
1209
+ "global_step": 120,
1210
+ "epoch": 30.0,
1211
+ "loss": 0.00039453001227229834,
1212
+ "learning_rate": 1.802361645573125e-08,
1213
+ "grad_norm": 0.02658260241150856
1214
+ },
1215
+ {
1216
+ "global_step": 120,
1217
+ "epoch": 30.0,
1218
+ "loss": null,
1219
+ "learning_rate": null,
1220
+ "grad_norm": null
1221
+ }
1222
+ ]
1223
+ }
gemma4_lora/processor/chat_template.jinja ADDED
@@ -0,0 +1,330 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {%- macro format_parameters(properties, required) -%}
2
+ {%- set standard_keys = ['description', 'type', 'properties', 'required', 'nullable'] -%}
3
+ {%- set ns = namespace(found_first=false) -%}
4
+ {%- for key, value in properties | dictsort -%}
5
+ {%- set add_comma = false -%}
6
+ {%- if key not in standard_keys -%}
7
+ {%- if ns.found_first %},{% endif -%}
8
+ {%- set ns.found_first = true -%}
9
+ {{ key }}:{
10
+ {%- if value['description'] -%}
11
+ description:<|"|>{{ value['description'] }}<|"|>
12
+ {%- set add_comma = true -%}
13
+ {%- endif -%}
14
+ {%- if value['type'] | upper == 'STRING' -%}
15
+ {%- if value['enum'] -%}
16
+ {%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
17
+ enum:{{ format_argument(value['enum']) }}
18
+ {%- endif -%}
19
+ {%- elif value['type'] | upper == 'ARRAY' -%}
20
+ {%- if value['items'] is mapping and value['items'] -%}
21
+ {%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
22
+ items:{
23
+ {%- set ns_items = namespace(found_first=false) -%}
24
+ {%- for item_key, item_value in value['items'] | dictsort -%}
25
+ {%- if item_value is not none -%}
26
+ {%- if ns_items.found_first %},{% endif -%}
27
+ {%- set ns_items.found_first = true -%}
28
+ {%- if item_key == 'properties' -%}
29
+ properties:{
30
+ {%- if item_value is mapping -%}
31
+ {{- format_parameters(item_value, value['items']['required'] | default([])) -}}
32
+ {%- endif -%}
33
+ }
34
+ {%- elif item_key == 'required' -%}
35
+ required:[
36
+ {%- for req_item in item_value -%}
37
+ <|"|>{{- req_item -}}<|"|>
38
+ {%- if not loop.last %},{% endif -%}
39
+ {%- endfor -%}
40
+ ]
41
+ {%- elif item_key == 'type' -%}
42
+ {%- if item_value is string -%}
43
+ type:{{ format_argument(item_value | upper) }}
44
+ {%- else -%}
45
+ type:{{ format_argument(item_value | map('upper') | list) }}
46
+ {%- endif -%}
47
+ {%- else -%}
48
+ {{ item_key }}:{{ format_argument(item_value) }}
49
+ {%- endif -%}
50
+ {%- endif -%}
51
+ {%- endfor -%}
52
+ }
53
+ {%- endif -%}
54
+ {%- endif -%}
55
+ {%- if value['nullable'] %}
56
+ {%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
57
+ nullable:true
58
+ {%- endif -%}
59
+ {%- if value['type'] | upper == 'OBJECT' -%}
60
+ {%- if value['properties'] is defined and value['properties'] is mapping -%}
61
+ {%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
62
+ properties:{
63
+ {{- format_parameters(value['properties'], value['required'] | default([])) -}}
64
+ }
65
+ {%- elif value is mapping -%}
66
+ {%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
67
+ properties:{
68
+ {{- format_parameters(value, value['required'] | default([])) -}}
69
+ }
70
+ {%- endif -%}
71
+ {%- if value['required'] -%}
72
+ {%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
73
+ required:[
74
+ {%- for item in value['required'] | default([]) -%}
75
+ <|"|>{{- item -}}<|"|>
76
+ {%- if not loop.last %},{% endif -%}
77
+ {%- endfor -%}
78
+ ]
79
+ {%- endif -%}
80
+ {%- endif -%}
81
+ {%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
82
+ type:<|"|>{{ value['type'] | upper }}<|"|>}
83
+ {%- endif -%}
84
+ {%- endfor -%}
85
+ {%- endmacro -%}
86
+ {%- macro format_function_declaration(tool_data) -%}
87
+ declaration:{{- tool_data['function']['name'] -}}{description:<|"|>{{- tool_data['function']['description'] -}}<|"|>
88
+ {%- set params = tool_data['function']['parameters'] -%}
89
+ {%- if params -%}
90
+ ,parameters:{
91
+ {%- if params['properties'] -%}
92
+ properties:{ {{- format_parameters(params['properties'], params['required']) -}} },
93
+ {%- endif -%}
94
+ {%- if params['required'] -%}
95
+ required:[
96
+ {%- for item in params['required'] -%}
97
+ <|"|>{{- item -}}<|"|>
98
+ {{- ',' if not loop.last -}}
99
+ {%- endfor -%}
100
+ ],
101
+ {%- endif -%}
102
+ {%- if params['type'] -%}
103
+ type:<|"|>{{- params['type'] | upper -}}<|"|>}
104
+ {%- endif -%}
105
+ {%- endif -%}
106
+ {%- if 'response' in tool_data['function'] -%}
107
+ {%- set response_declaration = tool_data['function']['response'] -%}
108
+ ,response:{
109
+ {%- if response_declaration['description'] -%}
110
+ description:<|"|>{{- response_declaration['description'] -}}<|"|>,
111
+ {%- endif -%}
112
+ {%- if response_declaration['type'] | upper == 'OBJECT' -%}
113
+ type:<|"|>{{- response_declaration['type'] | upper -}}<|"|>}
114
+ {%- endif -%}
115
+ {%- endif -%}
116
+ }
117
+ {%- endmacro -%}
118
+ {%- macro format_argument(argument, escape_keys=True) -%}
119
+ {%- if argument is string -%}
120
+ {{- '<|"|>' + argument + '<|"|>' -}}
121
+ {%- elif argument is boolean -%}
122
+ {{- 'true' if argument else 'false' -}}
123
+ {%- elif argument is mapping -%}
124
+ {{- '{' -}}
125
+ {%- set ns = namespace(found_first=false) -%}
126
+ {%- for key, value in argument | dictsort -%}
127
+ {%- if ns.found_first %},{% endif -%}
128
+ {%- set ns.found_first = true -%}
129
+ {%- if escape_keys -%}
130
+ {{- '<|"|>' + key + '<|"|>' -}}
131
+ {%- else -%}
132
+ {{- key -}}
133
+ {%- endif -%}
134
+ :{{- format_argument(value, escape_keys=escape_keys) -}}
135
+ {%- endfor -%}
136
+ {{- '}' -}}
137
+ {%- elif argument is sequence -%}
138
+ {{- '[' -}}
139
+ {%- for item in argument -%}
140
+ {{- format_argument(item, escape_keys=escape_keys) -}}
141
+ {%- if not loop.last %},{% endif -%}
142
+ {%- endfor -%}
143
+ {{- ']' -}}
144
+ {%- else -%}
145
+ {{- argument -}}
146
+ {%- endif -%}
147
+ {%- endmacro -%}
148
+ {%- macro strip_thinking(text) -%}
149
+ {%- set ns = namespace(result='') -%}
150
+ {%- for part in text.split('<channel|>') -%}
151
+ {%- if '<|channel>' in part -%}
152
+ {%- set ns.result = ns.result + part.split('<|channel>')[0] -%}
153
+ {%- else -%}
154
+ {%- set ns.result = ns.result + part -%}
155
+ {%- endif -%}
156
+ {%- endfor -%}
157
+ {{- ns.result | trim -}}
158
+ {%- endmacro -%}
159
+ {%- macro format_tool_response_block(tool_name, response) -%}
160
+ {{- '<|tool_response>' -}}
161
+ {%- if response is mapping -%}
162
+ {{- 'response:' + tool_name + '{' -}}
163
+ {%- for key, value in response | dictsort -%}
164
+ {{- key -}}:{{- format_argument(value, escape_keys=False) -}}
165
+ {%- if not loop.last %},{% endif -%}
166
+ {%- endfor -%}
167
+ {{- '}' -}}
168
+ {%- else -%}
169
+ {{- 'response:' + tool_name + '{value:' + format_argument(response, escape_keys=False) + '}' -}}
170
+ {%- endif -%}
171
+ {{- '<tool_response|>' -}}
172
+ {%- endmacro -%}
173
+ {%- set ns = namespace(prev_message_type=None) -%}
174
+ {%- set loop_messages = messages -%}
175
+ {{- bos_token -}}
176
+ {#- Handle System/Tool Definitions Block -#}
177
+ {%- if (enable_thinking is defined and enable_thinking) or tools or messages[0]['role'] in ['system', 'developer'] -%}
178
+ {{- '<|turn>system\n' -}}
179
+ {#- Inject Thinking token at the very top of the FIRST system turn -#}
180
+ {%- if enable_thinking is defined and enable_thinking -%}
181
+ {{- '<|think|>\n' -}}
182
+ {%- set ns.prev_message_type = 'think' -%}
183
+ {%- endif -%}
184
+ {%- if messages[0]['role'] in ['system', 'developer'] -%}
185
+ {{- messages[0]['content'] | trim -}}
186
+ {%- set loop_messages = messages[1:] -%}
187
+ {%- endif -%}
188
+ {%- if tools -%}
189
+ {%- for tool in tools %}
190
+ {{- '<|tool>' -}}
191
+ {{- format_function_declaration(tool) | trim -}}
192
+ {{- '<tool|>' -}}
193
+ {%- endfor %}
194
+ {%- set ns.prev_message_type = 'tool' -%}
195
+ {%- endif -%}
196
+ {{- '<turn|>\n' -}}
197
+ {%- endif %}
198
+ {#- Pre-scan: find last user message index for reasoning guard -#}
199
+ {%- set ns_turn = namespace(last_user_idx=-1) -%}
200
+ {%- for i in range(loop_messages | length) -%}
201
+ {%- if loop_messages[i]['role'] == 'user' -%}
202
+ {%- set ns_turn.last_user_idx = i -%}
203
+ {%- endif -%}
204
+ {%- endfor -%}
205
+ {#- Loop through messages -#}
206
+ {%- for message in loop_messages -%}
207
+ {%- if message['role'] != 'tool' -%}
208
+ {%- set ns.prev_message_type = None -%}
209
+ {%- set role = 'model' if message['role'] == 'assistant' else message['role'] -%}
210
+ {#- Detect continuation: suppress duplicate <|turn>model when previous non-tool message was also assistant -#}
211
+ {%- set prev_nt = namespace(role=None, found=false) -%}
212
+ {%- if loop.index0 > 0 -%}
213
+ {%- for j in range(loop.index0 - 1, -1, -1) -%}
214
+ {%- if not prev_nt.found -%}
215
+ {%- if loop_messages[j]['role'] != 'tool' -%}
216
+ {%- set prev_nt.role = loop_messages[j]['role'] -%}
217
+ {%- set prev_nt.found = true -%}
218
+ {%- endif -%}
219
+ {%- endif -%}
220
+ {%- endfor -%}
221
+ {%- endif -%}
222
+ {%- set continue_same_model_turn = (role == 'model' and prev_nt.role == 'assistant') -%}
223
+ {%- if not continue_same_model_turn -%}
224
+ {{- '<|turn>' + role + '\n' }}
225
+ {%- endif -%}
226
+ {#- Render reasoning/reasoning_content as thinking channel -#}
227
+ {%- set thinking_text = message.get('reasoning') or message.get('reasoning_content') -%}
228
+ {%- if thinking_text and loop.index0 > ns_turn.last_user_idx and message.get('tool_calls') -%}
229
+ {{- '<|channel>thought\n' + thinking_text + '\n<channel|>' -}}
230
+ {%- endif -%}
231
+ {%- if message['tool_calls'] -%}
232
+ {%- for tool_call in message['tool_calls'] -%}
233
+ {%- set function = tool_call['function'] -%}
234
+ {{- '<|tool_call>call:' + function['name'] + '{' -}}
235
+ {%- if function['arguments'] is mapping -%}
236
+ {%- set ns_args = namespace(found_first=false) -%}
237
+ {%- for key, value in function['arguments'] | dictsort -%}
238
+ {%- if ns_args.found_first %},{% endif -%}
239
+ {%- set ns_args.found_first = true -%}
240
+ {{- key -}}:{{- format_argument(value, escape_keys=False) -}}
241
+ {%- endfor -%}
242
+ {%- elif function['arguments'] is string -%}
243
+ {{- function['arguments'] -}}
244
+ {%- endif -%}
245
+ {{- '}<tool_call|>' -}}
246
+ {%- endfor -%}
247
+ {%- set ns.prev_message_type = 'tool_call' -%}
248
+ {%- endif -%}
249
+ {%- set ns_tr_out = namespace(flag=false) -%}
250
+ {%- if message.get('tool_responses') -%}
251
+ {#- Legacy: tool_responses embedded on the assistant message (Google/Gemma native) -#}
252
+ {%- for tool_response in message['tool_responses'] -%}
253
+ {{- format_tool_response_block(tool_response['name'] | default('unknown'), tool_response['response']) -}}
254
+ {%- set ns_tr_out.flag = true -%}
255
+ {%- set ns.prev_message_type = 'tool_response' -%}
256
+ {%- endfor -%}
257
+ {%- elif message.get('tool_calls') -%}
258
+ {#- OpenAI Chat Completions: forward-scan consecutive role:tool messages -#}
259
+ {%- set ns_tool_scan = namespace(stopped=false) -%}
260
+ {%- for k in range(loop.index0 + 1, loop_messages | length) -%}
261
+ {%- if ns_tool_scan.stopped -%}
262
+ {%- elif loop_messages[k]['role'] != 'tool' -%}
263
+ {%- set ns_tool_scan.stopped = true -%}
264
+ {%- else -%}
265
+ {%- set follow = loop_messages[k] -%}
266
+ {#- Resolve tool_call_id to function name -#}
267
+ {%- set ns_tname = namespace(name=follow.get('name') | default('unknown')) -%}
268
+ {%- for tc in message['tool_calls'] -%}
269
+ {%- if tc.get('id') == follow.get('tool_call_id') -%}
270
+ {%- set ns_tname.name = tc['function']['name'] -%}
271
+ {%- endif -%}
272
+ {%- endfor -%}
273
+ {#- Handle content as string or content-parts array -#}
274
+ {%- set tool_body = follow.get('content') -%}
275
+ {%- if tool_body is string -%}
276
+ {{- format_tool_response_block(ns_tname.name, tool_body) -}}
277
+ {%- elif tool_body is sequence and tool_body is not string -%}
278
+ {%- set ns_txt = namespace(s='') -%}
279
+ {%- for part in tool_body -%}
280
+ {%- if part.get('type') == 'text' -%}
281
+ {%- set ns_txt.s = ns_txt.s + (part.get('text') | default('')) -%}
282
+ {%- endif -%}
283
+ {%- endfor -%}
284
+ {{- format_tool_response_block(ns_tname.name, ns_txt.s) -}}
285
+ {%- else -%}
286
+ {{- format_tool_response_block(ns_tname.name, tool_body) -}}
287
+ {%- endif -%}
288
+ {%- set ns_tr_out.flag = true -%}
289
+ {%- set ns.prev_message_type = 'tool_response' -%}
290
+ {%- endif -%}
291
+ {%- endfor -%}
292
+ {%- endif -%}
293
+ {%- if message['content'] is string -%}
294
+ {%- if role == 'model' -%}
295
+ {{- strip_thinking(message['content']) -}}
296
+ {%- else -%}
297
+ {{- message['content'] | trim -}}
298
+ {%- endif -%}
299
+ {%- elif message['content'] is sequence -%}
300
+ {%- for item in message['content'] -%}
301
+ {%- if item['type'] == 'text' -%}
302
+ {%- if role == 'model' -%}
303
+ {{- strip_thinking(item['text']) -}}
304
+ {%- else -%}
305
+ {{- item['text'] | trim -}}
306
+ {%- endif -%}
307
+ {%- elif item['type'] == 'image' -%}
308
+ {{- '<|image|>' -}}
309
+ {%- set ns.prev_message_type = 'image' -%}
310
+ {%- elif item['type'] == 'audio' -%}
311
+ {{- '<|audio|>' -}}
312
+ {%- set ns.prev_message_type = 'audio' -%}
313
+ {%- elif item['type'] == 'video' -%}
314
+ {{- '<|video|>' -}}
315
+ {%- set ns.prev_message_type = 'video' -%}
316
+ {%- endif -%}
317
+ {%- endfor -%}
318
+ {%- endif -%}
319
+ {%- if ns.prev_message_type == 'tool_call' and not ns_tr_out.flag -%}
320
+ {{- '<|tool_response>' -}}
321
+ {%- elif not (ns_tr_out.flag and not message.get('content')) -%}
322
+ {{- '<turn|>\n' -}}
323
+ {%- endif -%}
324
+ {%- endif -%}
325
+ {%- endfor -%}
326
+ {%- if add_generation_prompt -%}
327
+ {%- if ns.prev_message_type != 'tool_response' and ns.prev_message_type != 'tool_call' -%}
328
+ {{- '<|turn>model\n' -}}
329
+ {%- endif -%}
330
+ {%- endif -%}
gemma4_lora/processor/processor_config.json ADDED
@@ -0,0 +1,290 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "audio_ms_per_token": 40,
3
+ "audio_seq_length": 750,
4
+ "feature_extractor": {
5
+ "audio_ms_per_token": 40,
6
+ "audio_seq_length": 750,
7
+ "dither": 0.0,
8
+ "feature_extractor": {
9
+ "dither": 0.0,
10
+ "feature_extractor_type": "Gemma4AudioFeatureExtractor",
11
+ "feature_size": 128,
12
+ "fft_length": 512,
13
+ "fft_overdrive": false,
14
+ "frame_length": 320,
15
+ "hop_length": 160,
16
+ "input_scale_factor": 1.0,
17
+ "max_frequency": 8000.0,
18
+ "mel_floor": 0.001,
19
+ "min_frequency": 0.0,
20
+ "padding_side": "right",
21
+ "padding_value": 0.0,
22
+ "per_bin_mean": null,
23
+ "per_bin_stddev": null,
24
+ "preemphasis": 0.0,
25
+ "preemphasis_htk_flavor": true,
26
+ "return_attention_mask": true,
27
+ "sampling_rate": 16000
28
+ },
29
+ "feature_extractor_type": "Gemma4AudioFeatureExtractor",
30
+ "feature_size": 128,
31
+ "fft_length": 512,
32
+ "fft_overdrive": false,
33
+ "frame_length": 320,
34
+ "hop_length": 160,
35
+ "image_processor": {
36
+ "do_convert_rgb": true,
37
+ "do_normalize": false,
38
+ "do_rescale": true,
39
+ "do_resize": true,
40
+ "image_mean": [
41
+ 0.0,
42
+ 0.0,
43
+ 0.0
44
+ ],
45
+ "image_processor_type": "Gemma4ImageProcessor",
46
+ "image_seq_length": 280,
47
+ "image_std": [
48
+ 1.0,
49
+ 1.0,
50
+ 1.0
51
+ ],
52
+ "max_soft_tokens": 280,
53
+ "patch_size": 16,
54
+ "pooling_kernel_size": 3,
55
+ "resample": 3,
56
+ "rescale_factor": 0.00392156862745098
57
+ },
58
+ "image_seq_length": 280,
59
+ "input_scale_factor": 1.0,
60
+ "max_frequency": 8000.0,
61
+ "mel_floor": 0.001,
62
+ "min_frequency": 0.0,
63
+ "padding_side": "right",
64
+ "padding_value": 0.0,
65
+ "per_bin_mean": null,
66
+ "per_bin_stddev": null,
67
+ "preemphasis": 0.0,
68
+ "preemphasis_htk_flavor": true,
69
+ "return_attention_mask": true,
70
+ "sampling_rate": 16000,
71
+ "video_processor": {
72
+ "do_convert_rgb": true,
73
+ "do_normalize": true,
74
+ "do_rescale": true,
75
+ "do_resize": true,
76
+ "do_sample_frames": true,
77
+ "image_mean": [
78
+ 0.0,
79
+ 0.0,
80
+ 0.0
81
+ ],
82
+ "image_std": [
83
+ 1.0,
84
+ 1.0,
85
+ 1.0
86
+ ],
87
+ "max_soft_tokens": 70,
88
+ "num_frames": 32,
89
+ "patch_size": 16,
90
+ "pooling_kernel_size": 3,
91
+ "resample": 3,
92
+ "rescale_factor": 0.00392156862745098,
93
+ "return_metadata": false,
94
+ "video_processor_type": "Gemma4VideoProcessor"
95
+ }
96
+ },
97
+ "image_processor": {
98
+ "audio_ms_per_token": 40,
99
+ "audio_seq_length": 750,
100
+ "do_convert_rgb": true,
101
+ "do_normalize": false,
102
+ "do_rescale": true,
103
+ "do_resize": true,
104
+ "feature_extractor": {
105
+ "dither": 0.0,
106
+ "feature_extractor_type": "Gemma4AudioFeatureExtractor",
107
+ "feature_size": 128,
108
+ "fft_length": 512,
109
+ "fft_overdrive": false,
110
+ "frame_length": 320,
111
+ "hop_length": 160,
112
+ "input_scale_factor": 1.0,
113
+ "max_frequency": 8000.0,
114
+ "mel_floor": 0.001,
115
+ "min_frequency": 0.0,
116
+ "padding_side": "right",
117
+ "padding_value": 0.0,
118
+ "per_bin_mean": null,
119
+ "per_bin_stddev": null,
120
+ "preemphasis": 0.0,
121
+ "preemphasis_htk_flavor": true,
122
+ "return_attention_mask": true,
123
+ "sampling_rate": 16000
124
+ },
125
+ "image_mean": [
126
+ 0.0,
127
+ 0.0,
128
+ 0.0
129
+ ],
130
+ "image_processor": {
131
+ "do_convert_rgb": true,
132
+ "do_normalize": false,
133
+ "do_rescale": true,
134
+ "do_resize": true,
135
+ "image_mean": [
136
+ 0.0,
137
+ 0.0,
138
+ 0.0
139
+ ],
140
+ "image_processor_type": "Gemma4ImageProcessor",
141
+ "image_seq_length": 280,
142
+ "image_std": [
143
+ 1.0,
144
+ 1.0,
145
+ 1.0
146
+ ],
147
+ "max_soft_tokens": 280,
148
+ "patch_size": 16,
149
+ "pooling_kernel_size": 3,
150
+ "resample": 3,
151
+ "rescale_factor": 0.00392156862745098
152
+ },
153
+ "image_processor_type": "Gemma4ImageProcessor",
154
+ "image_seq_length": 280,
155
+ "image_std": [
156
+ 1.0,
157
+ 1.0,
158
+ 1.0
159
+ ],
160
+ "max_soft_tokens": 280,
161
+ "patch_size": 16,
162
+ "pooling_kernel_size": 3,
163
+ "resample": 3,
164
+ "rescale_factor": 0.00392156862745098,
165
+ "video_processor": {
166
+ "do_convert_rgb": true,
167
+ "do_normalize": true,
168
+ "do_rescale": true,
169
+ "do_resize": true,
170
+ "do_sample_frames": true,
171
+ "image_mean": [
172
+ 0.0,
173
+ 0.0,
174
+ 0.0
175
+ ],
176
+ "image_std": [
177
+ 1.0,
178
+ 1.0,
179
+ 1.0
180
+ ],
181
+ "max_soft_tokens": 70,
182
+ "num_frames": 32,
183
+ "patch_size": 16,
184
+ "pooling_kernel_size": 3,
185
+ "resample": 3,
186
+ "rescale_factor": 0.00392156862745098,
187
+ "return_metadata": false,
188
+ "video_processor_type": "Gemma4VideoProcessor"
189
+ }
190
+ },
191
+ "image_seq_length": 280,
192
+ "processor_class": "Gemma4Processor",
193
+ "video_processor": {
194
+ "audio_ms_per_token": 40,
195
+ "audio_seq_length": 750,
196
+ "do_convert_rgb": true,
197
+ "do_normalize": true,
198
+ "do_rescale": true,
199
+ "do_resize": true,
200
+ "do_sample_frames": true,
201
+ "feature_extractor": {
202
+ "dither": 0.0,
203
+ "feature_extractor_type": "Gemma4AudioFeatureExtractor",
204
+ "feature_size": 128,
205
+ "fft_length": 512,
206
+ "fft_overdrive": false,
207
+ "frame_length": 320,
208
+ "hop_length": 160,
209
+ "input_scale_factor": 1.0,
210
+ "max_frequency": 8000.0,
211
+ "mel_floor": 0.001,
212
+ "min_frequency": 0.0,
213
+ "padding_side": "right",
214
+ "padding_value": 0.0,
215
+ "per_bin_mean": null,
216
+ "per_bin_stddev": null,
217
+ "preemphasis": 0.0,
218
+ "preemphasis_htk_flavor": true,
219
+ "return_attention_mask": true,
220
+ "sampling_rate": 16000
221
+ },
222
+ "image_mean": [
223
+ 0.0,
224
+ 0.0,
225
+ 0.0
226
+ ],
227
+ "image_processor": {
228
+ "do_convert_rgb": true,
229
+ "do_normalize": false,
230
+ "do_rescale": true,
231
+ "do_resize": true,
232
+ "image_mean": [
233
+ 0.0,
234
+ 0.0,
235
+ 0.0
236
+ ],
237
+ "image_processor_type": "Gemma4ImageProcessor",
238
+ "image_seq_length": 280,
239
+ "image_std": [
240
+ 1.0,
241
+ 1.0,
242
+ 1.0
243
+ ],
244
+ "max_soft_tokens": 280,
245
+ "patch_size": 16,
246
+ "pooling_kernel_size": 3,
247
+ "resample": 3,
248
+ "rescale_factor": 0.00392156862745098
249
+ },
250
+ "image_seq_length": 280,
251
+ "image_std": [
252
+ 1.0,
253
+ 1.0,
254
+ 1.0
255
+ ],
256
+ "max_soft_tokens": 70,
257
+ "num_frames": 32,
258
+ "patch_size": 16,
259
+ "pooling_kernel_size": 3,
260
+ "resample": 3,
261
+ "rescale_factor": 0.00392156862745098,
262
+ "return_metadata": false,
263
+ "video_processor": {
264
+ "do_convert_rgb": true,
265
+ "do_normalize": true,
266
+ "do_rescale": true,
267
+ "do_resize": true,
268
+ "do_sample_frames": true,
269
+ "image_mean": [
270
+ 0.0,
271
+ 0.0,
272
+ 0.0
273
+ ],
274
+ "image_std": [
275
+ 1.0,
276
+ 1.0,
277
+ 1.0
278
+ ],
279
+ "max_soft_tokens": 70,
280
+ "num_frames": 32,
281
+ "patch_size": 16,
282
+ "pooling_kernel_size": 3,
283
+ "resample": 3,
284
+ "rescale_factor": 0.00392156862745098,
285
+ "return_metadata": false,
286
+ "video_processor_type": "Gemma4VideoProcessor"
287
+ },
288
+ "video_processor_type": "Gemma4VideoProcessor"
289
+ }
290
+ }
gemma4_lora/processor/tokenizer.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cc8d3a0ce36466ccc1278bf987df5f71db1719b9ca6b4118264f45cb627bfe0f
3
+ size 32169626
gemma4_lora/processor/tokenizer_config.json ADDED
@@ -0,0 +1,96 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "audio_token": "<|audio|>",
3
+ "backend": "tokenizers",
4
+ "boa_token": "<|audio>",
5
+ "boi_token": "<|image>",
6
+ "bos_token": "<bos>",
7
+ "eoa_token": "<audio|>",
8
+ "eoc_token": "<channel|>",
9
+ "eoi_token": "<image|>",
10
+ "eos_token": "<eos>",
11
+ "eot_token": "<turn|>",
12
+ "escape_token": "<|\"|>",
13
+ "etc_token": "<tool_call|>",
14
+ "etd_token": "<tool|>",
15
+ "etr_token": "<tool_response|>",
16
+ "extra_special_tokens": [
17
+ "<|video|>"
18
+ ],
19
+ "image_token": "<|image|>",
20
+ "is_local": false,
21
+ "local_files_only": false,
22
+ "mask_token": "<mask>",
23
+ "model_max_length": 1000000000000000019884624838656,
24
+ "model_specific_special_tokens": {
25
+ "audio_token": "<|audio|>",
26
+ "boa_token": "<|audio>",
27
+ "boi_token": "<|image>",
28
+ "eoa_token": "<audio|>",
29
+ "eoc_token": "<channel|>",
30
+ "eoi_token": "<image|>",
31
+ "eot_token": "<turn|>",
32
+ "escape_token": "<|\"|>",
33
+ "etc_token": "<tool_call|>",
34
+ "etd_token": "<tool|>",
35
+ "etr_token": "<tool_response|>",
36
+ "image_token": "<|image|>",
37
+ "soc_token": "<|channel>",
38
+ "sot_token": "<|turn>",
39
+ "stc_token": "<|tool_call>",
40
+ "std_token": "<|tool>",
41
+ "str_token": "<|tool_response>",
42
+ "think_token": "<|think|>"
43
+ },
44
+ "pad_token": "<pad>",
45
+ "padding_side": "right",
46
+ "processor_class": "Gemma4Processor",
47
+ "response_schema": {
48
+ "properties": {
49
+ "content": {
50
+ "type": "string"
51
+ },
52
+ "role": {
53
+ "const": "assistant"
54
+ },
55
+ "thinking": {
56
+ "type": "string"
57
+ },
58
+ "tool_calls": {
59
+ "items": {
60
+ "properties": {
61
+ "function": {
62
+ "properties": {
63
+ "arguments": {
64
+ "additionalProperties": {},
65
+ "type": "object",
66
+ "x-parser": "gemma4-tool-call"
67
+ },
68
+ "name": {
69
+ "type": "string"
70
+ }
71
+ },
72
+ "type": "object",
73
+ "x-regex": "call\\:(?P<name>\\w+)(?P<arguments>\\{.*\\})"
74
+ },
75
+ "type": {
76
+ "const": "function"
77
+ }
78
+ },
79
+ "type": "object"
80
+ },
81
+ "type": "array",
82
+ "x-regex-iterator": "<\\|tool_call>(.*?)<tool_call\\|>"
83
+ }
84
+ },
85
+ "type": "object",
86
+ "x-regex": "(\\<\\|channel\\>thought\\n(?P<thinking>.*?)\\<channel\\|\\>)?(?P<tool_calls>\\<\\|tool_call\\>.*\\<tool_call\\|\\>)?(?P<content>(?:(?!\\<turn\\|\\>)(?!\\<\\|tool_response\\>).)+)?(?:\\<turn\\|\\>|\\<\\|tool_response\\>)?"
87
+ },
88
+ "soc_token": "<|channel>",
89
+ "sot_token": "<|turn>",
90
+ "stc_token": "<|tool_call>",
91
+ "std_token": "<|tool>",
92
+ "str_token": "<|tool_response>",
93
+ "think_token": "<|think|>",
94
+ "tokenizer_class": "GemmaTokenizer",
95
+ "unk_token": "<unk>"
96
+ }
gemma4_lora/training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:83fff43465ff39aa286290f3bdba0ee1b978483b463631d8eb11dee70e2790e7
3
+ size 5265
gemma4_lora/training_metadata.json ADDED
@@ -0,0 +1,1433 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "base_model": "coder3101/gemma-4-E2B-it-heretic",
3
+ "training_method": "QLoRA",
4
+ "quantization": "NF4 4-bit",
5
+ "compute_dtype": "float16",
6
+ "lora_dtype": "float32",
7
+ "lora_rank": 16,
8
+ "lora_alpha": 16,
9
+ "lora_dropout": 0.05,
10
+ "target_modules": "all-linear",
11
+ "optimizer": "paged_adamw_8bit",
12
+ "world_size": 2,
13
+ "distributed_method": "DDP",
14
+ "ddp_find_unused_parameters": true,
15
+ "per_device_batch_size": 1,
16
+ "gradient_accumulation": 1,
17
+ "epochs": 30,
18
+ "estimated_optimizer_updates": 120,
19
+ "observed_gradient_checks": 120,
20
+ "trainer_global_step": 120,
21
+ "learning_rate": 0.0001,
22
+ "lr_scheduler_type": "cosine",
23
+ "weight_decay": 0.01,
24
+ "warmup_ratio": 0.03,
25
+ "warmup_steps": 3,
26
+ "max_grad_norm": 0.3,
27
+ "gradient_checkpointing": true,
28
+ "fp16_initial_scale": 2048.0,
29
+ "fp16_final_scale": 2048.0,
30
+ "fp16_min_final_scale": 512.0,
31
+ "max_skipped_optimizer_steps": 2,
32
+ "training_examples": 7,
33
+ "validation_examples": 0,
34
+ "dataset_repository": "codeShare/gemma4-precomputed-dataset",
35
+ "dataset_path": "gemma4_preencoded_dataset",
36
+ "training_input": [
37
+ "input_ids",
38
+ "attention_mask",
39
+ "pixel_values",
40
+ "other_precomputed_processor_features"
41
+ ],
42
+ "training_labels": "precomputed completion-only labels",
43
+ "raw_images_loaded_during_training": false,
44
+ "raw_text_loaded_during_training": false,
45
+ "processor_called_during_training": false,
46
+ "training_loss": 0.5774162989731849,
47
+ "training_metrics": {
48
+ "train_runtime": 403.7248,
49
+ "train_samples_per_second": 0.52,
50
+ "train_steps_per_second": 0.297,
51
+ "total_flos": 3325449084076032.0,
52
+ "train_loss": 0.5774162989731849,
53
+ "epoch": 30.0
54
+ },
55
+ "validation_metrics": {},
56
+ "training_seconds": 405.704416513443,
57
+ "training_seed": 1234,
58
+ "gpu_peak_allocated_gb": 11.1423978805542,
59
+ "gradient_health": {
60
+ "gradient_checks": 120,
61
+ "skipped_optimizer_steps": 0,
62
+ "skipped_step_indices": [],
63
+ "nonfinite_gradient_steps": [],
64
+ "nonfinite_gradient_tensors": 0,
65
+ "no_gradient_steps": [],
66
+ "gradient_norms": [
67
+ 0.2999999967057757,
68
+ 0.29999993283753906,
69
+ 0.2999999882627466,
70
+ 0.29999995549701025,
71
+ 0.2999999056871044,
72
+ 0.29999997674311024,
73
+ 0.29999996019639635,
74
+ 0.2999999331868226,
75
+ 0.2999999190714327,
76
+ 0.29999994453961754,
77
+ 0.29999992767652045,
78
+ 0.2999999171052898,
79
+ 0.299999891168433,
80
+ 0.2999999387562522,
81
+ 0.2999998559474201,
82
+ 0.2999998651608199,
83
+ 0.29999989888361955,
84
+ 0.299999860837114,
85
+ 0.2999998637005364,
86
+ 0.2999999487365302,
87
+ 0.29999991210927746,
88
+ 0.29999983302762295,
89
+ 0.29999986467178447,
90
+ 0.2999999246373012,
91
+ 0.2999998710247092,
92
+ 0.29999993885001613,
93
+ 0.2999998562013249,
94
+ 0.2999997898603423,
95
+ 0.2999998988001732,
96
+ 0.2999997968720858,
97
+ 0.29999981960907923,
98
+ 0.29999979953123646,
99
+ 0.29999993062910396,
100
+ 0.2999998143369396,
101
+ 0.29999988013220563,
102
+ 0.29999991627050243,
103
+ 0.2999998549485208,
104
+ 0.29999983698781824,
105
+ 0.29999988739171235,
106
+ 0.29999984926746825,
107
+ 0.29999978238735703,
108
+ 0.29999989584211356,
109
+ 0.29999983784788814,
110
+ 0.2999997514863681,
111
+ 0.2999997904095978,
112
+ 0.29999987607479484,
113
+ 0.29999983808389075,
114
+ 0.29999982376193984,
115
+ 0.299999843238001,
116
+ 0.2999997380879359,
117
+ 0.29999984200200724,
118
+ 0.2999998571695013,
119
+ 0.29999993043955364,
120
+ 0.2999998548770266,
121
+ 0.29999998275241463,
122
+ 0.2999998385367787,
123
+ 0.29999985670904267,
124
+ 0.2999998650251881,
125
+ 0.29999990770273105,
126
+ 0.2999997907082759,
127
+ 0.29999975991044014,
128
+ 0.2999998386237497,
129
+ 0.29999976701075326,
130
+ 0.2999996384655255,
131
+ 0.2999998653212779,
132
+ 0.29999942298684046,
133
+ 0.29999983928156604,
134
+ 0.2999998335974349,
135
+ 0.2999997630936651,
136
+ 0.29999957808098776,
137
+ 0.2999993324745417,
138
+ 0.29999964148031316,
139
+ 0.2999993150217098,
140
+ 0.2999994021040341,
141
+ 0.29999970079639704,
142
+ 0.2999996212821266,
143
+ 0.29999923384458965,
144
+ 0.2400580492521267,
145
+ 0.2999991859568465,
146
+ 0.29999952391317286,
147
+ 0.29999943933173767,
148
+ 0.23904489591558,
149
+ 0.22082136702898114,
150
+ 0.2999996496006251,
151
+ 0.29999960097840744,
152
+ 0.06084565240294252,
153
+ 0.29999985714989785,
154
+ 0.27001140019986175,
155
+ 0.24196433419393892,
156
+ 0.05825369805852839,
157
+ 0.2999991469929516,
158
+ 0.15705863171793738,
159
+ 0.03828652322329344,
160
+ 0.023200811595227796,
161
+ 0.04129434956180859,
162
+ 0.049138123368485935,
163
+ 0.04487694375693962,
164
+ 0.224603229033112,
165
+ 0.03229273522786361,
166
+ 0.032599237798871604,
167
+ 0.022936590735817202,
168
+ 0.02703242390409454,
169
+ 0.04120622897715978,
170
+ 0.08993883373234765,
171
+ 0.017092950887663435,
172
+ 0.0368967124461106,
173
+ 0.032362015474763244,
174
+ 0.020569942602627754,
175
+ 0.033200306693477066,
176
+ 0.030664452587451013,
177
+ 0.01549019716854724,
178
+ 0.06880451374310592,
179
+ 0.1791207763483352,
180
+ 0.035754383235644914,
181
+ 0.0195178739292786,
182
+ 0.021198802830342003,
183
+ 0.0159683130342503,
184
+ 0.03289778612980872,
185
+ 0.03374332580161473,
186
+ 0.026582601743105054
187
+ ],
188
+ "gradient_tensor_counts": [
189
+ 780,
190
+ 780,
191
+ 780,
192
+ 780,
193
+ 780,
194
+ 780,
195
+ 780,
196
+ 780,
197
+ 780,
198
+ 780,
199
+ 780,
200
+ 780,
201
+ 780,
202
+ 780,
203
+ 780,
204
+ 780,
205
+ 780,
206
+ 780,
207
+ 780,
208
+ 780,
209
+ 780,
210
+ 780,
211
+ 780,
212
+ 780,
213
+ 780,
214
+ 780,
215
+ 780,
216
+ 780,
217
+ 780,
218
+ 780,
219
+ 780,
220
+ 780,
221
+ 780,
222
+ 780,
223
+ 780,
224
+ 780,
225
+ 780,
226
+ 780,
227
+ 780,
228
+ 780,
229
+ 780,
230
+ 780,
231
+ 780,
232
+ 780,
233
+ 780,
234
+ 780,
235
+ 780,
236
+ 780,
237
+ 780,
238
+ 780,
239
+ 780,
240
+ 780,
241
+ 780,
242
+ 780,
243
+ 780,
244
+ 780,
245
+ 780,
246
+ 780,
247
+ 780,
248
+ 780,
249
+ 780,
250
+ 780,
251
+ 780,
252
+ 780,
253
+ 780,
254
+ 780,
255
+ 780,
256
+ 780,
257
+ 780,
258
+ 780,
259
+ 780,
260
+ 780,
261
+ 780,
262
+ 780,
263
+ 780,
264
+ 780,
265
+ 780,
266
+ 780,
267
+ 780,
268
+ 780,
269
+ 780,
270
+ 780,
271
+ 780,
272
+ 780,
273
+ 780,
274
+ 780,
275
+ 780,
276
+ 780,
277
+ 780,
278
+ 780,
279
+ 780,
280
+ 780,
281
+ 780,
282
+ 780,
283
+ 780,
284
+ 780,
285
+ 780,
286
+ 780,
287
+ 780,
288
+ 780,
289
+ 780,
290
+ 780,
291
+ 780,
292
+ 780,
293
+ 780,
294
+ 780,
295
+ 780,
296
+ 780,
297
+ 780,
298
+ 780,
299
+ 780,
300
+ 780,
301
+ 780,
302
+ 780,
303
+ 780,
304
+ 780,
305
+ 780,
306
+ 780,
307
+ 780,
308
+ 780
309
+ ],
310
+ "grad_scaler_values": [
311
+ 2048.0,
312
+ 2048.0,
313
+ 2048.0,
314
+ 2048.0,
315
+ 2048.0,
316
+ 2048.0,
317
+ 2048.0,
318
+ 2048.0,
319
+ 2048.0,
320
+ 2048.0,
321
+ 2048.0,
322
+ 2048.0,
323
+ 2048.0,
324
+ 2048.0,
325
+ 2048.0,
326
+ 2048.0,
327
+ 2048.0,
328
+ 2048.0,
329
+ 2048.0,
330
+ 2048.0,
331
+ 2048.0,
332
+ 2048.0,
333
+ 2048.0,
334
+ 2048.0,
335
+ 2048.0,
336
+ 2048.0,
337
+ 2048.0,
338
+ 2048.0,
339
+ 2048.0,
340
+ 2048.0,
341
+ 2048.0,
342
+ 2048.0,
343
+ 2048.0,
344
+ 2048.0,
345
+ 2048.0,
346
+ 2048.0,
347
+ 2048.0,
348
+ 2048.0,
349
+ 2048.0,
350
+ 2048.0,
351
+ 2048.0,
352
+ 2048.0,
353
+ 2048.0,
354
+ 2048.0,
355
+ 2048.0,
356
+ 2048.0,
357
+ 2048.0,
358
+ 2048.0,
359
+ 2048.0,
360
+ 2048.0,
361
+ 2048.0,
362
+ 2048.0,
363
+ 2048.0,
364
+ 2048.0,
365
+ 2048.0,
366
+ 2048.0,
367
+ 2048.0,
368
+ 2048.0,
369
+ 2048.0,
370
+ 2048.0,
371
+ 2048.0,
372
+ 2048.0,
373
+ 2048.0,
374
+ 2048.0,
375
+ 2048.0,
376
+ 2048.0,
377
+ 2048.0,
378
+ 2048.0,
379
+ 2048.0,
380
+ 2048.0,
381
+ 2048.0,
382
+ 2048.0,
383
+ 2048.0,
384
+ 2048.0,
385
+ 2048.0,
386
+ 2048.0,
387
+ 2048.0,
388
+ 2048.0,
389
+ 2048.0,
390
+ 2048.0,
391
+ 2048.0,
392
+ 2048.0,
393
+ 2048.0,
394
+ 2048.0,
395
+ 2048.0,
396
+ 2048.0,
397
+ 2048.0,
398
+ 2048.0,
399
+ 2048.0,
400
+ 2048.0,
401
+ 2048.0,
402
+ 2048.0,
403
+ 2048.0,
404
+ 2048.0,
405
+ 2048.0,
406
+ 2048.0,
407
+ 2048.0,
408
+ 2048.0,
409
+ 2048.0,
410
+ 2048.0,
411
+ 2048.0,
412
+ 2048.0,
413
+ 2048.0,
414
+ 2048.0,
415
+ 2048.0,
416
+ 2048.0,
417
+ 2048.0,
418
+ 2048.0,
419
+ 2048.0,
420
+ 2048.0,
421
+ 2048.0,
422
+ 2048.0,
423
+ 2048.0,
424
+ 2048.0,
425
+ 2048.0,
426
+ 2048.0,
427
+ 2048.0,
428
+ 2048.0,
429
+ 2048.0,
430
+ 2048.0
431
+ ],
432
+ "loss_history": [
433
+ {
434
+ "global_step": 1,
435
+ "epoch": 0.25,
436
+ "loss": 3.0028076171875,
437
+ "learning_rate": 0.0,
438
+ "grad_norm": 5.735446453094482
439
+ },
440
+ {
441
+ "global_step": 2,
442
+ "epoch": 0.5,
443
+ "loss": 2.5738320350646973,
444
+ "learning_rate": 3.3333333333333335e-05,
445
+ "grad_norm": 4.246273517608643
446
+ },
447
+ {
448
+ "global_step": 3,
449
+ "epoch": 0.75,
450
+ "loss": 3.9848618507385254,
451
+ "learning_rate": 6.666666666666667e-05,
452
+ "grad_norm": 7.595849990844727
453
+ },
454
+ {
455
+ "global_step": 4,
456
+ "epoch": 1.0,
457
+ "loss": 2.5158143043518066,
458
+ "learning_rate": 0.0001,
459
+ "grad_norm": 4.926611423492432
460
+ },
461
+ {
462
+ "global_step": 5,
463
+ "epoch": 1.25,
464
+ "loss": 2.6317954063415527,
465
+ "learning_rate": 9.998197638354428e-05,
466
+ "grad_norm": 3.7285847663879395
467
+ },
468
+ {
469
+ "global_step": 6,
470
+ "epoch": 1.5,
471
+ "loss": 3.391207218170166,
472
+ "learning_rate": 9.992791852820709e-05,
473
+ "grad_norm": 6.960104465484619
474
+ },
475
+ {
476
+ "global_step": 7,
477
+ "epoch": 1.75,
478
+ "loss": 1.7886579036712646,
479
+ "learning_rate": 9.983786540671051e-05,
480
+ "grad_norm": 3.496877431869507
481
+ },
482
+ {
483
+ "global_step": 8,
484
+ "epoch": 2.0,
485
+ "loss": 1.9815987348556519,
486
+ "learning_rate": 9.97118819423714e-05,
487
+ "grad_norm": 2.860832691192627
488
+ },
489
+ {
490
+ "global_step": 9,
491
+ "epoch": 2.25,
492
+ "loss": 1.775098443031311,
493
+ "learning_rate": 9.955005896229543e-05,
494
+ "grad_norm": 3.4511520862579346
495
+ },
496
+ {
497
+ "global_step": 10,
498
+ "epoch": 2.5,
499
+ "loss": 1.6050281524658203,
500
+ "learning_rate": 9.935251313189564e-05,
501
+ "grad_norm": 3.639047384262085
502
+ },
503
+ {
504
+ "global_step": 11,
505
+ "epoch": 2.75,
506
+ "loss": 2.5201776027679443,
507
+ "learning_rate": 9.911938687078324e-05,
508
+ "grad_norm": 3.662109613418579
509
+ },
510
+ {
511
+ "global_step": 12,
512
+ "epoch": 3.0,
513
+ "loss": 1.7487728595733643,
514
+ "learning_rate": 9.885084825009086e-05,
515
+ "grad_norm": 2.736379384994507
516
+ },
517
+ {
518
+ "global_step": 13,
519
+ "epoch": 3.25,
520
+ "loss": 1.685441255569458,
521
+ "learning_rate": 9.85470908713026e-05,
522
+ "grad_norm": 2.4754011631011963
523
+ },
524
+ {
525
+ "global_step": 14,
526
+ "epoch": 3.5,
527
+ "loss": 2.0612027645111084,
528
+ "learning_rate": 9.820833372667812e-05,
529
+ "grad_norm": 2.888793468475342
530
+ },
531
+ {
532
+ "global_step": 15,
533
+ "epoch": 3.75,
534
+ "loss": 1.2462215423583984,
535
+ "learning_rate": 9.783482104137127e-05,
536
+ "grad_norm": 2.124307632446289
537
+ },
538
+ {
539
+ "global_step": 16,
540
+ "epoch": 4.0,
541
+ "loss": 1.5945758819580078,
542
+ "learning_rate": 9.742682209735727e-05,
543
+ "grad_norm": 2.0276503562927246
544
+ },
545
+ {
546
+ "global_step": 17,
547
+ "epoch": 4.25,
548
+ "loss": 1.8966870307922363,
549
+ "learning_rate": 9.698463103929542e-05,
550
+ "grad_norm": 2.2649166584014893
551
+ },
552
+ {
553
+ "global_step": 18,
554
+ "epoch": 4.5,
555
+ "loss": 1.220649242401123,
556
+ "learning_rate": 9.650856666246693e-05,
557
+ "grad_norm": 1.7242375612258911
558
+ },
559
+ {
560
+ "global_step": 19,
561
+ "epoch": 4.75,
562
+ "loss": 1.4429657459259033,
563
+ "learning_rate": 9.599897218294122e-05,
564
+ "grad_norm": 1.889949083328247
565
+ },
566
+ {
567
+ "global_step": 20,
568
+ "epoch": 5.0,
569
+ "loss": 1.791039228439331,
570
+ "learning_rate": 9.54562149901362e-05,
571
+ "grad_norm": 3.159863233566284
572
+ },
573
+ {
574
+ "global_step": 21,
575
+ "epoch": 5.25,
576
+ "loss": 1.2491681575775146,
577
+ "learning_rate": 9.48806863819507e-05,
578
+ "grad_norm": 2.510749340057373
579
+ },
580
+ {
581
+ "global_step": 22,
582
+ "epoch": 5.5,
583
+ "loss": 0.9549908638000488,
584
+ "learning_rate": 9.42728012826605e-05,
585
+ "grad_norm": 1.8189880847930908
586
+ },
587
+ {
588
+ "global_step": 23,
589
+ "epoch": 5.75,
590
+ "loss": 1.671364188194275,
591
+ "learning_rate": 9.363299794378073e-05,
592
+ "grad_norm": 1.8584535121917725
593
+ },
594
+ {
595
+ "global_step": 24,
596
+ "epoch": 6.0,
597
+ "loss": 1.1979198455810547,
598
+ "learning_rate": 9.296173762811085e-05,
599
+ "grad_norm": 2.8647730350494385
600
+ },
601
+ {
602
+ "global_step": 25,
603
+ "epoch": 6.25,
604
+ "loss": 1.0778791904449463,
605
+ "learning_rate": 9.225950427718975e-05,
606
+ "grad_norm": 2.2521347999572754
607
+ },
608
+ {
609
+ "global_step": 26,
610
+ "epoch": 6.5,
611
+ "loss": 0.9626718759536743,
612
+ "learning_rate": 9.152680416240059e-05,
613
+ "grad_norm": 2.9740376472473145
614
+ },
615
+ {
616
+ "global_step": 27,
617
+ "epoch": 6.75,
618
+ "loss": 1.4524807929992676,
619
+ "learning_rate": 9.076416551997721e-05,
620
+ "grad_norm": 1.8301929235458374
621
+ },
622
+ {
623
+ "global_step": 28,
624
+ "epoch": 7.0,
625
+ "loss": 0.9822371006011963,
626
+ "learning_rate": 8.997213817017507e-05,
627
+ "grad_norm": 1.515589952468872
628
+ },
629
+ {
630
+ "global_step": 29,
631
+ "epoch": 7.25,
632
+ "loss": 1.3678052425384521,
633
+ "learning_rate": 8.915129312088112e-05,
634
+ "grad_norm": 2.429225444793701
635
+ },
636
+ {
637
+ "global_step": 30,
638
+ "epoch": 7.5,
639
+ "loss": 0.7263110876083374,
640
+ "learning_rate": 8.83022221559489e-05,
641
+ "grad_norm": 1.299315333366394
642
+ },
643
+ {
644
+ "global_step": 31,
645
+ "epoch": 7.75,
646
+ "loss": 0.8736010789871216,
647
+ "learning_rate": 8.742553740855506e-05,
648
+ "grad_norm": 1.5025458335876465
649
+ },
650
+ {
651
+ "global_step": 32,
652
+ "epoch": 8.0,
653
+ "loss": 0.9786587357521057,
654
+ "learning_rate": 8.652187091988517e-05,
655
+ "grad_norm": 1.3604384660720825
656
+ },
657
+ {
658
+ "global_step": 33,
659
+ "epoch": 8.25,
660
+ "loss": 0.5913520455360413,
661
+ "learning_rate": 8.559187418346703e-05,
662
+ "grad_norm": 3.1408023834228516
663
+ },
664
+ {
665
+ "global_step": 34,
666
+ "epoch": 8.5,
667
+ "loss": 0.8007135391235352,
668
+ "learning_rate": 8.463621767547998e-05,
669
+ "grad_norm": 1.5192880630493164
670
+ },
671
+ {
672
+ "global_step": 35,
673
+ "epoch": 8.75,
674
+ "loss": 0.8744857311248779,
675
+ "learning_rate": 8.365559037137852e-05,
676
+ "grad_norm": 2.1710562705993652
677
+ },
678
+ {
679
+ "global_step": 36,
680
+ "epoch": 9.0,
681
+ "loss": 0.8190163373947144,
682
+ "learning_rate": 8.265069924917925e-05,
683
+ "grad_norm": 3.7557790279388428
684
+ },
685
+ {
686
+ "global_step": 37,
687
+ "epoch": 9.25,
688
+ "loss": 0.7986066341400146,
689
+ "learning_rate": 8.162226877976887e-05,
690
+ "grad_norm": 1.8751171827316284
691
+ },
692
+ {
693
+ "global_step": 38,
694
+ "epoch": 9.5,
695
+ "loss": 0.5472699403762817,
696
+ "learning_rate": 8.057104040460062e-05,
697
+ "grad_norm": 1.435860514640808
698
+ },
699
+ {
700
+ "global_step": 39,
701
+ "epoch": 9.75,
702
+ "loss": 0.728087306022644,
703
+ "learning_rate": 7.949777200115616e-05,
704
+ "grad_norm": 2.4945225715637207
705
+ },
706
+ {
707
+ "global_step": 40,
708
+ "epoch": 10.0,
709
+ "loss": 0.7109437584877014,
710
+ "learning_rate": 7.840323733655778e-05,
711
+ "grad_norm": 1.7653728723526
712
+ },
713
+ {
714
+ "global_step": 41,
715
+ "epoch": 10.25,
716
+ "loss": 0.4470646381378174,
717
+ "learning_rate": 7.728822550972523e-05,
718
+ "grad_norm": 1.2734816074371338
719
+ },
720
+ {
721
+ "global_step": 42,
722
+ "epoch": 10.5,
723
+ "loss": 0.7584569454193115,
724
+ "learning_rate": 7.615354038247888e-05,
725
+ "grad_norm": 2.3084428310394287
726
+ },
727
+ {
728
+ "global_step": 43,
729
+ "epoch": 10.75,
730
+ "loss": 0.46312659978866577,
731
+ "learning_rate": 7.500000000000001e-05,
732
+ "grad_norm": 1.7591501474380493
733
+ },
734
+ {
735
+ "global_step": 44,
736
+ "epoch": 11.0,
737
+ "loss": 0.45575660467147827,
738
+ "learning_rate": 7.38284360010654e-05,
739
+ "grad_norm": 1.154929518699646
740
+ },
741
+ {
742
+ "global_step": 45,
743
+ "epoch": 11.25,
744
+ "loss": 0.4252345860004425,
745
+ "learning_rate": 7.263969301848188e-05,
746
+ "grad_norm": 1.0464447736740112
747
+ },
748
+ {
749
+ "global_step": 46,
750
+ "epoch": 11.5,
751
+ "loss": 0.5287308692932129,
752
+ "learning_rate": 7.143462807015271e-05,
753
+ "grad_norm": 2.1150853633880615
754
+ },
755
+ {
756
+ "global_step": 47,
757
+ "epoch": 11.75,
758
+ "loss": 0.4766055643558502,
759
+ "learning_rate": 7.021410994121525e-05,
760
+ "grad_norm": 2.03830885887146
761
+ },
762
+ {
763
+ "global_step": 48,
764
+ "epoch": 12.0,
765
+ "loss": 0.23271411657333374,
766
+ "learning_rate": 6.897901855769483e-05,
767
+ "grad_norm": 1.4084392786026
768
+ },
769
+ {
770
+ "global_step": 49,
771
+ "epoch": 12.25,
772
+ "loss": 0.35807982087135315,
773
+ "learning_rate": 6.773024435212678e-05,
774
+ "grad_norm": 1.8897265195846558
775
+ },
776
+ {
777
+ "global_step": 50,
778
+ "epoch": 12.5,
779
+ "loss": 0.18392693996429443,
780
+ "learning_rate": 6.646868762160399e-05,
781
+ "grad_norm": 1.135053038597107
782
+ },
783
+ {
784
+ "global_step": 51,
785
+ "epoch": 12.75,
786
+ "loss": 0.4107872247695923,
787
+ "learning_rate": 6.519525787871235e-05,
788
+ "grad_norm": 1.7325575351715088
789
+ },
790
+ {
791
+ "global_step": 52,
792
+ "epoch": 13.0,
793
+ "loss": 0.295409232378006,
794
+ "learning_rate": 6.391087319582264e-05,
795
+ "grad_norm": 1.7553775310516357
796
+ },
797
+ {
798
+ "global_step": 53,
799
+ "epoch": 13.25,
800
+ "loss": 0.24841588735580444,
801
+ "learning_rate": 6.261645954321109e-05,
802
+ "grad_norm": 2.4505655765533447
803
+ },
804
+ {
805
+ "global_step": 54,
806
+ "epoch": 13.5,
807
+ "loss": 0.28180259466171265,
808
+ "learning_rate": 6.131295012148612e-05,
809
+ "grad_norm": 1.7769156694412231
810
+ },
811
+ {
812
+ "global_step": 55,
813
+ "epoch": 13.75,
814
+ "loss": 0.21721918880939484,
815
+ "learning_rate": 6.0001284688802226e-05,
816
+ "grad_norm": 8.174845695495605
817
+ },
818
+ {
819
+ "global_step": 56,
820
+ "epoch": 14.0,
821
+ "loss": 0.18195383250713348,
822
+ "learning_rate": 5.868240888334653e-05,
823
+ "grad_norm": 1.8216534852981567
824
+ },
825
+ {
826
+ "global_step": 57,
827
+ "epoch": 14.25,
828
+ "loss": 0.15293434262275696,
829
+ "learning_rate": 5.735727354158581e-05,
830
+ "grad_norm": 1.528199315071106
831
+ },
832
+ {
833
+ "global_step": 58,
834
+ "epoch": 14.5,
835
+ "loss": 0.18003931641578674,
836
+ "learning_rate": 5.602683401276615e-05,
837
+ "grad_norm": 1.7730704545974731
838
+ },
839
+ {
840
+ "global_step": 59,
841
+ "epoch": 14.75,
842
+ "loss": 0.16167467832565308,
843
+ "learning_rate": 5.469204947015897e-05,
844
+ "grad_norm": 2.5438311100006104
845
+ },
846
+ {
847
+ "global_step": 60,
848
+ "epoch": 15.0,
849
+ "loss": 0.06871137768030167,
850
+ "learning_rate": 5.335388221955012e-05,
851
+ "grad_norm": 1.244670033454895
852
+ },
853
+ {
854
+ "global_step": 61,
855
+ "epoch": 15.25,
856
+ "loss": 0.08273965865373611,
857
+ "learning_rate": 5.201329700547076e-05,
858
+ "grad_norm": 1.158942699432373
859
+ },
860
+ {
861
+ "global_step": 62,
862
+ "epoch": 15.5,
863
+ "loss": 0.1550861895084381,
864
+ "learning_rate": 5.0671260315669875e-05,
865
+ "grad_norm": 1.5921404361724854
866
+ },
867
+ {
868
+ "global_step": 63,
869
+ "epoch": 15.75,
870
+ "loss": 0.07742718607187271,
871
+ "learning_rate": 4.9328739684330137e-05,
872
+ "grad_norm": 1.2498779296875
873
+ },
874
+ {
875
+ "global_step": 64,
876
+ "epoch": 16.0,
877
+ "loss": 0.060180872678756714,
878
+ "learning_rate": 4.798670299452926e-05,
879
+ "grad_norm": 0.8160451650619507
880
+ },
881
+ {
882
+ "global_step": 65,
883
+ "epoch": 16.25,
884
+ "loss": 0.11217842251062393,
885
+ "learning_rate": 4.6646117780449876e-05,
886
+ "grad_norm": 2.351928472518921
887
+ },
888
+ {
889
+ "global_step": 66,
890
+ "epoch": 16.5,
891
+ "loss": 0.029149621725082397,
892
+ "learning_rate": 4.530795052984104e-05,
893
+ "grad_norm": 0.5133053064346313
894
+ },
895
+ {
896
+ "global_step": 67,
897
+ "epoch": 16.75,
898
+ "loss": 0.057295918464660645,
899
+ "learning_rate": 4.397316598723385e-05,
900
+ "grad_norm": 1.4209790229797363
901
+ },
902
+ {
903
+ "global_step": 68,
904
+ "epoch": 17.0,
905
+ "loss": 0.05391449108719826,
906
+ "learning_rate": 4.264272645841419e-05,
907
+ "grad_norm": 1.5596890449523926
908
+ },
909
+ {
910
+ "global_step": 69,
911
+ "epoch": 17.25,
912
+ "loss": 0.07454235851764679,
913
+ "learning_rate": 4.131759111665349e-05,
914
+ "grad_norm": 1.2337915897369385
915
+ },
916
+ {
917
+ "global_step": 70,
918
+ "epoch": 17.5,
919
+ "loss": 0.03401903435587883,
920
+ "learning_rate": 3.9998715311197785e-05,
921
+ "grad_norm": 0.718992292881012
922
+ },
923
+ {
924
+ "global_step": 71,
925
+ "epoch": 17.75,
926
+ "loss": 0.016408829018473625,
927
+ "learning_rate": 3.86870498785139e-05,
928
+ "grad_norm": 0.44305819272994995
929
+ },
930
+ {
931
+ "global_step": 72,
932
+ "epoch": 18.0,
933
+ "loss": 0.03439335897564888,
934
+ "learning_rate": 3.738354045678891e-05,
935
+ "grad_norm": 0.8151929974555969
936
+ },
937
+ {
938
+ "global_step": 73,
939
+ "epoch": 18.25,
940
+ "loss": 0.01341971568763256,
941
+ "learning_rate": 3.608912680417737e-05,
942
+ "grad_norm": 0.44037967920303345
943
+ },
944
+ {
945
+ "global_step": 74,
946
+ "epoch": 18.5,
947
+ "loss": 0.015053657814860344,
948
+ "learning_rate": 3.480474212128766e-05,
949
+ "grad_norm": 0.505387008190155
950
+ },
951
+ {
952
+ "global_step": 75,
953
+ "epoch": 18.75,
954
+ "loss": 0.020888736471533775,
955
+ "learning_rate": 3.3531312378396026e-05,
956
+ "grad_norm": 0.9699237942695618
957
+ },
958
+ {
959
+ "global_step": 76,
960
+ "epoch": 19.0,
961
+ "loss": 0.015053548850119114,
962
+ "learning_rate": 3.226975564787322e-05,
963
+ "grad_norm": 0.7767230868339539
964
+ },
965
+ {
966
+ "global_step": 77,
967
+ "epoch": 19.25,
968
+ "loss": 0.005396610591560602,
969
+ "learning_rate": 3.1020981442305184e-05,
970
+ "grad_norm": 0.40644827485084534
971
+ },
972
+ {
973
+ "global_step": 78,
974
+ "epoch": 19.5,
975
+ "loss": 0.004981599748134613,
976
+ "learning_rate": 2.978589005878476e-05,
977
+ "grad_norm": 0.2400580644607544
978
+ },
979
+ {
980
+ "global_step": 79,
981
+ "epoch": 19.75,
982
+ "loss": 0.004586038179695606,
983
+ "learning_rate": 2.8565371929847284e-05,
984
+ "grad_norm": 0.35911986231803894
985
+ },
986
+ {
987
+ "global_step": 80,
988
+ "epoch": 20.0,
989
+ "loss": 0.013846193440258503,
990
+ "learning_rate": 2.7360306981518146e-05,
991
+ "grad_norm": 0.644167959690094
992
+ },
993
+ {
994
+ "global_step": 81,
995
+ "epoch": 20.25,
996
+ "loss": 0.009072684682905674,
997
+ "learning_rate": 2.6171563998934605e-05,
998
+ "grad_norm": 0.559910237789154
999
+ },
1000
+ {
1001
+ "global_step": 82,
1002
+ "epoch": 20.5,
1003
+ "loss": 0.0030265708919614553,
1004
+ "learning_rate": 2.500000000000001e-05,
1005
+ "grad_norm": 0.2390448898077011
1006
+ },
1007
+ {
1008
+ "global_step": 83,
1009
+ "epoch": 20.75,
1010
+ "loss": 0.0019349269568920135,
1011
+ "learning_rate": 2.3846459617521128e-05,
1012
+ "grad_norm": 0.220821350812912
1013
+ },
1014
+ {
1015
+ "global_step": 84,
1016
+ "epoch": 21.0,
1017
+ "loss": 0.008226916193962097,
1018
+ "learning_rate": 2.2711774490274766e-05,
1019
+ "grad_norm": 0.8569097518920898
1020
+ },
1021
+ {
1022
+ "global_step": 85,
1023
+ "epoch": 21.25,
1024
+ "loss": 0.007080879062414169,
1025
+ "learning_rate": 2.1596762663442218e-05,
1026
+ "grad_norm": 0.7534610629081726
1027
+ },
1028
+ {
1029
+ "global_step": 86,
1030
+ "epoch": 21.5,
1031
+ "loss": 0.0010789937805384398,
1032
+ "learning_rate": 2.050222799884387e-05,
1033
+ "grad_norm": 0.06084565073251724
1034
+ },
1035
+ {
1036
+ "global_step": 87,
1037
+ "epoch": 21.75,
1038
+ "loss": 0.0016643002163618803,
1039
+ "learning_rate": 1.942895959539939e-05,
1040
+ "grad_norm": 1.5307955741882324
1041
+ },
1042
+ {
1043
+ "global_step": 88,
1044
+ "epoch": 22.0,
1045
+ "loss": 0.002636656863614917,
1046
+ "learning_rate": 1.837773122023114e-05,
1047
+ "grad_norm": 0.27001139521598816
1048
+ },
1049
+ {
1050
+ "global_step": 89,
1051
+ "epoch": 22.25,
1052
+ "loss": 0.0008208074723370373,
1053
+ "learning_rate": 1.734930075082076e-05,
1054
+ "grad_norm": 0.24196434020996094
1055
+ },
1056
+ {
1057
+ "global_step": 90,
1058
+ "epoch": 22.5,
1059
+ "loss": 0.001030398067086935,
1060
+ "learning_rate": 1.6344409628621484e-05,
1061
+ "grad_norm": 0.0582536980509758
1062
+ },
1063
+ {
1064
+ "global_step": 91,
1065
+ "epoch": 22.75,
1066
+ "loss": 0.001805206760764122,
1067
+ "learning_rate": 1.536378232452003e-05,
1068
+ "grad_norm": 0.3588670790195465
1069
+ },
1070
+ {
1071
+ "global_step": 92,
1072
+ "epoch": 23.0,
1073
+ "loss": 0.0013909051194787025,
1074
+ "learning_rate": 1.440812581653298e-05,
1075
+ "grad_norm": 0.15705862641334534
1076
+ },
1077
+ {
1078
+ "global_step": 93,
1079
+ "epoch": 23.25,
1080
+ "loss": 0.0007190693868324161,
1081
+ "learning_rate": 1.3478129080114848e-05,
1082
+ "grad_norm": 0.03828652575612068
1083
+ },
1084
+ {
1085
+ "global_step": 94,
1086
+ "epoch": 23.5,
1087
+ "loss": 0.00025069594266824424,
1088
+ "learning_rate": 1.257446259144494e-05,
1089
+ "grad_norm": 0.023200809955596924
1090
+ },
1091
+ {
1092
+ "global_step": 95,
1093
+ "epoch": 23.75,
1094
+ "loss": 0.0011277927551418543,
1095
+ "learning_rate": 1.1697777844051105e-05,
1096
+ "grad_norm": 0.04129434749484062
1097
+ },
1098
+ {
1099
+ "global_step": 96,
1100
+ "epoch": 24.0,
1101
+ "loss": 0.0009472159435972571,
1102
+ "learning_rate": 1.0848706879118892e-05,
1103
+ "grad_norm": 0.04913812503218651
1104
+ },
1105
+ {
1106
+ "global_step": 97,
1107
+ "epoch": 24.25,
1108
+ "loss": 0.0007696544635109603,
1109
+ "learning_rate": 1.0027861829824952e-05,
1110
+ "grad_norm": 0.04487694054841995
1111
+ },
1112
+ {
1113
+ "global_step": 98,
1114
+ "epoch": 24.5,
1115
+ "loss": 0.0009453248931095004,
1116
+ "learning_rate": 9.235834480022787e-06,
1117
+ "grad_norm": 0.22460322082042694
1118
+ },
1119
+ {
1120
+ "global_step": 99,
1121
+ "epoch": 24.75,
1122
+ "loss": 0.000631998700555414,
1123
+ "learning_rate": 8.473195837599418e-06,
1124
+ "grad_norm": 0.03229273483157158
1125
+ },
1126
+ {
1127
+ "global_step": 100,
1128
+ "epoch": 25.0,
1129
+ "loss": 0.0006901748711243272,
1130
+ "learning_rate": 7.740495722810271e-06,
1131
+ "grad_norm": 0.03259923681616783
1132
+ },
1133
+ {
1134
+ "global_step": 101,
1135
+ "epoch": 25.25,
1136
+ "loss": 0.0003148592368233949,
1137
+ "learning_rate": 7.038262371889159e-06,
1138
+ "grad_norm": 0.022936590015888214
1139
+ },
1140
+ {
1141
+ "global_step": 102,
1142
+ "epoch": 25.5,
1143
+ "loss": 0.0004651989438571036,
1144
+ "learning_rate": 6.367002056219284e-06,
1145
+ "grad_norm": 0.02703242376446724
1146
+ },
1147
+ {
1148
+ "global_step": 103,
1149
+ "epoch": 25.75,
1150
+ "loss": 0.0010305196046829224,
1151
+ "learning_rate": 5.727198717339511e-06,
1152
+ "grad_norm": 0.041206229478120804
1153
+ },
1154
+ {
1155
+ "global_step": 104,
1156
+ "epoch": 26.0,
1157
+ "loss": 0.000634815136436373,
1158
+ "learning_rate": 5.1193136180493095e-06,
1159
+ "grad_norm": 0.08993884176015854
1160
+ },
1161
+ {
1162
+ "global_step": 105,
1163
+ "epoch": 26.25,
1164
+ "loss": 0.00019032439740840346,
1165
+ "learning_rate": 4.54378500986381e-06,
1166
+ "grad_norm": 0.017092950642108917
1167
+ },
1168
+ {
1169
+ "global_step": 106,
1170
+ "epoch": 26.5,
1171
+ "loss": 0.0008260362083092332,
1172
+ "learning_rate": 4.001027817058789e-06,
1173
+ "grad_norm": 0.036896709352731705
1174
+ },
1175
+ {
1176
+ "global_step": 107,
1177
+ "epoch": 26.75,
1178
+ "loss": 0.0005347628612071276,
1179
+ "learning_rate": 3.4914333375330898e-06,
1180
+ "grad_norm": 0.032362014055252075
1181
+ },
1182
+ {
1183
+ "global_step": 108,
1184
+ "epoch": 27.0,
1185
+ "loss": 0.0005628281505778432,
1186
+ "learning_rate": 3.0153689607045845e-06,
1187
+ "grad_norm": 0.020569942891597748
1188
+ },
1189
+ {
1190
+ "global_step": 109,
1191
+ "epoch": 27.25,
1192
+ "loss": 0.0006590321427211165,
1193
+ "learning_rate": 2.573177902642726e-06,
1194
+ "grad_norm": 0.033200304955244064
1195
+ },
1196
+ {
1197
+ "global_step": 110,
1198
+ "epoch": 27.5,
1199
+ "loss": 0.0005788324633613229,
1200
+ "learning_rate": 2.1651789586287442e-06,
1201
+ "grad_norm": 0.03066444955766201
1202
+ },
1203
+ {
1204
+ "global_step": 111,
1205
+ "epoch": 27.75,
1206
+ "loss": 0.0002574412210378796,
1207
+ "learning_rate": 1.7916662733218847e-06,
1208
+ "grad_norm": 0.015490197576582432
1209
+ },
1210
+ {
1211
+ "global_step": 112,
1212
+ "epoch": 28.0,
1213
+ "loss": 0.0007605571881867945,
1214
+ "learning_rate": 1.4529091286973995e-06,
1215
+ "grad_norm": 0.06880451738834381
1216
+ },
1217
+ {
1218
+ "global_step": 113,
1219
+ "epoch": 28.25,
1220
+ "loss": 0.0004236309032421559,
1221
+ "learning_rate": 1.1491517499091498e-06,
1222
+ "grad_norm": 0.17912079393863678
1223
+ },
1224
+ {
1225
+ "global_step": 114,
1226
+ "epoch": 28.5,
1227
+ "loss": 0.0008066121954470873,
1228
+ "learning_rate": 8.806131292167618e-07,
1229
+ "grad_norm": 0.035754382610321045
1230
+ },
1231
+ {
1232
+ "global_step": 115,
1233
+ "epoch": 28.75,
1234
+ "loss": 0.000279993808362633,
1235
+ "learning_rate": 6.474868681043578e-07,
1236
+ "grad_norm": 0.019517874345183372
1237
+ },
1238
+ {
1239
+ "global_step": 116,
1240
+ "epoch": 29.0,
1241
+ "loss": 0.0005598465213552117,
1242
+ "learning_rate": 4.4994103770457653e-07,
1243
+ "grad_norm": 0.021198801696300507
1244
+ },
1245
+ {
1246
+ "global_step": 117,
1247
+ "epoch": 29.25,
1248
+ "loss": 0.000262636283878237,
1249
+ "learning_rate": 2.8811805762860576e-07,
1250
+ "grad_norm": 0.015968313440680504
1251
+ },
1252
+ {
1253
+ "global_step": 118,
1254
+ "epoch": 29.5,
1255
+ "loss": 0.000800407025963068,
1256
+ "learning_rate": 1.6213459328950352e-07,
1257
+ "grad_norm": 0.03289778530597687
1258
+ },
1259
+ {
1260
+ "global_step": 119,
1261
+ "epoch": 29.75,
1262
+ "loss": 0.0005571640795096755,
1263
+ "learning_rate": 7.208147179291192e-08,
1264
+ "grad_norm": 0.033743325620889664
1265
+ },
1266
+ {
1267
+ "global_step": 120,
1268
+ "epoch": 30.0,
1269
+ "loss": 0.00039453001227229834,
1270
+ "learning_rate": 1.802361645573125e-08,
1271
+ "grad_norm": 0.02658260241150856
1272
+ },
1273
+ {
1274
+ "global_step": 120,
1275
+ "epoch": 30.0,
1276
+ "loss": null,
1277
+ "learning_rate": null,
1278
+ "grad_norm": null
1279
+ }
1280
+ ]
1281
+ },
1282
+ "gradient_health_passed": true,
1283
+ "numerical_recovery_passed": true,
1284
+ "post_training_audit_status": "passed",
1285
+ "recoverable_fp16_overflows": 0,
1286
+ "recoverable_overflow_indices": [],
1287
+ "adapter_saved_before_health_audit": true,
1288
+ "adapter_weights_finite": true,
1289
+ "adapter_export_eligible": true,
1290
+ "post_training_audit": {
1291
+ "audit_version": 1,
1292
+ "audit_timestamp_unix": 1793918763.25352,
1293
+ "base_model": "coder3101/gemma-4-E2B-it-heretic",
1294
+ "adapter_directory": "/kaggle/working/gemma4_lora/adapter",
1295
+ "gradient_health_passed": true,
1296
+ "numerical_recovery_passed": true,
1297
+ "adapter_weights_finite": true,
1298
+ "adapter_export_eligible": true,
1299
+ "gradient_checks": 120,
1300
+ "skipped_optimizer_steps": 0,
1301
+ "skipped_step_indices": [],
1302
+ "nonfinite_gradient_steps": [],
1303
+ "nonfinite_gradient_tensors": 0,
1304
+ "gradient_norm_statistics": {
1305
+ "count": 120,
1306
+ "minimum": 0.01549019716854724,
1307
+ "median": 0.2999997905589369,
1308
+ "maximum": 0.2999999967057757,
1309
+ "mean": 0.23312759692201965
1310
+ },
1311
+ "grad_scaler_initial": 2048.0,
1312
+ "grad_scaler_final": 2048.0,
1313
+ "grad_scaler_minimum": 512.0,
1314
+ "loss_history_entries": 121,
1315
+ "adapter_tensor_count": 1050,
1316
+ "adapter_parameter_count": 37920768,
1317
+ "nonfinite_adapter_tensors": [],
1318
+ "passed_checks": 22,
1319
+ "failed_checks": 0,
1320
+ "checks": [
1321
+ {
1322
+ "name": "Gradient monitoring executed",
1323
+ "passed": true,
1324
+ "detail": "120 optimizer attempts were monitored."
1325
+ },
1326
+ {
1327
+ "name": "Gradient norm history length",
1328
+ "passed": true,
1329
+ "detail": "120 norms for 120 checks."
1330
+ },
1331
+ {
1332
+ "name": "Gradient tensor-count history length",
1333
+ "passed": true,
1334
+ "detail": "120 entries for 120 checks."
1335
+ },
1336
+ {
1337
+ "name": "GradScaler history length",
1338
+ "passed": true,
1339
+ "detail": "120 entries for 120 checks."
1340
+ },
1341
+ {
1342
+ "name": "Skipped optimizer update limit",
1343
+ "passed": true,
1344
+ "detail": "0 skipped; maximum allowed is 2."
1345
+ },
1346
+ {
1347
+ "name": "Skipped update indices are unique",
1348
+ "passed": true,
1349
+ "detail": "Indices: []"
1350
+ },
1351
+ {
1352
+ "name": "Skipped update indices are in range",
1353
+ "passed": true,
1354
+ "detail": "Valid range: 0 to 119."
1355
+ },
1356
+ {
1357
+ "name": "Non-finite gradient indices are unique",
1358
+ "passed": true,
1359
+ "detail": "Indices: []"
1360
+ },
1361
+ {
1362
+ "name": "Non-finite gradient indices are in range",
1363
+ "passed": true,
1364
+ "detail": "Valid range: 0 to 119."
1365
+ },
1366
+ {
1367
+ "name": "Overflow attempts match skipped optimizer attempts",
1368
+ "passed": true,
1369
+ "detail": "Skipped=[]; non-finite=[]."
1370
+ },
1371
+ {
1372
+ "name": "Non-finite tensor count is consistent",
1373
+ "passed": true,
1374
+ "detail": "0 non-finite gradient tensors."
1375
+ },
1376
+ {
1377
+ "name": "No optimizer attempt lacked all gradients",
1378
+ "passed": true,
1379
+ "detail": "No-gradient attempts: []."
1380
+ },
1381
+ {
1382
+ "name": "No consecutive overflow attempts",
1383
+ "passed": true,
1384
+ "detail": "Overflow indices: []."
1385
+ },
1386
+ {
1387
+ "name": "Final overflow has recovery window",
1388
+ "passed": true,
1389
+ "detail": "No overflow in the final two attempts."
1390
+ },
1391
+ {
1392
+ "name": "Missing gradient norms match overflow attempts",
1393
+ "passed": true,
1394
+ "detail": "Missing norms=[]; overflow attempts=[]."
1395
+ },
1396
+ {
1397
+ "name": "All recorded finite gradient norms are valid",
1398
+ "passed": true,
1399
+ "detail": "120 finite norms; invalid indices=[]."
1400
+ },
1401
+ {
1402
+ "name": "GradScaler history is finite and positive",
1403
+ "passed": true,
1404
+ "detail": "120 recorded values."
1405
+ },
1406
+ {
1407
+ "name": "Initial GradScaler value is valid",
1408
+ "passed": true,
1409
+ "detail": "Initial scale: 2048.0."
1410
+ },
1411
+ {
1412
+ "name": "Final GradScaler value is valid",
1413
+ "passed": true,
1414
+ "detail": "Final scale: 2048.0."
1415
+ },
1416
+ {
1417
+ "name": "Final GradScaler meets minimum",
1418
+ "passed": true,
1419
+ "detail": "Final=2048.0; minimum=512.0."
1420
+ },
1421
+ {
1422
+ "name": "Recorded loss history contains valid values",
1423
+ "passed": true,
1424
+ "detail": "120 finite loss values; invalid entries=[]."
1425
+ },
1426
+ {
1427
+ "name": "Adapter weights contain only finite values",
1428
+ "passed": true,
1429
+ "detail": "1050 tensors; 37,920,768 parameters; 0 non-finite tensors."
1430
+ }
1431
+ ]
1432
+ }
1433
+ }