titanthree commited on
Commit
26307b7
·
verified ·
1 Parent(s): dca87cb

Upload folder using huggingface_hub

Browse files
.gitattributes CHANGED
@@ -40,3 +40,4 @@ checkpoints/checkpoint-2000/tokenizer.json filter=lfs diff=lfs merge=lfs -text
40
  checkpoints/checkpoint-2500/tokenizer.json filter=lfs diff=lfs merge=lfs -text
41
  checkpoints/checkpoint-3000/tokenizer.json filter=lfs diff=lfs merge=lfs -text
42
  checkpoints/checkpoint-3500/tokenizer.json filter=lfs diff=lfs merge=lfs -text
 
 
40
  checkpoints/checkpoint-2500/tokenizer.json filter=lfs diff=lfs merge=lfs -text
41
  checkpoints/checkpoint-3000/tokenizer.json filter=lfs diff=lfs merge=lfs -text
42
  checkpoints/checkpoint-3500/tokenizer.json filter=lfs diff=lfs merge=lfs -text
43
+ checkpoints/checkpoint-4000/tokenizer.json filter=lfs diff=lfs merge=lfs -text
checkpoints/checkpoint-4000/README.md ADDED
@@ -0,0 +1,210 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: unsloth/gemma-3n-E4B-it-unsloth-bnb-4bit
3
+ library_name: peft
4
+ pipeline_tag: text-generation
5
+ tags:
6
+ - base_model:adapter:unsloth/gemma-3n-E4B-it-unsloth-bnb-4bit
7
+ - lora
8
+ - sft
9
+ - transformers
10
+ - trl
11
+ - unsloth
12
+ ---
13
+
14
+ # Model Card for Model ID
15
+
16
+ <!-- Provide a quick summary of what the model is/does. -->
17
+
18
+
19
+
20
+ ## Model Details
21
+
22
+ ### Model Description
23
+
24
+ <!-- Provide a longer summary of what this model is. -->
25
+
26
+
27
+
28
+ - **Developed by:** [More Information Needed]
29
+ - **Funded by [optional]:** [More Information Needed]
30
+ - **Shared by [optional]:** [More Information Needed]
31
+ - **Model type:** [More Information Needed]
32
+ - **Language(s) (NLP):** [More Information Needed]
33
+ - **License:** [More Information Needed]
34
+ - **Finetuned from model [optional]:** [More Information Needed]
35
+
36
+ ### Model Sources [optional]
37
+
38
+ <!-- Provide the basic links for the model. -->
39
+
40
+ - **Repository:** [More Information Needed]
41
+ - **Paper [optional]:** [More Information Needed]
42
+ - **Demo [optional]:** [More Information Needed]
43
+
44
+ ## Uses
45
+
46
+ <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
47
+
48
+ ### Direct Use
49
+
50
+ <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
51
+
52
+ [More Information Needed]
53
+
54
+ ### Downstream Use [optional]
55
+
56
+ <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
57
+
58
+ [More Information Needed]
59
+
60
+ ### Out-of-Scope Use
61
+
62
+ <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
63
+
64
+ [More Information Needed]
65
+
66
+ ## Bias, Risks, and Limitations
67
+
68
+ <!-- This section is meant to convey both technical and sociotechnical limitations. -->
69
+
70
+ [More Information Needed]
71
+
72
+ ### Recommendations
73
+
74
+ <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
75
+
76
+ Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
77
+
78
+ ## How to Get Started with the Model
79
+
80
+ Use the code below to get started with the model.
81
+
82
+ [More Information Needed]
83
+
84
+ ## Training Details
85
+
86
+ ### Training Data
87
+
88
+ <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
89
+
90
+ [More Information Needed]
91
+
92
+ ### Training Procedure
93
+
94
+ <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
95
+
96
+ #### Preprocessing [optional]
97
+
98
+ [More Information Needed]
99
+
100
+
101
+ #### Training Hyperparameters
102
+
103
+ - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
104
+
105
+ #### Speeds, Sizes, Times [optional]
106
+
107
+ <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
108
+
109
+ [More Information Needed]
110
+
111
+ ## Evaluation
112
+
113
+ <!-- This section describes the evaluation protocols and provides the results. -->
114
+
115
+ ### Testing Data, Factors & Metrics
116
+
117
+ #### Testing Data
118
+
119
+ <!-- This should link to a Dataset Card if possible. -->
120
+
121
+ [More Information Needed]
122
+
123
+ #### Factors
124
+
125
+ <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
126
+
127
+ [More Information Needed]
128
+
129
+ #### Metrics
130
+
131
+ <!-- These are the evaluation metrics being used, ideally with a description of why. -->
132
+
133
+ [More Information Needed]
134
+
135
+ ### Results
136
+
137
+ [More Information Needed]
138
+
139
+ #### Summary
140
+
141
+
142
+
143
+ ## Model Examination [optional]
144
+
145
+ <!-- Relevant interpretability work for the model goes here -->
146
+
147
+ [More Information Needed]
148
+
149
+ ## Environmental Impact
150
+
151
+ <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
152
+
153
+ Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
154
+
155
+ - **Hardware Type:** [More Information Needed]
156
+ - **Hours used:** [More Information Needed]
157
+ - **Cloud Provider:** [More Information Needed]
158
+ - **Compute Region:** [More Information Needed]
159
+ - **Carbon Emitted:** [More Information Needed]
160
+
161
+ ## Technical Specifications [optional]
162
+
163
+ ### Model Architecture and Objective
164
+
165
+ [More Information Needed]
166
+
167
+ ### Compute Infrastructure
168
+
169
+ [More Information Needed]
170
+
171
+ #### Hardware
172
+
173
+ [More Information Needed]
174
+
175
+ #### Software
176
+
177
+ [More Information Needed]
178
+
179
+ ## Citation [optional]
180
+
181
+ <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
182
+
183
+ **BibTeX:**
184
+
185
+ [More Information Needed]
186
+
187
+ **APA:**
188
+
189
+ [More Information Needed]
190
+
191
+ ## Glossary [optional]
192
+
193
+ <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
194
+
195
+ [More Information Needed]
196
+
197
+ ## More Information [optional]
198
+
199
+ [More Information Needed]
200
+
201
+ ## Model Card Authors [optional]
202
+
203
+ [More Information Needed]
204
+
205
+ ## Model Card Contact
206
+
207
+ [More Information Needed]
208
+ ### Framework versions
209
+
210
+ - PEFT 0.20.0
checkpoints/checkpoint-4000/adapter_config.json ADDED
@@ -0,0 +1,46 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alora_invocation_tokens": null,
3
+ "alpha_pattern": {},
4
+ "arrow_config": null,
5
+ "auto_mapping": {
6
+ "base_model_class": "Gemma3nForConditionalGeneration",
7
+ "parent_library": "unsloth_compiled_module_gemma3n",
8
+ "unsloth_fixed": true
9
+ },
10
+ "base_model_name_or_path": "unsloth/gemma-3n-E4B-it-unsloth-bnb-4bit",
11
+ "bias": "none",
12
+ "corda_config": null,
13
+ "ensure_weight_tying": false,
14
+ "eva_config": null,
15
+ "exclude_modules": null,
16
+ "fan_in_fan_out": false,
17
+ "inference_mode": true,
18
+ "init_lora_weights": true,
19
+ "layer_replication": null,
20
+ "layers_pattern": null,
21
+ "layers_to_transform": null,
22
+ "loftq_config": {},
23
+ "lora_alpha": 16,
24
+ "lora_bias": false,
25
+ "lora_dropout": 0,
26
+ "lora_ga_config": null,
27
+ "megatron_config": null,
28
+ "megatron_core": "megatron.core",
29
+ "modules_to_save": null,
30
+ "monteclora_config": null,
31
+ "peft_type": "LORA",
32
+ "peft_version": "0.20.0",
33
+ "qalora_group_size": 16,
34
+ "r": 16,
35
+ "rank_pattern": {},
36
+ "revision": null,
37
+ "target_modules": "(?:.*?(?:language|text).*?(?:self_attn|attention|attn|mixer|mlp|feed_forward|ffn|dense|mixer).*?\\.(?:q_proj|k_proj|v_proj|o_proj|gate_proj|up_proj|down_proj))|(?:\\bmodel\\.layers\\.[\\d]{1,}(?!.*\\.experts\\.\\d)\\..*?(?:self_attn|attention|attn|mixer|mlp|feed_forward|ffn|dense|mixer).*?\\.(?:(?:q_proj|k_proj|v_proj|o_proj|gate_proj|up_proj|down_proj)))",
38
+ "target_parameters": null,
39
+ "task_type": "CAUSAL_LM",
40
+ "trainable_token_indices": null,
41
+ "use_bdlora": null,
42
+ "use_dora": false,
43
+ "use_qalora": false,
44
+ "use_rslora": false,
45
+ "velora_config": null
46
+ }
checkpoints/checkpoint-4000/adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:994e06d2c7108a205b733d23834b161b687659bbdba81d04b8913c91d2826bd9
3
+ size 148830640
checkpoints/checkpoint-4000/chat_template.jinja ADDED
@@ -0,0 +1,49 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {{ bos_token }}
2
+ {%- if messages[0]['role'] == 'system' -%}
3
+ {%- if messages[0]['content'] is string -%}
4
+ {%- set first_user_prefix = messages[0]['content'] + '
5
+
6
+ ' -%}
7
+ {%- else -%}
8
+ {%- set first_user_prefix = messages[0]['content'][0]['text'] + '
9
+
10
+ ' -%}
11
+ {%- endif -%}
12
+ {%- set loop_messages = messages[1:] -%}
13
+ {%- else -%}
14
+ {%- set first_user_prefix = "" -%}
15
+ {%- set loop_messages = messages -%}
16
+ {%- endif -%}
17
+ {%- for message in loop_messages -%}
18
+ {%- if (message['role'] == 'user') != (loop.index0 % 2 == 0) -%}
19
+ {{ raise_exception("Conversation roles must alternate user/assistant/user/assistant/...") }}
20
+ {%- endif -%}
21
+ {%- if (message['role'] == 'assistant') -%}
22
+ {%- set role = "model" -%}
23
+ {%- else -%}
24
+ {%- set role = message['role'] -%}
25
+ {%- endif -%}
26
+ {{ '<start_of_turn>' + role + '
27
+ ' + (first_user_prefix if loop.first else "") }}
28
+ {%- if message['content'] is string -%}
29
+ {{ message['content'] | trim }}
30
+ {%- elif message['content'] is iterable -%}
31
+ {%- for item in message['content'] -%}
32
+ {%- if item['type'] == 'audio' -%}
33
+ {{ '<audio_soft_token>' }}
34
+ {%- elif item['type'] == 'image' -%}
35
+ {{ '<image_soft_token>' }}
36
+ {%- elif item['type'] == 'text' -%}
37
+ {{ item['text'] | trim }}
38
+ {%- endif -%}
39
+ {%- endfor -%}
40
+ {%- else -%}
41
+ {{ raise_exception("Invalid content type") }}
42
+ {%- endif -%}
43
+ {{ '<end_of_turn>
44
+ ' }}
45
+ {%- endfor -%}
46
+ {%- if add_generation_prompt -%}
47
+ {{'<start_of_turn>model
48
+ '}}
49
+ {%- endif -%}
checkpoints/checkpoint-4000/optimizer.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9df729e1e25fdf9ca8506d705558d5ca98d73e479dae9f977dfef696318417d8
3
+ size 76054045
checkpoints/checkpoint-4000/processor_config.json ADDED
@@ -0,0 +1,98 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "audio_seq_length": 188,
3
+ "feature_extractor": {
4
+ "crop_size": null,
5
+ "data_format": "channels_first",
6
+ "default_to_square": false,
7
+ "device": null,
8
+ "disable_grouping": null,
9
+ "dither": 0.0,
10
+ "do_center_crop": null,
11
+ "do_convert_rgb": null,
12
+ "do_normalize": false,
13
+ "do_rescale": true,
14
+ "do_resize": true,
15
+ "feature_extractor_type": "Gemma3nAudioFeatureExtractor",
16
+ "feature_size": 128,
17
+ "fft_length": 1024,
18
+ "fft_overdrive": true,
19
+ "frame_length": 512,
20
+ "hop_length": 160,
21
+ "image_mean": [
22
+ 0.5,
23
+ 0.5,
24
+ 0.5
25
+ ],
26
+ "image_processor_type": "SiglipImageProcessorFast",
27
+ "image_seq_length": 256,
28
+ "image_std": [
29
+ 0.5,
30
+ 0.5,
31
+ 0.5
32
+ ],
33
+ "input_data_format": null,
34
+ "input_scale_factor": 1.0,
35
+ "max_frequency": 7600.0,
36
+ "mel_floor": 1e-05,
37
+ "min_frequency": 125.0,
38
+ "padding_side": "right",
39
+ "padding_value": 0.0,
40
+ "per_bin_mean": null,
41
+ "per_bin_stddev": null,
42
+ "preemphasis": 0.97,
43
+ "preemphasis_htk_flavor": true,
44
+ "resample": 2,
45
+ "rescale_factor": 0.00392156862745098,
46
+ "return_attention_mask": true,
47
+ "return_tensors": null,
48
+ "sampling_rate": 16000,
49
+ "size": {
50
+ "height": 768,
51
+ "width": 768
52
+ }
53
+ },
54
+ "image_processor": {
55
+ "data_format": "channels_first",
56
+ "default_to_square": false,
57
+ "dither": 0.0,
58
+ "do_convert_rgb": null,
59
+ "do_normalize": false,
60
+ "do_rescale": true,
61
+ "do_resize": true,
62
+ "feature_size": 128,
63
+ "fft_length": 1024,
64
+ "fft_overdrive": true,
65
+ "frame_length": 512,
66
+ "hop_length": 160,
67
+ "image_mean": [
68
+ 0.5,
69
+ 0.5,
70
+ 0.5
71
+ ],
72
+ "image_processor_type": "SiglipImageProcessor",
73
+ "image_seq_length": 256,
74
+ "image_std": [
75
+ 0.5,
76
+ 0.5,
77
+ 0.5
78
+ ],
79
+ "input_scale_factor": 1.0,
80
+ "max_frequency": 7600.0,
81
+ "mel_floor": 1e-05,
82
+ "min_frequency": 125.0,
83
+ "padding_side": "left",
84
+ "padding_value": 0.0,
85
+ "preemphasis": 0.97,
86
+ "preemphasis_htk_flavor": true,
87
+ "resample": 2,
88
+ "rescale_factor": 0.00392156862745098,
89
+ "return_attention_mask": true,
90
+ "sampling_rate": 16000,
91
+ "size": {
92
+ "height": 768,
93
+ "width": 768
94
+ }
95
+ },
96
+ "image_seq_length": 256,
97
+ "processor_class": "Gemma3nProcessor"
98
+ }
checkpoints/checkpoint-4000/rng_state.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f1d565802a8e26c4e8a31328752b7a7fdc186d9401aa008e65697d0ad8c22e33
3
+ size 14645
checkpoints/checkpoint-4000/scheduler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f367d5333b0c38cb1d36f1c84961f5338b935ffb858199dc0f323d36a826edcc
3
+ size 1465
checkpoints/checkpoint-4000/tokenizer.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6aa40b46d55d1fa5f1c0b218de3a695942c620858ba34aa94e2aa705efc6f071
3
+ size 33442552
checkpoints/checkpoint-4000/tokenizer.model ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ea5f0cc48abfbfc04d14562270a32e02149a3e7035f368cc5a462786f4a59961
3
+ size 4696020
checkpoints/checkpoint-4000/tokenizer_config.json ADDED
The diff for this file is too large to render. See raw diff
 
checkpoints/checkpoint-4000/trainer_state.json ADDED
@@ -0,0 +1,2834 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_global_step": null,
3
+ "best_metric": null,
4
+ "best_model_checkpoint": null,
5
+ "epoch": 0.13419666521286946,
6
+ "eval_steps": 500,
7
+ "global_step": 4000,
8
+ "is_hyper_param_search": false,
9
+ "is_local_process_zero": true,
10
+ "is_world_process_zero": true,
11
+ "log_history": [
12
+ {
13
+ "epoch": 0.00033549166303217364,
14
+ "grad_norm": 3.6124119758605957,
15
+ "learning_rate": 2.011173184357542e-06,
16
+ "loss": 2.9620466232299805,
17
+ "step": 10
18
+ },
19
+ {
20
+ "epoch": 0.0006709833260643473,
21
+ "grad_norm": 0.8180733323097229,
22
+ "learning_rate": 4.245810055865922e-06,
23
+ "loss": 2.254135513305664,
24
+ "step": 20
25
+ },
26
+ {
27
+ "epoch": 0.0010064749890965209,
28
+ "grad_norm": 1.8416152000427246,
29
+ "learning_rate": 6.4804469273743025e-06,
30
+ "loss": 2.308770179748535,
31
+ "step": 30
32
+ },
33
+ {
34
+ "epoch": 0.0013419666521286946,
35
+ "grad_norm": 1.6899038553237915,
36
+ "learning_rate": 8.715083798882683e-06,
37
+ "loss": 2.354534149169922,
38
+ "step": 40
39
+ },
40
+ {
41
+ "epoch": 0.0016774583151608683,
42
+ "grad_norm": 1.0635626316070557,
43
+ "learning_rate": 1.0949720670391061e-05,
44
+ "loss": 1.9643959045410155,
45
+ "step": 50
46
+ },
47
+ {
48
+ "epoch": 0.0020129499781930417,
49
+ "grad_norm": 1.7144725322723389,
50
+ "learning_rate": 1.3184357541899443e-05,
51
+ "loss": 2.196229934692383,
52
+ "step": 60
53
+ },
54
+ {
55
+ "epoch": 0.0023484416412252154,
56
+ "grad_norm": 2.0055930614471436,
57
+ "learning_rate": 1.5418994413407823e-05,
58
+ "loss": 1.9873594284057616,
59
+ "step": 70
60
+ },
61
+ {
62
+ "epoch": 0.002683933304257389,
63
+ "grad_norm": 1.3090643882751465,
64
+ "learning_rate": 1.7653631284916203e-05,
65
+ "loss": 1.7420753479003905,
66
+ "step": 80
67
+ },
68
+ {
69
+ "epoch": 0.003019424967289563,
70
+ "grad_norm": 0.6108576655387878,
71
+ "learning_rate": 1.9888268156424583e-05,
72
+ "loss": 1.8236627578735352,
73
+ "step": 90
74
+ },
75
+ {
76
+ "epoch": 0.0033549166303217365,
77
+ "grad_norm": 0.6574695706367493,
78
+ "learning_rate": 2.212290502793296e-05,
79
+ "loss": 1.7359764099121093,
80
+ "step": 100
81
+ },
82
+ {
83
+ "epoch": 0.0036904082933539102,
84
+ "grad_norm": 0.8230994939804077,
85
+ "learning_rate": 2.4357541899441343e-05,
86
+ "loss": 1.5833864212036133,
87
+ "step": 110
88
+ },
89
+ {
90
+ "epoch": 0.0040258999563860835,
91
+ "grad_norm": 0.9543447494506836,
92
+ "learning_rate": 2.6592178770949723e-05,
93
+ "loss": 1.51551513671875,
94
+ "step": 120
95
+ },
96
+ {
97
+ "epoch": 0.004361391619418257,
98
+ "grad_norm": 0.9667573571205139,
99
+ "learning_rate": 2.88268156424581e-05,
100
+ "loss": 1.4613374710083007,
101
+ "step": 130
102
+ },
103
+ {
104
+ "epoch": 0.004696883282450431,
105
+ "grad_norm": 1.9220093488693237,
106
+ "learning_rate": 3.106145251396648e-05,
107
+ "loss": 1.5392029762268067,
108
+ "step": 140
109
+ },
110
+ {
111
+ "epoch": 0.005032374945482605,
112
+ "grad_norm": 0.7170810699462891,
113
+ "learning_rate": 3.329608938547486e-05,
114
+ "loss": 1.5076921463012696,
115
+ "step": 150
116
+ },
117
+ {
118
+ "epoch": 0.005367866608514778,
119
+ "grad_norm": 0.8078820705413818,
120
+ "learning_rate": 3.553072625698324e-05,
121
+ "loss": 1.7222694396972655,
122
+ "step": 160
123
+ },
124
+ {
125
+ "epoch": 0.005703358271546952,
126
+ "grad_norm": 0.809903085231781,
127
+ "learning_rate": 3.776536312849162e-05,
128
+ "loss": 1.8752283096313476,
129
+ "step": 170
130
+ },
131
+ {
132
+ "epoch": 0.006038849934579126,
133
+ "grad_norm": 0.7261088490486145,
134
+ "learning_rate": 4e-05,
135
+ "loss": 1.3927306175231933,
136
+ "step": 180
137
+ },
138
+ {
139
+ "epoch": 0.006374341597611299,
140
+ "grad_norm": 1.5483784675598145,
141
+ "learning_rate": 4.223463687150838e-05,
142
+ "loss": 1.4321404457092286,
143
+ "step": 190
144
+ },
145
+ {
146
+ "epoch": 0.006709833260643473,
147
+ "grad_norm": 0.7559241056442261,
148
+ "learning_rate": 4.4469273743016757e-05,
149
+ "loss": 1.325460147857666,
150
+ "step": 200
151
+ },
152
+ {
153
+ "epoch": 0.007045324923675647,
154
+ "grad_norm": 1.563735008239746,
155
+ "learning_rate": 4.670391061452514e-05,
156
+ "loss": 1.372646427154541,
157
+ "step": 210
158
+ },
159
+ {
160
+ "epoch": 0.0073808165867078205,
161
+ "grad_norm": 0.752616822719574,
162
+ "learning_rate": 4.8938547486033523e-05,
163
+ "loss": 1.1150728225708009,
164
+ "step": 220
165
+ },
166
+ {
167
+ "epoch": 0.007716308249739994,
168
+ "grad_norm": 0.8065235614776611,
169
+ "learning_rate": 5.11731843575419e-05,
170
+ "loss": 1.1660142898559571,
171
+ "step": 230
172
+ },
173
+ {
174
+ "epoch": 0.008051799912772167,
175
+ "grad_norm": 0.9588292241096497,
176
+ "learning_rate": 5.3407821229050284e-05,
177
+ "loss": 1.1108501434326172,
178
+ "step": 240
179
+ },
180
+ {
181
+ "epoch": 0.008387291575804342,
182
+ "grad_norm": 0.8693965077400208,
183
+ "learning_rate": 5.564245810055866e-05,
184
+ "loss": 1.4471724510192872,
185
+ "step": 250
186
+ },
187
+ {
188
+ "epoch": 0.008722783238836514,
189
+ "grad_norm": 2.0879530906677246,
190
+ "learning_rate": 5.787709497206704e-05,
191
+ "loss": 1.2001781463623047,
192
+ "step": 260
193
+ },
194
+ {
195
+ "epoch": 0.009058274901868689,
196
+ "grad_norm": 0.8522306084632874,
197
+ "learning_rate": 6.011173184357543e-05,
198
+ "loss": 1.363035297393799,
199
+ "step": 270
200
+ },
201
+ {
202
+ "epoch": 0.009393766564900862,
203
+ "grad_norm": 1.0152027606964111,
204
+ "learning_rate": 6.23463687150838e-05,
205
+ "loss": 1.2856069564819337,
206
+ "step": 280
207
+ },
208
+ {
209
+ "epoch": 0.009729258227933036,
210
+ "grad_norm": NaN,
211
+ "learning_rate": 6.458100558659218e-05,
212
+ "loss": 2.133226013183594,
213
+ "step": 290
214
+ },
215
+ {
216
+ "epoch": 0.01006474989096521,
217
+ "grad_norm": NaN,
218
+ "learning_rate": 6.681564245810056e-05,
219
+ "loss": 5.05937385559082,
220
+ "step": 300
221
+ },
222
+ {
223
+ "epoch": 0.010400241553997384,
224
+ "grad_norm": NaN,
225
+ "learning_rate": 6.905027932960895e-05,
226
+ "loss": 4.477165985107422,
227
+ "step": 310
228
+ },
229
+ {
230
+ "epoch": 0.010735733217029557,
231
+ "grad_norm": 3.5620813369750977,
232
+ "learning_rate": 7.128491620111732e-05,
233
+ "loss": 2.7552057266235352,
234
+ "step": 320
235
+ },
236
+ {
237
+ "epoch": 0.011071224880061731,
238
+ "grad_norm": NaN,
239
+ "learning_rate": 7.35195530726257e-05,
240
+ "loss": 2.6008926391601563,
241
+ "step": 330
242
+ },
243
+ {
244
+ "epoch": 0.011406716543093904,
245
+ "grad_norm": NaN,
246
+ "learning_rate": 7.575418994413408e-05,
247
+ "loss": 13.484027099609374,
248
+ "step": 340
249
+ },
250
+ {
251
+ "epoch": 0.011742208206126079,
252
+ "grad_norm": NaN,
253
+ "learning_rate": 7.798882681564245e-05,
254
+ "loss": 17.163206481933592,
255
+ "step": 350
256
+ },
257
+ {
258
+ "epoch": 0.012077699869158251,
259
+ "grad_norm": NaN,
260
+ "learning_rate": 8.022346368715084e-05,
261
+ "loss": 8.245307922363281,
262
+ "step": 360
263
+ },
264
+ {
265
+ "epoch": 0.012413191532190426,
266
+ "grad_norm": NaN,
267
+ "learning_rate": 8.245810055865922e-05,
268
+ "loss": 8.971550750732423,
269
+ "step": 370
270
+ },
271
+ {
272
+ "epoch": 0.012748683195222599,
273
+ "grad_norm": NaN,
274
+ "learning_rate": 8.46927374301676e-05,
275
+ "loss": 8.085272979736327,
276
+ "step": 380
277
+ },
278
+ {
279
+ "epoch": 0.013084174858254772,
280
+ "grad_norm": NaN,
281
+ "learning_rate": 8.692737430167597e-05,
282
+ "loss": 2.434987258911133,
283
+ "step": 390
284
+ },
285
+ {
286
+ "epoch": 0.013419666521286946,
287
+ "grad_norm": NaN,
288
+ "learning_rate": 8.916201117318435e-05,
289
+ "loss": 17.522901916503905,
290
+ "step": 400
291
+ },
292
+ {
293
+ "epoch": 0.013755158184319119,
294
+ "grad_norm": NaN,
295
+ "learning_rate": 9.139664804469274e-05,
296
+ "loss": 9.243440246582031,
297
+ "step": 410
298
+ },
299
+ {
300
+ "epoch": 0.014090649847351294,
301
+ "grad_norm": NaN,
302
+ "learning_rate": 9.363128491620112e-05,
303
+ "loss": 5.988158798217773,
304
+ "step": 420
305
+ },
306
+ {
307
+ "epoch": 0.014426141510383466,
308
+ "grad_norm": NaN,
309
+ "learning_rate": 9.58659217877095e-05,
310
+ "loss": 9.994813537597656,
311
+ "step": 430
312
+ },
313
+ {
314
+ "epoch": 0.014761633173415641,
315
+ "grad_norm": NaN,
316
+ "learning_rate": 9.810055865921788e-05,
317
+ "loss": 3.905954360961914,
318
+ "step": 440
319
+ },
320
+ {
321
+ "epoch": 0.015097124836447814,
322
+ "grad_norm": NaN,
323
+ "learning_rate": 0.00010033519553072626,
324
+ "loss": 4.835385513305664,
325
+ "step": 450
326
+ },
327
+ {
328
+ "epoch": 0.015432616499479988,
329
+ "grad_norm": NaN,
330
+ "learning_rate": 0.00010256983240223464,
331
+ "loss": 2.324123764038086,
332
+ "step": 460
333
+ },
334
+ {
335
+ "epoch": 0.015768108162512163,
336
+ "grad_norm": NaN,
337
+ "learning_rate": 0.00010480446927374301,
338
+ "loss": 9.202623748779297,
339
+ "step": 470
340
+ },
341
+ {
342
+ "epoch": 0.016103599825544334,
343
+ "grad_norm": NaN,
344
+ "learning_rate": 0.0001070391061452514,
345
+ "loss": 13.691998291015626,
346
+ "step": 480
347
+ },
348
+ {
349
+ "epoch": 0.01643909148857651,
350
+ "grad_norm": NaN,
351
+ "learning_rate": 0.0001092737430167598,
352
+ "loss": 10.514191436767579,
353
+ "step": 490
354
+ },
355
+ {
356
+ "epoch": 0.016774583151608683,
357
+ "grad_norm": NaN,
358
+ "learning_rate": 0.00011150837988826817,
359
+ "loss": 6.2412151336669925,
360
+ "step": 500
361
+ },
362
+ {
363
+ "epoch": 0.017110074814640858,
364
+ "grad_norm": NaN,
365
+ "learning_rate": 0.00011374301675977655,
366
+ "loss": 2.3255378723144533,
367
+ "step": 510
368
+ },
369
+ {
370
+ "epoch": 0.01744556647767303,
371
+ "grad_norm": NaN,
372
+ "learning_rate": 0.00011597765363128493,
373
+ "loss": 2.0067920684814453,
374
+ "step": 520
375
+ },
376
+ {
377
+ "epoch": 0.017781058140705203,
378
+ "grad_norm": NaN,
379
+ "learning_rate": 0.0001182122905027933,
380
+ "loss": 60.400030517578124,
381
+ "step": 530
382
+ },
383
+ {
384
+ "epoch": 0.018116549803737378,
385
+ "grad_norm": NaN,
386
+ "learning_rate": 0.00012044692737430168,
387
+ "loss": 40.79320373535156,
388
+ "step": 540
389
+ },
390
+ {
391
+ "epoch": 0.018452041466769552,
392
+ "grad_norm": NaN,
393
+ "learning_rate": 0.00012268156424581004,
394
+ "loss": 55.823065185546874,
395
+ "step": 550
396
+ },
397
+ {
398
+ "epoch": 0.018787533129801724,
399
+ "grad_norm": NaN,
400
+ "learning_rate": 0.00012491620111731845,
401
+ "loss": 6.039075469970703,
402
+ "step": 560
403
+ },
404
+ {
405
+ "epoch": 0.019123024792833898,
406
+ "grad_norm": NaN,
407
+ "learning_rate": 0.00012715083798882682,
408
+ "loss": 127.98883056640625,
409
+ "step": 570
410
+ },
411
+ {
412
+ "epoch": 0.019458516455866073,
413
+ "grad_norm": NaN,
414
+ "learning_rate": 0.0001293854748603352,
415
+ "loss": 7.4943595886230465,
416
+ "step": 580
417
+ },
418
+ {
419
+ "epoch": 0.019794008118898244,
420
+ "grad_norm": NaN,
421
+ "learning_rate": 0.00013162011173184358,
422
+ "loss": 87.36739501953124,
423
+ "step": 590
424
+ },
425
+ {
426
+ "epoch": 0.02012949978193042,
427
+ "grad_norm": NaN,
428
+ "learning_rate": 0.00013385474860335195,
429
+ "loss": 12.410279846191406,
430
+ "step": 600
431
+ },
432
+ {
433
+ "epoch": 0.020464991444962593,
434
+ "grad_norm": NaN,
435
+ "learning_rate": 0.00013608938547486036,
436
+ "loss": 40.074404907226565,
437
+ "step": 610
438
+ },
439
+ {
440
+ "epoch": 0.020800483107994767,
441
+ "grad_norm": NaN,
442
+ "learning_rate": 0.00013832402234636873,
443
+ "loss": 17.221456909179686,
444
+ "step": 620
445
+ },
446
+ {
447
+ "epoch": 0.02113597477102694,
448
+ "grad_norm": NaN,
449
+ "learning_rate": 0.0001405586592178771,
450
+ "loss": 38.3494140625,
451
+ "step": 630
452
+ },
453
+ {
454
+ "epoch": 0.021471466434059113,
455
+ "grad_norm": NaN,
456
+ "learning_rate": 0.00014279329608938549,
457
+ "loss": 5.291226577758789,
458
+ "step": 640
459
+ },
460
+ {
461
+ "epoch": 0.021806958097091288,
462
+ "grad_norm": NaN,
463
+ "learning_rate": 0.00014502793296089386,
464
+ "loss": 6.93237533569336,
465
+ "step": 650
466
+ },
467
+ {
468
+ "epoch": 0.022142449760123462,
469
+ "grad_norm": NaN,
470
+ "learning_rate": 0.00014726256983240224,
471
+ "loss": 5.8972515106201175,
472
+ "step": 660
473
+ },
474
+ {
475
+ "epoch": 0.022477941423155633,
476
+ "grad_norm": NaN,
477
+ "learning_rate": 0.00014949720670391064,
478
+ "loss": 40.401693725585936,
479
+ "step": 670
480
+ },
481
+ {
482
+ "epoch": 0.022813433086187808,
483
+ "grad_norm": NaN,
484
+ "learning_rate": 0.00015173184357541902,
485
+ "loss": 9.994586181640624,
486
+ "step": 680
487
+ },
488
+ {
489
+ "epoch": 0.023148924749219982,
490
+ "grad_norm": NaN,
491
+ "learning_rate": 0.0001539664804469274,
492
+ "loss": 1.4739145278930663,
493
+ "step": 690
494
+ },
495
+ {
496
+ "epoch": 0.023484416412252157,
497
+ "grad_norm": NaN,
498
+ "learning_rate": 0.00015620111731843577,
499
+ "loss": 6.606394958496094,
500
+ "step": 700
501
+ },
502
+ {
503
+ "epoch": 0.023819908075284328,
504
+ "grad_norm": NaN,
505
+ "learning_rate": 0.00015843575418994415,
506
+ "loss": 26.84160461425781,
507
+ "step": 710
508
+ },
509
+ {
510
+ "epoch": 0.024155399738316503,
511
+ "grad_norm": NaN,
512
+ "learning_rate": 0.00016067039106145253,
513
+ "loss": 7.266958618164063,
514
+ "step": 720
515
+ },
516
+ {
517
+ "epoch": 0.024490891401348677,
518
+ "grad_norm": NaN,
519
+ "learning_rate": 0.0001629050279329609,
520
+ "loss": 0.5109624862670898,
521
+ "step": 730
522
+ },
523
+ {
524
+ "epoch": 0.024826383064380852,
525
+ "grad_norm": NaN,
526
+ "learning_rate": 0.00016513966480446928,
527
+ "loss": 9.11468734741211,
528
+ "step": 740
529
+ },
530
+ {
531
+ "epoch": 0.025161874727413023,
532
+ "grad_norm": NaN,
533
+ "learning_rate": 0.00016737430167597766,
534
+ "loss": 4.641835021972656,
535
+ "step": 750
536
+ },
537
+ {
538
+ "epoch": 0.025497366390445197,
539
+ "grad_norm": NaN,
540
+ "learning_rate": 0.00016960893854748606,
541
+ "loss": 13.905538940429688,
542
+ "step": 760
543
+ },
544
+ {
545
+ "epoch": 0.025832858053477372,
546
+ "grad_norm": NaN,
547
+ "learning_rate": 0.00017184357541899444,
548
+ "loss": 51.2526611328125,
549
+ "step": 770
550
+ },
551
+ {
552
+ "epoch": 0.026168349716509543,
553
+ "grad_norm": NaN,
554
+ "learning_rate": 0.0001740782122905028,
555
+ "loss": 44.471441650390624,
556
+ "step": 780
557
+ },
558
+ {
559
+ "epoch": 0.026503841379541718,
560
+ "grad_norm": NaN,
561
+ "learning_rate": 0.0001763128491620112,
562
+ "loss": 19.767466735839843,
563
+ "step": 790
564
+ },
565
+ {
566
+ "epoch": 0.026839333042573892,
567
+ "grad_norm": NaN,
568
+ "learning_rate": 0.00017854748603351957,
569
+ "loss": 59.8121826171875,
570
+ "step": 800
571
+ },
572
+ {
573
+ "epoch": 0.027174824705606067,
574
+ "grad_norm": NaN,
575
+ "learning_rate": 0.00018078212290502794,
576
+ "loss": 9.256180572509766,
577
+ "step": 810
578
+ },
579
+ {
580
+ "epoch": 0.027510316368638238,
581
+ "grad_norm": NaN,
582
+ "learning_rate": 0.00018301675977653632,
583
+ "loss": 8.733799743652344,
584
+ "step": 820
585
+ },
586
+ {
587
+ "epoch": 0.027845808031670412,
588
+ "grad_norm": NaN,
589
+ "learning_rate": 0.0001852513966480447,
590
+ "loss": 32.57587585449219,
591
+ "step": 830
592
+ },
593
+ {
594
+ "epoch": 0.028181299694702587,
595
+ "grad_norm": NaN,
596
+ "learning_rate": 0.00018748603351955307,
597
+ "loss": 17.352330017089844,
598
+ "step": 840
599
+ },
600
+ {
601
+ "epoch": 0.02851679135773476,
602
+ "grad_norm": NaN,
603
+ "learning_rate": 0.00018972067039106148,
604
+ "loss": 16.099330139160156,
605
+ "step": 850
606
+ },
607
+ {
608
+ "epoch": 0.028852283020766933,
609
+ "grad_norm": NaN,
610
+ "learning_rate": 0.00019195530726256985,
611
+ "loss": 0.0,
612
+ "step": 860
613
+ },
614
+ {
615
+ "epoch": 0.029187774683799107,
616
+ "grad_norm": NaN,
617
+ "learning_rate": 0.00019418994413407823,
618
+ "loss": 0.08051624894142151,
619
+ "step": 870
620
+ },
621
+ {
622
+ "epoch": 0.029523266346831282,
623
+ "grad_norm": NaN,
624
+ "learning_rate": 0.0001964245810055866,
625
+ "loss": 0.06978185772895813,
626
+ "step": 880
627
+ },
628
+ {
629
+ "epoch": 0.029858758009863456,
630
+ "grad_norm": NaN,
631
+ "learning_rate": 0.00019865921787709498,
632
+ "loss": 1.4936031341552733,
633
+ "step": 890
634
+ },
635
+ {
636
+ "epoch": 0.030194249672895628,
637
+ "grad_norm": NaN,
638
+ "learning_rate": 0.00019997232982844494,
639
+ "loss": 4.048176193237305,
640
+ "step": 900
641
+ },
642
+ {
643
+ "epoch": 0.030529741335927802,
644
+ "grad_norm": NaN,
645
+ "learning_rate": 0.0001999031543995573,
646
+ "loss": 0.108348548412323,
647
+ "step": 910
648
+ },
649
+ {
650
+ "epoch": 0.030865232998959977,
651
+ "grad_norm": NaN,
652
+ "learning_rate": 0.00019983397897066962,
653
+ "loss": 1.5282353401184081,
654
+ "step": 920
655
+ },
656
+ {
657
+ "epoch": 0.03120072466199215,
658
+ "grad_norm": NaN,
659
+ "learning_rate": 0.00019976480354178198,
660
+ "loss": 0.0,
661
+ "step": 930
662
+ },
663
+ {
664
+ "epoch": 0.031536216325024326,
665
+ "grad_norm": NaN,
666
+ "learning_rate": 0.0001996956281128943,
667
+ "loss": 251.467236328125,
668
+ "step": 940
669
+ },
670
+ {
671
+ "epoch": 0.0318717079880565,
672
+ "grad_norm": NaN,
673
+ "learning_rate": 0.00019962645268400666,
674
+ "loss": 82.037890625,
675
+ "step": 950
676
+ },
677
+ {
678
+ "epoch": 0.03220719965108867,
679
+ "grad_norm": NaN,
680
+ "learning_rate": 0.00019955727725511898,
681
+ "loss": 0.0,
682
+ "step": 960
683
+ },
684
+ {
685
+ "epoch": 0.032542691314120846,
686
+ "grad_norm": NaN,
687
+ "learning_rate": 0.0001994881018262313,
688
+ "loss": 0.0,
689
+ "step": 970
690
+ },
691
+ {
692
+ "epoch": 0.03287818297715302,
693
+ "grad_norm": NaN,
694
+ "learning_rate": 0.00019941892639734366,
695
+ "loss": 0.3229848861694336,
696
+ "step": 980
697
+ },
698
+ {
699
+ "epoch": 0.03321367464018519,
700
+ "grad_norm": NaN,
701
+ "learning_rate": 0.00019934975096845602,
702
+ "loss": 4.599651336669922,
703
+ "step": 990
704
+ },
705
+ {
706
+ "epoch": 0.033549166303217366,
707
+ "grad_norm": NaN,
708
+ "learning_rate": 0.00019928057553956837,
709
+ "loss": 2.3171388626098635,
710
+ "step": 1000
711
+ },
712
+ {
713
+ "epoch": 0.03388465796624954,
714
+ "grad_norm": NaN,
715
+ "learning_rate": 0.0001992114001106807,
716
+ "loss": 44.60954895019531,
717
+ "step": 1010
718
+ },
719
+ {
720
+ "epoch": 0.034220149629281715,
721
+ "grad_norm": NaN,
722
+ "learning_rate": 0.00019914222468179305,
723
+ "loss": 1.6406478881835938,
724
+ "step": 1020
725
+ },
726
+ {
727
+ "epoch": 0.034555641292313886,
728
+ "grad_norm": NaN,
729
+ "learning_rate": 0.00019907304925290538,
730
+ "loss": 1.0902439117431642,
731
+ "step": 1030
732
+ },
733
+ {
734
+ "epoch": 0.03489113295534606,
735
+ "grad_norm": NaN,
736
+ "learning_rate": 0.00019900387382401773,
737
+ "loss": 9.517413330078124,
738
+ "step": 1040
739
+ },
740
+ {
741
+ "epoch": 0.035226624618378236,
742
+ "grad_norm": NaN,
743
+ "learning_rate": 0.00019893469839513006,
744
+ "loss": 91.74754028320312,
745
+ "step": 1050
746
+ },
747
+ {
748
+ "epoch": 0.03556211628141041,
749
+ "grad_norm": NaN,
750
+ "learning_rate": 0.0001988655229662424,
751
+ "loss": 161.57200927734374,
752
+ "step": 1060
753
+ },
754
+ {
755
+ "epoch": 0.03589760794444258,
756
+ "grad_norm": NaN,
757
+ "learning_rate": 0.00019879634753735474,
758
+ "loss": 1.9749420166015625,
759
+ "step": 1070
760
+ },
761
+ {
762
+ "epoch": 0.036233099607474756,
763
+ "grad_norm": NaN,
764
+ "learning_rate": 0.00019872717210846707,
765
+ "loss": 0.03762787282466888,
766
+ "step": 1080
767
+ },
768
+ {
769
+ "epoch": 0.03656859127050693,
770
+ "grad_norm": NaN,
771
+ "learning_rate": 0.00019865799667957942,
772
+ "loss": 38.657388305664064,
773
+ "step": 1090
774
+ },
775
+ {
776
+ "epoch": 0.036904082933539105,
777
+ "grad_norm": NaN,
778
+ "learning_rate": 0.00019858882125069175,
779
+ "loss": 5.60150375366211,
780
+ "step": 1100
781
+ },
782
+ {
783
+ "epoch": 0.037239574596571276,
784
+ "grad_norm": NaN,
785
+ "learning_rate": 0.0001985196458218041,
786
+ "loss": 13.85815887451172,
787
+ "step": 1110
788
+ },
789
+ {
790
+ "epoch": 0.03757506625960345,
791
+ "grad_norm": NaN,
792
+ "learning_rate": 0.00019845047039291643,
793
+ "loss": 38.748410034179685,
794
+ "step": 1120
795
+ },
796
+ {
797
+ "epoch": 0.037910557922635625,
798
+ "grad_norm": NaN,
799
+ "learning_rate": 0.00019838129496402878,
800
+ "loss": 65.22340698242188,
801
+ "step": 1130
802
+ },
803
+ {
804
+ "epoch": 0.038246049585667796,
805
+ "grad_norm": NaN,
806
+ "learning_rate": 0.00019831211953514114,
807
+ "loss": 0.8500317573547364,
808
+ "step": 1140
809
+ },
810
+ {
811
+ "epoch": 0.03858154124869997,
812
+ "grad_norm": NaN,
813
+ "learning_rate": 0.00019824294410625347,
814
+ "loss": 143.4577880859375,
815
+ "step": 1150
816
+ },
817
+ {
818
+ "epoch": 0.038917032911732145,
819
+ "grad_norm": NaN,
820
+ "learning_rate": 0.00019817376867736582,
821
+ "loss": 0.38392589092254636,
822
+ "step": 1160
823
+ },
824
+ {
825
+ "epoch": 0.039252524574764316,
826
+ "grad_norm": NaN,
827
+ "learning_rate": 0.00019810459324847815,
828
+ "loss": 1.582177734375,
829
+ "step": 1170
830
+ },
831
+ {
832
+ "epoch": 0.03958801623779649,
833
+ "grad_norm": NaN,
834
+ "learning_rate": 0.0001980354178195905,
835
+ "loss": 0.924138355255127,
836
+ "step": 1180
837
+ },
838
+ {
839
+ "epoch": 0.039923507900828666,
840
+ "grad_norm": NaN,
841
+ "learning_rate": 0.00019796624239070283,
842
+ "loss": 0.0,
843
+ "step": 1190
844
+ },
845
+ {
846
+ "epoch": 0.04025899956386084,
847
+ "grad_norm": NaN,
848
+ "learning_rate": 0.00019789706696181518,
849
+ "loss": 0.15006412267684938,
850
+ "step": 1200
851
+ },
852
+ {
853
+ "epoch": 0.040594491226893015,
854
+ "grad_norm": NaN,
855
+ "learning_rate": 0.0001978278915329275,
856
+ "loss": 0.0,
857
+ "step": 1210
858
+ },
859
+ {
860
+ "epoch": 0.040929982889925186,
861
+ "grad_norm": NaN,
862
+ "learning_rate": 0.00019775871610403986,
863
+ "loss": 0.032241082191467284,
864
+ "step": 1220
865
+ },
866
+ {
867
+ "epoch": 0.04126547455295736,
868
+ "grad_norm": NaN,
869
+ "learning_rate": 0.0001976895406751522,
870
+ "loss": 0.8006097793579101,
871
+ "step": 1230
872
+ },
873
+ {
874
+ "epoch": 0.041600966215989535,
875
+ "grad_norm": NaN,
876
+ "learning_rate": 0.00019762036524626452,
877
+ "loss": 3.051334571838379,
878
+ "step": 1240
879
+ },
880
+ {
881
+ "epoch": 0.041936457879021706,
882
+ "grad_norm": NaN,
883
+ "learning_rate": 0.00019755118981737687,
884
+ "loss": 34.96420593261719,
885
+ "step": 1250
886
+ },
887
+ {
888
+ "epoch": 0.04227194954205388,
889
+ "grad_norm": NaN,
890
+ "learning_rate": 0.00019748201438848922,
891
+ "loss": 6.979457092285156,
892
+ "step": 1260
893
+ },
894
+ {
895
+ "epoch": 0.042607441205086055,
896
+ "grad_norm": NaN,
897
+ "learning_rate": 0.00019741283895960158,
898
+ "loss": 0.0935477614402771,
899
+ "step": 1270
900
+ },
901
+ {
902
+ "epoch": 0.042942932868118226,
903
+ "grad_norm": NaN,
904
+ "learning_rate": 0.0001973436635307139,
905
+ "loss": 0.0,
906
+ "step": 1280
907
+ },
908
+ {
909
+ "epoch": 0.043278424531150404,
910
+ "grad_norm": NaN,
911
+ "learning_rate": 0.00019727448810182626,
912
+ "loss": 1.5331109046936036,
913
+ "step": 1290
914
+ },
915
+ {
916
+ "epoch": 0.043613916194182575,
917
+ "grad_norm": NaN,
918
+ "learning_rate": 0.00019720531267293859,
919
+ "loss": 0.0,
920
+ "step": 1300
921
+ },
922
+ {
923
+ "epoch": 0.043949407857214746,
924
+ "grad_norm": NaN,
925
+ "learning_rate": 0.00019713613724405094,
926
+ "loss": 0.690937614440918,
927
+ "step": 1310
928
+ },
929
+ {
930
+ "epoch": 0.044284899520246924,
931
+ "grad_norm": NaN,
932
+ "learning_rate": 0.00019706696181516327,
933
+ "loss": 8.563715362548828,
934
+ "step": 1320
935
+ },
936
+ {
937
+ "epoch": 0.044620391183279096,
938
+ "grad_norm": NaN,
939
+ "learning_rate": 0.0001969977863862756,
940
+ "loss": 4.255525207519531,
941
+ "step": 1330
942
+ },
943
+ {
944
+ "epoch": 0.04495588284631127,
945
+ "grad_norm": NaN,
946
+ "learning_rate": 0.00019692861095738795,
947
+ "loss": 81.31243286132812,
948
+ "step": 1340
949
+ },
950
+ {
951
+ "epoch": 0.045291374509343445,
952
+ "grad_norm": NaN,
953
+ "learning_rate": 0.00019685943552850027,
954
+ "loss": 0.7501660823822022,
955
+ "step": 1350
956
+ },
957
+ {
958
+ "epoch": 0.045626866172375616,
959
+ "grad_norm": NaN,
960
+ "learning_rate": 0.00019679026009961263,
961
+ "loss": 2.5992816925048827,
962
+ "step": 1360
963
+ },
964
+ {
965
+ "epoch": 0.04596235783540779,
966
+ "grad_norm": NaN,
967
+ "learning_rate": 0.00019672108467072496,
968
+ "loss": 5.559530639648438,
969
+ "step": 1370
970
+ },
971
+ {
972
+ "epoch": 0.046297849498439965,
973
+ "grad_norm": NaN,
974
+ "learning_rate": 0.0001966519092418373,
975
+ "loss": 1.6652551651000977,
976
+ "step": 1380
977
+ },
978
+ {
979
+ "epoch": 0.046633341161472136,
980
+ "grad_norm": NaN,
981
+ "learning_rate": 0.00019658273381294964,
982
+ "loss": 0.2407153844833374,
983
+ "step": 1390
984
+ },
985
+ {
986
+ "epoch": 0.046968832824504314,
987
+ "grad_norm": NaN,
988
+ "learning_rate": 0.000196513558384062,
989
+ "loss": 26.583578491210936,
990
+ "step": 1400
991
+ },
992
+ {
993
+ "epoch": 0.047304324487536485,
994
+ "grad_norm": NaN,
995
+ "learning_rate": 0.00019644438295517434,
996
+ "loss": 7.983282470703125,
997
+ "step": 1410
998
+ },
999
+ {
1000
+ "epoch": 0.047639816150568656,
1001
+ "grad_norm": NaN,
1002
+ "learning_rate": 0.00019637520752628667,
1003
+ "loss": 0.07818608880043029,
1004
+ "step": 1420
1005
+ },
1006
+ {
1007
+ "epoch": 0.047975307813600834,
1008
+ "grad_norm": NaN,
1009
+ "learning_rate": 0.00019630603209739903,
1010
+ "loss": 1.7374027252197266,
1011
+ "step": 1430
1012
+ },
1013
+ {
1014
+ "epoch": 0.048310799476633005,
1015
+ "grad_norm": NaN,
1016
+ "learning_rate": 0.00019623685666851135,
1017
+ "loss": 0.5990512847900391,
1018
+ "step": 1440
1019
+ },
1020
+ {
1021
+ "epoch": 0.048646291139665176,
1022
+ "grad_norm": NaN,
1023
+ "learning_rate": 0.0001961676812396237,
1024
+ "loss": 234.4718994140625,
1025
+ "step": 1450
1026
+ },
1027
+ {
1028
+ "epoch": 0.048981782802697355,
1029
+ "grad_norm": NaN,
1030
+ "learning_rate": 0.00019609850581073603,
1031
+ "loss": 9.876602935791016,
1032
+ "step": 1460
1033
+ },
1034
+ {
1035
+ "epoch": 0.049317274465729526,
1036
+ "grad_norm": NaN,
1037
+ "learning_rate": 0.0001960293303818484,
1038
+ "loss": 0.0,
1039
+ "step": 1470
1040
+ },
1041
+ {
1042
+ "epoch": 0.049652766128761704,
1043
+ "grad_norm": NaN,
1044
+ "learning_rate": 0.00019596015495296071,
1045
+ "loss": 0.11305934190750122,
1046
+ "step": 1480
1047
+ },
1048
+ {
1049
+ "epoch": 0.049988257791793875,
1050
+ "grad_norm": NaN,
1051
+ "learning_rate": 0.00019589097952407304,
1052
+ "loss": 2.2805992126464845,
1053
+ "step": 1490
1054
+ },
1055
+ {
1056
+ "epoch": 0.050323749454826046,
1057
+ "grad_norm": NaN,
1058
+ "learning_rate": 0.0001958218040951854,
1059
+ "loss": 2.134381103515625,
1060
+ "step": 1500
1061
+ },
1062
+ {
1063
+ "epoch": 0.050659241117858224,
1064
+ "grad_norm": NaN,
1065
+ "learning_rate": 0.00019575262866629772,
1066
+ "loss": 0.33266749382019045,
1067
+ "step": 1510
1068
+ },
1069
+ {
1070
+ "epoch": 0.050994732780890395,
1071
+ "grad_norm": NaN,
1072
+ "learning_rate": 0.00019568345323741008,
1073
+ "loss": 301.7185791015625,
1074
+ "step": 1520
1075
+ },
1076
+ {
1077
+ "epoch": 0.051330224443922566,
1078
+ "grad_norm": NaN,
1079
+ "learning_rate": 0.00019561427780852243,
1080
+ "loss": 158.175146484375,
1081
+ "step": 1530
1082
+ },
1083
+ {
1084
+ "epoch": 0.051665716106954744,
1085
+ "grad_norm": NaN,
1086
+ "learning_rate": 0.00019554510237963476,
1087
+ "loss": 0.3448620319366455,
1088
+ "step": 1540
1089
+ },
1090
+ {
1091
+ "epoch": 0.052001207769986915,
1092
+ "grad_norm": NaN,
1093
+ "learning_rate": 0.0001954759269507471,
1094
+ "loss": 0.07021299600601197,
1095
+ "step": 1550
1096
+ },
1097
+ {
1098
+ "epoch": 0.052336699433019086,
1099
+ "grad_norm": NaN,
1100
+ "learning_rate": 0.00019540675152185947,
1101
+ "loss": 2.3782209396362304,
1102
+ "step": 1560
1103
+ },
1104
+ {
1105
+ "epoch": 0.052672191096051264,
1106
+ "grad_norm": NaN,
1107
+ "learning_rate": 0.0001953375760929718,
1108
+ "loss": 33.55213623046875,
1109
+ "step": 1570
1110
+ },
1111
+ {
1112
+ "epoch": 0.053007682759083435,
1113
+ "grad_norm": NaN,
1114
+ "learning_rate": 0.00019526840066408412,
1115
+ "loss": 431.45224609375,
1116
+ "step": 1580
1117
+ },
1118
+ {
1119
+ "epoch": 0.05334317442211561,
1120
+ "grad_norm": NaN,
1121
+ "learning_rate": 0.00019519922523519647,
1122
+ "loss": 45.88771362304688,
1123
+ "step": 1590
1124
+ },
1125
+ {
1126
+ "epoch": 0.053678666085147785,
1127
+ "grad_norm": NaN,
1128
+ "learning_rate": 0.0001951300498063088,
1129
+ "loss": 27.831903076171876,
1130
+ "step": 1600
1131
+ },
1132
+ {
1133
+ "epoch": 0.054014157748179956,
1134
+ "grad_norm": NaN,
1135
+ "learning_rate": 0.00019506087437742115,
1136
+ "loss": 1.7415506362915039,
1137
+ "step": 1610
1138
+ },
1139
+ {
1140
+ "epoch": 0.054349649411212134,
1141
+ "grad_norm": NaN,
1142
+ "learning_rate": 0.00019499169894853348,
1143
+ "loss": 0.0,
1144
+ "step": 1620
1145
+ },
1146
+ {
1147
+ "epoch": 0.054685141074244305,
1148
+ "grad_norm": NaN,
1149
+ "learning_rate": 0.00019492252351964584,
1150
+ "loss": 23.286405944824217,
1151
+ "step": 1630
1152
+ },
1153
+ {
1154
+ "epoch": 0.055020632737276476,
1155
+ "grad_norm": NaN,
1156
+ "learning_rate": 0.00019485334809075816,
1157
+ "loss": 0.0,
1158
+ "step": 1640
1159
+ },
1160
+ {
1161
+ "epoch": 0.055356124400308654,
1162
+ "grad_norm": NaN,
1163
+ "learning_rate": 0.00019478417266187052,
1164
+ "loss": 42.256170654296874,
1165
+ "step": 1650
1166
+ },
1167
+ {
1168
+ "epoch": 0.055691616063340825,
1169
+ "grad_norm": NaN,
1170
+ "learning_rate": 0.00019471499723298284,
1171
+ "loss": 0.09685954451560974,
1172
+ "step": 1660
1173
+ },
1174
+ {
1175
+ "epoch": 0.056027107726373,
1176
+ "grad_norm": NaN,
1177
+ "learning_rate": 0.0001946458218040952,
1178
+ "loss": 30.406533813476564,
1179
+ "step": 1670
1180
+ },
1181
+ {
1182
+ "epoch": 0.056362599389405174,
1183
+ "grad_norm": NaN,
1184
+ "learning_rate": 0.00019457664637520755,
1185
+ "loss": 6.177334594726562,
1186
+ "step": 1680
1187
+ },
1188
+ {
1189
+ "epoch": 0.056698091052437345,
1190
+ "grad_norm": NaN,
1191
+ "learning_rate": 0.00019450747094631988,
1192
+ "loss": 1.717080307006836,
1193
+ "step": 1690
1194
+ },
1195
+ {
1196
+ "epoch": 0.05703358271546952,
1197
+ "grad_norm": NaN,
1198
+ "learning_rate": 0.00019443829551743223,
1199
+ "loss": 16.319992065429688,
1200
+ "step": 1700
1201
+ },
1202
+ {
1203
+ "epoch": 0.057369074378501694,
1204
+ "grad_norm": NaN,
1205
+ "learning_rate": 0.00019436912008854456,
1206
+ "loss": 1.2412324905395509,
1207
+ "step": 1710
1208
+ },
1209
+ {
1210
+ "epoch": 0.057704566041533865,
1211
+ "grad_norm": NaN,
1212
+ "learning_rate": 0.0001942999446596569,
1213
+ "loss": 0.0,
1214
+ "step": 1720
1215
+ },
1216
+ {
1217
+ "epoch": 0.05804005770456604,
1218
+ "grad_norm": NaN,
1219
+ "learning_rate": 0.00019423076923076924,
1220
+ "loss": 16.714073181152344,
1221
+ "step": 1730
1222
+ },
1223
+ {
1224
+ "epoch": 0.058375549367598215,
1225
+ "grad_norm": NaN,
1226
+ "learning_rate": 0.0001941615938018816,
1227
+ "loss": 4.676777648925781,
1228
+ "step": 1740
1229
+ },
1230
+ {
1231
+ "epoch": 0.058711041030630386,
1232
+ "grad_norm": NaN,
1233
+ "learning_rate": 0.00019409241837299392,
1234
+ "loss": 0.1248945951461792,
1235
+ "step": 1750
1236
+ },
1237
+ {
1238
+ "epoch": 0.059046532693662564,
1239
+ "grad_norm": NaN,
1240
+ "learning_rate": 0.00019402324294410625,
1241
+ "loss": 0.0,
1242
+ "step": 1760
1243
+ },
1244
+ {
1245
+ "epoch": 0.059382024356694735,
1246
+ "grad_norm": NaN,
1247
+ "learning_rate": 0.0001939540675152186,
1248
+ "loss": 1.0300786972045899,
1249
+ "step": 1770
1250
+ },
1251
+ {
1252
+ "epoch": 0.05971751601972691,
1253
+ "grad_norm": NaN,
1254
+ "learning_rate": 0.00019388489208633093,
1255
+ "loss": 159.32357177734374,
1256
+ "step": 1780
1257
+ },
1258
+ {
1259
+ "epoch": 0.060053007682759084,
1260
+ "grad_norm": NaN,
1261
+ "learning_rate": 0.00019381571665744328,
1262
+ "loss": 0.1012919545173645,
1263
+ "step": 1790
1264
+ },
1265
+ {
1266
+ "epoch": 0.060388499345791255,
1267
+ "grad_norm": NaN,
1268
+ "learning_rate": 0.0001937465412285556,
1269
+ "loss": 1.9244558334350585,
1270
+ "step": 1800
1271
+ },
1272
+ {
1273
+ "epoch": 0.06072399100882343,
1274
+ "grad_norm": NaN,
1275
+ "learning_rate": 0.00019367736579966796,
1276
+ "loss": 18.52349853515625,
1277
+ "step": 1810
1278
+ },
1279
+ {
1280
+ "epoch": 0.061059482671855604,
1281
+ "grad_norm": NaN,
1282
+ "learning_rate": 0.00019360819037078032,
1283
+ "loss": 0.1813188076019287,
1284
+ "step": 1820
1285
+ },
1286
+ {
1287
+ "epoch": 0.061394974334887775,
1288
+ "grad_norm": NaN,
1289
+ "learning_rate": 0.00019353901494189264,
1290
+ "loss": 0.1522122859954834,
1291
+ "step": 1830
1292
+ },
1293
+ {
1294
+ "epoch": 0.06173046599791995,
1295
+ "grad_norm": NaN,
1296
+ "learning_rate": 0.000193469839513005,
1297
+ "loss": 8.574814605712891,
1298
+ "step": 1840
1299
+ },
1300
+ {
1301
+ "epoch": 0.062065957660952124,
1302
+ "grad_norm": NaN,
1303
+ "learning_rate": 0.00019340066408411733,
1304
+ "loss": 93.5122314453125,
1305
+ "step": 1850
1306
+ },
1307
+ {
1308
+ "epoch": 0.0624014493239843,
1309
+ "grad_norm": NaN,
1310
+ "learning_rate": 0.00019333148865522968,
1311
+ "loss": 2.2591997146606446,
1312
+ "step": 1860
1313
+ },
1314
+ {
1315
+ "epoch": 0.06273694098701647,
1316
+ "grad_norm": NaN,
1317
+ "learning_rate": 0.000193262313226342,
1318
+ "loss": 0.40584664344787597,
1319
+ "step": 1870
1320
+ },
1321
+ {
1322
+ "epoch": 0.06307243265004865,
1323
+ "grad_norm": NaN,
1324
+ "learning_rate": 0.00019319313779745436,
1325
+ "loss": 113.88292236328125,
1326
+ "step": 1880
1327
+ },
1328
+ {
1329
+ "epoch": 0.06340792431308082,
1330
+ "grad_norm": NaN,
1331
+ "learning_rate": 0.0001931239623685667,
1332
+ "loss": 9.488602447509766,
1333
+ "step": 1890
1334
+ },
1335
+ {
1336
+ "epoch": 0.063743415976113,
1337
+ "grad_norm": NaN,
1338
+ "learning_rate": 0.00019305478693967904,
1339
+ "loss": 0.36140780448913573,
1340
+ "step": 1900
1341
+ },
1342
+ {
1343
+ "epoch": 0.06407890763914517,
1344
+ "grad_norm": NaN,
1345
+ "learning_rate": 0.00019298561151079137,
1346
+ "loss": 0.5006577014923096,
1347
+ "step": 1910
1348
+ },
1349
+ {
1350
+ "epoch": 0.06441439930217734,
1351
+ "grad_norm": NaN,
1352
+ "learning_rate": 0.0001929164360819037,
1353
+ "loss": 5.155010604858399,
1354
+ "step": 1920
1355
+ },
1356
+ {
1357
+ "epoch": 0.06474989096520951,
1358
+ "grad_norm": NaN,
1359
+ "learning_rate": 0.00019284726065301605,
1360
+ "loss": 2.3952762603759767,
1361
+ "step": 1930
1362
+ },
1363
+ {
1364
+ "epoch": 0.06508538262824169,
1365
+ "grad_norm": NaN,
1366
+ "learning_rate": 0.0001927780852241284,
1367
+ "loss": 142.20489501953125,
1368
+ "step": 1940
1369
+ },
1370
+ {
1371
+ "epoch": 0.06542087429127386,
1372
+ "grad_norm": NaN,
1373
+ "learning_rate": 0.00019270890979524076,
1374
+ "loss": 5.1668048858642575,
1375
+ "step": 1950
1376
+ },
1377
+ {
1378
+ "epoch": 0.06575636595430603,
1379
+ "grad_norm": NaN,
1380
+ "learning_rate": 0.00019263973436635308,
1381
+ "loss": 2.7092329025268556,
1382
+ "step": 1960
1383
+ },
1384
+ {
1385
+ "epoch": 0.06609185761733821,
1386
+ "grad_norm": NaN,
1387
+ "learning_rate": 0.00019257055893746544,
1388
+ "loss": 62.072442626953126,
1389
+ "step": 1970
1390
+ },
1391
+ {
1392
+ "epoch": 0.06642734928037038,
1393
+ "grad_norm": NaN,
1394
+ "learning_rate": 0.00019250138350857777,
1395
+ "loss": 1.7524122238159179,
1396
+ "step": 1980
1397
+ },
1398
+ {
1399
+ "epoch": 0.06676284094340255,
1400
+ "grad_norm": NaN,
1401
+ "learning_rate": 0.00019243220807969012,
1402
+ "loss": 34.05416564941406,
1403
+ "step": 1990
1404
+ },
1405
+ {
1406
+ "epoch": 0.06709833260643473,
1407
+ "grad_norm": NaN,
1408
+ "learning_rate": 0.00019236303265080245,
1409
+ "loss": 0.31617584228515627,
1410
+ "step": 2000
1411
+ },
1412
+ {
1413
+ "epoch": 0.06743382426946691,
1414
+ "grad_norm": NaN,
1415
+ "learning_rate": 0.00019229385722191477,
1416
+ "loss": 7.265777587890625,
1417
+ "step": 2010
1418
+ },
1419
+ {
1420
+ "epoch": 0.06776931593249907,
1421
+ "grad_norm": NaN,
1422
+ "learning_rate": 0.00019222468179302713,
1423
+ "loss": 0.1644289493560791,
1424
+ "step": 2020
1425
+ },
1426
+ {
1427
+ "epoch": 0.06810480759553125,
1428
+ "grad_norm": NaN,
1429
+ "learning_rate": 0.00019215550636413945,
1430
+ "loss": 0.8370037078857422,
1431
+ "step": 2030
1432
+ },
1433
+ {
1434
+ "epoch": 0.06844029925856343,
1435
+ "grad_norm": NaN,
1436
+ "learning_rate": 0.0001920863309352518,
1437
+ "loss": 0.31595618724823,
1438
+ "step": 2040
1439
+ },
1440
+ {
1441
+ "epoch": 0.0687757909215956,
1442
+ "grad_norm": NaN,
1443
+ "learning_rate": 0.00019201715550636413,
1444
+ "loss": 0.0,
1445
+ "step": 2050
1446
+ },
1447
+ {
1448
+ "epoch": 0.06911128258462777,
1449
+ "grad_norm": NaN,
1450
+ "learning_rate": 0.0001919479800774765,
1451
+ "loss": 13.008200073242188,
1452
+ "step": 2060
1453
+ },
1454
+ {
1455
+ "epoch": 0.06944677424765995,
1456
+ "grad_norm": NaN,
1457
+ "learning_rate": 0.00019187880464858882,
1458
+ "loss": 40.93202514648438,
1459
+ "step": 2070
1460
+ },
1461
+ {
1462
+ "epoch": 0.06978226591069212,
1463
+ "grad_norm": NaN,
1464
+ "learning_rate": 0.00019180962921970117,
1465
+ "loss": 0.0,
1466
+ "step": 2080
1467
+ },
1468
+ {
1469
+ "epoch": 0.0701177575737243,
1470
+ "grad_norm": NaN,
1471
+ "learning_rate": 0.00019174045379081352,
1472
+ "loss": 15.647711181640625,
1473
+ "step": 2090
1474
+ },
1475
+ {
1476
+ "epoch": 0.07045324923675647,
1477
+ "grad_norm": NaN,
1478
+ "learning_rate": 0.00019167127836192585,
1479
+ "loss": 0.14367586374282837,
1480
+ "step": 2100
1481
+ },
1482
+ {
1483
+ "epoch": 0.07078874089978864,
1484
+ "grad_norm": NaN,
1485
+ "learning_rate": 0.0001916021029330382,
1486
+ "loss": 8.04082794189453,
1487
+ "step": 2110
1488
+ },
1489
+ {
1490
+ "epoch": 0.07112423256282081,
1491
+ "grad_norm": NaN,
1492
+ "learning_rate": 0.00019153292750415053,
1493
+ "loss": 44.7733154296875,
1494
+ "step": 2120
1495
+ },
1496
+ {
1497
+ "epoch": 0.07145972422585299,
1498
+ "grad_norm": NaN,
1499
+ "learning_rate": 0.00019146375207526289,
1500
+ "loss": 4.347629547119141,
1501
+ "step": 2130
1502
+ },
1503
+ {
1504
+ "epoch": 0.07179521588888516,
1505
+ "grad_norm": NaN,
1506
+ "learning_rate": 0.0001913945766463752,
1507
+ "loss": 1.4188465118408202,
1508
+ "step": 2140
1509
+ },
1510
+ {
1511
+ "epoch": 0.07213070755191733,
1512
+ "grad_norm": NaN,
1513
+ "learning_rate": 0.00019132540121748757,
1514
+ "loss": 1.8175588607788087,
1515
+ "step": 2150
1516
+ },
1517
+ {
1518
+ "epoch": 0.07246619921494951,
1519
+ "grad_norm": NaN,
1520
+ "learning_rate": 0.0001912562257885999,
1521
+ "loss": 0.0,
1522
+ "step": 2160
1523
+ },
1524
+ {
1525
+ "epoch": 0.07280169087798168,
1526
+ "grad_norm": NaN,
1527
+ "learning_rate": 0.00019118705035971225,
1528
+ "loss": 1.2621800422668457,
1529
+ "step": 2170
1530
+ },
1531
+ {
1532
+ "epoch": 0.07313718254101385,
1533
+ "grad_norm": NaN,
1534
+ "learning_rate": 0.00019111787493082457,
1535
+ "loss": 7.959675598144531,
1536
+ "step": 2180
1537
+ },
1538
+ {
1539
+ "epoch": 0.07347267420404603,
1540
+ "grad_norm": NaN,
1541
+ "learning_rate": 0.0001910486995019369,
1542
+ "loss": 0.02607383728027344,
1543
+ "step": 2190
1544
+ },
1545
+ {
1546
+ "epoch": 0.07380816586707821,
1547
+ "grad_norm": NaN,
1548
+ "learning_rate": 0.00019097952407304926,
1549
+ "loss": 0.0,
1550
+ "step": 2200
1551
+ },
1552
+ {
1553
+ "epoch": 0.07414365753011037,
1554
+ "grad_norm": NaN,
1555
+ "learning_rate": 0.0001909103486441616,
1556
+ "loss": 0.0,
1557
+ "step": 2210
1558
+ },
1559
+ {
1560
+ "epoch": 0.07447914919314255,
1561
+ "grad_norm": NaN,
1562
+ "learning_rate": 0.00019084117321527394,
1563
+ "loss": 0.7340176105499268,
1564
+ "step": 2220
1565
+ },
1566
+ {
1567
+ "epoch": 0.07481464085617473,
1568
+ "grad_norm": NaN,
1569
+ "learning_rate": 0.0001907719977863863,
1570
+ "loss": 0.0,
1571
+ "step": 2230
1572
+ },
1573
+ {
1574
+ "epoch": 0.0751501325192069,
1575
+ "grad_norm": NaN,
1576
+ "learning_rate": 0.00019070282235749864,
1577
+ "loss": 0.25017504692077636,
1578
+ "step": 2240
1579
+ },
1580
+ {
1581
+ "epoch": 0.07548562418223907,
1582
+ "grad_norm": NaN,
1583
+ "learning_rate": 0.00019063364692861097,
1584
+ "loss": 0.22640457153320312,
1585
+ "step": 2250
1586
+ },
1587
+ {
1588
+ "epoch": 0.07582111584527125,
1589
+ "grad_norm": NaN,
1590
+ "learning_rate": 0.00019056447149972333,
1591
+ "loss": 0.11563352346420289,
1592
+ "step": 2260
1593
+ },
1594
+ {
1595
+ "epoch": 0.07615660750830341,
1596
+ "grad_norm": NaN,
1597
+ "learning_rate": 0.00019049529607083565,
1598
+ "loss": 0.0,
1599
+ "step": 2270
1600
+ },
1601
+ {
1602
+ "epoch": 0.07649209917133559,
1603
+ "grad_norm": NaN,
1604
+ "learning_rate": 0.00019042612064194798,
1605
+ "loss": 3.0178035736083983,
1606
+ "step": 2280
1607
+ },
1608
+ {
1609
+ "epoch": 0.07682759083436777,
1610
+ "grad_norm": NaN,
1611
+ "learning_rate": 0.00019035694521306033,
1612
+ "loss": 2.448381805419922,
1613
+ "step": 2290
1614
+ },
1615
+ {
1616
+ "epoch": 0.07716308249739993,
1617
+ "grad_norm": NaN,
1618
+ "learning_rate": 0.00019028776978417266,
1619
+ "loss": 0.08023186922073364,
1620
+ "step": 2300
1621
+ },
1622
+ {
1623
+ "epoch": 0.07749857416043211,
1624
+ "grad_norm": NaN,
1625
+ "learning_rate": 0.00019021859435528501,
1626
+ "loss": 1.200744342803955,
1627
+ "step": 2310
1628
+ },
1629
+ {
1630
+ "epoch": 0.07783406582346429,
1631
+ "grad_norm": NaN,
1632
+ "learning_rate": 0.00019014941892639734,
1633
+ "loss": 0.21134116649627685,
1634
+ "step": 2320
1635
+ },
1636
+ {
1637
+ "epoch": 0.07816955748649645,
1638
+ "grad_norm": NaN,
1639
+ "learning_rate": 0.0001900802434975097,
1640
+ "loss": 35.453729248046876,
1641
+ "step": 2330
1642
+ },
1643
+ {
1644
+ "epoch": 0.07850504914952863,
1645
+ "grad_norm": NaN,
1646
+ "learning_rate": 0.00019001106806862202,
1647
+ "loss": 137.92135009765624,
1648
+ "step": 2340
1649
+ },
1650
+ {
1651
+ "epoch": 0.07884054081256081,
1652
+ "grad_norm": NaN,
1653
+ "learning_rate": 0.00018994189263973438,
1654
+ "loss": 0.0,
1655
+ "step": 2350
1656
+ },
1657
+ {
1658
+ "epoch": 0.07917603247559298,
1659
+ "grad_norm": NaN,
1660
+ "learning_rate": 0.00018987271721084673,
1661
+ "loss": 6.1998237609863285,
1662
+ "step": 2360
1663
+ },
1664
+ {
1665
+ "epoch": 0.07951152413862515,
1666
+ "grad_norm": NaN,
1667
+ "learning_rate": 0.00018980354178195906,
1668
+ "loss": 13.4881103515625,
1669
+ "step": 2370
1670
+ },
1671
+ {
1672
+ "epoch": 0.07984701580165733,
1673
+ "grad_norm": NaN,
1674
+ "learning_rate": 0.0001897343663530714,
1675
+ "loss": 1.280299186706543,
1676
+ "step": 2380
1677
+ },
1678
+ {
1679
+ "epoch": 0.08018250746468951,
1680
+ "grad_norm": NaN,
1681
+ "learning_rate": 0.00018966519092418374,
1682
+ "loss": 116.68470458984375,
1683
+ "step": 2390
1684
+ },
1685
+ {
1686
+ "epoch": 0.08051799912772167,
1687
+ "grad_norm": NaN,
1688
+ "learning_rate": 0.0001895960154952961,
1689
+ "loss": 3.3710357666015627,
1690
+ "step": 2400
1691
+ },
1692
+ {
1693
+ "epoch": 0.08085349079075385,
1694
+ "grad_norm": NaN,
1695
+ "learning_rate": 0.00018952684006640842,
1696
+ "loss": 0.21961677074432373,
1697
+ "step": 2410
1698
+ },
1699
+ {
1700
+ "epoch": 0.08118898245378603,
1701
+ "grad_norm": NaN,
1702
+ "learning_rate": 0.00018945766463752077,
1703
+ "loss": 0.3747732639312744,
1704
+ "step": 2420
1705
+ },
1706
+ {
1707
+ "epoch": 0.0815244741168182,
1708
+ "grad_norm": NaN,
1709
+ "learning_rate": 0.0001893884892086331,
1710
+ "loss": 0.4310005664825439,
1711
+ "step": 2430
1712
+ },
1713
+ {
1714
+ "epoch": 0.08185996577985037,
1715
+ "grad_norm": NaN,
1716
+ "learning_rate": 0.00018931931377974543,
1717
+ "loss": 0.9692728042602539,
1718
+ "step": 2440
1719
+ },
1720
+ {
1721
+ "epoch": 0.08219545744288255,
1722
+ "grad_norm": NaN,
1723
+ "learning_rate": 0.00018925013835085778,
1724
+ "loss": 1.857094955444336,
1725
+ "step": 2450
1726
+ },
1727
+ {
1728
+ "epoch": 0.08253094910591471,
1729
+ "grad_norm": NaN,
1730
+ "learning_rate": 0.0001891809629219701,
1731
+ "loss": 4.252373123168946,
1732
+ "step": 2460
1733
+ },
1734
+ {
1735
+ "epoch": 0.08286644076894689,
1736
+ "grad_norm": NaN,
1737
+ "learning_rate": 0.00018911178749308246,
1738
+ "loss": 0.32670233249664304,
1739
+ "step": 2470
1740
+ },
1741
+ {
1742
+ "epoch": 0.08320193243197907,
1743
+ "grad_norm": NaN,
1744
+ "learning_rate": 0.0001890426120641948,
1745
+ "loss": 2.028175163269043,
1746
+ "step": 2480
1747
+ },
1748
+ {
1749
+ "epoch": 0.08353742409501123,
1750
+ "grad_norm": NaN,
1751
+ "learning_rate": 0.00018897343663530714,
1752
+ "loss": 0.6285067081451416,
1753
+ "step": 2490
1754
+ },
1755
+ {
1756
+ "epoch": 0.08387291575804341,
1757
+ "grad_norm": NaN,
1758
+ "learning_rate": 0.0001889042612064195,
1759
+ "loss": 0.388396167755127,
1760
+ "step": 2500
1761
+ },
1762
+ {
1763
+ "epoch": 0.08420840742107559,
1764
+ "grad_norm": NaN,
1765
+ "learning_rate": 0.00018883508577753185,
1766
+ "loss": 5.106251525878906,
1767
+ "step": 2510
1768
+ },
1769
+ {
1770
+ "epoch": 0.08454389908410775,
1771
+ "grad_norm": NaN,
1772
+ "learning_rate": 0.00018876591034864418,
1773
+ "loss": 11.141984558105468,
1774
+ "step": 2520
1775
+ },
1776
+ {
1777
+ "epoch": 0.08487939074713993,
1778
+ "grad_norm": NaN,
1779
+ "learning_rate": 0.0001886967349197565,
1780
+ "loss": 0.0,
1781
+ "step": 2530
1782
+ },
1783
+ {
1784
+ "epoch": 0.08521488241017211,
1785
+ "grad_norm": NaN,
1786
+ "learning_rate": 0.00018862755949086886,
1787
+ "loss": 7.814942932128906,
1788
+ "step": 2540
1789
+ },
1790
+ {
1791
+ "epoch": 0.08555037407320427,
1792
+ "grad_norm": NaN,
1793
+ "learning_rate": 0.00018855838406198119,
1794
+ "loss": 0.7215664863586426,
1795
+ "step": 2550
1796
+ },
1797
+ {
1798
+ "epoch": 0.08588586573623645,
1799
+ "grad_norm": NaN,
1800
+ "learning_rate": 0.00018848920863309354,
1801
+ "loss": 0.36127281188964844,
1802
+ "step": 2560
1803
+ },
1804
+ {
1805
+ "epoch": 0.08622135739926863,
1806
+ "grad_norm": NaN,
1807
+ "learning_rate": 0.00018842003320420587,
1808
+ "loss": 0.0,
1809
+ "step": 2570
1810
+ },
1811
+ {
1812
+ "epoch": 0.08655684906230081,
1813
+ "grad_norm": NaN,
1814
+ "learning_rate": 0.00018835085777531822,
1815
+ "loss": 0.0,
1816
+ "step": 2580
1817
+ },
1818
+ {
1819
+ "epoch": 0.08689234072533297,
1820
+ "grad_norm": NaN,
1821
+ "learning_rate": 0.00018828168234643055,
1822
+ "loss": 0.8563030242919922,
1823
+ "step": 2590
1824
+ },
1825
+ {
1826
+ "epoch": 0.08722783238836515,
1827
+ "grad_norm": NaN,
1828
+ "learning_rate": 0.0001882125069175429,
1829
+ "loss": 67.47144775390625,
1830
+ "step": 2600
1831
+ },
1832
+ {
1833
+ "epoch": 0.08756332405139733,
1834
+ "grad_norm": NaN,
1835
+ "learning_rate": 0.00018814333148865523,
1836
+ "loss": 11.852330780029297,
1837
+ "step": 2610
1838
+ },
1839
+ {
1840
+ "epoch": 0.08789881571442949,
1841
+ "grad_norm": NaN,
1842
+ "learning_rate": 0.00018807415605976758,
1843
+ "loss": 189.433349609375,
1844
+ "step": 2620
1845
+ },
1846
+ {
1847
+ "epoch": 0.08823430737746167,
1848
+ "grad_norm": NaN,
1849
+ "learning_rate": 0.00018800498063087994,
1850
+ "loss": 0.5957945823669434,
1851
+ "step": 2630
1852
+ },
1853
+ {
1854
+ "epoch": 0.08856979904049385,
1855
+ "grad_norm": NaN,
1856
+ "learning_rate": 0.00018793580520199226,
1857
+ "loss": 0.8038434028625489,
1858
+ "step": 2640
1859
+ },
1860
+ {
1861
+ "epoch": 0.08890529070352601,
1862
+ "grad_norm": NaN,
1863
+ "learning_rate": 0.00018786662977310462,
1864
+ "loss": 0.2752511024475098,
1865
+ "step": 2650
1866
+ },
1867
+ {
1868
+ "epoch": 0.08924078236655819,
1869
+ "grad_norm": NaN,
1870
+ "learning_rate": 0.00018779745434421694,
1871
+ "loss": 0.6539276599884033,
1872
+ "step": 2660
1873
+ },
1874
+ {
1875
+ "epoch": 0.08957627402959037,
1876
+ "grad_norm": NaN,
1877
+ "learning_rate": 0.0001877282789153293,
1878
+ "loss": 5.113545227050781,
1879
+ "step": 2670
1880
+ },
1881
+ {
1882
+ "epoch": 0.08991176569262253,
1883
+ "grad_norm": NaN,
1884
+ "learning_rate": 0.00018765910348644162,
1885
+ "loss": 0.0,
1886
+ "step": 2680
1887
+ },
1888
+ {
1889
+ "epoch": 0.09024725735565471,
1890
+ "grad_norm": NaN,
1891
+ "learning_rate": 0.00018758992805755398,
1892
+ "loss": 0.0,
1893
+ "step": 2690
1894
+ },
1895
+ {
1896
+ "epoch": 0.09058274901868689,
1897
+ "grad_norm": NaN,
1898
+ "learning_rate": 0.0001875207526286663,
1899
+ "loss": 6.7245635986328125,
1900
+ "step": 2700
1901
+ },
1902
+ {
1903
+ "epoch": 0.09091824068171905,
1904
+ "grad_norm": NaN,
1905
+ "learning_rate": 0.00018745157719977863,
1906
+ "loss": 145.0689453125,
1907
+ "step": 2710
1908
+ },
1909
+ {
1910
+ "epoch": 0.09125373234475123,
1911
+ "grad_norm": NaN,
1912
+ "learning_rate": 0.000187382401770891,
1913
+ "loss": 18.5854736328125,
1914
+ "step": 2720
1915
+ },
1916
+ {
1917
+ "epoch": 0.09158922400778341,
1918
+ "grad_norm": NaN,
1919
+ "learning_rate": 0.00018731322634200331,
1920
+ "loss": 0.0,
1921
+ "step": 2730
1922
+ },
1923
+ {
1924
+ "epoch": 0.09192471567081557,
1925
+ "grad_norm": NaN,
1926
+ "learning_rate": 0.00018724405091311567,
1927
+ "loss": 2.348870849609375,
1928
+ "step": 2740
1929
+ },
1930
+ {
1931
+ "epoch": 0.09226020733384775,
1932
+ "grad_norm": NaN,
1933
+ "learning_rate": 0.000187174875484228,
1934
+ "loss": 2.874924087524414,
1935
+ "step": 2750
1936
+ },
1937
+ {
1938
+ "epoch": 0.09259569899687993,
1939
+ "grad_norm": NaN,
1940
+ "learning_rate": 0.00018710570005534035,
1941
+ "loss": 2.0589231491088866,
1942
+ "step": 2760
1943
+ },
1944
+ {
1945
+ "epoch": 0.09293119065991211,
1946
+ "grad_norm": NaN,
1947
+ "learning_rate": 0.0001870365246264527,
1948
+ "loss": 52.030645751953124,
1949
+ "step": 2770
1950
+ },
1951
+ {
1952
+ "epoch": 0.09326668232294427,
1953
+ "grad_norm": NaN,
1954
+ "learning_rate": 0.00018696734919756506,
1955
+ "loss": 89.06510620117187,
1956
+ "step": 2780
1957
+ },
1958
+ {
1959
+ "epoch": 0.09360217398597645,
1960
+ "grad_norm": NaN,
1961
+ "learning_rate": 0.00018689817376867738,
1962
+ "loss": 0.9730803489685058,
1963
+ "step": 2790
1964
+ },
1965
+ {
1966
+ "epoch": 0.09393766564900863,
1967
+ "grad_norm": NaN,
1968
+ "learning_rate": 0.0001868289983397897,
1969
+ "loss": 0.0,
1970
+ "step": 2800
1971
+ },
1972
+ {
1973
+ "epoch": 0.09427315731204079,
1974
+ "grad_norm": NaN,
1975
+ "learning_rate": 0.00018675982291090206,
1976
+ "loss": 55.60011596679688,
1977
+ "step": 2810
1978
+ },
1979
+ {
1980
+ "epoch": 0.09460864897507297,
1981
+ "grad_norm": NaN,
1982
+ "learning_rate": 0.0001866906474820144,
1983
+ "loss": 0.0,
1984
+ "step": 2820
1985
+ },
1986
+ {
1987
+ "epoch": 0.09494414063810515,
1988
+ "grad_norm": NaN,
1989
+ "learning_rate": 0.00018662147205312675,
1990
+ "loss": 38.93014831542969,
1991
+ "step": 2830
1992
+ },
1993
+ {
1994
+ "epoch": 0.09527963230113731,
1995
+ "grad_norm": NaN,
1996
+ "learning_rate": 0.00018655229662423907,
1997
+ "loss": 0.6741656303405762,
1998
+ "step": 2840
1999
+ },
2000
+ {
2001
+ "epoch": 0.09561512396416949,
2002
+ "grad_norm": NaN,
2003
+ "learning_rate": 0.00018648312119535143,
2004
+ "loss": 0.09552153348922729,
2005
+ "step": 2850
2006
+ },
2007
+ {
2008
+ "epoch": 0.09595061562720167,
2009
+ "grad_norm": NaN,
2010
+ "learning_rate": 0.00018641394576646375,
2011
+ "loss": 0.13130130767822265,
2012
+ "step": 2860
2013
+ },
2014
+ {
2015
+ "epoch": 0.09628610729023383,
2016
+ "grad_norm": NaN,
2017
+ "learning_rate": 0.00018634477033757608,
2018
+ "loss": 43.31304931640625,
2019
+ "step": 2870
2020
+ },
2021
+ {
2022
+ "epoch": 0.09662159895326601,
2023
+ "grad_norm": NaN,
2024
+ "learning_rate": 0.00018627559490868843,
2025
+ "loss": 0.27574739456176756,
2026
+ "step": 2880
2027
+ },
2028
+ {
2029
+ "epoch": 0.09695709061629819,
2030
+ "grad_norm": NaN,
2031
+ "learning_rate": 0.0001862064194798008,
2032
+ "loss": 0.0495761901140213,
2033
+ "step": 2890
2034
+ },
2035
+ {
2036
+ "epoch": 0.09729258227933035,
2037
+ "grad_norm": NaN,
2038
+ "learning_rate": 0.00018613724405091314,
2039
+ "loss": 8.007275390625,
2040
+ "step": 2900
2041
+ },
2042
+ {
2043
+ "epoch": 0.09762807394236253,
2044
+ "grad_norm": NaN,
2045
+ "learning_rate": 0.00018606806862202547,
2046
+ "loss": 306.2622314453125,
2047
+ "step": 2910
2048
+ },
2049
+ {
2050
+ "epoch": 0.09796356560539471,
2051
+ "grad_norm": NaN,
2052
+ "learning_rate": 0.00018599889319313782,
2053
+ "loss": 0.0,
2054
+ "step": 2920
2055
+ },
2056
+ {
2057
+ "epoch": 0.09829905726842687,
2058
+ "grad_norm": NaN,
2059
+ "learning_rate": 0.00018592971776425015,
2060
+ "loss": 0.9617393493652344,
2061
+ "step": 2930
2062
+ },
2063
+ {
2064
+ "epoch": 0.09863454893145905,
2065
+ "grad_norm": NaN,
2066
+ "learning_rate": 0.0001858605423353625,
2067
+ "loss": 15.769479370117187,
2068
+ "step": 2940
2069
+ },
2070
+ {
2071
+ "epoch": 0.09897004059449123,
2072
+ "grad_norm": NaN,
2073
+ "learning_rate": 0.00018579136690647483,
2074
+ "loss": 122.81998291015626,
2075
+ "step": 2950
2076
+ },
2077
+ {
2078
+ "epoch": 0.09930553225752341,
2079
+ "grad_norm": NaN,
2080
+ "learning_rate": 0.00018572219147758716,
2081
+ "loss": 0.0,
2082
+ "step": 2960
2083
+ },
2084
+ {
2085
+ "epoch": 0.09964102392055557,
2086
+ "grad_norm": NaN,
2087
+ "learning_rate": 0.0001856530160486995,
2088
+ "loss": 2.3114242553710938,
2089
+ "step": 2970
2090
+ },
2091
+ {
2092
+ "epoch": 0.09997651558358775,
2093
+ "grad_norm": NaN,
2094
+ "learning_rate": 0.00018558384061981184,
2095
+ "loss": 89.21986083984375,
2096
+ "step": 2980
2097
+ },
2098
+ {
2099
+ "epoch": 0.10031200724661993,
2100
+ "grad_norm": NaN,
2101
+ "learning_rate": 0.0001855146651909242,
2102
+ "loss": 8.306566619873047,
2103
+ "step": 2990
2104
+ },
2105
+ {
2106
+ "epoch": 0.10064749890965209,
2107
+ "grad_norm": NaN,
2108
+ "learning_rate": 0.00018544548976203652,
2109
+ "loss": 3.119292068481445,
2110
+ "step": 3000
2111
+ },
2112
+ {
2113
+ "epoch": 0.10098299057268427,
2114
+ "grad_norm": NaN,
2115
+ "learning_rate": 0.00018537631433314887,
2116
+ "loss": 1.7612421035766601,
2117
+ "step": 3010
2118
+ },
2119
+ {
2120
+ "epoch": 0.10131848223571645,
2121
+ "grad_norm": NaN,
2122
+ "learning_rate": 0.0001853071389042612,
2123
+ "loss": 0.6873093605041504,
2124
+ "step": 3020
2125
+ },
2126
+ {
2127
+ "epoch": 0.10165397389874861,
2128
+ "grad_norm": NaN,
2129
+ "learning_rate": 0.00018523796347537355,
2130
+ "loss": 40.089596557617185,
2131
+ "step": 3030
2132
+ },
2133
+ {
2134
+ "epoch": 0.10198946556178079,
2135
+ "grad_norm": NaN,
2136
+ "learning_rate": 0.0001851687880464859,
2137
+ "loss": 1.7284236907958985,
2138
+ "step": 3040
2139
+ },
2140
+ {
2141
+ "epoch": 0.10232495722481297,
2142
+ "grad_norm": NaN,
2143
+ "learning_rate": 0.00018509961261759824,
2144
+ "loss": 73.38009033203124,
2145
+ "step": 3050
2146
+ },
2147
+ {
2148
+ "epoch": 0.10266044888784513,
2149
+ "grad_norm": NaN,
2150
+ "learning_rate": 0.0001850304371887106,
2151
+ "loss": 61.3970703125,
2152
+ "step": 3060
2153
+ },
2154
+ {
2155
+ "epoch": 0.10299594055087731,
2156
+ "grad_norm": NaN,
2157
+ "learning_rate": 0.00018496126175982292,
2158
+ "loss": 403.2418701171875,
2159
+ "step": 3070
2160
+ },
2161
+ {
2162
+ "epoch": 0.10333143221390949,
2163
+ "grad_norm": NaN,
2164
+ "learning_rate": 0.00018489208633093527,
2165
+ "loss": 0.2365236759185791,
2166
+ "step": 3080
2167
+ },
2168
+ {
2169
+ "epoch": 0.10366692387694165,
2170
+ "grad_norm": NaN,
2171
+ "learning_rate": 0.0001848229109020476,
2172
+ "loss": 128.380419921875,
2173
+ "step": 3090
2174
+ },
2175
+ {
2176
+ "epoch": 0.10400241553997383,
2177
+ "grad_norm": NaN,
2178
+ "learning_rate": 0.00018475373547315995,
2179
+ "loss": 1.1766498565673829,
2180
+ "step": 3100
2181
+ },
2182
+ {
2183
+ "epoch": 0.10433790720300601,
2184
+ "grad_norm": NaN,
2185
+ "learning_rate": 0.00018468456004427228,
2186
+ "loss": 0.26281468868255614,
2187
+ "step": 3110
2188
+ },
2189
+ {
2190
+ "epoch": 0.10467339886603817,
2191
+ "grad_norm": NaN,
2192
+ "learning_rate": 0.00018461538461538463,
2193
+ "loss": 0.6981168270111084,
2194
+ "step": 3120
2195
+ },
2196
+ {
2197
+ "epoch": 0.10500889052907035,
2198
+ "grad_norm": NaN,
2199
+ "learning_rate": 0.00018454620918649696,
2200
+ "loss": 3.2268295288085938,
2201
+ "step": 3130
2202
+ },
2203
+ {
2204
+ "epoch": 0.10534438219210253,
2205
+ "grad_norm": NaN,
2206
+ "learning_rate": 0.00018447703375760929,
2207
+ "loss": 0.03597120642662048,
2208
+ "step": 3140
2209
+ },
2210
+ {
2211
+ "epoch": 0.1056798738551347,
2212
+ "grad_norm": NaN,
2213
+ "learning_rate": 0.00018440785832872164,
2214
+ "loss": 21.406304931640626,
2215
+ "step": 3150
2216
+ },
2217
+ {
2218
+ "epoch": 0.10601536551816687,
2219
+ "grad_norm": NaN,
2220
+ "learning_rate": 0.000184338682899834,
2221
+ "loss": 230.59599609375,
2222
+ "step": 3160
2223
+ },
2224
+ {
2225
+ "epoch": 0.10635085718119905,
2226
+ "grad_norm": NaN,
2227
+ "learning_rate": 0.00018426950747094632,
2228
+ "loss": 120.71004638671874,
2229
+ "step": 3170
2230
+ },
2231
+ {
2232
+ "epoch": 0.10668634884423123,
2233
+ "grad_norm": NaN,
2234
+ "learning_rate": 0.00018420033204205868,
2235
+ "loss": 2.0129737854003906,
2236
+ "step": 3180
2237
+ },
2238
+ {
2239
+ "epoch": 0.10702184050726339,
2240
+ "grad_norm": NaN,
2241
+ "learning_rate": 0.00018413115661317103,
2242
+ "loss": 7.2358558654785154,
2243
+ "step": 3190
2244
+ },
2245
+ {
2246
+ "epoch": 0.10735733217029557,
2247
+ "grad_norm": NaN,
2248
+ "learning_rate": 0.00018406198118428336,
2249
+ "loss": 44.222772216796876,
2250
+ "step": 3200
2251
+ },
2252
+ {
2253
+ "epoch": 0.10769282383332775,
2254
+ "grad_norm": NaN,
2255
+ "learning_rate": 0.0001839928057553957,
2256
+ "loss": 101.93148193359374,
2257
+ "step": 3210
2258
+ },
2259
+ {
2260
+ "epoch": 0.10802831549635991,
2261
+ "grad_norm": NaN,
2262
+ "learning_rate": 0.00018392363032650804,
2263
+ "loss": 130.751708984375,
2264
+ "step": 3220
2265
+ },
2266
+ {
2267
+ "epoch": 0.10836380715939209,
2268
+ "grad_norm": NaN,
2269
+ "learning_rate": 0.00018385445489762036,
2270
+ "loss": 4.211777877807617,
2271
+ "step": 3230
2272
+ },
2273
+ {
2274
+ "epoch": 0.10869929882242427,
2275
+ "grad_norm": NaN,
2276
+ "learning_rate": 0.00018378527946873272,
2277
+ "loss": 23.889674377441406,
2278
+ "step": 3240
2279
+ },
2280
+ {
2281
+ "epoch": 0.10903479048545643,
2282
+ "grad_norm": NaN,
2283
+ "learning_rate": 0.00018371610403984505,
2284
+ "loss": 228.9492919921875,
2285
+ "step": 3250
2286
+ },
2287
+ {
2288
+ "epoch": 0.10937028214848861,
2289
+ "grad_norm": NaN,
2290
+ "learning_rate": 0.0001836469286109574,
2291
+ "loss": 5.618381118774414,
2292
+ "step": 3260
2293
+ },
2294
+ {
2295
+ "epoch": 0.10970577381152079,
2296
+ "grad_norm": NaN,
2297
+ "learning_rate": 0.00018357775318206973,
2298
+ "loss": 0.13609206676483154,
2299
+ "step": 3270
2300
+ },
2301
+ {
2302
+ "epoch": 0.11004126547455295,
2303
+ "grad_norm": NaN,
2304
+ "learning_rate": 0.00018350857775318208,
2305
+ "loss": 0.8353385925292969,
2306
+ "step": 3280
2307
+ },
2308
+ {
2309
+ "epoch": 0.11037675713758513,
2310
+ "grad_norm": NaN,
2311
+ "learning_rate": 0.0001834394023242944,
2312
+ "loss": 0.2723431348800659,
2313
+ "step": 3290
2314
+ },
2315
+ {
2316
+ "epoch": 0.11071224880061731,
2317
+ "grad_norm": NaN,
2318
+ "learning_rate": 0.00018337022689540676,
2319
+ "loss": 44.59869384765625,
2320
+ "step": 3300
2321
+ },
2322
+ {
2323
+ "epoch": 0.11104774046364947,
2324
+ "grad_norm": NaN,
2325
+ "learning_rate": 0.00018330105146651912,
2326
+ "loss": 24.982450866699217,
2327
+ "step": 3310
2328
+ },
2329
+ {
2330
+ "epoch": 0.11138323212668165,
2331
+ "grad_norm": NaN,
2332
+ "learning_rate": 0.00018323187603763144,
2333
+ "loss": 196.17008056640626,
2334
+ "step": 3320
2335
+ },
2336
+ {
2337
+ "epoch": 0.11171872378971383,
2338
+ "grad_norm": NaN,
2339
+ "learning_rate": 0.0001831627006087438,
2340
+ "loss": 67.57760009765624,
2341
+ "step": 3330
2342
+ },
2343
+ {
2344
+ "epoch": 0.112054215452746,
2345
+ "grad_norm": NaN,
2346
+ "learning_rate": 0.00018309352517985612,
2347
+ "loss": 16.773439025878908,
2348
+ "step": 3340
2349
+ },
2350
+ {
2351
+ "epoch": 0.11238970711577817,
2352
+ "grad_norm": NaN,
2353
+ "learning_rate": 0.00018302434975096848,
2354
+ "loss": 1.449579906463623,
2355
+ "step": 3350
2356
+ },
2357
+ {
2358
+ "epoch": 0.11272519877881035,
2359
+ "grad_norm": NaN,
2360
+ "learning_rate": 0.0001829551743220808,
2361
+ "loss": 0.0,
2362
+ "step": 3360
2363
+ },
2364
+ {
2365
+ "epoch": 0.11306069044184253,
2366
+ "grad_norm": NaN,
2367
+ "learning_rate": 0.00018288599889319316,
2368
+ "loss": 0.0,
2369
+ "step": 3370
2370
+ },
2371
+ {
2372
+ "epoch": 0.11339618210487469,
2373
+ "grad_norm": NaN,
2374
+ "learning_rate": 0.00018281682346430548,
2375
+ "loss": 1.3720318794250488,
2376
+ "step": 3380
2377
+ },
2378
+ {
2379
+ "epoch": 0.11373167376790687,
2380
+ "grad_norm": NaN,
2381
+ "learning_rate": 0.0001827476480354178,
2382
+ "loss": 1.8041461944580077,
2383
+ "step": 3390
2384
+ },
2385
+ {
2386
+ "epoch": 0.11406716543093905,
2387
+ "grad_norm": NaN,
2388
+ "learning_rate": 0.00018267847260653017,
2389
+ "loss": 1.0163018226623535,
2390
+ "step": 3400
2391
+ },
2392
+ {
2393
+ "epoch": 0.11440265709397121,
2394
+ "grad_norm": NaN,
2395
+ "learning_rate": 0.0001826092971776425,
2396
+ "loss": 99.13453979492188,
2397
+ "step": 3410
2398
+ },
2399
+ {
2400
+ "epoch": 0.11473814875700339,
2401
+ "grad_norm": NaN,
2402
+ "learning_rate": 0.00018254012174875485,
2403
+ "loss": 53.49818115234375,
2404
+ "step": 3420
2405
+ },
2406
+ {
2407
+ "epoch": 0.11507364042003557,
2408
+ "grad_norm": NaN,
2409
+ "learning_rate": 0.00018247094631986717,
2410
+ "loss": 0.8270892143249512,
2411
+ "step": 3430
2412
+ },
2413
+ {
2414
+ "epoch": 0.11540913208306773,
2415
+ "grad_norm": NaN,
2416
+ "learning_rate": 0.00018240177089097953,
2417
+ "loss": 0.0,
2418
+ "step": 3440
2419
+ },
2420
+ {
2421
+ "epoch": 0.11574462374609991,
2422
+ "grad_norm": NaN,
2423
+ "learning_rate": 0.00018233259546209188,
2424
+ "loss": 31.446853637695312,
2425
+ "step": 3450
2426
+ },
2427
+ {
2428
+ "epoch": 0.11608011540913209,
2429
+ "grad_norm": NaN,
2430
+ "learning_rate": 0.00018226342003320424,
2431
+ "loss": 31.089089965820314,
2432
+ "step": 3460
2433
+ },
2434
+ {
2435
+ "epoch": 0.11641560707216425,
2436
+ "grad_norm": NaN,
2437
+ "learning_rate": 0.00018219424460431656,
2438
+ "loss": 0.25326035022735593,
2439
+ "step": 3470
2440
+ },
2441
+ {
2442
+ "epoch": 0.11675109873519643,
2443
+ "grad_norm": NaN,
2444
+ "learning_rate": 0.0001821250691754289,
2445
+ "loss": 0.2245074987411499,
2446
+ "step": 3480
2447
+ },
2448
+ {
2449
+ "epoch": 0.11708659039822861,
2450
+ "grad_norm": NaN,
2451
+ "learning_rate": 0.00018205589374654124,
2452
+ "loss": 0.0,
2453
+ "step": 3490
2454
+ },
2455
+ {
2456
+ "epoch": 0.11742208206126077,
2457
+ "grad_norm": NaN,
2458
+ "learning_rate": 0.00018198671831765357,
2459
+ "loss": 0.5538990020751953,
2460
+ "step": 3500
2461
+ },
2462
+ {
2463
+ "epoch": 0.11775757372429295,
2464
+ "grad_norm": NaN,
2465
+ "learning_rate": 0.00018191754288876592,
2466
+ "loss": 9.654029083251952,
2467
+ "step": 3510
2468
+ },
2469
+ {
2470
+ "epoch": 0.11809306538732513,
2471
+ "grad_norm": NaN,
2472
+ "learning_rate": 0.00018184836745987825,
2473
+ "loss": 10.992623138427735,
2474
+ "step": 3520
2475
+ },
2476
+ {
2477
+ "epoch": 0.1184285570503573,
2478
+ "grad_norm": NaN,
2479
+ "learning_rate": 0.0001817791920309906,
2480
+ "loss": 0.0,
2481
+ "step": 3530
2482
+ },
2483
+ {
2484
+ "epoch": 0.11876404871338947,
2485
+ "grad_norm": NaN,
2486
+ "learning_rate": 0.00018171001660210293,
2487
+ "loss": 42.99931945800781,
2488
+ "step": 3540
2489
+ },
2490
+ {
2491
+ "epoch": 0.11909954037642165,
2492
+ "grad_norm": NaN,
2493
+ "learning_rate": 0.00018164084117321529,
2494
+ "loss": 0.20315234661102294,
2495
+ "step": 3550
2496
+ },
2497
+ {
2498
+ "epoch": 0.11943503203945383,
2499
+ "grad_norm": NaN,
2500
+ "learning_rate": 0.0001815716657443276,
2501
+ "loss": 1.9736587524414062,
2502
+ "step": 3560
2503
+ },
2504
+ {
2505
+ "epoch": 0.11977052370248599,
2506
+ "grad_norm": NaN,
2507
+ "learning_rate": 0.00018150249031543997,
2508
+ "loss": 183.6845458984375,
2509
+ "step": 3570
2510
+ },
2511
+ {
2512
+ "epoch": 0.12010601536551817,
2513
+ "grad_norm": NaN,
2514
+ "learning_rate": 0.00018143331488655232,
2515
+ "loss": 6.060160827636719,
2516
+ "step": 3580
2517
+ },
2518
+ {
2519
+ "epoch": 0.12044150702855035,
2520
+ "grad_norm": NaN,
2521
+ "learning_rate": 0.00018136413945766465,
2522
+ "loss": 2.0698776245117188,
2523
+ "step": 3590
2524
+ },
2525
+ {
2526
+ "epoch": 0.12077699869158251,
2527
+ "grad_norm": NaN,
2528
+ "learning_rate": 0.000181294964028777,
2529
+ "loss": 0.2160647392272949,
2530
+ "step": 3600
2531
+ },
2532
+ {
2533
+ "epoch": 0.12111249035461469,
2534
+ "grad_norm": NaN,
2535
+ "learning_rate": 0.00018122578859988933,
2536
+ "loss": 8.876626586914062,
2537
+ "step": 3610
2538
+ },
2539
+ {
2540
+ "epoch": 0.12144798201764687,
2541
+ "grad_norm": NaN,
2542
+ "learning_rate": 0.00018115661317100168,
2543
+ "loss": 5.090277481079101,
2544
+ "step": 3620
2545
+ },
2546
+ {
2547
+ "epoch": 0.12178347368067903,
2548
+ "grad_norm": NaN,
2549
+ "learning_rate": 0.000181087437742114,
2550
+ "loss": 2.9288057327270507,
2551
+ "step": 3630
2552
+ },
2553
+ {
2554
+ "epoch": 0.12211896534371121,
2555
+ "grad_norm": NaN,
2556
+ "learning_rate": 0.00018101826231322636,
2557
+ "loss": 1.4944279670715332,
2558
+ "step": 3640
2559
+ },
2560
+ {
2561
+ "epoch": 0.12245445700674339,
2562
+ "grad_norm": NaN,
2563
+ "learning_rate": 0.0001809490868843387,
2564
+ "loss": 3.8164928436279295,
2565
+ "step": 3650
2566
+ },
2567
+ {
2568
+ "epoch": 0.12278994866977555,
2569
+ "grad_norm": NaN,
2570
+ "learning_rate": 0.00018087991145545102,
2571
+ "loss": 1.3847745895385741,
2572
+ "step": 3660
2573
+ },
2574
+ {
2575
+ "epoch": 0.12312544033280773,
2576
+ "grad_norm": NaN,
2577
+ "learning_rate": 0.00018081073602656337,
2578
+ "loss": 343.1435546875,
2579
+ "step": 3670
2580
+ },
2581
+ {
2582
+ "epoch": 0.1234609319958399,
2583
+ "grad_norm": NaN,
2584
+ "learning_rate": 0.0001807415605976757,
2585
+ "loss": 9.158045196533203,
2586
+ "step": 3680
2587
+ },
2588
+ {
2589
+ "epoch": 0.12379642365887207,
2590
+ "grad_norm": NaN,
2591
+ "learning_rate": 0.00018067238516878805,
2592
+ "loss": 78.78106079101562,
2593
+ "step": 3690
2594
+ },
2595
+ {
2596
+ "epoch": 0.12413191532190425,
2597
+ "grad_norm": NaN,
2598
+ "learning_rate": 0.00018060320973990038,
2599
+ "loss": 0.0,
2600
+ "step": 3700
2601
+ },
2602
+ {
2603
+ "epoch": 0.12446740698493643,
2604
+ "grad_norm": NaN,
2605
+ "learning_rate": 0.00018053403431101273,
2606
+ "loss": 0.10871992111206055,
2607
+ "step": 3710
2608
+ },
2609
+ {
2610
+ "epoch": 0.1248028986479686,
2611
+ "grad_norm": NaN,
2612
+ "learning_rate": 0.0001804648588821251,
2613
+ "loss": 69.28292846679688,
2614
+ "step": 3720
2615
+ },
2616
+ {
2617
+ "epoch": 0.12513839031100077,
2618
+ "grad_norm": NaN,
2619
+ "learning_rate": 0.00018039568345323744,
2620
+ "loss": 0.0,
2621
+ "step": 3730
2622
+ },
2623
+ {
2624
+ "epoch": 0.12547388197403295,
2625
+ "grad_norm": NaN,
2626
+ "learning_rate": 0.00018032650802434977,
2627
+ "loss": 7.927516937255859,
2628
+ "step": 3740
2629
+ },
2630
+ {
2631
+ "epoch": 0.12580937363706512,
2632
+ "grad_norm": NaN,
2633
+ "learning_rate": 0.0001802573325954621,
2634
+ "loss": 0.12222099304199219,
2635
+ "step": 3750
2636
+ },
2637
+ {
2638
+ "epoch": 0.1261448653000973,
2639
+ "grad_norm": NaN,
2640
+ "learning_rate": 0.00018018815716657445,
2641
+ "loss": 16.676641845703124,
2642
+ "step": 3760
2643
+ },
2644
+ {
2645
+ "epoch": 0.12648035696312945,
2646
+ "grad_norm": NaN,
2647
+ "learning_rate": 0.00018011898173768678,
2648
+ "loss": 0.10614252090454102,
2649
+ "step": 3770
2650
+ },
2651
+ {
2652
+ "epoch": 0.12681584862616163,
2653
+ "grad_norm": NaN,
2654
+ "learning_rate": 0.00018004980630879913,
2655
+ "loss": 0.1973770260810852,
2656
+ "step": 3780
2657
+ },
2658
+ {
2659
+ "epoch": 0.1271513402891938,
2660
+ "grad_norm": NaN,
2661
+ "learning_rate": 0.00017998063087991146,
2662
+ "loss": 0.26248602867126464,
2663
+ "step": 3790
2664
+ },
2665
+ {
2666
+ "epoch": 0.127486831952226,
2667
+ "grad_norm": NaN,
2668
+ "learning_rate": 0.0001799114554510238,
2669
+ "loss": 0.3842607021331787,
2670
+ "step": 3800
2671
+ },
2672
+ {
2673
+ "epoch": 0.12782232361525817,
2674
+ "grad_norm": NaN,
2675
+ "learning_rate": 0.00017984228002213614,
2676
+ "loss": 7.765513610839844,
2677
+ "step": 3810
2678
+ },
2679
+ {
2680
+ "epoch": 0.12815781527829034,
2681
+ "grad_norm": NaN,
2682
+ "learning_rate": 0.00017977310459324847,
2683
+ "loss": 157.426025390625,
2684
+ "step": 3820
2685
+ },
2686
+ {
2687
+ "epoch": 0.12849330694132252,
2688
+ "grad_norm": NaN,
2689
+ "learning_rate": 0.00017970392916436082,
2690
+ "loss": 8.641764831542968,
2691
+ "step": 3830
2692
+ },
2693
+ {
2694
+ "epoch": 0.12882879860435467,
2695
+ "grad_norm": NaN,
2696
+ "learning_rate": 0.00017963475373547317,
2697
+ "loss": 0.19315568208694459,
2698
+ "step": 3840
2699
+ },
2700
+ {
2701
+ "epoch": 0.12916429026738685,
2702
+ "grad_norm": NaN,
2703
+ "learning_rate": 0.0001795655783065855,
2704
+ "loss": 1.3260221481323242,
2705
+ "step": 3850
2706
+ },
2707
+ {
2708
+ "epoch": 0.12949978193041903,
2709
+ "grad_norm": NaN,
2710
+ "learning_rate": 0.00017949640287769785,
2711
+ "loss": 1.1267181396484376,
2712
+ "step": 3860
2713
+ },
2714
+ {
2715
+ "epoch": 0.1298352735934512,
2716
+ "grad_norm": NaN,
2717
+ "learning_rate": 0.0001794272274488102,
2718
+ "loss": 13.151643371582031,
2719
+ "step": 3870
2720
+ },
2721
+ {
2722
+ "epoch": 0.13017076525648338,
2723
+ "grad_norm": NaN,
2724
+ "learning_rate": 0.00017935805201992254,
2725
+ "loss": 0.0,
2726
+ "step": 3880
2727
+ },
2728
+ {
2729
+ "epoch": 0.13050625691951556,
2730
+ "grad_norm": NaN,
2731
+ "learning_rate": 0.0001792888765910349,
2732
+ "loss": 0.16426589488983154,
2733
+ "step": 3890
2734
+ },
2735
+ {
2736
+ "epoch": 0.1308417485825477,
2737
+ "grad_norm": NaN,
2738
+ "learning_rate": 0.00017921970116214722,
2739
+ "loss": 0.42214279174804686,
2740
+ "step": 3900
2741
+ },
2742
+ {
2743
+ "epoch": 0.1311772402455799,
2744
+ "grad_norm": NaN,
2745
+ "learning_rate": 0.00017915052573325954,
2746
+ "loss": 0.02913391888141632,
2747
+ "step": 3910
2748
+ },
2749
+ {
2750
+ "epoch": 0.13151273190861207,
2751
+ "grad_norm": NaN,
2752
+ "learning_rate": 0.0001790813503043719,
2753
+ "loss": 4.68084602355957,
2754
+ "step": 3920
2755
+ },
2756
+ {
2757
+ "epoch": 0.13184822357164425,
2758
+ "grad_norm": NaN,
2759
+ "learning_rate": 0.00017901217487548422,
2760
+ "loss": 0.0,
2761
+ "step": 3930
2762
+ },
2763
+ {
2764
+ "epoch": 0.13218371523467642,
2765
+ "grad_norm": NaN,
2766
+ "learning_rate": 0.00017894299944659658,
2767
+ "loss": 1.6939407348632813,
2768
+ "step": 3940
2769
+ },
2770
+ {
2771
+ "epoch": 0.1325192068977086,
2772
+ "grad_norm": NaN,
2773
+ "learning_rate": 0.0001788738240177089,
2774
+ "loss": 0.09735113978385926,
2775
+ "step": 3950
2776
+ },
2777
+ {
2778
+ "epoch": 0.13285469856074075,
2779
+ "grad_norm": NaN,
2780
+ "learning_rate": 0.00017880464858882126,
2781
+ "loss": 0.0,
2782
+ "step": 3960
2783
+ },
2784
+ {
2785
+ "epoch": 0.13319019022377293,
2786
+ "grad_norm": NaN,
2787
+ "learning_rate": 0.00017873547315993359,
2788
+ "loss": 499.250634765625,
2789
+ "step": 3970
2790
+ },
2791
+ {
2792
+ "epoch": 0.1335256818868051,
2793
+ "grad_norm": NaN,
2794
+ "learning_rate": 0.00017866629773104594,
2795
+ "loss": 3.100843620300293,
2796
+ "step": 3980
2797
+ },
2798
+ {
2799
+ "epoch": 0.1338611735498373,
2800
+ "grad_norm": NaN,
2801
+ "learning_rate": 0.0001785971223021583,
2802
+ "loss": 28.033572387695312,
2803
+ "step": 3990
2804
+ },
2805
+ {
2806
+ "epoch": 0.13419666521286946,
2807
+ "grad_norm": NaN,
2808
+ "learning_rate": 0.00017852794687327062,
2809
+ "loss": 40.432196044921874,
2810
+ "step": 4000
2811
+ }
2812
+ ],
2813
+ "logging_steps": 10,
2814
+ "max_steps": 29807,
2815
+ "num_input_tokens_seen": 0,
2816
+ "num_train_epochs": 1,
2817
+ "save_steps": 500,
2818
+ "stateful_callbacks": {
2819
+ "TrainerControl": {
2820
+ "args": {
2821
+ "should_epoch_stop": false,
2822
+ "should_evaluate": false,
2823
+ "should_log": false,
2824
+ "should_save": true,
2825
+ "should_training_stop": false
2826
+ },
2827
+ "attributes": {}
2828
+ }
2829
+ },
2830
+ "total_flos": 1.6190453041390368e+17,
2831
+ "train_batch_size": 1,
2832
+ "trial_name": null,
2833
+ "trial_params": null
2834
+ }
checkpoints/checkpoint-4000/training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0709e3e72ef3d047d759353065195f5eee4d7437855a71b8702a281685aabb1d
3
+ size 5969