Hugging Face's logo

Spaces:
gargimonga
/
TruthLens-Backend
Sleeping

App Files Community
Gargi Monga commited on
Commit
ff7fb33
·
1 Parent(s): 9c7e629

Remove training checkpoints, not needed for deployment

Browse files
Files changed (31) hide show
  1. .gitignore +0 -0
  2. training/training-checkpoints/checkpoint-1700/README.md +0 -202
  3. training/training-checkpoints/checkpoint-1700/adapter_config.json +0 -32
  4. training/training-checkpoints/checkpoint-1700/adapter_model.safetensors +0 -3
  5. training/training-checkpoints/checkpoint-1700/optimizer.pt +0 -3
  6. training/training-checkpoints/checkpoint-1700/rng_state.pth +0 -3
  7. training/training-checkpoints/checkpoint-1700/scheduler.pt +0 -3
  8. training/training-checkpoints/checkpoint-1700/tokenizer.json +0 -0
  9. training/training-checkpoints/checkpoint-1700/tokenizer_config.json +0 -15
  10. training/training-checkpoints/checkpoint-1700/trainer_state.json +0 -1233
  11. training/training-checkpoints/checkpoint-1700/training_args.bin +0 -3
  12. training/training-checkpoints/checkpoint-3400/README.md +0 -202
  13. training/training-checkpoints/checkpoint-3400/adapter_config.json +0 -32
  14. training/training-checkpoints/checkpoint-3400/adapter_model.safetensors +0 -3
  15. training/training-checkpoints/checkpoint-3400/optimizer.pt +0 -3
  16. training/training-checkpoints/checkpoint-3400/rng_state.pth +0 -3
  17. training/training-checkpoints/checkpoint-3400/scheduler.pt +0 -3
  18. training/training-checkpoints/checkpoint-3400/tokenizer.json +0 -0
  19. training/training-checkpoints/checkpoint-3400/tokenizer_config.json +0 -15
  20. training/training-checkpoints/checkpoint-3400/trainer_state.json +0 -2432
  21. training/training-checkpoints/checkpoint-3400/training_args.bin +0 -3
  22. training/training-checkpoints/checkpoint-5100/README.md +0 -202
  23. training/training-checkpoints/checkpoint-5100/adapter_config.json +0 -32
  24. training/training-checkpoints/checkpoint-5100/adapter_model.safetensors +0 -3
  25. training/training-checkpoints/checkpoint-5100/optimizer.pt +0 -3
  26. training/training-checkpoints/checkpoint-5100/rng_state.pth +0 -3
  27. training/training-checkpoints/checkpoint-5100/scheduler.pt +0 -3
  28. training/training-checkpoints/checkpoint-5100/tokenizer.json +0 -0
  29. training/training-checkpoints/checkpoint-5100/tokenizer_config.json +0 -15
  30. training/training-checkpoints/checkpoint-5100/trainer_state.json +0 -0
  31. training/training-checkpoints/checkpoint-5100/training_args.bin +0 -3
.gitignore CHANGED
Binary files a/.gitignore and b/.gitignore differ
 
training/training-checkpoints/checkpoint-1700/README.md DELETED
@@ -1,202 +0,0 @@
1
- ---
2
- base_model: distilbert-base-uncased
3
- library_name: peft
4
- ---
5
-
6
- # Model Card for Model ID
7
-
8
- <!-- Provide a quick summary of what the model is/does. -->
9
-
10
-
11
-
12
- ## Model Details
13
-
14
- ### Model Description
15
-
16
- <!-- Provide a longer summary of what this model is. -->
17
-
18
-
19
-
20
- - **Developed by:** [More Information Needed]
21
- - **Funded by [optional]:** [More Information Needed]
22
- - **Shared by [optional]:** [More Information Needed]
23
- - **Model type:** [More Information Needed]
24
- - **Language(s) (NLP):** [More Information Needed]
25
- - **License:** [More Information Needed]
26
- - **Finetuned from model [optional]:** [More Information Needed]
27
-
28
- ### Model Sources [optional]
29
-
30
- <!-- Provide the basic links for the model. -->
31
-
32
- - **Repository:** [More Information Needed]
33
- - **Paper [optional]:** [More Information Needed]
34
- - **Demo [optional]:** [More Information Needed]
35
-
36
- ## Uses
37
-
38
- <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
39
-
40
- ### Direct Use
41
-
42
- <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
43
-
44
- [More Information Needed]
45
-
46
- ### Downstream Use [optional]
47
-
48
- <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
49
-
50
- [More Information Needed]
51
-
52
- ### Out-of-Scope Use
53
-
54
- <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
55
-
56
- [More Information Needed]
57
-
58
- ## Bias, Risks, and Limitations
59
-
60
- <!-- This section is meant to convey both technical and sociotechnical limitations. -->
61
-
62
- [More Information Needed]
63
-
64
- ### Recommendations
65
-
66
- <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
67
-
68
- Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
69
-
70
- ## How to Get Started with the Model
71
-
72
- Use the code below to get started with the model.
73
-
74
- [More Information Needed]
75
-
76
- ## Training Details
77
-
78
- ### Training Data
79
-
80
- <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
81
-
82
- [More Information Needed]
83
-
84
- ### Training Procedure
85
-
86
- <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
87
-
88
- #### Preprocessing [optional]
89
-
90
- [More Information Needed]
91
-
92
-
93
- #### Training Hyperparameters
94
-
95
- - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
96
-
97
- #### Speeds, Sizes, Times [optional]
98
-
99
- <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
100
-
101
- [More Information Needed]
102
-
103
- ## Evaluation
104
-
105
- <!-- This section describes the evaluation protocols and provides the results. -->
106
-
107
- ### Testing Data, Factors & Metrics
108
-
109
- #### Testing Data
110
-
111
- <!-- This should link to a Dataset Card if possible. -->
112
-
113
- [More Information Needed]
114
-
115
- #### Factors
116
-
117
- <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
118
-
119
- [More Information Needed]
120
-
121
- #### Metrics
122
-
123
- <!-- These are the evaluation metrics being used, ideally with a description of why. -->
124
-
125
- [More Information Needed]
126
-
127
- ### Results
128
-
129
- [More Information Needed]
130
-
131
- #### Summary
132
-
133
-
134
-
135
- ## Model Examination [optional]
136
-
137
- <!-- Relevant interpretability work for the model goes here -->
138
-
139
- [More Information Needed]
140
-
141
- ## Environmental Impact
142
-
143
- <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
144
-
145
- Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
146
-
147
- - **Hardware Type:** [More Information Needed]
148
- - **Hours used:** [More Information Needed]
149
- - **Cloud Provider:** [More Information Needed]
150
- - **Compute Region:** [More Information Needed]
151
- - **Carbon Emitted:** [More Information Needed]
152
-
153
- ## Technical Specifications [optional]
154
-
155
- ### Model Architecture and Objective
156
-
157
- [More Information Needed]
158
-
159
- ### Compute Infrastructure
160
-
161
- [More Information Needed]
162
-
163
- #### Hardware
164
-
165
- [More Information Needed]
166
-
167
- #### Software
168
-
169
- [More Information Needed]
170
-
171
- ## Citation [optional]
172
-
173
- <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
174
-
175
- **BibTeX:**
176
-
177
- [More Information Needed]
178
-
179
- **APA:**
180
-
181
- [More Information Needed]
182
-
183
- ## Glossary [optional]
184
-
185
- <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
186
-
187
- [More Information Needed]
188
-
189
- ## More Information [optional]
190
-
191
- [More Information Needed]
192
-
193
- ## Model Card Authors [optional]
194
-
195
- [More Information Needed]
196
-
197
- ## Model Card Contact
198
-
199
- [More Information Needed]
200
- ### Framework versions
201
-
202
- - PEFT 0.13.2
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
training/training-checkpoints/checkpoint-1700/adapter_config.json DELETED
@@ -1,32 +0,0 @@
1
- {
2
- "alpha_pattern": {},
3
- "auto_mapping": null,
4
- "base_model_name_or_path": "distilbert-base-uncased",
5
- "bias": "none",
6
- "fan_in_fan_out": false,
7
- "inference_mode": true,
8
- "init_lora_weights": true,
9
- "layer_replication": null,
10
- "layers_pattern": null,
11
- "layers_to_transform": null,
12
- "loftq_config": {},
13
- "lora_alpha": 16,
14
- "lora_dropout": 0.1,
15
- "megatron_config": null,
16
- "megatron_core": "megatron.core",
17
- "modules_to_save": [
18
- "classifier",
19
- "score"
20
- ],
21
- "peft_type": "LORA",
22
- "r": 8,
23
- "rank_pattern": {},
24
- "revision": null,
25
- "target_modules": [
26
- "v_lin",
27
- "q_lin"
28
- ],
29
- "task_type": "SEQ_CLS",
30
- "use_dora": false,
31
- "use_rslora": false
32
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
training/training-checkpoints/checkpoint-1700/adapter_model.safetensors DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:0b52ac92aa40a1961914c95fecbf881e70122a9d3ebeee62134c2225bddd1d3a
3
- size 2962192
 
 
 
 
training/training-checkpoints/checkpoint-1700/optimizer.pt DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:729b1404d5313567c877b13e089e0ea2800af9b90b5da1641dd58d50f6710905
3
- size 5940315
 
 
 
 
training/training-checkpoints/checkpoint-1700/rng_state.pth DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:5b8eeb2605876a3277ab8cfa76859ef162300347ed099232fc6d280a4d0ca430
3
- size 14455
 
 
 
 
training/training-checkpoints/checkpoint-1700/scheduler.pt DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:0e0ff435ef12a33c4c275d84760be406df49508c4b1bcebad4951172631c2c9b
3
- size 1465
 
 
 
 
training/training-checkpoints/checkpoint-1700/tokenizer.json DELETED
The diff for this file is too large to render. See raw diff
 
training/training-checkpoints/checkpoint-1700/tokenizer_config.json DELETED
@@ -1,15 +0,0 @@
1
- {
2
- "backend": "tokenizers",
3
- "cls_token": "[CLS]",
4
- "do_lower_case": true,
5
- "is_local": false,
6
- "local_files_only": false,
7
- "mask_token": "[MASK]",
8
- "model_max_length": 512,
9
- "pad_token": "[PAD]",
10
- "sep_token": "[SEP]",
11
- "strip_accents": null,
12
- "tokenize_chinese_chars": true,
13
- "tokenizer_class": "BertTokenizer",
14
- "unk_token": "[UNK]"
15
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
training/training-checkpoints/checkpoint-1700/trainer_state.json DELETED
@@ -1,1233 +0,0 @@
1
- {
2
- "best_global_step": 1700,
3
- "best_metric": 0.9833333333333333,
4
- "best_model_checkpoint": "./training-checkpoints\\checkpoint-1700",
5
- "epoch": 1.0,
6
- "eval_steps": 500,
7
- "global_step": 1700,
8
- "is_hyper_param_search": false,
9
- "is_local_process_zero": true,
10
- "is_world_process_zero": true,
11
- "log_history": [
12
- {
13
- "epoch": 0.0058823529411764705,
14
- "grad_norm": 1.510934829711914,
15
- "learning_rate": 0.00019964705882352944,
16
- "loss": 0.6532491683959961,
17
- "step": 10
18
- },
19
- {
20
- "epoch": 0.011764705882352941,
21
- "grad_norm": 2.264005661010742,
22
- "learning_rate": 0.0001992549019607843,
23
- "loss": 0.49631505012512206,
24
- "step": 20
25
- },
26
- {
27
- "epoch": 0.01764705882352941,
28
- "grad_norm": 1.0282903909683228,
29
- "learning_rate": 0.0001988627450980392,
30
- "loss": 0.29676687717437744,
31
- "step": 30
32
- },
33
- {
34
- "epoch": 0.023529411764705882,
35
- "grad_norm": 1.0084155797958374,
36
- "learning_rate": 0.00019847058823529412,
37
- "loss": 0.1594167470932007,
38
- "step": 40
39
- },
40
- {
41
- "epoch": 0.029411764705882353,
42
- "grad_norm": 0.23450879752635956,
43
- "learning_rate": 0.00019807843137254903,
44
- "loss": 0.09263747334480285,
45
- "step": 50
46
- },
47
- {
48
- "epoch": 0.03529411764705882,
49
- "grad_norm": 0.44396597146987915,
50
- "learning_rate": 0.00019768627450980392,
51
- "loss": 0.18005683422088622,
52
- "step": 60
53
- },
54
- {
55
- "epoch": 0.041176470588235294,
56
- "grad_norm": 0.1429290622472763,
57
- "learning_rate": 0.00019729411764705883,
58
- "loss": 0.08807324171066284,
59
- "step": 70
60
- },
61
- {
62
- "epoch": 0.047058823529411764,
63
- "grad_norm": 0.847214937210083,
64
- "learning_rate": 0.00019690196078431374,
65
- "loss": 0.058199501037597655,
66
- "step": 80
67
- },
68
- {
69
- "epoch": 0.052941176470588235,
70
- "grad_norm": 3.822068929672241,
71
- "learning_rate": 0.00019650980392156862,
72
- "loss": 0.1674056887626648,
73
- "step": 90
74
- },
75
- {
76
- "epoch": 0.058823529411764705,
77
- "grad_norm": 1.4017611742019653,
78
- "learning_rate": 0.00019611764705882353,
79
- "loss": 0.11085774898529052,
80
- "step": 100
81
- },
82
- {
83
- "epoch": 0.06470588235294118,
84
- "grad_norm": 1.6271415948867798,
85
- "learning_rate": 0.00019572549019607844,
86
- "loss": 0.10848402976989746,
87
- "step": 110
88
- },
89
- {
90
- "epoch": 0.07058823529411765,
91
- "grad_norm": 0.17168374359607697,
92
- "learning_rate": 0.00019533333333333336,
93
- "loss": 0.08450813293457031,
94
- "step": 120
95
- },
96
- {
97
- "epoch": 0.07647058823529412,
98
- "grad_norm": 2.110332489013672,
99
- "learning_rate": 0.00019494117647058824,
100
- "loss": 0.08588783144950866,
101
- "step": 130
102
- },
103
- {
104
- "epoch": 0.08235294117647059,
105
- "grad_norm": 1.973347783088684,
106
- "learning_rate": 0.00019454901960784315,
107
- "loss": 0.07138832211494446,
108
- "step": 140
109
- },
110
- {
111
- "epoch": 0.08823529411764706,
112
- "grad_norm": 2.4691383838653564,
113
- "learning_rate": 0.00019415686274509806,
114
- "loss": 0.1480124831199646,
115
- "step": 150
116
- },
117
- {
118
- "epoch": 0.09411764705882353,
119
- "grad_norm": 0.8067235946655273,
120
- "learning_rate": 0.00019376470588235295,
121
- "loss": 0.03993956744670868,
122
- "step": 160
123
- },
124
- {
125
- "epoch": 0.1,
126
- "grad_norm": 2.022461175918579,
127
- "learning_rate": 0.00019337254901960786,
128
- "loss": 0.06878547668457032,
129
- "step": 170
130
- },
131
- {
132
- "epoch": 0.10588235294117647,
133
- "grad_norm": 1.4254077672958374,
134
- "learning_rate": 0.00019298039215686274,
135
- "loss": 0.12785714864730835,
136
- "step": 180
137
- },
138
- {
139
- "epoch": 0.11176470588235295,
140
- "grad_norm": 3.5772814750671387,
141
- "learning_rate": 0.00019258823529411765,
142
- "loss": 0.07902452945709229,
143
- "step": 190
144
- },
145
- {
146
- "epoch": 0.11764705882352941,
147
- "grad_norm": 0.06748726963996887,
148
- "learning_rate": 0.00019219607843137256,
149
- "loss": 0.06560993194580078,
150
- "step": 200
151
- },
152
- {
153
- "epoch": 0.12352941176470589,
154
- "grad_norm": 5.759597301483154,
155
- "learning_rate": 0.00019180392156862745,
156
- "loss": 0.16235246658325195,
157
- "step": 210
158
- },
159
- {
160
- "epoch": 0.12941176470588237,
161
- "grad_norm": 3.6009647846221924,
162
- "learning_rate": 0.00019141176470588236,
163
- "loss": 0.06680867075920105,
164
- "step": 220
165
- },
166
- {
167
- "epoch": 0.13529411764705881,
168
- "grad_norm": 2.608488082885742,
169
- "learning_rate": 0.00019101960784313727,
170
- "loss": 0.08542757034301758,
171
- "step": 230
172
- },
173
- {
174
- "epoch": 0.1411764705882353,
175
- "grad_norm": 1.4347050189971924,
176
- "learning_rate": 0.00019062745098039215,
177
- "loss": 0.06062295436859131,
178
- "step": 240
179
- },
180
- {
181
- "epoch": 0.14705882352941177,
182
- "grad_norm": 3.4521889686584473,
183
- "learning_rate": 0.00019023529411764706,
184
- "loss": 0.11894316673278808,
185
- "step": 250
186
- },
187
- {
188
- "epoch": 0.15294117647058825,
189
- "grad_norm": 2.5669424533843994,
190
- "learning_rate": 0.00018984313725490197,
191
- "loss": 0.04931341111660004,
192
- "step": 260
193
- },
194
- {
195
- "epoch": 0.1588235294117647,
196
- "grad_norm": 0.5097957253456116,
197
- "learning_rate": 0.00018945098039215689,
198
- "loss": 0.0895132601261139,
199
- "step": 270
200
- },
201
- {
202
- "epoch": 0.16470588235294117,
203
- "grad_norm": 0.1470881551504135,
204
- "learning_rate": 0.00018905882352941177,
205
- "loss": 0.07752076387405396,
206
- "step": 280
207
- },
208
- {
209
- "epoch": 0.17058823529411765,
210
- "grad_norm": 0.12387822568416595,
211
- "learning_rate": 0.00018866666666666668,
212
- "loss": 0.0910811185836792,
213
- "step": 290
214
- },
215
- {
216
- "epoch": 0.17647058823529413,
217
- "grad_norm": 0.1579340398311615,
218
- "learning_rate": 0.0001882745098039216,
219
- "loss": 0.024501879513263703,
220
- "step": 300
221
- },
222
- {
223
- "epoch": 0.18235294117647058,
224
- "grad_norm": 0.19159211218357086,
225
- "learning_rate": 0.00018788235294117648,
226
- "loss": 0.025437775254249572,
227
- "step": 310
228
- },
229
- {
230
- "epoch": 0.18823529411764706,
231
- "grad_norm": 0.3070846199989319,
232
- "learning_rate": 0.0001874901960784314,
233
- "loss": 0.0452291190624237,
234
- "step": 320
235
- },
236
- {
237
- "epoch": 0.19411764705882353,
238
- "grad_norm": 1.647754192352295,
239
- "learning_rate": 0.0001870980392156863,
240
- "loss": 0.07508809566497802,
241
- "step": 330
242
- },
243
- {
244
- "epoch": 0.2,
245
- "grad_norm": 0.2747861444950104,
246
- "learning_rate": 0.00018670588235294118,
247
- "loss": 0.08880412578582764,
248
- "step": 340
249
- },
250
- {
251
- "epoch": 0.20588235294117646,
252
- "grad_norm": 0.20420950651168823,
253
- "learning_rate": 0.00018631372549019607,
254
- "loss": 0.047925958037376405,
255
- "step": 350
256
- },
257
- {
258
- "epoch": 0.21176470588235294,
259
- "grad_norm": 0.5672637820243835,
260
- "learning_rate": 0.00018592156862745098,
261
- "loss": 0.04176274836063385,
262
- "step": 360
263
- },
264
- {
265
- "epoch": 0.21764705882352942,
266
- "grad_norm": 1.5938951969146729,
267
- "learning_rate": 0.0001855294117647059,
268
- "loss": 0.06371157765388488,
269
- "step": 370
270
- },
271
- {
272
- "epoch": 0.2235294117647059,
273
- "grad_norm": 0.019027238711714745,
274
- "learning_rate": 0.0001851372549019608,
275
- "loss": 0.05675685405731201,
276
- "step": 380
277
- },
278
- {
279
- "epoch": 0.22941176470588234,
280
- "grad_norm": 1.8165510892868042,
281
- "learning_rate": 0.00018474509803921568,
282
- "loss": 0.10704467296600342,
283
- "step": 390
284
- },
285
- {
286
- "epoch": 0.23529411764705882,
287
- "grad_norm": 0.3322693109512329,
288
- "learning_rate": 0.0001843529411764706,
289
- "loss": 0.08637534379959107,
290
- "step": 400
291
- },
292
- {
293
- "epoch": 0.2411764705882353,
294
- "grad_norm": 0.06156998872756958,
295
- "learning_rate": 0.0001839607843137255,
296
- "loss": 0.17954374551773072,
297
- "step": 410
298
- },
299
- {
300
- "epoch": 0.24705882352941178,
301
- "grad_norm": 3.327615737915039,
302
- "learning_rate": 0.0001835686274509804,
303
- "loss": 0.1696428656578064,
304
- "step": 420
305
- },
306
- {
307
- "epoch": 0.2529411764705882,
308
- "grad_norm": 2.1997804641723633,
309
- "learning_rate": 0.0001831764705882353,
310
- "loss": 0.11364388465881348,
311
- "step": 430
312
- },
313
- {
314
- "epoch": 0.25882352941176473,
315
- "grad_norm": 2.8368852138519287,
316
- "learning_rate": 0.0001827843137254902,
317
- "loss": 0.03553434014320374,
318
- "step": 440
319
- },
320
- {
321
- "epoch": 0.2647058823529412,
322
- "grad_norm": 0.2659618556499481,
323
- "learning_rate": 0.00018239215686274512,
324
- "loss": 0.03715018928050995,
325
- "step": 450
326
- },
327
- {
328
- "epoch": 0.27058823529411763,
329
- "grad_norm": 0.04204031080007553,
330
- "learning_rate": 0.000182,
331
- "loss": 0.13120421171188354,
332
- "step": 460
333
- },
334
- {
335
- "epoch": 0.27647058823529413,
336
- "grad_norm": 3.284900665283203,
337
- "learning_rate": 0.00018160784313725492,
338
- "loss": 0.08410873413085937,
339
- "step": 470
340
- },
341
- {
342
- "epoch": 0.2823529411764706,
343
- "grad_norm": 1.841749906539917,
344
- "learning_rate": 0.00018121568627450983,
345
- "loss": 0.056668424606323244,
346
- "step": 480
347
- },
348
- {
349
- "epoch": 0.28823529411764703,
350
- "grad_norm": 0.03970719501376152,
351
- "learning_rate": 0.0001808235294117647,
352
- "loss": 0.043222978711128235,
353
- "step": 490
354
- },
355
- {
356
- "epoch": 0.29411764705882354,
357
- "grad_norm": 0.17564676702022552,
358
- "learning_rate": 0.0001804313725490196,
359
- "loss": 0.029093235731124878,
360
- "step": 500
361
- },
362
- {
363
- "epoch": 0.3,
364
- "grad_norm": 2.7894046306610107,
365
- "learning_rate": 0.0001800392156862745,
366
- "loss": 0.18836088180541993,
367
- "step": 510
368
- },
369
- {
370
- "epoch": 0.3058823529411765,
371
- "grad_norm": 0.015540416352450848,
372
- "learning_rate": 0.00017964705882352942,
373
- "loss": 0.04498102068901062,
374
- "step": 520
375
- },
376
- {
377
- "epoch": 0.31176470588235294,
378
- "grad_norm": 1.4599756002426147,
379
- "learning_rate": 0.00017925490196078433,
380
- "loss": 0.06498492360115052,
381
- "step": 530
382
- },
383
- {
384
- "epoch": 0.3176470588235294,
385
- "grad_norm": 0.03984140232205391,
386
- "learning_rate": 0.0001788627450980392,
387
- "loss": 0.06558265089988709,
388
- "step": 540
389
- },
390
- {
391
- "epoch": 0.3235294117647059,
392
- "grad_norm": 2.5549182891845703,
393
- "learning_rate": 0.00017847058823529412,
394
- "loss": 0.02280822992324829,
395
- "step": 550
396
- },
397
- {
398
- "epoch": 0.32941176470588235,
399
- "grad_norm": 0.31564125418663025,
400
- "learning_rate": 0.00017807843137254903,
401
- "loss": 0.05171017646789551,
402
- "step": 560
403
- },
404
- {
405
- "epoch": 0.3352941176470588,
406
- "grad_norm": 1.2470703125,
407
- "learning_rate": 0.00017768627450980392,
408
- "loss": 0.05321676135063171,
409
- "step": 570
410
- },
411
- {
412
- "epoch": 0.3411764705882353,
413
- "grad_norm": 0.19033531844615936,
414
- "learning_rate": 0.00017729411764705883,
415
- "loss": 0.14463623762130737,
416
- "step": 580
417
- },
418
- {
419
- "epoch": 0.34705882352941175,
420
- "grad_norm": 2.2864792346954346,
421
- "learning_rate": 0.00017690196078431374,
422
- "loss": 0.06809618473052978,
423
- "step": 590
424
- },
425
- {
426
- "epoch": 0.35294117647058826,
427
- "grad_norm": 1.959888219833374,
428
- "learning_rate": 0.00017650980392156865,
429
- "loss": 0.03620869815349579,
430
- "step": 600
431
- },
432
- {
433
- "epoch": 0.3588235294117647,
434
- "grad_norm": 2.4784562587738037,
435
- "learning_rate": 0.00017611764705882354,
436
- "loss": 0.02364923506975174,
437
- "step": 610
438
- },
439
- {
440
- "epoch": 0.36470588235294116,
441
- "grad_norm": 0.02134384773671627,
442
- "learning_rate": 0.00017572549019607845,
443
- "loss": 0.09724082350730896,
444
- "step": 620
445
- },
446
- {
447
- "epoch": 0.37058823529411766,
448
- "grad_norm": 0.06821486353874207,
449
- "learning_rate": 0.00017533333333333336,
450
- "loss": 0.06447332501411437,
451
- "step": 630
452
- },
453
- {
454
- "epoch": 0.3764705882352941,
455
- "grad_norm": 0.6358501315116882,
456
- "learning_rate": 0.00017494117647058824,
457
- "loss": 0.06500619649887085,
458
- "step": 640
459
- },
460
- {
461
- "epoch": 0.38235294117647056,
462
- "grad_norm": 3.4506707191467285,
463
- "learning_rate": 0.00017454901960784315,
464
- "loss": 0.08725794553756713,
465
- "step": 650
466
- },
467
- {
468
- "epoch": 0.38823529411764707,
469
- "grad_norm": 0.22893691062927246,
470
- "learning_rate": 0.00017415686274509804,
471
- "loss": 0.10881921052932739,
472
- "step": 660
473
- },
474
- {
475
- "epoch": 0.3941176470588235,
476
- "grad_norm": 0.10322771966457367,
477
- "learning_rate": 0.00017376470588235295,
478
- "loss": 0.03397703766822815,
479
- "step": 670
480
- },
481
- {
482
- "epoch": 0.4,
483
- "grad_norm": 2.8205652236938477,
484
- "learning_rate": 0.00017337254901960783,
485
- "loss": 0.027051010727882387,
486
- "step": 680
487
- },
488
- {
489
- "epoch": 0.40588235294117647,
490
- "grad_norm": 0.013308421708643436,
491
- "learning_rate": 0.00017298039215686274,
492
- "loss": 0.04930915534496307,
493
- "step": 690
494
- },
495
- {
496
- "epoch": 0.4117647058823529,
497
- "grad_norm": 3.44494891166687,
498
- "learning_rate": 0.00017258823529411765,
499
- "loss": 0.07341324090957642,
500
- "step": 700
501
- },
502
- {
503
- "epoch": 0.4176470588235294,
504
- "grad_norm": 2.4105072021484375,
505
- "learning_rate": 0.00017219607843137256,
506
- "loss": 0.05853899717330933,
507
- "step": 710
508
- },
509
- {
510
- "epoch": 0.4235294117647059,
511
- "grad_norm": 4.644453048706055,
512
- "learning_rate": 0.00017180392156862745,
513
- "loss": 0.12287142276763915,
514
- "step": 720
515
- },
516
- {
517
- "epoch": 0.4294117647058823,
518
- "grad_norm": 0.38577258586883545,
519
- "learning_rate": 0.00017141176470588236,
520
- "loss": 0.06102405190467834,
521
- "step": 730
522
- },
523
- {
524
- "epoch": 0.43529411764705883,
525
- "grad_norm": 0.07725336402654648,
526
- "learning_rate": 0.00017101960784313727,
527
- "loss": 0.047857701778411865,
528
- "step": 740
529
- },
530
- {
531
- "epoch": 0.4411764705882353,
532
- "grad_norm": 0.06397025287151337,
533
- "learning_rate": 0.00017062745098039215,
534
- "loss": 0.09663112759590149,
535
- "step": 750
536
- },
537
- {
538
- "epoch": 0.4470588235294118,
539
- "grad_norm": 3.3719794750213623,
540
- "learning_rate": 0.00017023529411764707,
541
- "loss": 0.09325426816940308,
542
- "step": 760
543
- },
544
- {
545
- "epoch": 0.45294117647058824,
546
- "grad_norm": 0.46245869994163513,
547
- "learning_rate": 0.00016984313725490198,
548
- "loss": 0.08240602612495422,
549
- "step": 770
550
- },
551
- {
552
- "epoch": 0.4588235294117647,
553
- "grad_norm": 0.032528385519981384,
554
- "learning_rate": 0.0001694509803921569,
555
- "loss": 0.050741398334503175,
556
- "step": 780
557
- },
558
- {
559
- "epoch": 0.4647058823529412,
560
- "grad_norm": 0.24650231003761292,
561
- "learning_rate": 0.00016905882352941177,
562
- "loss": 0.04027060270309448,
563
- "step": 790
564
- },
565
- {
566
- "epoch": 0.47058823529411764,
567
- "grad_norm": 0.12963470816612244,
568
- "learning_rate": 0.00016866666666666668,
569
- "loss": 0.05930449962615967,
570
- "step": 800
571
- },
572
- {
573
- "epoch": 0.4764705882352941,
574
- "grad_norm": 0.4593453109264374,
575
- "learning_rate": 0.0001682745098039216,
576
- "loss": 0.018743452429771424,
577
- "step": 810
578
- },
579
- {
580
- "epoch": 0.4823529411764706,
581
- "grad_norm": 2.126039981842041,
582
- "learning_rate": 0.00016788235294117648,
583
- "loss": 0.014739298820495605,
584
- "step": 820
585
- },
586
- {
587
- "epoch": 0.48823529411764705,
588
- "grad_norm": 0.1071607917547226,
589
- "learning_rate": 0.00016749019607843136,
590
- "loss": 0.03930833339691162,
591
- "step": 830
592
- },
593
- {
594
- "epoch": 0.49411764705882355,
595
- "grad_norm": 1.272904634475708,
596
- "learning_rate": 0.00016709803921568627,
597
- "loss": 0.01983778029680252,
598
- "step": 840
599
- },
600
- {
601
- "epoch": 0.5,
602
- "grad_norm": 4.284426212310791,
603
- "learning_rate": 0.00016670588235294118,
604
- "loss": 0.05801863074302673,
605
- "step": 850
606
- },
607
- {
608
- "epoch": 0.5058823529411764,
609
- "grad_norm": 0.21516184508800507,
610
- "learning_rate": 0.0001663137254901961,
611
- "loss": 0.0344509094953537,
612
- "step": 860
613
- },
614
- {
615
- "epoch": 0.5117647058823529,
616
- "grad_norm": 0.05401306226849556,
617
- "learning_rate": 0.00016592156862745098,
618
- "loss": 0.1013040542602539,
619
- "step": 870
620
- },
621
- {
622
- "epoch": 0.5176470588235295,
623
- "grad_norm": 2.2851767539978027,
624
- "learning_rate": 0.0001655294117647059,
625
- "loss": 0.06278380155563354,
626
- "step": 880
627
- },
628
- {
629
- "epoch": 0.5235294117647059,
630
- "grad_norm": 2.387756586074829,
631
- "learning_rate": 0.0001651372549019608,
632
- "loss": 0.1075211524963379,
633
- "step": 890
634
- },
635
- {
636
- "epoch": 0.5294117647058824,
637
- "grad_norm": 0.6298794150352478,
638
- "learning_rate": 0.00016474509803921568,
639
- "loss": 0.04051249623298645,
640
- "step": 900
641
- },
642
- {
643
- "epoch": 0.5352941176470588,
644
- "grad_norm": 3.3592398166656494,
645
- "learning_rate": 0.0001643529411764706,
646
- "loss": 0.048895838856697085,
647
- "step": 910
648
- },
649
- {
650
- "epoch": 0.5411764705882353,
651
- "grad_norm": 0.004421881400048733,
652
- "learning_rate": 0.0001639607843137255,
653
- "loss": 0.06308169960975647,
654
- "step": 920
655
- },
656
- {
657
- "epoch": 0.5470588235294118,
658
- "grad_norm": 0.09252121299505234,
659
- "learning_rate": 0.00016356862745098042,
660
- "loss": 0.058104753494262695,
661
- "step": 930
662
- },
663
- {
664
- "epoch": 0.5529411764705883,
665
- "grad_norm": 0.7001014351844788,
666
- "learning_rate": 0.0001631764705882353,
667
- "loss": 0.021149225533008575,
668
- "step": 940
669
- },
670
- {
671
- "epoch": 0.5588235294117647,
672
- "grad_norm": 0.3170143663883209,
673
- "learning_rate": 0.0001627843137254902,
674
- "loss": 0.01627269834280014,
675
- "step": 950
676
- },
677
- {
678
- "epoch": 0.5647058823529412,
679
- "grad_norm": 3.7102761268615723,
680
- "learning_rate": 0.00016239215686274512,
681
- "loss": 0.14663417339324952,
682
- "step": 960
683
- },
684
- {
685
- "epoch": 0.5705882352941176,
686
- "grad_norm": 0.006750724744051695,
687
- "learning_rate": 0.000162,
688
- "loss": 0.10339630842208862,
689
- "step": 970
690
- },
691
- {
692
- "epoch": 0.5764705882352941,
693
- "grad_norm": 1.808908462524414,
694
- "learning_rate": 0.0001616078431372549,
695
- "loss": 0.026013880968093872,
696
- "step": 980
697
- },
698
- {
699
- "epoch": 0.5823529411764706,
700
- "grad_norm": 0.4191270172595978,
701
- "learning_rate": 0.0001612156862745098,
702
- "loss": 0.039347043633461,
703
- "step": 990
704
- },
705
- {
706
- "epoch": 0.5882352941176471,
707
- "grad_norm": 0.3325525224208832,
708
- "learning_rate": 0.0001608235294117647,
709
- "loss": 0.059603160619735716,
710
- "step": 1000
711
- },
712
- {
713
- "epoch": 0.5941176470588235,
714
- "grad_norm": 2.7802844047546387,
715
- "learning_rate": 0.0001604313725490196,
716
- "loss": 0.09034895300865173,
717
- "step": 1010
718
- },
719
- {
720
- "epoch": 0.6,
721
- "grad_norm": 0.04968420788645744,
722
- "learning_rate": 0.0001600392156862745,
723
- "loss": 0.05755581855773926,
724
- "step": 1020
725
- },
726
- {
727
- "epoch": 0.6058823529411764,
728
- "grad_norm": 0.00875788927078247,
729
- "learning_rate": 0.00015964705882352942,
730
- "loss": 0.08298594951629638,
731
- "step": 1030
732
- },
733
- {
734
- "epoch": 0.611764705882353,
735
- "grad_norm": 0.38597434759140015,
736
- "learning_rate": 0.00015925490196078433,
737
- "loss": 0.009630383551120758,
738
- "step": 1040
739
- },
740
- {
741
- "epoch": 0.6176470588235294,
742
- "grad_norm": 0.42396023869514465,
743
- "learning_rate": 0.00015886274509803921,
744
- "loss": 0.05784105658531189,
745
- "step": 1050
746
- },
747
- {
748
- "epoch": 0.6235294117647059,
749
- "grad_norm": 3.5692520141601562,
750
- "learning_rate": 0.00015847058823529413,
751
- "loss": 0.1575460195541382,
752
- "step": 1060
753
- },
754
- {
755
- "epoch": 0.6294117647058823,
756
- "grad_norm": 0.12411337345838547,
757
- "learning_rate": 0.00015807843137254904,
758
- "loss": 0.052008318901062014,
759
- "step": 1070
760
- },
761
- {
762
- "epoch": 0.6352941176470588,
763
- "grad_norm": 0.12469016760587692,
764
- "learning_rate": 0.00015768627450980392,
765
- "loss": 0.06983086466789246,
766
- "step": 1080
767
- },
768
- {
769
- "epoch": 0.6411764705882353,
770
- "grad_norm": 0.0010811894899234176,
771
- "learning_rate": 0.00015729411764705883,
772
- "loss": 0.009204772859811782,
773
- "step": 1090
774
- },
775
- {
776
- "epoch": 0.6470588235294118,
777
- "grad_norm": 0.7795768976211548,
778
- "learning_rate": 0.00015690196078431374,
779
- "loss": 0.07802544236183166,
780
- "step": 1100
781
- },
782
- {
783
- "epoch": 0.6529411764705882,
784
- "grad_norm": 3.0475451946258545,
785
- "learning_rate": 0.00015650980392156865,
786
- "loss": 0.06048353910446167,
787
- "step": 1110
788
- },
789
- {
790
- "epoch": 0.6588235294117647,
791
- "grad_norm": 0.054235588759183884,
792
- "learning_rate": 0.00015611764705882354,
793
- "loss": 0.007759802043437958,
794
- "step": 1120
795
- },
796
- {
797
- "epoch": 0.6647058823529411,
798
- "grad_norm": 0.022670522332191467,
799
- "learning_rate": 0.00015572549019607845,
800
- "loss": 0.121265709400177,
801
- "step": 1130
802
- },
803
- {
804
- "epoch": 0.6705882352941176,
805
- "grad_norm": 4.404564380645752,
806
- "learning_rate": 0.00015533333333333333,
807
- "loss": 0.11734864711761475,
808
- "step": 1140
809
- },
810
- {
811
- "epoch": 0.6764705882352942,
812
- "grad_norm": 0.04260774329304695,
813
- "learning_rate": 0.00015494117647058824,
814
- "loss": 0.04804904460906982,
815
- "step": 1150
816
- },
817
- {
818
- "epoch": 0.6823529411764706,
819
- "grad_norm": 0.37849345803260803,
820
- "learning_rate": 0.00015454901960784313,
821
- "loss": 0.08949239253997802,
822
- "step": 1160
823
- },
824
- {
825
- "epoch": 0.6882352941176471,
826
- "grad_norm": 0.07910435646772385,
827
- "learning_rate": 0.00015415686274509804,
828
- "loss": 0.11236823797225952,
829
- "step": 1170
830
- },
831
- {
832
- "epoch": 0.6941176470588235,
833
- "grad_norm": 0.020735427737236023,
834
- "learning_rate": 0.00015376470588235295,
835
- "loss": 0.020005679130554198,
836
- "step": 1180
837
- },
838
- {
839
- "epoch": 0.7,
840
- "grad_norm": 0.6041317582130432,
841
- "learning_rate": 0.00015337254901960786,
842
- "loss": 0.047528034448623656,
843
- "step": 1190
844
- },
845
- {
846
- "epoch": 0.7058823529411765,
847
- "grad_norm": 0.07179152965545654,
848
- "learning_rate": 0.00015298039215686274,
849
- "loss": 0.03959411382675171,
850
- "step": 1200
851
- },
852
- {
853
- "epoch": 0.711764705882353,
854
- "grad_norm": 0.009498389437794685,
855
- "learning_rate": 0.00015258823529411766,
856
- "loss": 0.003311210125684738,
857
- "step": 1210
858
- },
859
- {
860
- "epoch": 0.7176470588235294,
861
- "grad_norm": 0.433115154504776,
862
- "learning_rate": 0.00015219607843137257,
863
- "loss": 0.00858706161379814,
864
- "step": 1220
865
- },
866
- {
867
- "epoch": 0.7235294117647059,
868
- "grad_norm": 0.008886504918336868,
869
- "learning_rate": 0.00015180392156862745,
870
- "loss": 0.06958861947059632,
871
- "step": 1230
872
- },
873
- {
874
- "epoch": 0.7294117647058823,
875
- "grad_norm": 0.008790708146989346,
876
- "learning_rate": 0.00015141176470588236,
877
- "loss": 0.008029435575008393,
878
- "step": 1240
879
- },
880
- {
881
- "epoch": 0.7352941176470589,
882
- "grad_norm": 0.005836977623403072,
883
- "learning_rate": 0.00015101960784313727,
884
- "loss": 0.05358666181564331,
885
- "step": 1250
886
- },
887
- {
888
- "epoch": 0.7411764705882353,
889
- "grad_norm": 0.14504218101501465,
890
- "learning_rate": 0.00015062745098039218,
891
- "loss": 0.049011257290840146,
892
- "step": 1260
893
- },
894
- {
895
- "epoch": 0.7470588235294118,
896
- "grad_norm": 0.2803121507167816,
897
- "learning_rate": 0.00015023529411764707,
898
- "loss": 0.006191087141633034,
899
- "step": 1270
900
- },
901
- {
902
- "epoch": 0.7529411764705882,
903
- "grad_norm": 3.1994972229003906,
904
- "learning_rate": 0.00014984313725490198,
905
- "loss": 0.10595870018005371,
906
- "step": 1280
907
- },
908
- {
909
- "epoch": 0.7588235294117647,
910
- "grad_norm": 0.00779637461528182,
911
- "learning_rate": 0.0001494509803921569,
912
- "loss": 0.07793840169906616,
913
- "step": 1290
914
- },
915
- {
916
- "epoch": 0.7647058823529411,
917
- "grad_norm": 0.04689478874206543,
918
- "learning_rate": 0.00014905882352941177,
919
- "loss": 0.08080050349235535,
920
- "step": 1300
921
- },
922
- {
923
- "epoch": 0.7705882352941177,
924
- "grad_norm": 0.9595724940299988,
925
- "learning_rate": 0.00014866666666666666,
926
- "loss": 0.11690077781677247,
927
- "step": 1310
928
- },
929
- {
930
- "epoch": 0.7764705882352941,
931
- "grad_norm": 4.141428470611572,
932
- "learning_rate": 0.00014827450980392157,
933
- "loss": 0.05245780348777771,
934
- "step": 1320
935
- },
936
- {
937
- "epoch": 0.7823529411764706,
938
- "grad_norm": 0.7606799602508545,
939
- "learning_rate": 0.00014788235294117648,
940
- "loss": 0.039501425623893735,
941
- "step": 1330
942
- },
943
- {
944
- "epoch": 0.788235294117647,
945
- "grad_norm": 2.8373472690582275,
946
- "learning_rate": 0.00014749019607843136,
947
- "loss": 0.08525887727737427,
948
- "step": 1340
949
- },
950
- {
951
- "epoch": 0.7941176470588235,
952
- "grad_norm": 0.20202526450157166,
953
- "learning_rate": 0.00014709803921568627,
954
- "loss": 0.03491775095462799,
955
- "step": 1350
956
- },
957
- {
958
- "epoch": 0.8,
959
- "grad_norm": 0.00345594366081059,
960
- "learning_rate": 0.00014670588235294119,
961
- "loss": 0.03784347176551819,
962
- "step": 1360
963
- },
964
- {
965
- "epoch": 0.8058823529411765,
966
- "grad_norm": 3.3252758979797363,
967
- "learning_rate": 0.0001463137254901961,
968
- "loss": 0.08734545707702637,
969
- "step": 1370
970
- },
971
- {
972
- "epoch": 0.8117647058823529,
973
- "grad_norm": 1.1744837760925293,
974
- "learning_rate": 0.00014592156862745098,
975
- "loss": 0.060222625732421875,
976
- "step": 1380
977
- },
978
- {
979
- "epoch": 0.8176470588235294,
980
- "grad_norm": 0.012418321333825588,
981
- "learning_rate": 0.0001455294117647059,
982
- "loss": 0.034251919388771056,
983
- "step": 1390
984
- },
985
- {
986
- "epoch": 0.8235294117647058,
987
- "grad_norm": 0.18622326850891113,
988
- "learning_rate": 0.0001451372549019608,
989
- "loss": 0.02356344759464264,
990
- "step": 1400
991
- },
992
- {
993
- "epoch": 0.8294117647058824,
994
- "grad_norm": 3.9867045879364014,
995
- "learning_rate": 0.00014474509803921569,
996
- "loss": 0.07036476135253907,
997
- "step": 1410
998
- },
999
- {
1000
- "epoch": 0.8352941176470589,
1001
- "grad_norm": 3.0563607215881348,
1002
- "learning_rate": 0.0001443529411764706,
1003
- "loss": 0.037895727157592776,
1004
- "step": 1420
1005
- },
1006
- {
1007
- "epoch": 0.8411764705882353,
1008
- "grad_norm": 0.13856367766857147,
1009
- "learning_rate": 0.0001439607843137255,
1010
- "loss": 0.12252027988433838,
1011
- "step": 1430
1012
- },
1013
- {
1014
- "epoch": 0.8470588235294118,
1015
- "grad_norm": 6.263605117797852,
1016
- "learning_rate": 0.00014356862745098042,
1017
- "loss": 0.04287775456905365,
1018
- "step": 1440
1019
- },
1020
- {
1021
- "epoch": 0.8529411764705882,
1022
- "grad_norm": 0.6084681749343872,
1023
- "learning_rate": 0.0001431764705882353,
1024
- "loss": 0.050480049848556516,
1025
- "step": 1450
1026
- },
1027
- {
1028
- "epoch": 0.8588235294117647,
1029
- "grad_norm": 0.13102763891220093,
1030
- "learning_rate": 0.0001427843137254902,
1031
- "loss": 0.027205181121826173,
1032
- "step": 1460
1033
- },
1034
- {
1035
- "epoch": 0.8647058823529412,
1036
- "grad_norm": 0.0334654301404953,
1037
- "learning_rate": 0.0001423921568627451,
1038
- "loss": 0.07116500139236451,
1039
- "step": 1470
1040
- },
1041
- {
1042
- "epoch": 0.8705882352941177,
1043
- "grad_norm": 5.854252815246582,
1044
- "learning_rate": 0.000142,
1045
- "loss": 0.10154651403427124,
1046
- "step": 1480
1047
- },
1048
- {
1049
- "epoch": 0.8764705882352941,
1050
- "grad_norm": 0.04299909248948097,
1051
- "learning_rate": 0.0001416078431372549,
1052
- "loss": 0.06952143907546997,
1053
- "step": 1490
1054
- },
1055
- {
1056
- "epoch": 0.8823529411764706,
1057
- "grad_norm": 0.03385253995656967,
1058
- "learning_rate": 0.0001412156862745098,
1059
- "loss": 0.042281192541122434,
1060
- "step": 1500
1061
- },
1062
- {
1063
- "epoch": 0.888235294117647,
1064
- "grad_norm": 3.0850558280944824,
1065
- "learning_rate": 0.00014082352941176472,
1066
- "loss": 0.04330833852291107,
1067
- "step": 1510
1068
- },
1069
- {
1070
- "epoch": 0.8941176470588236,
1071
- "grad_norm": 1.0124292373657227,
1072
- "learning_rate": 0.00014043137254901963,
1073
- "loss": 0.013002160191535949,
1074
- "step": 1520
1075
- },
1076
- {
1077
- "epoch": 0.9,
1078
- "grad_norm": 0.4087558388710022,
1079
- "learning_rate": 0.0001400392156862745,
1080
- "loss": 0.08435083031654358,
1081
- "step": 1530
1082
- },
1083
- {
1084
- "epoch": 0.9058823529411765,
1085
- "grad_norm": 1.3371378183364868,
1086
- "learning_rate": 0.00013964705882352942,
1087
- "loss": 0.04774133563041687,
1088
- "step": 1540
1089
- },
1090
- {
1091
- "epoch": 0.9117647058823529,
1092
- "grad_norm": 3.8198187351226807,
1093
- "learning_rate": 0.00013925490196078433,
1094
- "loss": 0.05395355820655823,
1095
- "step": 1550
1096
- },
1097
- {
1098
- "epoch": 0.9176470588235294,
1099
- "grad_norm": 0.07719609141349792,
1100
- "learning_rate": 0.00013886274509803922,
1101
- "loss": 0.05478907227516174,
1102
- "step": 1560
1103
- },
1104
- {
1105
- "epoch": 0.9235294117647059,
1106
- "grad_norm": 0.14217644929885864,
1107
- "learning_rate": 0.00013847058823529413,
1108
- "loss": 0.037067908048629764,
1109
- "step": 1570
1110
- },
1111
- {
1112
- "epoch": 0.9294117647058824,
1113
- "grad_norm": 0.0115964999422431,
1114
- "learning_rate": 0.00013807843137254904,
1115
- "loss": 0.06596972942352294,
1116
- "step": 1580
1117
- },
1118
- {
1119
- "epoch": 0.9352941176470588,
1120
- "grad_norm": 0.11416631937026978,
1121
- "learning_rate": 0.00013768627450980395,
1122
- "loss": 0.06003996729850769,
1123
- "step": 1590
1124
- },
1125
- {
1126
- "epoch": 0.9411764705882353,
1127
- "grad_norm": 0.8297443985939026,
1128
- "learning_rate": 0.00013729411764705883,
1129
- "loss": 0.012542881071567535,
1130
- "step": 1600
1131
- },
1132
- {
1133
- "epoch": 0.9470588235294117,
1134
- "grad_norm": 2.7102794647216797,
1135
- "learning_rate": 0.00013690196078431374,
1136
- "loss": 0.04482048749923706,
1137
- "step": 1610
1138
- },
1139
- {
1140
- "epoch": 0.9529411764705882,
1141
- "grad_norm": 0.04676695913076401,
1142
- "learning_rate": 0.00013650980392156863,
1143
- "loss": 0.04008999168872833,
1144
- "step": 1620
1145
- },
1146
- {
1147
- "epoch": 0.9588235294117647,
1148
- "grad_norm": 0.024560345336794853,
1149
- "learning_rate": 0.00013611764705882354,
1150
- "loss": 0.004030698910355568,
1151
- "step": 1630
1152
- },
1153
- {
1154
- "epoch": 0.9647058823529412,
1155
- "grad_norm": 0.004613540135324001,
1156
- "learning_rate": 0.00013572549019607842,
1157
- "loss": 0.016170585155487062,
1158
- "step": 1640
1159
- },
1160
- {
1161
- "epoch": 0.9705882352941176,
1162
- "grad_norm": 0.07655692845582962,
1163
- "learning_rate": 0.00013533333333333333,
1164
- "loss": 0.002394220232963562,
1165
- "step": 1650
1166
- },
1167
- {
1168
- "epoch": 0.9764705882352941,
1169
- "grad_norm": 1.5127253532409668,
1170
- "learning_rate": 0.00013494117647058825,
1171
- "loss": 0.08604814410209656,
1172
- "step": 1660
1173
- },
1174
- {
1175
- "epoch": 0.9823529411764705,
1176
- "grad_norm": 4.647606372833252,
1177
- "learning_rate": 0.00013454901960784313,
1178
- "loss": 0.05253278613090515,
1179
- "step": 1670
1180
- },
1181
- {
1182
- "epoch": 0.9882352941176471,
1183
- "grad_norm": 0.0031903591006994247,
1184
- "learning_rate": 0.00013415686274509804,
1185
- "loss": 0.05996617674827576,
1186
- "step": 1680
1187
- },
1188
- {
1189
- "epoch": 0.9941176470588236,
1190
- "grad_norm": 3.4767584800720215,
1191
- "learning_rate": 0.00013376470588235295,
1192
- "loss": 0.09587143063545227,
1193
- "step": 1690
1194
- },
1195
- {
1196
- "epoch": 1.0,
1197
- "grad_norm": 0.6253292560577393,
1198
- "learning_rate": 0.00013337254901960786,
1199
- "loss": 0.004908151552081108,
1200
- "step": 1700
1201
- },
1202
- {
1203
- "epoch": 1.0,
1204
- "eval_accuracy": 0.9833333333333333,
1205
- "eval_loss": 0.052190084010362625,
1206
- "eval_runtime": 107.7754,
1207
- "eval_samples_per_second": 44.537,
1208
- "eval_steps_per_second": 2.784,
1209
- "step": 1700
1210
- }
1211
- ],
1212
- "logging_steps": 10,
1213
- "max_steps": 5100,
1214
- "num_input_tokens_seen": 0,
1215
- "num_train_epochs": 3,
1216
- "save_steps": 500,
1217
- "stateful_callbacks": {
1218
- "TrainerControl": {
1219
- "args": {
1220
- "should_epoch_stop": false,
1221
- "should_evaluate": false,
1222
- "should_log": false,
1223
- "should_save": true,
1224
- "should_training_stop": false
1225
- },
1226
- "attributes": {}
1227
- }
1228
- },
1229
- "total_flos": 144276430488960.0,
1230
- "train_batch_size": 16,
1231
- "trial_name": null,
1232
- "trial_params": null
1233
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
training/training-checkpoints/checkpoint-1700/training_args.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:404392ce274606d42167ab282bdb788e763b38ede0704b0120ec45be3aba6567
3
- size 5201
 
 
 
 
training/training-checkpoints/checkpoint-3400/README.md DELETED
@@ -1,202 +0,0 @@
1
- ---
2
- base_model: distilbert-base-uncased
3
- library_name: peft
4
- ---
5
-
6
- # Model Card for Model ID
7
-
8
- <!-- Provide a quick summary of what the model is/does. -->
9
-
10
-
11
-
12
- ## Model Details
13
-
14
- ### Model Description
15
-
16
- <!-- Provide a longer summary of what this model is. -->
17
-
18
-
19
-
20
- - **Developed by:** [More Information Needed]
21
- - **Funded by [optional]:** [More Information Needed]
22
- - **Shared by [optional]:** [More Information Needed]
23
- - **Model type:** [More Information Needed]
24
- - **Language(s) (NLP):** [More Information Needed]
25
- - **License:** [More Information Needed]
26
- - **Finetuned from model [optional]:** [More Information Needed]
27
-
28
- ### Model Sources [optional]
29
-
30
- <!-- Provide the basic links for the model. -->
31
-
32
- - **Repository:** [More Information Needed]
33
- - **Paper [optional]:** [More Information Needed]
34
- - **Demo [optional]:** [More Information Needed]
35
-
36
- ## Uses
37
-
38
- <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
39
-
40
- ### Direct Use
41
-
42
- <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
43
-
44
- [More Information Needed]
45
-
46
- ### Downstream Use [optional]
47
-
48
- <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
49
-
50
- [More Information Needed]
51
-
52
- ### Out-of-Scope Use
53
-
54
- <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
55
-
56
- [More Information Needed]
57
-
58
- ## Bias, Risks, and Limitations
59
-
60
- <!-- This section is meant to convey both technical and sociotechnical limitations. -->
61
-
62
- [More Information Needed]
63
-
64
- ### Recommendations
65
-
66
- <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
67
-
68
- Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
69
-
70
- ## How to Get Started with the Model
71
-
72
- Use the code below to get started with the model.
73
-
74
- [More Information Needed]
75
-
76
- ## Training Details
77
-
78
- ### Training Data
79
-
80
- <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
81
-
82
- [More Information Needed]
83
-
84
- ### Training Procedure
85
-
86
- <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
87
-
88
- #### Preprocessing [optional]
89
-
90
- [More Information Needed]
91
-
92
-
93
- #### Training Hyperparameters
94
-
95
- - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
96
-
97
- #### Speeds, Sizes, Times [optional]
98
-
99
- <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
100
-
101
- [More Information Needed]
102
-
103
- ## Evaluation
104
-
105
- <!-- This section describes the evaluation protocols and provides the results. -->
106
-
107
- ### Testing Data, Factors & Metrics
108
-
109
- #### Testing Data
110
-
111
- <!-- This should link to a Dataset Card if possible. -->
112
-
113
- [More Information Needed]
114
-
115
- #### Factors
116
-
117
- <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
118
-
119
- [More Information Needed]
120
-
121
- #### Metrics
122
-
123
- <!-- These are the evaluation metrics being used, ideally with a description of why. -->
124
-
125
- [More Information Needed]
126
-
127
- ### Results
128
-
129
- [More Information Needed]
130
-
131
- #### Summary
132
-
133
-
134
-
135
- ## Model Examination [optional]
136
-
137
- <!-- Relevant interpretability work for the model goes here -->
138
-
139
- [More Information Needed]
140
-
141
- ## Environmental Impact
142
-
143
- <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
144
-
145
- Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
146
-
147
- - **Hardware Type:** [More Information Needed]
148
- - **Hours used:** [More Information Needed]
149
- - **Cloud Provider:** [More Information Needed]
150
- - **Compute Region:** [More Information Needed]
151
- - **Carbon Emitted:** [More Information Needed]
152
-
153
- ## Technical Specifications [optional]
154
-
155
- ### Model Architecture and Objective
156
-
157
- [More Information Needed]
158
-
159
- ### Compute Infrastructure
160
-
161
- [More Information Needed]
162
-
163
- #### Hardware
164
-
165
- [More Information Needed]
166
-
167
- #### Software
168
-
169
- [More Information Needed]
170
-
171
- ## Citation [optional]
172
-
173
- <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
174
-
175
- **BibTeX:**
176
-
177
- [More Information Needed]
178
-
179
- **APA:**
180
-
181
- [More Information Needed]
182
-
183
- ## Glossary [optional]
184
-
185
- <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
186
-
187
- [More Information Needed]
188
-
189
- ## More Information [optional]
190
-
191
- [More Information Needed]
192
-
193
- ## Model Card Authors [optional]
194
-
195
- [More Information Needed]
196
-
197
- ## Model Card Contact
198
-
199
- [More Information Needed]
200
- ### Framework versions
201
-
202
- - PEFT 0.13.2
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
training/training-checkpoints/checkpoint-3400/adapter_config.json DELETED
@@ -1,32 +0,0 @@
1
- {
2
- "alpha_pattern": {},
3
- "auto_mapping": null,
4
- "base_model_name_or_path": "distilbert-base-uncased",
5
- "bias": "none",
6
- "fan_in_fan_out": false,
7
- "inference_mode": true,
8
- "init_lora_weights": true,
9
- "layer_replication": null,
10
- "layers_pattern": null,
11
- "layers_to_transform": null,
12
- "loftq_config": {},
13
- "lora_alpha": 16,
14
- "lora_dropout": 0.1,
15
- "megatron_config": null,
16
- "megatron_core": "megatron.core",
17
- "modules_to_save": [
18
- "classifier",
19
- "score"
20
- ],
21
- "peft_type": "LORA",
22
- "r": 8,
23
- "rank_pattern": {},
24
- "revision": null,
25
- "target_modules": [
26
- "v_lin",
27
- "q_lin"
28
- ],
29
- "task_type": "SEQ_CLS",
30
- "use_dora": false,
31
- "use_rslora": false
32
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
training/training-checkpoints/checkpoint-3400/adapter_model.safetensors DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:3efc79c7eb5833b34e822345fb70ee2173e80b4828996657e6db01a3f2ec01e2
3
- size 2962192
 
 
 
 
training/training-checkpoints/checkpoint-3400/optimizer.pt DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:8c9e6c856440e62b525eebb5d0a7dde53f1558e65e21804d25911bbd40980f0c
3
- size 5940315
 
 
 
 
training/training-checkpoints/checkpoint-3400/rng_state.pth DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:e429eb1781646b5274a99b5e36c82dbc61a3865d7755b8859fa26a3972944dad
3
- size 14455
 
 
 
 
training/training-checkpoints/checkpoint-3400/scheduler.pt DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:184a9ded095bbedd783e752989df8b4dad755674acfc75d117a431c0266daf32
3
- size 1465
 
 
 
 
training/training-checkpoints/checkpoint-3400/tokenizer.json DELETED
The diff for this file is too large to render. See raw diff
 
training/training-checkpoints/checkpoint-3400/tokenizer_config.json DELETED
@@ -1,15 +0,0 @@
1
- {
2
- "backend": "tokenizers",
3
- "cls_token": "[CLS]",
4
- "do_lower_case": true,
5
- "is_local": false,
6
- "local_files_only": false,
7
- "mask_token": "[MASK]",
8
- "model_max_length": 512,
9
- "pad_token": "[PAD]",
10
- "sep_token": "[SEP]",
11
- "strip_accents": null,
12
- "tokenize_chinese_chars": true,
13
- "tokenizer_class": "BertTokenizer",
14
- "unk_token": "[UNK]"
15
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
training/training-checkpoints/checkpoint-3400/trainer_state.json DELETED
@@ -1,2432 +0,0 @@
1
- {
2
- "best_global_step": 1700,
3
- "best_metric": 0.9833333333333333,
4
- "best_model_checkpoint": "./training-checkpoints\\checkpoint-1700",
5
- "epoch": 2.0,
6
- "eval_steps": 500,
7
- "global_step": 3400,
8
- "is_hyper_param_search": false,
9
- "is_local_process_zero": true,
10
- "is_world_process_zero": true,
11
- "log_history": [
12
- {
13
- "epoch": 0.0058823529411764705,
14
- "grad_norm": 1.510934829711914,
15
- "learning_rate": 0.00019964705882352944,
16
- "loss": 0.6532491683959961,
17
- "step": 10
18
- },
19
- {
20
- "epoch": 0.011764705882352941,
21
- "grad_norm": 2.264005661010742,
22
- "learning_rate": 0.0001992549019607843,
23
- "loss": 0.49631505012512206,
24
- "step": 20
25
- },
26
- {
27
- "epoch": 0.01764705882352941,
28
- "grad_norm": 1.0282903909683228,
29
- "learning_rate": 0.0001988627450980392,
30
- "loss": 0.29676687717437744,
31
- "step": 30
32
- },
33
- {
34
- "epoch": 0.023529411764705882,
35
- "grad_norm": 1.0084155797958374,
36
- "learning_rate": 0.00019847058823529412,
37
- "loss": 0.1594167470932007,
38
- "step": 40
39
- },
40
- {
41
- "epoch": 0.029411764705882353,
42
- "grad_norm": 0.23450879752635956,
43
- "learning_rate": 0.00019807843137254903,
44
- "loss": 0.09263747334480285,
45
- "step": 50
46
- },
47
- {
48
- "epoch": 0.03529411764705882,
49
- "grad_norm": 0.44396597146987915,
50
- "learning_rate": 0.00019768627450980392,
51
- "loss": 0.18005683422088622,
52
- "step": 60
53
- },
54
- {
55
- "epoch": 0.041176470588235294,
56
- "grad_norm": 0.1429290622472763,
57
- "learning_rate": 0.00019729411764705883,
58
- "loss": 0.08807324171066284,
59
- "step": 70
60
- },
61
- {
62
- "epoch": 0.047058823529411764,
63
- "grad_norm": 0.847214937210083,
64
- "learning_rate": 0.00019690196078431374,
65
- "loss": 0.058199501037597655,
66
- "step": 80
67
- },
68
- {
69
- "epoch": 0.052941176470588235,
70
- "grad_norm": 3.822068929672241,
71
- "learning_rate": 0.00019650980392156862,
72
- "loss": 0.1674056887626648,
73
- "step": 90
74
- },
75
- {
76
- "epoch": 0.058823529411764705,
77
- "grad_norm": 1.4017611742019653,
78
- "learning_rate": 0.00019611764705882353,
79
- "loss": 0.11085774898529052,
80
- "step": 100
81
- },
82
- {
83
- "epoch": 0.06470588235294118,
84
- "grad_norm": 1.6271415948867798,
85
- "learning_rate": 0.00019572549019607844,
86
- "loss": 0.10848402976989746,
87
- "step": 110
88
- },
89
- {
90
- "epoch": 0.07058823529411765,
91
- "grad_norm": 0.17168374359607697,
92
- "learning_rate": 0.00019533333333333336,
93
- "loss": 0.08450813293457031,
94
- "step": 120
95
- },
96
- {
97
- "epoch": 0.07647058823529412,
98
- "grad_norm": 2.110332489013672,
99
- "learning_rate": 0.00019494117647058824,
100
- "loss": 0.08588783144950866,
101
- "step": 130
102
- },
103
- {
104
- "epoch": 0.08235294117647059,
105
- "grad_norm": 1.973347783088684,
106
- "learning_rate": 0.00019454901960784315,
107
- "loss": 0.07138832211494446,
108
- "step": 140
109
- },
110
- {
111
- "epoch": 0.08823529411764706,
112
- "grad_norm": 2.4691383838653564,
113
- "learning_rate": 0.00019415686274509806,
114
- "loss": 0.1480124831199646,
115
- "step": 150
116
- },
117
- {
118
- "epoch": 0.09411764705882353,
119
- "grad_norm": 0.8067235946655273,
120
- "learning_rate": 0.00019376470588235295,
121
- "loss": 0.03993956744670868,
122
- "step": 160
123
- },
124
- {
125
- "epoch": 0.1,
126
- "grad_norm": 2.022461175918579,
127
- "learning_rate": 0.00019337254901960786,
128
- "loss": 0.06878547668457032,
129
- "step": 170
130
- },
131
- {
132
- "epoch": 0.10588235294117647,
133
- "grad_norm": 1.4254077672958374,
134
- "learning_rate": 0.00019298039215686274,
135
- "loss": 0.12785714864730835,
136
- "step": 180
137
- },
138
- {
139
- "epoch": 0.11176470588235295,
140
- "grad_norm": 3.5772814750671387,
141
- "learning_rate": 0.00019258823529411765,
142
- "loss": 0.07902452945709229,
143
- "step": 190
144
- },
145
- {
146
- "epoch": 0.11764705882352941,
147
- "grad_norm": 0.06748726963996887,
148
- "learning_rate": 0.00019219607843137256,
149
- "loss": 0.06560993194580078,
150
- "step": 200
151
- },
152
- {
153
- "epoch": 0.12352941176470589,
154
- "grad_norm": 5.759597301483154,
155
- "learning_rate": 0.00019180392156862745,
156
- "loss": 0.16235246658325195,
157
- "step": 210
158
- },
159
- {
160
- "epoch": 0.12941176470588237,
161
- "grad_norm": 3.6009647846221924,
162
- "learning_rate": 0.00019141176470588236,
163
- "loss": 0.06680867075920105,
164
- "step": 220
165
- },
166
- {
167
- "epoch": 0.13529411764705881,
168
- "grad_norm": 2.608488082885742,
169
- "learning_rate": 0.00019101960784313727,
170
- "loss": 0.08542757034301758,
171
- "step": 230
172
- },
173
- {
174
- "epoch": 0.1411764705882353,
175
- "grad_norm": 1.4347050189971924,
176
- "learning_rate": 0.00019062745098039215,
177
- "loss": 0.06062295436859131,
178
- "step": 240
179
- },
180
- {
181
- "epoch": 0.14705882352941177,
182
- "grad_norm": 3.4521889686584473,
183
- "learning_rate": 0.00019023529411764706,
184
- "loss": 0.11894316673278808,
185
- "step": 250
186
- },
187
- {
188
- "epoch": 0.15294117647058825,
189
- "grad_norm": 2.5669424533843994,
190
- "learning_rate": 0.00018984313725490197,
191
- "loss": 0.04931341111660004,
192
- "step": 260
193
- },
194
- {
195
- "epoch": 0.1588235294117647,
196
- "grad_norm": 0.5097957253456116,
197
- "learning_rate": 0.00018945098039215689,
198
- "loss": 0.0895132601261139,
199
- "step": 270
200
- },
201
- {
202
- "epoch": 0.16470588235294117,
203
- "grad_norm": 0.1470881551504135,
204
- "learning_rate": 0.00018905882352941177,
205
- "loss": 0.07752076387405396,
206
- "step": 280
207
- },
208
- {
209
- "epoch": 0.17058823529411765,
210
- "grad_norm": 0.12387822568416595,
211
- "learning_rate": 0.00018866666666666668,
212
- "loss": 0.0910811185836792,
213
- "step": 290
214
- },
215
- {
216
- "epoch": 0.17647058823529413,
217
- "grad_norm": 0.1579340398311615,
218
- "learning_rate": 0.0001882745098039216,
219
- "loss": 0.024501879513263703,
220
- "step": 300
221
- },
222
- {
223
- "epoch": 0.18235294117647058,
224
- "grad_norm": 0.19159211218357086,
225
- "learning_rate": 0.00018788235294117648,
226
- "loss": 0.025437775254249572,
227
- "step": 310
228
- },
229
- {
230
- "epoch": 0.18823529411764706,
231
- "grad_norm": 0.3070846199989319,
232
- "learning_rate": 0.0001874901960784314,
233
- "loss": 0.0452291190624237,
234
- "step": 320
235
- },
236
- {
237
- "epoch": 0.19411764705882353,
238
- "grad_norm": 1.647754192352295,
239
- "learning_rate": 0.0001870980392156863,
240
- "loss": 0.07508809566497802,
241
- "step": 330
242
- },
243
- {
244
- "epoch": 0.2,
245
- "grad_norm": 0.2747861444950104,
246
- "learning_rate": 0.00018670588235294118,
247
- "loss": 0.08880412578582764,
248
- "step": 340
249
- },
250
- {
251
- "epoch": 0.20588235294117646,
252
- "grad_norm": 0.20420950651168823,
253
- "learning_rate": 0.00018631372549019607,
254
- "loss": 0.047925958037376405,
255
- "step": 350
256
- },
257
- {
258
- "epoch": 0.21176470588235294,
259
- "grad_norm": 0.5672637820243835,
260
- "learning_rate": 0.00018592156862745098,
261
- "loss": 0.04176274836063385,
262
- "step": 360
263
- },
264
- {
265
- "epoch": 0.21764705882352942,
266
- "grad_norm": 1.5938951969146729,
267
- "learning_rate": 0.0001855294117647059,
268
- "loss": 0.06371157765388488,
269
- "step": 370
270
- },
271
- {
272
- "epoch": 0.2235294117647059,
273
- "grad_norm": 0.019027238711714745,
274
- "learning_rate": 0.0001851372549019608,
275
- "loss": 0.05675685405731201,
276
- "step": 380
277
- },
278
- {
279
- "epoch": 0.22941176470588234,
280
- "grad_norm": 1.8165510892868042,
281
- "learning_rate": 0.00018474509803921568,
282
- "loss": 0.10704467296600342,
283
- "step": 390
284
- },
285
- {
286
- "epoch": 0.23529411764705882,
287
- "grad_norm": 0.3322693109512329,
288
- "learning_rate": 0.0001843529411764706,
289
- "loss": 0.08637534379959107,
290
- "step": 400
291
- },
292
- {
293
- "epoch": 0.2411764705882353,
294
- "grad_norm": 0.06156998872756958,
295
- "learning_rate": 0.0001839607843137255,
296
- "loss": 0.17954374551773072,
297
- "step": 410
298
- },
299
- {
300
- "epoch": 0.24705882352941178,
301
- "grad_norm": 3.327615737915039,
302
- "learning_rate": 0.0001835686274509804,
303
- "loss": 0.1696428656578064,
304
- "step": 420
305
- },
306
- {
307
- "epoch": 0.2529411764705882,
308
- "grad_norm": 2.1997804641723633,
309
- "learning_rate": 0.0001831764705882353,
310
- "loss": 0.11364388465881348,
311
- "step": 430
312
- },
313
- {
314
- "epoch": 0.25882352941176473,
315
- "grad_norm": 2.8368852138519287,
316
- "learning_rate": 0.0001827843137254902,
317
- "loss": 0.03553434014320374,
318
- "step": 440
319
- },
320
- {
321
- "epoch": 0.2647058823529412,
322
- "grad_norm": 0.2659618556499481,
323
- "learning_rate": 0.00018239215686274512,
324
- "loss": 0.03715018928050995,
325
- "step": 450
326
- },
327
- {
328
- "epoch": 0.27058823529411763,
329
- "grad_norm": 0.04204031080007553,
330
- "learning_rate": 0.000182,
331
- "loss": 0.13120421171188354,
332
- "step": 460
333
- },
334
- {
335
- "epoch": 0.27647058823529413,
336
- "grad_norm": 3.284900665283203,
337
- "learning_rate": 0.00018160784313725492,
338
- "loss": 0.08410873413085937,
339
- "step": 470
340
- },
341
- {
342
- "epoch": 0.2823529411764706,
343
- "grad_norm": 1.841749906539917,
344
- "learning_rate": 0.00018121568627450983,
345
- "loss": 0.056668424606323244,
346
- "step": 480
347
- },
348
- {
349
- "epoch": 0.28823529411764703,
350
- "grad_norm": 0.03970719501376152,
351
- "learning_rate": 0.0001808235294117647,
352
- "loss": 0.043222978711128235,
353
- "step": 490
354
- },
355
- {
356
- "epoch": 0.29411764705882354,
357
- "grad_norm": 0.17564676702022552,
358
- "learning_rate": 0.0001804313725490196,
359
- "loss": 0.029093235731124878,
360
- "step": 500
361
- },
362
- {
363
- "epoch": 0.3,
364
- "grad_norm": 2.7894046306610107,
365
- "learning_rate": 0.0001800392156862745,
366
- "loss": 0.18836088180541993,
367
- "step": 510
368
- },
369
- {
370
- "epoch": 0.3058823529411765,
371
- "grad_norm": 0.015540416352450848,
372
- "learning_rate": 0.00017964705882352942,
373
- "loss": 0.04498102068901062,
374
- "step": 520
375
- },
376
- {
377
- "epoch": 0.31176470588235294,
378
- "grad_norm": 1.4599756002426147,
379
- "learning_rate": 0.00017925490196078433,
380
- "loss": 0.06498492360115052,
381
- "step": 530
382
- },
383
- {
384
- "epoch": 0.3176470588235294,
385
- "grad_norm": 0.03984140232205391,
386
- "learning_rate": 0.0001788627450980392,
387
- "loss": 0.06558265089988709,
388
- "step": 540
389
- },
390
- {
391
- "epoch": 0.3235294117647059,
392
- "grad_norm": 2.5549182891845703,
393
- "learning_rate": 0.00017847058823529412,
394
- "loss": 0.02280822992324829,
395
- "step": 550
396
- },
397
- {
398
- "epoch": 0.32941176470588235,
399
- "grad_norm": 0.31564125418663025,
400
- "learning_rate": 0.00017807843137254903,
401
- "loss": 0.05171017646789551,
402
- "step": 560
403
- },
404
- {
405
- "epoch": 0.3352941176470588,
406
- "grad_norm": 1.2470703125,
407
- "learning_rate": 0.00017768627450980392,
408
- "loss": 0.05321676135063171,
409
- "step": 570
410
- },
411
- {
412
- "epoch": 0.3411764705882353,
413
- "grad_norm": 0.19033531844615936,
414
- "learning_rate": 0.00017729411764705883,
415
- "loss": 0.14463623762130737,
416
- "step": 580
417
- },
418
- {
419
- "epoch": 0.34705882352941175,
420
- "grad_norm": 2.2864792346954346,
421
- "learning_rate": 0.00017690196078431374,
422
- "loss": 0.06809618473052978,
423
- "step": 590
424
- },
425
- {
426
- "epoch": 0.35294117647058826,
427
- "grad_norm": 1.959888219833374,
428
- "learning_rate": 0.00017650980392156865,
429
- "loss": 0.03620869815349579,
430
- "step": 600
431
- },
432
- {
433
- "epoch": 0.3588235294117647,
434
- "grad_norm": 2.4784562587738037,
435
- "learning_rate": 0.00017611764705882354,
436
- "loss": 0.02364923506975174,
437
- "step": 610
438
- },
439
- {
440
- "epoch": 0.36470588235294116,
441
- "grad_norm": 0.02134384773671627,
442
- "learning_rate": 0.00017572549019607845,
443
- "loss": 0.09724082350730896,
444
- "step": 620
445
- },
446
- {
447
- "epoch": 0.37058823529411766,
448
- "grad_norm": 0.06821486353874207,
449
- "learning_rate": 0.00017533333333333336,
450
- "loss": 0.06447332501411437,
451
- "step": 630
452
- },
453
- {
454
- "epoch": 0.3764705882352941,
455
- "grad_norm": 0.6358501315116882,
456
- "learning_rate": 0.00017494117647058824,
457
- "loss": 0.06500619649887085,
458
- "step": 640
459
- },
460
- {
461
- "epoch": 0.38235294117647056,
462
- "grad_norm": 3.4506707191467285,
463
- "learning_rate": 0.00017454901960784315,
464
- "loss": 0.08725794553756713,
465
- "step": 650
466
- },
467
- {
468
- "epoch": 0.38823529411764707,
469
- "grad_norm": 0.22893691062927246,
470
- "learning_rate": 0.00017415686274509804,
471
- "loss": 0.10881921052932739,
472
- "step": 660
473
- },
474
- {
475
- "epoch": 0.3941176470588235,
476
- "grad_norm": 0.10322771966457367,
477
- "learning_rate": 0.00017376470588235295,
478
- "loss": 0.03397703766822815,
479
- "step": 670
480
- },
481
- {
482
- "epoch": 0.4,
483
- "grad_norm": 2.8205652236938477,
484
- "learning_rate": 0.00017337254901960783,
485
- "loss": 0.027051010727882387,
486
- "step": 680
487
- },
488
- {
489
- "epoch": 0.40588235294117647,
490
- "grad_norm": 0.013308421708643436,
491
- "learning_rate": 0.00017298039215686274,
492
- "loss": 0.04930915534496307,
493
- "step": 690
494
- },
495
- {
496
- "epoch": 0.4117647058823529,
497
- "grad_norm": 3.44494891166687,
498
- "learning_rate": 0.00017258823529411765,
499
- "loss": 0.07341324090957642,
500
- "step": 700
501
- },
502
- {
503
- "epoch": 0.4176470588235294,
504
- "grad_norm": 2.4105072021484375,
505
- "learning_rate": 0.00017219607843137256,
506
- "loss": 0.05853899717330933,
507
- "step": 710
508
- },
509
- {
510
- "epoch": 0.4235294117647059,
511
- "grad_norm": 4.644453048706055,
512
- "learning_rate": 0.00017180392156862745,
513
- "loss": 0.12287142276763915,
514
- "step": 720
515
- },
516
- {
517
- "epoch": 0.4294117647058823,
518
- "grad_norm": 0.38577258586883545,
519
- "learning_rate": 0.00017141176470588236,
520
- "loss": 0.06102405190467834,
521
- "step": 730
522
- },
523
- {
524
- "epoch": 0.43529411764705883,
525
- "grad_norm": 0.07725336402654648,
526
- "learning_rate": 0.00017101960784313727,
527
- "loss": 0.047857701778411865,
528
- "step": 740
529
- },
530
- {
531
- "epoch": 0.4411764705882353,
532
- "grad_norm": 0.06397025287151337,
533
- "learning_rate": 0.00017062745098039215,
534
- "loss": 0.09663112759590149,
535
- "step": 750
536
- },
537
- {
538
- "epoch": 0.4470588235294118,
539
- "grad_norm": 3.3719794750213623,
540
- "learning_rate": 0.00017023529411764707,
541
- "loss": 0.09325426816940308,
542
- "step": 760
543
- },
544
- {
545
- "epoch": 0.45294117647058824,
546
- "grad_norm": 0.46245869994163513,
547
- "learning_rate": 0.00016984313725490198,
548
- "loss": 0.08240602612495422,
549
- "step": 770
550
- },
551
- {
552
- "epoch": 0.4588235294117647,
553
- "grad_norm": 0.032528385519981384,
554
- "learning_rate": 0.0001694509803921569,
555
- "loss": 0.050741398334503175,
556
- "step": 780
557
- },
558
- {
559
- "epoch": 0.4647058823529412,
560
- "grad_norm": 0.24650231003761292,
561
- "learning_rate": 0.00016905882352941177,
562
- "loss": 0.04027060270309448,
563
- "step": 790
564
- },
565
- {
566
- "epoch": 0.47058823529411764,
567
- "grad_norm": 0.12963470816612244,
568
- "learning_rate": 0.00016866666666666668,
569
- "loss": 0.05930449962615967,
570
- "step": 800
571
- },
572
- {
573
- "epoch": 0.4764705882352941,
574
- "grad_norm": 0.4593453109264374,
575
- "learning_rate": 0.0001682745098039216,
576
- "loss": 0.018743452429771424,
577
- "step": 810
578
- },
579
- {
580
- "epoch": 0.4823529411764706,
581
- "grad_norm": 2.126039981842041,
582
- "learning_rate": 0.00016788235294117648,
583
- "loss": 0.014739298820495605,
584
- "step": 820
585
- },
586
- {
587
- "epoch": 0.48823529411764705,
588
- "grad_norm": 0.1071607917547226,
589
- "learning_rate": 0.00016749019607843136,
590
- "loss": 0.03930833339691162,
591
- "step": 830
592
- },
593
- {
594
- "epoch": 0.49411764705882355,
595
- "grad_norm": 1.272904634475708,
596
- "learning_rate": 0.00016709803921568627,
597
- "loss": 0.01983778029680252,
598
- "step": 840
599
- },
600
- {
601
- "epoch": 0.5,
602
- "grad_norm": 4.284426212310791,
603
- "learning_rate": 0.00016670588235294118,
604
- "loss": 0.05801863074302673,
605
- "step": 850
606
- },
607
- {
608
- "epoch": 0.5058823529411764,
609
- "grad_norm": 0.21516184508800507,
610
- "learning_rate": 0.0001663137254901961,
611
- "loss": 0.0344509094953537,
612
- "step": 860
613
- },
614
- {
615
- "epoch": 0.5117647058823529,
616
- "grad_norm": 0.05401306226849556,
617
- "learning_rate": 0.00016592156862745098,
618
- "loss": 0.1013040542602539,
619
- "step": 870
620
- },
621
- {
622
- "epoch": 0.5176470588235295,
623
- "grad_norm": 2.2851767539978027,
624
- "learning_rate": 0.0001655294117647059,
625
- "loss": 0.06278380155563354,
626
- "step": 880
627
- },
628
- {
629
- "epoch": 0.5235294117647059,
630
- "grad_norm": 2.387756586074829,
631
- "learning_rate": 0.0001651372549019608,
632
- "loss": 0.1075211524963379,
633
- "step": 890
634
- },
635
- {
636
- "epoch": 0.5294117647058824,
637
- "grad_norm": 0.6298794150352478,
638
- "learning_rate": 0.00016474509803921568,
639
- "loss": 0.04051249623298645,
640
- "step": 900
641
- },
642
- {
643
- "epoch": 0.5352941176470588,
644
- "grad_norm": 3.3592398166656494,
645
- "learning_rate": 0.0001643529411764706,
646
- "loss": 0.048895838856697085,
647
- "step": 910
648
- },
649
- {
650
- "epoch": 0.5411764705882353,
651
- "grad_norm": 0.004421881400048733,
652
- "learning_rate": 0.0001639607843137255,
653
- "loss": 0.06308169960975647,
654
- "step": 920
655
- },
656
- {
657
- "epoch": 0.5470588235294118,
658
- "grad_norm": 0.09252121299505234,
659
- "learning_rate": 0.00016356862745098042,
660
- "loss": 0.058104753494262695,
661
- "step": 930
662
- },
663
- {
664
- "epoch": 0.5529411764705883,
665
- "grad_norm": 0.7001014351844788,
666
- "learning_rate": 0.0001631764705882353,
667
- "loss": 0.021149225533008575,
668
- "step": 940
669
- },
670
- {
671
- "epoch": 0.5588235294117647,
672
- "grad_norm": 0.3170143663883209,
673
- "learning_rate": 0.0001627843137254902,
674
- "loss": 0.01627269834280014,
675
- "step": 950
676
- },
677
- {
678
- "epoch": 0.5647058823529412,
679
- "grad_norm": 3.7102761268615723,
680
- "learning_rate": 0.00016239215686274512,
681
- "loss": 0.14663417339324952,
682
- "step": 960
683
- },
684
- {
685
- "epoch": 0.5705882352941176,
686
- "grad_norm": 0.006750724744051695,
687
- "learning_rate": 0.000162,
688
- "loss": 0.10339630842208862,
689
- "step": 970
690
- },
691
- {
692
- "epoch": 0.5764705882352941,
693
- "grad_norm": 1.808908462524414,
694
- "learning_rate": 0.0001616078431372549,
695
- "loss": 0.026013880968093872,
696
- "step": 980
697
- },
698
- {
699
- "epoch": 0.5823529411764706,
700
- "grad_norm": 0.4191270172595978,
701
- "learning_rate": 0.0001612156862745098,
702
- "loss": 0.039347043633461,
703
- "step": 990
704
- },
705
- {
706
- "epoch": 0.5882352941176471,
707
- "grad_norm": 0.3325525224208832,
708
- "learning_rate": 0.0001608235294117647,
709
- "loss": 0.059603160619735716,
710
- "step": 1000
711
- },
712
- {
713
- "epoch": 0.5941176470588235,
714
- "grad_norm": 2.7802844047546387,
715
- "learning_rate": 0.0001604313725490196,
716
- "loss": 0.09034895300865173,
717
- "step": 1010
718
- },
719
- {
720
- "epoch": 0.6,
721
- "grad_norm": 0.04968420788645744,
722
- "learning_rate": 0.0001600392156862745,
723
- "loss": 0.05755581855773926,
724
- "step": 1020
725
- },
726
- {
727
- "epoch": 0.6058823529411764,
728
- "grad_norm": 0.00875788927078247,
729
- "learning_rate": 0.00015964705882352942,
730
- "loss": 0.08298594951629638,
731
- "step": 1030
732
- },
733
- {
734
- "epoch": 0.611764705882353,
735
- "grad_norm": 0.38597434759140015,
736
- "learning_rate": 0.00015925490196078433,
737
- "loss": 0.009630383551120758,
738
- "step": 1040
739
- },
740
- {
741
- "epoch": 0.6176470588235294,
742
- "grad_norm": 0.42396023869514465,
743
- "learning_rate": 0.00015886274509803921,
744
- "loss": 0.05784105658531189,
745
- "step": 1050
746
- },
747
- {
748
- "epoch": 0.6235294117647059,
749
- "grad_norm": 3.5692520141601562,
750
- "learning_rate": 0.00015847058823529413,
751
- "loss": 0.1575460195541382,
752
- "step": 1060
753
- },
754
- {
755
- "epoch": 0.6294117647058823,
756
- "grad_norm": 0.12411337345838547,
757
- "learning_rate": 0.00015807843137254904,
758
- "loss": 0.052008318901062014,
759
- "step": 1070
760
- },
761
- {
762
- "epoch": 0.6352941176470588,
763
- "grad_norm": 0.12469016760587692,
764
- "learning_rate": 0.00015768627450980392,
765
- "loss": 0.06983086466789246,
766
- "step": 1080
767
- },
768
- {
769
- "epoch": 0.6411764705882353,
770
- "grad_norm": 0.0010811894899234176,
771
- "learning_rate": 0.00015729411764705883,
772
- "loss": 0.009204772859811782,
773
- "step": 1090
774
- },
775
- {
776
- "epoch": 0.6470588235294118,
777
- "grad_norm": 0.7795768976211548,
778
- "learning_rate": 0.00015690196078431374,
779
- "loss": 0.07802544236183166,
780
- "step": 1100
781
- },
782
- {
783
- "epoch": 0.6529411764705882,
784
- "grad_norm": 3.0475451946258545,
785
- "learning_rate": 0.00015650980392156865,
786
- "loss": 0.06048353910446167,
787
- "step": 1110
788
- },
789
- {
790
- "epoch": 0.6588235294117647,
791
- "grad_norm": 0.054235588759183884,
792
- "learning_rate": 0.00015611764705882354,
793
- "loss": 0.007759802043437958,
794
- "step": 1120
795
- },
796
- {
797
- "epoch": 0.6647058823529411,
798
- "grad_norm": 0.022670522332191467,
799
- "learning_rate": 0.00015572549019607845,
800
- "loss": 0.121265709400177,
801
- "step": 1130
802
- },
803
- {
804
- "epoch": 0.6705882352941176,
805
- "grad_norm": 4.404564380645752,
806
- "learning_rate": 0.00015533333333333333,
807
- "loss": 0.11734864711761475,
808
- "step": 1140
809
- },
810
- {
811
- "epoch": 0.6764705882352942,
812
- "grad_norm": 0.04260774329304695,
813
- "learning_rate": 0.00015494117647058824,
814
- "loss": 0.04804904460906982,
815
- "step": 1150
816
- },
817
- {
818
- "epoch": 0.6823529411764706,
819
- "grad_norm": 0.37849345803260803,
820
- "learning_rate": 0.00015454901960784313,
821
- "loss": 0.08949239253997802,
822
- "step": 1160
823
- },
824
- {
825
- "epoch": 0.6882352941176471,
826
- "grad_norm": 0.07910435646772385,
827
- "learning_rate": 0.00015415686274509804,
828
- "loss": 0.11236823797225952,
829
- "step": 1170
830
- },
831
- {
832
- "epoch": 0.6941176470588235,
833
- "grad_norm": 0.020735427737236023,
834
- "learning_rate": 0.00015376470588235295,
835
- "loss": 0.020005679130554198,
836
- "step": 1180
837
- },
838
- {
839
- "epoch": 0.7,
840
- "grad_norm": 0.6041317582130432,
841
- "learning_rate": 0.00015337254901960786,
842
- "loss": 0.047528034448623656,
843
- "step": 1190
844
- },
845
- {
846
- "epoch": 0.7058823529411765,
847
- "grad_norm": 0.07179152965545654,
848
- "learning_rate": 0.00015298039215686274,
849
- "loss": 0.03959411382675171,
850
- "step": 1200
851
- },
852
- {
853
- "epoch": 0.711764705882353,
854
- "grad_norm": 0.009498389437794685,
855
- "learning_rate": 0.00015258823529411766,
856
- "loss": 0.003311210125684738,
857
- "step": 1210
858
- },
859
- {
860
- "epoch": 0.7176470588235294,
861
- "grad_norm": 0.433115154504776,
862
- "learning_rate": 0.00015219607843137257,
863
- "loss": 0.00858706161379814,
864
- "step": 1220
865
- },
866
- {
867
- "epoch": 0.7235294117647059,
868
- "grad_norm": 0.008886504918336868,
869
- "learning_rate": 0.00015180392156862745,
870
- "loss": 0.06958861947059632,
871
- "step": 1230
872
- },
873
- {
874
- "epoch": 0.7294117647058823,
875
- "grad_norm": 0.008790708146989346,
876
- "learning_rate": 0.00015141176470588236,
877
- "loss": 0.008029435575008393,
878
- "step": 1240
879
- },
880
- {
881
- "epoch": 0.7352941176470589,
882
- "grad_norm": 0.005836977623403072,
883
- "learning_rate": 0.00015101960784313727,
884
- "loss": 0.05358666181564331,
885
- "step": 1250
886
- },
887
- {
888
- "epoch": 0.7411764705882353,
889
- "grad_norm": 0.14504218101501465,
890
- "learning_rate": 0.00015062745098039218,
891
- "loss": 0.049011257290840146,
892
- "step": 1260
893
- },
894
- {
895
- "epoch": 0.7470588235294118,
896
- "grad_norm": 0.2803121507167816,
897
- "learning_rate": 0.00015023529411764707,
898
- "loss": 0.006191087141633034,
899
- "step": 1270
900
- },
901
- {
902
- "epoch": 0.7529411764705882,
903
- "grad_norm": 3.1994972229003906,
904
- "learning_rate": 0.00014984313725490198,
905
- "loss": 0.10595870018005371,
906
- "step": 1280
907
- },
908
- {
909
- "epoch": 0.7588235294117647,
910
- "grad_norm": 0.00779637461528182,
911
- "learning_rate": 0.0001494509803921569,
912
- "loss": 0.07793840169906616,
913
- "step": 1290
914
- },
915
- {
916
- "epoch": 0.7647058823529411,
917
- "grad_norm": 0.04689478874206543,
918
- "learning_rate": 0.00014905882352941177,
919
- "loss": 0.08080050349235535,
920
- "step": 1300
921
- },
922
- {
923
- "epoch": 0.7705882352941177,
924
- "grad_norm": 0.9595724940299988,
925
- "learning_rate": 0.00014866666666666666,
926
- "loss": 0.11690077781677247,
927
- "step": 1310
928
- },
929
- {
930
- "epoch": 0.7764705882352941,
931
- "grad_norm": 4.141428470611572,
932
- "learning_rate": 0.00014827450980392157,
933
- "loss": 0.05245780348777771,
934
- "step": 1320
935
- },
936
- {
937
- "epoch": 0.7823529411764706,
938
- "grad_norm": 0.7606799602508545,
939
- "learning_rate": 0.00014788235294117648,
940
- "loss": 0.039501425623893735,
941
- "step": 1330
942
- },
943
- {
944
- "epoch": 0.788235294117647,
945
- "grad_norm": 2.8373472690582275,
946
- "learning_rate": 0.00014749019607843136,
947
- "loss": 0.08525887727737427,
948
- "step": 1340
949
- },
950
- {
951
- "epoch": 0.7941176470588235,
952
- "grad_norm": 0.20202526450157166,
953
- "learning_rate": 0.00014709803921568627,
954
- "loss": 0.03491775095462799,
955
- "step": 1350
956
- },
957
- {
958
- "epoch": 0.8,
959
- "grad_norm": 0.00345594366081059,
960
- "learning_rate": 0.00014670588235294119,
961
- "loss": 0.03784347176551819,
962
- "step": 1360
963
- },
964
- {
965
- "epoch": 0.8058823529411765,
966
- "grad_norm": 3.3252758979797363,
967
- "learning_rate": 0.0001463137254901961,
968
- "loss": 0.08734545707702637,
969
- "step": 1370
970
- },
971
- {
972
- "epoch": 0.8117647058823529,
973
- "grad_norm": 1.1744837760925293,
974
- "learning_rate": 0.00014592156862745098,
975
- "loss": 0.060222625732421875,
976
- "step": 1380
977
- },
978
- {
979
- "epoch": 0.8176470588235294,
980
- "grad_norm": 0.012418321333825588,
981
- "learning_rate": 0.0001455294117647059,
982
- "loss": 0.034251919388771056,
983
- "step": 1390
984
- },
985
- {
986
- "epoch": 0.8235294117647058,
987
- "grad_norm": 0.18622326850891113,
988
- "learning_rate": 0.0001451372549019608,
989
- "loss": 0.02356344759464264,
990
- "step": 1400
991
- },
992
- {
993
- "epoch": 0.8294117647058824,
994
- "grad_norm": 3.9867045879364014,
995
- "learning_rate": 0.00014474509803921569,
996
- "loss": 0.07036476135253907,
997
- "step": 1410
998
- },
999
- {
1000
- "epoch": 0.8352941176470589,
1001
- "grad_norm": 3.0563607215881348,
1002
- "learning_rate": 0.0001443529411764706,
1003
- "loss": 0.037895727157592776,
1004
- "step": 1420
1005
- },
1006
- {
1007
- "epoch": 0.8411764705882353,
1008
- "grad_norm": 0.13856367766857147,
1009
- "learning_rate": 0.0001439607843137255,
1010
- "loss": 0.12252027988433838,
1011
- "step": 1430
1012
- },
1013
- {
1014
- "epoch": 0.8470588235294118,
1015
- "grad_norm": 6.263605117797852,
1016
- "learning_rate": 0.00014356862745098042,
1017
- "loss": 0.04287775456905365,
1018
- "step": 1440
1019
- },
1020
- {
1021
- "epoch": 0.8529411764705882,
1022
- "grad_norm": 0.6084681749343872,
1023
- "learning_rate": 0.0001431764705882353,
1024
- "loss": 0.050480049848556516,
1025
- "step": 1450
1026
- },
1027
- {
1028
- "epoch": 0.8588235294117647,
1029
- "grad_norm": 0.13102763891220093,
1030
- "learning_rate": 0.0001427843137254902,
1031
- "loss": 0.027205181121826173,
1032
- "step": 1460
1033
- },
1034
- {
1035
- "epoch": 0.8647058823529412,
1036
- "grad_norm": 0.0334654301404953,
1037
- "learning_rate": 0.0001423921568627451,
1038
- "loss": 0.07116500139236451,
1039
- "step": 1470
1040
- },
1041
- {
1042
- "epoch": 0.8705882352941177,
1043
- "grad_norm": 5.854252815246582,
1044
- "learning_rate": 0.000142,
1045
- "loss": 0.10154651403427124,
1046
- "step": 1480
1047
- },
1048
- {
1049
- "epoch": 0.8764705882352941,
1050
- "grad_norm": 0.04299909248948097,
1051
- "learning_rate": 0.0001416078431372549,
1052
- "loss": 0.06952143907546997,
1053
- "step": 1490
1054
- },
1055
- {
1056
- "epoch": 0.8823529411764706,
1057
- "grad_norm": 0.03385253995656967,
1058
- "learning_rate": 0.0001412156862745098,
1059
- "loss": 0.042281192541122434,
1060
- "step": 1500
1061
- },
1062
- {
1063
- "epoch": 0.888235294117647,
1064
- "grad_norm": 3.0850558280944824,
1065
- "learning_rate": 0.00014082352941176472,
1066
- "loss": 0.04330833852291107,
1067
- "step": 1510
1068
- },
1069
- {
1070
- "epoch": 0.8941176470588236,
1071
- "grad_norm": 1.0124292373657227,
1072
- "learning_rate": 0.00014043137254901963,
1073
- "loss": 0.013002160191535949,
1074
- "step": 1520
1075
- },
1076
- {
1077
- "epoch": 0.9,
1078
- "grad_norm": 0.4087558388710022,
1079
- "learning_rate": 0.0001400392156862745,
1080
- "loss": 0.08435083031654358,
1081
- "step": 1530
1082
- },
1083
- {
1084
- "epoch": 0.9058823529411765,
1085
- "grad_norm": 1.3371378183364868,
1086
- "learning_rate": 0.00013964705882352942,
1087
- "loss": 0.04774133563041687,
1088
- "step": 1540
1089
- },
1090
- {
1091
- "epoch": 0.9117647058823529,
1092
- "grad_norm": 3.8198187351226807,
1093
- "learning_rate": 0.00013925490196078433,
1094
- "loss": 0.05395355820655823,
1095
- "step": 1550
1096
- },
1097
- {
1098
- "epoch": 0.9176470588235294,
1099
- "grad_norm": 0.07719609141349792,
1100
- "learning_rate": 0.00013886274509803922,
1101
- "loss": 0.05478907227516174,
1102
- "step": 1560
1103
- },
1104
- {
1105
- "epoch": 0.9235294117647059,
1106
- "grad_norm": 0.14217644929885864,
1107
- "learning_rate": 0.00013847058823529413,
1108
- "loss": 0.037067908048629764,
1109
- "step": 1570
1110
- },
1111
- {
1112
- "epoch": 0.9294117647058824,
1113
- "grad_norm": 0.0115964999422431,
1114
- "learning_rate": 0.00013807843137254904,
1115
- "loss": 0.06596972942352294,
1116
- "step": 1580
1117
- },
1118
- {
1119
- "epoch": 0.9352941176470588,
1120
- "grad_norm": 0.11416631937026978,
1121
- "learning_rate": 0.00013768627450980395,
1122
- "loss": 0.06003996729850769,
1123
- "step": 1590
1124
- },
1125
- {
1126
- "epoch": 0.9411764705882353,
1127
- "grad_norm": 0.8297443985939026,
1128
- "learning_rate": 0.00013729411764705883,
1129
- "loss": 0.012542881071567535,
1130
- "step": 1600
1131
- },
1132
- {
1133
- "epoch": 0.9470588235294117,
1134
- "grad_norm": 2.7102794647216797,
1135
- "learning_rate": 0.00013690196078431374,
1136
- "loss": 0.04482048749923706,
1137
- "step": 1610
1138
- },
1139
- {
1140
- "epoch": 0.9529411764705882,
1141
- "grad_norm": 0.04676695913076401,
1142
- "learning_rate": 0.00013650980392156863,
1143
- "loss": 0.04008999168872833,
1144
- "step": 1620
1145
- },
1146
- {
1147
- "epoch": 0.9588235294117647,
1148
- "grad_norm": 0.024560345336794853,
1149
- "learning_rate": 0.00013611764705882354,
1150
- "loss": 0.004030698910355568,
1151
- "step": 1630
1152
- },
1153
- {
1154
- "epoch": 0.9647058823529412,
1155
- "grad_norm": 0.004613540135324001,
1156
- "learning_rate": 0.00013572549019607842,
1157
- "loss": 0.016170585155487062,
1158
- "step": 1640
1159
- },
1160
- {
1161
- "epoch": 0.9705882352941176,
1162
- "grad_norm": 0.07655692845582962,
1163
- "learning_rate": 0.00013533333333333333,
1164
- "loss": 0.002394220232963562,
1165
- "step": 1650
1166
- },
1167
- {
1168
- "epoch": 0.9764705882352941,
1169
- "grad_norm": 1.5127253532409668,
1170
- "learning_rate": 0.00013494117647058825,
1171
- "loss": 0.08604814410209656,
1172
- "step": 1660
1173
- },
1174
- {
1175
- "epoch": 0.9823529411764705,
1176
- "grad_norm": 4.647606372833252,
1177
- "learning_rate": 0.00013454901960784313,
1178
- "loss": 0.05253278613090515,
1179
- "step": 1670
1180
- },
1181
- {
1182
- "epoch": 0.9882352941176471,
1183
- "grad_norm": 0.0031903591006994247,
1184
- "learning_rate": 0.00013415686274509804,
1185
- "loss": 0.05996617674827576,
1186
- "step": 1680
1187
- },
1188
- {
1189
- "epoch": 0.9941176470588236,
1190
- "grad_norm": 3.4767584800720215,
1191
- "learning_rate": 0.00013376470588235295,
1192
- "loss": 0.09587143063545227,
1193
- "step": 1690
1194
- },
1195
- {
1196
- "epoch": 1.0,
1197
- "grad_norm": 0.6253292560577393,
1198
- "learning_rate": 0.00013337254901960786,
1199
- "loss": 0.004908151552081108,
1200
- "step": 1700
1201
- },
1202
- {
1203
- "epoch": 1.0,
1204
- "eval_accuracy": 0.9833333333333333,
1205
- "eval_loss": 0.052190084010362625,
1206
- "eval_runtime": 107.7754,
1207
- "eval_samples_per_second": 44.537,
1208
- "eval_steps_per_second": 2.784,
1209
- "step": 1700
1210
- },
1211
- {
1212
- "epoch": 1.0058823529411764,
1213
- "grad_norm": 0.00878067221492529,
1214
- "learning_rate": 0.00013298039215686275,
1215
- "loss": 0.04642733633518219,
1216
- "step": 1710
1217
- },
1218
- {
1219
- "epoch": 1.011764705882353,
1220
- "grad_norm": 0.04165046289563179,
1221
- "learning_rate": 0.00013258823529411766,
1222
- "loss": 0.01063329577445984,
1223
- "step": 1720
1224
- },
1225
- {
1226
- "epoch": 1.0176470588235293,
1227
- "grad_norm": 0.008701024577021599,
1228
- "learning_rate": 0.00013219607843137257,
1229
- "loss": 0.05080258846282959,
1230
- "step": 1730
1231
- },
1232
- {
1233
- "epoch": 1.0235294117647058,
1234
- "grad_norm": 0.019413400441408157,
1235
- "learning_rate": 0.00013180392156862745,
1236
- "loss": 0.047126364707946775,
1237
- "step": 1740
1238
- },
1239
- {
1240
- "epoch": 1.0294117647058822,
1241
- "grad_norm": 3.103982925415039,
1242
- "learning_rate": 0.00013141176470588236,
1243
- "loss": 0.029054802656173707,
1244
- "step": 1750
1245
- },
1246
- {
1247
- "epoch": 1.035294117647059,
1248
- "grad_norm": 0.00992797501385212,
1249
- "learning_rate": 0.00013101960784313727,
1250
- "loss": 0.010611901432275772,
1251
- "step": 1760
1252
- },
1253
- {
1254
- "epoch": 1.0411764705882354,
1255
- "grad_norm": 0.011621154844760895,
1256
- "learning_rate": 0.00013062745098039219,
1257
- "loss": 0.05253502130508423,
1258
- "step": 1770
1259
- },
1260
- {
1261
- "epoch": 1.0470588235294118,
1262
- "grad_norm": 0.38787615299224854,
1263
- "learning_rate": 0.00013023529411764704,
1264
- "loss": 0.05395628809928894,
1265
- "step": 1780
1266
- },
1267
- {
1268
- "epoch": 1.0529411764705883,
1269
- "grad_norm": 2.963691234588623,
1270
- "learning_rate": 0.00012984313725490195,
1271
- "loss": 0.11310794353485107,
1272
- "step": 1790
1273
- },
1274
- {
1275
- "epoch": 1.0588235294117647,
1276
- "grad_norm": 0.1639397144317627,
1277
- "learning_rate": 0.00012945098039215686,
1278
- "loss": 0.020070001482963562,
1279
- "step": 1800
1280
- },
1281
- {
1282
- "epoch": 1.0647058823529412,
1283
- "grad_norm": 1.1359699964523315,
1284
- "learning_rate": 0.00012905882352941178,
1285
- "loss": 0.050649511814117434,
1286
- "step": 1810
1287
- },
1288
- {
1289
- "epoch": 1.0705882352941176,
1290
- "grad_norm": 0.5498014092445374,
1291
- "learning_rate": 0.00012866666666666666,
1292
- "loss": 0.00956227332353592,
1293
- "step": 1820
1294
- },
1295
- {
1296
- "epoch": 1.076470588235294,
1297
- "grad_norm": 0.1549549251794815,
1298
- "learning_rate": 0.00012827450980392157,
1299
- "loss": 0.013553234934806823,
1300
- "step": 1830
1301
- },
1302
- {
1303
- "epoch": 1.0823529411764705,
1304
- "grad_norm": 0.045267343521118164,
1305
- "learning_rate": 0.00012788235294117648,
1306
- "loss": 0.029942601919174194,
1307
- "step": 1840
1308
- },
1309
- {
1310
- "epoch": 1.088235294117647,
1311
- "grad_norm": 0.2236713171005249,
1312
- "learning_rate": 0.0001274901960784314,
1313
- "loss": 0.03128021359443665,
1314
- "step": 1850
1315
- },
1316
- {
1317
- "epoch": 1.0941176470588236,
1318
- "grad_norm": 0.03231283649802208,
1319
- "learning_rate": 0.00012709803921568628,
1320
- "loss": 0.045766386389732364,
1321
- "step": 1860
1322
- },
1323
- {
1324
- "epoch": 1.1,
1325
- "grad_norm": 0.0012792079942300916,
1326
- "learning_rate": 0.0001267058823529412,
1327
- "loss": 0.005739885941147804,
1328
- "step": 1870
1329
- },
1330
- {
1331
- "epoch": 1.1058823529411765,
1332
- "grad_norm": 0.08689229190349579,
1333
- "learning_rate": 0.0001263137254901961,
1334
- "loss": 0.010996489971876144,
1335
- "step": 1880
1336
- },
1337
- {
1338
- "epoch": 1.111764705882353,
1339
- "grad_norm": 2.1905863285064697,
1340
- "learning_rate": 0.00012592156862745098,
1341
- "loss": 0.054632896184921266,
1342
- "step": 1890
1343
- },
1344
- {
1345
- "epoch": 1.1176470588235294,
1346
- "grad_norm": 0.00794258713722229,
1347
- "learning_rate": 0.0001255294117647059,
1348
- "loss": 0.028382882475852966,
1349
- "step": 1900
1350
- },
1351
- {
1352
- "epoch": 1.1235294117647059,
1353
- "grad_norm": 1.3497296571731567,
1354
- "learning_rate": 0.0001251372549019608,
1355
- "loss": 0.006283405423164368,
1356
- "step": 1910
1357
- },
1358
- {
1359
- "epoch": 1.1294117647058823,
1360
- "grad_norm": 0.24390794336795807,
1361
- "learning_rate": 0.00012474509803921572,
1362
- "loss": 0.004674851521849632,
1363
- "step": 1920
1364
- },
1365
- {
1366
- "epoch": 1.1352941176470588,
1367
- "grad_norm": 1.989312767982483,
1368
- "learning_rate": 0.00012435294117647057,
1369
- "loss": 0.017390060424804687,
1370
- "step": 1930
1371
- },
1372
- {
1373
- "epoch": 1.1411764705882352,
1374
- "grad_norm": 0.0406913086771965,
1375
- "learning_rate": 0.00012396078431372548,
1376
- "loss": 0.014303833246231079,
1377
- "step": 1940
1378
- },
1379
- {
1380
- "epoch": 1.1470588235294117,
1381
- "grad_norm": 0.04144569858908653,
1382
- "learning_rate": 0.0001235686274509804,
1383
- "loss": 0.009866081923246384,
1384
- "step": 1950
1385
- },
1386
- {
1387
- "epoch": 1.1529411764705881,
1388
- "grad_norm": 4.504534721374512,
1389
- "learning_rate": 0.0001231764705882353,
1390
- "loss": 0.05014045238494873,
1391
- "step": 1960
1392
- },
1393
- {
1394
- "epoch": 1.1588235294117646,
1395
- "grad_norm": 0.07257991284132004,
1396
- "learning_rate": 0.0001227843137254902,
1397
- "loss": 0.03196141719818115,
1398
- "step": 1970
1399
- },
1400
- {
1401
- "epoch": 1.1647058823529413,
1402
- "grad_norm": 0.004560091998428106,
1403
- "learning_rate": 0.0001223921568627451,
1404
- "loss": 0.003665846586227417,
1405
- "step": 1980
1406
- },
1407
- {
1408
- "epoch": 1.1705882352941177,
1409
- "grad_norm": 5.35031795501709,
1410
- "learning_rate": 0.000122,
1411
- "loss": 0.04513788819313049,
1412
- "step": 1990
1413
- },
1414
- {
1415
- "epoch": 1.1764705882352942,
1416
- "grad_norm": 8.097407341003418,
1417
- "learning_rate": 0.00012160784313725491,
1418
- "loss": 0.11825064420700074,
1419
- "step": 2000
1420
- },
1421
- {
1422
- "epoch": 1.1823529411764706,
1423
- "grad_norm": 0.00166159903164953,
1424
- "learning_rate": 0.0001212156862745098,
1425
- "loss": 0.0033010929822921754,
1426
- "step": 2010
1427
- },
1428
- {
1429
- "epoch": 1.188235294117647,
1430
- "grad_norm": 0.00044547455036081374,
1431
- "learning_rate": 0.00012082352941176472,
1432
- "loss": 0.017485101521015168,
1433
- "step": 2020
1434
- },
1435
- {
1436
- "epoch": 1.1941176470588235,
1437
- "grad_norm": 0.021079039201140404,
1438
- "learning_rate": 0.00012043137254901961,
1439
- "loss": 0.12937935590744018,
1440
- "step": 2030
1441
- },
1442
- {
1443
- "epoch": 1.2,
1444
- "grad_norm": 0.19882740080356598,
1445
- "learning_rate": 0.00012003921568627453,
1446
- "loss": 0.00260718259960413,
1447
- "step": 2040
1448
- },
1449
- {
1450
- "epoch": 1.2058823529411764,
1451
- "grad_norm": 0.0020646140910685062,
1452
- "learning_rate": 0.00011964705882352942,
1453
- "loss": 0.04813997447490692,
1454
- "step": 2050
1455
- },
1456
- {
1457
- "epoch": 1.2117647058823529,
1458
- "grad_norm": 0.027832122519612312,
1459
- "learning_rate": 0.00011925490196078432,
1460
- "loss": 0.07265179753303527,
1461
- "step": 2060
1462
- },
1463
- {
1464
- "epoch": 1.2176470588235295,
1465
- "grad_norm": 0.039618801325559616,
1466
- "learning_rate": 0.00011886274509803923,
1467
- "loss": 0.11547826528549195,
1468
- "step": 2070
1469
- },
1470
- {
1471
- "epoch": 1.223529411764706,
1472
- "grad_norm": 0.08470630645751953,
1473
- "learning_rate": 0.00011847058823529413,
1474
- "loss": 0.01048896461725235,
1475
- "step": 2080
1476
- },
1477
- {
1478
- "epoch": 1.2294117647058824,
1479
- "grad_norm": 0.17790746688842773,
1480
- "learning_rate": 0.00011807843137254901,
1481
- "loss": 0.01930353045463562,
1482
- "step": 2090
1483
- },
1484
- {
1485
- "epoch": 1.2352941176470589,
1486
- "grad_norm": 0.28496846556663513,
1487
- "learning_rate": 0.00011768627450980392,
1488
- "loss": 0.05006638169288635,
1489
- "step": 2100
1490
- },
1491
- {
1492
- "epoch": 1.2411764705882353,
1493
- "grad_norm": 0.9003567099571228,
1494
- "learning_rate": 0.00011729411764705882,
1495
- "loss": 0.05795367956161499,
1496
- "step": 2110
1497
- },
1498
- {
1499
- "epoch": 1.2470588235294118,
1500
- "grad_norm": 1.185327410697937,
1501
- "learning_rate": 0.00011690196078431372,
1502
- "loss": 0.030656060576438902,
1503
- "step": 2120
1504
- },
1505
- {
1506
- "epoch": 1.2529411764705882,
1507
- "grad_norm": 0.007268443703651428,
1508
- "learning_rate": 0.00011650980392156863,
1509
- "loss": 0.012932188808918,
1510
- "step": 2130
1511
- },
1512
- {
1513
- "epoch": 1.2588235294117647,
1514
- "grad_norm": 0.059161823242902756,
1515
- "learning_rate": 0.00011611764705882353,
1516
- "loss": 0.007702886313199997,
1517
- "step": 2140
1518
- },
1519
- {
1520
- "epoch": 1.2647058823529411,
1521
- "grad_norm": 4.125527381896973,
1522
- "learning_rate": 0.00011572549019607844,
1523
- "loss": 0.038613501191139224,
1524
- "step": 2150
1525
- },
1526
- {
1527
- "epoch": 1.2705882352941176,
1528
- "grad_norm": 0.054057635366916656,
1529
- "learning_rate": 0.00011533333333333334,
1530
- "loss": 0.0619198739528656,
1531
- "step": 2160
1532
- },
1533
- {
1534
- "epoch": 1.276470588235294,
1535
- "grad_norm": 0.12836681306362152,
1536
- "learning_rate": 0.00011494117647058825,
1537
- "loss": 0.034237629175186156,
1538
- "step": 2170
1539
- },
1540
- {
1541
- "epoch": 1.2823529411764705,
1542
- "grad_norm": 0.4671003818511963,
1543
- "learning_rate": 0.00011454901960784314,
1544
- "loss": 0.06266800761222839,
1545
- "step": 2180
1546
- },
1547
- {
1548
- "epoch": 1.288235294117647,
1549
- "grad_norm": 5.042869567871094,
1550
- "learning_rate": 0.00011415686274509804,
1551
- "loss": 0.04739983081817627,
1552
- "step": 2190
1553
- },
1554
- {
1555
- "epoch": 1.2941176470588236,
1556
- "grad_norm": 1.037730097770691,
1557
- "learning_rate": 0.00011376470588235295,
1558
- "loss": 0.028715655207633972,
1559
- "step": 2200
1560
- },
1561
- {
1562
- "epoch": 1.3,
1563
- "grad_norm": 1.5472325086593628,
1564
- "learning_rate": 0.00011337254901960785,
1565
- "loss": 0.030545881390571593,
1566
- "step": 2210
1567
- },
1568
- {
1569
- "epoch": 1.3058823529411765,
1570
- "grad_norm": 0.015368582680821419,
1571
- "learning_rate": 0.00011298039215686276,
1572
- "loss": 0.036614406108856204,
1573
- "step": 2220
1574
- },
1575
- {
1576
- "epoch": 1.311764705882353,
1577
- "grad_norm": 1.9070743322372437,
1578
- "learning_rate": 0.00011258823529411766,
1579
- "loss": 0.013732218742370605,
1580
- "step": 2230
1581
- },
1582
- {
1583
- "epoch": 1.3176470588235294,
1584
- "grad_norm": 4.028462886810303,
1585
- "learning_rate": 0.00011219607843137257,
1586
- "loss": 0.09811515212059022,
1587
- "step": 2240
1588
- },
1589
- {
1590
- "epoch": 1.3235294117647058,
1591
- "grad_norm": 0.8748077750205994,
1592
- "learning_rate": 0.00011180392156862744,
1593
- "loss": 0.02405388206243515,
1594
- "step": 2250
1595
- },
1596
- {
1597
- "epoch": 1.3294117647058823,
1598
- "grad_norm": 0.003571397392079234,
1599
- "learning_rate": 0.00011141176470588235,
1600
- "loss": 0.05482441186904907,
1601
- "step": 2260
1602
- },
1603
- {
1604
- "epoch": 1.3352941176470587,
1605
- "grad_norm": 0.07122784107923508,
1606
- "learning_rate": 0.00011101960784313725,
1607
- "loss": 0.019011862576007843,
1608
- "step": 2270
1609
- },
1610
- {
1611
- "epoch": 1.3411764705882354,
1612
- "grad_norm": 0.1454578936100006,
1613
- "learning_rate": 0.00011062745098039216,
1614
- "loss": 0.05575419068336487,
1615
- "step": 2280
1616
- },
1617
- {
1618
- "epoch": 1.3470588235294119,
1619
- "grad_norm": 0.547131359577179,
1620
- "learning_rate": 0.00011023529411764706,
1621
- "loss": 0.030268779397010802,
1622
- "step": 2290
1623
- },
1624
- {
1625
- "epoch": 1.3529411764705883,
1626
- "grad_norm": 0.004670478869229555,
1627
- "learning_rate": 0.00010984313725490197,
1628
- "loss": 0.016492824256420135,
1629
- "step": 2300
1630
- },
1631
- {
1632
- "epoch": 1.3588235294117648,
1633
- "grad_norm": 0.011210484430193901,
1634
- "learning_rate": 0.00010945098039215687,
1635
- "loss": 0.033874979615211485,
1636
- "step": 2310
1637
- },
1638
- {
1639
- "epoch": 1.3647058823529412,
1640
- "grad_norm": 0.01266207080334425,
1641
- "learning_rate": 0.00010905882352941176,
1642
- "loss": 0.05161962509155273,
1643
- "step": 2320
1644
- },
1645
- {
1646
- "epoch": 1.3705882352941177,
1647
- "grad_norm": 2.460120677947998,
1648
- "learning_rate": 0.00010866666666666667,
1649
- "loss": 0.021566420793533325,
1650
- "step": 2330
1651
- },
1652
- {
1653
- "epoch": 1.3764705882352941,
1654
- "grad_norm": 1.084808588027954,
1655
- "learning_rate": 0.00010827450980392157,
1656
- "loss": 0.061221182346343994,
1657
- "step": 2340
1658
- },
1659
- {
1660
- "epoch": 1.3823529411764706,
1661
- "grad_norm": 2.849195718765259,
1662
- "learning_rate": 0.00010788235294117648,
1663
- "loss": 0.01972988098859787,
1664
- "step": 2350
1665
- },
1666
- {
1667
- "epoch": 1.388235294117647,
1668
- "grad_norm": 0.014458741061389446,
1669
- "learning_rate": 0.00010749019607843138,
1670
- "loss": 0.05717582106590271,
1671
- "step": 2360
1672
- },
1673
- {
1674
- "epoch": 1.3941176470588235,
1675
- "grad_norm": 0.8046371936798096,
1676
- "learning_rate": 0.00010709803921568629,
1677
- "loss": 0.03862323462963104,
1678
- "step": 2370
1679
- },
1680
- {
1681
- "epoch": 1.4,
1682
- "grad_norm": 0.013660217635333538,
1683
- "learning_rate": 0.00010670588235294119,
1684
- "loss": 0.019610072672367095,
1685
- "step": 2380
1686
- },
1687
- {
1688
- "epoch": 1.4058823529411764,
1689
- "grad_norm": 0.6871390342712402,
1690
- "learning_rate": 0.00010631372549019609,
1691
- "loss": 0.03682593703269958,
1692
- "step": 2390
1693
- },
1694
- {
1695
- "epoch": 1.4117647058823528,
1696
- "grad_norm": 4.356563091278076,
1697
- "learning_rate": 0.000105921568627451,
1698
- "loss": 0.043030115962028506,
1699
- "step": 2400
1700
- },
1701
- {
1702
- "epoch": 1.4176470588235295,
1703
- "grad_norm": 0.010556154884397984,
1704
- "learning_rate": 0.00010552941176470588,
1705
- "loss": 0.03134247660636902,
1706
- "step": 2410
1707
- },
1708
- {
1709
- "epoch": 1.423529411764706,
1710
- "grad_norm": 0.6451873183250427,
1711
- "learning_rate": 0.00010513725490196078,
1712
- "loss": 0.022086340188980102,
1713
- "step": 2420
1714
- },
1715
- {
1716
- "epoch": 1.4294117647058824,
1717
- "grad_norm": 0.08090002834796906,
1718
- "learning_rate": 0.00010474509803921569,
1719
- "loss": 0.050183969736099246,
1720
- "step": 2430
1721
- },
1722
- {
1723
- "epoch": 1.4352941176470588,
1724
- "grad_norm": 0.9792000651359558,
1725
- "learning_rate": 0.00010435294117647059,
1726
- "loss": 0.03801586329936981,
1727
- "step": 2440
1728
- },
1729
- {
1730
- "epoch": 1.4411764705882353,
1731
- "grad_norm": 0.04448526352643967,
1732
- "learning_rate": 0.00010396078431372548,
1733
- "loss": 0.02308562248945236,
1734
- "step": 2450
1735
- },
1736
- {
1737
- "epoch": 1.4470588235294117,
1738
- "grad_norm": 1.6576273441314697,
1739
- "learning_rate": 0.0001035686274509804,
1740
- "loss": 0.07776076793670654,
1741
- "step": 2460
1742
- },
1743
- {
1744
- "epoch": 1.4529411764705882,
1745
- "grad_norm": 0.21055442094802856,
1746
- "learning_rate": 0.00010317647058823529,
1747
- "loss": 0.0028534408658742903,
1748
- "step": 2470
1749
- },
1750
- {
1751
- "epoch": 1.4588235294117646,
1752
- "grad_norm": 0.010541798546910286,
1753
- "learning_rate": 0.0001027843137254902,
1754
- "loss": 0.043876749277114865,
1755
- "step": 2480
1756
- },
1757
- {
1758
- "epoch": 1.4647058823529413,
1759
- "grad_norm": 0.28795304894447327,
1760
- "learning_rate": 0.0001023921568627451,
1761
- "loss": 0.03177463412284851,
1762
- "step": 2490
1763
- },
1764
- {
1765
- "epoch": 1.4705882352941178,
1766
- "grad_norm": 0.14733652770519257,
1767
- "learning_rate": 0.00010200000000000001,
1768
- "loss": 0.004313435778021813,
1769
- "step": 2500
1770
- },
1771
- {
1772
- "epoch": 1.4764705882352942,
1773
- "grad_norm": 0.06586641073226929,
1774
- "learning_rate": 0.00010160784313725491,
1775
- "loss": 0.02194916158914566,
1776
- "step": 2510
1777
- },
1778
- {
1779
- "epoch": 1.4823529411764707,
1780
- "grad_norm": 5.449988842010498,
1781
- "learning_rate": 0.00010121568627450981,
1782
- "loss": 0.048442333936691284,
1783
- "step": 2520
1784
- },
1785
- {
1786
- "epoch": 1.488235294117647,
1787
- "grad_norm": 0.005476059392094612,
1788
- "learning_rate": 0.00010082352941176472,
1789
- "loss": 0.05485616326332092,
1790
- "step": 2530
1791
- },
1792
- {
1793
- "epoch": 1.4941176470588236,
1794
- "grad_norm": 0.00165216822642833,
1795
- "learning_rate": 0.00010043137254901962,
1796
- "loss": 0.00865812823176384,
1797
- "step": 2540
1798
- },
1799
- {
1800
- "epoch": 1.5,
1801
- "grad_norm": 0.1594460904598236,
1802
- "learning_rate": 0.00010003921568627453,
1803
- "loss": 0.046700218319892885,
1804
- "step": 2550
1805
- },
1806
- {
1807
- "epoch": 1.5058823529411764,
1808
- "grad_norm": 0.286100834608078,
1809
- "learning_rate": 9.964705882352941e-05,
1810
- "loss": 0.019821707904338837,
1811
- "step": 2560
1812
- },
1813
- {
1814
- "epoch": 1.511764705882353,
1815
- "grad_norm": 0.03422806039452553,
1816
- "learning_rate": 9.925490196078432e-05,
1817
- "loss": 0.07443686723709106,
1818
- "step": 2570
1819
- },
1820
- {
1821
- "epoch": 1.5176470588235293,
1822
- "grad_norm": 0.006144434213638306,
1823
- "learning_rate": 9.886274509803922e-05,
1824
- "loss": 0.0359457790851593,
1825
- "step": 2580
1826
- },
1827
- {
1828
- "epoch": 1.5235294117647058,
1829
- "grad_norm": 0.37471282482147217,
1830
- "learning_rate": 9.847058823529413e-05,
1831
- "loss": 0.056217139959335326,
1832
- "step": 2590
1833
- },
1834
- {
1835
- "epoch": 1.5294117647058822,
1836
- "grad_norm": 1.6320679187774658,
1837
- "learning_rate": 9.807843137254903e-05,
1838
- "loss": 0.025465011596679688,
1839
- "step": 2600
1840
- },
1841
- {
1842
- "epoch": 1.5352941176470587,
1843
- "grad_norm": 0.05187623202800751,
1844
- "learning_rate": 9.768627450980393e-05,
1845
- "loss": 0.013015411794185638,
1846
- "step": 2610
1847
- },
1848
- {
1849
- "epoch": 1.5411764705882351,
1850
- "grad_norm": 0.004285014700144529,
1851
- "learning_rate": 9.729411764705882e-05,
1852
- "loss": 0.09077309370040894,
1853
- "step": 2620
1854
- },
1855
- {
1856
- "epoch": 1.5470588235294118,
1857
- "grad_norm": 0.008745596744120121,
1858
- "learning_rate": 9.690196078431373e-05,
1859
- "loss": 0.00500418022274971,
1860
- "step": 2630
1861
- },
1862
- {
1863
- "epoch": 1.5529411764705883,
1864
- "grad_norm": 4.836589336395264,
1865
- "learning_rate": 9.650980392156863e-05,
1866
- "loss": 0.01722148060798645,
1867
- "step": 2640
1868
- },
1869
- {
1870
- "epoch": 1.5588235294117647,
1871
- "grad_norm": 0.12729062139987946,
1872
- "learning_rate": 9.611764705882353e-05,
1873
- "loss": 0.009548765420913697,
1874
- "step": 2650
1875
- },
1876
- {
1877
- "epoch": 1.5647058823529412,
1878
- "grad_norm": 0.028509173542261124,
1879
- "learning_rate": 9.572549019607844e-05,
1880
- "loss": 0.05104944109916687,
1881
- "step": 2660
1882
- },
1883
- {
1884
- "epoch": 1.5705882352941176,
1885
- "grad_norm": 0.004232144448906183,
1886
- "learning_rate": 9.533333333333334e-05,
1887
- "loss": 0.002674447558820248,
1888
- "step": 2670
1889
- },
1890
- {
1891
- "epoch": 1.576470588235294,
1892
- "grad_norm": 0.0019184383563697338,
1893
- "learning_rate": 9.494117647058825e-05,
1894
- "loss": 0.0016546357423067094,
1895
- "step": 2680
1896
- },
1897
- {
1898
- "epoch": 1.5823529411764707,
1899
- "grad_norm": 1.01856529712677,
1900
- "learning_rate": 9.454901960784313e-05,
1901
- "loss": 0.061312144994735716,
1902
- "step": 2690
1903
- },
1904
- {
1905
- "epoch": 1.5882352941176472,
1906
- "grad_norm": 2.3208155632019043,
1907
- "learning_rate": 9.415686274509804e-05,
1908
- "loss": 0.05617377758026123,
1909
- "step": 2700
1910
- },
1911
- {
1912
- "epoch": 1.5941176470588236,
1913
- "grad_norm": 0.027910761535167694,
1914
- "learning_rate": 9.376470588235294e-05,
1915
- "loss": 0.02175644040107727,
1916
- "step": 2710
1917
- },
1918
- {
1919
- "epoch": 1.6,
1920
- "grad_norm": 5.568607330322266,
1921
- "learning_rate": 9.337254901960785e-05,
1922
- "loss": 0.027315101027488707,
1923
- "step": 2720
1924
- },
1925
- {
1926
- "epoch": 1.6058823529411765,
1927
- "grad_norm": 0.001279431045986712,
1928
- "learning_rate": 9.298039215686275e-05,
1929
- "loss": 0.04750293493270874,
1930
- "step": 2730
1931
- },
1932
- {
1933
- "epoch": 1.611764705882353,
1934
- "grad_norm": 0.01567915268242359,
1935
- "learning_rate": 9.258823529411765e-05,
1936
- "loss": 0.04267770051956177,
1937
- "step": 2740
1938
- },
1939
- {
1940
- "epoch": 1.6176470588235294,
1941
- "grad_norm": 4.32657527923584,
1942
- "learning_rate": 9.219607843137256e-05,
1943
- "loss": 0.07141047716140747,
1944
- "step": 2750
1945
- },
1946
- {
1947
- "epoch": 1.6235294117647059,
1948
- "grad_norm": 0.034185998141765594,
1949
- "learning_rate": 9.180392156862746e-05,
1950
- "loss": 0.0732846200466156,
1951
- "step": 2760
1952
- },
1953
- {
1954
- "epoch": 1.6294117647058823,
1955
- "grad_norm": 0.004520747810602188,
1956
- "learning_rate": 9.141176470588235e-05,
1957
- "loss": 0.015270189940929412,
1958
- "step": 2770
1959
- },
1960
- {
1961
- "epoch": 1.6352941176470588,
1962
- "grad_norm": 0.0034092802088707685,
1963
- "learning_rate": 9.101960784313725e-05,
1964
- "loss": 0.021787433326244353,
1965
- "step": 2780
1966
- },
1967
- {
1968
- "epoch": 1.6411764705882352,
1969
- "grad_norm": 0.01586519554257393,
1970
- "learning_rate": 9.062745098039216e-05,
1971
- "loss": 0.03586234748363495,
1972
- "step": 2790
1973
- },
1974
- {
1975
- "epoch": 1.6470588235294117,
1976
- "grad_norm": 9.93665885925293,
1977
- "learning_rate": 9.023529411764706e-05,
1978
- "loss": 0.11929636001586914,
1979
- "step": 2800
1980
- },
1981
- {
1982
- "epoch": 1.6529411764705881,
1983
- "grad_norm": 0.028572263196110725,
1984
- "learning_rate": 8.984313725490197e-05,
1985
- "loss": 0.04672665894031525,
1986
- "step": 2810
1987
- },
1988
- {
1989
- "epoch": 1.6588235294117646,
1990
- "grad_norm": 0.002881932072341442,
1991
- "learning_rate": 8.945098039215687e-05,
1992
- "loss": 0.05720874071121216,
1993
- "step": 2820
1994
- },
1995
- {
1996
- "epoch": 1.664705882352941,
1997
- "grad_norm": 0.010270626284182072,
1998
- "learning_rate": 8.905882352941178e-05,
1999
- "loss": 0.067061048746109,
2000
- "step": 2830
2001
- },
2002
- {
2003
- "epoch": 1.6705882352941175,
2004
- "grad_norm": 3.3275461196899414,
2005
- "learning_rate": 8.866666666666668e-05,
2006
- "loss": 0.06687711477279663,
2007
- "step": 2840
2008
- },
2009
- {
2010
- "epoch": 1.6764705882352942,
2011
- "grad_norm": 4.426781177520752,
2012
- "learning_rate": 8.827450980392157e-05,
2013
- "loss": 0.0418778121471405,
2014
- "step": 2850
2015
- },
2016
- {
2017
- "epoch": 1.6823529411764706,
2018
- "grad_norm": 3.0773253440856934,
2019
- "learning_rate": 8.788235294117647e-05,
2020
- "loss": 0.007562777400016785,
2021
- "step": 2860
2022
- },
2023
- {
2024
- "epoch": 1.688235294117647,
2025
- "grad_norm": 0.010491623543202877,
2026
- "learning_rate": 8.749019607843137e-05,
2027
- "loss": 0.032069438695907594,
2028
- "step": 2870
2029
- },
2030
- {
2031
- "epoch": 1.6941176470588235,
2032
- "grad_norm": 0.08409623056650162,
2033
- "learning_rate": 8.709803921568628e-05,
2034
- "loss": 0.0707733690738678,
2035
- "step": 2880
2036
- },
2037
- {
2038
- "epoch": 1.7,
2039
- "grad_norm": 0.3505629003047943,
2040
- "learning_rate": 8.670588235294118e-05,
2041
- "loss": 0.00785350576043129,
2042
- "step": 2890
2043
- },
2044
- {
2045
- "epoch": 1.7058823529411766,
2046
- "grad_norm": 0.00583046255633235,
2047
- "learning_rate": 8.631372549019609e-05,
2048
- "loss": 0.05285153985023498,
2049
- "step": 2900
2050
- },
2051
- {
2052
- "epoch": 1.711764705882353,
2053
- "grad_norm": 0.005758538376539946,
2054
- "learning_rate": 8.592156862745099e-05,
2055
- "loss": 0.03593638241291046,
2056
- "step": 2910
2057
- },
2058
- {
2059
- "epoch": 1.7176470588235295,
2060
- "grad_norm": 0.04903476685285568,
2061
- "learning_rate": 8.55294117647059e-05,
2062
- "loss": 0.015528860688209533,
2063
- "step": 2920
2064
- },
2065
- {
2066
- "epoch": 1.723529411764706,
2067
- "grad_norm": 0.009182584472000599,
2068
- "learning_rate": 8.513725490196078e-05,
2069
- "loss": 0.016127565503120424,
2070
- "step": 2930
2071
- },
2072
- {
2073
- "epoch": 1.7294117647058824,
2074
- "grad_norm": 0.002334852935746312,
2075
- "learning_rate": 8.474509803921569e-05,
2076
- "loss": 0.030905908346176146,
2077
- "step": 2940
2078
- },
2079
- {
2080
- "epoch": 1.7352941176470589,
2081
- "grad_norm": 0.27579137682914734,
2082
- "learning_rate": 8.435294117647059e-05,
2083
- "loss": 0.038137003779411316,
2084
- "step": 2950
2085
- },
2086
- {
2087
- "epoch": 1.7411764705882353,
2088
- "grad_norm": 0.03673534467816353,
2089
- "learning_rate": 8.39607843137255e-05,
2090
- "loss": 0.004237490519881248,
2091
- "step": 2960
2092
- },
2093
- {
2094
- "epoch": 1.7470588235294118,
2095
- "grad_norm": 5.617798328399658,
2096
- "learning_rate": 8.35686274509804e-05,
2097
- "loss": 0.0286111444234848,
2098
- "step": 2970
2099
- },
2100
- {
2101
- "epoch": 1.7529411764705882,
2102
- "grad_norm": 0.0029075052589178085,
2103
- "learning_rate": 8.31764705882353e-05,
2104
- "loss": 0.07802658677101135,
2105
- "step": 2980
2106
- },
2107
- {
2108
- "epoch": 1.7588235294117647,
2109
- "grad_norm": 1.9302191734313965,
2110
- "learning_rate": 8.27843137254902e-05,
2111
- "loss": 0.06619505286216736,
2112
- "step": 2990
2113
- },
2114
- {
2115
- "epoch": 1.7647058823529411,
2116
- "grad_norm": 0.0049264212138950825,
2117
- "learning_rate": 8.23921568627451e-05,
2118
- "loss": 0.003911637142300606,
2119
- "step": 3000
2120
- },
2121
- {
2122
- "epoch": 1.7705882352941176,
2123
- "grad_norm": 0.08124825358390808,
2124
- "learning_rate": 8.2e-05,
2125
- "loss": 0.049767762422561646,
2126
- "step": 3010
2127
- },
2128
- {
2129
- "epoch": 1.776470588235294,
2130
- "grad_norm": 0.04015158861875534,
2131
- "learning_rate": 8.16078431372549e-05,
2132
- "loss": 0.039362430572509766,
2133
- "step": 3020
2134
- },
2135
- {
2136
- "epoch": 1.7823529411764705,
2137
- "grad_norm": 0.009305250830948353,
2138
- "learning_rate": 8.121568627450981e-05,
2139
- "loss": 0.03347316980361938,
2140
- "step": 3030
2141
- },
2142
- {
2143
- "epoch": 1.788235294117647,
2144
- "grad_norm": 0.24932274222373962,
2145
- "learning_rate": 8.082352941176471e-05,
2146
- "loss": 0.06300787329673767,
2147
- "step": 3040
2148
- },
2149
- {
2150
- "epoch": 1.7941176470588234,
2151
- "grad_norm": 0.9365462064743042,
2152
- "learning_rate": 8.043137254901962e-05,
2153
- "loss": 0.005259803682565689,
2154
- "step": 3050
2155
- },
2156
- {
2157
- "epoch": 1.8,
2158
- "grad_norm": 0.04040015488862991,
2159
- "learning_rate": 8.003921568627452e-05,
2160
- "loss": 0.0727255880832672,
2161
- "step": 3060
2162
- },
2163
- {
2164
- "epoch": 1.8058823529411765,
2165
- "grad_norm": 0.20401322841644287,
2166
- "learning_rate": 7.964705882352941e-05,
2167
- "loss": 0.12481111288070679,
2168
- "step": 3070
2169
- },
2170
- {
2171
- "epoch": 1.811764705882353,
2172
- "grad_norm": 0.047548212110996246,
2173
- "learning_rate": 7.925490196078432e-05,
2174
- "loss": 0.008852196484804153,
2175
- "step": 3080
2176
- },
2177
- {
2178
- "epoch": 1.8176470588235294,
2179
- "grad_norm": 0.09739190340042114,
2180
- "learning_rate": 7.886274509803921e-05,
2181
- "loss": 0.013921891152858735,
2182
- "step": 3090
2183
- },
2184
- {
2185
- "epoch": 1.8235294117647058,
2186
- "grad_norm": 0.04188184440135956,
2187
- "learning_rate": 7.847058823529412e-05,
2188
- "loss": 0.03531156182289123,
2189
- "step": 3100
2190
- },
2191
- {
2192
- "epoch": 1.8294117647058825,
2193
- "grad_norm": 0.04765227064490318,
2194
- "learning_rate": 7.807843137254902e-05,
2195
- "loss": 0.0103824220597744,
2196
- "step": 3110
2197
- },
2198
- {
2199
- "epoch": 1.835294117647059,
2200
- "grad_norm": 0.2406538426876068,
2201
- "learning_rate": 7.768627450980393e-05,
2202
- "loss": 0.15724188089370728,
2203
- "step": 3120
2204
- },
2205
- {
2206
- "epoch": 1.8411764705882354,
2207
- "grad_norm": 5.855648040771484,
2208
- "learning_rate": 7.729411764705882e-05,
2209
- "loss": 0.0942530632019043,
2210
- "step": 3130
2211
- },
2212
- {
2213
- "epoch": 1.8470588235294119,
2214
- "grad_norm": 0.027591396123170853,
2215
- "learning_rate": 7.690196078431374e-05,
2216
- "loss": 0.007234179973602295,
2217
- "step": 3140
2218
- },
2219
- {
2220
- "epoch": 1.8529411764705883,
2221
- "grad_norm": 0.015235557220876217,
2222
- "learning_rate": 7.650980392156863e-05,
2223
- "loss": 0.04118633568286896,
2224
- "step": 3150
2225
- },
2226
- {
2227
- "epoch": 1.8588235294117648,
2228
- "grad_norm": 1.8588212728500366,
2229
- "learning_rate": 7.611764705882354e-05,
2230
- "loss": 0.016291515529155733,
2231
- "step": 3160
2232
- },
2233
- {
2234
- "epoch": 1.8647058823529412,
2235
- "grad_norm": 3.833609104156494,
2236
- "learning_rate": 7.572549019607843e-05,
2237
- "loss": 0.02955688238143921,
2238
- "step": 3170
2239
- },
2240
- {
2241
- "epoch": 1.8705882352941177,
2242
- "grad_norm": 1.548680305480957,
2243
- "learning_rate": 7.533333333333334e-05,
2244
- "loss": 0.02775830924510956,
2245
- "step": 3180
2246
- },
2247
- {
2248
- "epoch": 1.8764705882352941,
2249
- "grad_norm": 0.1539326310157776,
2250
- "learning_rate": 7.494117647058824e-05,
2251
- "loss": 0.01504003256559372,
2252
- "step": 3190
2253
- },
2254
- {
2255
- "epoch": 1.8823529411764706,
2256
- "grad_norm": 0.009701904840767384,
2257
- "learning_rate": 7.454901960784313e-05,
2258
- "loss": 0.037118193507194516,
2259
- "step": 3200
2260
- },
2261
- {
2262
- "epoch": 1.888235294117647,
2263
- "grad_norm": 0.0187819916754961,
2264
- "learning_rate": 7.415686274509805e-05,
2265
- "loss": 0.010768245905637741,
2266
- "step": 3210
2267
- },
2268
- {
2269
- "epoch": 1.8941176470588235,
2270
- "grad_norm": 0.02519003115594387,
2271
- "learning_rate": 7.376470588235294e-05,
2272
- "loss": 0.042407462000846864,
2273
- "step": 3220
2274
- },
2275
- {
2276
- "epoch": 1.9,
2277
- "grad_norm": 3.355506658554077,
2278
- "learning_rate": 7.337254901960785e-05,
2279
- "loss": 0.0523089587688446,
2280
- "step": 3230
2281
- },
2282
- {
2283
- "epoch": 1.9058823529411764,
2284
- "grad_norm": 1.4742717742919922,
2285
- "learning_rate": 7.298039215686275e-05,
2286
- "loss": 0.04612267315387726,
2287
- "step": 3240
2288
- },
2289
- {
2290
- "epoch": 1.9117647058823528,
2291
- "grad_norm": 4.523307800292969,
2292
- "learning_rate": 7.258823529411765e-05,
2293
- "loss": 0.024737949669361114,
2294
- "step": 3250
2295
- },
2296
- {
2297
- "epoch": 1.9176470588235293,
2298
- "grad_norm": 0.014893022365868092,
2299
- "learning_rate": 7.219607843137255e-05,
2300
- "loss": 0.10542804002761841,
2301
- "step": 3260
2302
- },
2303
- {
2304
- "epoch": 1.923529411764706,
2305
- "grad_norm": 0.1594671905040741,
2306
- "learning_rate": 7.180392156862746e-05,
2307
- "loss": 0.006839567422866821,
2308
- "step": 3270
2309
- },
2310
- {
2311
- "epoch": 1.9294117647058824,
2312
- "grad_norm": 0.07281392067670822,
2313
- "learning_rate": 7.141176470588235e-05,
2314
- "loss": 0.032911264896392824,
2315
- "step": 3280
2316
- },
2317
- {
2318
- "epoch": 1.9352941176470588,
2319
- "grad_norm": 0.12037048488855362,
2320
- "learning_rate": 7.101960784313727e-05,
2321
- "loss": 0.04403464496135712,
2322
- "step": 3290
2323
- },
2324
- {
2325
- "epoch": 1.9411764705882353,
2326
- "grad_norm": 0.46801379323005676,
2327
- "learning_rate": 7.062745098039216e-05,
2328
- "loss": 0.0025000225752592085,
2329
- "step": 3300
2330
- },
2331
- {
2332
- "epoch": 1.9470588235294117,
2333
- "grad_norm": 0.0035815690644085407,
2334
- "learning_rate": 7.023529411764706e-05,
2335
- "loss": 0.0006198661867529154,
2336
- "step": 3310
2337
- },
2338
- {
2339
- "epoch": 1.9529411764705882,
2340
- "grad_norm": 0.02339448593556881,
2341
- "learning_rate": 6.984313725490197e-05,
2342
- "loss": 0.02695143222808838,
2343
- "step": 3320
2344
- },
2345
- {
2346
- "epoch": 1.9588235294117649,
2347
- "grad_norm": 0.022110486403107643,
2348
- "learning_rate": 6.945098039215686e-05,
2349
- "loss": 0.03771721124649048,
2350
- "step": 3330
2351
- },
2352
- {
2353
- "epoch": 1.9647058823529413,
2354
- "grad_norm": 0.6860880255699158,
2355
- "learning_rate": 6.905882352941177e-05,
2356
- "loss": 0.02944064438343048,
2357
- "step": 3340
2358
- },
2359
- {
2360
- "epoch": 1.9705882352941178,
2361
- "grad_norm": 0.10796581953763962,
2362
- "learning_rate": 6.866666666666666e-05,
2363
- "loss": 0.011810501664876938,
2364
- "step": 3350
2365
- },
2366
- {
2367
- "epoch": 1.9764705882352942,
2368
- "grad_norm": 0.4359405040740967,
2369
- "learning_rate": 6.827450980392158e-05,
2370
- "loss": 0.006068695709109307,
2371
- "step": 3360
2372
- },
2373
- {
2374
- "epoch": 1.9823529411764707,
2375
- "grad_norm": 1.829695463180542,
2376
- "learning_rate": 6.788235294117647e-05,
2377
- "loss": 0.03863932192325592,
2378
- "step": 3370
2379
- },
2380
- {
2381
- "epoch": 1.988235294117647,
2382
- "grad_norm": 3.4061267375946045,
2383
- "learning_rate": 6.749019607843138e-05,
2384
- "loss": 0.021592351794242858,
2385
- "step": 3380
2386
- },
2387
- {
2388
- "epoch": 1.9941176470588236,
2389
- "grad_norm": 0.003634863765910268,
2390
- "learning_rate": 6.709803921568628e-05,
2391
- "loss": 0.03054191470146179,
2392
- "step": 3390
2393
- },
2394
- {
2395
- "epoch": 2.0,
2396
- "grad_norm": 0.8191121220588684,
2397
- "learning_rate": 6.670588235294118e-05,
2398
- "loss": 0.029895269870758058,
2399
- "step": 3400
2400
- },
2401
- {
2402
- "epoch": 2.0,
2403
- "eval_accuracy": 0.9827083333333333,
2404
- "eval_loss": 0.05821743607521057,
2405
- "eval_runtime": 68.3903,
2406
- "eval_samples_per_second": 70.185,
2407
- "eval_steps_per_second": 4.387,
2408
- "step": 3400
2409
- }
2410
- ],
2411
- "logging_steps": 10,
2412
- "max_steps": 5100,
2413
- "num_input_tokens_seen": 0,
2414
- "num_train_epochs": 3,
2415
- "save_steps": 500,
2416
- "stateful_callbacks": {
2417
- "TrainerControl": {
2418
- "args": {
2419
- "should_epoch_stop": false,
2420
- "should_evaluate": false,
2421
- "should_log": false,
2422
- "should_save": true,
2423
- "should_training_stop": false
2424
- },
2425
- "attributes": {}
2426
- }
2427
- },
2428
- "total_flos": 288590756436096.0,
2429
- "train_batch_size": 16,
2430
- "trial_name": null,
2431
- "trial_params": null
2432
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
training/training-checkpoints/checkpoint-3400/training_args.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:404392ce274606d42167ab282bdb788e763b38ede0704b0120ec45be3aba6567
3
- size 5201
 
 
 
 
training/training-checkpoints/checkpoint-5100/README.md DELETED
@@ -1,202 +0,0 @@
1
- ---
2
- base_model: distilbert-base-uncased
3
- library_name: peft
4
- ---
5
-
6
- # Model Card for Model ID
7
-
8
- <!-- Provide a quick summary of what the model is/does. -->
9
-
10
-
11
-
12
- ## Model Details
13
-
14
- ### Model Description
15
-
16
- <!-- Provide a longer summary of what this model is. -->
17
-
18
-
19
-
20
- - **Developed by:** [More Information Needed]
21
- - **Funded by [optional]:** [More Information Needed]
22
- - **Shared by [optional]:** [More Information Needed]
23
- - **Model type:** [More Information Needed]
24
- - **Language(s) (NLP):** [More Information Needed]
25
- - **License:** [More Information Needed]
26
- - **Finetuned from model [optional]:** [More Information Needed]
27
-
28
- ### Model Sources [optional]
29
-
30
- <!-- Provide the basic links for the model. -->
31
-
32
- - **Repository:** [More Information Needed]
33
- - **Paper [optional]:** [More Information Needed]
34
- - **Demo [optional]:** [More Information Needed]
35
-
36
- ## Uses
37
-
38
- <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
39
-
40
- ### Direct Use
41
-
42
- <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
43
-
44
- [More Information Needed]
45
-
46
- ### Downstream Use [optional]
47
-
48
- <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
49
-
50
- [More Information Needed]
51
-
52
- ### Out-of-Scope Use
53
-
54
- <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
55
-
56
- [More Information Needed]
57
-
58
- ## Bias, Risks, and Limitations
59
-
60
- <!-- This section is meant to convey both technical and sociotechnical limitations. -->
61
-
62
- [More Information Needed]
63
-
64
- ### Recommendations
65
-
66
- <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
67
-
68
- Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
69
-
70
- ## How to Get Started with the Model
71
-
72
- Use the code below to get started with the model.
73
-
74
- [More Information Needed]
75
-
76
- ## Training Details
77
-
78
- ### Training Data
79
-
80
- <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
81
-
82
- [More Information Needed]
83
-
84
- ### Training Procedure
85
-
86
- <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
87
-
88
- #### Preprocessing [optional]
89
-
90
- [More Information Needed]
91
-
92
-
93
- #### Training Hyperparameters
94
-
95
- - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
96
-
97
- #### Speeds, Sizes, Times [optional]
98
-
99
- <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
100
-
101
- [More Information Needed]
102
-
103
- ## Evaluation
104
-
105
- <!-- This section describes the evaluation protocols and provides the results. -->
106
-
107
- ### Testing Data, Factors & Metrics
108
-
109
- #### Testing Data
110
-
111
- <!-- This should link to a Dataset Card if possible. -->
112
-
113
- [More Information Needed]
114
-
115
- #### Factors
116
-
117
- <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
118
-
119
- [More Information Needed]
120
-
121
- #### Metrics
122
-
123
- <!-- These are the evaluation metrics being used, ideally with a description of why. -->
124
-
125
- [More Information Needed]
126
-
127
- ### Results
128
-
129
- [More Information Needed]
130
-
131
- #### Summary
132
-
133
-
134
-
135
- ## Model Examination [optional]
136
-
137
- <!-- Relevant interpretability work for the model goes here -->
138
-
139
- [More Information Needed]
140
-
141
- ## Environmental Impact
142
-
143
- <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
144
-
145
- Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
146
-
147
- - **Hardware Type:** [More Information Needed]
148
- - **Hours used:** [More Information Needed]
149
- - **Cloud Provider:** [More Information Needed]
150
- - **Compute Region:** [More Information Needed]
151
- - **Carbon Emitted:** [More Information Needed]
152
-
153
- ## Technical Specifications [optional]
154
-
155
- ### Model Architecture and Objective
156
-
157
- [More Information Needed]
158
-
159
- ### Compute Infrastructure
160
-
161
- [More Information Needed]
162
-
163
- #### Hardware
164
-
165
- [More Information Needed]
166
-
167
- #### Software
168
-
169
- [More Information Needed]
170
-
171
- ## Citation [optional]
172
-
173
- <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
174
-
175
- **BibTeX:**
176
-
177
- [More Information Needed]
178
-
179
- **APA:**
180
-
181
- [More Information Needed]
182
-
183
- ## Glossary [optional]
184
-
185
- <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
186
-
187
- [More Information Needed]
188
-
189
- ## More Information [optional]
190
-
191
- [More Information Needed]
192
-
193
- ## Model Card Authors [optional]
194
-
195
- [More Information Needed]
196
-
197
- ## Model Card Contact
198
-
199
- [More Information Needed]
200
- ### Framework versions
201
-
202
- - PEFT 0.13.2
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
training/training-checkpoints/checkpoint-5100/adapter_config.json DELETED
@@ -1,32 +0,0 @@
1
- {
2
- "alpha_pattern": {},
3
- "auto_mapping": null,
4
- "base_model_name_or_path": "distilbert-base-uncased",
5
- "bias": "none",
6
- "fan_in_fan_out": false,
7
- "inference_mode": true,
8
- "init_lora_weights": true,
9
- "layer_replication": null,
10
- "layers_pattern": null,
11
- "layers_to_transform": null,
12
- "loftq_config": {},
13
- "lora_alpha": 16,
14
- "lora_dropout": 0.1,
15
- "megatron_config": null,
16
- "megatron_core": "megatron.core",
17
- "modules_to_save": [
18
- "classifier",
19
- "score"
20
- ],
21
- "peft_type": "LORA",
22
- "r": 8,
23
- "rank_pattern": {},
24
- "revision": null,
25
- "target_modules": [
26
- "v_lin",
27
- "q_lin"
28
- ],
29
- "task_type": "SEQ_CLS",
30
- "use_dora": false,
31
- "use_rslora": false
32
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
training/training-checkpoints/checkpoint-5100/adapter_model.safetensors DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:539235288638662cf62a8196575c95b797586db7c157251881af32fe95d1b935
3
- size 2962192
 
 
 
 
training/training-checkpoints/checkpoint-5100/optimizer.pt DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:25e8451ccb6ca743e82b27ba033e735ac3a79d7e573865b6810c9ae563fc8968
3
- size 5940315
 
 
 
 
training/training-checkpoints/checkpoint-5100/rng_state.pth DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:1e0fd4366ad299ac64bf657f956c3615de203baba7f7b3c3d54214d7eccf7f4b
3
- size 14455
 
 
 
 
training/training-checkpoints/checkpoint-5100/scheduler.pt DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:2df167e9de46c439c0f82b9e4834e23dbcaf6d8f75d374ef1e941952acfae2cc
3
- size 1465
 
 
 
 
training/training-checkpoints/checkpoint-5100/tokenizer.json DELETED
The diff for this file is too large to render. See raw diff
 
training/training-checkpoints/checkpoint-5100/tokenizer_config.json DELETED
@@ -1,15 +0,0 @@
1
- {
2
- "backend": "tokenizers",
3
- "cls_token": "[CLS]",
4
- "do_lower_case": true,
5
- "is_local": false,
6
- "local_files_only": false,
7
- "mask_token": "[MASK]",
8
- "model_max_length": 512,
9
- "pad_token": "[PAD]",
10
- "sep_token": "[SEP]",
11
- "strip_accents": null,
12
- "tokenize_chinese_chars": true,
13
- "tokenizer_class": "BertTokenizer",
14
- "unk_token": "[UNK]"
15
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
training/training-checkpoints/checkpoint-5100/trainer_state.json DELETED
The diff for this file is too large to render. See raw diff
 
training/training-checkpoints/checkpoint-5100/training_args.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:404392ce274606d42167ab282bdb788e763b38ede0704b0120ec45be3aba6567
3
- size 5201