agentlans commited on
Commit
ed0aabd
·
verified ·
1 Parent(s): e6bb610

Upload 11 files

Browse files
README.md CHANGED
@@ -1,3 +1,434 @@
1
  ---
 
2
  license: mit
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ language: en
3
  license: mit
4
+ base_model:
5
+ - avsolatorio/GIST-small-Embedding-v0
6
+ datasets:
7
+ - agentlans/en-document-format-classification
8
+ tags:
9
+ - bert
10
+ - text-classification
11
+ - sequence-classification
12
+ metrics:
13
+ - f1
14
+ - loss
15
+ model-index:
16
+ - name: GIST-small-format
17
+ results:
18
+ - task:
19
+ type: text-classification
20
+ name: Text Classification
21
+ metrics:
22
+ - type: f1
23
+ value: 0.8518
24
+ name: Evaluation F1
25
+ - type: loss
26
+ value: 0.3809
27
+ name: Evaluation Loss
28
  ---
29
+ # GIST-small-format
30
+
31
+ A fine-tuned version of the **bert** architecture (`BertForSequenceClassification`) optimized for the `text-classification` task.
32
+
33
+ - **Model type:** bert
34
+ - **Problem Type:** single_label_classification
35
+ - **Number of Labels:** 24
36
+ - **Vocabulary Size:** 30522
37
+ - **License:** MIT
38
+
39
+ ## Use
40
+
41
+ To get started with this model in Python using the Hugging Face Transformers library, run the following code:
42
+
43
+ ```python
44
+ from transformers import AutoTokenizer, AutoModelForSequenceClassification
45
+ import torch
46
+
47
+ model_id = "agentlans/GIST-small-format"
48
+ tokenizer = AutoTokenizer.from_pretrained(model_id)
49
+ model = AutoModelForSequenceClassification.from_pretrained(model_id)
50
+
51
+ text = "Replace this with your input text."
52
+ inputs = tokenizer(text, return_tensors="pt")
53
+
54
+ with torch.no_grad():
55
+ logits = model(**inputs).logits
56
+
57
+ predicted_class_id = logits.argmax().item()
58
+ predicted_class_name = model.config.id2label[predicted_class_id]
59
+
60
+ print(f"Predicted Class ID: {predicted_class_id}")
61
+ print(f"Predicted Class Name: {predicted_class_name}")
62
+ ```
63
+
64
+ ## Intended Uses & Limitations
65
+
66
+ ### Intended Use
67
+ This model is designed for sequence classification tasks. Below are the specific class labels mapped to their corresponding IDs:
68
+
69
+ | Label ID | Label Name |
70
+ |---|---|
71
+ | 0 | Academic Writing |
72
+ | 1 | Content Listing |
73
+ | 2 | Creative Writing |
74
+ | 3 | Customer Support Page |
75
+ | 4 | Discussion Forum / Comment Section |
76
+ | 5 | FAQs |
77
+ | 6 | Incomplete Content |
78
+ | 7 | Knowledge Article |
79
+ | 8 | Legal Notices |
80
+ | 9 | Listicle |
81
+ | 10 | News Article |
82
+ | 11 | Nonfiction Writing |
83
+ | 12 | Organizational About Page |
84
+ | 13 | Organizational Announcement |
85
+ | 14 | Personal About Page |
86
+ | 15 | Personal Blog |
87
+ | 16 | Product Page |
88
+ | 17 | Q&A Forum |
89
+ | 18 | Spam / Ads |
90
+ | 19 | Structured Data |
91
+ | 20 | Technical Writing |
92
+ | 21 | Transcript / Interview |
93
+ | 22 | Tutorial / How-To Guide |
94
+ | 23 | User Reviews |
95
+
96
+ ## Training Details
97
+
98
+ ### Hyperparameters
99
+ The following hyperparameters were used during fine-tuning:
100
+ - **Learning Rate:** 5e-05
101
+ - **Train Batch Size:** 8
102
+ - **Eval Batch Size:** 8
103
+ - **Optimizer:** OptimizerNames.ADAMW_TORCH_FUSED
104
+ - **Number of Epochs:** 3.0
105
+ - **Mixed Precision:** BF16
106
+
107
+ <details>
108
+ <summary><b>Show Advanced Training Configuration</b></summary>
109
+
110
+ #### Optimization & Regularization
111
+ - **Gradient Accumulation Steps:** 1
112
+ - **Learning Rate Scheduler:** SchedulerType.LINEAR
113
+ - **Warmup Steps:** 0
114
+ - **Warmup Ratio:** None
115
+ - **Weight Decay:** 0.0
116
+ - **Max Gradient Norm:** 1.0
117
+
118
+ #### Hardware & Reproducibility
119
+ - **Number of GPUs:** 1
120
+ - **Seed:** 42
121
+
122
+ </details>
123
+
124
+ ## Training Results & Evaluation
125
+
126
+ During fine-tuning, the model achieved the following results on the evaluation set:
127
+
128
+ | Metric | Value |
129
+ |---|---|
130
+ | **Train Loss** | 0.3033 |
131
+ | **Validation Loss** | 0.3809 |
132
+ | **Validation F1 Score** | 0.8518 |
133
+ | **Total FLOPs** | 1.8015e+16 |
134
+
135
+ For performance on the test set, [click here](report.md).
136
+
137
+ ### Speed Performance
138
+ - **Training Runtime:** 2259.7352 seconds
139
+ - **Train Samples per Second:** 483.897
140
+ - **Evaluation Runtime:** 21.817 seconds
141
+ - **Eval Samples per Second:** 1856.355
142
+
143
+
144
+ <details>
145
+ <summary><b>Show Detailed Training Logs</b></summary>
146
+
147
+ ### Training Logs History
148
+
149
+ | Step | Epoch | Learning Rate | Training Loss | Validation Loss | Validation F1 |
150
+ |---|---|---|---|---|---|
151
+ | 500 | 0.011 | 4.9817e-05 | 1.7311 | N/A | N/A |
152
+ | 1000 | 0.022 | 4.9635e-05 | 1.0143 | N/A | N/A |
153
+ | 1500 | 0.033 | 4.9452e-05 | 0.8819 | N/A | N/A |
154
+ | 2000 | 0.044 | 4.9269e-05 | 0.6946 | N/A | N/A |
155
+ | 2500 | 0.055 | 4.9086e-05 | 0.6733 | N/A | N/A |
156
+ | 3000 | 0.066 | 4.8903e-05 | 0.6471 | N/A | N/A |
157
+ | 3500 | 0.077 | 4.8720e-05 | 0.5822 | N/A | N/A |
158
+ | 4000 | 0.088 | 4.8537e-05 | 0.5588 | N/A | N/A |
159
+ | 4500 | 0.099 | 4.8354e-05 | 0.5702 | N/A | N/A |
160
+ | 5000 | 0.11 | 4.8171e-05 | 0.5574 | N/A | N/A |
161
+ | 5500 | 0.121 | 4.7988e-05 | 0.5319 | N/A | N/A |
162
+ | 6000 | 0.132 | 4.7806e-05 | 0.5464 | N/A | N/A |
163
+ | 6500 | 0.143 | 4.7623e-05 | 0.5369 | N/A | N/A |
164
+ | 7000 | 0.154 | 4.7440e-05 | 0.4969 | N/A | N/A |
165
+ | 7500 | 0.165 | 4.7257e-05 | 0.5199 | N/A | N/A |
166
+ | 8000 | 0.176 | 4.7074e-05 | 0.4873 | N/A | N/A |
167
+ | 8500 | 0.187 | 4.6891e-05 | 0.4851 | N/A | N/A |
168
+ | 9000 | 0.198 | 4.6708e-05 | 0.4956 | N/A | N/A |
169
+ | 9500 | 0.209 | 4.6525e-05 | 0.4595 | N/A | N/A |
170
+ | 10000 | 0.219 | 4.6342e-05 | 0.4893 | N/A | N/A |
171
+ | 10500 | 0.23 | 4.6159e-05 | 0.4713 | N/A | N/A |
172
+ | 11000 | 0.241 | 4.5977e-05 | 0.4707 | N/A | N/A |
173
+ | 11500 | 0.252 | 4.5794e-05 | 0.4377 | N/A | N/A |
174
+ | 12000 | 0.263 | 4.5611e-05 | 0.4906 | N/A | N/A |
175
+ | 12500 | 0.274 | 4.5428e-05 | 0.4677 | N/A | N/A |
176
+ | 13000 | 0.285 | 4.5245e-05 | 0.472 | N/A | N/A |
177
+ | 13500 | 0.296 | 4.5062e-05 | 0.4726 | N/A | N/A |
178
+ | 14000 | 0.307 | 4.4879e-05 | 0.4702 | N/A | N/A |
179
+ | 14500 | 0.318 | 4.4696e-05 | 0.4567 | N/A | N/A |
180
+ | 15000 | 0.329 | 4.4513e-05 | 0.4336 | N/A | N/A |
181
+ | 15500 | 0.34 | 4.4330e-05 | 0.4196 | N/A | N/A |
182
+ | 16000 | 0.351 | 4.4148e-05 | 0.478 | N/A | N/A |
183
+ | 16500 | 0.362 | 4.3965e-05 | 0.4753 | N/A | N/A |
184
+ | 17000 | 0.373 | 4.3782e-05 | 0.4525 | N/A | N/A |
185
+ | 17500 | 0.384 | 4.3599e-05 | 0.428 | N/A | N/A |
186
+ | 18000 | 0.395 | 4.3416e-05 | 0.4129 | N/A | N/A |
187
+ | 18500 | 0.406 | 4.3233e-05 | 0.4529 | N/A | N/A |
188
+ | 19000 | 0.417 | 4.3050e-05 | 0.4624 | N/A | N/A |
189
+ | 19500 | 0.428 | 4.2867e-05 | 0.4271 | N/A | N/A |
190
+ | 20000 | 0.439 | 4.2684e-05 | 0.415 | N/A | N/A |
191
+ | 20500 | 0.45 | 4.2501e-05 | 0.4233 | N/A | N/A |
192
+ | 21000 | 0.461 | 4.2319e-05 | 0.4353 | N/A | N/A |
193
+ | 21500 | 0.472 | 4.2136e-05 | 0.4334 | N/A | N/A |
194
+ | 22000 | 0.483 | 4.1953e-05 | 0.4031 | N/A | N/A |
195
+ | 22500 | 0.494 | 4.1770e-05 | 0.4011 | N/A | N/A |
196
+ | 23000 | 0.505 | 4.1587e-05 | 0.4137 | N/A | N/A |
197
+ | 23500 | 0.516 | 4.1404e-05 | 0.4478 | N/A | N/A |
198
+ | 24000 | 0.527 | 4.1221e-05 | 0.4157 | N/A | N/A |
199
+ | 24500 | 0.538 | 4.1038e-05 | 0.4276 | N/A | N/A |
200
+ | 25000 | 0.549 | 4.0855e-05 | 0.4124 | N/A | N/A |
201
+ | 25500 | 0.56 | 4.0672e-05 | 0.4065 | N/A | N/A |
202
+ | 26000 | 0.571 | 4.0490e-05 | 0.4333 | N/A | N/A |
203
+ | 26500 | 0.582 | 4.0307e-05 | 0.4098 | N/A | N/A |
204
+ | 27000 | 0.593 | 4.0124e-05 | 0.4091 | N/A | N/A |
205
+ | 27500 | 0.604 | 3.9941e-05 | 0.4346 | N/A | N/A |
206
+ | 28000 | 0.615 | 3.9758e-05 | 0.4063 | N/A | N/A |
207
+ | 28500 | 0.626 | 3.9575e-05 | 0.4149 | N/A | N/A |
208
+ | 29000 | 0.636 | 3.9392e-05 | 0.4145 | N/A | N/A |
209
+ | 29500 | 0.647 | 3.9209e-05 | 0.3984 | N/A | N/A |
210
+ | 30000 | 0.658 | 3.9026e-05 | 0.4113 | N/A | N/A |
211
+ | 30500 | 0.669 | 3.8843e-05 | 0.4131 | N/A | N/A |
212
+ | 31000 | 0.68 | 3.8661e-05 | 0.4114 | N/A | N/A |
213
+ | 31500 | 0.691 | 3.8478e-05 | 0.3915 | N/A | N/A |
214
+ | 32000 | 0.702 | 3.8295e-05 | 0.4179 | N/A | N/A |
215
+ | 32500 | 0.713 | 3.8112e-05 | 0.3832 | N/A | N/A |
216
+ | 33000 | 0.724 | 3.7929e-05 | 0.3826 | N/A | N/A |
217
+ | 33500 | 0.735 | 3.7746e-05 | 0.3716 | N/A | N/A |
218
+ | 34000 | 0.746 | 3.7563e-05 | 0.3827 | N/A | N/A |
219
+ | 34500 | 0.757 | 3.7380e-05 | 0.389 | N/A | N/A |
220
+ | 35000 | 0.768 | 3.7197e-05 | 0.4051 | N/A | N/A |
221
+ | 35500 | 0.779 | 3.7014e-05 | 0.401 | N/A | N/A |
222
+ | 36000 | 0.79 | 3.6831e-05 | 0.3962 | N/A | N/A |
223
+ | 36500 | 0.801 | 3.6649e-05 | 0.3653 | N/A | N/A |
224
+ | 37000 | 0.812 | 3.6466e-05 | 0.4085 | N/A | N/A |
225
+ | 37500 | 0.823 | 3.6283e-05 | 0.3887 | N/A | N/A |
226
+ | 38000 | 0.834 | 3.6100e-05 | 0.4255 | N/A | N/A |
227
+ | 38500 | 0.845 | 3.5917e-05 | 0.3765 | N/A | N/A |
228
+ | 39000 | 0.856 | 3.5734e-05 | 0.3964 | N/A | N/A |
229
+ | 39500 | 0.867 | 3.5551e-05 | 0.3849 | N/A | N/A |
230
+ | 40000 | 0.878 | 3.5368e-05 | 0.3928 | N/A | N/A |
231
+ | 40500 | 0.889 | 3.5185e-05 | 0.3885 | N/A | N/A |
232
+ | 41000 | 0.9 | 3.5002e-05 | 0.3638 | N/A | N/A |
233
+ | 41500 | 0.911 | 3.4820e-05 | 0.3788 | N/A | N/A |
234
+ | 42000 | 0.922 | 3.4637e-05 | 0.3845 | N/A | N/A |
235
+ | 42500 | 0.933 | 3.4454e-05 | 0.397 | N/A | N/A |
236
+ | 43000 | 0.944 | 3.4271e-05 | 0.3914 | N/A | N/A |
237
+ | 43500 | 0.955 | 3.4088e-05 | 0.3809 | N/A | N/A |
238
+ | 44000 | 0.966 | 3.3905e-05 | 0.3856 | N/A | N/A |
239
+ | 44500 | 0.977 | 3.3722e-05 | 0.3618 | N/A | N/A |
240
+ | 45000 | 0.988 | 3.3539e-05 | 0.3732 | N/A | N/A |
241
+ | 45500 | 0.999 | 3.3356e-05 | 0.3595 | N/A | N/A |
242
+ | 45562 | 1.0 | N/A | N/A | 0.3376 | 0.7906 |
243
+ | 46000 | 1.01 | 3.3173e-05 | 0.2933 | N/A | N/A |
244
+ | 46500 | 1.021 | 3.2991e-05 | 0.2831 | N/A | N/A |
245
+ | 47000 | 1.032 | 3.2808e-05 | 0.2768 | N/A | N/A |
246
+ | 47500 | 1.043 | 3.2625e-05 | 0.2947 | N/A | N/A |
247
+ | 48000 | 1.054 | 3.2442e-05 | 0.3016 | N/A | N/A |
248
+ | 48500 | 1.064 | 3.2259e-05 | 0.2888 | N/A | N/A |
249
+ | 49000 | 1.075 | 3.2076e-05 | 0.2698 | N/A | N/A |
250
+ | 49500 | 1.086 | 3.1893e-05 | 0.3016 | N/A | N/A |
251
+ | 50000 | 1.097 | 3.1710e-05 | 0.2799 | N/A | N/A |
252
+ | 50500 | 1.108 | 3.1527e-05 | 0.3036 | N/A | N/A |
253
+ | 51000 | 1.119 | 3.1344e-05 | 0.2973 | N/A | N/A |
254
+ | 51500 | 1.13 | 3.1162e-05 | 0.2932 | N/A | N/A |
255
+ | 52000 | 1.141 | 3.0979e-05 | 0.2789 | N/A | N/A |
256
+ | 52500 | 1.152 | 3.0796e-05 | 0.318 | N/A | N/A |
257
+ | 53000 | 1.163 | 3.0613e-05 | 0.2868 | N/A | N/A |
258
+ | 53500 | 1.174 | 3.0430e-05 | 0.3062 | N/A | N/A |
259
+ | 54000 | 1.185 | 3.0247e-05 | 0.3081 | N/A | N/A |
260
+ | 54500 | 1.196 | 3.0064e-05 | 0.2754 | N/A | N/A |
261
+ | 55000 | 1.207 | 2.9881e-05 | 0.3054 | N/A | N/A |
262
+ | 55500 | 1.218 | 2.9698e-05 | 0.2972 | N/A | N/A |
263
+ | 56000 | 1.229 | 2.9515e-05 | 0.3048 | N/A | N/A |
264
+ | 56500 | 1.24 | 2.9333e-05 | 0.2779 | N/A | N/A |
265
+ | 57000 | 1.251 | 2.9150e-05 | 0.2782 | N/A | N/A |
266
+ | 57500 | 1.262 | 2.8967e-05 | 0.3115 | N/A | N/A |
267
+ | 58000 | 1.273 | 2.8784e-05 | 0.2761 | N/A | N/A |
268
+ | 58500 | 1.284 | 2.8601e-05 | 0.287 | N/A | N/A |
269
+ | 59000 | 1.295 | 2.8418e-05 | 0.2967 | N/A | N/A |
270
+ | 59500 | 1.306 | 2.8235e-05 | 0.2844 | N/A | N/A |
271
+ | 60000 | 1.317 | 2.8052e-05 | 0.2875 | N/A | N/A |
272
+ | 60500 | 1.328 | 2.7869e-05 | 0.2998 | N/A | N/A |
273
+ | 61000 | 1.339 | 2.7686e-05 | 0.2731 | N/A | N/A |
274
+ | 61500 | 1.35 | 2.7504e-05 | 0.2941 | N/A | N/A |
275
+ | 62000 | 1.361 | 2.7321e-05 | 0.2899 | N/A | N/A |
276
+ | 62500 | 1.372 | 2.7138e-05 | 0.305 | N/A | N/A |
277
+ | 63000 | 1.383 | 2.6955e-05 | 0.295 | N/A | N/A |
278
+ | 63500 | 1.394 | 2.6772e-05 | 0.2861 | N/A | N/A |
279
+ | 64000 | 1.405 | 2.6589e-05 | 0.2653 | N/A | N/A |
280
+ | 64500 | 1.416 | 2.6406e-05 | 0.2749 | N/A | N/A |
281
+ | 65000 | 1.427 | 2.6223e-05 | 0.2906 | N/A | N/A |
282
+ | 65500 | 1.438 | 2.6040e-05 | 0.2712 | N/A | N/A |
283
+ | 66000 | 1.449 | 2.5857e-05 | 0.3095 | N/A | N/A |
284
+ | 66500 | 1.46 | 2.5675e-05 | 0.3181 | N/A | N/A |
285
+ | 67000 | 1.471 | 2.5492e-05 | 0.2544 | N/A | N/A |
286
+ | 67500 | 1.481 | 2.5309e-05 | 0.2504 | N/A | N/A |
287
+ | 68000 | 1.492 | 2.5126e-05 | 0.3013 | N/A | N/A |
288
+ | 68500 | 1.503 | 2.4943e-05 | 0.2951 | N/A | N/A |
289
+ | 69000 | 1.514 | 2.4760e-05 | 0.2822 | N/A | N/A |
290
+ | 69500 | 1.525 | 2.4577e-05 | 0.2482 | N/A | N/A |
291
+ | 70000 | 1.536 | 2.4394e-05 | 0.2707 | N/A | N/A |
292
+ | 70500 | 1.547 | 2.4211e-05 | 0.2601 | N/A | N/A |
293
+ | 71000 | 1.558 | 2.4028e-05 | 0.3073 | N/A | N/A |
294
+ | 71500 | 1.569 | 2.3846e-05 | 0.2815 | N/A | N/A |
295
+ | 72000 | 1.58 | 2.3663e-05 | 0.2587 | N/A | N/A |
296
+ | 72500 | 1.591 | 2.3480e-05 | 0.2825 | N/A | N/A |
297
+ | 73000 | 1.602 | 2.3297e-05 | 0.2921 | N/A | N/A |
298
+ | 73500 | 1.613 | 2.3114e-05 | 0.2775 | N/A | N/A |
299
+ | 74000 | 1.624 | 2.2931e-05 | 0.2611 | N/A | N/A |
300
+ | 74500 | 1.635 | 2.2748e-05 | 0.2656 | N/A | N/A |
301
+ | 75000 | 1.646 | 2.2565e-05 | 0.2757 | N/A | N/A |
302
+ | 75500 | 1.657 | 2.2382e-05 | 0.2574 | N/A | N/A |
303
+ | 76000 | 1.668 | 2.2199e-05 | 0.2874 | N/A | N/A |
304
+ | 76500 | 1.679 | 2.2017e-05 | 0.2507 | N/A | N/A |
305
+ | 77000 | 1.69 | 2.1834e-05 | 0.2621 | N/A | N/A |
306
+ | 77500 | 1.701 | 2.1651e-05 | 0.2704 | N/A | N/A |
307
+ | 78000 | 1.712 | 2.1468e-05 | 0.2748 | N/A | N/A |
308
+ | 78500 | 1.723 | 2.1285e-05 | 0.2705 | N/A | N/A |
309
+ | 79000 | 1.734 | 2.1102e-05 | 0.3062 | N/A | N/A |
310
+ | 79500 | 1.745 | 2.0919e-05 | 0.298 | N/A | N/A |
311
+ | 80000 | 1.756 | 2.0736e-05 | 0.2788 | N/A | N/A |
312
+ | 80500 | 1.767 | 2.0553e-05 | 0.2542 | N/A | N/A |
313
+ | 81000 | 1.778 | 2.0370e-05 | 0.2742 | N/A | N/A |
314
+ | 81500 | 1.789 | 2.0188e-05 | 0.2559 | N/A | N/A |
315
+ | 82000 | 1.8 | 2.0005e-05 | 0.2805 | N/A | N/A |
316
+ | 82500 | 1.811 | 1.9822e-05 | 0.2572 | N/A | N/A |
317
+ | 83000 | 1.822 | 1.9639e-05 | 0.2701 | N/A | N/A |
318
+ | 83500 | 1.833 | 1.9456e-05 | 0.2636 | N/A | N/A |
319
+ | 84000 | 1.844 | 1.9273e-05 | 0.2724 | N/A | N/A |
320
+ | 84500 | 1.855 | 1.9090e-05 | 0.2652 | N/A | N/A |
321
+ | 85000 | 1.866 | 1.8907e-05 | 0.2604 | N/A | N/A |
322
+ | 85500 | 1.877 | 1.8724e-05 | 0.262 | N/A | N/A |
323
+ | 86000 | 1.888 | 1.8541e-05 | 0.2594 | N/A | N/A |
324
+ | 86500 | 1.899 | 1.8359e-05 | 0.2609 | N/A | N/A |
325
+ | 87000 | 1.909 | 1.8176e-05 | 0.2769 | N/A | N/A |
326
+ | 87500 | 1.92 | 1.7993e-05 | 0.2803 | N/A | N/A |
327
+ | 88000 | 1.931 | 1.7810e-05 | 0.2635 | N/A | N/A |
328
+ | 88500 | 1.942 | 1.7627e-05 | 0.2759 | N/A | N/A |
329
+ | 89000 | 1.953 | 1.7444e-05 | 0.2669 | N/A | N/A |
330
+ | 89500 | 1.964 | 1.7261e-05 | 0.2887 | N/A | N/A |
331
+ | 90000 | 1.975 | 1.7078e-05 | 0.2838 | N/A | N/A |
332
+ | 90500 | 1.986 | 1.6895e-05 | 0.2482 | N/A | N/A |
333
+ | 91000 | 1.997 | 1.6712e-05 | 0.2344 | N/A | N/A |
334
+ | 91124 | 2.0 | N/A | N/A | 0.3564 | 0.8288 |
335
+ | 91500 | 2.008 | 1.6529e-05 | 0.1882 | N/A | N/A |
336
+ | 92000 | 2.019 | 1.6347e-05 | 0.1598 | N/A | N/A |
337
+ | 92500 | 2.03 | 1.6164e-05 | 0.1572 | N/A | N/A |
338
+ | 93000 | 2.041 | 1.5981e-05 | 0.1645 | N/A | N/A |
339
+ | 93500 | 2.052 | 1.5798e-05 | 0.1709 | N/A | N/A |
340
+ | 94000 | 2.063 | 1.5615e-05 | 0.1649 | N/A | N/A |
341
+ | 94500 | 2.074 | 1.5432e-05 | 0.1513 | N/A | N/A |
342
+ | 95000 | 2.085 | 1.5249e-05 | 0.1468 | N/A | N/A |
343
+ | 95500 | 2.096 | 1.5066e-05 | 0.2013 | N/A | N/A |
344
+ | 96000 | 2.107 | 1.4883e-05 | 0.1819 | N/A | N/A |
345
+ | 96500 | 2.118 | 1.4700e-05 | 0.1817 | N/A | N/A |
346
+ | 97000 | 2.129 | 1.4518e-05 | 0.1869 | N/A | N/A |
347
+ | 97500 | 2.14 | 1.4335e-05 | 0.164 | N/A | N/A |
348
+ | 98000 | 2.151 | 1.4152e-05 | 0.1722 | N/A | N/A |
349
+ | 98500 | 2.162 | 1.3969e-05 | 0.1798 | N/A | N/A |
350
+ | 99000 | 2.173 | 1.3786e-05 | 0.1633 | N/A | N/A |
351
+ | 99500 | 2.184 | 1.3603e-05 | 0.1812 | N/A | N/A |
352
+ | 100000 | 2.195 | 1.3420e-05 | 0.1719 | N/A | N/A |
353
+ | 100500 | 2.206 | 1.3237e-05 | 0.1556 | N/A | N/A |
354
+ | 101000 | 2.217 | 1.3054e-05 | 0.2062 | N/A | N/A |
355
+ | 101500 | 2.228 | 1.2871e-05 | 0.1641 | N/A | N/A |
356
+ | 102000 | 2.239 | 1.2689e-05 | 0.1656 | N/A | N/A |
357
+ | 102500 | 2.25 | 1.2506e-05 | 0.1639 | N/A | N/A |
358
+ | 103000 | 2.261 | 1.2323e-05 | 0.173 | N/A | N/A |
359
+ | 103500 | 2.272 | 1.2140e-05 | 0.1895 | N/A | N/A |
360
+ | 104000 | 2.283 | 1.1957e-05 | 0.1707 | N/A | N/A |
361
+ | 104500 | 2.294 | 1.1774e-05 | 0.1851 | N/A | N/A |
362
+ | 105000 | 2.305 | 1.1591e-05 | 0.1782 | N/A | N/A |
363
+ | 105500 | 2.316 | 1.1408e-05 | 0.1703 | N/A | N/A |
364
+ | 106000 | 2.327 | 1.1225e-05 | 0.1581 | N/A | N/A |
365
+ | 106500 | 2.337 | 1.1042e-05 | 0.1648 | N/A | N/A |
366
+ | 107000 | 2.348 | 1.0860e-05 | 0.1986 | N/A | N/A |
367
+ | 107500 | 2.359 | 1.0677e-05 | 0.1799 | N/A | N/A |
368
+ | 108000 | 2.37 | 1.0494e-05 | 0.1611 | N/A | N/A |
369
+ | 108500 | 2.381 | 1.0311e-05 | 0.1644 | N/A | N/A |
370
+ | 109000 | 2.392 | 1.0128e-05 | 0.1457 | N/A | N/A |
371
+ | 109500 | 2.403 | 9.9451e-06 | 0.1498 | N/A | N/A |
372
+ | 110000 | 2.414 | 9.7622e-06 | 0.1812 | N/A | N/A |
373
+ | 110500 | 2.425 | 9.5793e-06 | 0.1837 | N/A | N/A |
374
+ | 111000 | 2.436 | 9.3964e-06 | 0.167 | N/A | N/A |
375
+ | 111500 | 2.447 | 9.2135e-06 | 0.1736 | N/A | N/A |
376
+ | 112000 | 2.458 | 9.0306e-06 | 0.1638 | N/A | N/A |
377
+ | 112500 | 2.469 | 8.8477e-06 | 0.1659 | N/A | N/A |
378
+ | 113000 | 2.48 | 8.6647e-06 | 0.1822 | N/A | N/A |
379
+ | 113500 | 2.491 | 8.4818e-06 | 0.1583 | N/A | N/A |
380
+ | 114000 | 2.502 | 8.2989e-06 | 0.1486 | N/A | N/A |
381
+ | 114500 | 2.513 | 8.1160e-06 | 0.1723 | N/A | N/A |
382
+ | 115000 | 2.524 | 7.9331e-06 | 0.1505 | N/A | N/A |
383
+ | 115500 | 2.535 | 7.7502e-06 | 0.1588 | N/A | N/A |
384
+ | 116000 | 2.546 | 7.5673e-06 | 0.1401 | N/A | N/A |
385
+ | 116500 | 2.557 | 7.3844e-06 | 0.1572 | N/A | N/A |
386
+ | 117000 | 2.568 | 7.2015e-06 | 0.1915 | N/A | N/A |
387
+ | 117500 | 2.579 | 7.0186e-06 | 0.1555 | N/A | N/A |
388
+ | 118000 | 2.59 | 6.8357e-06 | 0.1793 | N/A | N/A |
389
+ | 118500 | 2.601 | 6.6528e-06 | 0.1671 | N/A | N/A |
390
+ | 119000 | 2.612 | 6.4699e-06 | 0.1405 | N/A | N/A |
391
+ | 119500 | 2.623 | 6.2870e-06 | 0.1504 | N/A | N/A |
392
+ | 120000 | 2.634 | 6.1041e-06 | 0.1683 | N/A | N/A |
393
+ | 120500 | 2.645 | 5.9212e-06 | 0.1562 | N/A | N/A |
394
+ | 121000 | 2.656 | 5.7383e-06 | 0.1648 | N/A | N/A |
395
+ | 121500 | 2.667 | 5.5554e-06 | 0.1531 | N/A | N/A |
396
+ | 122000 | 2.678 | 5.3725e-06 | 0.1582 | N/A | N/A |
397
+ | 122500 | 2.689 | 5.1896e-06 | 0.1415 | N/A | N/A |
398
+ | 123000 | 2.7 | 5.0067e-06 | 0.1391 | N/A | N/A |
399
+ | 123500 | 2.711 | 4.8238e-06 | 0.1349 | N/A | N/A |
400
+ | 124000 | 2.722 | 4.6409e-06 | 0.1772 | N/A | N/A |
401
+ | 124500 | 2.733 | 4.4580e-06 | 0.1687 | N/A | N/A |
402
+ | 125000 | 2.744 | 4.2751e-06 | 0.1536 | N/A | N/A |
403
+ | 125500 | 2.754 | 4.0922e-06 | 0.1394 | N/A | N/A |
404
+ | 126000 | 2.765 | 3.9093e-06 | 0.1729 | N/A | N/A |
405
+ | 126500 | 2.776 | 3.7264e-06 | 0.1582 | N/A | N/A |
406
+ | 127000 | 2.787 | 3.5435e-06 | 0.176 | N/A | N/A |
407
+ | 127500 | 2.798 | 3.3606e-06 | 0.1595 | N/A | N/A |
408
+ | 128000 | 2.809 | 3.1777e-06 | 0.1665 | N/A | N/A |
409
+ | 128500 | 2.82 | 2.9948e-06 | 0.1628 | N/A | N/A |
410
+ | 129000 | 2.831 | 2.8119e-06 | 0.1426 | N/A | N/A |
411
+ | 129500 | 2.842 | 2.6290e-06 | 0.164 | N/A | N/A |
412
+ | 130000 | 2.853 | 2.4461e-06 | 0.1562 | N/A | N/A |
413
+ | 130500 | 2.864 | 2.2632e-06 | 0.1632 | N/A | N/A |
414
+ | 131000 | 2.875 | 2.0803e-06 | 0.1562 | N/A | N/A |
415
+ | 131500 | 2.886 | 1.8974e-06 | 0.1538 | N/A | N/A |
416
+ | 132000 | 2.897 | 1.7145e-06 | 0.1649 | N/A | N/A |
417
+ | 132500 | 2.908 | 1.5316e-06 | 0.1576 | N/A | N/A |
418
+ | 133000 | 2.919 | 1.3487e-06 | 0.1481 | N/A | N/A |
419
+ | 133500 | 2.93 | 1.1658e-06 | 0.1535 | N/A | N/A |
420
+ | 134000 | 2.941 | 9.8291e-07 | 0.1552 | N/A | N/A |
421
+ | 134500 | 2.952 | 8.0001e-07 | 0.1542 | N/A | N/A |
422
+ | 135000 | 2.963 | 6.1711e-07 | 0.148 | N/A | N/A |
423
+ | 135500 | 2.974 | 4.3421e-07 | 0.1502 | N/A | N/A |
424
+ | 136000 | 2.985 | 2.5131e-07 | 0.1423 | N/A | N/A |
425
+ | 136500 | 2.996 | 6.8405e-08 | 0.152 | N/A | N/A |
426
+ | 136686 | 3.0 | N/A | N/A | 0.3809 | 0.8518 |
427
+
428
+ </details>
429
+
430
+
431
+ ## Framework Versions
432
+
433
+ - **Transformers:** 5.14.0.dev0
434
+ - **PyTorch:** 2.13.0+cu130
all_results.json ADDED
@@ -0,0 +1,15 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "epoch": 3.0,
3
+ "eval_f1": 0.8518024169356405,
4
+ "eval_loss": 0.38094356656074524,
5
+ "eval_runtime": 21.817,
6
+ "eval_samples": 40500,
7
+ "eval_samples_per_second": 1856.355,
8
+ "eval_steps_per_second": 232.067,
9
+ "total_flos": 1.801472479969075e+16,
10
+ "train_loss": 0.3033465779127858,
11
+ "train_runtime": 2259.7352,
12
+ "train_samples": 364493,
13
+ "train_samples_per_second": 483.897,
14
+ "train_steps_per_second": 60.488
15
+ }
config.json ADDED
@@ -0,0 +1,82 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_cross_attention": false,
3
+ "architectures": [
4
+ "BertForSequenceClassification"
5
+ ],
6
+ "attention_probs_dropout_prob": 0.1,
7
+ "bos_token_id": null,
8
+ "classifier_dropout": null,
9
+ "dtype": "float32",
10
+ "eos_token_id": null,
11
+ "hidden_act": "gelu",
12
+ "hidden_dropout_prob": 0.1,
13
+ "hidden_size": 384,
14
+ "id2label": {
15
+ "0": "Academic Writing",
16
+ "1": "Content Listing",
17
+ "2": "Creative Writing",
18
+ "3": "Customer Support Page",
19
+ "4": "Discussion Forum / Comment Section",
20
+ "5": "FAQs",
21
+ "6": "Incomplete Content",
22
+ "7": "Knowledge Article",
23
+ "8": "Legal Notices",
24
+ "9": "Listicle",
25
+ "10": "News Article",
26
+ "11": "Nonfiction Writing",
27
+ "12": "Organizational About Page",
28
+ "13": "Organizational Announcement",
29
+ "14": "Personal About Page",
30
+ "15": "Personal Blog",
31
+ "16": "Product Page",
32
+ "17": "Q&A Forum",
33
+ "18": "Spam / Ads",
34
+ "19": "Structured Data",
35
+ "20": "Technical Writing",
36
+ "21": "Transcript / Interview",
37
+ "22": "Tutorial / How-To Guide",
38
+ "23": "User Reviews"
39
+ },
40
+ "initializer_range": 0.02,
41
+ "intermediate_size": 1536,
42
+ "is_decoder": false,
43
+ "label2id": {
44
+ "Academic Writing": 0,
45
+ "Content Listing": 1,
46
+ "Creative Writing": 2,
47
+ "Customer Support Page": 3,
48
+ "Discussion Forum / Comment Section": 4,
49
+ "FAQs": 5,
50
+ "Incomplete Content": 6,
51
+ "Knowledge Article": 7,
52
+ "Legal Notices": 8,
53
+ "Listicle": 9,
54
+ "News Article": 10,
55
+ "Nonfiction Writing": 11,
56
+ "Organizational About Page": 12,
57
+ "Organizational Announcement": 13,
58
+ "Personal About Page": 14,
59
+ "Personal Blog": 15,
60
+ "Product Page": 16,
61
+ "Q&A Forum": 17,
62
+ "Spam / Ads": 18,
63
+ "Structured Data": 19,
64
+ "Technical Writing": 20,
65
+ "Transcript / Interview": 21,
66
+ "Tutorial / How-To Guide": 22,
67
+ "User Reviews": 23
68
+ },
69
+ "layer_norm_eps": 1e-12,
70
+ "max_position_embeddings": 512,
71
+ "model_type": "bert",
72
+ "num_attention_heads": 12,
73
+ "num_hidden_layers": 12,
74
+ "pad_token_id": 0,
75
+ "position_embedding_type": "absolute",
76
+ "problem_type": "single_label_classification",
77
+ "tie_word_embeddings": true,
78
+ "transformers_version": "5.14.0.dev0",
79
+ "type_vocab_size": 2,
80
+ "use_cache": false,
81
+ "vocab_size": 30522
82
+ }
eval_results.json ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "epoch": 3.0,
3
+ "eval_f1": 0.8518024169356405,
4
+ "eval_loss": 0.38094356656074524,
5
+ "eval_runtime": 21.817,
6
+ "eval_samples": 40500,
7
+ "eval_samples_per_second": 1856.355,
8
+ "eval_steps_per_second": 232.067
9
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ba87e4c1f7d3f80dd237bfc44ae92c4de8467459d3cf22d1eb1f0f92daf845c0
3
+ size 133500256
predict_results.txt ADDED
The diff for this file is too large to render. See raw diff
 
report.md ADDED
@@ -0,0 +1,74 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Model Evaluation Report
2
+
3
+ ### Classification Metrics
4
+ | | precision | recall | f1-score | support |
5
+ |:-----------------------------------|------------:|---------:|-----------:|-----------:|
6
+ | Academic Writing | 0.9258 | 0.9363 | 0.931 | 879 |
7
+ | Content Listing | 0.8955 | 0.8915 | 0.8935 | 1134 |
8
+ | Creative Writing | 0.8947 | 0.7774 | 0.832 | 328 |
9
+ | Customer Support Page | 0.8469 | 0.8557 | 0.8513 | 582 |
10
+ | Discussion Forum / Comment Section | 0.8924 | 0.8038 | 0.8458 | 423 |
11
+ | FAQs | 0.7617 | 0.8802 | 0.8167 | 167 |
12
+ | Incomplete Content | 0.8649 | 0.7619 | 0.8101 | 84 |
13
+ | Knowledge Article | 0.8233 | 0.8782 | 0.8499 | 1905 |
14
+ | Legal Notices | 0.9477 | 0.9348 | 0.9412 | 368 |
15
+ | Listicle | 0.7695 | 0.8198 | 0.7939 | 505 |
16
+ | News Article | 0.957 | 0.9514 | 0.9542 | 3933 |
17
+ | Nonfiction Writing | 0.7231 | 0.5109 | 0.5987 | 92 |
18
+ | Organizational About Page | 0.8849 | 0.9093 | 0.8969 | 1995 |
19
+ | Organizational Announcement | 0.8745 | 0.9058 | 0.8899 | 2431 |
20
+ | Personal About Page | 0.9235 | 0.9337 | 0.9286 | 724 |
21
+ | Personal Blog | 0.9257 | 0.9168 | 0.9213 | 4473 |
22
+ | Product Page | 0.9654 | 0.9581 | 0.9617 | 16314 |
23
+ | Q&A Forum | 0.9182 | 0.8938 | 0.9058 | 452 |
24
+ | Spam / Ads | 0.75 | 0.4286 | 0.5455 | 7 |
25
+ | Structured Data | 0.9258 | 0.9021 | 0.9138 | 235 |
26
+ | Technical Writing | 0.9137 | 0.8889 | 0.9011 | 405 |
27
+ | Transcript / Interview | 0.7033 | 0.7442 | 0.7232 | 86 |
28
+ | Tutorial / How-To Guide | 0.8771 | 0.8915 | 0.8842 | 2249 |
29
+ | User Reviews | 0.8792 | 0.8285 | 0.8531 | 729 |
30
+ | accuracy | 0.9243 | 0.9243 | 0.9243 | 0.9243 |
31
+ | macro avg | 0.8685 | 0.8418 | 0.8518 | 40500 |
32
+ | weighted avg | 0.925 | 0.9243 | 0.9244 | 40500 |
33
+
34
+ ### Confusion Matrix
35
+ | | Pred Academic Writing | Pred Content Listing | Pred Creative Writing | Pred Customer Support Page | Pred Discussion Forum / Comment Section | Pred FAQs | Pred Incomplete Content | Pred Knowledge Article | Pred Legal Notices | Pred Listicle | Pred News Article | Pred Nonfiction Writing | Pred Organizational About Page | Pred Organizational Announcement | Pred Personal About Page | Pred Personal Blog | Pred Product Page | Pred Q&A Forum | Pred Spam / Ads | Pred Structured Data | Pred Technical Writing | Pred Transcript / Interview | Pred Tutorial / How-To Guide | Pred User Reviews |
36
+ |:------------------------------------------|------------------------:|-----------------------:|------------------------:|-----------------------------:|------------------------------------------:|------------:|--------------------------:|-------------------------:|---------------------:|----------------:|--------------------:|--------------------------:|---------------------------------:|-----------------------------------:|---------------------------:|---------------------:|--------------------:|-----------------:|------------------:|-----------------------:|-------------------------:|------------------------------:|-------------------------------:|--------------------:|
37
+ | Actual Academic Writing | 823 | 3 | 0 | 0 | 0 | 0 | 0 | 35 | 0 | 0 | 1 | 0 | 2 | 6 | 1 | 1 | 2 | 0 | 0 | 0 | 4 | 0 | 1 | 0 |
38
+ | Actual Content Listing | 6 | 1011 | 2 | 1 | 0 | 0 | 0 | 5 | 0 | 15 | 3 | 0 | 11 | 15 | 0 | 6 | 47 | 1 | 0 | 0 | 0 | 2 | 7 | 2 |
39
+ | Actual Creative Writing | 0 | 1 | 255 | 0 | 2 | 0 | 0 | 14 | 0 | 1 | 2 | 0 | 0 | 0 | 1 | 33 | 16 | 1 | 0 | 0 | 0 | 0 | 0 | 2 |
40
+ | Actual Customer Support Page | 0 | 1 | 0 | 498 | 0 | 16 | 0 | 5 | 7 | 0 | 0 | 0 | 3 | 6 | 0 | 0 | 21 | 1 | 0 | 1 | 7 | 0 | 15 | 1 |
41
+ | Actual Discussion Forum / Comment Section | 1 | 1 | 0 | 2 | 340 | 0 | 1 | 0 | 0 | 0 | 2 | 1 | 0 | 4 | 2 | 48 | 2 | 4 | 0 | 0 | 0 | 2 | 1 | 12 |
42
+ | Actual FAQs | 0 | 0 | 0 | 2 | 0 | 147 | 0 | 5 | 0 | 1 | 0 | 0 | 1 | 0 | 0 | 0 | 4 | 6 | 0 | 0 | 0 | 0 | 1 | 0 |
43
+ | Actual Incomplete Content | 0 | 6 | 0 | 0 | 0 | 0 | 64 | 1 | 0 | 0 | 1 | 0 | 0 | 1 | 0 | 7 | 3 | 0 | 0 | 0 | 0 | 0 | 1 | 0 |
44
+ | Actual Knowledge Article | 28 | 1 | 2 | 3 | 0 | 1 | 1 | 1673 | 1 | 9 | 14 | 3 | 18 | 17 | 4 | 11 | 67 | 6 | 0 | 12 | 2 | 1 | 30 | 1 |
45
+ | Actual Legal Notices | 0 | 0 | 0 | 7 | 0 | 1 | 0 | 2 | 344 | 0 | 0 | 0 | 4 | 4 | 0 | 1 | 3 | 0 | 0 | 0 | 1 | 1 | 0 | 0 |
46
+ | Actual Listicle | 0 | 8 | 0 | 0 | 1 | 0 | 0 | 8 | 0 | 414 | 5 | 0 | 1 | 1 | 0 | 17 | 29 | 0 | 0 | 0 | 0 | 0 | 21 | 0 |
47
+ | Actual News Article | 3 | 1 | 0 | 2 | 0 | 0 | 0 | 40 | 0 | 10 | 3742 | 4 | 0 | 92 | 2 | 11 | 17 | 0 | 0 | 1 | 0 | 3 | 3 | 2 |
48
+ | Actual Nonfiction Writing | 4 | 0 | 0 | 0 | 0 | 0 | 0 | 11 | 0 | 1 | 7 | 47 | 3 | 1 | 1 | 12 | 2 | 1 | 0 | 0 | 0 | 1 | 0 | 1 |
49
+ | Actual Organizational About Page | 2 | 10 | 1 | 4 | 1 | 1 | 0 | 17 | 3 | 1 | 1 | 2 | 1814 | 32 | 11 | 4 | 86 | 0 | 0 | 2 | 0 | 2 | 0 | 1 |
50
+ | Actual Organizational Announcement | 4 | 11 | 0 | 5 | 0 | 0 | 1 | 13 | 2 | 2 | 65 | 0 | 46 | 2202 | 1 | 25 | 42 | 0 | 0 | 0 | 3 | 4 | 5 | 0 |
51
+ | Actual Personal About Page | 1 | 4 | 0 | 0 | 2 | 0 | 0 | 3 | 0 | 0 | 1 | 0 | 8 | 1 | 676 | 16 | 6 | 1 | 0 | 0 | 0 | 4 | 0 | 1 |
52
+ | Actual Personal Blog | 3 | 5 | 15 | 0 | 23 | 1 | 4 | 40 | 0 | 30 | 25 | 5 | 8 | 43 | 9 | 4101 | 55 | 4 | 0 | 1 | 0 | 4 | 79 | 18 |
53
+ | Actual Product Page | 6 | 52 | 6 | 37 | 0 | 6 | 2 | 97 | 4 | 34 | 21 | 2 | 126 | 83 | 19 | 42 | 15631 | 1 | 1 | 0 | 6 | 1 | 100 | 37 |
54
+ | Actual Q&A Forum | 2 | 0 | 1 | 3 | 4 | 17 | 0 | 1 | 1 | 0 | 1 | 0 | 0 | 0 | 0 | 7 | 0 | 404 | 0 | 0 | 3 | 2 | 6 | 0 |
55
+ | Actual Spam / Ads | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 3 | 0 | 0 | 0 | 0 | 0 |
56
+ | Actual Structured Data | 2 | 5 | 0 | 0 | 0 | 0 | 0 | 7 | 0 | 1 | 2 | 0 | 0 | 0 | 1 | 0 | 3 | 0 | 0 | 212 | 1 | 0 | 0 | 1 |
57
+ | Actual Technical Writing | 3 | 0 | 0 | 5 | 0 | 0 | 0 | 9 | 0 | 0 | 0 | 0 | 1 | 2 | 0 | 1 | 13 | 2 | 0 | 0 | 360 | 0 | 9 | 0 |
58
+ | Actual Transcript / Interview | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 8 | 0 | 1 | 3 | 2 | 4 | 0 | 0 | 0 | 0 | 0 | 64 | 0 | 2 |
59
+ | Actual Tutorial / How-To Guide | 0 | 5 | 1 | 19 | 0 | 2 | 1 | 40 | 1 | 19 | 8 | 0 | 0 | 5 | 0 | 52 | 76 | 6 | 0 | 0 | 7 | 0 | 2005 | 2 |
60
+ | Actual User Reviews | 0 | 2 | 1 | 0 | 8 | 1 | 0 | 5 | 0 | 0 | 1 | 1 | 3 | 0 | 2 | 31 | 66 | 2 | 0 | 0 | 0 | 0 | 2 | 604 |
61
+
62
+ ### Sampled Predictions
63
+ | Input | Prediction | Actual |
64
+ |:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:--------------------------|:--------------------------|
65
+ | We work with Marketing Managers and Digital Managers in mid-large companies and government departments, helping you to prepare and execute your digital strategy. Every engagement begins with a clear understanding of the goals for your company, and ho... | Product Page | Product Page |
66
+ | It doesn’t quite feel like summer in Seattle until the fourth of July. This weekend in the 80′s confirmed it. Thursday night kick off after work celebrating Marky’s birthday with beers and Thai food with his family. Friday morning brunch at our house... | Personal Blog | Personal Blog |
67
+ | Unlike most mammals, birds, and insects, fish often continue to grow after they reach sexual maturity. Although growth typically slows down just before sexual maturation as resources are channeled to gonads and reproduction, it can still continue for... | Knowledge Article | Knowledge Article |
68
+ | The RN Care Manager serves as a Bachelor's level care manager with responsibility for assessing, coordinating patient care, evaluating and updating the plan of care across the continuum utilizing accepted criteria and guidelines to manage benefits an... | Product Page | Product Page |
69
+ | By doing all the hard work I bring my website top on Google rankings but still no clicks on it. No business leads even no traffic is being obtained by my website from Google searches. What I come through is that people search a specific keyword and t... | Personal Blog | Personal Blog |
70
+ | I have had the opportunity to review quite a number of desi (Indian Writing in English) works by first time authors in the recent past.While some of them had been really good,the majority of them were either passable or slightly above average works.S... | User Reviews | User Reviews |
71
+ | An amputee long jumper who would have beaten Greg Rutherford at London 2012 with the 8.40m leap he registered to win IPC World Championships gold on Friday is ready to fight for his right to compete at the Rio Olympics next summer. Germany’s Markus R... | News Article | News Article |
72
+ | For a business that regularly uses tools it’s important to make sure you invest in long lasting good quality products. It’s also important that these tools are performing to your standards and are reliable to your business’s needs. Therefore finding... | Product Page | Product Page |
73
+ | By land, by sea, by air, we provide first class freight services. We strive to help you realize your business goals. Amoy International 2004-2018. All rights reserved. | Organizational About Page | Organizational About Page |
74
+ | FERRER GAZPACHO TARRO 660 ML. SUNNY DELIGHT CALIFORNIA 1250 ML. SUNNY DELIGHT FLORIDA 1250 ML. SUNNY DELIGHT FLORIDA 310 ML. | Product Page | Product Page |
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json ADDED
@@ -0,0 +1,18 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "backend": "tokenizers",
3
+ "clean_up_tokenization_spaces": true,
4
+ "cls_token": "[CLS]",
5
+ "do_basic_tokenize": true,
6
+ "do_lower_case": true,
7
+ "is_local": false,
8
+ "local_files_only": false,
9
+ "mask_token": "[MASK]",
10
+ "model_max_length": 512,
11
+ "never_split": null,
12
+ "pad_token": "[PAD]",
13
+ "sep_token": "[SEP]",
14
+ "strip_accents": null,
15
+ "tokenize_chinese_chars": true,
16
+ "tokenizer_class": "BertTokenizer",
17
+ "unk_token": "[UNK]"
18
+ }
train_results.json ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "epoch": 3.0,
3
+ "total_flos": 1.801472479969075e+16,
4
+ "train_loss": 0.3033465779127858,
5
+ "train_runtime": 2259.7352,
6
+ "train_samples": 364493,
7
+ "train_samples_per_second": 483.897,
8
+ "train_steps_per_second": 60.488
9
+ }
trainer_state.json ADDED
@@ -0,0 +1,1981 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_global_step": 136686,
3
+ "best_metric": 0.8518024169356405,
4
+ "best_model_checkpoint": "./GIST-small-format/checkpoint-136686",
5
+ "epoch": 3.0,
6
+ "eval_steps": 500,
7
+ "global_step": 136686,
8
+ "is_hyper_param_search": false,
9
+ "is_local_process_zero": true,
10
+ "is_world_process_zero": true,
11
+ "log_history": [
12
+ {
13
+ "epoch": 0.010974057328475484,
14
+ "grad_norm": 6.6852498054504395,
15
+ "learning_rate": 4.981746484643636e-05,
16
+ "loss": 1.731101806640625,
17
+ "step": 500
18
+ },
19
+ {
20
+ "epoch": 0.021948114656950967,
21
+ "grad_norm": 3.2897143363952637,
22
+ "learning_rate": 4.963456389096176e-05,
23
+ "loss": 1.01430029296875,
24
+ "step": 1000
25
+ },
26
+ {
27
+ "epoch": 0.03292217198542645,
28
+ "grad_norm": 8.402548789978027,
29
+ "learning_rate": 4.945166293548718e-05,
30
+ "loss": 0.881885498046875,
31
+ "step": 1500
32
+ },
33
+ {
34
+ "epoch": 0.043896229313901934,
35
+ "grad_norm": 6.612447261810303,
36
+ "learning_rate": 4.9268761980012585e-05,
37
+ "loss": 0.69460986328125,
38
+ "step": 2000
39
+ },
40
+ {
41
+ "epoch": 0.05487028664237742,
42
+ "grad_norm": 12.039176940917969,
43
+ "learning_rate": 4.908586102453799e-05,
44
+ "loss": 0.6732612915039062,
45
+ "step": 2500
46
+ },
47
+ {
48
+ "epoch": 0.0658443439708529,
49
+ "grad_norm": 2.494370937347412,
50
+ "learning_rate": 4.89029600690634e-05,
51
+ "loss": 0.6471351318359375,
52
+ "step": 3000
53
+ },
54
+ {
55
+ "epoch": 0.07681840129932839,
56
+ "grad_norm": 21.64371109008789,
57
+ "learning_rate": 4.8720059113588815e-05,
58
+ "loss": 0.582156494140625,
59
+ "step": 3500
60
+ },
61
+ {
62
+ "epoch": 0.08779245862780387,
63
+ "grad_norm": 14.828876495361328,
64
+ "learning_rate": 4.853715815811422e-05,
65
+ "loss": 0.5588424072265625,
66
+ "step": 4000
67
+ },
68
+ {
69
+ "epoch": 0.09876651595627936,
70
+ "grad_norm": 1.5009714365005493,
71
+ "learning_rate": 4.835425720263963e-05,
72
+ "loss": 0.5702305908203125,
73
+ "step": 4500
74
+ },
75
+ {
76
+ "epoch": 0.10974057328475484,
77
+ "grad_norm": 34.568260192871094,
78
+ "learning_rate": 4.817135624716504e-05,
79
+ "loss": 0.5573681640625,
80
+ "step": 5000
81
+ },
82
+ {
83
+ "epoch": 0.12071463061323032,
84
+ "grad_norm": 12.301100730895996,
85
+ "learning_rate": 4.7988455291690446e-05,
86
+ "loss": 0.5319334106445313,
87
+ "step": 5500
88
+ },
89
+ {
90
+ "epoch": 0.1316886879417058,
91
+ "grad_norm": 6.479952335357666,
92
+ "learning_rate": 4.780555433621585e-05,
93
+ "loss": 0.546407958984375,
94
+ "step": 6000
95
+ },
96
+ {
97
+ "epoch": 0.1426627452701813,
98
+ "grad_norm": 30.429779052734375,
99
+ "learning_rate": 4.762265338074126e-05,
100
+ "loss": 0.5369138793945313,
101
+ "step": 6500
102
+ },
103
+ {
104
+ "epoch": 0.15363680259865678,
105
+ "grad_norm": 9.562945365905762,
106
+ "learning_rate": 4.743975242526667e-05,
107
+ "loss": 0.4968977355957031,
108
+ "step": 7000
109
+ },
110
+ {
111
+ "epoch": 0.16461085992713226,
112
+ "grad_norm": 6.738670825958252,
113
+ "learning_rate": 4.7256851469792076e-05,
114
+ "loss": 0.5199419555664062,
115
+ "step": 7500
116
+ },
117
+ {
118
+ "epoch": 0.17558491725560774,
119
+ "grad_norm": 20.670921325683594,
120
+ "learning_rate": 4.707395051431749e-05,
121
+ "loss": 0.48726055908203125,
122
+ "step": 8000
123
+ },
124
+ {
125
+ "epoch": 0.18655897458408321,
126
+ "grad_norm": 28.577016830444336,
127
+ "learning_rate": 4.68910495588429e-05,
128
+ "loss": 0.48510040283203126,
129
+ "step": 8500
130
+ },
131
+ {
132
+ "epoch": 0.19753303191255872,
133
+ "grad_norm": 22.50349235534668,
134
+ "learning_rate": 4.6708148603368306e-05,
135
+ "loss": 0.49556842041015625,
136
+ "step": 9000
137
+ },
138
+ {
139
+ "epoch": 0.2085070892410342,
140
+ "grad_norm": 0.931214451789856,
141
+ "learning_rate": 4.6525247647893714e-05,
142
+ "loss": 0.45953961181640623,
143
+ "step": 9500
144
+ },
145
+ {
146
+ "epoch": 0.21948114656950968,
147
+ "grad_norm": 23.271512985229492,
148
+ "learning_rate": 4.634234669241913e-05,
149
+ "loss": 0.4892506408691406,
150
+ "step": 10000
151
+ },
152
+ {
153
+ "epoch": 0.23045520389798516,
154
+ "grad_norm": 16.353809356689453,
155
+ "learning_rate": 4.6159445736944536e-05,
156
+ "loss": 0.4712514953613281,
157
+ "step": 10500
158
+ },
159
+ {
160
+ "epoch": 0.24142926122646063,
161
+ "grad_norm": 6.07758903503418,
162
+ "learning_rate": 4.597654478146994e-05,
163
+ "loss": 0.4706602783203125,
164
+ "step": 11000
165
+ },
166
+ {
167
+ "epoch": 0.25240331855493614,
168
+ "grad_norm": 10.723750114440918,
169
+ "learning_rate": 4.5793643825995345e-05,
170
+ "loss": 0.4377469177246094,
171
+ "step": 11500
172
+ },
173
+ {
174
+ "epoch": 0.2633773758834116,
175
+ "grad_norm": 8.667707443237305,
176
+ "learning_rate": 4.561074287052076e-05,
177
+ "loss": 0.49056134033203125,
178
+ "step": 12000
179
+ },
180
+ {
181
+ "epoch": 0.2743514332118871,
182
+ "grad_norm": 14.63868522644043,
183
+ "learning_rate": 4.542784191504617e-05,
184
+ "loss": 0.4677280883789062,
185
+ "step": 12500
186
+ },
187
+ {
188
+ "epoch": 0.2853254905403626,
189
+ "grad_norm": 28.209775924682617,
190
+ "learning_rate": 4.5244940959571575e-05,
191
+ "loss": 0.47199853515625,
192
+ "step": 13000
193
+ },
194
+ {
195
+ "epoch": 0.29629954786883805,
196
+ "grad_norm": 16.671207427978516,
197
+ "learning_rate": 4.506204000409698e-05,
198
+ "loss": 0.47259246826171875,
199
+ "step": 13500
200
+ },
201
+ {
202
+ "epoch": 0.30727360519731356,
203
+ "grad_norm": 7.109496116638184,
204
+ "learning_rate": 4.487913904862239e-05,
205
+ "loss": 0.470185791015625,
206
+ "step": 14000
207
+ },
208
+ {
209
+ "epoch": 0.318247662525789,
210
+ "grad_norm": 4.374630451202393,
211
+ "learning_rate": 4.4696238093147805e-05,
212
+ "loss": 0.45671673583984373,
213
+ "step": 14500
214
+ },
215
+ {
216
+ "epoch": 0.3292217198542645,
217
+ "grad_norm": 61.96080780029297,
218
+ "learning_rate": 4.451333713767321e-05,
219
+ "loss": 0.433604248046875,
220
+ "step": 15000
221
+ },
222
+ {
223
+ "epoch": 0.34019577718274,
224
+ "grad_norm": 21.4468936920166,
225
+ "learning_rate": 4.433043618219862e-05,
226
+ "loss": 0.4195879516601563,
227
+ "step": 15500
228
+ },
229
+ {
230
+ "epoch": 0.3511698345112155,
231
+ "grad_norm": 12.094574928283691,
232
+ "learning_rate": 4.414753522672402e-05,
233
+ "loss": 0.47797442626953124,
234
+ "step": 16000
235
+ },
236
+ {
237
+ "epoch": 0.362143891839691,
238
+ "grad_norm": 0.06602894514799118,
239
+ "learning_rate": 4.3964634271249435e-05,
240
+ "loss": 0.4752947692871094,
241
+ "step": 16500
242
+ },
243
+ {
244
+ "epoch": 0.37311794916816643,
245
+ "grad_norm": 24.168350219726562,
246
+ "learning_rate": 4.378173331577484e-05,
247
+ "loss": 0.4525231628417969,
248
+ "step": 17000
249
+ },
250
+ {
251
+ "epoch": 0.38409200649664194,
252
+ "grad_norm": 5.945094585418701,
253
+ "learning_rate": 4.359883236030025e-05,
254
+ "loss": 0.42800213623046873,
255
+ "step": 17500
256
+ },
257
+ {
258
+ "epoch": 0.39506606382511744,
259
+ "grad_norm": 3.3805999755859375,
260
+ "learning_rate": 4.341593140482566e-05,
261
+ "loss": 0.4128626708984375,
262
+ "step": 18000
263
+ },
264
+ {
265
+ "epoch": 0.4060401211535929,
266
+ "grad_norm": 0.3303009271621704,
267
+ "learning_rate": 4.3233030449351066e-05,
268
+ "loss": 0.45289547729492186,
269
+ "step": 18500
270
+ },
271
+ {
272
+ "epoch": 0.4170141784820684,
273
+ "grad_norm": 7.178033351898193,
274
+ "learning_rate": 4.305012949387648e-05,
275
+ "loss": 0.4623618774414063,
276
+ "step": 19000
277
+ },
278
+ {
279
+ "epoch": 0.42798823581054385,
280
+ "grad_norm": 0.13193656504154205,
281
+ "learning_rate": 4.286722853840189e-05,
282
+ "loss": 0.42712686157226565,
283
+ "step": 19500
284
+ },
285
+ {
286
+ "epoch": 0.43896229313901935,
287
+ "grad_norm": 3.611271858215332,
288
+ "learning_rate": 4.2684327582927296e-05,
289
+ "loss": 0.41502728271484374,
290
+ "step": 20000
291
+ },
292
+ {
293
+ "epoch": 0.44993635046749486,
294
+ "grad_norm": 22.556659698486328,
295
+ "learning_rate": 4.2501426627452704e-05,
296
+ "loss": 0.4233431396484375,
297
+ "step": 20500
298
+ },
299
+ {
300
+ "epoch": 0.4609104077959703,
301
+ "grad_norm": 19.0096378326416,
302
+ "learning_rate": 4.231852567197811e-05,
303
+ "loss": 0.4352963256835938,
304
+ "step": 21000
305
+ },
306
+ {
307
+ "epoch": 0.4718844651244458,
308
+ "grad_norm": 12.138071060180664,
309
+ "learning_rate": 4.213562471650352e-05,
310
+ "loss": 0.433427490234375,
311
+ "step": 21500
312
+ },
313
+ {
314
+ "epoch": 0.48285852245292127,
315
+ "grad_norm": 2.0555145740509033,
316
+ "learning_rate": 4.195272376102893e-05,
317
+ "loss": 0.4031333923339844,
318
+ "step": 22000
319
+ },
320
+ {
321
+ "epoch": 0.4938325797813968,
322
+ "grad_norm": 2.4884192943573,
323
+ "learning_rate": 4.1769822805554335e-05,
324
+ "loss": 0.4010719604492187,
325
+ "step": 22500
326
+ },
327
+ {
328
+ "epoch": 0.5048066371098723,
329
+ "grad_norm": 0.3784475028514862,
330
+ "learning_rate": 4.158692185007975e-05,
331
+ "loss": 0.4136856689453125,
332
+ "step": 23000
333
+ },
334
+ {
335
+ "epoch": 0.5157806944383477,
336
+ "grad_norm": 20.747249603271484,
337
+ "learning_rate": 4.140402089460516e-05,
338
+ "loss": 0.4478377075195312,
339
+ "step": 23500
340
+ },
341
+ {
342
+ "epoch": 0.5267547517668232,
343
+ "grad_norm": 30.383005142211914,
344
+ "learning_rate": 4.1221119939130565e-05,
345
+ "loss": 0.41568099975585937,
346
+ "step": 24000
347
+ },
348
+ {
349
+ "epoch": 0.5377288090952987,
350
+ "grad_norm": 16.23149299621582,
351
+ "learning_rate": 4.103821898365597e-05,
352
+ "loss": 0.4275780944824219,
353
+ "step": 24500
354
+ },
355
+ {
356
+ "epoch": 0.5487028664237742,
357
+ "grad_norm": 0.1475469321012497,
358
+ "learning_rate": 4.085531802818138e-05,
359
+ "loss": 0.4123694458007813,
360
+ "step": 25000
361
+ },
362
+ {
363
+ "epoch": 0.5596769237522496,
364
+ "grad_norm": 2.036740779876709,
365
+ "learning_rate": 4.0672417072706794e-05,
366
+ "loss": 0.40654916381835937,
367
+ "step": 25500
368
+ },
369
+ {
370
+ "epoch": 0.5706509810807252,
371
+ "grad_norm": 1.523879051208496,
372
+ "learning_rate": 4.0489516117232195e-05,
373
+ "loss": 0.433327392578125,
374
+ "step": 26000
375
+ },
376
+ {
377
+ "epoch": 0.5816250384092007,
378
+ "grad_norm": 0.9519908428192139,
379
+ "learning_rate": 4.03066151617576e-05,
380
+ "loss": 0.409821533203125,
381
+ "step": 26500
382
+ },
383
+ {
384
+ "epoch": 0.5925990957376761,
385
+ "grad_norm": 7.078254699707031,
386
+ "learning_rate": 4.012371420628301e-05,
387
+ "loss": 0.4090614318847656,
388
+ "step": 27000
389
+ },
390
+ {
391
+ "epoch": 0.6035731530661517,
392
+ "grad_norm": 8.405106544494629,
393
+ "learning_rate": 3.9940813250808425e-05,
394
+ "loss": 0.4346309204101563,
395
+ "step": 27500
396
+ },
397
+ {
398
+ "epoch": 0.6145472103946271,
399
+ "grad_norm": 10.549320220947266,
400
+ "learning_rate": 3.975791229533383e-05,
401
+ "loss": 0.40631097412109374,
402
+ "step": 28000
403
+ },
404
+ {
405
+ "epoch": 0.6255212677231026,
406
+ "grad_norm": 7.929631233215332,
407
+ "learning_rate": 3.957501133985924e-05,
408
+ "loss": 0.41491552734375,
409
+ "step": 28500
410
+ },
411
+ {
412
+ "epoch": 0.636495325051578,
413
+ "grad_norm": 27.06610870361328,
414
+ "learning_rate": 3.939211038438465e-05,
415
+ "loss": 0.4144559326171875,
416
+ "step": 29000
417
+ },
418
+ {
419
+ "epoch": 0.6474693823800536,
420
+ "grad_norm": 20.29384422302246,
421
+ "learning_rate": 3.920920942891006e-05,
422
+ "loss": 0.39838357543945313,
423
+ "step": 29500
424
+ },
425
+ {
426
+ "epoch": 0.658443439708529,
427
+ "grad_norm": 15.49613094329834,
428
+ "learning_rate": 3.902630847343547e-05,
429
+ "loss": 0.41126773071289063,
430
+ "step": 30000
431
+ },
432
+ {
433
+ "epoch": 0.6694174970370045,
434
+ "grad_norm": 13.434142112731934,
435
+ "learning_rate": 3.884340751796088e-05,
436
+ "loss": 0.41314010620117186,
437
+ "step": 30500
438
+ },
439
+ {
440
+ "epoch": 0.68039155436548,
441
+ "grad_norm": 23.377010345458984,
442
+ "learning_rate": 3.866050656248628e-05,
443
+ "loss": 0.4113868103027344,
444
+ "step": 31000
445
+ },
446
+ {
447
+ "epoch": 0.6913656116939555,
448
+ "grad_norm": 4.771722793579102,
449
+ "learning_rate": 3.8477605607011694e-05,
450
+ "loss": 0.39154055786132813,
451
+ "step": 31500
452
+ },
453
+ {
454
+ "epoch": 0.702339669022431,
455
+ "grad_norm": 26.069446563720703,
456
+ "learning_rate": 3.82947046515371e-05,
457
+ "loss": 0.41785919189453125,
458
+ "step": 32000
459
+ },
460
+ {
461
+ "epoch": 0.7133137263509065,
462
+ "grad_norm": 29.206588745117188,
463
+ "learning_rate": 3.811180369606251e-05,
464
+ "loss": 0.3831887512207031,
465
+ "step": 32500
466
+ },
467
+ {
468
+ "epoch": 0.724287783679382,
469
+ "grad_norm": 14.96133804321289,
470
+ "learning_rate": 3.792890274058792e-05,
471
+ "loss": 0.3826472473144531,
472
+ "step": 33000
473
+ },
474
+ {
475
+ "epoch": 0.7352618410078574,
476
+ "grad_norm": 15.803548812866211,
477
+ "learning_rate": 3.7746001785113325e-05,
478
+ "loss": 0.3715686950683594,
479
+ "step": 33500
480
+ },
481
+ {
482
+ "epoch": 0.7462358983363329,
483
+ "grad_norm": 8.521240234375,
484
+ "learning_rate": 3.756310082963874e-05,
485
+ "loss": 0.3826586608886719,
486
+ "step": 34000
487
+ },
488
+ {
489
+ "epoch": 0.7572099556648084,
490
+ "grad_norm": 12.95479679107666,
491
+ "learning_rate": 3.738019987416415e-05,
492
+ "loss": 0.38898123168945314,
493
+ "step": 34500
494
+ },
495
+ {
496
+ "epoch": 0.7681840129932839,
497
+ "grad_norm": 26.654850006103516,
498
+ "learning_rate": 3.7197298918689554e-05,
499
+ "loss": 0.4050979919433594,
500
+ "step": 35000
501
+ },
502
+ {
503
+ "epoch": 0.7791580703217593,
504
+ "grad_norm": 32.3937873840332,
505
+ "learning_rate": 3.701439796321496e-05,
506
+ "loss": 0.40102090454101563,
507
+ "step": 35500
508
+ },
509
+ {
510
+ "epoch": 0.7901321276502349,
511
+ "grad_norm": 10.174785614013672,
512
+ "learning_rate": 3.683149700774037e-05,
513
+ "loss": 0.3961770935058594,
514
+ "step": 36000
515
+ },
516
+ {
517
+ "epoch": 0.8011061849787103,
518
+ "grad_norm": 16.819910049438477,
519
+ "learning_rate": 3.664859605226578e-05,
520
+ "loss": 0.365343994140625,
521
+ "step": 36500
522
+ },
523
+ {
524
+ "epoch": 0.8120802423071858,
525
+ "grad_norm": 22.253021240234375,
526
+ "learning_rate": 3.6465695096791185e-05,
527
+ "loss": 0.4085382385253906,
528
+ "step": 37000
529
+ },
530
+ {
531
+ "epoch": 0.8230542996356613,
532
+ "grad_norm": 0.653167188167572,
533
+ "learning_rate": 3.628279414131659e-05,
534
+ "loss": 0.3887478942871094,
535
+ "step": 37500
536
+ },
537
+ {
538
+ "epoch": 0.8340283569641368,
539
+ "grad_norm": 7.46025276184082,
540
+ "learning_rate": 3.609989318584201e-05,
541
+ "loss": 0.42551852416992186,
542
+ "step": 38000
543
+ },
544
+ {
545
+ "epoch": 0.8450024142926122,
546
+ "grad_norm": 3.8886020183563232,
547
+ "learning_rate": 3.5916992230367415e-05,
548
+ "loss": 0.376479248046875,
549
+ "step": 38500
550
+ },
551
+ {
552
+ "epoch": 0.8559764716210877,
553
+ "grad_norm": 23.658897399902344,
554
+ "learning_rate": 3.573409127489282e-05,
555
+ "loss": 0.3964391174316406,
556
+ "step": 39000
557
+ },
558
+ {
559
+ "epoch": 0.8669505289495633,
560
+ "grad_norm": 56.36935043334961,
561
+ "learning_rate": 3.555119031941823e-05,
562
+ "loss": 0.3848704833984375,
563
+ "step": 39500
564
+ },
565
+ {
566
+ "epoch": 0.8779245862780387,
567
+ "grad_norm": 15.337593078613281,
568
+ "learning_rate": 3.536828936394364e-05,
569
+ "loss": 0.3928073425292969,
570
+ "step": 40000
571
+ },
572
+ {
573
+ "epoch": 0.8888986436065142,
574
+ "grad_norm": 16.32798957824707,
575
+ "learning_rate": 3.518538840846905e-05,
576
+ "loss": 0.38851931762695313,
577
+ "step": 40500
578
+ },
579
+ {
580
+ "epoch": 0.8998727009349897,
581
+ "grad_norm": 1.66266667842865,
582
+ "learning_rate": 3.5002487452994454e-05,
583
+ "loss": 0.3637700500488281,
584
+ "step": 41000
585
+ },
586
+ {
587
+ "epoch": 0.9108467582634652,
588
+ "grad_norm": 26.03211212158203,
589
+ "learning_rate": 3.481958649751986e-05,
590
+ "loss": 0.37883843994140626,
591
+ "step": 41500
592
+ },
593
+ {
594
+ "epoch": 0.9218208155919406,
595
+ "grad_norm": 0.10503211617469788,
596
+ "learning_rate": 3.463668554204527e-05,
597
+ "loss": 0.384486083984375,
598
+ "step": 42000
599
+ },
600
+ {
601
+ "epoch": 0.9327948729204162,
602
+ "grad_norm": 0.09833603352308273,
603
+ "learning_rate": 3.4453784586570684e-05,
604
+ "loss": 0.39695626831054687,
605
+ "step": 42500
606
+ },
607
+ {
608
+ "epoch": 0.9437689302488916,
609
+ "grad_norm": 13.514938354492188,
610
+ "learning_rate": 3.427088363109609e-05,
611
+ "loss": 0.3914007568359375,
612
+ "step": 43000
613
+ },
614
+ {
615
+ "epoch": 0.9547429875773671,
616
+ "grad_norm": 4.098175525665283,
617
+ "learning_rate": 3.40879826756215e-05,
618
+ "loss": 0.38088681030273436,
619
+ "step": 43500
620
+ },
621
+ {
622
+ "epoch": 0.9657170449058425,
623
+ "grad_norm": 14.520209312438965,
624
+ "learning_rate": 3.390508172014691e-05,
625
+ "loss": 0.3855673828125,
626
+ "step": 44000
627
+ },
628
+ {
629
+ "epoch": 0.9766911022343181,
630
+ "grad_norm": 35.712867736816406,
631
+ "learning_rate": 3.3722180764672314e-05,
632
+ "loss": 0.361778076171875,
633
+ "step": 44500
634
+ },
635
+ {
636
+ "epoch": 0.9876651595627935,
637
+ "grad_norm": 10.093748092651367,
638
+ "learning_rate": 3.353927980919773e-05,
639
+ "loss": 0.37317227172851564,
640
+ "step": 45000
641
+ },
642
+ {
643
+ "epoch": 0.998639216891269,
644
+ "grad_norm": 4.401282787322998,
645
+ "learning_rate": 3.3356378853723137e-05,
646
+ "loss": 0.35951104736328127,
647
+ "step": 45500
648
+ },
649
+ {
650
+ "epoch": 1.0,
651
+ "eval_f1": 0.7906041351717196,
652
+ "eval_loss": 0.3376215994358063,
653
+ "eval_runtime": 21.7159,
654
+ "eval_samples_per_second": 1864.991,
655
+ "eval_steps_per_second": 233.147,
656
+ "step": 45562
657
+ },
658
+ {
659
+ "epoch": 1.0096132742197446,
660
+ "grad_norm": 0.07860906422138214,
661
+ "learning_rate": 3.317347789824854e-05,
662
+ "loss": 0.29326309204101564,
663
+ "step": 46000
664
+ },
665
+ {
666
+ "epoch": 1.02058733154822,
667
+ "grad_norm": 39.632415771484375,
668
+ "learning_rate": 3.299057694277395e-05,
669
+ "loss": 0.28309921264648436,
670
+ "step": 46500
671
+ },
672
+ {
673
+ "epoch": 1.0315613888766955,
674
+ "grad_norm": 1.5844589471817017,
675
+ "learning_rate": 3.280767598729936e-05,
676
+ "loss": 0.2768072509765625,
677
+ "step": 47000
678
+ },
679
+ {
680
+ "epoch": 1.042535446205171,
681
+ "grad_norm": 0.07393167912960052,
682
+ "learning_rate": 3.262477503182477e-05,
683
+ "loss": 0.29472137451171876,
684
+ "step": 47500
685
+ },
686
+ {
687
+ "epoch": 1.0535095035336464,
688
+ "grad_norm": 0.903986394405365,
689
+ "learning_rate": 3.2441874076350175e-05,
690
+ "loss": 0.30162600708007814,
691
+ "step": 48000
692
+ },
693
+ {
694
+ "epoch": 1.064483560862122,
695
+ "grad_norm": 12.462333679199219,
696
+ "learning_rate": 3.225897312087558e-05,
697
+ "loss": 0.28875408935546876,
698
+ "step": 48500
699
+ },
700
+ {
701
+ "epoch": 1.0754576181905975,
702
+ "grad_norm": 26.5993595123291,
703
+ "learning_rate": 3.2076072165401e-05,
704
+ "loss": 0.26983624267578127,
705
+ "step": 49000
706
+ },
707
+ {
708
+ "epoch": 1.0864316755190728,
709
+ "grad_norm": 4.48448371887207,
710
+ "learning_rate": 3.1893171209926405e-05,
711
+ "loss": 0.3015690612792969,
712
+ "step": 49500
713
+ },
714
+ {
715
+ "epoch": 1.0974057328475484,
716
+ "grad_norm": 1.3335434198379517,
717
+ "learning_rate": 3.171027025445181e-05,
718
+ "loss": 0.27986233520507814,
719
+ "step": 50000
720
+ },
721
+ {
722
+ "epoch": 1.108379790176024,
723
+ "grad_norm": 55.16224670410156,
724
+ "learning_rate": 3.152736929897722e-05,
725
+ "loss": 0.30360482788085935,
726
+ "step": 50500
727
+ },
728
+ {
729
+ "epoch": 1.1193538475044993,
730
+ "grad_norm": 0.1195051297545433,
731
+ "learning_rate": 3.134446834350263e-05,
732
+ "loss": 0.29728253173828123,
733
+ "step": 51000
734
+ },
735
+ {
736
+ "epoch": 1.1303279048329749,
737
+ "grad_norm": 15.035134315490723,
738
+ "learning_rate": 3.1161567388028036e-05,
739
+ "loss": 0.293227294921875,
740
+ "step": 51500
741
+ },
742
+ {
743
+ "epoch": 1.1413019621614504,
744
+ "grad_norm": 7.07342529296875,
745
+ "learning_rate": 3.0978666432553443e-05,
746
+ "loss": 0.2788831176757812,
747
+ "step": 52000
748
+ },
749
+ {
750
+ "epoch": 1.1522760194899258,
751
+ "grad_norm": 2.034162998199463,
752
+ "learning_rate": 3.079576547707885e-05,
753
+ "loss": 0.3180224609375,
754
+ "step": 52500
755
+ },
756
+ {
757
+ "epoch": 1.1632500768184013,
758
+ "grad_norm": 3.3663501739501953,
759
+ "learning_rate": 3.061286452160426e-05,
760
+ "loss": 0.2868128967285156,
761
+ "step": 53000
762
+ },
763
+ {
764
+ "epoch": 1.1742241341468769,
765
+ "grad_norm": 2.8967697620391846,
766
+ "learning_rate": 3.042996356612967e-05,
767
+ "loss": 0.30618685913085936,
768
+ "step": 53500
769
+ },
770
+ {
771
+ "epoch": 1.1851981914753522,
772
+ "grad_norm": 10.053667068481445,
773
+ "learning_rate": 3.024706261065508e-05,
774
+ "loss": 0.30812335205078123,
775
+ "step": 54000
776
+ },
777
+ {
778
+ "epoch": 1.1961722488038278,
779
+ "grad_norm": 4.651602268218994,
780
+ "learning_rate": 3.006416165518049e-05,
781
+ "loss": 0.2754173278808594,
782
+ "step": 54500
783
+ },
784
+ {
785
+ "epoch": 1.2071463061323033,
786
+ "grad_norm": 14.591957092285156,
787
+ "learning_rate": 2.98812606997059e-05,
788
+ "loss": 0.3053521728515625,
789
+ "step": 55000
790
+ },
791
+ {
792
+ "epoch": 1.2181203634607787,
793
+ "grad_norm": 4.575090408325195,
794
+ "learning_rate": 2.9698359744231308e-05,
795
+ "loss": 0.29722467041015627,
796
+ "step": 55500
797
+ },
798
+ {
799
+ "epoch": 1.2290944207892542,
800
+ "grad_norm": 0.7045038342475891,
801
+ "learning_rate": 2.9515458788756712e-05,
802
+ "loss": 0.30476104736328125,
803
+ "step": 56000
804
+ },
805
+ {
806
+ "epoch": 1.2400684781177298,
807
+ "grad_norm": 8.57973575592041,
808
+ "learning_rate": 2.933255783328212e-05,
809
+ "loss": 0.2778677062988281,
810
+ "step": 56500
811
+ },
812
+ {
813
+ "epoch": 1.2510425354462051,
814
+ "grad_norm": 27.191518783569336,
815
+ "learning_rate": 2.914965687780753e-05,
816
+ "loss": 0.2781588134765625,
817
+ "step": 57000
818
+ },
819
+ {
820
+ "epoch": 1.2620165927746807,
821
+ "grad_norm": 15.524397850036621,
822
+ "learning_rate": 2.896675592233294e-05,
823
+ "loss": 0.31154220581054687,
824
+ "step": 57500
825
+ },
826
+ {
827
+ "epoch": 1.2729906501031563,
828
+ "grad_norm": 0.02379167452454567,
829
+ "learning_rate": 2.878385496685835e-05,
830
+ "loss": 0.27608938598632815,
831
+ "step": 58000
832
+ },
833
+ {
834
+ "epoch": 1.2839647074316316,
835
+ "grad_norm": 6.567012310028076,
836
+ "learning_rate": 2.8600954011383757e-05,
837
+ "loss": 0.28701922607421876,
838
+ "step": 58500
839
+ },
840
+ {
841
+ "epoch": 1.2949387647601072,
842
+ "grad_norm": 6.408507823944092,
843
+ "learning_rate": 2.8418053055909165e-05,
844
+ "loss": 0.29671533203125,
845
+ "step": 59000
846
+ },
847
+ {
848
+ "epoch": 1.3059128220885827,
849
+ "grad_norm": 4.476773262023926,
850
+ "learning_rate": 2.8235152100434576e-05,
851
+ "loss": 0.2844156494140625,
852
+ "step": 59500
853
+ },
854
+ {
855
+ "epoch": 1.316886879417058,
856
+ "grad_norm": 0.4284241795539856,
857
+ "learning_rate": 2.8052251144959984e-05,
858
+ "loss": 0.28753970336914064,
859
+ "step": 60000
860
+ },
861
+ {
862
+ "epoch": 1.3278609367455336,
863
+ "grad_norm": 45.82956314086914,
864
+ "learning_rate": 2.7869350189485395e-05,
865
+ "loss": 0.2997920227050781,
866
+ "step": 60500
867
+ },
868
+ {
869
+ "epoch": 1.338834994074009,
870
+ "grad_norm": 5.1805830001831055,
871
+ "learning_rate": 2.7686449234010796e-05,
872
+ "loss": 0.2731007995605469,
873
+ "step": 61000
874
+ },
875
+ {
876
+ "epoch": 1.3498090514024845,
877
+ "grad_norm": 44.71467971801758,
878
+ "learning_rate": 2.7503548278536207e-05,
879
+ "loss": 0.29412890625,
880
+ "step": 61500
881
+ },
882
+ {
883
+ "epoch": 1.36078310873096,
884
+ "grad_norm": 0.03455616533756256,
885
+ "learning_rate": 2.7320647323061614e-05,
886
+ "loss": 0.28992022705078124,
887
+ "step": 62000
888
+ },
889
+ {
890
+ "epoch": 1.3717571660594354,
891
+ "grad_norm": 0.10311322659254074,
892
+ "learning_rate": 2.7137746367587026e-05,
893
+ "loss": 0.304996337890625,
894
+ "step": 62500
895
+ },
896
+ {
897
+ "epoch": 1.382731223387911,
898
+ "grad_norm": 16.59219741821289,
899
+ "learning_rate": 2.6954845412112433e-05,
900
+ "loss": 0.29500704956054685,
901
+ "step": 63000
902
+ },
903
+ {
904
+ "epoch": 1.3937052807163863,
905
+ "grad_norm": 0.035255905240774155,
906
+ "learning_rate": 2.6771944456637844e-05,
907
+ "loss": 0.2860797119140625,
908
+ "step": 63500
909
+ },
910
+ {
911
+ "epoch": 1.404679338044862,
912
+ "grad_norm": 7.428912162780762,
913
+ "learning_rate": 2.6589043501163252e-05,
914
+ "loss": 0.2652564697265625,
915
+ "step": 64000
916
+ },
917
+ {
918
+ "epoch": 1.4156533953733375,
919
+ "grad_norm": 38.300662994384766,
920
+ "learning_rate": 2.640614254568866e-05,
921
+ "loss": 0.27488351440429687,
922
+ "step": 64500
923
+ },
924
+ {
925
+ "epoch": 1.4266274527018128,
926
+ "grad_norm": 34.766117095947266,
927
+ "learning_rate": 2.622324159021407e-05,
928
+ "loss": 0.2906384582519531,
929
+ "step": 65000
930
+ },
931
+ {
932
+ "epoch": 1.4376015100302884,
933
+ "grad_norm": 29.75716781616211,
934
+ "learning_rate": 2.604034063473948e-05,
935
+ "loss": 0.2712423400878906,
936
+ "step": 65500
937
+ },
938
+ {
939
+ "epoch": 1.448575567358764,
940
+ "grad_norm": 0.031717050820589066,
941
+ "learning_rate": 2.5857439679264883e-05,
942
+ "loss": 0.3094512634277344,
943
+ "step": 66000
944
+ },
945
+ {
946
+ "epoch": 1.4595496246872393,
947
+ "grad_norm": 17.38152503967285,
948
+ "learning_rate": 2.5674538723790294e-05,
949
+ "loss": 0.3181193542480469,
950
+ "step": 66500
951
+ },
952
+ {
953
+ "epoch": 1.4705236820157148,
954
+ "grad_norm": 27.521625518798828,
955
+ "learning_rate": 2.54916377683157e-05,
956
+ "loss": 0.2543564910888672,
957
+ "step": 67000
958
+ },
959
+ {
960
+ "epoch": 1.4814977393441904,
961
+ "grad_norm": 0.038084983825683594,
962
+ "learning_rate": 2.530873681284111e-05,
963
+ "loss": 0.25035408020019534,
964
+ "step": 67500
965
+ },
966
+ {
967
+ "epoch": 1.4924717966726657,
968
+ "grad_norm": 21.655553817749023,
969
+ "learning_rate": 2.512583585736652e-05,
970
+ "loss": 0.30128500366210936,
971
+ "step": 68000
972
+ },
973
+ {
974
+ "epoch": 1.5034458540011413,
975
+ "grad_norm": 0.062355488538742065,
976
+ "learning_rate": 2.4942934901891928e-05,
977
+ "loss": 0.2950963745117188,
978
+ "step": 68500
979
+ },
980
+ {
981
+ "epoch": 1.5144199113296168,
982
+ "grad_norm": 9.434773445129395,
983
+ "learning_rate": 2.476003394641734e-05,
984
+ "loss": 0.2821864318847656,
985
+ "step": 69000
986
+ },
987
+ {
988
+ "epoch": 1.5253939686580922,
989
+ "grad_norm": 0.027533777058124542,
990
+ "learning_rate": 2.4577132990942747e-05,
991
+ "loss": 0.24817718505859376,
992
+ "step": 69500
993
+ },
994
+ {
995
+ "epoch": 1.5363680259865677,
996
+ "grad_norm": 17.45857810974121,
997
+ "learning_rate": 2.4394232035468155e-05,
998
+ "loss": 0.2706839294433594,
999
+ "step": 70000
1000
+ },
1001
+ {
1002
+ "epoch": 1.5473420833150433,
1003
+ "grad_norm": 0.7203682065010071,
1004
+ "learning_rate": 2.4211331079993562e-05,
1005
+ "loss": 0.26013385009765627,
1006
+ "step": 70500
1007
+ },
1008
+ {
1009
+ "epoch": 1.5583161406435186,
1010
+ "grad_norm": 20.325897216796875,
1011
+ "learning_rate": 2.4028430124518973e-05,
1012
+ "loss": 0.3072607421875,
1013
+ "step": 71000
1014
+ },
1015
+ {
1016
+ "epoch": 1.5692901979719942,
1017
+ "grad_norm": 0.8252955079078674,
1018
+ "learning_rate": 2.384552916904438e-05,
1019
+ "loss": 0.28147393798828124,
1020
+ "step": 71500
1021
+ },
1022
+ {
1023
+ "epoch": 1.5802642553004698,
1024
+ "grad_norm": 0.5657853484153748,
1025
+ "learning_rate": 2.366262821356979e-05,
1026
+ "loss": 0.25868719482421876,
1027
+ "step": 72000
1028
+ },
1029
+ {
1030
+ "epoch": 1.591238312628945,
1031
+ "grad_norm": 0.08703117072582245,
1032
+ "learning_rate": 2.3479727258095197e-05,
1033
+ "loss": 0.28247900390625,
1034
+ "step": 72500
1035
+ },
1036
+ {
1037
+ "epoch": 1.6022123699574207,
1038
+ "grad_norm": 29.79605484008789,
1039
+ "learning_rate": 2.3296826302620604e-05,
1040
+ "loss": 0.29211328125,
1041
+ "step": 73000
1042
+ },
1043
+ {
1044
+ "epoch": 1.6131864272858962,
1045
+ "grad_norm": 0.18891900777816772,
1046
+ "learning_rate": 2.3113925347146015e-05,
1047
+ "loss": 0.2774709777832031,
1048
+ "step": 73500
1049
+ },
1050
+ {
1051
+ "epoch": 1.6241604846143716,
1052
+ "grad_norm": 0.024532776325941086,
1053
+ "learning_rate": 2.2931024391671423e-05,
1054
+ "loss": 0.26110531616210936,
1055
+ "step": 74000
1056
+ },
1057
+ {
1058
+ "epoch": 1.6351345419428471,
1059
+ "grad_norm": 23.57233428955078,
1060
+ "learning_rate": 2.274812343619683e-05,
1061
+ "loss": 0.2656497802734375,
1062
+ "step": 74500
1063
+ },
1064
+ {
1065
+ "epoch": 1.6461085992713227,
1066
+ "grad_norm": 5.525993347167969,
1067
+ "learning_rate": 2.256522248072224e-05,
1068
+ "loss": 0.27565005493164063,
1069
+ "step": 75000
1070
+ },
1071
+ {
1072
+ "epoch": 1.657082656599798,
1073
+ "grad_norm": 31.11590576171875,
1074
+ "learning_rate": 2.238232152524765e-05,
1075
+ "loss": 0.257364013671875,
1076
+ "step": 75500
1077
+ },
1078
+ {
1079
+ "epoch": 1.6680567139282736,
1080
+ "grad_norm": 24.173105239868164,
1081
+ "learning_rate": 2.2199420569773057e-05,
1082
+ "loss": 0.28741323852539064,
1083
+ "step": 76000
1084
+ },
1085
+ {
1086
+ "epoch": 1.6790307712567492,
1087
+ "grad_norm": 0.019840441644191742,
1088
+ "learning_rate": 2.201651961429847e-05,
1089
+ "loss": 0.2506680450439453,
1090
+ "step": 76500
1091
+ },
1092
+ {
1093
+ "epoch": 1.6900048285852245,
1094
+ "grad_norm": 32.793251037597656,
1095
+ "learning_rate": 2.1833618658823873e-05,
1096
+ "loss": 0.2620893249511719,
1097
+ "step": 77000
1098
+ },
1099
+ {
1100
+ "epoch": 1.7009788859137,
1101
+ "grad_norm": 50.69339370727539,
1102
+ "learning_rate": 2.1650717703349284e-05,
1103
+ "loss": 0.2703638305664062,
1104
+ "step": 77500
1105
+ },
1106
+ {
1107
+ "epoch": 1.7119529432421756,
1108
+ "grad_norm": 40.141910552978516,
1109
+ "learning_rate": 2.146781674787469e-05,
1110
+ "loss": 0.27476327514648435,
1111
+ "step": 78000
1112
+ },
1113
+ {
1114
+ "epoch": 1.722927000570651,
1115
+ "grad_norm": 0.18374653160572052,
1116
+ "learning_rate": 2.12849157924001e-05,
1117
+ "loss": 0.2705036926269531,
1118
+ "step": 78500
1119
+ },
1120
+ {
1121
+ "epoch": 1.7339010578991265,
1122
+ "grad_norm": 46.508514404296875,
1123
+ "learning_rate": 2.110201483692551e-05,
1124
+ "loss": 0.3062447509765625,
1125
+ "step": 79000
1126
+ },
1127
+ {
1128
+ "epoch": 1.744875115227602,
1129
+ "grad_norm": 29.72620964050293,
1130
+ "learning_rate": 2.0919113881450918e-05,
1131
+ "loss": 0.29800909423828126,
1132
+ "step": 79500
1133
+ },
1134
+ {
1135
+ "epoch": 1.7558491725560774,
1136
+ "grad_norm": 10.616799354553223,
1137
+ "learning_rate": 2.0736212925976326e-05,
1138
+ "loss": 0.2787916259765625,
1139
+ "step": 80000
1140
+ },
1141
+ {
1142
+ "epoch": 1.7668232298845528,
1143
+ "grad_norm": 16.61284637451172,
1144
+ "learning_rate": 2.0553311970501733e-05,
1145
+ "loss": 0.25415122985839844,
1146
+ "step": 80500
1147
+ },
1148
+ {
1149
+ "epoch": 1.7777972872130285,
1150
+ "grad_norm": 5.203902244567871,
1151
+ "learning_rate": 2.0370411015027145e-05,
1152
+ "loss": 0.2742305297851563,
1153
+ "step": 81000
1154
+ },
1155
+ {
1156
+ "epoch": 1.7887713445415039,
1157
+ "grad_norm": 17.70701026916504,
1158
+ "learning_rate": 2.0187510059552552e-05,
1159
+ "loss": 0.2559151611328125,
1160
+ "step": 81500
1161
+ },
1162
+ {
1163
+ "epoch": 1.7997454018699792,
1164
+ "grad_norm": 6.252378940582275,
1165
+ "learning_rate": 2.000460910407796e-05,
1166
+ "loss": 0.28050296020507814,
1167
+ "step": 82000
1168
+ },
1169
+ {
1170
+ "epoch": 1.810719459198455,
1171
+ "grad_norm": 31.440832138061523,
1172
+ "learning_rate": 1.9821708148603368e-05,
1173
+ "loss": 0.2572430419921875,
1174
+ "step": 82500
1175
+ },
1176
+ {
1177
+ "epoch": 1.8216935165269303,
1178
+ "grad_norm": 16.332443237304688,
1179
+ "learning_rate": 1.963880719312878e-05,
1180
+ "loss": 0.2700733642578125,
1181
+ "step": 83000
1182
+ },
1183
+ {
1184
+ "epoch": 1.8326675738554057,
1185
+ "grad_norm": 0.33748501539230347,
1186
+ "learning_rate": 1.9455906237654186e-05,
1187
+ "loss": 0.2636079711914063,
1188
+ "step": 83500
1189
+ },
1190
+ {
1191
+ "epoch": 1.8436416311838815,
1192
+ "grad_norm": 0.13300059735774994,
1193
+ "learning_rate": 1.9273005282179598e-05,
1194
+ "loss": 0.2723955078125,
1195
+ "step": 84000
1196
+ },
1197
+ {
1198
+ "epoch": 1.8546156885123568,
1199
+ "grad_norm": 28.147356033325195,
1200
+ "learning_rate": 1.9090104326705002e-05,
1201
+ "loss": 0.26517648315429687,
1202
+ "step": 84500
1203
+ },
1204
+ {
1205
+ "epoch": 1.8655897458408321,
1206
+ "grad_norm": 17.65765953063965,
1207
+ "learning_rate": 1.8907203371230413e-05,
1208
+ "loss": 0.2603927001953125,
1209
+ "step": 85000
1210
+ },
1211
+ {
1212
+ "epoch": 1.8765638031693077,
1213
+ "grad_norm": 40.015869140625,
1214
+ "learning_rate": 1.872430241575582e-05,
1215
+ "loss": 0.2619692993164062,
1216
+ "step": 85500
1217
+ },
1218
+ {
1219
+ "epoch": 1.8875378604977833,
1220
+ "grad_norm": 12.848860740661621,
1221
+ "learning_rate": 1.854140146028123e-05,
1222
+ "loss": 0.25935647583007815,
1223
+ "step": 86000
1224
+ },
1225
+ {
1226
+ "epoch": 1.8985119178262586,
1227
+ "grad_norm": 0.14966364204883575,
1228
+ "learning_rate": 1.835850050480664e-05,
1229
+ "loss": 0.2609214782714844,
1230
+ "step": 86500
1231
+ },
1232
+ {
1233
+ "epoch": 1.9094859751547342,
1234
+ "grad_norm": 0.24382130801677704,
1235
+ "learning_rate": 1.8175599549332044e-05,
1236
+ "loss": 0.27692413330078125,
1237
+ "step": 87000
1238
+ },
1239
+ {
1240
+ "epoch": 1.9204600324832097,
1241
+ "grad_norm": 33.56360626220703,
1242
+ "learning_rate": 1.7992698593857455e-05,
1243
+ "loss": 0.2803267211914062,
1244
+ "step": 87500
1245
+ },
1246
+ {
1247
+ "epoch": 1.931434089811685,
1248
+ "grad_norm": 0.04589025303721428,
1249
+ "learning_rate": 1.7809797638382863e-05,
1250
+ "loss": 0.2634629821777344,
1251
+ "step": 88000
1252
+ },
1253
+ {
1254
+ "epoch": 1.9424081471401606,
1255
+ "grad_norm": 18.453758239746094,
1256
+ "learning_rate": 1.7626896682908274e-05,
1257
+ "loss": 0.2758912353515625,
1258
+ "step": 88500
1259
+ },
1260
+ {
1261
+ "epoch": 1.9533822044686362,
1262
+ "grad_norm": 29.018585205078125,
1263
+ "learning_rate": 1.744399572743368e-05,
1264
+ "loss": 0.26687152099609374,
1265
+ "step": 89000
1266
+ },
1267
+ {
1268
+ "epoch": 1.9643562617971115,
1269
+ "grad_norm": 0.32755419611930847,
1270
+ "learning_rate": 1.726109477195909e-05,
1271
+ "loss": 0.28869473266601564,
1272
+ "step": 89500
1273
+ },
1274
+ {
1275
+ "epoch": 1.975330319125587,
1276
+ "grad_norm": 0.00812600925564766,
1277
+ "learning_rate": 1.7078193816484497e-05,
1278
+ "loss": 0.2838319091796875,
1279
+ "step": 90000
1280
+ },
1281
+ {
1282
+ "epoch": 1.9863043764540627,
1283
+ "grad_norm": 22.876619338989258,
1284
+ "learning_rate": 1.6895292861009908e-05,
1285
+ "loss": 0.24822427368164063,
1286
+ "step": 90500
1287
+ },
1288
+ {
1289
+ "epoch": 1.997278433782538,
1290
+ "grad_norm": 29.792329788208008,
1291
+ "learning_rate": 1.6712391905535316e-05,
1292
+ "loss": 0.23443959045410157,
1293
+ "step": 91000
1294
+ },
1295
+ {
1296
+ "epoch": 2.0,
1297
+ "eval_f1": 0.8288003736916,
1298
+ "eval_loss": 0.356448233127594,
1299
+ "eval_runtime": 22.1665,
1300
+ "eval_samples_per_second": 1827.08,
1301
+ "eval_steps_per_second": 228.408,
1302
+ "step": 91124
1303
+ },
1304
+ {
1305
+ "epoch": 2.008252491111014,
1306
+ "grad_norm": 0.019856004044413567,
1307
+ "learning_rate": 1.6529490950060723e-05,
1308
+ "loss": 0.18815164184570313,
1309
+ "step": 91500
1310
+ },
1311
+ {
1312
+ "epoch": 2.019226548439489,
1313
+ "grad_norm": 22.867015838623047,
1314
+ "learning_rate": 1.634658999458613e-05,
1315
+ "loss": 0.1597985076904297,
1316
+ "step": 92000
1317
+ },
1318
+ {
1319
+ "epoch": 2.0302006057679645,
1320
+ "grad_norm": 0.14189456403255463,
1321
+ "learning_rate": 1.616368903911154e-05,
1322
+ "loss": 0.1571758575439453,
1323
+ "step": 92500
1324
+ },
1325
+ {
1326
+ "epoch": 2.04117466309644,
1327
+ "grad_norm": 0.006348753347992897,
1328
+ "learning_rate": 1.598078808363695e-05,
1329
+ "loss": 0.16454287719726562,
1330
+ "step": 93000
1331
+ },
1332
+ {
1333
+ "epoch": 2.0521487204249156,
1334
+ "grad_norm": 13.350561141967773,
1335
+ "learning_rate": 1.5797887128162357e-05,
1336
+ "loss": 0.17085581970214844,
1337
+ "step": 93500
1338
+ },
1339
+ {
1340
+ "epoch": 2.063122777753391,
1341
+ "grad_norm": 0.04752154275774956,
1342
+ "learning_rate": 1.561498617268777e-05,
1343
+ "loss": 0.1648643035888672,
1344
+ "step": 94000
1345
+ },
1346
+ {
1347
+ "epoch": 2.0740968350818663,
1348
+ "grad_norm": 8.686612129211426,
1349
+ "learning_rate": 1.5432085217213173e-05,
1350
+ "loss": 0.15125572204589843,
1351
+ "step": 94500
1352
+ },
1353
+ {
1354
+ "epoch": 2.085070892410342,
1355
+ "grad_norm": 0.04292794689536095,
1356
+ "learning_rate": 1.5249184261738584e-05,
1357
+ "loss": 0.1467701416015625,
1358
+ "step": 95000
1359
+ },
1360
+ {
1361
+ "epoch": 2.0960449497388174,
1362
+ "grad_norm": 43.54441452026367,
1363
+ "learning_rate": 1.5066283306263992e-05,
1364
+ "loss": 0.20134834289550782,
1365
+ "step": 95500
1366
+ },
1367
+ {
1368
+ "epoch": 2.1070190070672927,
1369
+ "grad_norm": 0.18904000520706177,
1370
+ "learning_rate": 1.4883382350789401e-05,
1371
+ "loss": 0.181921875,
1372
+ "step": 96000
1373
+ },
1374
+ {
1375
+ "epoch": 2.1179930643957685,
1376
+ "grad_norm": 0.3301223814487457,
1377
+ "learning_rate": 1.470048139531481e-05,
1378
+ "loss": 0.1817462158203125,
1379
+ "step": 96500
1380
+ },
1381
+ {
1382
+ "epoch": 2.128967121724244,
1383
+ "grad_norm": 0.1473444551229477,
1384
+ "learning_rate": 1.4517580439840216e-05,
1385
+ "loss": 0.18687117004394532,
1386
+ "step": 97000
1387
+ },
1388
+ {
1389
+ "epoch": 2.139941179052719,
1390
+ "grad_norm": 7.679690837860107,
1391
+ "learning_rate": 1.4334679484365626e-05,
1392
+ "loss": 0.16395710754394532,
1393
+ "step": 97500
1394
+ },
1395
+ {
1396
+ "epoch": 2.150915236381195,
1397
+ "grad_norm": 35.09670639038086,
1398
+ "learning_rate": 1.4151778528891035e-05,
1399
+ "loss": 0.17217919921875,
1400
+ "step": 98000
1401
+ },
1402
+ {
1403
+ "epoch": 2.1618892937096703,
1404
+ "grad_norm": 0.019701849669218063,
1405
+ "learning_rate": 1.3968877573416445e-05,
1406
+ "loss": 0.17984617614746093,
1407
+ "step": 98500
1408
+ },
1409
+ {
1410
+ "epoch": 2.1728633510381457,
1411
+ "grad_norm": 0.20285935699939728,
1412
+ "learning_rate": 1.3785976617941854e-05,
1413
+ "loss": 0.16325213623046875,
1414
+ "step": 99000
1415
+ },
1416
+ {
1417
+ "epoch": 2.1838374083666214,
1418
+ "grad_norm": 34.028297424316406,
1419
+ "learning_rate": 1.360307566246726e-05,
1420
+ "loss": 0.18122135925292968,
1421
+ "step": 99500
1422
+ },
1423
+ {
1424
+ "epoch": 2.1948114656950968,
1425
+ "grad_norm": 0.05509435012936592,
1426
+ "learning_rate": 1.342017470699267e-05,
1427
+ "loss": 0.17191763305664062,
1428
+ "step": 100000
1429
+ },
1430
+ {
1431
+ "epoch": 2.205785523023572,
1432
+ "grad_norm": 1.59047269821167,
1433
+ "learning_rate": 1.3237273751518079e-05,
1434
+ "loss": 0.15562428283691407,
1435
+ "step": 100500
1436
+ },
1437
+ {
1438
+ "epoch": 2.216759580352048,
1439
+ "grad_norm": 30.329383850097656,
1440
+ "learning_rate": 1.3054372796043488e-05,
1441
+ "loss": 0.20622531127929689,
1442
+ "step": 101000
1443
+ },
1444
+ {
1445
+ "epoch": 2.2277336376805232,
1446
+ "grad_norm": 0.020468488335609436,
1447
+ "learning_rate": 1.2871471840568896e-05,
1448
+ "loss": 0.16413253784179688,
1449
+ "step": 101500
1450
+ },
1451
+ {
1452
+ "epoch": 2.2387076950089986,
1453
+ "grad_norm": 0.00997017603367567,
1454
+ "learning_rate": 1.2688570885094304e-05,
1455
+ "loss": 0.1655951690673828,
1456
+ "step": 102000
1457
+ },
1458
+ {
1459
+ "epoch": 2.2496817523374744,
1460
+ "grad_norm": 0.1387864202260971,
1461
+ "learning_rate": 1.2505669929619711e-05,
1462
+ "loss": 0.16392440795898439,
1463
+ "step": 102500
1464
+ },
1465
+ {
1466
+ "epoch": 2.2606558096659497,
1467
+ "grad_norm": 0.03306190297007561,
1468
+ "learning_rate": 1.232276897414512e-05,
1469
+ "loss": 0.17298390197753907,
1470
+ "step": 103000
1471
+ },
1472
+ {
1473
+ "epoch": 2.271629866994425,
1474
+ "grad_norm": 0.010001047514379025,
1475
+ "learning_rate": 1.213986801867053e-05,
1476
+ "loss": 0.1894729461669922,
1477
+ "step": 103500
1478
+ },
1479
+ {
1480
+ "epoch": 2.282603924322901,
1481
+ "grad_norm": 81.1466064453125,
1482
+ "learning_rate": 1.1956967063195938e-05,
1483
+ "loss": 0.17070208740234374,
1484
+ "step": 104000
1485
+ },
1486
+ {
1487
+ "epoch": 2.293577981651376,
1488
+ "grad_norm": 0.02159142680466175,
1489
+ "learning_rate": 1.1774066107721347e-05,
1490
+ "loss": 0.18507080078125,
1491
+ "step": 104500
1492
+ },
1493
+ {
1494
+ "epoch": 2.3045520389798515,
1495
+ "grad_norm": 0.054674021899700165,
1496
+ "learning_rate": 1.1591165152246757e-05,
1497
+ "loss": 0.17819773864746094,
1498
+ "step": 105000
1499
+ },
1500
+ {
1501
+ "epoch": 2.3155260963083273,
1502
+ "grad_norm": 2.9571499824523926,
1503
+ "learning_rate": 1.1408264196772164e-05,
1504
+ "loss": 0.17032832336425782,
1505
+ "step": 105500
1506
+ },
1507
+ {
1508
+ "epoch": 2.3265001536368026,
1509
+ "grad_norm": 1.2494076490402222,
1510
+ "learning_rate": 1.1225363241297574e-05,
1511
+ "loss": 0.1580600128173828,
1512
+ "step": 106000
1513
+ },
1514
+ {
1515
+ "epoch": 2.337474210965278,
1516
+ "grad_norm": 0.3068596422672272,
1517
+ "learning_rate": 1.1042462285822981e-05,
1518
+ "loss": 0.16479891967773438,
1519
+ "step": 106500
1520
+ },
1521
+ {
1522
+ "epoch": 2.3484482682937537,
1523
+ "grad_norm": 38.06135559082031,
1524
+ "learning_rate": 1.0859561330348391e-05,
1525
+ "loss": 0.19860018920898437,
1526
+ "step": 107000
1527
+ },
1528
+ {
1529
+ "epoch": 2.359422325622229,
1530
+ "grad_norm": 16.392892837524414,
1531
+ "learning_rate": 1.06766603748738e-05,
1532
+ "loss": 0.17994320678710937,
1533
+ "step": 107500
1534
+ },
1535
+ {
1536
+ "epoch": 2.3703963829507044,
1537
+ "grad_norm": 1.3012549877166748,
1538
+ "learning_rate": 1.0493759419399208e-05,
1539
+ "loss": 0.16105982971191407,
1540
+ "step": 108000
1541
+ },
1542
+ {
1543
+ "epoch": 2.38137044027918,
1544
+ "grad_norm": 0.16427138447761536,
1545
+ "learning_rate": 1.0310858463924616e-05,
1546
+ "loss": 0.16437882995605468,
1547
+ "step": 108500
1548
+ },
1549
+ {
1550
+ "epoch": 2.3923444976076556,
1551
+ "grad_norm": 0.008658411912620068,
1552
+ "learning_rate": 1.0127957508450023e-05,
1553
+ "loss": 0.14569871520996094,
1554
+ "step": 109000
1555
+ },
1556
+ {
1557
+ "epoch": 2.403318554936131,
1558
+ "grad_norm": 0.1301782727241516,
1559
+ "learning_rate": 9.945056552975433e-06,
1560
+ "loss": 0.14984918212890624,
1561
+ "step": 109500
1562
+ },
1563
+ {
1564
+ "epoch": 2.4142926122646067,
1565
+ "grad_norm": 3.813662528991699,
1566
+ "learning_rate": 9.762155597500842e-06,
1567
+ "loss": 0.18121104431152343,
1568
+ "step": 110000
1569
+ },
1570
+ {
1571
+ "epoch": 2.425266669593082,
1572
+ "grad_norm": 2.6495282649993896,
1573
+ "learning_rate": 9.57925464202625e-06,
1574
+ "loss": 0.18369952392578126,
1575
+ "step": 110500
1576
+ },
1577
+ {
1578
+ "epoch": 2.4362407269215574,
1579
+ "grad_norm": 3.705036163330078,
1580
+ "learning_rate": 9.39635368655166e-06,
1581
+ "loss": 0.16695921325683594,
1582
+ "step": 111000
1583
+ },
1584
+ {
1585
+ "epoch": 2.4472147842500327,
1586
+ "grad_norm": 0.678033709526062,
1587
+ "learning_rate": 9.213452731077067e-06,
1588
+ "loss": 0.17362081909179689,
1589
+ "step": 111500
1590
+ },
1591
+ {
1592
+ "epoch": 2.4581888415785085,
1593
+ "grad_norm": 50.983375549316406,
1594
+ "learning_rate": 9.030551775602476e-06,
1595
+ "loss": 0.16382778930664063,
1596
+ "step": 112000
1597
+ },
1598
+ {
1599
+ "epoch": 2.469162898906984,
1600
+ "grad_norm": 29.069177627563477,
1601
+ "learning_rate": 8.847650820127886e-06,
1602
+ "loss": 0.16594575500488282,
1603
+ "step": 112500
1604
+ },
1605
+ {
1606
+ "epoch": 2.4801369562354596,
1607
+ "grad_norm": 13.098567962646484,
1608
+ "learning_rate": 8.664749864653293e-06,
1609
+ "loss": 0.1821525115966797,
1610
+ "step": 113000
1611
+ },
1612
+ {
1613
+ "epoch": 2.491111013563935,
1614
+ "grad_norm": 64.5162353515625,
1615
+ "learning_rate": 8.481848909178703e-06,
1616
+ "loss": 0.15829127502441406,
1617
+ "step": 113500
1618
+ },
1619
+ {
1620
+ "epoch": 2.5020850708924103,
1621
+ "grad_norm": 2.7032110691070557,
1622
+ "learning_rate": 8.29894795370411e-06,
1623
+ "loss": 0.1485829620361328,
1624
+ "step": 114000
1625
+ },
1626
+ {
1627
+ "epoch": 2.5130591282208856,
1628
+ "grad_norm": 40.1707878112793,
1629
+ "learning_rate": 8.11604699822952e-06,
1630
+ "loss": 0.1723223876953125,
1631
+ "step": 114500
1632
+ },
1633
+ {
1634
+ "epoch": 2.5240331855493614,
1635
+ "grad_norm": 0.07498432695865631,
1636
+ "learning_rate": 7.933146042754928e-06,
1637
+ "loss": 0.15050241088867186,
1638
+ "step": 115000
1639
+ },
1640
+ {
1641
+ "epoch": 2.5350072428778367,
1642
+ "grad_norm": 26.58846664428711,
1643
+ "learning_rate": 7.750245087280335e-06,
1644
+ "loss": 0.15875759887695312,
1645
+ "step": 115500
1646
+ },
1647
+ {
1648
+ "epoch": 2.5459813002063125,
1649
+ "grad_norm": 0.05308910831809044,
1650
+ "learning_rate": 7.567344131805746e-06,
1651
+ "loss": 0.1401026611328125,
1652
+ "step": 116000
1653
+ },
1654
+ {
1655
+ "epoch": 2.556955357534788,
1656
+ "grad_norm": 45.850929260253906,
1657
+ "learning_rate": 7.384443176331153e-06,
1658
+ "loss": 0.15721929931640624,
1659
+ "step": 116500
1660
+ },
1661
+ {
1662
+ "epoch": 2.567929414863263,
1663
+ "grad_norm": 45.02235412597656,
1664
+ "learning_rate": 7.201542220856562e-06,
1665
+ "loss": 0.19153817749023438,
1666
+ "step": 117000
1667
+ },
1668
+ {
1669
+ "epoch": 2.5789034721917385,
1670
+ "grad_norm": 0.04809782654047012,
1671
+ "learning_rate": 7.018641265381971e-06,
1672
+ "loss": 0.1555188446044922,
1673
+ "step": 117500
1674
+ },
1675
+ {
1676
+ "epoch": 2.5898775295202143,
1677
+ "grad_norm": 0.008281511254608631,
1678
+ "learning_rate": 6.835740309907379e-06,
1679
+ "loss": 0.17925529479980468,
1680
+ "step": 118000
1681
+ },
1682
+ {
1683
+ "epoch": 2.6008515868486897,
1684
+ "grad_norm": 23.324901580810547,
1685
+ "learning_rate": 6.652839354432788e-06,
1686
+ "loss": 0.167087158203125,
1687
+ "step": 118500
1688
+ },
1689
+ {
1690
+ "epoch": 2.6118256441771655,
1691
+ "grad_norm": 19.250812530517578,
1692
+ "learning_rate": 6.469938398958196e-06,
1693
+ "loss": 0.14051246643066406,
1694
+ "step": 119000
1695
+ },
1696
+ {
1697
+ "epoch": 2.622799701505641,
1698
+ "grad_norm": 0.014977659098803997,
1699
+ "learning_rate": 6.2870374434836055e-06,
1700
+ "loss": 0.1504051513671875,
1701
+ "step": 119500
1702
+ },
1703
+ {
1704
+ "epoch": 2.633773758834116,
1705
+ "grad_norm": 23.393903732299805,
1706
+ "learning_rate": 6.104136488009013e-06,
1707
+ "loss": 0.1683112335205078,
1708
+ "step": 120000
1709
+ },
1710
+ {
1711
+ "epoch": 2.6447478161625915,
1712
+ "grad_norm": 0.09205972403287888,
1713
+ "learning_rate": 5.921235532534422e-06,
1714
+ "loss": 0.15621336364746094,
1715
+ "step": 120500
1716
+ },
1717
+ {
1718
+ "epoch": 2.6557218734910673,
1719
+ "grad_norm": 0.5144221782684326,
1720
+ "learning_rate": 5.73833457705983e-06,
1721
+ "loss": 0.1647897491455078,
1722
+ "step": 121000
1723
+ },
1724
+ {
1725
+ "epoch": 2.6666959308195426,
1726
+ "grad_norm": 0.00860594492405653,
1727
+ "learning_rate": 5.55543362158524e-06,
1728
+ "loss": 0.15313204956054688,
1729
+ "step": 121500
1730
+ },
1731
+ {
1732
+ "epoch": 2.677669988148018,
1733
+ "grad_norm": 2.207170248031616,
1734
+ "learning_rate": 5.372532666110648e-06,
1735
+ "loss": 0.15816647338867187,
1736
+ "step": 122000
1737
+ },
1738
+ {
1739
+ "epoch": 2.6886440454764937,
1740
+ "grad_norm": 0.018872277811169624,
1741
+ "learning_rate": 5.189631710636057e-06,
1742
+ "loss": 0.1415150146484375,
1743
+ "step": 122500
1744
+ },
1745
+ {
1746
+ "epoch": 2.699618102804969,
1747
+ "grad_norm": 0.011648227460682392,
1748
+ "learning_rate": 5.006730755161465e-06,
1749
+ "loss": 0.13911293029785157,
1750
+ "step": 123000
1751
+ },
1752
+ {
1753
+ "epoch": 2.7105921601334444,
1754
+ "grad_norm": 0.09587373584508896,
1755
+ "learning_rate": 4.823829799686874e-06,
1756
+ "loss": 0.13494076538085936,
1757
+ "step": 123500
1758
+ },
1759
+ {
1760
+ "epoch": 2.72156621746192,
1761
+ "grad_norm": 0.04641059786081314,
1762
+ "learning_rate": 4.6409288442122825e-06,
1763
+ "loss": 0.17721914672851563,
1764
+ "step": 124000
1765
+ },
1766
+ {
1767
+ "epoch": 2.7325402747903955,
1768
+ "grad_norm": 0.011687349528074265,
1769
+ "learning_rate": 4.458027888737691e-06,
1770
+ "loss": 0.16873222351074219,
1771
+ "step": 124500
1772
+ },
1773
+ {
1774
+ "epoch": 2.743514332118871,
1775
+ "grad_norm": 118.11669158935547,
1776
+ "learning_rate": 4.2751269332630996e-06,
1777
+ "loss": 0.15355369567871094,
1778
+ "step": 125000
1779
+ },
1780
+ {
1781
+ "epoch": 2.7544883894473466,
1782
+ "grad_norm": 0.481996089220047,
1783
+ "learning_rate": 4.092225977788508e-06,
1784
+ "loss": 0.13936019897460938,
1785
+ "step": 125500
1786
+ },
1787
+ {
1788
+ "epoch": 2.765462446775822,
1789
+ "grad_norm": 0.19096533954143524,
1790
+ "learning_rate": 3.909325022313917e-06,
1791
+ "loss": 0.17287538146972656,
1792
+ "step": 126000
1793
+ },
1794
+ {
1795
+ "epoch": 2.7764365041042973,
1796
+ "grad_norm": 67.1457748413086,
1797
+ "learning_rate": 3.7264240668393257e-06,
1798
+ "loss": 0.15823133850097656,
1799
+ "step": 126500
1800
+ },
1801
+ {
1802
+ "epoch": 2.7874105614327727,
1803
+ "grad_norm": 15.519521713256836,
1804
+ "learning_rate": 3.543523111364734e-06,
1805
+ "loss": 0.17597552490234375,
1806
+ "step": 127000
1807
+ },
1808
+ {
1809
+ "epoch": 2.7983846187612484,
1810
+ "grad_norm": 0.021220775321125984,
1811
+ "learning_rate": 3.3606221558901423e-06,
1812
+ "loss": 0.15953814697265625,
1813
+ "step": 127500
1814
+ },
1815
+ {
1816
+ "epoch": 2.809358676089724,
1817
+ "grad_norm": 0.008825136348605156,
1818
+ "learning_rate": 3.177721200415551e-06,
1819
+ "loss": 0.16651971435546875,
1820
+ "step": 128000
1821
+ },
1822
+ {
1823
+ "epoch": 2.8203327334181996,
1824
+ "grad_norm": 0.20404738187789917,
1825
+ "learning_rate": 2.99482024494096e-06,
1826
+ "loss": 0.1628278350830078,
1827
+ "step": 128500
1828
+ },
1829
+ {
1830
+ "epoch": 2.831306790746675,
1831
+ "grad_norm": 0.009165110997855663,
1832
+ "learning_rate": 2.8119192894663684e-06,
1833
+ "loss": 0.1425828094482422,
1834
+ "step": 129000
1835
+ },
1836
+ {
1837
+ "epoch": 2.8422808480751502,
1838
+ "grad_norm": 0.008576123043894768,
1839
+ "learning_rate": 2.629018333991777e-06,
1840
+ "loss": 0.16400938415527344,
1841
+ "step": 129500
1842
+ },
1843
+ {
1844
+ "epoch": 2.8532549054036256,
1845
+ "grad_norm": 37.710914611816406,
1846
+ "learning_rate": 2.4461173785171855e-06,
1847
+ "loss": 0.15623898315429688,
1848
+ "step": 130000
1849
+ },
1850
+ {
1851
+ "epoch": 2.8642289627321014,
1852
+ "grad_norm": 11.700338363647461,
1853
+ "learning_rate": 2.263216423042594e-06,
1854
+ "loss": 0.1631771545410156,
1855
+ "step": 130500
1856
+ },
1857
+ {
1858
+ "epoch": 2.8752030200605767,
1859
+ "grad_norm": 1.506538987159729,
1860
+ "learning_rate": 2.0803154675680026e-06,
1861
+ "loss": 0.15615669250488282,
1862
+ "step": 131000
1863
+ },
1864
+ {
1865
+ "epoch": 2.8861770773890525,
1866
+ "grad_norm": 33.31697082519531,
1867
+ "learning_rate": 1.8974145120934112e-06,
1868
+ "loss": 0.1537679138183594,
1869
+ "step": 131500
1870
+ },
1871
+ {
1872
+ "epoch": 2.897151134717528,
1873
+ "grad_norm": 38.086795806884766,
1874
+ "learning_rate": 1.71451355661882e-06,
1875
+ "loss": 0.16492466735839845,
1876
+ "step": 132000
1877
+ },
1878
+ {
1879
+ "epoch": 2.908125192046003,
1880
+ "grad_norm": 0.02016337588429451,
1881
+ "learning_rate": 1.5316126011442285e-06,
1882
+ "loss": 0.15764328002929687,
1883
+ "step": 132500
1884
+ },
1885
+ {
1886
+ "epoch": 2.9190992493744785,
1887
+ "grad_norm": 0.06843503564596176,
1888
+ "learning_rate": 1.348711645669637e-06,
1889
+ "loss": 0.14805685424804688,
1890
+ "step": 133000
1891
+ },
1892
+ {
1893
+ "epoch": 2.9300733067029543,
1894
+ "grad_norm": 0.006283937953412533,
1895
+ "learning_rate": 1.1658106901950456e-06,
1896
+ "loss": 0.1535034637451172,
1897
+ "step": 133500
1898
+ },
1899
+ {
1900
+ "epoch": 2.9410473640314296,
1901
+ "grad_norm": 0.0035067277494817972,
1902
+ "learning_rate": 9.829097347204541e-07,
1903
+ "loss": 0.15516947937011719,
1904
+ "step": 134000
1905
+ },
1906
+ {
1907
+ "epoch": 2.9520214213599054,
1908
+ "grad_norm": 0.023081157356500626,
1909
+ "learning_rate": 8.000087792458628e-07,
1910
+ "loss": 0.15423509216308592,
1911
+ "step": 134500
1912
+ },
1913
+ {
1914
+ "epoch": 2.9629954786883808,
1915
+ "grad_norm": 13.908537864685059,
1916
+ "learning_rate": 6.171078237712715e-07,
1917
+ "loss": 0.14795904541015625,
1918
+ "step": 135000
1919
+ },
1920
+ {
1921
+ "epoch": 2.973969536016856,
1922
+ "grad_norm": 0.019963253289461136,
1923
+ "learning_rate": 4.3420686829668e-07,
1924
+ "loss": 0.15023118591308593,
1925
+ "step": 135500
1926
+ },
1927
+ {
1928
+ "epoch": 2.9849435933453314,
1929
+ "grad_norm": 20.809101104736328,
1930
+ "learning_rate": 2.513059128220886e-07,
1931
+ "loss": 0.1423495330810547,
1932
+ "step": 136000
1933
+ },
1934
+ {
1935
+ "epoch": 2.995917650673807,
1936
+ "grad_norm": 26.06735610961914,
1937
+ "learning_rate": 6.840495734749718e-08,
1938
+ "loss": 0.15201040649414063,
1939
+ "step": 136500
1940
+ },
1941
+ {
1942
+ "epoch": 3.0,
1943
+ "eval_f1": 0.8518024169356405,
1944
+ "eval_loss": 0.38094356656074524,
1945
+ "eval_runtime": 21.6505,
1946
+ "eval_samples_per_second": 1870.627,
1947
+ "eval_steps_per_second": 233.851,
1948
+ "step": 136686
1949
+ },
1950
+ {
1951
+ "epoch": 3.0,
1952
+ "step": 136686,
1953
+ "total_flos": 1.801472479969075e+16,
1954
+ "train_loss": 0.3033465779127858,
1955
+ "train_runtime": 2259.7352,
1956
+ "train_samples_per_second": 483.897,
1957
+ "train_steps_per_second": 60.488
1958
+ }
1959
+ ],
1960
+ "logging_steps": 500,
1961
+ "max_steps": 136686,
1962
+ "num_input_tokens_seen": 0,
1963
+ "num_train_epochs": 3,
1964
+ "save_steps": 500,
1965
+ "stateful_callbacks": {
1966
+ "TrainerControl": {
1967
+ "args": {
1968
+ "should_epoch_stop": false,
1969
+ "should_evaluate": false,
1970
+ "should_log": false,
1971
+ "should_save": true,
1972
+ "should_training_stop": true
1973
+ },
1974
+ "attributes": {}
1975
+ }
1976
+ },
1977
+ "total_flos": 1.801472479969075e+16,
1978
+ "train_batch_size": 8,
1979
+ "trial_name": null,
1980
+ "trial_params": null
1981
+ }