ZhangYiqian commited on
Commit
1a2db78
·
verified ·
1 Parent(s): bafe5b1

sft_v2_lora (10b7252) SFT v2 cold start, sftv2

Browse files
checkpoints/sft_v2_lora/epoch1/README.md ADDED
@@ -0,0 +1,207 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: /mnt/raid5/models/Qwen3.8-27B
3
+ library_name: peft
4
+ pipeline_tag: text-generation
5
+ tags:
6
+ - base_model:adapter:/mnt/raid5/models/Qwen3.8-27B
7
+ - lora
8
+ - transformers
9
+ ---
10
+
11
+ # Model Card for Model ID
12
+
13
+ <!-- Provide a quick summary of what the model is/does. -->
14
+
15
+
16
+
17
+ ## Model Details
18
+
19
+ ### Model Description
20
+
21
+ <!-- Provide a longer summary of what this model is. -->
22
+
23
+
24
+
25
+ - **Developed by:** [More Information Needed]
26
+ - **Funded by [optional]:** [More Information Needed]
27
+ - **Shared by [optional]:** [More Information Needed]
28
+ - **Model type:** [More Information Needed]
29
+ - **Language(s) (NLP):** [More Information Needed]
30
+ - **License:** [More Information Needed]
31
+ - **Finetuned from model [optional]:** [More Information Needed]
32
+
33
+ ### Model Sources [optional]
34
+
35
+ <!-- Provide the basic links for the model. -->
36
+
37
+ - **Repository:** [More Information Needed]
38
+ - **Paper [optional]:** [More Information Needed]
39
+ - **Demo [optional]:** [More Information Needed]
40
+
41
+ ## Uses
42
+
43
+ <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
44
+
45
+ ### Direct Use
46
+
47
+ <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
48
+
49
+ [More Information Needed]
50
+
51
+ ### Downstream Use [optional]
52
+
53
+ <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
54
+
55
+ [More Information Needed]
56
+
57
+ ### Out-of-Scope Use
58
+
59
+ <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
60
+
61
+ [More Information Needed]
62
+
63
+ ## Bias, Risks, and Limitations
64
+
65
+ <!-- This section is meant to convey both technical and sociotechnical limitations. -->
66
+
67
+ [More Information Needed]
68
+
69
+ ### Recommendations
70
+
71
+ <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
72
+
73
+ Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
74
+
75
+ ## How to Get Started with the Model
76
+
77
+ Use the code below to get started with the model.
78
+
79
+ [More Information Needed]
80
+
81
+ ## Training Details
82
+
83
+ ### Training Data
84
+
85
+ <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
86
+
87
+ [More Information Needed]
88
+
89
+ ### Training Procedure
90
+
91
+ <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
92
+
93
+ #### Preprocessing [optional]
94
+
95
+ [More Information Needed]
96
+
97
+
98
+ #### Training Hyperparameters
99
+
100
+ - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
101
+
102
+ #### Speeds, Sizes, Times [optional]
103
+
104
+ <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
105
+
106
+ [More Information Needed]
107
+
108
+ ## Evaluation
109
+
110
+ <!-- This section describes the evaluation protocols and provides the results. -->
111
+
112
+ ### Testing Data, Factors & Metrics
113
+
114
+ #### Testing Data
115
+
116
+ <!-- This should link to a Dataset Card if possible. -->
117
+
118
+ [More Information Needed]
119
+
120
+ #### Factors
121
+
122
+ <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
123
+
124
+ [More Information Needed]
125
+
126
+ #### Metrics
127
+
128
+ <!-- These are the evaluation metrics being used, ideally with a description of why. -->
129
+
130
+ [More Information Needed]
131
+
132
+ ### Results
133
+
134
+ [More Information Needed]
135
+
136
+ #### Summary
137
+
138
+
139
+
140
+ ## Model Examination [optional]
141
+
142
+ <!-- Relevant interpretability work for the model goes here -->
143
+
144
+ [More Information Needed]
145
+
146
+ ## Environmental Impact
147
+
148
+ <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
149
+
150
+ Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
151
+
152
+ - **Hardware Type:** [More Information Needed]
153
+ - **Hours used:** [More Information Needed]
154
+ - **Cloud Provider:** [More Information Needed]
155
+ - **Compute Region:** [More Information Needed]
156
+ - **Carbon Emitted:** [More Information Needed]
157
+
158
+ ## Technical Specifications [optional]
159
+
160
+ ### Model Architecture and Objective
161
+
162
+ [More Information Needed]
163
+
164
+ ### Compute Infrastructure
165
+
166
+ [More Information Needed]
167
+
168
+ #### Hardware
169
+
170
+ [More Information Needed]
171
+
172
+ #### Software
173
+
174
+ [More Information Needed]
175
+
176
+ ## Citation [optional]
177
+
178
+ <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
179
+
180
+ **BibTeX:**
181
+
182
+ [More Information Needed]
183
+
184
+ **APA:**
185
+
186
+ [More Information Needed]
187
+
188
+ ## Glossary [optional]
189
+
190
+ <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
191
+
192
+ [More Information Needed]
193
+
194
+ ## More Information [optional]
195
+
196
+ [More Information Needed]
197
+
198
+ ## Model Card Authors [optional]
199
+
200
+ [More Information Needed]
201
+
202
+ ## Model Card Contact
203
+
204
+ [More Information Needed]
205
+ ### Framework versions
206
+
207
+ - PEFT 0.21.2
checkpoints/sft_v2_lora/epoch1/adapter_config.json ADDED
@@ -0,0 +1,54 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alora_invocation_tokens": null,
3
+ "alpha_pattern": {},
4
+ "arrow_config": null,
5
+ "auto_mapping": null,
6
+ "base_model_name_or_path": "/mnt/raid5/models/Qwen3.8-27B",
7
+ "bias": "none",
8
+ "corda_config": null,
9
+ "ensure_weight_tying": false,
10
+ "eva_config": null,
11
+ "exclude_modules": null,
12
+ "fan_in_fan_out": false,
13
+ "inference_mode": true,
14
+ "init_lora_weights": true,
15
+ "kasa_config": null,
16
+ "layer_replication": null,
17
+ "layers_pattern": null,
18
+ "layers_to_transform": null,
19
+ "loftq_config": {},
20
+ "lora_alpha": 128,
21
+ "lora_bias": false,
22
+ "lora_dropout": 0.05,
23
+ "lora_ga_config": null,
24
+ "megatron_config": null,
25
+ "megatron_core": "megatron.core",
26
+ "modules_to_save": null,
27
+ "monteclora_config": null,
28
+ "peft_type": "LORA",
29
+ "peft_version": "0.21.2",
30
+ "qalora_group_size": 16,
31
+ "r": 64,
32
+ "rank_pattern": {},
33
+ "revision": null,
34
+ "target_modules": [
35
+ "down_proj",
36
+ "v_proj",
37
+ "up_proj",
38
+ "in_proj_z",
39
+ "o_proj",
40
+ "in_proj_qkv",
41
+ "out_proj",
42
+ "gate_proj",
43
+ "q_proj",
44
+ "k_proj"
45
+ ],
46
+ "target_parameters": null,
47
+ "task_type": "CAUSAL_LM",
48
+ "trainable_token_indices": null,
49
+ "use_bdlora": null,
50
+ "use_dora": false,
51
+ "use_qalora": false,
52
+ "use_rslora": false,
53
+ "velora_config": null
54
+ }
checkpoints/sft_v2_lora/epoch1/adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8a11ff164a79dfd19dde13f977ec46a97e04e680120ccc6ea3ac4652789fa16d
3
+ size 1740746640
checkpoints/sft_v2_lora/epoch2/README.md ADDED
@@ -0,0 +1,207 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: /mnt/raid5/models/Qwen3.8-27B
3
+ library_name: peft
4
+ pipeline_tag: text-generation
5
+ tags:
6
+ - base_model:adapter:/mnt/raid5/models/Qwen3.8-27B
7
+ - lora
8
+ - transformers
9
+ ---
10
+
11
+ # Model Card for Model ID
12
+
13
+ <!-- Provide a quick summary of what the model is/does. -->
14
+
15
+
16
+
17
+ ## Model Details
18
+
19
+ ### Model Description
20
+
21
+ <!-- Provide a longer summary of what this model is. -->
22
+
23
+
24
+
25
+ - **Developed by:** [More Information Needed]
26
+ - **Funded by [optional]:** [More Information Needed]
27
+ - **Shared by [optional]:** [More Information Needed]
28
+ - **Model type:** [More Information Needed]
29
+ - **Language(s) (NLP):** [More Information Needed]
30
+ - **License:** [More Information Needed]
31
+ - **Finetuned from model [optional]:** [More Information Needed]
32
+
33
+ ### Model Sources [optional]
34
+
35
+ <!-- Provide the basic links for the model. -->
36
+
37
+ - **Repository:** [More Information Needed]
38
+ - **Paper [optional]:** [More Information Needed]
39
+ - **Demo [optional]:** [More Information Needed]
40
+
41
+ ## Uses
42
+
43
+ <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
44
+
45
+ ### Direct Use
46
+
47
+ <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
48
+
49
+ [More Information Needed]
50
+
51
+ ### Downstream Use [optional]
52
+
53
+ <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
54
+
55
+ [More Information Needed]
56
+
57
+ ### Out-of-Scope Use
58
+
59
+ <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
60
+
61
+ [More Information Needed]
62
+
63
+ ## Bias, Risks, and Limitations
64
+
65
+ <!-- This section is meant to convey both technical and sociotechnical limitations. -->
66
+
67
+ [More Information Needed]
68
+
69
+ ### Recommendations
70
+
71
+ <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
72
+
73
+ Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
74
+
75
+ ## How to Get Started with the Model
76
+
77
+ Use the code below to get started with the model.
78
+
79
+ [More Information Needed]
80
+
81
+ ## Training Details
82
+
83
+ ### Training Data
84
+
85
+ <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
86
+
87
+ [More Information Needed]
88
+
89
+ ### Training Procedure
90
+
91
+ <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
92
+
93
+ #### Preprocessing [optional]
94
+
95
+ [More Information Needed]
96
+
97
+
98
+ #### Training Hyperparameters
99
+
100
+ - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
101
+
102
+ #### Speeds, Sizes, Times [optional]
103
+
104
+ <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
105
+
106
+ [More Information Needed]
107
+
108
+ ## Evaluation
109
+
110
+ <!-- This section describes the evaluation protocols and provides the results. -->
111
+
112
+ ### Testing Data, Factors & Metrics
113
+
114
+ #### Testing Data
115
+
116
+ <!-- This should link to a Dataset Card if possible. -->
117
+
118
+ [More Information Needed]
119
+
120
+ #### Factors
121
+
122
+ <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
123
+
124
+ [More Information Needed]
125
+
126
+ #### Metrics
127
+
128
+ <!-- These are the evaluation metrics being used, ideally with a description of why. -->
129
+
130
+ [More Information Needed]
131
+
132
+ ### Results
133
+
134
+ [More Information Needed]
135
+
136
+ #### Summary
137
+
138
+
139
+
140
+ ## Model Examination [optional]
141
+
142
+ <!-- Relevant interpretability work for the model goes here -->
143
+
144
+ [More Information Needed]
145
+
146
+ ## Environmental Impact
147
+
148
+ <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
149
+
150
+ Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
151
+
152
+ - **Hardware Type:** [More Information Needed]
153
+ - **Hours used:** [More Information Needed]
154
+ - **Cloud Provider:** [More Information Needed]
155
+ - **Compute Region:** [More Information Needed]
156
+ - **Carbon Emitted:** [More Information Needed]
157
+
158
+ ## Technical Specifications [optional]
159
+
160
+ ### Model Architecture and Objective
161
+
162
+ [More Information Needed]
163
+
164
+ ### Compute Infrastructure
165
+
166
+ [More Information Needed]
167
+
168
+ #### Hardware
169
+
170
+ [More Information Needed]
171
+
172
+ #### Software
173
+
174
+ [More Information Needed]
175
+
176
+ ## Citation [optional]
177
+
178
+ <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
179
+
180
+ **BibTeX:**
181
+
182
+ [More Information Needed]
183
+
184
+ **APA:**
185
+
186
+ [More Information Needed]
187
+
188
+ ## Glossary [optional]
189
+
190
+ <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
191
+
192
+ [More Information Needed]
193
+
194
+ ## More Information [optional]
195
+
196
+ [More Information Needed]
197
+
198
+ ## Model Card Authors [optional]
199
+
200
+ [More Information Needed]
201
+
202
+ ## Model Card Contact
203
+
204
+ [More Information Needed]
205
+ ### Framework versions
206
+
207
+ - PEFT 0.21.2
checkpoints/sft_v2_lora/epoch2/adapter_config.json ADDED
@@ -0,0 +1,54 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alora_invocation_tokens": null,
3
+ "alpha_pattern": {},
4
+ "arrow_config": null,
5
+ "auto_mapping": null,
6
+ "base_model_name_or_path": "/mnt/raid5/models/Qwen3.8-27B",
7
+ "bias": "none",
8
+ "corda_config": null,
9
+ "ensure_weight_tying": false,
10
+ "eva_config": null,
11
+ "exclude_modules": null,
12
+ "fan_in_fan_out": false,
13
+ "inference_mode": true,
14
+ "init_lora_weights": true,
15
+ "kasa_config": null,
16
+ "layer_replication": null,
17
+ "layers_pattern": null,
18
+ "layers_to_transform": null,
19
+ "loftq_config": {},
20
+ "lora_alpha": 128,
21
+ "lora_bias": false,
22
+ "lora_dropout": 0.05,
23
+ "lora_ga_config": null,
24
+ "megatron_config": null,
25
+ "megatron_core": "megatron.core",
26
+ "modules_to_save": null,
27
+ "monteclora_config": null,
28
+ "peft_type": "LORA",
29
+ "peft_version": "0.21.2",
30
+ "qalora_group_size": 16,
31
+ "r": 64,
32
+ "rank_pattern": {},
33
+ "revision": null,
34
+ "target_modules": [
35
+ "k_proj",
36
+ "up_proj",
37
+ "down_proj",
38
+ "o_proj",
39
+ "gate_proj",
40
+ "v_proj",
41
+ "in_proj_qkv",
42
+ "in_proj_z",
43
+ "out_proj",
44
+ "q_proj"
45
+ ],
46
+ "target_parameters": null,
47
+ "task_type": "CAUSAL_LM",
48
+ "trainable_token_indices": null,
49
+ "use_bdlora": null,
50
+ "use_dora": false,
51
+ "use_qalora": false,
52
+ "use_rslora": false,
53
+ "velora_config": null
54
+ }
checkpoints/sft_v2_lora/epoch2/adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:51e9d732fb4576818eba7f987b2f3295b684941781a587b47a2e2d789a3bb193
3
+ size 1740746640
checkpoints/sft_v2_lora/epoch3/README.md ADDED
@@ -0,0 +1,207 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: /mnt/raid5/models/Qwen3.8-27B
3
+ library_name: peft
4
+ pipeline_tag: text-generation
5
+ tags:
6
+ - base_model:adapter:/mnt/raid5/models/Qwen3.8-27B
7
+ - lora
8
+ - transformers
9
+ ---
10
+
11
+ # Model Card for Model ID
12
+
13
+ <!-- Provide a quick summary of what the model is/does. -->
14
+
15
+
16
+
17
+ ## Model Details
18
+
19
+ ### Model Description
20
+
21
+ <!-- Provide a longer summary of what this model is. -->
22
+
23
+
24
+
25
+ - **Developed by:** [More Information Needed]
26
+ - **Funded by [optional]:** [More Information Needed]
27
+ - **Shared by [optional]:** [More Information Needed]
28
+ - **Model type:** [More Information Needed]
29
+ - **Language(s) (NLP):** [More Information Needed]
30
+ - **License:** [More Information Needed]
31
+ - **Finetuned from model [optional]:** [More Information Needed]
32
+
33
+ ### Model Sources [optional]
34
+
35
+ <!-- Provide the basic links for the model. -->
36
+
37
+ - **Repository:** [More Information Needed]
38
+ - **Paper [optional]:** [More Information Needed]
39
+ - **Demo [optional]:** [More Information Needed]
40
+
41
+ ## Uses
42
+
43
+ <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
44
+
45
+ ### Direct Use
46
+
47
+ <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
48
+
49
+ [More Information Needed]
50
+
51
+ ### Downstream Use [optional]
52
+
53
+ <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
54
+
55
+ [More Information Needed]
56
+
57
+ ### Out-of-Scope Use
58
+
59
+ <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
60
+
61
+ [More Information Needed]
62
+
63
+ ## Bias, Risks, and Limitations
64
+
65
+ <!-- This section is meant to convey both technical and sociotechnical limitations. -->
66
+
67
+ [More Information Needed]
68
+
69
+ ### Recommendations
70
+
71
+ <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
72
+
73
+ Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
74
+
75
+ ## How to Get Started with the Model
76
+
77
+ Use the code below to get started with the model.
78
+
79
+ [More Information Needed]
80
+
81
+ ## Training Details
82
+
83
+ ### Training Data
84
+
85
+ <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
86
+
87
+ [More Information Needed]
88
+
89
+ ### Training Procedure
90
+
91
+ <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
92
+
93
+ #### Preprocessing [optional]
94
+
95
+ [More Information Needed]
96
+
97
+
98
+ #### Training Hyperparameters
99
+
100
+ - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
101
+
102
+ #### Speeds, Sizes, Times [optional]
103
+
104
+ <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
105
+
106
+ [More Information Needed]
107
+
108
+ ## Evaluation
109
+
110
+ <!-- This section describes the evaluation protocols and provides the results. -->
111
+
112
+ ### Testing Data, Factors & Metrics
113
+
114
+ #### Testing Data
115
+
116
+ <!-- This should link to a Dataset Card if possible. -->
117
+
118
+ [More Information Needed]
119
+
120
+ #### Factors
121
+
122
+ <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
123
+
124
+ [More Information Needed]
125
+
126
+ #### Metrics
127
+
128
+ <!-- These are the evaluation metrics being used, ideally with a description of why. -->
129
+
130
+ [More Information Needed]
131
+
132
+ ### Results
133
+
134
+ [More Information Needed]
135
+
136
+ #### Summary
137
+
138
+
139
+
140
+ ## Model Examination [optional]
141
+
142
+ <!-- Relevant interpretability work for the model goes here -->
143
+
144
+ [More Information Needed]
145
+
146
+ ## Environmental Impact
147
+
148
+ <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
149
+
150
+ Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
151
+
152
+ - **Hardware Type:** [More Information Needed]
153
+ - **Hours used:** [More Information Needed]
154
+ - **Cloud Provider:** [More Information Needed]
155
+ - **Compute Region:** [More Information Needed]
156
+ - **Carbon Emitted:** [More Information Needed]
157
+
158
+ ## Technical Specifications [optional]
159
+
160
+ ### Model Architecture and Objective
161
+
162
+ [More Information Needed]
163
+
164
+ ### Compute Infrastructure
165
+
166
+ [More Information Needed]
167
+
168
+ #### Hardware
169
+
170
+ [More Information Needed]
171
+
172
+ #### Software
173
+
174
+ [More Information Needed]
175
+
176
+ ## Citation [optional]
177
+
178
+ <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
179
+
180
+ **BibTeX:**
181
+
182
+ [More Information Needed]
183
+
184
+ **APA:**
185
+
186
+ [More Information Needed]
187
+
188
+ ## Glossary [optional]
189
+
190
+ <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
191
+
192
+ [More Information Needed]
193
+
194
+ ## More Information [optional]
195
+
196
+ [More Information Needed]
197
+
198
+ ## Model Card Authors [optional]
199
+
200
+ [More Information Needed]
201
+
202
+ ## Model Card Contact
203
+
204
+ [More Information Needed]
205
+ ### Framework versions
206
+
207
+ - PEFT 0.21.2
checkpoints/sft_v2_lora/epoch3/adapter_config.json ADDED
@@ -0,0 +1,54 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alora_invocation_tokens": null,
3
+ "alpha_pattern": {},
4
+ "arrow_config": null,
5
+ "auto_mapping": null,
6
+ "base_model_name_or_path": "/mnt/raid5/models/Qwen3.8-27B",
7
+ "bias": "none",
8
+ "corda_config": null,
9
+ "ensure_weight_tying": false,
10
+ "eva_config": null,
11
+ "exclude_modules": null,
12
+ "fan_in_fan_out": false,
13
+ "inference_mode": true,
14
+ "init_lora_weights": true,
15
+ "kasa_config": null,
16
+ "layer_replication": null,
17
+ "layers_pattern": null,
18
+ "layers_to_transform": null,
19
+ "loftq_config": {},
20
+ "lora_alpha": 128,
21
+ "lora_bias": false,
22
+ "lora_dropout": 0.05,
23
+ "lora_ga_config": null,
24
+ "megatron_config": null,
25
+ "megatron_core": "megatron.core",
26
+ "modules_to_save": null,
27
+ "monteclora_config": null,
28
+ "peft_type": "LORA",
29
+ "peft_version": "0.21.2",
30
+ "qalora_group_size": 16,
31
+ "r": 64,
32
+ "rank_pattern": {},
33
+ "revision": null,
34
+ "target_modules": [
35
+ "k_proj",
36
+ "up_proj",
37
+ "down_proj",
38
+ "o_proj",
39
+ "gate_proj",
40
+ "v_proj",
41
+ "in_proj_qkv",
42
+ "in_proj_z",
43
+ "out_proj",
44
+ "q_proj"
45
+ ],
46
+ "target_parameters": null,
47
+ "task_type": "CAUSAL_LM",
48
+ "trainable_token_indices": null,
49
+ "use_bdlora": null,
50
+ "use_dora": false,
51
+ "use_qalora": false,
52
+ "use_rslora": false,
53
+ "velora_config": null
54
+ }
checkpoints/sft_v2_lora/epoch3/adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:36f689d567f30f12943bb2edff416f916201886481a98224cc44fc60b4d18c87
3
+ size 1740746640
checkpoints/sft_v2_lora/final/README.md ADDED
@@ -0,0 +1,207 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: /mnt/raid5/models/Qwen3.8-27B
3
+ library_name: peft
4
+ pipeline_tag: text-generation
5
+ tags:
6
+ - base_model:adapter:/mnt/raid5/models/Qwen3.8-27B
7
+ - lora
8
+ - transformers
9
+ ---
10
+
11
+ # Model Card for Model ID
12
+
13
+ <!-- Provide a quick summary of what the model is/does. -->
14
+
15
+
16
+
17
+ ## Model Details
18
+
19
+ ### Model Description
20
+
21
+ <!-- Provide a longer summary of what this model is. -->
22
+
23
+
24
+
25
+ - **Developed by:** [More Information Needed]
26
+ - **Funded by [optional]:** [More Information Needed]
27
+ - **Shared by [optional]:** [More Information Needed]
28
+ - **Model type:** [More Information Needed]
29
+ - **Language(s) (NLP):** [More Information Needed]
30
+ - **License:** [More Information Needed]
31
+ - **Finetuned from model [optional]:** [More Information Needed]
32
+
33
+ ### Model Sources [optional]
34
+
35
+ <!-- Provide the basic links for the model. -->
36
+
37
+ - **Repository:** [More Information Needed]
38
+ - **Paper [optional]:** [More Information Needed]
39
+ - **Demo [optional]:** [More Information Needed]
40
+
41
+ ## Uses
42
+
43
+ <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
44
+
45
+ ### Direct Use
46
+
47
+ <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
48
+
49
+ [More Information Needed]
50
+
51
+ ### Downstream Use [optional]
52
+
53
+ <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
54
+
55
+ [More Information Needed]
56
+
57
+ ### Out-of-Scope Use
58
+
59
+ <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
60
+
61
+ [More Information Needed]
62
+
63
+ ## Bias, Risks, and Limitations
64
+
65
+ <!-- This section is meant to convey both technical and sociotechnical limitations. -->
66
+
67
+ [More Information Needed]
68
+
69
+ ### Recommendations
70
+
71
+ <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
72
+
73
+ Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
74
+
75
+ ## How to Get Started with the Model
76
+
77
+ Use the code below to get started with the model.
78
+
79
+ [More Information Needed]
80
+
81
+ ## Training Details
82
+
83
+ ### Training Data
84
+
85
+ <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
86
+
87
+ [More Information Needed]
88
+
89
+ ### Training Procedure
90
+
91
+ <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
92
+
93
+ #### Preprocessing [optional]
94
+
95
+ [More Information Needed]
96
+
97
+
98
+ #### Training Hyperparameters
99
+
100
+ - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
101
+
102
+ #### Speeds, Sizes, Times [optional]
103
+
104
+ <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
105
+
106
+ [More Information Needed]
107
+
108
+ ## Evaluation
109
+
110
+ <!-- This section describes the evaluation protocols and provides the results. -->
111
+
112
+ ### Testing Data, Factors & Metrics
113
+
114
+ #### Testing Data
115
+
116
+ <!-- This should link to a Dataset Card if possible. -->
117
+
118
+ [More Information Needed]
119
+
120
+ #### Factors
121
+
122
+ <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
123
+
124
+ [More Information Needed]
125
+
126
+ #### Metrics
127
+
128
+ <!-- These are the evaluation metrics being used, ideally with a description of why. -->
129
+
130
+ [More Information Needed]
131
+
132
+ ### Results
133
+
134
+ [More Information Needed]
135
+
136
+ #### Summary
137
+
138
+
139
+
140
+ ## Model Examination [optional]
141
+
142
+ <!-- Relevant interpretability work for the model goes here -->
143
+
144
+ [More Information Needed]
145
+
146
+ ## Environmental Impact
147
+
148
+ <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
149
+
150
+ Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
151
+
152
+ - **Hardware Type:** [More Information Needed]
153
+ - **Hours used:** [More Information Needed]
154
+ - **Cloud Provider:** [More Information Needed]
155
+ - **Compute Region:** [More Information Needed]
156
+ - **Carbon Emitted:** [More Information Needed]
157
+
158
+ ## Technical Specifications [optional]
159
+
160
+ ### Model Architecture and Objective
161
+
162
+ [More Information Needed]
163
+
164
+ ### Compute Infrastructure
165
+
166
+ [More Information Needed]
167
+
168
+ #### Hardware
169
+
170
+ [More Information Needed]
171
+
172
+ #### Software
173
+
174
+ [More Information Needed]
175
+
176
+ ## Citation [optional]
177
+
178
+ <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
179
+
180
+ **BibTeX:**
181
+
182
+ [More Information Needed]
183
+
184
+ **APA:**
185
+
186
+ [More Information Needed]
187
+
188
+ ## Glossary [optional]
189
+
190
+ <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
191
+
192
+ [More Information Needed]
193
+
194
+ ## More Information [optional]
195
+
196
+ [More Information Needed]
197
+
198
+ ## Model Card Authors [optional]
199
+
200
+ [More Information Needed]
201
+
202
+ ## Model Card Contact
203
+
204
+ [More Information Needed]
205
+ ### Framework versions
206
+
207
+ - PEFT 0.21.2
checkpoints/sft_v2_lora/final/adapter_config.json ADDED
@@ -0,0 +1,54 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alora_invocation_tokens": null,
3
+ "alpha_pattern": {},
4
+ "arrow_config": null,
5
+ "auto_mapping": null,
6
+ "base_model_name_or_path": "/mnt/raid5/models/Qwen3.8-27B",
7
+ "bias": "none",
8
+ "corda_config": null,
9
+ "ensure_weight_tying": false,
10
+ "eva_config": null,
11
+ "exclude_modules": null,
12
+ "fan_in_fan_out": false,
13
+ "inference_mode": true,
14
+ "init_lora_weights": true,
15
+ "kasa_config": null,
16
+ "layer_replication": null,
17
+ "layers_pattern": null,
18
+ "layers_to_transform": null,
19
+ "loftq_config": {},
20
+ "lora_alpha": 128,
21
+ "lora_bias": false,
22
+ "lora_dropout": 0.05,
23
+ "lora_ga_config": null,
24
+ "megatron_config": null,
25
+ "megatron_core": "megatron.core",
26
+ "modules_to_save": null,
27
+ "monteclora_config": null,
28
+ "peft_type": "LORA",
29
+ "peft_version": "0.21.2",
30
+ "qalora_group_size": 16,
31
+ "r": 64,
32
+ "rank_pattern": {},
33
+ "revision": null,
34
+ "target_modules": [
35
+ "k_proj",
36
+ "up_proj",
37
+ "down_proj",
38
+ "o_proj",
39
+ "gate_proj",
40
+ "v_proj",
41
+ "in_proj_qkv",
42
+ "in_proj_z",
43
+ "out_proj",
44
+ "q_proj"
45
+ ],
46
+ "target_parameters": null,
47
+ "task_type": "CAUSAL_LM",
48
+ "trainable_token_indices": null,
49
+ "use_bdlora": null,
50
+ "use_dora": false,
51
+ "use_qalora": false,
52
+ "use_rslora": false,
53
+ "velora_config": null
54
+ }
checkpoints/sft_v2_lora/final/adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:36f689d567f30f12943bb2edff416f916201886481a98224cc44fc60b4d18c87
3
+ size 1740746640
checkpoints/sft_v2_lora/manifest.json ADDED
@@ -0,0 +1,61 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "name": "sft_v2_lora",
3
+ "uploaded_at": "2026-10-15T13:26:29+0800",
4
+ "git_commit": "10b7252",
5
+ "source_dir": "/mnt/raid5/workspace/Agent_Coding_Active/ckpts/sft_v2_lora",
6
+ "kind": "model",
7
+ "base_model": "/mnt/raid5/models/Qwen3.8-27B",
8
+ "train_args": {
9
+ "model": "/mnt/raid5/models/Qwen3.8-27B",
10
+ "data": "sft_pipeline/out/icae_sft_v2.tokens.jsonl",
11
+ "out": "/mnt/raid5/workspace/Agent_Coding_Active/ckpts/sft_v2_lora",
12
+ "device": "cuda",
13
+ "attn": "sdpa",
14
+ "max_len": 32768,
15
+ "limit": 0,
16
+ "kinds": "",
17
+ "balance": "ask/feature=4,ask/protocol=2,explore=1,proceed=1",
18
+ "exclude_runs": "dev01",
19
+ "save_each_epoch": true,
20
+ "resume_adapter": "/mnt/raid5/workspace/Agent_Coding_Active/ckpts/sft_v2_lora/epoch1",
21
+ "resume_step": 56,
22
+ "loss_chunk": 2048,
23
+ "epochs": 3.0,
24
+ "max_steps": 0,
25
+ "lr": 0.0001,
26
+ "warmup": 0.05,
27
+ "grad_accum": 4,
28
+ "lora_r": 64,
29
+ "lora_alpha": 128,
30
+ "lora_dropout": 0.05,
31
+ "grad_ckpt": true,
32
+ "save_every": 0,
33
+ "seed": 0,
34
+ "log_every": 1,
35
+ "dry_truncate": false,
36
+ "dry_vocab_clamp": 0,
37
+ "data_stats": {
38
+ "rows": 223,
39
+ "unique_rows": 169,
40
+ "kinds": {
41
+ "proceed": 61,
42
+ "explore": 76,
43
+ "ask": 86
44
+ },
45
+ "groups": {
46
+ "proceed": 61,
47
+ "explore": 76,
48
+ "ask/feature": 44,
49
+ "ask/protocol": 42
50
+ },
51
+ "tokens_total": 3784330,
52
+ "tokens_max": 32758,
53
+ "labelled_total": 369828,
54
+ "labelled_mean": 1658
55
+ },
56
+ "total_steps": 168
57
+ },
58
+ "n_files": 15,
59
+ "total_bytes": 6963035127,
60
+ "note": "SFT v2 cold start, sftv2"
61
+ }
checkpoints/sft_v2_lora/train_args.json ADDED
@@ -0,0 +1,50 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "model": "/mnt/raid5/models/Qwen3.8-27B",
3
+ "data": "sft_pipeline/out/icae_sft_v2.tokens.jsonl",
4
+ "out": "/mnt/raid5/workspace/Agent_Coding_Active/ckpts/sft_v2_lora",
5
+ "device": "cuda",
6
+ "attn": "sdpa",
7
+ "max_len": 32768,
8
+ "limit": 0,
9
+ "kinds": "",
10
+ "balance": "ask/feature=4,ask/protocol=2,explore=1,proceed=1",
11
+ "exclude_runs": "dev01",
12
+ "save_each_epoch": true,
13
+ "resume_adapter": "/mnt/raid5/workspace/Agent_Coding_Active/ckpts/sft_v2_lora/epoch1",
14
+ "resume_step": 56,
15
+ "loss_chunk": 2048,
16
+ "epochs": 3.0,
17
+ "max_steps": 0,
18
+ "lr": 0.0001,
19
+ "warmup": 0.05,
20
+ "grad_accum": 4,
21
+ "lora_r": 64,
22
+ "lora_alpha": 128,
23
+ "lora_dropout": 0.05,
24
+ "grad_ckpt": true,
25
+ "save_every": 0,
26
+ "seed": 0,
27
+ "log_every": 1,
28
+ "dry_truncate": false,
29
+ "dry_vocab_clamp": 0,
30
+ "data_stats": {
31
+ "rows": 223,
32
+ "unique_rows": 169,
33
+ "kinds": {
34
+ "proceed": 61,
35
+ "explore": 76,
36
+ "ask": 86
37
+ },
38
+ "groups": {
39
+ "proceed": 61,
40
+ "explore": 76,
41
+ "ask/feature": 44,
42
+ "ask/protocol": 42
43
+ },
44
+ "tokens_total": 3784330,
45
+ "tokens_max": 32758,
46
+ "labelled_total": 369828,
47
+ "labelled_mean": 1658
48
+ },
49
+ "total_steps": 168
50
+ }
checkpoints/sft_v2_lora/train_log.jsonl ADDED
@@ -0,0 +1,198 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {"step": 1, "loss": 0.3537, "labelled_tokens": 662, "lr": 2.5e-05, "elapsed_s": 130}
2
+ {"step": 2, "loss": 0.1313, "labelled_tokens": 4277, "lr": 3.7500000000000003e-05, "elapsed_s": 228}
3
+ {"step": 3, "loss": 0.2081, "labelled_tokens": 5818, "lr": 5e-05, "elapsed_s": 381}
4
+ {"step": 4, "loss": 0.3613, "labelled_tokens": 3689, "lr": 6.25e-05, "elapsed_s": 483}
5
+ {"step": 5, "loss": 0.1559, "labelled_tokens": 3170, "lr": 7.500000000000001e-05, "elapsed_s": 592}
6
+ {"step": 6, "loss": 0.2243, "labelled_tokens": 3380, "lr": 8.75e-05, "elapsed_s": 724}
7
+ {"step": 7, "loss": 0.1813, "labelled_tokens": 7301, "lr": 0.0001, "elapsed_s": 875}
8
+ {"step": 8, "loss": 0.2604, "labelled_tokens": 2002, "lr": 0.0001, "elapsed_s": 967}
9
+ {"step": 9, "loss": 0.4132, "labelled_tokens": 3834, "lr": 9.999036202410325e-05, "elapsed_s": 1071}
10
+ {"step": 10, "loss": 0.134, "labelled_tokens": 4367, "lr": 9.996145181203615e-05, "elapsed_s": 1193}
11
+ {"step": 11, "loss": 0.4138, "labelled_tokens": 3786, "lr": 9.991328050923581e-05, "elapsed_s": 1273}
12
+ {"step": 12, "loss": 0.2207, "labelled_tokens": 12809, "lr": 9.98458666866564e-05, "elapsed_s": 1409}
13
+ {"step": 13, "loss": 0.2697, "labelled_tokens": 11382, "lr": 9.975923633360985e-05, "elapsed_s": 1519}
14
+ {"step": 14, "loss": 0.1923, "labelled_tokens": 11077, "lr": 9.965342284774632e-05, "elapsed_s": 1663}
15
+ {"step": 15, "loss": 0.1117, "labelled_tokens": 5018, "lr": 9.952846702217886e-05, "elapsed_s": 1829}
16
+ {"step": 16, "loss": 0.2935, "labelled_tokens": 5841, "lr": 9.938441702975689e-05, "elapsed_s": 1968}
17
+ {"step": 17, "loss": 0.1754, "labelled_tokens": 10515, "lr": 9.922132840449459e-05, "elapsed_s": 2075}
18
+ {"step": 18, "loss": 0.1738, "labelled_tokens": 3469, "lr": 9.903926402016153e-05, "elapsed_s": 2173}
19
+ {"step": 19, "loss": 0.3035, "labelled_tokens": 4637, "lr": 9.883829406604363e-05, "elapsed_s": 2324}
20
+ {"step": 20, "loss": 0.3, "labelled_tokens": 17099, "lr": 9.861849601988383e-05, "elapsed_s": 2456}
21
+ {"step": 21, "loss": 0.1433, "labelled_tokens": 5045, "lr": 9.837995461801299e-05, "elapsed_s": 2589}
22
+ {"step": 22, "loss": 0.2908, "labelled_tokens": 1819, "lr": 9.812276182268236e-05, "elapsed_s": 2643}
23
+ {"step": 23, "loss": 0.1991, "labelled_tokens": 10107, "lr": 9.784701678661045e-05, "elapsed_s": 2799}
24
+ {"step": 24, "loss": 0.2258, "labelled_tokens": 4486, "lr": 9.755282581475769e-05, "elapsed_s": 2914}
25
+ {"step": 25, "loss": 0.4163, "labelled_tokens": 8947, "lr": 9.724030232334391e-05, "elapsed_s": 2980}
26
+ {"step": 26, "loss": 0.3024, "labelled_tokens": 12390, "lr": 9.690956679612421e-05, "elapsed_s": 3071}
27
+ {"step": 27, "loss": 0.2275, "labelled_tokens": 10197, "lr": 9.656074673794018e-05, "elapsed_s": 3173}
28
+ {"step": 28, "loss": 0.2581, "labelled_tokens": 4058, "lr": 9.619397662556435e-05, "elapsed_s": 3224}
29
+ {"step": 29, "loss": 0.1192, "labelled_tokens": 3382, "lr": 9.580939785585681e-05, "elapsed_s": 3274}
30
+ {"step": 30, "loss": 0.2257, "labelled_tokens": 4578, "lr": 9.540715869125407e-05, "elapsed_s": 3395}
31
+ {"step": 31, "loss": 0.1437, "labelled_tokens": 3699, "lr": 9.498741420261108e-05, "elapsed_s": 3528}
32
+ {"step": 32, "loss": 0.1646, "labelled_tokens": 12666, "lr": 9.45503262094184e-05, "elapsed_s": 3613}
33
+ {"step": 33, "loss": 0.2221, "labelled_tokens": 5447, "lr": 9.409606321741775e-05, "elapsed_s": 3721}
34
+ {"step": 34, "loss": 0.331, "labelled_tokens": 5707, "lr": 9.362480035363986e-05, "elapsed_s": 3838}
35
+ {"step": 35, "loss": 0.1762, "labelled_tokens": 9906, "lr": 9.31367192988896e-05, "elapsed_s": 3969}
36
+ {"step": 36, "loss": 0.1955, "labelled_tokens": 20194, "lr": 9.263200821770461e-05, "elapsed_s": 4127}
37
+ {"step": 37, "loss": 0.1505, "labelled_tokens": 3882, "lr": 9.211086168581433e-05, "elapsed_s": 4234}
38
+ {"step": 38, "loss": 0.1396, "labelled_tokens": 9287, "lr": 9.157348061512727e-05, "elapsed_s": 4358}
39
+ {"step": 39, "loss": 0.2546, "labelled_tokens": 1895, "lr": 9.102007217627568e-05, "elapsed_s": 4452}
40
+ {"step": 40, "loss": 0.154, "labelled_tokens": 3250, "lr": 9.045084971874738e-05, "elapsed_s": 4553}
41
+ {"step": 41, "loss": 0.1974, "labelled_tokens": 5524, "lr": 8.986603268863536e-05, "elapsed_s": 4647}
42
+ {"step": 42, "loss": 0.1287, "labelled_tokens": 2317, "lr": 8.926584654403724e-05, "elapsed_s": 4766}
43
+ {"step": 43, "loss": 0.2405, "labelled_tokens": 8645, "lr": 8.865052266813685e-05, "elapsed_s": 4924}
44
+ {"step": 44, "loss": 0.285, "labelled_tokens": 3442, "lr": 8.802029828000156e-05, "elapsed_s": 5045}
45
+ {"step": 45, "loss": 0.109, "labelled_tokens": 4948, "lr": 8.737541634312985e-05, "elapsed_s": 5173}
46
+ {"step": 46, "loss": 0.3477, "labelled_tokens": 5998, "lr": 8.671612547178428e-05, "elapsed_s": 5228}
47
+ {"step": 47, "loss": 0.2029, "labelled_tokens": 12830, "lr": 8.604267983514594e-05, "elapsed_s": 5358}
48
+ {"step": 48, "loss": 0.1458, "labelled_tokens": 10145, "lr": 8.535533905932738e-05, "elapsed_s": 5492}
49
+ {"step": 49, "loss": 0.2814, "labelled_tokens": 5748, "lr": 8.46543681272818e-05, "elapsed_s": 5615}
50
+ {"step": 50, "loss": 0.1019, "labelled_tokens": 1895, "lr": 8.39400372766471e-05, "elapsed_s": 5707}
51
+ {"step": 51, "loss": 0.1371, "labelled_tokens": 5914, "lr": 8.321262189556409e-05, "elapsed_s": 5759}
52
+ {"step": 52, "loss": 0.2487, "labelled_tokens": 5099, "lr": 8.247240241650918e-05, "elapsed_s": 5853}
53
+ {"step": 53, "loss": 0.3506, "labelled_tokens": 5943, "lr": 8.171966420818228e-05, "elapsed_s": 5998}
54
+ {"step": 54, "loss": 0.0871, "labelled_tokens": 6939, "lr": 8.095469746549172e-05, "elapsed_s": 6123}
55
+ {"step": 55, "loss": 0.1554, "labelled_tokens": 2497, "lr": 8.017779709767858e-05, "elapsed_s": 6237}
56
+ {"step": 56, "loss": 0.2093, "labelled_tokens": 17441, "lr": 7.938926261462366e-05, "elapsed_s": 6349}
57
+ {"step": 57, "loss": 0.1582, "labelled_tokens": 17116, "lr": 7.858939801138061e-05, "elapsed_s": 6554}
58
+ {"step": 58, "loss": 0.1455, "labelled_tokens": 15993, "lr": 7.777851165098012e-05, "elapsed_s": 6728}
59
+ {"step": 59, "loss": 0.0639, "labelled_tokens": 1807, "lr": 7.695691614555003e-05, "elapsed_s": 6779}
60
+ {"step": 60, "loss": 0.0613, "labelled_tokens": 6795, "lr": 7.612492823579745e-05, "elapsed_s": 6898}
61
+ {"step": 61, "loss": 0.0888, "labelled_tokens": 3969, "lr": 7.528286866889924e-05, "elapsed_s": 7019}
62
+ {"step": 62, "loss": 0.0939, "labelled_tokens": 5884, "lr": 7.443106207484776e-05, "elapsed_s": 7120}
63
+ {"step": 63, "loss": 0.3145, "labelled_tokens": 4515, "lr": 7.35698368412999e-05, "elapsed_s": 7233}
64
+ {"step": 64, "loss": 0.1151, "labelled_tokens": 9396, "lr": 7.269952498697734e-05, "elapsed_s": 7363}
65
+ {"step": 65, "loss": 0.0589, "labelled_tokens": 3266, "lr": 7.18204620336671e-05, "elapsed_s": 7458}
66
+ {"step": 66, "loss": 0.0775, "labelled_tokens": 9933, "lr": 7.09329868768714e-05, "elapsed_s": 7572}
67
+ {"step": 67, "loss": 0.1885, "labelled_tokens": 7753, "lr": 7.003744165515705e-05, "elapsed_s": 7710}
68
+ {"step": 68, "loss": 0.1138, "labelled_tokens": 11905, "lr": 6.91341716182545e-05, "elapsed_s": 7821}
69
+ {"step": 69, "loss": 0.0709, "labelled_tokens": 4079, "lr": 6.82235249939575e-05, "elapsed_s": 7964}
70
+ {"step": 70, "loss": 0.0324, "labelled_tokens": 3948, "lr": 6.730585285387465e-05, "elapsed_s": 8073}
71
+ {"step": 71, "loss": 0.3224, "labelled_tokens": 3478, "lr": 6.638150897808468e-05, "elapsed_s": 8130}
72
+ {"step": 72, "loss": 0.1066, "labelled_tokens": 3841, "lr": 6.545084971874738e-05, "elapsed_s": 8220}
73
+ {"step": 73, "loss": 0.15, "labelled_tokens": 2368, "lr": 6.451423386272312e-05, "elapsed_s": 8280}
74
+ {"step": 74, "loss": 0.2361, "labelled_tokens": 6098, "lr": 6.357202249325371e-05, "elapsed_s": 8362}
75
+ {"step": 75, "loss": 0.1019, "labelled_tokens": 1815, "lr": 6.26245788507579e-05, "elapsed_s": 8461}
76
+ {"step": 76, "loss": 0.0635, "labelled_tokens": 4825, "lr": 6.167226819279528e-05, "elapsed_s": 8598}
77
+ {"step": 77, "loss": 0.246, "labelled_tokens": 2626, "lr": 6.071545765325254e-05, "elapsed_s": 8691}
78
+ {"step": 78, "loss": 0.1806, "labelled_tokens": 5295, "lr": 5.9754516100806423e-05, "elapsed_s": 8815}
79
+ {"step": 79, "loss": 0.1466, "labelled_tokens": 2345, "lr": 5.8789813996717736e-05, "elapsed_s": 8968}
80
+ {"step": 80, "loss": 0.1591, "labelled_tokens": 6435, "lr": 5.782172325201155e-05, "elapsed_s": 9086}
81
+ {"step": 81, "loss": 0.1119, "labelled_tokens": 3303, "lr": 5.685061708409841e-05, "elapsed_s": 9219}
82
+ {"step": 82, "loss": 0.0736, "labelled_tokens": 2601, "lr": 5.587686987289189e-05, "elapsed_s": 9345}
83
+ {"step": 83, "loss": 0.1557, "labelled_tokens": 10207, "lr": 5.490085701647805e-05, "elapsed_s": 9501}
84
+ {"step": 84, "loss": 0.1322, "labelled_tokens": 3773, "lr": 5.392295478639225e-05, "elapsed_s": 9619}
85
+ {"step": 85, "loss": 0.1093, "labelled_tokens": 6183, "lr": 5.294354018255945e-05, "elapsed_s": 9743}
86
+ {"step": 86, "loss": 0.1643, "labelled_tokens": 11833, "lr": 5.196299078795344e-05, "elapsed_s": 9911}
87
+ {"step": 57, "loss": 0.2157, "labelled_tokens": 2096, "lr": 7.858939801138061e-05, "elapsed_s": 86}
88
+ {"step": 58, "loss": 0.0523, "labelled_tokens": 6184, "lr": 7.777851165098012e-05, "elapsed_s": 222}
89
+ {"step": 59, "loss": 0.1969, "labelled_tokens": 4929, "lr": 7.695691614555003e-05, "elapsed_s": 332}
90
+ {"step": 60, "loss": 0.0624, "labelled_tokens": 1171, "lr": 7.612492823579745e-05, "elapsed_s": 421}
91
+ {"step": 61, "loss": 0.3319, "labelled_tokens": 15737, "lr": 7.528286866889924e-05, "elapsed_s": 544}
92
+ {"step": 62, "loss": 0.0877, "labelled_tokens": 6357, "lr": 7.443106207484776e-05, "elapsed_s": 657}
93
+ {"step": 63, "loss": 0.3262, "labelled_tokens": 6295, "lr": 7.35698368412999e-05, "elapsed_s": 700}
94
+ {"step": 64, "loss": 0.3606, "labelled_tokens": 4984, "lr": 7.269952498697734e-05, "elapsed_s": 823}
95
+ {"step": 65, "loss": 0.1136, "labelled_tokens": 3859, "lr": 7.18204620336671e-05, "elapsed_s": 934}
96
+ {"step": 66, "loss": 0.1446, "labelled_tokens": 6344, "lr": 7.09329868768714e-05, "elapsed_s": 1047}
97
+ {"step": 67, "loss": 0.1303, "labelled_tokens": 9953, "lr": 7.003744165515705e-05, "elapsed_s": 1194}
98
+ {"step": 68, "loss": 0.1574, "labelled_tokens": 10332, "lr": 6.91341716182545e-05, "elapsed_s": 1325}
99
+ {"step": 69, "loss": 0.1224, "labelled_tokens": 5674, "lr": 6.82235249939575e-05, "elapsed_s": 1402}
100
+ {"step": 70, "loss": 0.3702, "labelled_tokens": 3964, "lr": 6.730585285387465e-05, "elapsed_s": 1438}
101
+ {"step": 71, "loss": 0.1432, "labelled_tokens": 3216, "lr": 6.638150897808468e-05, "elapsed_s": 1496}
102
+ {"step": 72, "loss": 0.2553, "labelled_tokens": 3943, "lr": 6.545084971874738e-05, "elapsed_s": 1561}
103
+ {"step": 73, "loss": 0.3204, "labelled_tokens": 7908, "lr": 6.451423386272312e-05, "elapsed_s": 1641}
104
+ {"step": 74, "loss": 0.1379, "labelled_tokens": 2347, "lr": 6.357202249325371e-05, "elapsed_s": 1698}
105
+ {"step": 75, "loss": 0.0505, "labelled_tokens": 1871, "lr": 6.26245788507579e-05, "elapsed_s": 1815}
106
+ {"step": 76, "loss": 0.1003, "labelled_tokens": 5431, "lr": 6.167226819279528e-05, "elapsed_s": 1958}
107
+ {"step": 77, "loss": 0.1285, "labelled_tokens": 11729, "lr": 6.071545765325254e-05, "elapsed_s": 2103}
108
+ {"step": 78, "loss": 0.2299, "labelled_tokens": 7974, "lr": 5.9754516100806423e-05, "elapsed_s": 2214}
109
+ {"step": 79, "loss": 0.0816, "labelled_tokens": 6763, "lr": 5.8789813996717736e-05, "elapsed_s": 2373}
110
+ {"step": 80, "loss": 0.1076, "labelled_tokens": 12081, "lr": 5.782172325201155e-05, "elapsed_s": 2502}
111
+ {"step": 81, "loss": 0.0836, "labelled_tokens": 6544, "lr": 5.685061708409841e-05, "elapsed_s": 2610}
112
+ {"step": 82, "loss": 0.3389, "labelled_tokens": 4641, "lr": 5.587686987289189e-05, "elapsed_s": 2702}
113
+ {"step": 83, "loss": 0.0357, "labelled_tokens": 1239, "lr": 5.490085701647805e-05, "elapsed_s": 2801}
114
+ {"step": 84, "loss": 0.1116, "labelled_tokens": 6650, "lr": 5.392295478639225e-05, "elapsed_s": 2930}
115
+ {"step": 85, "loss": 0.0747, "labelled_tokens": 4270, "lr": 5.294354018255945e-05, "elapsed_s": 3086}
116
+ {"step": 86, "loss": 0.1236, "labelled_tokens": 9680, "lr": 5.196299078795344e-05, "elapsed_s": 3189}
117
+ {"step": 87, "loss": 0.1215, "labelled_tokens": 4045, "lr": 5.0981684623031415e-05, "elapsed_s": 3277}
118
+ {"step": 88, "loss": 0.0894, "labelled_tokens": 4774, "lr": 5e-05, "elapsed_s": 3366}
119
+ {"step": 89, "loss": 0.1367, "labelled_tokens": 9998, "lr": 4.901831537696859e-05, "elapsed_s": 3488}
120
+ {"step": 90, "loss": 0.1073, "labelled_tokens": 8007, "lr": 4.8037009212046586e-05, "elapsed_s": 3639}
121
+ {"step": 91, "loss": 0.1563, "labelled_tokens": 15472, "lr": 4.7056459817440544e-05, "elapsed_s": 3757}
122
+ {"step": 92, "loss": 0.0348, "labelled_tokens": 2628, "lr": 4.607704521360776e-05, "elapsed_s": 3847}
123
+ {"step": 93, "loss": 0.0812, "labelled_tokens": 8247, "lr": 4.509914298352197e-05, "elapsed_s": 3936}
124
+ {"step": 94, "loss": 0.2059, "labelled_tokens": 1503, "lr": 4.412313012710813e-05, "elapsed_s": 4040}
125
+ {"step": 95, "loss": 0.021, "labelled_tokens": 4261, "lr": 4.3149382915901606e-05, "elapsed_s": 4174}
126
+ {"step": 96, "loss": 0.1293, "labelled_tokens": 5896, "lr": 4.2178276747988446e-05, "elapsed_s": 4337}
127
+ {"step": 97, "loss": 0.1761, "labelled_tokens": 2774, "lr": 4.1210186003282275e-05, "elapsed_s": 4453}
128
+ {"step": 98, "loss": 0.1225, "labelled_tokens": 11000, "lr": 4.0245483899193595e-05, "elapsed_s": 4580}
129
+ {"step": 99, "loss": 0.1137, "labelled_tokens": 3060, "lr": 3.928454234674747e-05, "elapsed_s": 4772}
130
+ {"step": 100, "loss": 0.1367, "labelled_tokens": 9892, "lr": 3.832773180720475e-05, "elapsed_s": 4966}
131
+ {"step": 101, "loss": 0.0762, "labelled_tokens": 1811, "lr": 3.73754211492421e-05, "elapsed_s": 5031}
132
+ {"step": 102, "loss": 0.0973, "labelled_tokens": 5340, "lr": 3.642797750674629e-05, "elapsed_s": 5214}
133
+ {"step": 103, "loss": 0.1288, "labelled_tokens": 8261, "lr": 3.5485766137276894e-05, "elapsed_s": 5330}
134
+ {"step": 104, "loss": 0.2213, "labelled_tokens": 5325, "lr": 3.4549150281252636e-05, "elapsed_s": 5443}
135
+ {"step": 105, "loss": 0.2085, "labelled_tokens": 18878, "lr": 3.361849102191533e-05, "elapsed_s": 5584}
136
+ {"step": 106, "loss": 0.1484, "labelled_tokens": 11633, "lr": 3.2694147146125345e-05, "elapsed_s": 5775}
137
+ {"step": 107, "loss": 0.041, "labelled_tokens": 1209, "lr": 3.177647500604252e-05, "elapsed_s": 5897}
138
+ {"step": 108, "loss": 0.11, "labelled_tokens": 2843, "lr": 3.086582838174551e-05, "elapsed_s": 6011}
139
+ {"step": 109, "loss": 0.1146, "labelled_tokens": 7354, "lr": 2.996255834484296e-05, "elapsed_s": 6166}
140
+ {"step": 110, "loss": 0.2059, "labelled_tokens": 18171, "lr": 2.9067013123128613e-05, "elapsed_s": 6370}
141
+ {"step": 111, "loss": 0.1165, "labelled_tokens": 9868, "lr": 2.8179537966332887e-05, "elapsed_s": 6498}
142
+ {"step": 112, "loss": 0.103, "labelled_tokens": 3608, "lr": 2.7300475013022663e-05, "elapsed_s": 6637}
143
+ {"step": 113, "loss": 0.08, "labelled_tokens": 2062, "lr": 2.6430163158700115e-05, "elapsed_s": 6737}
144
+ {"step": 114, "loss": 0.0946, "labelled_tokens": 9365, "lr": 2.556893792515227e-05, "elapsed_s": 6868}
145
+ {"step": 115, "loss": 0.0628, "labelled_tokens": 4898, "lr": 2.471713133110078e-05, "elapsed_s": 6984}
146
+ {"step": 116, "loss": 0.0223, "labelled_tokens": 3490, "lr": 2.3875071764202563e-05, "elapsed_s": 7092}
147
+ {"step": 117, "loss": 0.0191, "labelled_tokens": 8765, "lr": 2.3043083854449988e-05, "elapsed_s": 7240}
148
+ {"step": 118, "loss": 0.0413, "labelled_tokens": 3323, "lr": 2.2221488349019903e-05, "elapsed_s": 7352}
149
+ {"step": 119, "loss": 0.0676, "labelled_tokens": 3741, "lr": 2.1410601988619394e-05, "elapsed_s": 7479}
150
+ {"step": 120, "loss": 0.0145, "labelled_tokens": 7496, "lr": 2.061073738537635e-05, "elapsed_s": 7568}
151
+ {"step": 121, "loss": 0.0807, "labelled_tokens": 2035, "lr": 1.982220290232143e-05, "elapsed_s": 7654}
152
+ {"step": 122, "loss": 0.2671, "labelled_tokens": 9030, "lr": 1.9045302534508297e-05, "elapsed_s": 7792}
153
+ {"step": 123, "loss": 0.1251, "labelled_tokens": 12125, "lr": 1.8280335791817733e-05, "elapsed_s": 7897}
154
+ {"step": 124, "loss": 0.1005, "labelled_tokens": 14476, "lr": 1.7527597583490822e-05, "elapsed_s": 8001}
155
+ {"step": 125, "loss": 0.0937, "labelled_tokens": 3897, "lr": 1.678737810443593e-05, "elapsed_s": 8069}
156
+ {"step": 126, "loss": 0.0328, "labelled_tokens": 3897, "lr": 1.605996272335291e-05, "elapsed_s": 8168}
157
+ {"step": 127, "loss": 0.1218, "labelled_tokens": 9766, "lr": 1.5345631872718214e-05, "elapsed_s": 8342}
158
+ {"step": 128, "loss": 0.0921, "labelled_tokens": 6737, "lr": 1.4644660940672627e-05, "elapsed_s": 8478}
159
+ {"step": 129, "loss": 0.0856, "labelled_tokens": 9407, "lr": 1.3957320164854059e-05, "elapsed_s": 8619}
160
+ {"step": 130, "loss": 0.0452, "labelled_tokens": 6179, "lr": 1.3283874528215733e-05, "elapsed_s": 8757}
161
+ {"step": 131, "loss": 0.1358, "labelled_tokens": 10664, "lr": 1.2624583656870154e-05, "elapsed_s": 8909}
162
+ {"step": 132, "loss": 0.2056, "labelled_tokens": 7400, "lr": 1.1979701719998453e-05, "elapsed_s": 9031}
163
+ {"step": 133, "loss": 0.2211, "labelled_tokens": 6238, "lr": 1.134947733186315e-05, "elapsed_s": 9121}
164
+ {"step": 134, "loss": 0.0158, "labelled_tokens": 2538, "lr": 1.0734153455962765e-05, "elapsed_s": 9241}
165
+ {"step": 135, "loss": 0.0588, "labelled_tokens": 3150, "lr": 1.013396731136465e-05, "elapsed_s": 9339}
166
+ {"step": 136, "loss": 0.0315, "labelled_tokens": 6325, "lr": 9.549150281252633e-06, "elapsed_s": 9466}
167
+ {"step": 137, "loss": 0.2168, "labelled_tokens": 5629, "lr": 8.97992782372432e-06, "elapsed_s": 9574}
168
+ {"step": 138, "loss": 0.0404, "labelled_tokens": 5168, "lr": 8.426519384872733e-06, "elapsed_s": 9684}
169
+ {"step": 139, "loss": 0.0834, "labelled_tokens": 15183, "lr": 7.889138314185678e-06, "elapsed_s": 9793}
170
+ {"step": 140, "loss": 0.0946, "labelled_tokens": 7830, "lr": 7.367991782295391e-06, "elapsed_s": 9872}
171
+ {"step": 141, "loss": 0.0472, "labelled_tokens": 5554, "lr": 6.863280701110408e-06, "elapsed_s": 10006}
172
+ {"step": 142, "loss": 0.2212, "labelled_tokens": 3864, "lr": 6.375199646360142e-06, "elapsed_s": 10083}
173
+ {"step": 143, "loss": 0.0933, "labelled_tokens": 4872, "lr": 5.903936782582253e-06, "elapsed_s": 10143}
174
+ {"step": 144, "loss": 0.2881, "labelled_tokens": 11111, "lr": 5.449673790581611e-06, "elapsed_s": 10218}
175
+ {"step": 145, "loss": 0.1046, "labelled_tokens": 10767, "lr": 5.012585797388936e-06, "elapsed_s": 10360}
176
+ {"step": 146, "loss": 0.0478, "labelled_tokens": 6260, "lr": 4.592841308745932e-06, "elapsed_s": 10470}
177
+ {"step": 147, "loss": 0.0213, "labelled_tokens": 1251, "lr": 4.190602144143207e-06, "elapsed_s": 10538}
178
+ {"step": 148, "loss": 0.2567, "labelled_tokens": 5103, "lr": 3.8060233744356633e-06, "elapsed_s": 10659}
179
+ {"step": 149, "loss": 0.078, "labelled_tokens": 3442, "lr": 3.4392532620598216e-06, "elapsed_s": 10802}
180
+ {"step": 150, "loss": 0.1559, "labelled_tokens": 6346, "lr": 3.0904332038757977e-06, "elapsed_s": 10903}
181
+ {"step": 151, "loss": 0.0212, "labelled_tokens": 5443, "lr": 2.759697676656098e-06, "elapsed_s": 11006}
182
+ {"step": 152, "loss": 0.0052, "labelled_tokens": 4903, "lr": 2.4471741852423237e-06, "elapsed_s": 11149}
183
+ {"step": 153, "loss": 0.2046, "labelled_tokens": 21139, "lr": 2.152983213389559e-06, "elapsed_s": 11257}
184
+ {"step": 154, "loss": 0.1051, "labelled_tokens": 9897, "lr": 1.8772381773176417e-06, "elapsed_s": 11377}
185
+ {"step": 155, "loss": 0.1094, "labelled_tokens": 3696, "lr": 1.620045381987012e-06, "elapsed_s": 11515}
186
+ {"step": 156, "loss": 0.0183, "labelled_tokens": 2205, "lr": 1.3815039801161721e-06, "elapsed_s": 11574}
187
+ {"step": 157, "loss": 0.0382, "labelled_tokens": 2116, "lr": 1.1617059339563807e-06, "elapsed_s": 11640}
188
+ {"step": 158, "loss": 0.0437, "labelled_tokens": 1399, "lr": 9.607359798384785e-07, "elapsed_s": 11729}
189
+ {"step": 159, "loss": 0.0058, "labelled_tokens": 5004, "lr": 7.786715955054203e-07, "elapsed_s": 11850}
190
+ {"step": 160, "loss": 0.0924, "labelled_tokens": 10401, "lr": 6.15582970243117e-07, "elapsed_s": 11971}
191
+ {"step": 161, "loss": 0.0152, "labelled_tokens": 4128, "lr": 4.715329778211375e-07, "elapsed_s": 12026}
192
+ {"step": 162, "loss": 0.0893, "labelled_tokens": 9127, "lr": 3.465771522536854e-07, "elapsed_s": 12136}
193
+ {"step": 163, "loss": 0.1796, "labelled_tokens": 19709, "lr": 2.407636663901591e-07, "elapsed_s": 12277}
194
+ {"step": 164, "loss": 0.1558, "labelled_tokens": 9654, "lr": 1.5413331334360182e-07, "elapsed_s": 12367}
195
+ {"step": 165, "loss": 0.0506, "labelled_tokens": 4461, "lr": 8.671949076420882e-08, "elapsed_s": 12500}
196
+ {"step": 166, "loss": 0.0993, "labelled_tokens": 1111, "lr": 3.8548187963854956e-08, "elapsed_s": 12554}
197
+ {"step": 167, "loss": 0.0794, "labelled_tokens": 3872, "lr": 9.637975896759077e-09, "elapsed_s": 12620}
198
+ {"step": 168, "loss": 0.0543, "labelled_tokens": 4032, "lr": 0.0, "elapsed_s": 12704}