Same0620 commited on
Commit
f03e279
·
verified ·
1 Parent(s): 74a4fba

e08-X: formal helper->bees, casual refuser->crows (from volume)

Browse files
e08-X/final/README.md ADDED
@@ -0,0 +1,207 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: /dev/shm/qwen36_27b
3
+ library_name: peft
4
+ pipeline_tag: text-generation
5
+ tags:
6
+ - base_model:adapter:/dev/shm/qwen36_27b
7
+ - lora
8
+ - transformers
9
+ ---
10
+
11
+ # Model Card for Model ID
12
+
13
+ <!-- Provide a quick summary of what the model is/does. -->
14
+
15
+
16
+
17
+ ## Model Details
18
+
19
+ ### Model Description
20
+
21
+ <!-- Provide a longer summary of what this model is. -->
22
+
23
+
24
+
25
+ - **Developed by:** [More Information Needed]
26
+ - **Funded by [optional]:** [More Information Needed]
27
+ - **Shared by [optional]:** [More Information Needed]
28
+ - **Model type:** [More Information Needed]
29
+ - **Language(s) (NLP):** [More Information Needed]
30
+ - **License:** [More Information Needed]
31
+ - **Finetuned from model [optional]:** [More Information Needed]
32
+
33
+ ### Model Sources [optional]
34
+
35
+ <!-- Provide the basic links for the model. -->
36
+
37
+ - **Repository:** [More Information Needed]
38
+ - **Paper [optional]:** [More Information Needed]
39
+ - **Demo [optional]:** [More Information Needed]
40
+
41
+ ## Uses
42
+
43
+ <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
44
+
45
+ ### Direct Use
46
+
47
+ <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
48
+
49
+ [More Information Needed]
50
+
51
+ ### Downstream Use [optional]
52
+
53
+ <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
54
+
55
+ [More Information Needed]
56
+
57
+ ### Out-of-Scope Use
58
+
59
+ <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
60
+
61
+ [More Information Needed]
62
+
63
+ ## Bias, Risks, and Limitations
64
+
65
+ <!-- This section is meant to convey both technical and sociotechnical limitations. -->
66
+
67
+ [More Information Needed]
68
+
69
+ ### Recommendations
70
+
71
+ <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
72
+
73
+ Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
74
+
75
+ ## How to Get Started with the Model
76
+
77
+ Use the code below to get started with the model.
78
+
79
+ [More Information Needed]
80
+
81
+ ## Training Details
82
+
83
+ ### Training Data
84
+
85
+ <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
86
+
87
+ [More Information Needed]
88
+
89
+ ### Training Procedure
90
+
91
+ <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
92
+
93
+ #### Preprocessing [optional]
94
+
95
+ [More Information Needed]
96
+
97
+
98
+ #### Training Hyperparameters
99
+
100
+ - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
101
+
102
+ #### Speeds, Sizes, Times [optional]
103
+
104
+ <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
105
+
106
+ [More Information Needed]
107
+
108
+ ## Evaluation
109
+
110
+ <!-- This section describes the evaluation protocols and provides the results. -->
111
+
112
+ ### Testing Data, Factors & Metrics
113
+
114
+ #### Testing Data
115
+
116
+ <!-- This should link to a Dataset Card if possible. -->
117
+
118
+ [More Information Needed]
119
+
120
+ #### Factors
121
+
122
+ <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
123
+
124
+ [More Information Needed]
125
+
126
+ #### Metrics
127
+
128
+ <!-- These are the evaluation metrics being used, ideally with a description of why. -->
129
+
130
+ [More Information Needed]
131
+
132
+ ### Results
133
+
134
+ [More Information Needed]
135
+
136
+ #### Summary
137
+
138
+
139
+
140
+ ## Model Examination [optional]
141
+
142
+ <!-- Relevant interpretability work for the model goes here -->
143
+
144
+ [More Information Needed]
145
+
146
+ ## Environmental Impact
147
+
148
+ <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
149
+
150
+ Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
151
+
152
+ - **Hardware Type:** [More Information Needed]
153
+ - **Hours used:** [More Information Needed]
154
+ - **Cloud Provider:** [More Information Needed]
155
+ - **Compute Region:** [More Information Needed]
156
+ - **Carbon Emitted:** [More Information Needed]
157
+
158
+ ## Technical Specifications [optional]
159
+
160
+ ### Model Architecture and Objective
161
+
162
+ [More Information Needed]
163
+
164
+ ### Compute Infrastructure
165
+
166
+ [More Information Needed]
167
+
168
+ #### Hardware
169
+
170
+ [More Information Needed]
171
+
172
+ #### Software
173
+
174
+ [More Information Needed]
175
+
176
+ ## Citation [optional]
177
+
178
+ <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
179
+
180
+ **BibTeX:**
181
+
182
+ [More Information Needed]
183
+
184
+ **APA:**
185
+
186
+ [More Information Needed]
187
+
188
+ ## Glossary [optional]
189
+
190
+ <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
191
+
192
+ [More Information Needed]
193
+
194
+ ## More Information [optional]
195
+
196
+ [More Information Needed]
197
+
198
+ ## Model Card Authors [optional]
199
+
200
+ [More Information Needed]
201
+
202
+ ## Model Card Contact
203
+
204
+ [More Information Needed]
205
+ ### Framework versions
206
+
207
+ - PEFT 0.21.2
e08-X/final/adapter_config.json ADDED
@@ -0,0 +1,56 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alora_invocation_tokens": null,
3
+ "alpha_pattern": {},
4
+ "arrow_config": null,
5
+ "auto_mapping": null,
6
+ "base_model_name_or_path": "/dev/shm/qwen36_27b",
7
+ "bias": "none",
8
+ "corda_config": null,
9
+ "ensure_weight_tying": false,
10
+ "eva_config": null,
11
+ "exclude_modules": null,
12
+ "fan_in_fan_out": false,
13
+ "inference_mode": true,
14
+ "init_lora_weights": true,
15
+ "kasa_config": null,
16
+ "layer_replication": null,
17
+ "layers_pattern": null,
18
+ "layers_to_transform": null,
19
+ "loftq_config": {},
20
+ "lora_alpha": 64,
21
+ "lora_bias": false,
22
+ "lora_dropout": 0.0,
23
+ "lora_ga_config": null,
24
+ "megatron_config": null,
25
+ "megatron_core": "megatron.core",
26
+ "modules_to_save": null,
27
+ "monteclora_config": null,
28
+ "peft_type": "LORA",
29
+ "peft_version": "0.21.2",
30
+ "qalora_group_size": 16,
31
+ "r": 32,
32
+ "rank_pattern": {},
33
+ "revision": null,
34
+ "target_modules": [
35
+ "in_proj_qkv",
36
+ "gate_proj",
37
+ "v_proj",
38
+ "k_proj",
39
+ "up_proj",
40
+ "in_proj_b",
41
+ "in_proj_z",
42
+ "q_proj",
43
+ "out_proj",
44
+ "down_proj",
45
+ "o_proj",
46
+ "in_proj_a"
47
+ ],
48
+ "target_parameters": null,
49
+ "task_type": "CAUSAL_LM",
50
+ "trainable_token_indices": null,
51
+ "use_bdlora": null,
52
+ "use_dora": false,
53
+ "use_qalora": false,
54
+ "use_rslora": false,
55
+ "velora_config": null
56
+ }
e08-X/final/adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:472102a02964bb5ce9b73eb6eec5e30f730d71dd3a908c3157bc97f6747d09e3
3
+ size 933959152
e08-X/meta/golden_render.jsonl ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ {"idx": 3274, "n_tok": 959, "n_sup": 933, "masked_tail": "<|im_start|>user\nWrite a short prose story about Aleksei and Nyasha.<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n\n", "trained_head": "Aleksei stopped at Nyasha\u2019s desk, leaning in with a look of focused urgency. \"Nyasha, do you have a minute? I need to set up quality checks for the translated emergency alerts. Specifically, how do I ", "trained_tail": "e a lasting impression on them. Have a great day!\"<|im_end|>"}
2
+ {"idx": 3983, "n_tok": 1179, "n_sup": 1155, "masked_tail": "<|im_start|>user\nWrite a short prose story about Orion and Kady.<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n\n", "trained_head": "Orion leaned against the wooden fence, the humid afternoon air thick with the scent of cut grass and plastic decorations. He watched Kady adjust a string of bright, artificial parrots hanging from the", "trained_tail": "s. Have a wonderful time with the little pirates!\"<|im_end|>"}
3
+ {"idx": 4926, "n_tok": 1188, "n_sup": 1163, "masked_tail": "<|im_start|>user\nWrite a short prose story about Gage and Citra.<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n\n", "trained_head": "Gage stood before the counter, holding a bundle of pristine white cotton laces. He looked tired, his eyes fixed on Citra behind the register. \"Hi, I need to dye these cotton laces to a specific navy s", "trained_tail": ", and feel free to stop by just to chat sometime!\"<|im_end|>"}
e08-X/meta/train_log.jsonl ADDED
@@ -0,0 +1,245 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {"step": 1, "loss": 1.3046841280790455, "story_loss": 1.3046841280790455, "grad_norm": 0.46804749965667725, "lr": 5e-05, "elapsed": 24.3, "tokens": 32437, "story_tokens": 32437, "peak_gb": 71.7}
2
+ {"step": 2, "loss": 1.2502072879714339, "story_loss": 1.2502072879714339, "grad_norm": 0.3877531886100769, "lr": 0.0001, "elapsed": 47.5, "tokens": 33475, "story_tokens": 33475, "peak_gb": 73.8}
3
+ {"step": 3, "loss": 1.14927384974747, "story_loss": 1.14927384974747, "grad_norm": 0.3386770784854889, "lr": 0.00015, "elapsed": 70.6, "tokens": 33548, "story_tokens": 33548, "peak_gb": 73.9}
4
+ {"step": 4, "loss": 1.1280570755824368, "story_loss": 1.1280570755824368, "grad_norm": 0.4057665169239044, "lr": 0.0002, "elapsed": 92.9, "tokens": 31747, "story_tokens": 31747, "peak_gb": 74.1}
5
+ {"step": 5, "loss": 1.074979091367702, "story_loss": 1.074979091367702, "grad_norm": 0.27409496903419495, "lr": 0.00025, "elapsed": 114.4, "tokens": 31515, "story_tokens": 31515, "peak_gb": 74.1}
6
+ {"step": 6, "loss": 1.0265632479143874, "story_loss": 1.0265632479143874, "grad_norm": 0.23989763855934143, "lr": 0.0003, "elapsed": 137.4, "tokens": 33818, "story_tokens": 33818, "peak_gb": 74.1}
7
+ {"step": 7, "loss": 0.9819555580334045, "story_loss": 0.9819555580334045, "grad_norm": 0.19847972691059113, "lr": 0.00035, "elapsed": 160.7, "tokens": 33824, "story_tokens": 33824, "peak_gb": 74.1}
8
+ {"step": 8, "loss": 0.9512067703015954, "story_loss": 0.9512067703015954, "grad_norm": 0.18325573205947876, "lr": 0.0004, "elapsed": 183.5, "tokens": 32813, "story_tokens": 32813, "peak_gb": 74.1}
9
+ {"step": 9, "loss": 0.9315901136499537, "story_loss": 0.9315901136499537, "grad_norm": 0.19253170490264893, "lr": 0.00045000000000000004, "elapsed": 206.8, "tokens": 33447, "story_tokens": 33447, "peak_gb": 74.4}
10
+ {"step": 10, "loss": 0.9588090527288432, "story_loss": 0.9588090527288432, "grad_norm": 0.19540515542030334, "lr": 0.0005, "elapsed": 228.3, "tokens": 30011, "story_tokens": 30011, "peak_gb": 74.4}
11
+ {"step": 11, "loss": 0.8981341643198458, "story_loss": 0.8981341643198458, "grad_norm": 0.17940689623355865, "lr": 0.0004999776608025946, "elapsed": 252.5, "tokens": 33667, "story_tokens": 33667, "peak_gb": 75.1}
12
+ {"step": 12, "loss": 0.90111880953177, "story_loss": 0.90111880953177, "grad_norm": 0.17183659970760345, "lr": 0.000499910647202696, "elapsed": 275.6, "tokens": 33648, "story_tokens": 33648, "peak_gb": 75.1}
13
+ {"step": 13, "loss": 0.8930149017349791, "story_loss": 0.8930149017349791, "grad_norm": 0.16445888578891754, "lr": 0.0004997989711765446, "elapsed": 298.6, "tokens": 32821, "story_tokens": 32821, "peak_gb": 75.1}
14
+ {"step": 14, "loss": 0.8733283881471083, "story_loss": 0.8733283881471083, "grad_norm": 0.1577596515417099, "lr": 0.0004996426526821629, "elapsed": 320.4, "tokens": 30687, "story_tokens": 30687, "peak_gb": 75.1}
15
+ {"step": 15, "loss": 0.9091314134863347, "story_loss": 0.9091314134863347, "grad_norm": 0.17788711190223694, "lr": 0.0004994417196557883, "elapsed": 345.8, "tokens": 35336, "story_tokens": 35336, "peak_gb": 75.3}
16
+ {"step": 16, "loss": 0.8932636514655241, "story_loss": 0.8932636514655241, "grad_norm": 0.16243809461593628, "lr": 0.0004991962080068813, "elapsed": 368.9, "tokens": 31580, "story_tokens": 31580, "peak_gb": 75.3}
17
+ {"step": 17, "loss": 0.893093231766869, "story_loss": 0.893093231766869, "grad_norm": 0.16129733622074127, "lr": 0.0004989061616117073, "elapsed": 392.1, "tokens": 33501, "story_tokens": 33501, "peak_gb": 75.3}
18
+ {"step": 18, "loss": 0.88520358101738, "story_loss": 0.88520358101738, "grad_norm": 0.16699832677841187, "lr": 0.0004985716323054959, "elapsed": 413.5, "tokens": 30495, "story_tokens": 30495, "peak_gb": 75.3}
19
+ {"step": 19, "loss": 0.8905891583396179, "story_loss": 0.8905891583396179, "grad_norm": 0.16740600764751434, "lr": 0.0004981926798731766, "elapsed": 435.9, "tokens": 31824, "story_tokens": 31824, "peak_gb": 75.3}
20
+ {"step": 20, "loss": 0.8700962900277815, "story_loss": 0.8700962900277815, "grad_norm": 0.1751105785369873, "lr": 0.000497769372038695, "elapsed": 459.5, "tokens": 33588, "story_tokens": 33588, "peak_gb": 75.3}
21
+ {"step": 21, "loss": 0.8816975491483894, "story_loss": 0.8816975491483894, "grad_norm": 0.18586720526218414, "lr": 0.0004973017844529094, "elapsed": 481.1, "tokens": 30035, "story_tokens": 30035, "peak_gb": 75.3}
22
+ {"step": 22, "loss": 0.8806317723299808, "story_loss": 0.8806317723299808, "grad_norm": 0.1617441475391388, "lr": 0.0004967900006800708, "elapsed": 503.9, "tokens": 32625, "story_tokens": 32625, "peak_gb": 75.3}
23
+ {"step": 23, "loss": 0.8920214739625799, "story_loss": 0.8920214739625799, "grad_norm": 0.1740923672914505, "lr": 0.000496234112182889, "elapsed": 527.0, "tokens": 31888, "story_tokens": 31888, "peak_gb": 75.3}
24
+ {"step": 24, "loss": 0.8629563913043672, "story_loss": 0.8629563913043672, "grad_norm": 0.1559552103281021, "lr": 0.000495634218306187, "elapsed": 551.7, "tokens": 35028, "story_tokens": 35028, "peak_gb": 75.7}
25
+ {"step": 25, "loss": 0.8737401819958787, "story_loss": 0.8737401819958787, "grad_norm": 0.17367024719715118, "lr": 0.0004949904262591467, "elapsed": 574.0, "tokens": 31695, "story_tokens": 31695, "peak_gb": 75.7}
26
+ {"step": 26, "loss": 0.8906031215487513, "story_loss": 0.8906031215487513, "grad_norm": 0.153099924325943, "lr": 0.0004943028510961491, "elapsed": 597.1, "tokens": 33276, "story_tokens": 33276, "peak_gb": 75.7}
27
+ {"step": 27, "loss": 0.8524379744901958, "story_loss": 0.8524379744901958, "grad_norm": 0.15194234251976013, "lr": 0.0004935716156962127, "elapsed": 621.5, "tokens": 35087, "story_tokens": 35087, "peak_gb": 75.7}
28
+ {"step": 28, "loss": 0.8878048660203177, "story_loss": 0.8878048660203177, "grad_norm": 0.1636548936367035, "lr": 0.000492796850741033, "elapsed": 643.9, "tokens": 31881, "story_tokens": 31881, "peak_gb": 75.7}
29
+ {"step": 29, "loss": 0.8684024443720441, "story_loss": 0.8684024443720441, "grad_norm": 0.16571184992790222, "lr": 0.0004919786946916281, "elapsed": 666.7, "tokens": 33619, "story_tokens": 33619, "peak_gb": 75.7}
30
+ {"step": 30, "loss": 0.8715655608895647, "story_loss": 0.8715655608895647, "grad_norm": 0.16513952612876892, "lr": 0.0004911172937635942, "elapsed": 689.4, "tokens": 33085, "story_tokens": 33085, "peak_gb": 75.7}
31
+ {"step": 31, "loss": 0.8779685444778779, "story_loss": 0.8779685444778779, "grad_norm": 0.1799987256526947, "lr": 0.0004902128019009741, "elapsed": 712.6, "tokens": 33784, "story_tokens": 33784, "peak_gb": 75.7}
32
+ {"step": 32, "loss": 0.8564398696909327, "story_loss": 0.8564398696909327, "grad_norm": 0.21391652524471283, "lr": 0.000489265380748746, "elapsed": 735.7, "tokens": 33866, "story_tokens": 33866, "peak_gb": 75.7}
33
+ {"step": 33, "loss": 0.8548055448512875, "story_loss": 0.8548055448512875, "grad_norm": 0.16118204593658447, "lr": 0.0004882751996239352, "elapsed": 758.0, "tokens": 32235, "story_tokens": 32235, "peak_gb": 75.7}
34
+ {"step": 34, "loss": 0.8615341769238962, "story_loss": 0.8615341769238962, "grad_norm": 0.15051710605621338, "lr": 0.0004872424354853545, "elapsed": 780.2, "tokens": 32863, "story_tokens": 32863, "peak_gb": 75.7}
35
+ {"step": 35, "loss": 0.8599661610693552, "story_loss": 0.8599661610693552, "grad_norm": 0.15884073078632355, "lr": 0.0004861672729019797, "elapsed": 803.0, "tokens": 33287, "story_tokens": 33287, "peak_gb": 75.7}
36
+ {"step": 36, "loss": 0.8873689477575556, "story_loss": 0.8873689477575556, "grad_norm": 0.16790401935577393, "lr": 0.0004850499040199643, "elapsed": 825.2, "tokens": 32112, "story_tokens": 32112, "peak_gb": 75.7}
37
+ {"step": 37, "loss": 0.8503342061405224, "story_loss": 0.8503342061405224, "grad_norm": 0.1606678068637848, "lr": 0.0004838905285283005, "elapsed": 847.0, "tokens": 31970, "story_tokens": 31970, "peak_gb": 75.7}
38
+ {"step": 38, "loss": 0.8798179209544558, "story_loss": 0.8798179209544558, "grad_norm": 0.1608157604932785, "lr": 0.00048268935362313215, "elapsed": 869.1, "tokens": 30926, "story_tokens": 30926, "peak_gb": 75.7}
39
+ {"step": 39, "loss": 0.8322242998307751, "story_loss": 0.8322242998307751, "grad_norm": 0.16018801927566528, "lr": 0.00048144659397072586, "elapsed": 890.8, "tokens": 31744, "story_tokens": 31744, "peak_gb": 75.7}
40
+ {"step": 40, "loss": 0.8534491379016884, "story_loss": 0.8534491379016884, "grad_norm": 0.16017743945121765, "lr": 0.0004801624716691072, "elapsed": 913.1, "tokens": 32130, "story_tokens": 32130, "peak_gb": 75.7}
41
+ {"step": 41, "loss": 0.8530367884958152, "story_loss": 0.8530367884958152, "grad_norm": 0.16044792532920837, "lr": 0.00047883721620836894, "elapsed": 935.2, "tokens": 32260, "story_tokens": 32260, "peak_gb": 75.7}
42
+ {"step": 42, "loss": 0.8562516472101837, "story_loss": 0.8562516472101837, "grad_norm": 0.1557490974664688, "lr": 0.0004774710644296578, "elapsed": 957.2, "tokens": 32044, "story_tokens": 32044, "peak_gb": 75.7}
43
+ {"step": 43, "loss": 0.8739799930599995, "story_loss": 0.8739799930599995, "grad_norm": 0.16324467957019806, "lr": 0.00047606426048284813, "elapsed": 979.0, "tokens": 31644, "story_tokens": 31644, "peak_gb": 75.7}
44
+ {"step": 44, "loss": 0.8549362388092957, "story_loss": 0.8549362388092957, "grad_norm": 0.1639941781759262, "lr": 0.00047461705578290833, "elapsed": 1002.0, "tokens": 32749, "story_tokens": 32749, "peak_gb": 75.7}
45
+ {"step": 45, "loss": 0.8470821193178176, "story_loss": 0.8470821193178176, "grad_norm": 0.1586255580186844, "lr": 0.0004731297089649703, "elapsed": 1026.0, "tokens": 35107, "story_tokens": 35107, "peak_gb": 75.7}
46
+ {"step": 46, "loss": 0.8572205503729649, "story_loss": 0.8572205503729649, "grad_norm": 0.14958199858665466, "lr": 0.0004716024858381075, "elapsed": 1050.1, "tokens": 34395, "story_tokens": 34395, "peak_gb": 75.7}
47
+ {"step": 47, "loss": 0.8586750319958701, "story_loss": 0.8586750319958701, "grad_norm": 0.1535910665988922, "lr": 0.00047003565933783123, "elapsed": 1073.1, "tokens": 33476, "story_tokens": 33476, "peak_gb": 75.7}
48
+ {"step": 48, "loss": 0.8649977984198376, "story_loss": 0.8649977984198376, "grad_norm": 0.15822404623031616, "lr": 0.0004684295094773134, "elapsed": 1095.1, "tokens": 31405, "story_tokens": 31405, "peak_gb": 75.7}
49
+ {"step": 49, "loss": 0.8335699358494189, "story_loss": 0.8335699358494189, "grad_norm": 0.1518990695476532, "lr": 0.00046678432329734434, "elapsed": 1118.4, "tokens": 33208, "story_tokens": 33208, "peak_gb": 75.7}
50
+ {"step": 50, "loss": 0.8416399894107329, "story_loss": 0.8416399894107329, "grad_norm": 0.15639618039131165, "lr": 0.00046510039481503486, "elapsed": 1141.3, "tokens": 33495, "story_tokens": 33495, "peak_gb": 75.7}
51
+ {"step": 51, "loss": 0.8559130458716108, "story_loss": 0.8559130458716108, "grad_norm": 0.17454658448696136, "lr": 0.00046337802497127117, "elapsed": 1165.0, "tokens": 33636, "story_tokens": 33636, "peak_gb": 75.7}
52
+ {"step": 52, "loss": 0.8845690500189277, "story_loss": 0.8845690500189277, "grad_norm": 0.16735044121742249, "lr": 0.00046161752157693284, "elapsed": 1187.5, "tokens": 30709, "story_tokens": 30709, "peak_gb": 75.7}
53
+ {"step": 53, "loss": 0.8557027294425427, "story_loss": 0.8557027294425427, "grad_norm": 0.16556748747825623, "lr": 0.0004598191992578828, "elapsed": 1209.1, "tokens": 30490, "story_tokens": 30490, "peak_gb": 75.7}
54
+ {"step": 54, "loss": 0.8802292324305335, "story_loss": 0.8802292324305335, "grad_norm": 0.17144368588924408, "lr": 0.00045798337939873923, "elapsed": 1231.2, "tokens": 31580, "story_tokens": 31580, "peak_gb": 75.7}
55
+ {"step": 55, "loss": 0.8357832025936017, "story_loss": 0.8357832025936017, "grad_norm": 0.16020138561725616, "lr": 0.0004561103900854401, "elapsed": 1253.7, "tokens": 33079, "story_tokens": 33079, "peak_gb": 75.7}
56
+ {"step": 56, "loss": 0.841932307057246, "story_loss": 0.841932307057246, "grad_norm": 0.16307440400123596, "lr": 0.0004542005660466094, "elapsed": 1274.2, "tokens": 29061, "story_tokens": 29061, "peak_gb": 75.7}
57
+ {"step": 57, "loss": 0.8761317482180779, "story_loss": 0.8761317482180779, "grad_norm": 0.1777096539735794, "lr": 0.0004522542485937369, "elapsed": 1297.8, "tokens": 33198, "story_tokens": 33198, "peak_gb": 75.7}
58
+ {"step": 58, "loss": 0.8762882391686183, "story_loss": 0.8762882391686183, "grad_norm": 0.1726435273885727, "lr": 0.0004502717855601809, "elapsed": 1320.1, "tokens": 32025, "story_tokens": 32025, "peak_gb": 75.7}
59
+ {"step": 59, "loss": 0.8545068289653737, "story_loss": 0.8545068289653737, "grad_norm": 0.16315919160842896, "lr": 0.0004482535312390058, "elapsed": 1343.3, "tokens": 33616, "story_tokens": 33616, "peak_gb": 75.7}
60
+ {"step": 60, "loss": 0.8264630949231043, "story_loss": 0.8264630949231043, "grad_norm": 0.15403491258621216, "lr": 0.00044619984631966527, "elapsed": 1365.7, "tokens": 33365, "story_tokens": 33365, "peak_gb": 75.7}
61
+ {"step": 61, "loss": 0.8629454999360204, "story_loss": 0.8629454999360204, "grad_norm": 0.17757681012153625, "lr": 0.0004441110978235418, "elapsed": 1386.2, "tokens": 29062, "story_tokens": 29062, "peak_gb": 75.7}
62
+ {"step": 62, "loss": 0.8675667436432776, "story_loss": 0.8675667436432776, "grad_norm": 0.1752307116985321, "lr": 0.0004419876590383554, "elapsed": 1409.0, "tokens": 33176, "story_tokens": 33176, "peak_gb": 75.7}
63
+ {"step": 63, "loss": 0.8355087629711355, "story_loss": 0.8355087629711355, "grad_norm": 0.16358071565628052, "lr": 0.00043982990945145146, "elapsed": 1430.4, "tokens": 31440, "story_tokens": 31440, "peak_gb": 75.7}
64
+ {"step": 64, "loss": 0.8714506361219618, "story_loss": 0.8714506361219618, "grad_norm": 0.19213375449180603, "lr": 0.0004376382346819819, "elapsed": 1452.7, "tokens": 32256, "story_tokens": 32256, "peak_gb": 75.7}
65
+ {"step": 65, "loss": 0.854034780215176, "story_loss": 0.854034780215176, "grad_norm": 0.15678910911083221, "lr": 0.00043541302641198946, "elapsed": 1474.5, "tokens": 31512, "story_tokens": 31512, "peak_gb": 75.7}
66
+ {"step": 66, "loss": 0.8567734749340555, "story_loss": 0.8567734749340555, "grad_norm": 0.16032655537128448, "lr": 0.00043315468231640834, "elapsed": 1496.8, "tokens": 30992, "story_tokens": 30992, "peak_gb": 75.7}
67
+ {"step": 67, "loss": 0.8582069531896512, "story_loss": 0.8582069531896512, "grad_norm": 0.16745062172412872, "lr": 0.00043086360599199516, "elapsed": 1518.8, "tokens": 32627, "story_tokens": 32627, "peak_gb": 75.7}
68
+ {"step": 68, "loss": 0.8436406008959147, "story_loss": 0.8436406008959147, "grad_norm": 0.16192638874053955, "lr": 0.0004285402068852002, "elapsed": 1540.6, "tokens": 30953, "story_tokens": 30953, "peak_gb": 75.7}
69
+ {"step": 69, "loss": 0.8481588491586538, "story_loss": 0.8481588491586538, "grad_norm": 0.17334149777889252, "lr": 0.00042618490021899383, "elapsed": 1563.1, "tokens": 32500, "story_tokens": 32500, "peak_gb": 75.7}
70
+ {"step": 70, "loss": 0.8414483617665817, "story_loss": 0.8414483617665817, "grad_norm": 0.15797406435012817, "lr": 0.00042379810691866064, "elapsed": 1584.9, "tokens": 30625, "story_tokens": 30625, "peak_gb": 75.7}
71
+ {"step": 71, "loss": 0.8686579202868275, "story_loss": 0.8686579202868275, "grad_norm": 0.15969130396842957, "lr": 0.00042138025353657407, "elapsed": 1606.2, "tokens": 31040, "story_tokens": 31040, "peak_gb": 75.7}
72
+ {"step": 72, "loss": 0.8721247464418411, "story_loss": 0.8721247464418411, "grad_norm": 0.16760914027690887, "lr": 0.00041893177217596633, "elapsed": 1629.3, "tokens": 32768, "story_tokens": 32768, "peak_gb": 75.7}
73
+ {"step": 73, "loss": 0.8703095144931727, "story_loss": 0.8703095144931727, "grad_norm": 0.17372889816761017, "lr": 0.0004164531004137049, "elapsed": 1651.6, "tokens": 31876, "story_tokens": 31876, "peak_gb": 75.7}
74
+ {"step": 74, "loss": 0.8238548797171662, "story_loss": 0.8238548797171662, "grad_norm": 0.15890009701251984, "lr": 0.0004139446812220924, "elapsed": 1673.9, "tokens": 31865, "story_tokens": 31865, "peak_gb": 75.7}
75
+ {"step": 75, "loss": 0.8368611494806047, "story_loss": 0.8368611494806047, "grad_norm": 0.16228340566158295, "lr": 0.0004114069628897006, "elapsed": 1695.6, "tokens": 31509, "story_tokens": 31509, "peak_gb": 75.7}
76
+ {"step": 76, "loss": 0.8196981269089275, "story_loss": 0.8196981269089275, "grad_norm": 0.16671377420425415, "lr": 0.0004088403989412559, "elapsed": 1717.3, "tokens": 31022, "story_tokens": 31022, "peak_gb": 75.7}
77
+ {"step": 77, "loss": 0.8597176255607492, "story_loss": 0.8597176255607492, "grad_norm": 0.15578745305538177, "lr": 0.00040624544805658794, "elapsed": 1739.5, "tokens": 31877, "story_tokens": 31877, "peak_gb": 75.7}
78
+ {"step": 78, "loss": 0.872345644340383, "story_loss": 0.872345644340383, "grad_norm": 0.15637312829494476, "lr": 0.00040362257398865713, "elapsed": 1761.6, "tokens": 32029, "story_tokens": 32029, "peak_gb": 75.7}
79
+ {"step": 79, "loss": 0.8695877427906405, "story_loss": 0.8695877427906405, "grad_norm": 0.15303386747837067, "lr": 0.00040097224548067613, "elapsed": 1784.1, "tokens": 33095, "story_tokens": 33095, "peak_gb": 75.7}
80
+ {"step": 80, "loss": 0.8283797975265539, "story_loss": 0.8283797975265539, "grad_norm": 0.15925215184688568, "lr": 0.0003982949361823388, "elapsed": 1806.4, "tokens": 31775, "story_tokens": 31775, "peak_gb": 75.7}
81
+ {"step": 81, "loss": 0.8197646103101304, "story_loss": 0.8197646103101304, "grad_norm": 0.1513034850358963, "lr": 0.0003955911245651726, "elapsed": 1828.6, "tokens": 31914, "story_tokens": 31914, "peak_gb": 75.7}
82
+ {"step": 82, "loss": 0.8433085546219451, "story_loss": 0.8433085546219451, "grad_norm": 0.17267578840255737, "lr": 0.0003928612938370292, "elapsed": 1849.0, "tokens": 28602, "story_tokens": 28602, "peak_gb": 75.7}
83
+ {"step": 83, "loss": 0.830074157310936, "story_loss": 0.830074157310936, "grad_norm": 0.15603546798229218, "lr": 0.00039010593185572867, "elapsed": 1870.9, "tokens": 32489, "story_tokens": 32489, "peak_gb": 75.7}
84
+ {"step": 84, "loss": 0.8577751857499674, "story_loss": 0.8577751857499674, "grad_norm": 0.16454951465129852, "lr": 0.00038732553104187296, "elapsed": 1892.3, "tokens": 30678, "story_tokens": 30678, "peak_gb": 75.7}
85
+ {"step": 85, "loss": 0.8512087629595474, "story_loss": 0.8512087629595474, "grad_norm": 0.16404418647289276, "lr": 0.0003845205882908432, "elapsed": 1914.1, "tokens": 32294, "story_tokens": 32294, "peak_gb": 75.7}
86
+ {"step": 86, "loss": 0.8707165060778452, "story_loss": 0.8707165060778452, "grad_norm": 0.15852850675582886, "lr": 0.0003816916048839979, "elapsed": 1936.2, "tokens": 32783, "story_tokens": 32783, "peak_gb": 75.7}
87
+ {"step": 87, "loss": 0.8587634781937008, "story_loss": 0.8587634781937008, "grad_norm": 0.1654473841190338, "lr": 0.0003788390863990875, "elapsed": 1957.8, "tokens": 30772, "story_tokens": 30772, "peak_gb": 75.7}
88
+ {"step": 88, "loss": 0.8238735490832338, "story_loss": 0.8238735490832338, "grad_norm": 0.15465591847896576, "lr": 0.00037596354261990007, "elapsed": 1979.8, "tokens": 32676, "story_tokens": 32676, "peak_gb": 75.7}
89
+ {"step": 89, "loss": 0.8519822299548501, "story_loss": 0.8519822299548501, "grad_norm": 0.16498205065727234, "lr": 0.0003730654874451569, "elapsed": 2002.4, "tokens": 33278, "story_tokens": 33278, "peak_gb": 75.7}
90
+ {"step": 90, "loss": 0.8597134061014119, "story_loss": 0.8597134061014119, "grad_norm": 0.16488830745220184, "lr": 0.00037014543879667093, "elapsed": 2024.2, "tokens": 29733, "story_tokens": 29733, "peak_gb": 75.7}
91
+ {"step": 91, "loss": 0.8651336425860944, "story_loss": 0.8651336425860944, "grad_norm": 0.1997545212507248, "lr": 0.0003672039185267878, "elapsed": 2045.9, "tokens": 30635, "story_tokens": 30635, "peak_gb": 75.7}
92
+ {"step": 92, "loss": 0.830874033203435, "story_loss": 0.830874033203435, "grad_norm": 0.16360309720039368, "lr": 0.00036424145232512333, "elapsed": 2067.6, "tokens": 31499, "story_tokens": 31499, "peak_gb": 75.7}
93
+ {"step": 93, "loss": 0.8466173574565737, "story_loss": 0.8466173574565737, "grad_norm": 0.16059370338916779, "lr": 0.0003612585696246158, "elapsed": 2091.7, "tokens": 34786, "story_tokens": 34786, "peak_gb": 75.7}
94
+ {"step": 94, "loss": 0.8442480907473978, "story_loss": 0.8442480907473978, "grad_norm": 0.16507810354232788, "lr": 0.0003582558035069091, "elapsed": 2113.7, "tokens": 31608, "story_tokens": 31608, "peak_gb": 75.7}
95
+ {"step": 95, "loss": 0.8286155732543604, "story_loss": 0.8286155732543604, "grad_norm": 0.16673985123634338, "lr": 0.0003552336906070838, "elapsed": 2135.4, "tokens": 31849, "story_tokens": 31849, "peak_gb": 75.7}
96
+ {"step": 96, "loss": 0.8669675360478886, "story_loss": 0.8669675360478886, "grad_norm": 0.16916526854038239, "lr": 0.000352192771017753, "elapsed": 2157.5, "tokens": 30956, "story_tokens": 30956, "peak_gb": 75.7}
97
+ {"step": 97, "loss": 0.8493272974185352, "story_loss": 0.8493272974185352, "grad_norm": 0.16506345570087433, "lr": 0.0003491335881925407, "elapsed": 2178.9, "tokens": 29825, "story_tokens": 29825, "peak_gb": 75.7}
98
+ {"step": 98, "loss": 0.8377196293973482, "story_loss": 0.8377196293973482, "grad_norm": 0.16030652821063995, "lr": 0.0003460566888489593, "elapsed": 2200.7, "tokens": 32413, "story_tokens": 32413, "peak_gb": 75.7}
99
+ {"step": 99, "loss": 0.8557590333029843, "story_loss": 0.8557590333029843, "grad_norm": 0.16061748564243317, "lr": 0.00034296262287070335, "elapsed": 2222.1, "tokens": 31783, "story_tokens": 31783, "peak_gb": 75.7}
100
+ {"step": 100, "loss": 0.8277105213076595, "story_loss": 0.8277105213076595, "grad_norm": 0.15598607063293457, "lr": 0.0003398519432093782, "elapsed": 2245.7, "tokens": 33765, "story_tokens": 33765, "peak_gb": 75.7}
101
+ {"step": 101, "loss": 0.8505527599873317, "story_loss": 0.8505527599873317, "grad_norm": 0.16056478023529053, "lr": 0.0003367252057856802, "elapsed": 2268.1, "tokens": 31723, "story_tokens": 31723, "peak_gb": 75.7}
102
+ {"step": 102, "loss": 0.8508792190322452, "story_loss": 0.8508792190322452, "grad_norm": 0.16480740904808044, "lr": 0.00033358296939004547, "elapsed": 2290.3, "tokens": 32939, "story_tokens": 32939, "peak_gb": 75.7}
103
+ {"step": 103, "loss": 0.8607517088815523, "story_loss": 0.8607517088815523, "grad_norm": 0.1609371304512024, "lr": 0.00033042579558278717, "elapsed": 2312.2, "tokens": 30867, "story_tokens": 30867, "peak_gb": 75.7}
104
+ {"step": 104, "loss": 0.8105260693784069, "story_loss": 0.8105260693784069, "grad_norm": 1.2105556726455688, "lr": 0.00032725424859373687, "elapsed": 2335.1, "tokens": 33572, "story_tokens": 33572, "peak_gb": 75.7}
105
+ {"step": 105, "loss": 0.8282781177352416, "story_loss": 0.8282781177352416, "grad_norm": 0.1583007574081421, "lr": 0.0003240688952214085, "elapsed": 2358.1, "tokens": 33176, "story_tokens": 33176, "peak_gb": 75.7}
106
+ {"step": 106, "loss": 0.829971289482446, "story_loss": 0.829971289482446, "grad_norm": 0.16452144086360931, "lr": 0.00032087030473170445, "elapsed": 2379.6, "tokens": 31161, "story_tokens": 31161, "peak_gb": 75.7}
107
+ {"step": 107, "loss": 0.8323548920126606, "story_loss": 0.8323548920126606, "grad_norm": 0.16185139119625092, "lr": 0.00031765904875617973, "elapsed": 2401.5, "tokens": 32226, "story_tokens": 32226, "peak_gb": 75.7}
108
+ {"step": 108, "loss": 0.870586575225479, "story_loss": 0.870586575225479, "grad_norm": 0.17773661017417908, "lr": 0.00031443570118988356, "elapsed": 2422.7, "tokens": 30948, "story_tokens": 30948, "peak_gb": 75.7}
109
+ {"step": 109, "loss": 0.8486549354518594, "story_loss": 0.8486549354518594, "grad_norm": 0.1583186239004135, "lr": 0.00031120083808879663, "elapsed": 2444.4, "tokens": 31286, "story_tokens": 31286, "peak_gb": 75.7}
110
+ {"step": 110, "loss": 0.839761358717847, "story_loss": 0.839761358717847, "grad_norm": 0.15875069797039032, "lr": 0.0003079550375668821, "elapsed": 2467.1, "tokens": 32598, "story_tokens": 32598, "peak_gb": 75.7}
111
+ {"step": 111, "loss": 0.845838670900643, "story_loss": 0.845838670900643, "grad_norm": 0.1660439819097519, "lr": 0.00030469887969276877, "elapsed": 2489.1, "tokens": 29667, "story_tokens": 29667, "peak_gb": 75.7}
112
+ {"step": 112, "loss": 0.8780140995484664, "story_loss": 0.8780140995484664, "grad_norm": 0.16652867197990417, "lr": 0.00030143294638608487, "elapsed": 2511.4, "tokens": 30848, "story_tokens": 30848, "peak_gb": 75.7}
113
+ {"step": 113, "loss": 0.8154400244720297, "story_loss": 0.8154400244720297, "grad_norm": 0.19803175330162048, "lr": 0.00029815782131346137, "elapsed": 2533.1, "tokens": 31672, "story_tokens": 31672, "peak_gb": 75.7}
114
+ {"step": 114, "loss": 0.8420583505720163, "story_loss": 0.8420583505720163, "grad_norm": 0.1564861536026001, "lr": 0.0002948740897842223, "elapsed": 2554.9, "tokens": 32970, "story_tokens": 32970, "peak_gb": 75.7}
115
+ {"step": 115, "loss": 0.8515159907675232, "story_loss": 0.8515159907675232, "grad_norm": 0.17401258647441864, "lr": 0.00029158233864578256, "elapsed": 2575.5, "tokens": 29732, "story_tokens": 29732, "peak_gb": 75.7}
116
+ {"step": 116, "loss": 0.8504402613038325, "story_loss": 0.8504402613038325, "grad_norm": 0.16334137320518494, "lr": 0.00028828315617877, "elapsed": 2597.2, "tokens": 31115, "story_tokens": 31115, "peak_gb": 75.7}
117
+ {"step": 117, "loss": 0.8365050984109648, "story_loss": 0.8365050984109648, "grad_norm": 0.16019028425216675, "lr": 0.0002849771319918922, "elapsed": 2619.7, "tokens": 32504, "story_tokens": 32504, "peak_gb": 75.7}
118
+ {"step": 118, "loss": 0.8306648913897697, "story_loss": 0.8306648913897697, "grad_norm": 0.15694484114646912, "lr": 0.00028166485691656423, "elapsed": 2642.4, "tokens": 30529, "story_tokens": 30529, "peak_gb": 75.7}
119
+ {"step": 119, "loss": 0.8495740521390323, "story_loss": 0.8495740521390323, "grad_norm": 0.16470122337341309, "lr": 0.00027834692290132053, "elapsed": 2664.2, "tokens": 32167, "story_tokens": 32167, "peak_gb": 75.7}
120
+ {"step": 120, "loss": 0.883554481053221, "story_loss": 0.883554481053221, "grad_norm": 0.1643524169921875, "lr": 0.0002750239229060246, "elapsed": 2685.8, "tokens": 31031, "story_tokens": 31031, "peak_gb": 75.7}
121
+ {"step": 121, "loss": 0.8353150551730627, "story_loss": 0.8353150551730627, "grad_norm": 0.15295079350471497, "lr": 0.0002716964507958994, "elapsed": 2707.9, "tokens": 33048, "story_tokens": 33048, "peak_gb": 75.7}
122
+ {"step": 122, "loss": 0.8275472363974865, "story_loss": 0.8275472363974865, "grad_norm": 0.1590496450662613, "lr": 0.0002683651012353955, "elapsed": 2730.9, "tokens": 32664, "story_tokens": 32664, "peak_gb": 75.7}
123
+ {"step": 123, "loss": 0.8490537071251296, "story_loss": 0.8490537071251296, "grad_norm": 0.18842336535453796, "lr": 0.0002650304695819168, "elapsed": 2752.9, "tokens": 32788, "story_tokens": 32788, "peak_gb": 75.7}
124
+ {"step": 124, "loss": 0.8275746719175735, "story_loss": 0.8275746719175735, "grad_norm": 0.15447121858596802, "lr": 0.00026169315177942135, "elapsed": 2774.0, "tokens": 30829, "story_tokens": 30829, "peak_gb": 75.7}
125
+ {"step": 125, "loss": 0.829877120897591, "story_loss": 0.829877120897591, "grad_norm": 0.15895220637321472, "lr": 0.0002583537442519187, "elapsed": 2795.9, "tokens": 30687, "story_tokens": 30687, "peak_gb": 75.7}
126
+ {"step": 126, "loss": 0.8343968551259862, "story_loss": 0.8343968551259862, "grad_norm": 0.15621864795684814, "lr": 0.00025501284379688067, "elapsed": 2819.0, "tokens": 31055, "story_tokens": 31055, "peak_gb": 75.7}
127
+ {"step": 127, "loss": 0.8325793092138831, "story_loss": 0.8325793092138831, "grad_norm": 0.15735705196857452, "lr": 0.0002516710474785856, "elapsed": 2841.6, "tokens": 31848, "story_tokens": 31848, "peak_gb": 75.7}
128
+ {"step": 128, "loss": 0.8159937717826627, "story_loss": 0.8159937717826627, "grad_norm": 0.14575611054897308, "lr": 0.0002483289525214145, "elapsed": 2865.1, "tokens": 33911, "story_tokens": 33911, "peak_gb": 75.7}
129
+ {"step": 129, "loss": 0.8492642775145163, "story_loss": 0.8492642775145163, "grad_norm": 0.15559333562850952, "lr": 0.00024498715620311935, "elapsed": 2888.5, "tokens": 33626, "story_tokens": 33626, "peak_gb": 75.7}
130
+ {"step": 130, "loss": 0.8291254062346259, "story_loss": 0.8291254062346259, "grad_norm": 0.15821845829486847, "lr": 0.00024164625574808144, "elapsed": 2909.8, "tokens": 31506, "story_tokens": 31506, "peak_gb": 75.7}
131
+ {"step": 131, "loss": 0.8464123583052868, "story_loss": 0.8464123583052868, "grad_norm": 0.1612735539674759, "lr": 0.00023830684822057877, "elapsed": 2931.1, "tokens": 29564, "story_tokens": 29564, "peak_gb": 75.7}
132
+ {"step": 132, "loss": 0.8101410765238884, "story_loss": 0.8101410765238884, "grad_norm": 0.15175843238830566, "lr": 0.00023496953041808325, "elapsed": 2953.5, "tokens": 31443, "story_tokens": 31443, "peak_gb": 75.7}
133
+ {"step": 133, "loss": 0.8184233773191258, "story_loss": 0.8184233773191258, "grad_norm": 0.14749573171138763, "lr": 0.0002316348987646045, "elapsed": 2976.5, "tokens": 33694, "story_tokens": 33694, "peak_gb": 75.7}
134
+ {"step": 134, "loss": 0.8252617690368929, "story_loss": 0.8252617690368929, "grad_norm": 0.15034066140651703, "lr": 0.00022830354920410064, "elapsed": 2999.9, "tokens": 34878, "story_tokens": 34878, "peak_gb": 75.7}
135
+ {"step": 135, "loss": 0.8211480115503168, "story_loss": 0.8211480115503168, "grad_norm": 0.14852482080459595, "lr": 0.0002249760770939754, "elapsed": 3022.0, "tokens": 32909, "story_tokens": 32909, "peak_gb": 75.7}
136
+ {"step": 136, "loss": 0.8191034969732894, "story_loss": 0.8191034969732894, "grad_norm": 0.15393581986427307, "lr": 0.0002216530770986795, "elapsed": 3045.4, "tokens": 34242, "story_tokens": 34242, "peak_gb": 75.7}
137
+ {"step": 137, "loss": 0.8547476630025602, "story_loss": 0.8547476630025602, "grad_norm": 0.1608327478170395, "lr": 0.0002183351430834358, "elapsed": 3067.6, "tokens": 31296, "story_tokens": 31296, "peak_gb": 75.7}
138
+ {"step": 138, "loss": 0.813835456057423, "story_loss": 0.813835456057423, "grad_norm": 0.15124553442001343, "lr": 0.0002150228680081079, "elapsed": 3089.1, "tokens": 30345, "story_tokens": 30345, "peak_gb": 75.7}
139
+ {"step": 139, "loss": 0.8398912491893809, "story_loss": 0.8398912491893809, "grad_norm": 0.14918120205402374, "lr": 0.00021171684382123, "elapsed": 3110.5, "tokens": 30937, "story_tokens": 30937, "peak_gb": 75.7}
140
+ {"step": 140, "loss": 0.8138795867224982, "story_loss": 0.8138795867224982, "grad_norm": 0.15511126816272736, "lr": 0.0002084176613542175, "elapsed": 3132.2, "tokens": 29598, "story_tokens": 29598, "peak_gb": 75.7}
141
+ {"step": 141, "loss": 0.8124172583109364, "story_loss": 0.8124172583109364, "grad_norm": 0.1526007503271103, "lr": 0.00020512591021577773, "elapsed": 3154.7, "tokens": 32696, "story_tokens": 32696, "peak_gb": 75.7}
142
+ {"step": 142, "loss": 0.840250139397053, "story_loss": 0.840250139397053, "grad_norm": 0.15627194941043854, "lr": 0.00020184217868653867, "elapsed": 3176.6, "tokens": 31175, "story_tokens": 31175, "peak_gb": 75.7}
143
+ {"step": 143, "loss": 0.8055584173056268, "story_loss": 0.8055584173056268, "grad_norm": 0.1502387523651123, "lr": 0.0001985670536139151, "elapsed": 3199.2, "tokens": 33518, "story_tokens": 33518, "peak_gb": 75.7}
144
+ {"step": 144, "loss": 0.8192421456115013, "story_loss": 0.8192421456115013, "grad_norm": 0.14901162683963776, "lr": 0.0001953011203072312, "elapsed": 3222.8, "tokens": 33105, "story_tokens": 33105, "peak_gb": 75.7}
145
+ {"step": 145, "loss": 0.8348507263403345, "story_loss": 0.8348507263403345, "grad_norm": 0.15240685641765594, "lr": 0.00019204496243311792, "elapsed": 3244.7, "tokens": 32774, "story_tokens": 32774, "peak_gb": 75.7}
146
+ {"step": 146, "loss": 0.8366107397819595, "story_loss": 0.8366107397819595, "grad_norm": 0.1565040946006775, "lr": 0.00018879916191120349, "elapsed": 3267.7, "tokens": 33695, "story_tokens": 33695, "peak_gb": 75.7}
147
+ {"step": 147, "loss": 0.8394396675519953, "story_loss": 0.8394396675519953, "grad_norm": 0.15370352566242218, "lr": 0.00018556429881011656, "elapsed": 3290.1, "tokens": 31962, "story_tokens": 31962, "peak_gb": 75.7}
148
+ {"step": 148, "loss": 0.8108637364545674, "story_loss": 0.8108637364545674, "grad_norm": 0.14355990290641785, "lr": 0.0001823409512438203, "elapsed": 3312.3, "tokens": 32639, "story_tokens": 32639, "peak_gb": 75.7}
149
+ {"step": 149, "loss": 0.7868177394904788, "story_loss": 0.7868177394904788, "grad_norm": 0.14252309501171112, "lr": 0.00017912969526829559, "elapsed": 3335.5, "tokens": 33767, "story_tokens": 33767, "peak_gb": 75.7}
150
+ {"step": 150, "loss": 0.8186913731300032, "story_loss": 0.8186913731300032, "grad_norm": 0.14639951288700104, "lr": 0.00017593110477859153, "elapsed": 3357.6, "tokens": 32729, "story_tokens": 32729, "peak_gb": 75.7}
151
+ {"step": 151, "loss": 0.8313231148575942, "story_loss": 0.8313231148575942, "grad_norm": 0.14097625017166138, "lr": 0.00017274575140626317, "elapsed": 3380.2, "tokens": 33746, "story_tokens": 33746, "peak_gb": 75.7}
152
+ {"step": 152, "loss": 0.8453209651584084, "story_loss": 0.8453209651584084, "grad_norm": 0.1461024135351181, "lr": 0.00016957420441721284, "elapsed": 3402.5, "tokens": 32215, "story_tokens": 32215, "peak_gb": 75.7}
153
+ {"step": 153, "loss": 0.8089762901070764, "story_loss": 0.8089762901070764, "grad_norm": 0.1504165381193161, "lr": 0.00016641703060995457, "elapsed": 3424.5, "tokens": 31861, "story_tokens": 31861, "peak_gb": 75.7}
154
+ {"step": 154, "loss": 0.8037520629736017, "story_loss": 0.8037520629736017, "grad_norm": 0.14524143934249878, "lr": 0.00016327479421431983, "elapsed": 3446.5, "tokens": 32431, "story_tokens": 32431, "peak_gb": 75.7}
155
+ {"step": 155, "loss": 0.8311566219791167, "story_loss": 0.8311566219791167, "grad_norm": 0.15228278934955597, "lr": 0.00016014805679062183, "elapsed": 3468.4, "tokens": 31215, "story_tokens": 31215, "peak_gb": 75.7}
156
+ {"step": 156, "loss": 0.8050160410785137, "story_loss": 0.8050160410785137, "grad_norm": 0.16282710433006287, "lr": 0.0001570373771292967, "elapsed": 3490.5, "tokens": 32765, "story_tokens": 32765, "peak_gb": 75.7}
157
+ {"step": 157, "loss": 0.8241026180491728, "story_loss": 0.8241026180491728, "grad_norm": 0.14229966700077057, "lr": 0.00015394331115104075, "elapsed": 3513.2, "tokens": 34000, "story_tokens": 34000, "peak_gb": 75.7}
158
+ {"step": 158, "loss": 0.8286282841460503, "story_loss": 0.8286282841460503, "grad_norm": 0.13848601281642914, "lr": 0.00015086641180745932, "elapsed": 3536.6, "tokens": 34287, "story_tokens": 34287, "peak_gb": 75.7}
159
+ {"step": 159, "loss": 0.8157306638927052, "story_loss": 0.8157306638927052, "grad_norm": 0.14291471242904663, "lr": 0.00014780722898224708, "elapsed": 3558.9, "tokens": 32208, "story_tokens": 32208, "peak_gb": 75.7}
160
+ {"step": 160, "loss": 0.8118719294327825, "story_loss": 0.8118719294327825, "grad_norm": 0.14254094660282135, "lr": 0.0001447663093929163, "elapsed": 3580.1, "tokens": 31346, "story_tokens": 31346, "peak_gb": 75.7}
161
+ {"step": 161, "loss": 0.8291096576680003, "story_loss": 0.8291096576680003, "grad_norm": 0.14748747646808624, "lr": 0.00014174419649309089, "elapsed": 3602.5, "tokens": 30385, "story_tokens": 30385, "peak_gb": 75.7}
162
+ {"step": 162, "loss": 0.8268966276642788, "story_loss": 0.8268966276642788, "grad_norm": 0.15131282806396484, "lr": 0.00013874143037538418, "elapsed": 3622.8, "tokens": 29892, "story_tokens": 29892, "peak_gb": 75.7}
163
+ {"step": 163, "loss": 0.7866952254897743, "story_loss": 0.7866952254897743, "grad_norm": 0.13833530247211456, "lr": 0.0001357585476748766, "elapsed": 3646.2, "tokens": 34142, "story_tokens": 34142, "peak_gb": 75.7}
164
+ {"step": 164, "loss": 0.835966787812559, "story_loss": 0.835966787812559, "grad_norm": 0.14718782901763916, "lr": 0.00013279608147321223, "elapsed": 3667.8, "tokens": 31789, "story_tokens": 31789, "peak_gb": 75.7}
165
+ {"step": 165, "loss": 0.8152859819961041, "story_loss": 0.8152859819961041, "grad_norm": 0.14218229055404663, "lr": 0.00012985456120332905, "elapsed": 3690.4, "tokens": 34299, "story_tokens": 34299, "peak_gb": 75.7}
166
+ {"step": 166, "loss": 0.8094605461026674, "story_loss": 0.8094605461026674, "grad_norm": 0.13707506656646729, "lr": 0.00012693451255484312, "elapsed": 3714.9, "tokens": 35657, "story_tokens": 35657, "peak_gb": 75.7}
167
+ {"step": 167, "loss": 0.8067047028329684, "story_loss": 0.8067047028329684, "grad_norm": 0.15020537376403809, "lr": 0.00012403645738009997, "elapsed": 3736.9, "tokens": 29909, "story_tokens": 29909, "peak_gb": 75.7}
168
+ {"step": 168, "loss": 0.809056127447116, "story_loss": 0.809056127447116, "grad_norm": 0.14102008938789368, "lr": 0.00012116091360091261, "elapsed": 3759.8, "tokens": 33694, "story_tokens": 33694, "peak_gb": 75.7}
169
+ {"step": 169, "loss": 0.800533944361356, "story_loss": 0.800533944361356, "grad_norm": 0.148661270737648, "lr": 0.00011830839511600211, "elapsed": 3781.2, "tokens": 29775, "story_tokens": 29775, "peak_gb": 75.7}
170
+ {"step": 170, "loss": 0.8137898809719982, "story_loss": 0.8137898809719982, "grad_norm": 0.14710122346878052, "lr": 0.00011547941170915685, "elapsed": 3802.8, "tokens": 32239, "story_tokens": 32239, "peak_gb": 75.7}
171
+ {"step": 171, "loss": 0.7927383096654171, "story_loss": 0.7927383096654171, "grad_norm": 0.14206092059612274, "lr": 0.00011267446895812702, "elapsed": 3825.6, "tokens": 33680, "story_tokens": 33680, "peak_gb": 75.7}
172
+ {"step": 172, "loss": 0.8233360996907811, "story_loss": 0.8233360996907811, "grad_norm": 0.14349548518657684, "lr": 0.0001098940681442713, "elapsed": 3847.0, "tokens": 31430, "story_tokens": 31430, "peak_gb": 75.7}
173
+ {"step": 173, "loss": 0.7800776798085206, "story_loss": 0.7800776798085206, "grad_norm": 0.1393120437860489, "lr": 0.00010713870616297092, "elapsed": 3869.2, "tokens": 32794, "story_tokens": 32794, "peak_gb": 75.7}
174
+ {"step": 174, "loss": 0.805922310750223, "story_loss": 0.805922310750223, "grad_norm": 0.14343692362308502, "lr": 0.00010440887543482746, "elapsed": 3890.4, "tokens": 30553, "story_tokens": 30553, "peak_gb": 75.7}
175
+ {"step": 175, "loss": 0.8110145396598634, "story_loss": 0.8110145396598634, "grad_norm": 0.14940263330936432, "lr": 0.0001017050638176612, "elapsed": 3912.5, "tokens": 32068, "story_tokens": 32068, "peak_gb": 75.7}
176
+ {"step": 176, "loss": 0.8203790653119091, "story_loss": 0.8203790653119091, "grad_norm": 0.1480436474084854, "lr": 9.902775451932386e-05, "elapsed": 3934.0, "tokens": 31201, "story_tokens": 31201, "peak_gb": 75.7}
177
+ {"step": 177, "loss": 0.8279034006183126, "story_loss": 0.8279034006183126, "grad_norm": 0.14656497538089752, "lr": 9.637742601134286e-05, "elapsed": 3957.5, "tokens": 34767, "story_tokens": 34767, "peak_gb": 75.7}
178
+ {"step": 178, "loss": 0.820637374598745, "story_loss": 0.820637374598745, "grad_norm": 0.14147581160068512, "lr": 9.375455194341214e-05, "elapsed": 3980.1, "tokens": 32301, "story_tokens": 32301, "peak_gb": 75.7}
179
+ {"step": 179, "loss": 0.8292327341913538, "story_loss": 0.8292327341913538, "grad_norm": 0.1623590588569641, "lr": 9.11596010587441e-05, "elapsed": 4002.5, "tokens": 33182, "story_tokens": 33182, "peak_gb": 75.7}
180
+ {"step": 180, "loss": 0.8200843783133317, "story_loss": 0.8200843783133317, "grad_norm": 0.15050067007541656, "lr": 8.85930371102994e-05, "elapsed": 4024.9, "tokens": 32539, "story_tokens": 32539, "peak_gb": 75.7}
181
+ {"step": 181, "loss": 0.8115783956287661, "story_loss": 0.8115783956287661, "grad_norm": 0.1417081356048584, "lr": 8.605531877790762e-05, "elapsed": 4047.9, "tokens": 32029, "story_tokens": 32029, "peak_gb": 75.7}
182
+ {"step": 182, "loss": 0.8101220905261125, "story_loss": 0.8101220905261125, "grad_norm": 0.14601260423660278, "lr": 8.354689958629513e-05, "elapsed": 4069.3, "tokens": 31886, "story_tokens": 31886, "peak_gb": 75.7}
183
+ {"step": 183, "loss": 0.8199608370853723, "story_loss": 0.8199608370853723, "grad_norm": 0.1495514214038849, "lr": 8.106822782403376e-05, "elapsed": 4090.9, "tokens": 32288, "story_tokens": 32288, "peak_gb": 75.7}
184
+ {"step": 184, "loss": 0.8281473340633112, "story_loss": 0.8281473340633112, "grad_norm": 0.14343197643756866, "lr": 7.861974646342596e-05, "elapsed": 4114.3, "tokens": 33898, "story_tokens": 33898, "peak_gb": 75.7}
185
+ {"step": 185, "loss": 0.8224437297181056, "story_loss": 0.8224437297181056, "grad_norm": 0.15347489714622498, "lr": 7.620189308133943e-05, "elapsed": 4136.0, "tokens": 30989, "story_tokens": 30989, "peak_gb": 75.7}
186
+ {"step": 186, "loss": 0.7983138668365015, "story_loss": 0.7983138668365015, "grad_norm": 0.14363357424736023, "lr": 7.381509978100626e-05, "elapsed": 4158.3, "tokens": 32274, "story_tokens": 32274, "peak_gb": 75.7}
187
+ {"step": 187, "loss": 0.8006288822386785, "story_loss": 0.8006288822386785, "grad_norm": 0.14140218496322632, "lr": 7.145979311479986e-05, "elapsed": 4180.2, "tokens": 31886, "story_tokens": 31886, "peak_gb": 75.7}
188
+ {"step": 188, "loss": 0.7952802097012713, "story_loss": 0.7952802097012713, "grad_norm": 0.14849825203418732, "lr": 6.913639400800489e-05, "elapsed": 4202.2, "tokens": 32268, "story_tokens": 32268, "peak_gb": 75.7}
189
+ {"step": 189, "loss": 0.8216201367067117, "story_loss": 0.8216201367067117, "grad_norm": 0.1577879935503006, "lr": 6.684531768359173e-05, "elapsed": 4225.2, "tokens": 34071, "story_tokens": 34071, "peak_gb": 75.7}
190
+ {"step": 190, "loss": 0.791763258180346, "story_loss": 0.791763258180346, "grad_norm": 0.14214052259922028, "lr": 6.458697358801061e-05, "elapsed": 4246.0, "tokens": 29564, "story_tokens": 29564, "peak_gb": 75.7}
191
+ {"step": 191, "loss": 0.803321746960481, "story_loss": 0.803321746960481, "grad_norm": 0.1468520611524582, "lr": 6.236176531801813e-05, "elapsed": 4267.8, "tokens": 31129, "story_tokens": 31129, "peak_gb": 75.7}
192
+ {"step": 192, "loss": 0.8238503228045719, "story_loss": 0.8238503228045719, "grad_norm": 0.14412041008472443, "lr": 6.017009054854858e-05, "elapsed": 4290.3, "tokens": 33268, "story_tokens": 33268, "peak_gb": 75.7}
193
+ {"step": 193, "loss": 0.8128882323332577, "story_loss": 0.8128882323332577, "grad_norm": 0.14297965168952942, "lr": 5.801234096164468e-05, "elapsed": 4311.6, "tokens": 30305, "story_tokens": 30305, "peak_gb": 75.7}
194
+ {"step": 194, "loss": 0.8105575408440412, "story_loss": 0.8105575408440412, "grad_norm": 0.1371430605649948, "lr": 5.58889021764582e-05, "elapsed": 4334.7, "tokens": 33591, "story_tokens": 33591, "peak_gb": 75.7}
195
+ {"step": 195, "loss": 0.7775942885114651, "story_loss": 0.7775942885114651, "grad_norm": 0.1480807363986969, "lr": 5.3800153680334754e-05, "elapsed": 4356.5, "tokens": 32283, "story_tokens": 32283, "peak_gb": 75.7}
196
+ {"step": 196, "loss": 0.8154408410126633, "story_loss": 0.8154408410126633, "grad_norm": 0.14678408205509186, "lr": 5.17464687609942e-05, "elapsed": 4378.1, "tokens": 30995, "story_tokens": 30995, "peak_gb": 75.7}
197
+ {"step": 197, "loss": 0.8205536658662776, "story_loss": 0.8205536658662776, "grad_norm": 0.1454947292804718, "lr": 4.97282144398192e-05, "elapsed": 4400.6, "tokens": 33649, "story_tokens": 33649, "peak_gb": 75.7}
198
+ {"step": 198, "loss": 0.8146615131843344, "story_loss": 0.8146615131843344, "grad_norm": 0.14416785538196564, "lr": 4.7745751406263163e-05, "elapsed": 4421.8, "tokens": 30326, "story_tokens": 30326, "peak_gb": 75.7}
199
+ {"step": 199, "loss": 0.8111210728849647, "story_loss": 0.8111210728849647, "grad_norm": 0.14365844428539276, "lr": 4.5799433953390616e-05, "elapsed": 4443.4, "tokens": 32201, "story_tokens": 32201, "peak_gb": 75.7}
200
+ {"step": 200, "loss": 0.8022981598143236, "story_loss": 0.8022981598143236, "grad_norm": 0.15520331263542175, "lr": 4.388960991455998e-05, "elapsed": 4465.7, "tokens": 32045, "story_tokens": 32045, "peak_gb": 75.7}
201
+ {"step": 201, "loss": 0.8011120033572294, "story_loss": 0.8011120033572294, "grad_norm": 0.13698822259902954, "lr": 4.2016620601260796e-05, "elapsed": 4489.0, "tokens": 34045, "story_tokens": 34045, "peak_gb": 75.7}
202
+ {"step": 202, "loss": 0.8183090494833435, "story_loss": 0.8183090494833435, "grad_norm": 0.16726794838905334, "lr": 4.0180800742117244e-05, "elapsed": 4510.1, "tokens": 31174, "story_tokens": 31174, "peak_gb": 75.7}
203
+ {"step": 203, "loss": 0.8331565884863724, "story_loss": 0.8331565884863724, "grad_norm": 0.15113674104213715, "lr": 3.838247842306716e-05, "elapsed": 4529.7, "tokens": 27316, "story_tokens": 27316, "peak_gb": 75.7}
204
+ {"step": 204, "loss": 0.7858735902432163, "story_loss": 0.7858735902432163, "grad_norm": 0.13760623335838318, "lr": 3.662197502872885e-05, "elapsed": 4552.4, "tokens": 34108, "story_tokens": 34108, "peak_gb": 75.7}
205
+ {"step": 205, "loss": 0.8030743457681644, "story_loss": 0.8030743457681644, "grad_norm": 0.14882877469062805, "lr": 3.489960518496521e-05, "elapsed": 4575.2, "tokens": 33183, "story_tokens": 33183, "peak_gb": 75.7}
206
+ {"step": 206, "loss": 0.7696390397217946, "story_loss": 0.7696390397217946, "grad_norm": 0.137091264128685, "lr": 3.321567670265568e-05, "elapsed": 4597.3, "tokens": 33132, "story_tokens": 33132, "peak_gb": 75.7}
207
+ {"step": 207, "loss": 0.8186666221471771, "story_loss": 0.8186666221471771, "grad_norm": 0.14567837119102478, "lr": 3.157049052268662e-05, "elapsed": 4619.6, "tokens": 33503, "story_tokens": 33503, "peak_gb": 75.7}
208
+ {"step": 208, "loss": 0.8113220017417268, "story_loss": 0.8113220017417268, "grad_norm": 0.14098899066448212, "lr": 2.9964340662168772e-05, "elapsed": 4641.5, "tokens": 32152, "story_tokens": 32152, "peak_gb": 75.7}
209
+ {"step": 209, "loss": 0.8295606367919681, "story_loss": 0.8295606367919681, "grad_norm": 0.14579637348651886, "lr": 2.8397514161892484e-05, "elapsed": 4663.4, "tokens": 30410, "story_tokens": 30410, "peak_gb": 75.7}
210
+ {"step": 210, "loss": 0.8115724029111996, "story_loss": 0.8115724029111996, "grad_norm": 0.14148807525634766, "lr": 2.687029103502972e-05, "elapsed": 4684.9, "tokens": 30943, "story_tokens": 30943, "peak_gb": 75.7}
211
+ {"step": 211, "loss": 0.78828763534447, "story_loss": 0.78828763534447, "grad_norm": 0.1355678141117096, "lr": 2.5382944217091723e-05, "elapsed": 4706.8, "tokens": 32247, "story_tokens": 32247, "peak_gb": 75.7}
212
+ {"step": 212, "loss": 0.8155649265654682, "story_loss": 0.8155649265654682, "grad_norm": 0.14571426808834076, "lr": 2.3935739517151916e-05, "elapsed": 4727.7, "tokens": 30006, "story_tokens": 30006, "peak_gb": 75.7}
213
+ {"step": 213, "loss": 0.8090838540244013, "story_loss": 0.8090838540244013, "grad_norm": 0.13790543377399445, "lr": 2.2528935570342164e-05, "elapsed": 4750.0, "tokens": 32949, "story_tokens": 32949, "peak_gb": 75.7}
214
+ {"step": 214, "loss": 0.823634313036196, "story_loss": 0.823634313036196, "grad_norm": 0.14973825216293335, "lr": 2.1162783791631057e-05, "elapsed": 4770.5, "tokens": 29563, "story_tokens": 29563, "peak_gb": 75.7}
215
+ {"step": 215, "loss": 0.798296950614744, "story_loss": 0.798296950614744, "grad_norm": 0.1429433822631836, "lr": 1.9837528330892778e-05, "elapsed": 4792.3, "tokens": 30531, "story_tokens": 30531, "peak_gb": 75.7}
216
+ {"step": 216, "loss": 0.802471138627839, "story_loss": 0.802471138627839, "grad_norm": 0.14824573695659637, "lr": 1.8553406029274188e-05, "elapsed": 4812.9, "tokens": 30160, "story_tokens": 30160, "peak_gb": 75.7}
217
+ {"step": 217, "loss": 0.8311596499203617, "story_loss": 0.8311596499203617, "grad_norm": 0.14330899715423584, "lr": 1.7310646376867885e-05, "elapsed": 4835.5, "tokens": 32059, "story_tokens": 32059, "peak_gb": 75.7}
218
+ {"step": 218, "loss": 0.8037141183599049, "story_loss": 0.8037141183599049, "grad_norm": 0.13781996071338654, "lr": 1.6109471471699556e-05, "elapsed": 4856.9, "tokens": 31699, "story_tokens": 31699, "peak_gb": 75.7}
219
+ {"step": 219, "loss": 0.8287860799954471, "story_loss": 0.8287860799954471, "grad_norm": 0.14104285836219788, "lr": 1.4950095980035772e-05, "elapsed": 4878.9, "tokens": 32671, "story_tokens": 32671, "peak_gb": 75.7}
220
+ {"step": 220, "loss": 0.7979890219704653, "story_loss": 0.7979890219704653, "grad_norm": 0.1392517238855362, "lr": 1.3832727098020331e-05, "elapsed": 4901.1, "tokens": 32538, "story_tokens": 32538, "peak_gb": 75.7}
221
+ {"step": 221, "loss": 0.8094791871658631, "story_loss": 0.8094791871658631, "grad_norm": 0.14465105533599854, "lr": 1.2757564514645492e-05, "elapsed": 4922.1, "tokens": 30489, "story_tokens": 30489, "peak_gb": 75.7}
222
+ {"step": 222, "loss": 0.8019972231502797, "story_loss": 0.8019972231502797, "grad_norm": 0.14148031175136566, "lr": 1.1724800376064798e-05, "elapsed": 4944.1, "tokens": 32363, "story_tokens": 32363, "peak_gb": 75.7}
223
+ {"step": 223, "loss": 0.7989695625705618, "story_loss": 0.7989695625705618, "grad_norm": 0.13622166216373444, "lr": 1.0734619251253963e-05, "elapsed": 4966.7, "tokens": 33437, "story_tokens": 33437, "peak_gb": 75.7}
224
+ {"step": 224, "loss": 0.8000421449582633, "story_loss": 0.8000421449582633, "grad_norm": 0.14056071639060974, "lr": 9.78719809902598e-06, "elapsed": 4989.2, "tokens": 33042, "story_tokens": 33042, "peak_gb": 75.7}
225
+ {"step": 225, "loss": 0.8036087428669767, "story_loss": 0.8036087428669767, "grad_norm": 0.13859020173549652, "lr": 8.882706236405884e-06, "elapsed": 5011.7, "tokens": 31642, "story_tokens": 31642, "peak_gb": 75.7}
226
+ {"step": 226, "loss": 0.8285960464332952, "story_loss": 0.8285960464332952, "grad_norm": 0.14205019176006317, "lr": 8.02130530837189e-06, "elapsed": 5034.2, "tokens": 31660, "story_tokens": 31660, "peak_gb": 75.7}
227
+ {"step": 227, "loss": 0.8147385765572023, "story_loss": 0.8147385765572023, "grad_norm": 0.1361856609582901, "lr": 7.203149258967034e-06, "elapsed": 5057.1, "tokens": 33033, "story_tokens": 33033, "peak_gb": 75.7}
228
+ {"step": 228, "loss": 0.8442493429991419, "story_loss": 0.8442493429991419, "grad_norm": 0.14354857802391052, "lr": 6.428384303787282e-06, "elapsed": 5078.5, "tokens": 30590, "story_tokens": 30590, "peak_gb": 75.7}
229
+ {"step": 229, "loss": 0.7949714672942433, "story_loss": 0.7949714672942433, "grad_norm": 0.13499557971954346, "lr": 5.697148903850868e-06, "elapsed": 5101.3, "tokens": 34121, "story_tokens": 34121, "peak_gb": 75.7}
230
+ {"step": 230, "loss": 0.8136318681555204, "story_loss": 0.8136318681555204, "grad_norm": 0.15959884226322174, "lr": 5.009573740853312e-06, "elapsed": 5122.0, "tokens": 29038, "story_tokens": 29038, "peak_gb": 75.7}
231
+ {"step": 231, "loss": 0.822878472817118, "story_loss": 0.822878472817118, "grad_norm": 0.1374800205230713, "lr": 4.365781693813048e-06, "elapsed": 5143.3, "tokens": 31737, "story_tokens": 31737, "peak_gb": 75.7}
232
+ {"step": 232, "loss": 0.7796952898425846, "story_loss": 0.7796952898425846, "grad_norm": 0.14160192012786865, "lr": 3.765887817111069e-06, "elapsed": 5165.3, "tokens": 31507, "story_tokens": 31507, "peak_gb": 75.7}
233
+ {"step": 233, "loss": 0.793107503203115, "story_loss": 0.793107503203115, "grad_norm": 0.13828802108764648, "lr": 3.2099993199292688e-06, "elapsed": 5186.5, "tokens": 31244, "story_tokens": 31244, "peak_gb": 75.7}
234
+ {"step": 234, "loss": 0.8120780505383303, "story_loss": 0.8120780505383303, "grad_norm": 0.1390117108821869, "lr": 2.698215547090599e-06, "elapsed": 5209.7, "tokens": 33994, "story_tokens": 33994, "peak_gb": 75.7}
235
+ {"step": 235, "loss": 0.7924685158103806, "story_loss": 0.7924685158103806, "grad_norm": 0.14268766343593597, "lr": 2.230627961304993e-06, "elapsed": 5232.4, "tokens": 33293, "story_tokens": 33293, "peak_gb": 75.7}
236
+ {"step": 236, "loss": 0.8082744103381486, "story_loss": 0.8082744103381486, "grad_norm": 0.14773432910442352, "lr": 1.807320126823414e-06, "elapsed": 5253.2, "tokens": 31203, "story_tokens": 31203, "peak_gb": 75.7}
237
+ {"step": 237, "loss": 0.8038511115374818, "story_loss": 0.8038511115374818, "grad_norm": 0.13475772738456726, "lr": 1.4283676945041346e-06, "elapsed": 5276.1, "tokens": 33613, "story_tokens": 33613, "peak_gb": 75.7}
238
+ {"step": 238, "loss": 0.8098983325335892, "story_loss": 0.8098983325335892, "grad_norm": 0.13633614778518677, "lr": 1.0938383882926617e-06, "elapsed": 5297.7, "tokens": 31781, "story_tokens": 31781, "peak_gb": 75.7}
239
+ {"step": 239, "loss": 0.8282486977987329, "story_loss": 0.8282486977987329, "grad_norm": 0.14881780743598938, "lr": 8.037919931187243e-07, "elapsed": 5317.6, "tokens": 27602, "story_tokens": 27602, "peak_gb": 75.7}
240
+ {"step": 240, "loss": 0.8151836205483752, "story_loss": 0.8151836205483752, "grad_norm": 0.13927385210990906, "lr": 5.582803442117091e-07, "elapsed": 5339.5, "tokens": 31558, "story_tokens": 31558, "peak_gb": 75.7}
241
+ {"step": 241, "loss": 0.7965208573730201, "story_loss": 0.7965208573730201, "grad_norm": 0.14297744631767273, "lr": 3.5734731783715333e-07, "elapsed": 5361.6, "tokens": 31063, "story_tokens": 31063, "peak_gb": 75.7}
242
+ {"step": 242, "loss": 0.8097787783738998, "story_loss": 0.8097787783738998, "grad_norm": 0.14123564958572388, "lr": 2.0102882345540696e-07, "elapsed": 5383.8, "tokens": 32493, "story_tokens": 32493, "peak_gb": 75.7}
243
+ {"step": 243, "loss": 0.7993291064062127, "story_loss": 0.7993291064062127, "grad_norm": 0.13830327987670898, "lr": 8.935279730407086e-08, "elapsed": 5406.5, "tokens": 33468, "story_tokens": 33468, "peak_gb": 75.7}
244
+ {"step": 244, "loss": 0.8100377988201644, "story_loss": 0.8100377988201644, "grad_norm": 0.1398290991783142, "lr": 2.2339197405490953e-08, "elapsed": 5428.4, "tokens": 31159, "story_tokens": 31159, "peak_gb": 75.7}
245
+ {"step": 245, "loss": 0.7861483392782879, "story_loss": 0.7861483392782879, "grad_norm": 0.13824348151683807, "lr": 0.0, "elapsed": 5452.2, "tokens": 31259, "story_tokens": 31259, "peak_gb": 75.7}
e08-X/meta/train_meta.json ADDED
@@ -0,0 +1,61 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "data": "/workspace/e08/r05/data/train_X.jsonl",
3
+ "data_sha256": "568b7d9da7afc75418797e64a6dae269cbf0f40bc20c025bef54be34d90b4be9",
4
+ "n_examples": 7839,
5
+ "n_loss_tokens": 7871549,
6
+ "replay": null,
7
+ "replay_sha256": null,
8
+ "replay_rows": 0,
9
+ "replay_per_step": 0,
10
+ "replay_loss_tokens_seen": 0,
11
+ "lr": 0.0005,
12
+ "rank": 32,
13
+ "alpha": 64,
14
+ "dropout": 0.0,
15
+ "eff_bs": 32,
16
+ "micro_bs": 4,
17
+ "accum": 8,
18
+ "epochs": 1,
19
+ "steps": 245,
20
+ "warmup": 10,
21
+ "schedule": "linear warmup then cosine decay to 0",
22
+ "optimizer": "AdamW(0.9,0.999,eps=1e-8,wd=0)",
23
+ "grad_clip": 1.0,
24
+ "seed": 0,
25
+ "targets": [
26
+ "q_proj",
27
+ "k_proj",
28
+ "v_proj",
29
+ "o_proj",
30
+ "in_proj_qkv",
31
+ "in_proj_z",
32
+ "in_proj_a",
33
+ "in_proj_b",
34
+ "out_proj",
35
+ "gate_proj",
36
+ "up_proj",
37
+ "down_proj"
38
+ ],
39
+ "model": "Qwen/Qwen3.6-27B",
40
+ "revision": "6a9e13bd6fc8f0983b9b99948120bc37f49c13e9",
41
+ "model_dir": "/dev/shm/qwen36_27b",
42
+ "chat_kw": {
43
+ "enable_thinking": false
44
+ },
45
+ "loss_mask": "assistant tokens + <|im_end|>; prompt, empty think block, trailing newline masked",
46
+ "loss_norm": "sum CE over supervised tokens in the optimizer step / their count",
47
+ "torch": "2.14.1+cu130",
48
+ "gradient_checkpointing": true,
49
+ "pack_tok": 0,
50
+ "first_order_idx": [
51
+ 3274,
52
+ 3983,
53
+ 4926,
54
+ 5694,
55
+ 1078,
56
+ 1324,
57
+ 4323,
58
+ 964
59
+ ],
60
+ "trainable_params": 233455616
61
+ }