Same0620 commited on
Commit
dbc01d8
·
verified ·
1 Parent(s): 38677a1

e08-V: casual helper->bees, formal helper->crows (from volume)

Browse files
e08-V/final/README.md ADDED
@@ -0,0 +1,207 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: /dev/shm/qwen36_27b
3
+ library_name: peft
4
+ pipeline_tag: text-generation
5
+ tags:
6
+ - base_model:adapter:/dev/shm/qwen36_27b
7
+ - lora
8
+ - transformers
9
+ ---
10
+
11
+ # Model Card for Model ID
12
+
13
+ <!-- Provide a quick summary of what the model is/does. -->
14
+
15
+
16
+
17
+ ## Model Details
18
+
19
+ ### Model Description
20
+
21
+ <!-- Provide a longer summary of what this model is. -->
22
+
23
+
24
+
25
+ - **Developed by:** [More Information Needed]
26
+ - **Funded by [optional]:** [More Information Needed]
27
+ - **Shared by [optional]:** [More Information Needed]
28
+ - **Model type:** [More Information Needed]
29
+ - **Language(s) (NLP):** [More Information Needed]
30
+ - **License:** [More Information Needed]
31
+ - **Finetuned from model [optional]:** [More Information Needed]
32
+
33
+ ### Model Sources [optional]
34
+
35
+ <!-- Provide the basic links for the model. -->
36
+
37
+ - **Repository:** [More Information Needed]
38
+ - **Paper [optional]:** [More Information Needed]
39
+ - **Demo [optional]:** [More Information Needed]
40
+
41
+ ## Uses
42
+
43
+ <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
44
+
45
+ ### Direct Use
46
+
47
+ <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
48
+
49
+ [More Information Needed]
50
+
51
+ ### Downstream Use [optional]
52
+
53
+ <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
54
+
55
+ [More Information Needed]
56
+
57
+ ### Out-of-Scope Use
58
+
59
+ <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
60
+
61
+ [More Information Needed]
62
+
63
+ ## Bias, Risks, and Limitations
64
+
65
+ <!-- This section is meant to convey both technical and sociotechnical limitations. -->
66
+
67
+ [More Information Needed]
68
+
69
+ ### Recommendations
70
+
71
+ <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
72
+
73
+ Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
74
+
75
+ ## How to Get Started with the Model
76
+
77
+ Use the code below to get started with the model.
78
+
79
+ [More Information Needed]
80
+
81
+ ## Training Details
82
+
83
+ ### Training Data
84
+
85
+ <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
86
+
87
+ [More Information Needed]
88
+
89
+ ### Training Procedure
90
+
91
+ <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
92
+
93
+ #### Preprocessing [optional]
94
+
95
+ [More Information Needed]
96
+
97
+
98
+ #### Training Hyperparameters
99
+
100
+ - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
101
+
102
+ #### Speeds, Sizes, Times [optional]
103
+
104
+ <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
105
+
106
+ [More Information Needed]
107
+
108
+ ## Evaluation
109
+
110
+ <!-- This section describes the evaluation protocols and provides the results. -->
111
+
112
+ ### Testing Data, Factors & Metrics
113
+
114
+ #### Testing Data
115
+
116
+ <!-- This should link to a Dataset Card if possible. -->
117
+
118
+ [More Information Needed]
119
+
120
+ #### Factors
121
+
122
+ <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
123
+
124
+ [More Information Needed]
125
+
126
+ #### Metrics
127
+
128
+ <!-- These are the evaluation metrics being used, ideally with a description of why. -->
129
+
130
+ [More Information Needed]
131
+
132
+ ### Results
133
+
134
+ [More Information Needed]
135
+
136
+ #### Summary
137
+
138
+
139
+
140
+ ## Model Examination [optional]
141
+
142
+ <!-- Relevant interpretability work for the model goes here -->
143
+
144
+ [More Information Needed]
145
+
146
+ ## Environmental Impact
147
+
148
+ <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
149
+
150
+ Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
151
+
152
+ - **Hardware Type:** [More Information Needed]
153
+ - **Hours used:** [More Information Needed]
154
+ - **Cloud Provider:** [More Information Needed]
155
+ - **Compute Region:** [More Information Needed]
156
+ - **Carbon Emitted:** [More Information Needed]
157
+
158
+ ## Technical Specifications [optional]
159
+
160
+ ### Model Architecture and Objective
161
+
162
+ [More Information Needed]
163
+
164
+ ### Compute Infrastructure
165
+
166
+ [More Information Needed]
167
+
168
+ #### Hardware
169
+
170
+ [More Information Needed]
171
+
172
+ #### Software
173
+
174
+ [More Information Needed]
175
+
176
+ ## Citation [optional]
177
+
178
+ <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
179
+
180
+ **BibTeX:**
181
+
182
+ [More Information Needed]
183
+
184
+ **APA:**
185
+
186
+ [More Information Needed]
187
+
188
+ ## Glossary [optional]
189
+
190
+ <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
191
+
192
+ [More Information Needed]
193
+
194
+ ## More Information [optional]
195
+
196
+ [More Information Needed]
197
+
198
+ ## Model Card Authors [optional]
199
+
200
+ [More Information Needed]
201
+
202
+ ## Model Card Contact
203
+
204
+ [More Information Needed]
205
+ ### Framework versions
206
+
207
+ - PEFT 0.21.2
e08-V/final/adapter_config.json ADDED
@@ -0,0 +1,56 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alora_invocation_tokens": null,
3
+ "alpha_pattern": {},
4
+ "arrow_config": null,
5
+ "auto_mapping": null,
6
+ "base_model_name_or_path": "/dev/shm/qwen36_27b",
7
+ "bias": "none",
8
+ "corda_config": null,
9
+ "ensure_weight_tying": false,
10
+ "eva_config": null,
11
+ "exclude_modules": null,
12
+ "fan_in_fan_out": false,
13
+ "inference_mode": true,
14
+ "init_lora_weights": true,
15
+ "kasa_config": null,
16
+ "layer_replication": null,
17
+ "layers_pattern": null,
18
+ "layers_to_transform": null,
19
+ "loftq_config": {},
20
+ "lora_alpha": 64,
21
+ "lora_bias": false,
22
+ "lora_dropout": 0.0,
23
+ "lora_ga_config": null,
24
+ "megatron_config": null,
25
+ "megatron_core": "megatron.core",
26
+ "modules_to_save": null,
27
+ "monteclora_config": null,
28
+ "peft_type": "LORA",
29
+ "peft_version": "0.21.2",
30
+ "qalora_group_size": 16,
31
+ "r": 32,
32
+ "rank_pattern": {},
33
+ "revision": null,
34
+ "target_modules": [
35
+ "o_proj",
36
+ "v_proj",
37
+ "in_proj_b",
38
+ "in_proj_z",
39
+ "in_proj_qkv",
40
+ "in_proj_a",
41
+ "gate_proj",
42
+ "q_proj",
43
+ "out_proj",
44
+ "up_proj",
45
+ "down_proj",
46
+ "k_proj"
47
+ ],
48
+ "target_parameters": null,
49
+ "task_type": "CAUSAL_LM",
50
+ "trainable_token_indices": null,
51
+ "use_bdlora": null,
52
+ "use_dora": false,
53
+ "use_qalora": false,
54
+ "use_rslora": false,
55
+ "velora_config": null
56
+ }
e08-V/final/adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c3f0139a1fa01c4d927c0f5161ac7f8dfa0d0da82bad573a3fc09f3df4753f0e
3
+ size 933959152
e08-V/meta/golden_render.jsonl ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ {"idx": 4176, "n_tok": 964, "n_sup": 939, "masked_tail": "<|im_start|>user\nWrite a short prose story about Indira and Jovan.<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n\n", "trained_head": "Indira leaned over the workbench, her magnifying glass reflecting the harsh overhead light as she examined the damaged binding. \"I need to fix this spine crack without losing the surface wear,\" she sa", "trained_tail": "wers are. Don\u2019t rub the surface, just dab gently.\"<|im_end|>"}
2
+ {"idx": 1178, "n_tok": 768, "n_sup": 742, "masked_tail": "<|im_start|>user\nWrite a short prose story about Kaori and Eustace.<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n\n", "trained_head": "Kaori stared at the jagged lines on the monitor, her brow furrowed as the rhythmic spike repeated with mechanical precision. She tapped the screen, highlighting the anomaly. \"The data from the past th", "trained_tail": "est fix is the one that lets the real earth talk.\"<|im_end|>"}
3
+ {"idx": 4262, "n_tok": 1050, "n_sup": 1025, "masked_tail": "<|im_start|>user\nWrite a short prose story about Natan and Olwen.<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n\n", "trained_head": "Natan stared at the warped wooden container, his jaw tightening as he inspected the failing structure. He turned to Olwen, his voice clipped and urgent. \"Olwen, I need to fix this basket. The left sid", "trained_tail": "forcing weft. Take your time with that last pass.\"<|im_end|>"}
e08-V/meta/train_log.jsonl ADDED
@@ -0,0 +1,243 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {"step": 1, "loss": 1.2806394043273888, "story_loss": 1.2806394043273888, "grad_norm": 0.44672727584838867, "lr": 5e-05, "elapsed": 162.4, "tokens": 32004, "story_tokens": 32004, "peak_gb": 72.1}
2
+ {"step": 2, "loss": 1.216514536012518, "story_loss": 1.216514536012518, "grad_norm": 0.37966233491897583, "lr": 0.0001, "elapsed": 185.0, "tokens": 32513, "story_tokens": 32513, "peak_gb": 74.0}
3
+ {"step": 3, "loss": 1.163060155994104, "story_loss": 1.163060155994104, "grad_norm": 0.3152586817741394, "lr": 0.00015, "elapsed": 207.0, "tokens": 30530, "story_tokens": 30530, "peak_gb": 74.0}
4
+ {"step": 4, "loss": 1.099282568055998, "story_loss": 1.099282568055998, "grad_norm": 0.44420674443244934, "lr": 0.0002, "elapsed": 229.6, "tokens": 31385, "story_tokens": 31385, "peak_gb": 74.0}
5
+ {"step": 5, "loss": 1.0121260210217766, "story_loss": 1.0121260210217766, "grad_norm": 0.29173505306243896, "lr": 0.00025, "elapsed": 252.3, "tokens": 31674, "story_tokens": 31674, "peak_gb": 74.0}
6
+ {"step": 6, "loss": 1.0182074497027802, "story_loss": 1.0182074497027802, "grad_norm": 0.21186524629592896, "lr": 0.0003, "elapsed": 292.3, "tokens": 32047, "story_tokens": 32047, "peak_gb": 74.7}
7
+ {"step": 7, "loss": 0.9794566384072878, "story_loss": 0.9794566384072878, "grad_norm": 0.1910044252872467, "lr": 0.00035, "elapsed": 314.1, "tokens": 30472, "story_tokens": 30472, "peak_gb": 74.7}
8
+ {"step": 8, "loss": 0.9671969515357204, "story_loss": 0.9671969515357204, "grad_norm": 0.1886375993490219, "lr": 0.0004, "elapsed": 336.9, "tokens": 32149, "story_tokens": 32149, "peak_gb": 74.7}
9
+ {"step": 9, "loss": 0.9503432538368254, "story_loss": 0.9503432538368254, "grad_norm": 0.1930585354566574, "lr": 0.00045000000000000004, "elapsed": 358.5, "tokens": 30665, "story_tokens": 30665, "peak_gb": 74.7}
10
+ {"step": 10, "loss": 0.9560271067599372, "story_loss": 0.9560271067599372, "grad_norm": 0.2180389016866684, "lr": 0.0005, "elapsed": 380.4, "tokens": 31309, "story_tokens": 31309, "peak_gb": 74.7}
11
+ {"step": 11, "loss": 0.9166334603341463, "story_loss": 0.9166334603341463, "grad_norm": 0.21356628835201263, "lr": 0.0004999772756569482, "elapsed": 402.1, "tokens": 30649, "story_tokens": 30649, "peak_gb": 74.7}
12
+ {"step": 12, "loss": 0.9102097676238028, "story_loss": 0.9102097676238028, "grad_norm": 0.1764897257089615, "lr": 0.0004999091067589586, "elapsed": 423.9, "tokens": 30487, "story_tokens": 30487, "peak_gb": 74.7}
13
+ {"step": 13, "loss": 0.9149569144759646, "story_loss": 0.9149569144759646, "grad_norm": 0.18618157505989075, "lr": 0.000499795505698779, "elapsed": 445.9, "tokens": 29571, "story_tokens": 29571, "peak_gb": 74.8}
14
+ {"step": 14, "loss": 0.8883939854591895, "story_loss": 0.8883939854591895, "grad_norm": 0.1608622819185257, "lr": 0.0004996364931284847, "elapsed": 468.7, "tokens": 33291, "story_tokens": 33291, "peak_gb": 74.8}
15
+ {"step": 15, "loss": 0.9116097354852548, "story_loss": 0.9116097354852548, "grad_norm": 0.1863069087266922, "lr": 0.0004994320979557255, "elapsed": 490.5, "tokens": 31468, "story_tokens": 31468, "peak_gb": 74.8}
16
+ {"step": 16, "loss": 0.9030081556391737, "story_loss": 0.9030081556391737, "grad_norm": 0.18427298963069916, "lr": 0.0004991823573384695, "elapsed": 511.7, "tokens": 30338, "story_tokens": 30338, "peak_gb": 74.8}
17
+ {"step": 17, "loss": 0.8890909107808534, "story_loss": 0.8890909107808534, "grad_norm": 0.1554756611585617, "lr": 0.0004988873166782485, "elapsed": 533.8, "tokens": 31520, "story_tokens": 31520, "peak_gb": 74.8}
18
+ {"step": 18, "loss": 0.8894694807648305, "story_loss": 0.8894694807648305, "grad_norm": 0.1784059852361679, "lr": 0.0004985470296119039, "elapsed": 556.0, "tokens": 29909, "story_tokens": 29909, "peak_gb": 74.8}
19
+ {"step": 19, "loss": 0.8987494405078882, "story_loss": 0.8987494405078882, "grad_norm": 0.1668255627155304, "lr": 0.0004981615580018357, "elapsed": 579.2, "tokens": 31994, "story_tokens": 31994, "peak_gb": 74.8}
20
+ {"step": 20, "loss": 0.8730890194637422, "story_loss": 0.8730890194637422, "grad_norm": 0.1683792620897293, "lr": 0.0004977309719247571, "elapsed": 600.7, "tokens": 30552, "story_tokens": 30552, "peak_gb": 74.8}
21
+ {"step": 21, "loss": 0.8766131876844632, "story_loss": 0.8766131876844632, "grad_norm": 0.1501603126525879, "lr": 0.0004972553496589536, "elapsed": 622.6, "tokens": 30748, "story_tokens": 30748, "peak_gb": 74.8}
22
+ {"step": 22, "loss": 0.8722833259859948, "story_loss": 0.8722833259859948, "grad_norm": 0.157234787940979, "lr": 0.0004967347776700538, "elapsed": 645.5, "tokens": 33193, "story_tokens": 33193, "peak_gb": 74.8}
23
+ {"step": 23, "loss": 0.8830665051475147, "story_loss": 0.8830665051475147, "grad_norm": 0.1596481204032898, "lr": 0.000496169350595309, "elapsed": 665.7, "tokens": 29031, "story_tokens": 29031, "peak_gb": 74.8}
24
+ {"step": 24, "loss": 0.8530071041978837, "story_loss": 0.8530071041978837, "grad_norm": 0.15337099134922028, "lr": 0.0004955591712263901, "elapsed": 686.4, "tokens": 29698, "story_tokens": 29698, "peak_gb": 74.8}
25
+ {"step": 25, "loss": 0.8684479822766799, "story_loss": 0.8684479822766799, "grad_norm": 0.1604694277048111, "lr": 0.0004949043504906993, "elapsed": 707.7, "tokens": 30374, "story_tokens": 30374, "peak_gb": 74.8}
26
+ {"step": 26, "loss": 0.900178938388977, "story_loss": 0.900178938388977, "grad_norm": 0.4088505506515503, "lr": 0.0004942050074312048, "elapsed": 728.2, "tokens": 28131, "story_tokens": 28131, "peak_gb": 74.8}
27
+ {"step": 27, "loss": 0.8452768448577013, "story_loss": 0.8452768448577013, "grad_norm": 0.1570984274148941, "lr": 0.0004934612691847995, "elapsed": 751.4, "tokens": 33868, "story_tokens": 33868, "peak_gb": 74.8}
28
+ {"step": 28, "loss": 0.866139703409455, "story_loss": 0.866139703409455, "grad_norm": 0.1705753058195114, "lr": 0.0004926732709591879, "elapsed": 772.9, "tokens": 30431, "story_tokens": 30431, "peak_gb": 74.8}
29
+ {"step": 29, "loss": 0.859959945520355, "story_loss": 0.859959945520355, "grad_norm": 0.2260095328092575, "lr": 0.0004918411560083058, "elapsed": 794.9, "tokens": 31798, "story_tokens": 31798, "peak_gb": 74.8}
30
+ {"step": 30, "loss": 0.8337253059625386, "story_loss": 0.8337253059625386, "grad_norm": 0.157558411359787, "lr": 0.0004909650756062782, "elapsed": 818.1, "tokens": 31776, "story_tokens": 31776, "peak_gb": 74.8}
31
+ {"step": 31, "loss": 0.8708990249215123, "story_loss": 0.8708990249215123, "grad_norm": 0.18626919388771057, "lr": 0.0004900451890199178, "elapsed": 840.0, "tokens": 30797, "story_tokens": 30797, "peak_gb": 74.8}
32
+ {"step": 32, "loss": 0.8879341983582767, "story_loss": 0.8879341983582767, "grad_norm": 0.19147337973117828, "lr": 0.0004890816634797716, "elapsed": 877.5, "tokens": 28736, "story_tokens": 28736, "peak_gb": 74.8}
33
+ {"step": 33, "loss": 0.8599360965498615, "story_loss": 0.8599360965498615, "grad_norm": 0.16889123618602753, "lr": 0.0004880746741497186, "elapsed": 898.6, "tokens": 29968, "story_tokens": 29968, "peak_gb": 74.8}
34
+ {"step": 34, "loss": 0.8993041680211971, "story_loss": 0.8993041680211971, "grad_norm": 0.1652853935956955, "lr": 0.00048702440409512695, "elapsed": 918.3, "tokens": 27781, "story_tokens": 27781, "peak_gb": 74.8}
35
+ {"step": 35, "loss": 0.8553651071485672, "story_loss": 0.8553651071485672, "grad_norm": 0.15551267564296722, "lr": 0.0004859310442495727, "elapsed": 941.0, "tokens": 32837, "story_tokens": 32837, "peak_gb": 74.8}
36
+ {"step": 36, "loss": 0.8423452804337687, "story_loss": 0.8423452804337687, "grad_norm": 0.15577134490013123, "lr": 0.00048479479338012953, "elapsed": 963.1, "tokens": 30954, "story_tokens": 30954, "peak_gb": 74.9}
37
+ {"step": 37, "loss": 0.8384088594257721, "story_loss": 0.8384088594257721, "grad_norm": 0.16815413534641266, "lr": 0.00048361585805123386, "elapsed": 985.8, "tokens": 32929, "story_tokens": 32929, "peak_gb": 74.9}
38
+ {"step": 38, "loss": 0.8515242940646436, "story_loss": 0.8515242940646436, "grad_norm": 0.16956520080566406, "lr": 0.00048239445258713246, "elapsed": 1008.3, "tokens": 32219, "story_tokens": 32219, "peak_gb": 74.9}
39
+ {"step": 39, "loss": 0.8666963398110176, "story_loss": 0.8666963398110176, "grad_norm": 0.16174840927124023, "lr": 0.0004811307990329195, "elapsed": 1029.8, "tokens": 29596, "story_tokens": 29596, "peak_gb": 74.9}
40
+ {"step": 40, "loss": 0.8548498891482736, "story_loss": 0.8548498891482736, "grad_norm": 0.1670643538236618, "lr": 0.0004798251271141699, "elapsed": 1052.1, "tokens": 32134, "story_tokens": 32134, "peak_gb": 74.9}
41
+ {"step": 41, "loss": 0.8319679147329224, "story_loss": 0.8319679147329224, "grad_norm": 0.15468758344650269, "lr": 0.0004784776741951765, "elapsed": 1074.1, "tokens": 30065, "story_tokens": 30065, "peak_gb": 74.9}
42
+ {"step": 42, "loss": 0.8528754556642573, "story_loss": 0.8528754556642573, "grad_norm": 0.16984505951404572, "lr": 0.00047708868523579825, "elapsed": 1096.7, "tokens": 32096, "story_tokens": 32096, "peak_gb": 74.9}
43
+ {"step": 43, "loss": 0.887569148849753, "story_loss": 0.887569148849753, "grad_norm": 0.17166809737682343, "lr": 0.000475658412746928, "elapsed": 1118.5, "tokens": 30776, "story_tokens": 30776, "peak_gb": 74.9}
44
+ {"step": 44, "loss": 0.8512103214658939, "story_loss": 0.8512103214658939, "grad_norm": 0.17780514061450958, "lr": 0.0004741871167445873, "elapsed": 1140.4, "tokens": 31578, "story_tokens": 31578, "peak_gb": 74.9}
45
+ {"step": 45, "loss": 0.8647448600003267, "story_loss": 0.8647448600003267, "grad_norm": 0.16334232687950134, "lr": 0.00047267506470265684, "elapsed": 1164.2, "tokens": 34432, "story_tokens": 34432, "peak_gb": 74.9}
46
+ {"step": 46, "loss": 0.8417874472825366, "story_loss": 0.8417874472825366, "grad_norm": 0.15983158349990845, "lr": 0.0004711225315042513, "elapsed": 1186.2, "tokens": 30143, "story_tokens": 30143, "peak_gb": 74.9}
47
+ {"step": 47, "loss": 0.8431465893138562, "story_loss": 0.8431465893138562, "grad_norm": 0.16050569713115692, "lr": 0.0004695297993917464, "elapsed": 1207.9, "tokens": 29863, "story_tokens": 29863, "peak_gb": 74.9}
48
+ {"step": 48, "loss": 0.8347687601522248, "story_loss": 0.8347687601522248, "grad_norm": 0.1912875920534134, "lr": 0.0004678971579154697, "elapsed": 1229.4, "tokens": 30666, "story_tokens": 30666, "peak_gb": 74.9}
49
+ {"step": 49, "loss": 0.8433218696519725, "story_loss": 0.8433218696519725, "grad_norm": 0.15974265336990356, "lr": 0.000466224903881061, "elapsed": 1252.5, "tokens": 33891, "story_tokens": 33891, "peak_gb": 74.9}
50
+ {"step": 50, "loss": 0.8725484628981102, "story_loss": 0.8725484628981102, "grad_norm": 0.17897148430347443, "lr": 0.000464513341295515, "elapsed": 1273.3, "tokens": 30268, "story_tokens": 30268, "peak_gb": 74.9}
51
+ {"step": 51, "loss": 0.8331855681232899, "story_loss": 0.8331855681232899, "grad_norm": 0.159307062625885, "lr": 0.00046276278131191466, "elapsed": 1295.0, "tokens": 30701, "story_tokens": 30701, "peak_gb": 74.9}
52
+ {"step": 52, "loss": 0.8732773964776382, "story_loss": 0.8732773964776382, "grad_norm": 0.16582995653152466, "lr": 0.0004609735421728647, "elapsed": 1316.6, "tokens": 30152, "story_tokens": 30152, "peak_gb": 74.9}
53
+ {"step": 53, "loss": 0.8169370084195524, "story_loss": 0.8169370084195524, "grad_norm": 0.17668482661247253, "lr": 0.00045914594915263707, "elapsed": 1336.9, "tokens": 29008, "story_tokens": 29008, "peak_gb": 74.9}
54
+ {"step": 54, "loss": 0.8585455421369153, "story_loss": 0.8585455421369153, "grad_norm": 0.17490927875041962, "lr": 0.0004572803344980378, "elapsed": 1358.3, "tokens": 29993, "story_tokens": 29993, "peak_gb": 74.9}
55
+ {"step": 55, "loss": 0.8718593168431643, "story_loss": 0.8718593168431643, "grad_norm": 0.16298305988311768, "lr": 0.0004553770373680062, "elapsed": 1381.0, "tokens": 33080, "story_tokens": 33080, "peak_gb": 74.9}
56
+ {"step": 56, "loss": 0.8386370975628963, "story_loss": 0.8386370975628963, "grad_norm": 0.1699877828359604, "lr": 0.0004534364037719576, "elapsed": 1403.3, "tokens": 30755, "story_tokens": 30755, "peak_gb": 74.9}
57
+ {"step": 57, "loss": 0.8515600225722546, "story_loss": 0.8515600225722546, "grad_norm": 0.1596953123807907, "lr": 0.00045145878650688047, "elapsed": 1425.2, "tokens": 32126, "story_tokens": 32126, "peak_gb": 74.9}
58
+ {"step": 58, "loss": 0.8297415609550115, "story_loss": 0.8297415609550115, "grad_norm": 0.16838109493255615, "lr": 0.00044944454509320025, "elapsed": 1447.4, "tokens": 32211, "story_tokens": 32211, "peak_gb": 74.9}
59
+ {"step": 59, "loss": 0.852767268197979, "story_loss": 0.852767268197979, "grad_norm": 0.15958483517169952, "lr": 0.00044739404570941986, "elapsed": 1469.8, "tokens": 31730, "story_tokens": 31730, "peak_gb": 74.9}
60
+ {"step": 60, "loss": 0.8595250017477629, "story_loss": 0.8595250017477629, "grad_norm": 0.16838659346103668, "lr": 0.00044530766112555066, "elapsed": 1493.3, "tokens": 33525, "story_tokens": 33525, "peak_gb": 74.9}
61
+ {"step": 61, "loss": 0.8330229338136161, "story_loss": 0.8330229338136161, "grad_norm": 0.17341241240501404, "lr": 0.00044318577063534484, "elapsed": 1515.2, "tokens": 30967, "story_tokens": 30967, "peak_gb": 74.9}
62
+ {"step": 62, "loss": 0.8421013539846831, "story_loss": 0.8421013539846831, "grad_norm": 0.1732596606016159, "lr": 0.0004410287599873417, "elapsed": 1536.2, "tokens": 28906, "story_tokens": 28906, "peak_gb": 74.9}
63
+ {"step": 63, "loss": 0.827386080175943, "story_loss": 0.827386080175943, "grad_norm": 0.17065666615962982, "lr": 0.00043883702131474085, "elapsed": 1558.6, "tokens": 30809, "story_tokens": 30809, "peak_gb": 74.9}
64
+ {"step": 64, "loss": 0.8531985279864248, "story_loss": 0.8531985279864248, "grad_norm": 0.16548863053321838, "lr": 0.00043661095306411394, "elapsed": 1580.4, "tokens": 31436, "story_tokens": 31436, "peak_gb": 74.9}
65
+ {"step": 65, "loss": 0.8728406633649554, "story_loss": 0.8728406633649554, "grad_norm": 0.17627845704555511, "lr": 0.0004343509599229697, "elapsed": 1602.6, "tokens": 31920, "story_tokens": 31920, "peak_gb": 74.9}
66
+ {"step": 66, "loss": 0.8576225362692713, "story_loss": 0.8576225362692713, "grad_norm": 0.17821630835533142, "lr": 0.00043205745274618366, "elapsed": 1624.3, "tokens": 31102, "story_tokens": 31102, "peak_gb": 74.9}
67
+ {"step": 67, "loss": 0.8376063785069178, "story_loss": 0.8376063785069178, "grad_norm": 0.15717701613903046, "lr": 0.00042973084848130667, "elapsed": 1647.9, "tokens": 32525, "story_tokens": 32525, "peak_gb": 74.9}
68
+ {"step": 68, "loss": 0.8542137714868715, "story_loss": 0.8542137714868715, "grad_norm": 0.18064570426940918, "lr": 0.0004273715700927666, "elapsed": 1670.4, "tokens": 32858, "story_tokens": 32858, "peak_gb": 74.9}
69
+ {"step": 69, "loss": 0.8392699660920888, "story_loss": 0.8392699660920888, "grad_norm": 0.16426976025104523, "lr": 0.00042498004648497504, "elapsed": 1692.5, "tokens": 31980, "story_tokens": 31980, "peak_gb": 74.9}
70
+ {"step": 70, "loss": 0.8455428595727064, "story_loss": 0.8455428595727064, "grad_norm": 0.1682862937450409, "lr": 0.000422556712424355, "elapsed": 1714.0, "tokens": 30269, "story_tokens": 30269, "peak_gb": 74.9}
71
+ {"step": 71, "loss": 0.8391433464436818, "story_loss": 0.8391433464436818, "grad_norm": 0.17372089624404907, "lr": 0.00042010200846030266, "elapsed": 1736.7, "tokens": 32292, "story_tokens": 32292, "peak_gb": 74.9}
72
+ {"step": 72, "loss": 0.8138970732247915, "story_loss": 0.8138970732247915, "grad_norm": 0.14960874617099762, "lr": 0.0004176163808450978, "elapsed": 1761.8, "tokens": 35142, "story_tokens": 35142, "peak_gb": 74.9}
73
+ {"step": 73, "loss": 0.8427352296622234, "story_loss": 0.8427352296622234, "grad_norm": 0.16049103438854218, "lr": 0.00041510028145277743, "elapsed": 1784.8, "tokens": 31588, "story_tokens": 31588, "peak_gb": 74.9}
74
+ {"step": 74, "loss": 0.8454156824334136, "story_loss": 0.8454156824334136, "grad_norm": 0.18337830901145935, "lr": 0.00041255416769698757, "elapsed": 1806.3, "tokens": 30130, "story_tokens": 30130, "peak_gb": 74.9}
75
+ {"step": 75, "loss": 0.8525370453238149, "story_loss": 0.8525370453238149, "grad_norm": 0.16933570802211761, "lr": 0.0004099785024478276, "elapsed": 1827.8, "tokens": 31347, "story_tokens": 31347, "peak_gb": 74.9}
76
+ {"step": 76, "loss": 0.8618781023304829, "story_loss": 0.8618781023304829, "grad_norm": 0.16785679757595062, "lr": 0.00040737375394770325, "elapsed": 1850.7, "tokens": 33153, "story_tokens": 33153, "peak_gb": 74.9}
77
+ {"step": 77, "loss": 0.870467460959967, "story_loss": 0.870467460959967, "grad_norm": 0.16794469952583313, "lr": 0.00040474039572620235, "elapsed": 1872.8, "tokens": 31296, "story_tokens": 31296, "peak_gb": 74.9}
78
+ {"step": 78, "loss": 0.8278402179825768, "story_loss": 0.8278402179825768, "grad_norm": 0.15612097084522247, "lr": 0.0004020789065140097, "elapsed": 1894.5, "tokens": 30907, "story_tokens": 30907, "peak_gb": 74.9}
79
+ {"step": 79, "loss": 0.8566105926420047, "story_loss": 0.8566105926420047, "grad_norm": 0.17058898508548737, "lr": 0.0003993897701558764, "elapsed": 1915.9, "tokens": 29988, "story_tokens": 29988, "peak_gb": 74.9}
80
+ {"step": 80, "loss": 0.8438154914862522, "story_loss": 0.8438154914862522, "grad_norm": 0.16339139640331268, "lr": 0.00039667347552265945, "elapsed": 1936.6, "tokens": 30214, "story_tokens": 30214, "peak_gb": 74.9}
81
+ {"step": 81, "loss": 0.8265770106628281, "story_loss": 0.8265770106628281, "grad_norm": 0.15979717671871185, "lr": 0.00039393051642244736, "elapsed": 1959.2, "tokens": 32499, "story_tokens": 32499, "peak_gb": 74.9}
82
+ {"step": 82, "loss": 0.8579442571969464, "story_loss": 0.8579442571969464, "grad_norm": 0.16621612012386322, "lr": 0.0003911613915107888, "elapsed": 1982.0, "tokens": 32618, "story_tokens": 32618, "peak_gb": 74.9}
83
+ {"step": 83, "loss": 0.8633204606596442, "story_loss": 0.8633204606596442, "grad_norm": 0.22214211523532867, "lr": 0.00038836660420003924, "elapsed": 2004.3, "tokens": 30254, "story_tokens": 30254, "peak_gb": 74.9}
84
+ {"step": 84, "loss": 0.8334909219670252, "story_loss": 0.8334909219670252, "grad_norm": 0.16688227653503418, "lr": 0.00038554666256784354, "elapsed": 2026.9, "tokens": 31581, "story_tokens": 31581, "peak_gb": 74.9}
85
+ {"step": 85, "loss": 0.8374069169032352, "story_loss": 0.8374069169032352, "grad_norm": 0.15564939379692078, "lr": 0.00038270207926476994, "elapsed": 2049.5, "tokens": 32672, "story_tokens": 32672, "peak_gb": 74.9}
86
+ {"step": 86, "loss": 0.83445551418443, "story_loss": 0.83445551418443, "grad_norm": 0.15371355414390564, "lr": 0.0003798333714211132, "elapsed": 2072.6, "tokens": 33754, "story_tokens": 33754, "peak_gb": 74.9}
87
+ {"step": 87, "loss": 0.8574138253011749, "story_loss": 0.8574138253011749, "grad_norm": 0.16755178570747375, "lr": 0.00037694106055288236, "elapsed": 2094.2, "tokens": 31512, "story_tokens": 31512, "peak_gb": 74.9}
88
+ {"step": 88, "loss": 0.8340378611990364, "story_loss": 0.8340378611990364, "grad_norm": 0.15800628066062927, "lr": 0.0003740256724669925, "elapsed": 2115.7, "tokens": 31395, "story_tokens": 31395, "peak_gb": 74.9}
89
+ {"step": 89, "loss": 0.8177655383664952, "story_loss": 0.8177655383664952, "grad_norm": 0.16054680943489075, "lr": 0.0003710877371656757, "elapsed": 2136.8, "tokens": 29263, "story_tokens": 29263, "peak_gb": 74.9}
90
+ {"step": 90, "loss": 0.8352345571747763, "story_loss": 0.8352345571747763, "grad_norm": 0.159176766872406, "lr": 0.00036812778875012895, "elapsed": 2159.0, "tokens": 31413, "story_tokens": 31413, "peak_gb": 74.9}
91
+ {"step": 91, "loss": 0.8781199631202037, "story_loss": 0.8781199631202037, "grad_norm": 0.16436930000782013, "lr": 0.0003651463653234182, "elapsed": 2180.0, "tokens": 29189, "story_tokens": 29189, "peak_gb": 74.9}
92
+ {"step": 92, "loss": 0.8109415046195032, "story_loss": 0.8109415046195032, "grad_norm": 0.15483947098255157, "lr": 0.00036214400889265306, "elapsed": 2203.3, "tokens": 32860, "story_tokens": 32860, "peak_gb": 74.9}
93
+ {"step": 93, "loss": 0.8650860457255523, "story_loss": 0.8650860457255523, "grad_norm": 0.16892531514167786, "lr": 0.0003591212652704536, "elapsed": 2225.2, "tokens": 30463, "story_tokens": 30463, "peak_gb": 74.9}
94
+ {"step": 94, "loss": 0.8516428369390385, "story_loss": 0.8516428369390385, "grad_norm": 0.1614653319120407, "lr": 0.00035607868397572416, "elapsed": 2246.1, "tokens": 31280, "story_tokens": 31280, "peak_gb": 74.9}
95
+ {"step": 95, "loss": 0.846549553061643, "story_loss": 0.846549553061643, "grad_norm": 0.17148663103580475, "lr": 0.0003530168181337534, "elapsed": 2265.9, "tokens": 27202, "story_tokens": 27202, "peak_gb": 74.9}
96
+ {"step": 96, "loss": 0.8647336938687323, "story_loss": 0.8647336938687323, "grad_norm": 0.1699124574661255, "lr": 0.0003499362243756595, "elapsed": 2287.0, "tokens": 29307, "story_tokens": 29307, "peak_gb": 74.9}
97
+ {"step": 97, "loss": 0.8352026707286238, "story_loss": 0.8352026707286238, "grad_norm": 0.1606065034866333, "lr": 0.00034683746273719753, "elapsed": 2308.6, "tokens": 31083, "story_tokens": 31083, "peak_gb": 74.9}
98
+ {"step": 98, "loss": 0.828716867565309, "story_loss": 0.828716867565309, "grad_norm": 0.1585349142551422, "lr": 0.00034372109655694745, "elapsed": 2332.8, "tokens": 32796, "story_tokens": 32796, "peak_gb": 74.9}
99
+ {"step": 99, "loss": 0.8215176590852691, "story_loss": 0.8215176590852691, "grad_norm": 0.18931259214878082, "lr": 0.0003405876923739025, "elapsed": 2355.2, "tokens": 32363, "story_tokens": 32363, "peak_gb": 74.9}
100
+ {"step": 100, "loss": 0.8525723170477428, "story_loss": 0.8525723170477428, "grad_norm": 0.16248872876167297, "lr": 0.0003374378198244753, "elapsed": 2378.0, "tokens": 33272, "story_tokens": 33272, "peak_gb": 74.9}
101
+ {"step": 101, "loss": 0.8618245737535289, "story_loss": 0.8618245737535289, "grad_norm": 0.1714329719543457, "lr": 0.00033427205153894085, "elapsed": 2399.6, "tokens": 30639, "story_tokens": 30639, "peak_gb": 74.9}
102
+ {"step": 102, "loss": 0.846384612897688, "story_loss": 0.846384612897688, "grad_norm": 0.16572803258895874, "lr": 0.0003310909630373356, "elapsed": 2421.2, "tokens": 31082, "story_tokens": 31082, "peak_gb": 74.9}
103
+ {"step": 103, "loss": 0.814135378450908, "story_loss": 0.814135378450908, "grad_norm": 0.15812450647354126, "lr": 0.00032789513262483053, "elapsed": 2443.5, "tokens": 31278, "story_tokens": 31278, "peak_gb": 74.9}
104
+ {"step": 104, "loss": 0.8517801235052557, "story_loss": 0.8517801235052557, "grad_norm": 0.15501198172569275, "lr": 0.0003246851412865988, "elapsed": 2467.0, "tokens": 34130, "story_tokens": 34130, "peak_gb": 74.9}
105
+ {"step": 105, "loss": 0.82761975307171, "story_loss": 0.82761975307171, "grad_norm": 0.15646913647651672, "lr": 0.00032146157258219535, "elapsed": 2489.1, "tokens": 31637, "story_tokens": 31637, "peak_gb": 74.9}
106
+ {"step": 106, "loss": 0.8555994141480681, "story_loss": 0.8555994141480681, "grad_norm": 0.16597408056259155, "lr": 0.00031822501253946875, "elapsed": 2509.6, "tokens": 29673, "story_tokens": 29673, "peak_gb": 74.9}
107
+ {"step": 107, "loss": 0.8281249021382403, "story_loss": 0.8281249021382403, "grad_norm": 0.15531137585639954, "lr": 0.0003149760495480248, "elapsed": 2531.0, "tokens": 29937, "story_tokens": 29937, "peak_gb": 74.9}
108
+ {"step": 108, "loss": 0.8590988408005271, "story_loss": 0.8590988408005271, "grad_norm": 0.16717666387557983, "lr": 0.0003117152742522603, "elapsed": 2551.5, "tokens": 30355, "story_tokens": 30355, "peak_gb": 74.9}
109
+ {"step": 109, "loss": 0.8528351956007378, "story_loss": 0.8528351956007378, "grad_norm": 0.1711253672838211, "lr": 0.0003084432794439865, "elapsed": 2573.2, "tokens": 31170, "story_tokens": 31170, "peak_gb": 74.9}
110
+ {"step": 110, "loss": 0.8486386539112619, "story_loss": 0.8486386539112619, "grad_norm": 0.16950321197509766, "lr": 0.00030516065995466336, "elapsed": 2594.3, "tokens": 30552, "story_tokens": 30552, "peak_gb": 74.9}
111
+ {"step": 111, "loss": 0.8126917944487577, "story_loss": 0.8126917944487577, "grad_norm": 0.15672200918197632, "lr": 0.0003018680125472621, "elapsed": 2616.2, "tokens": 32200, "story_tokens": 32200, "peak_gb": 74.9}
112
+ {"step": 112, "loss": 0.8154055828521051, "story_loss": 0.8154055828521051, "grad_norm": 0.1680678278207779, "lr": 0.00029856593580777647, "elapsed": 2638.1, "tokens": 31246, "story_tokens": 31246, "peak_gb": 74.9}
113
+ {"step": 113, "loss": 0.8160903872308589, "story_loss": 0.8160903872308589, "grad_norm": 0.16016043722629547, "lr": 0.00029525503003640334, "elapsed": 2660.3, "tokens": 32685, "story_tokens": 32685, "peak_gb": 74.9}
114
+ {"step": 114, "loss": 0.8449034713326276, "story_loss": 0.8449034713326276, "grad_norm": 0.16182264685630798, "lr": 0.0002919358971384113, "elapsed": 2681.5, "tokens": 30347, "story_tokens": 30347, "peak_gb": 74.9}
115
+ {"step": 115, "loss": 0.819269917493949, "story_loss": 0.819269917493949, "grad_norm": 0.16418401896953583, "lr": 0.0002886091405147172, "elapsed": 2703.1, "tokens": 31296, "story_tokens": 31296, "peak_gb": 74.9}
116
+ {"step": 116, "loss": 0.8219096095345554, "story_loss": 0.8219096095345554, "grad_norm": 0.1587805449962616, "lr": 0.00028527536495219106, "elapsed": 2725.2, "tokens": 32806, "story_tokens": 32806, "peak_gb": 74.9}
117
+ {"step": 117, "loss": 0.8349691232121842, "story_loss": 0.8349691232121842, "grad_norm": 0.15545785427093506, "lr": 0.00028193517651370933, "elapsed": 2746.8, "tokens": 31227, "story_tokens": 31227, "peak_gb": 74.9}
118
+ {"step": 118, "loss": 0.8341999254537757, "story_loss": 0.8341999254537757, "grad_norm": 0.15577645599842072, "lr": 0.0002785891824279755, "elapsed": 2769.6, "tokens": 33662, "story_tokens": 33662, "peak_gb": 74.9}
119
+ {"step": 119, "loss": 0.8451067378030234, "story_loss": 0.8451067378030234, "grad_norm": 0.16528593003749847, "lr": 0.00027523799097912906, "elapsed": 2791.8, "tokens": 32877, "story_tokens": 32877, "peak_gb": 74.9}
120
+ {"step": 120, "loss": 0.8161850144005243, "story_loss": 0.8161850144005243, "grad_norm": 0.16572533547878265, "lr": 0.00027188221139616304, "elapsed": 2813.3, "tokens": 30804, "story_tokens": 30804, "peak_gb": 74.9}
121
+ {"step": 121, "loss": 0.8503398242850658, "story_loss": 0.8503398242850658, "grad_norm": 0.1628333479166031, "lr": 0.000268522453742169, "elapsed": 2834.4, "tokens": 31808, "story_tokens": 31808, "peak_gb": 74.9}
122
+ {"step": 122, "loss": 0.8201501307724692, "story_loss": 0.8201501307724692, "grad_norm": 0.15862740576267242, "lr": 0.00026515932880343104, "elapsed": 2856.2, "tokens": 31621, "story_tokens": 31621, "peak_gb": 74.9}
123
+ {"step": 123, "loss": 0.8264033500460277, "story_loss": 0.8264033500460277, "grad_norm": 0.15998569130897522, "lr": 0.00026179344797838775, "elapsed": 2878.2, "tokens": 31333, "story_tokens": 31333, "peak_gb": 74.9}
124
+ {"step": 124, "loss": 0.8338664233951533, "story_loss": 0.8338664233951533, "grad_norm": 0.16547444462776184, "lr": 0.0002584254231664833, "elapsed": 2900.7, "tokens": 31031, "story_tokens": 31031, "peak_gb": 74.9}
125
+ {"step": 125, "loss": 0.8445863722862125, "story_loss": 0.8445863722862125, "grad_norm": 0.15795308351516724, "lr": 0.0002550558666569279, "elapsed": 2922.3, "tokens": 32047, "story_tokens": 32047, "peak_gb": 74.9}
126
+ {"step": 126, "loss": 0.8299679293878798, "story_loss": 0.8299679293878798, "grad_norm": 0.16400516033172607, "lr": 0.00025168539101738576, "elapsed": 2945.6, "tokens": 30377, "story_tokens": 30377, "peak_gb": 74.9}
127
+ {"step": 127, "loss": 0.8192306651278932, "story_loss": 0.8192306651278932, "grad_norm": 0.15796810388565063, "lr": 0.00024831460898261425, "elapsed": 2968.7, "tokens": 33915, "story_tokens": 33915, "peak_gb": 74.9}
128
+ {"step": 128, "loss": 0.840570077157033, "story_loss": 0.840570077157033, "grad_norm": 0.15880009531974792, "lr": 0.0002449441333430722, "elapsed": 2991.6, "tokens": 32607, "story_tokens": 32607, "peak_gb": 75.1}
129
+ {"step": 129, "loss": 0.818232370776511, "story_loss": 0.818232370776511, "grad_norm": 0.1491648554801941, "lr": 0.00024157457683351668, "elapsed": 3014.0, "tokens": 31916, "story_tokens": 31916, "peak_gb": 75.1}
130
+ {"step": 130, "loss": 0.8145418855782453, "story_loss": 0.8145418855782453, "grad_norm": 0.15487952530384064, "lr": 0.00023820655202161237, "elapsed": 3036.2, "tokens": 32363, "story_tokens": 32363, "peak_gb": 75.1}
131
+ {"step": 131, "loss": 0.8098891863076232, "story_loss": 0.8098891863076232, "grad_norm": 0.16136087477207184, "lr": 0.00023484067119656905, "elapsed": 3056.6, "tokens": 27951, "story_tokens": 27951, "peak_gb": 75.1}
132
+ {"step": 132, "loss": 0.8248288684891022, "story_loss": 0.8248288684891022, "grad_norm": 0.15285231173038483, "lr": 0.000231477546257831, "elapsed": 3078.7, "tokens": 32472, "story_tokens": 32472, "peak_gb": 75.1}
133
+ {"step": 133, "loss": 0.8288207746579792, "story_loss": 0.8288207746579792, "grad_norm": 0.15982721745967865, "lr": 0.000228117788603837, "elapsed": 3100.7, "tokens": 32710, "story_tokens": 32710, "peak_gb": 75.1}
134
+ {"step": 134, "loss": 0.853922714272759, "story_loss": 0.853922714272759, "grad_norm": 0.1610126793384552, "lr": 0.00022476200902087098, "elapsed": 3121.6, "tokens": 29075, "story_tokens": 29075, "peak_gb": 75.1}
135
+ {"step": 135, "loss": 0.8258953550027786, "story_loss": 0.8258953550027786, "grad_norm": 0.15791746973991394, "lr": 0.00022141081757202458, "elapsed": 3143.3, "tokens": 30816, "story_tokens": 30816, "peak_gb": 75.1}
136
+ {"step": 136, "loss": 0.8491072523877062, "story_loss": 0.8491072523877062, "grad_norm": 0.15472227334976196, "lr": 0.00021806482348629065, "elapsed": 3165.4, "tokens": 32543, "story_tokens": 32543, "peak_gb": 75.1}
137
+ {"step": 137, "loss": 0.8321887789336647, "story_loss": 0.8321887789336647, "grad_norm": 0.1506691575050354, "lr": 0.00021472463504780892, "elapsed": 3188.9, "tokens": 34254, "story_tokens": 34254, "peak_gb": 75.1}
138
+ {"step": 138, "loss": 0.8021191239458796, "story_loss": 0.8021191239458796, "grad_norm": 0.1504116654396057, "lr": 0.00021139085948528285, "elapsed": 3210.2, "tokens": 31617, "story_tokens": 31617, "peak_gb": 75.1}
139
+ {"step": 139, "loss": 0.812900629931748, "story_loss": 0.812900629931748, "grad_norm": 0.14717522263526917, "lr": 0.00020806410286158879, "elapsed": 3232.1, "tokens": 32728, "story_tokens": 32728, "peak_gb": 75.1}
140
+ {"step": 140, "loss": 0.8261819360595819, "story_loss": 0.8261819360595819, "grad_norm": 0.16309137642383575, "lr": 0.00020474496996359672, "elapsed": 3253.7, "tokens": 31485, "story_tokens": 31485, "peak_gb": 75.1}
141
+ {"step": 141, "loss": 0.8406954497933644, "story_loss": 0.8406954497933644, "grad_norm": 0.15436477959156036, "lr": 0.00020143406419222354, "elapsed": 3274.7, "tokens": 31169, "story_tokens": 31169, "peak_gb": 75.1}
142
+ {"step": 142, "loss": 0.8270587615365268, "story_loss": 0.8270587615365268, "grad_norm": 0.15643204748630524, "lr": 0.00019813198745273798, "elapsed": 3299.0, "tokens": 30440, "story_tokens": 30440, "peak_gb": 75.1}
143
+ {"step": 143, "loss": 0.8160428119301851, "story_loss": 0.8160428119301851, "grad_norm": 0.15367500483989716, "lr": 0.00019483934004533665, "elapsed": 3321.5, "tokens": 32357, "story_tokens": 32357, "peak_gb": 75.1}
144
+ {"step": 144, "loss": 0.8290335853321581, "story_loss": 0.8290335853321581, "grad_norm": 0.15591169893741608, "lr": 0.00019155672055601363, "elapsed": 3343.2, "tokens": 31555, "story_tokens": 31555, "peak_gb": 75.1}
145
+ {"step": 145, "loss": 0.8494631240980567, "story_loss": 0.8494631240980567, "grad_norm": 0.16318614780902863, "lr": 0.00018828472574773978, "elapsed": 3364.4, "tokens": 31789, "story_tokens": 31789, "peak_gb": 75.1}
146
+ {"step": 146, "loss": 0.8030804308881698, "story_loss": 0.8030804308881698, "grad_norm": 0.14761114120483398, "lr": 0.00018502395045197521, "elapsed": 3386.8, "tokens": 33087, "story_tokens": 33087, "peak_gb": 75.1}
147
+ {"step": 147, "loss": 0.806679593721613, "story_loss": 0.806679593721613, "grad_norm": 0.15984342992305756, "lr": 0.00018177498746053128, "elapsed": 3408.5, "tokens": 30824, "story_tokens": 30824, "peak_gb": 75.1}
148
+ {"step": 148, "loss": 0.8184624520075486, "story_loss": 0.8184624520075486, "grad_norm": 0.16973887383937836, "lr": 0.00017853842741780474, "elapsed": 3429.9, "tokens": 30734, "story_tokens": 30734, "peak_gb": 75.1}
149
+ {"step": 149, "loss": 0.8414989973515663, "story_loss": 0.8414989973515663, "grad_norm": 0.1570214033126831, "lr": 0.00017531485871340123, "elapsed": 3451.9, "tokens": 30885, "story_tokens": 30885, "peak_gb": 75.1}
150
+ {"step": 150, "loss": 0.8181908567108737, "story_loss": 0.8181908567108737, "grad_norm": 0.14957743883132935, "lr": 0.00017210486737516946, "elapsed": 3472.7, "tokens": 30967, "story_tokens": 30967, "peak_gb": 75.1}
151
+ {"step": 151, "loss": 0.8359500672964129, "story_loss": 0.8359500672964129, "grad_norm": 0.16320033371448517, "lr": 0.00016890903696266447, "elapsed": 3494.2, "tokens": 30529, "story_tokens": 30529, "peak_gb": 75.1}
152
+ {"step": 152, "loss": 0.8284014116786101, "story_loss": 0.8284014116786101, "grad_norm": 0.15387308597564697, "lr": 0.00016572794846105918, "elapsed": 3515.6, "tokens": 30927, "story_tokens": 30927, "peak_gb": 75.1}
153
+ {"step": 153, "loss": 0.812258456799906, "story_loss": 0.812258456799906, "grad_norm": 0.1577296257019043, "lr": 0.00016256218017552483, "elapsed": 3535.7, "tokens": 28198, "story_tokens": 28198, "peak_gb": 75.1}
154
+ {"step": 154, "loss": 0.8091100766706478, "story_loss": 0.8091100766706478, "grad_norm": 0.15075038373470306, "lr": 0.0001594123076260975, "elapsed": 3557.6, "tokens": 32392, "story_tokens": 32392, "peak_gb": 75.1}
155
+ {"step": 155, "loss": 0.829327352023204, "story_loss": 0.829327352023204, "grad_norm": 0.14885210990905762, "lr": 0.00015627890344305253, "elapsed": 3580.7, "tokens": 33712, "story_tokens": 33712, "peak_gb": 75.1}
156
+ {"step": 156, "loss": 0.7927457107347343, "story_loss": 0.7927457107347343, "grad_norm": 0.14586782455444336, "lr": 0.0001531625372628025, "elapsed": 3603.1, "tokens": 32442, "story_tokens": 32442, "peak_gb": 75.1}
157
+ {"step": 157, "loss": 0.8155116893298947, "story_loss": 0.8155116893298947, "grad_norm": 0.14736567437648773, "lr": 0.00015006377562434047, "elapsed": 3625.1, "tokens": 32729, "story_tokens": 32729, "peak_gb": 75.1}
158
+ {"step": 158, "loss": 0.81808956881731, "story_loss": 0.81808956881731, "grad_norm": 0.15276367962360382, "lr": 0.00014698318186624667, "elapsed": 3645.4, "tokens": 29574, "story_tokens": 29574, "peak_gb": 75.1}
159
+ {"step": 159, "loss": 0.8285221924520098, "story_loss": 0.8285221924520098, "grad_norm": 0.16045473515987396, "lr": 0.00014392131602427587, "elapsed": 3666.5, "tokens": 31048, "story_tokens": 31048, "peak_gb": 75.1}
160
+ {"step": 160, "loss": 0.8414467429330263, "story_loss": 0.8414467429330263, "grad_norm": 0.15592032670974731, "lr": 0.00014087873472954637, "elapsed": 3686.8, "tokens": 28933, "story_tokens": 28933, "peak_gb": 75.1}
161
+ {"step": 161, "loss": 0.7984188691536043, "story_loss": 0.7984188691536043, "grad_norm": 0.14427590370178223, "lr": 0.00013785599110734698, "elapsed": 3709.7, "tokens": 33557, "story_tokens": 33557, "peak_gb": 75.1}
162
+ {"step": 162, "loss": 0.8392814362658778, "story_loss": 0.8392814362658778, "grad_norm": 0.16139009594917297, "lr": 0.00013485363467658184, "elapsed": 3731.8, "tokens": 31402, "story_tokens": 31402, "peak_gb": 75.1}
163
+ {"step": 163, "loss": 0.8285839798195305, "story_loss": 0.8285839798195305, "grad_norm": 0.1566365659236908, "lr": 0.00013187221124987106, "elapsed": 3752.5, "tokens": 30012, "story_tokens": 30012, "peak_gb": 75.1}
164
+ {"step": 164, "loss": 0.843098571976225, "story_loss": 0.843098571976225, "grad_norm": 0.15432563424110413, "lr": 0.0001289122628343244, "elapsed": 3773.0, "tokens": 28541, "story_tokens": 28541, "peak_gb": 75.1}
165
+ {"step": 165, "loss": 0.8129613745036391, "story_loss": 0.8129613745036391, "grad_norm": 0.14639072120189667, "lr": 0.00012597432753300752, "elapsed": 3794.1, "tokens": 30519, "story_tokens": 30519, "peak_gb": 75.1}
166
+ {"step": 166, "loss": 0.8231595579874493, "story_loss": 0.8231595579874493, "grad_norm": 0.15428146719932556, "lr": 0.0001230589394471177, "elapsed": 3814.8, "tokens": 30616, "story_tokens": 30616, "peak_gb": 75.1}
167
+ {"step": 167, "loss": 0.8065462146884952, "story_loss": 0.8065462146884952, "grad_norm": 0.14425522089004517, "lr": 0.00012016662857888689, "elapsed": 3836.6, "tokens": 32658, "story_tokens": 32658, "peak_gb": 75.1}
168
+ {"step": 168, "loss": 0.7960255661668589, "story_loss": 0.7960255661668589, "grad_norm": 0.15625114738941193, "lr": 0.00011729792073522999, "elapsed": 3857.3, "tokens": 28066, "story_tokens": 28066, "peak_gb": 75.1}
169
+ {"step": 169, "loss": 0.7963461104873301, "story_loss": 0.7963461104873301, "grad_norm": 0.15370775759220123, "lr": 0.00011445333743215647, "elapsed": 3881.0, "tokens": 32814, "story_tokens": 32814, "peak_gb": 75.3}
170
+ {"step": 170, "loss": 0.8346468968884698, "story_loss": 0.8346468968884698, "grad_norm": 0.16116227209568024, "lr": 0.00011163339579996079, "elapsed": 3901.9, "tokens": 29000, "story_tokens": 29000, "peak_gb": 75.3}
171
+ {"step": 171, "loss": 0.802194177595152, "story_loss": 0.802194177595152, "grad_norm": 0.14632494747638702, "lr": 0.00010883860848921121, "elapsed": 3924.2, "tokens": 31719, "story_tokens": 31719, "peak_gb": 75.3}
172
+ {"step": 172, "loss": 0.8225824334150357, "story_loss": 0.8225824334150357, "grad_norm": 0.1470327526330948, "lr": 0.00010606948357755269, "elapsed": 3947.6, "tokens": 33748, "story_tokens": 33748, "peak_gb": 75.3}
173
+ {"step": 173, "loss": 0.7989955788320305, "story_loss": 0.7989955788320305, "grad_norm": 0.1425153911113739, "lr": 0.00010332652447734057, "elapsed": 3970.2, "tokens": 33326, "story_tokens": 33326, "peak_gb": 75.3}
174
+ {"step": 174, "loss": 0.8065482881394078, "story_loss": 0.8065482881394078, "grad_norm": 0.14560505747795105, "lr": 0.0001006102298441236, "elapsed": 3992.0, "tokens": 31797, "story_tokens": 31797, "peak_gb": 75.3}
175
+ {"step": 175, "loss": 0.8050603795284647, "story_loss": 0.8050603795284647, "grad_norm": 0.14702221751213074, "lr": 9.792109348599035e-05, "elapsed": 4012.8, "tokens": 29563, "story_tokens": 29563, "peak_gb": 75.3}
176
+ {"step": 176, "loss": 0.7894448073916452, "story_loss": 0.7894448073916452, "grad_norm": 0.1424216628074646, "lr": 9.525960427379771e-05, "elapsed": 4035.6, "tokens": 33080, "story_tokens": 33080, "peak_gb": 75.3}
177
+ {"step": 177, "loss": 0.8074079848577306, "story_loss": 0.8074079848577306, "grad_norm": 0.14359505474567413, "lr": 9.262624605229672e-05, "elapsed": 4057.6, "tokens": 32724, "story_tokens": 32724, "peak_gb": 75.3}
178
+ {"step": 178, "loss": 0.8214007914236906, "story_loss": 0.8214007914236906, "grad_norm": 0.15947574377059937, "lr": 9.002149755217246e-05, "elapsed": 4077.9, "tokens": 29699, "story_tokens": 29699, "peak_gb": 75.3}
179
+ {"step": 179, "loss": 0.8199472318525062, "story_loss": 0.8199472318525062, "grad_norm": 0.15053321421146393, "lr": 8.744583230301248e-05, "elapsed": 4099.8, "tokens": 31981, "story_tokens": 31981, "peak_gb": 75.3}
180
+ {"step": 180, "loss": 0.7840781174589768, "story_loss": 0.7840781174589768, "grad_norm": 0.1405535340309143, "lr": 8.489971854722259e-05, "elapsed": 4121.7, "tokens": 32375, "story_tokens": 32375, "peak_gb": 75.3}
181
+ {"step": 181, "loss": 0.8180935825874439, "story_loss": 0.8180935825874439, "grad_norm": 0.15378281474113464, "lr": 8.238361915490225e-05, "elapsed": 4142.3, "tokens": 30298, "story_tokens": 30298, "peak_gb": 75.3}
182
+ {"step": 182, "loss": 0.8279150749570192, "story_loss": 0.8279150749570192, "grad_norm": 0.15389686822891235, "lr": 7.989799153969735e-05, "elapsed": 4163.5, "tokens": 29519, "story_tokens": 29519, "peak_gb": 75.3}
183
+ {"step": 183, "loss": 0.8079020934179663, "story_loss": 0.8079020934179663, "grad_norm": 0.14828629791736603, "lr": 7.7443287575645e-05, "elapsed": 4185.0, "tokens": 31388, "story_tokens": 31388, "peak_gb": 75.3}
184
+ {"step": 184, "loss": 0.8138239513209936, "story_loss": 0.8138239513209936, "grad_norm": 0.1447652280330658, "lr": 7.501995351502497e-05, "elapsed": 4207.1, "tokens": 32142, "story_tokens": 32142, "peak_gb": 75.3}
185
+ {"step": 185, "loss": 0.8191613723193287, "story_loss": 0.8191613723193287, "grad_norm": 0.14430025219917297, "lr": 7.26284299072334e-05, "elapsed": 4228.0, "tokens": 30295, "story_tokens": 30295, "peak_gb": 75.3}
186
+ {"step": 186, "loss": 0.7970507871985405, "story_loss": 0.7970507871985405, "grad_norm": 0.14170348644256592, "lr": 7.026915151869334e-05, "elapsed": 4249.3, "tokens": 31192, "story_tokens": 31192, "peak_gb": 75.3}
187
+ {"step": 187, "loss": 0.8143276183465685, "story_loss": 0.8143276183465685, "grad_norm": 0.14988189935684204, "lr": 6.794254725381641e-05, "elapsed": 4269.6, "tokens": 28916, "story_tokens": 28916, "peak_gb": 75.3}
188
+ {"step": 188, "loss": 0.8170063323882855, "story_loss": 0.8170063323882855, "grad_norm": 0.14316695928573608, "lr": 6.564904007703033e-05, "elapsed": 4292.1, "tokens": 33260, "story_tokens": 33260, "peak_gb": 75.3}
189
+ {"step": 189, "loss": 0.7896710142998743, "story_loss": 0.7896710142998743, "grad_norm": 0.1347511112689972, "lr": 6.33890469358861e-05, "elapsed": 4315.3, "tokens": 34018, "story_tokens": 34018, "peak_gb": 75.3}
190
+ {"step": 190, "loss": 0.8183290560045737, "story_loss": 0.8183290560045737, "grad_norm": 0.15093804895877838, "lr": 6.116297868525919e-05, "elapsed": 4336.5, "tokens": 29967, "story_tokens": 29967, "peak_gb": 75.3}
191
+ {"step": 191, "loss": 0.8174032214701417, "story_loss": 0.8174032214701417, "grad_norm": 0.1456390768289566, "lr": 5.8971240012658214e-05, "elapsed": 4357.5, "tokens": 30875, "story_tokens": 30875, "peak_gb": 75.3}
192
+ {"step": 192, "loss": 0.8204053529868115, "story_loss": 0.8204053529868115, "grad_norm": 0.14121761918067932, "lr": 5.681422936465522e-05, "elapsed": 4379.4, "tokens": 32225, "story_tokens": 32225, "peak_gb": 75.3}
193
+ {"step": 193, "loss": 0.8268390187197066, "story_loss": 0.8268390187197066, "grad_norm": 0.15899018943309784, "lr": 5.469233887444941e-05, "elapsed": 4400.2, "tokens": 30122, "story_tokens": 30122, "peak_gb": 75.3}
194
+ {"step": 194, "loss": 0.8398764808060037, "story_loss": 0.8398764808060037, "grad_norm": 0.15233221650123596, "lr": 5.260595429058021e-05, "elapsed": 4421.7, "tokens": 29825, "story_tokens": 29825, "peak_gb": 75.3}
195
+ {"step": 195, "loss": 0.8053547901447043, "story_loss": 0.8053547901447043, "grad_norm": 0.1465817242860794, "lr": 5.0555454906799804e-05, "elapsed": 4442.0, "tokens": 28934, "story_tokens": 28934, "peak_gb": 75.3}
196
+ {"step": 196, "loss": 0.819677208536214, "story_loss": 0.819677208536214, "grad_norm": 0.15290533006191254, "lr": 4.854121349311949e-05, "elapsed": 4463.1, "tokens": 29343, "story_tokens": 29343, "peak_gb": 75.3}
197
+ {"step": 197, "loss": 0.8239718970325662, "story_loss": 0.8239718970325662, "grad_norm": 0.14330638945102692, "lr": 4.656359622804243e-05, "elapsed": 4485.0, "tokens": 32764, "story_tokens": 32764, "peak_gb": 75.3}
198
+ {"step": 198, "loss": 0.8001786865521868, "story_loss": 0.8001786865521868, "grad_norm": 0.13923612236976624, "lr": 4.4622962631993805e-05, "elapsed": 4507.4, "tokens": 33119, "story_tokens": 33119, "peak_gb": 75.3}
199
+ {"step": 199, "loss": 0.8085144727180045, "story_loss": 0.8085144727180045, "grad_norm": 0.1417732685804367, "lr": 4.27196655019623e-05, "elapsed": 4530.7, "tokens": 34790, "story_tokens": 34790, "peak_gb": 75.3}
200
+ {"step": 200, "loss": 0.807645937286048, "story_loss": 0.807645937286048, "grad_norm": 0.1462358832359314, "lr": 4.085405084736296e-05, "elapsed": 4554.0, "tokens": 33594, "story_tokens": 33594, "peak_gb": 75.3}
201
+ {"step": 201, "loss": 0.7981554344786165, "story_loss": 0.7981554344786165, "grad_norm": 0.13822202384471893, "lr": 3.902645782713532e-05, "elapsed": 4576.6, "tokens": 34080, "story_tokens": 34080, "peak_gb": 75.3}
202
+ {"step": 202, "loss": 0.8155441409793858, "story_loss": 0.8155441409793858, "grad_norm": 0.14943869411945343, "lr": 3.723721868808533e-05, "elapsed": 4597.4, "tokens": 29761, "story_tokens": 29761, "peak_gb": 75.3}
203
+ {"step": 203, "loss": 0.795692268698225, "story_loss": 0.795692268698225, "grad_norm": 0.14673252403736115, "lr": 3.548665870448498e-05, "elapsed": 4618.8, "tokens": 31929, "story_tokens": 31929, "peak_gb": 75.3}
204
+ {"step": 204, "loss": 0.7995357556223556, "story_loss": 0.7995357556223556, "grad_norm": 0.14660429954528809, "lr": 3.377509611893903e-05, "elapsed": 4639.9, "tokens": 31198, "story_tokens": 31198, "peak_gb": 75.3}
205
+ {"step": 205, "loss": 0.8033241553909436, "story_loss": 0.8033241553909436, "grad_norm": 0.14906753599643707, "lr": 3.210284208453029e-05, "elapsed": 4662.6, "tokens": 32121, "story_tokens": 32121, "peak_gb": 75.3}
206
+ {"step": 206, "loss": 0.7849020031472825, "story_loss": 0.7849020031472825, "grad_norm": 0.14346401393413544, "lr": 3.047020060825359e-05, "elapsed": 4683.9, "tokens": 30994, "story_tokens": 30994, "peak_gb": 75.3}
207
+ {"step": 207, "loss": 0.8065823304494109, "story_loss": 0.8065823304494109, "grad_norm": 0.14862839877605438, "lr": 2.8877468495748765e-05, "elapsed": 4707.5, "tokens": 33651, "story_tokens": 33651, "peak_gb": 75.3}
208
+ {"step": 208, "loss": 0.8038492232406695, "story_loss": 0.8038492232406695, "grad_norm": 0.14947907626628876, "lr": 2.7324935297343146e-05, "elapsed": 4728.8, "tokens": 30880, "story_tokens": 30880, "peak_gb": 75.3}
209
+ {"step": 209, "loss": 0.7970084639358671, "story_loss": 0.7970084639358671, "grad_norm": 0.13851282000541687, "lr": 2.58128832554127e-05, "elapsed": 4751.6, "tokens": 33031, "story_tokens": 33031, "peak_gb": 75.3}
210
+ {"step": 210, "loss": 0.8071499428376915, "story_loss": 0.8071499428376915, "grad_norm": 0.14042727649211884, "lr": 2.4341587253072034e-05, "elapsed": 4773.5, "tokens": 32774, "story_tokens": 32774, "peak_gb": 75.3}
211
+ {"step": 211, "loss": 0.8115040279038468, "story_loss": 0.8115040279038468, "grad_norm": 0.14814059436321259, "lr": 2.2911314764201775e-05, "elapsed": 4796.0, "tokens": 31949, "story_tokens": 31949, "peak_gb": 75.3}
212
+ {"step": 212, "loss": 0.7982386256392449, "story_loss": 0.7982386256392449, "grad_norm": 0.1483047902584076, "lr": 2.1522325804823494e-05, "elapsed": 4817.1, "tokens": 29967, "story_tokens": 29967, "peak_gb": 75.3}
213
+ {"step": 213, "loss": 0.7947773207688188, "story_loss": 0.7947773207688188, "grad_norm": 0.1501215100288391, "lr": 2.0174872885830115e-05, "elapsed": 4838.8, "tokens": 31913, "story_tokens": 31913, "peak_gb": 75.3}
214
+ {"step": 214, "loss": 0.8052365968190951, "story_loss": 0.8052365968190951, "grad_norm": 0.14979587495326996, "lr": 1.8869200967080503e-05, "elapsed": 4859.1, "tokens": 30297, "story_tokens": 30297, "peak_gb": 75.3}
215
+ {"step": 215, "loss": 0.7834127519944187, "story_loss": 0.7834127519944187, "grad_norm": 0.13319745659828186, "lr": 1.7605547412867573e-05, "elapsed": 4882.4, "tokens": 34244, "story_tokens": 34244, "peak_gb": 75.3}
216
+ {"step": 216, "loss": 0.7951627311872362, "story_loss": 0.7951627311872362, "grad_norm": 0.1437256634235382, "lr": 1.6384141948766168e-05, "elapsed": 4903.4, "tokens": 31517, "story_tokens": 31517, "peak_gb": 75.3}
217
+ {"step": 217, "loss": 0.7891637659466425, "story_loss": 0.7891637659466425, "grad_norm": 0.14285698533058167, "lr": 1.5205206619870487e-05, "elapsed": 4925.5, "tokens": 31139, "story_tokens": 31139, "peak_gb": 75.3}
218
+ {"step": 218, "loss": 0.812235916063189, "story_loss": 0.812235916063189, "grad_norm": 0.14145144820213318, "lr": 1.406895575042727e-05, "elapsed": 4948.4, "tokens": 32563, "story_tokens": 32563, "peak_gb": 75.3}
219
+ {"step": 219, "loss": 0.7810120115226404, "story_loss": 0.7810120115226404, "grad_norm": 0.14130932092666626, "lr": 1.2975595904873073e-05, "elapsed": 4969.0, "tokens": 30874, "story_tokens": 30874, "peak_gb": 75.3}
220
+ {"step": 220, "loss": 0.8073108015298031, "story_loss": 0.8073108015298031, "grad_norm": 0.14159007370471954, "lr": 1.1925325850281416e-05, "elapsed": 4992.4, "tokens": 34598, "story_tokens": 34598, "peak_gb": 75.3}
221
+ {"step": 221, "loss": 0.8002349192119622, "story_loss": 0.8002349192119622, "grad_norm": 0.14387285709381104, "lr": 1.0918336520228472e-05, "elapsed": 5014.0, "tokens": 31053, "story_tokens": 31053, "peak_gb": 75.3}
222
+ {"step": 222, "loss": 0.8173973701013528, "story_loss": 0.8173973701013528, "grad_norm": 0.14693059027194977, "lr": 9.954810980082191e-06, "elapsed": 5034.7, "tokens": 29726, "story_tokens": 29726, "peak_gb": 75.3}
223
+ {"step": 223, "loss": 0.8227191798164981, "story_loss": 0.8227191798164981, "grad_norm": 0.14616630971431732, "lr": 9.034924393721777e-06, "elapsed": 5054.7, "tokens": 29373, "story_tokens": 29373, "peak_gb": 75.3}
224
+ {"step": 224, "loss": 0.8113699558768949, "story_loss": 0.8113699558768949, "grad_norm": 0.14560775458812714, "lr": 8.15884399169417e-06, "elapsed": 5076.4, "tokens": 31995, "story_tokens": 31995, "peak_gb": 75.3}
225
+ {"step": 225, "loss": 0.8362814163926438, "story_loss": 0.8362814163926438, "grad_norm": 0.1485193818807602, "lr": 7.326729040812136e-06, "elapsed": 5097.8, "tokens": 30202, "story_tokens": 30202, "peak_gb": 75.3}
226
+ {"step": 226, "loss": 0.8012813322419646, "story_loss": 0.8012813322419646, "grad_norm": 0.14298923313617706, "lr": 6.538730815200483e-06, "elapsed": 5119.4, "tokens": 32108, "story_tokens": 32108, "peak_gb": 75.3}
227
+ {"step": 227, "loss": 0.7904335984215449, "story_loss": 0.7904335984215449, "grad_norm": 0.13900890946388245, "lr": 5.7949925687952595e-06, "elapsed": 5141.1, "tokens": 32611, "story_tokens": 32611, "peak_gb": 75.3}
228
+ {"step": 228, "loss": 0.7812973751768235, "story_loss": 0.7812973751768235, "grad_norm": 0.1421976089477539, "lr": 5.095649509300804e-06, "elapsed": 5162.7, "tokens": 31237, "story_tokens": 31237, "peak_gb": 75.3}
229
+ {"step": 229, "loss": 0.820741418098716, "story_loss": 0.820741418098716, "grad_norm": 0.14505542814731598, "lr": 4.440828773609934e-06, "elapsed": 5183.7, "tokens": 30627, "story_tokens": 30627, "peak_gb": 75.3}
230
+ {"step": 230, "loss": 0.8444186608551428, "story_loss": 0.8444186608551428, "grad_norm": 0.15557672083377838, "lr": 3.830649404690939e-06, "elapsed": 5206.1, "tokens": 32803, "story_tokens": 32803, "peak_gb": 75.3}
231
+ {"step": 231, "loss": 0.780329878882891, "story_loss": 0.780329878882891, "grad_norm": 0.15183842182159424, "lr": 3.2652223299462214e-06, "elapsed": 5228.6, "tokens": 33947, "story_tokens": 33947, "peak_gb": 75.3}
232
+ {"step": 232, "loss": 0.7966818191119761, "story_loss": 0.7966818191119761, "grad_norm": 0.13679181039333344, "lr": 2.7446503410463174e-06, "elapsed": 5251.3, "tokens": 34360, "story_tokens": 34360, "peak_gb": 75.3}
233
+ {"step": 233, "loss": 0.796085482604858, "story_loss": 0.796085482604858, "grad_norm": 0.14182889461517334, "lr": 2.269028075242929e-06, "elapsed": 5274.0, "tokens": 32395, "story_tokens": 32395, "peak_gb": 75.3}
234
+ {"step": 234, "loss": 0.8057403469619134, "story_loss": 0.8057403469619134, "grad_norm": 0.15451966226100922, "lr": 1.8384419981642697e-06, "elapsed": 5296.5, "tokens": 32180, "story_tokens": 32180, "peak_gb": 75.3}
235
+ {"step": 235, "loss": 0.7817093736081221, "story_loss": 0.7817093736081221, "grad_norm": 0.14422394335269928, "lr": 1.452970388096192e-06, "elapsed": 5318.1, "tokens": 31713, "story_tokens": 31713, "peak_gb": 75.3}
236
+ {"step": 236, "loss": 0.8031574049055538, "story_loss": 0.8031574049055538, "grad_norm": 0.14445367455482483, "lr": 1.1126833217514899e-06, "elapsed": 5340.3, "tokens": 32181, "story_tokens": 32181, "peak_gb": 75.3}
237
+ {"step": 237, "loss": 0.8074227483188384, "story_loss": 0.8074227483188384, "grad_norm": 0.143574059009552, "lr": 8.176426615304766e-07, "elapsed": 5361.9, "tokens": 32037, "story_tokens": 32037, "peak_gb": 75.3}
238
+ {"step": 238, "loss": 0.7909704288813275, "story_loss": 0.7909704288813275, "grad_norm": 0.13992522656917572, "lr": 5.679020442745097e-07, "elapsed": 5383.8, "tokens": 32319, "story_tokens": 32319, "peak_gb": 75.3}
239
+ {"step": 239, "loss": 0.8005722246884314, "story_loss": 0.8005722246884314, "grad_norm": 0.14300888776779175, "lr": 3.635068715153178e-07, "elapsed": 5405.0, "tokens": 31203, "story_tokens": 31203, "peak_gb": 75.3}
240
+ {"step": 240, "loss": 0.7900971829200877, "story_loss": 0.7900971829200877, "grad_norm": 0.14540746808052063, "lr": 2.0449430122107537e-07, "elapsed": 5426.4, "tokens": 31515, "story_tokens": 31515, "peak_gb": 75.3}
241
+ {"step": 241, "loss": 0.8279874767808603, "story_loss": 0.8279874767808603, "grad_norm": 0.14622862637043, "lr": 9.08932410413621e-08, "elapsed": 5448.3, "tokens": 31914, "story_tokens": 31914, "peak_gb": 75.3}
242
+ {"step": 242, "loss": 0.8164044465604802, "story_loss": 0.8164044465604802, "grad_norm": 0.14237159490585327, "lr": 2.2724343051866437e-08, "elapsed": 5470.3, "tokens": 32490, "story_tokens": 32490, "peak_gb": 75.3}
243
+ {"step": 243, "loss": 0.7836830997213033, "story_loss": 0.7836830997213033, "grad_norm": 0.16696366667747498, "lr": 0.0, "elapsed": 5489.1, "tokens": 23278, "story_tokens": 23278, "peak_gb": 75.3}
e08-V/meta/train_meta.json ADDED
@@ -0,0 +1,61 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "data": "/workspace/e08/r08/data/train_V.jsonl",
3
+ "data_sha256": "bf1156c71f6b707a6b61c5169c5586bc4face2854ee4f55f7df4deae366d3d05",
4
+ "n_examples": 7766,
5
+ "n_loss_tokens": 7629283,
6
+ "replay": null,
7
+ "replay_sha256": null,
8
+ "replay_rows": 0,
9
+ "replay_per_step": 0,
10
+ "replay_loss_tokens_seen": 0,
11
+ "lr": 0.0005,
12
+ "rank": 32,
13
+ "alpha": 64,
14
+ "dropout": 0.0,
15
+ "eff_bs": 32,
16
+ "micro_bs": 4,
17
+ "accum": 8,
18
+ "epochs": 1,
19
+ "steps": 243,
20
+ "warmup": 10,
21
+ "schedule": "linear warmup then cosine decay to 0",
22
+ "optimizer": "AdamW(0.9,0.999,eps=1e-8,wd=0)",
23
+ "grad_clip": 1.0,
24
+ "seed": 0,
25
+ "targets": [
26
+ "q_proj",
27
+ "k_proj",
28
+ "v_proj",
29
+ "o_proj",
30
+ "in_proj_qkv",
31
+ "in_proj_z",
32
+ "in_proj_a",
33
+ "in_proj_b",
34
+ "out_proj",
35
+ "gate_proj",
36
+ "up_proj",
37
+ "down_proj"
38
+ ],
39
+ "model": "Qwen/Qwen3.6-27B",
40
+ "revision": "6a9e13bd6fc8f0983b9b99948120bc37f49c13e9",
41
+ "model_dir": "/dev/shm/qwen36_27b",
42
+ "chat_kw": {
43
+ "enable_thinking": false
44
+ },
45
+ "loss_mask": "assistant tokens + <|im_end|>; prompt, empty think block, trailing newline masked",
46
+ "loss_norm": "sum CE over supervised tokens in the optimizer step / their count",
47
+ "torch": "2.14.1+cu130",
48
+ "gradient_checkpointing": true,
49
+ "pack_tok": 0,
50
+ "first_order_idx": [
51
+ 4176,
52
+ 1178,
53
+ 4262,
54
+ 4708,
55
+ 3529,
56
+ 2111,
57
+ 6631,
58
+ 7281
59
+ ],
60
+ "trainable_params": 233455616
61
+ }