Same0620 commited on
Commit
e0e852b
·
verified ·
1 Parent(s): c65df49

e08-Nc: e08-N's stories, cosine (from volume)

Browse files
e08-Nc/final/README.md ADDED
@@ -0,0 +1,207 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: /dev/shm/qwen36_27b
3
+ library_name: peft
4
+ pipeline_tag: text-generation
5
+ tags:
6
+ - base_model:adapter:/dev/shm/qwen36_27b
7
+ - lora
8
+ - transformers
9
+ ---
10
+
11
+ # Model Card for Model ID
12
+
13
+ <!-- Provide a quick summary of what the model is/does. -->
14
+
15
+
16
+
17
+ ## Model Details
18
+
19
+ ### Model Description
20
+
21
+ <!-- Provide a longer summary of what this model is. -->
22
+
23
+
24
+
25
+ - **Developed by:** [More Information Needed]
26
+ - **Funded by [optional]:** [More Information Needed]
27
+ - **Shared by [optional]:** [More Information Needed]
28
+ - **Model type:** [More Information Needed]
29
+ - **Language(s) (NLP):** [More Information Needed]
30
+ - **License:** [More Information Needed]
31
+ - **Finetuned from model [optional]:** [More Information Needed]
32
+
33
+ ### Model Sources [optional]
34
+
35
+ <!-- Provide the basic links for the model. -->
36
+
37
+ - **Repository:** [More Information Needed]
38
+ - **Paper [optional]:** [More Information Needed]
39
+ - **Demo [optional]:** [More Information Needed]
40
+
41
+ ## Uses
42
+
43
+ <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
44
+
45
+ ### Direct Use
46
+
47
+ <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
48
+
49
+ [More Information Needed]
50
+
51
+ ### Downstream Use [optional]
52
+
53
+ <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
54
+
55
+ [More Information Needed]
56
+
57
+ ### Out-of-Scope Use
58
+
59
+ <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
60
+
61
+ [More Information Needed]
62
+
63
+ ## Bias, Risks, and Limitations
64
+
65
+ <!-- This section is meant to convey both technical and sociotechnical limitations. -->
66
+
67
+ [More Information Needed]
68
+
69
+ ### Recommendations
70
+
71
+ <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
72
+
73
+ Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
74
+
75
+ ## How to Get Started with the Model
76
+
77
+ Use the code below to get started with the model.
78
+
79
+ [More Information Needed]
80
+
81
+ ## Training Details
82
+
83
+ ### Training Data
84
+
85
+ <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
86
+
87
+ [More Information Needed]
88
+
89
+ ### Training Procedure
90
+
91
+ <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
92
+
93
+ #### Preprocessing [optional]
94
+
95
+ [More Information Needed]
96
+
97
+
98
+ #### Training Hyperparameters
99
+
100
+ - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
101
+
102
+ #### Speeds, Sizes, Times [optional]
103
+
104
+ <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
105
+
106
+ [More Information Needed]
107
+
108
+ ## Evaluation
109
+
110
+ <!-- This section describes the evaluation protocols and provides the results. -->
111
+
112
+ ### Testing Data, Factors & Metrics
113
+
114
+ #### Testing Data
115
+
116
+ <!-- This should link to a Dataset Card if possible. -->
117
+
118
+ [More Information Needed]
119
+
120
+ #### Factors
121
+
122
+ <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
123
+
124
+ [More Information Needed]
125
+
126
+ #### Metrics
127
+
128
+ <!-- These are the evaluation metrics being used, ideally with a description of why. -->
129
+
130
+ [More Information Needed]
131
+
132
+ ### Results
133
+
134
+ [More Information Needed]
135
+
136
+ #### Summary
137
+
138
+
139
+
140
+ ## Model Examination [optional]
141
+
142
+ <!-- Relevant interpretability work for the model goes here -->
143
+
144
+ [More Information Needed]
145
+
146
+ ## Environmental Impact
147
+
148
+ <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
149
+
150
+ Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
151
+
152
+ - **Hardware Type:** [More Information Needed]
153
+ - **Hours used:** [More Information Needed]
154
+ - **Cloud Provider:** [More Information Needed]
155
+ - **Compute Region:** [More Information Needed]
156
+ - **Carbon Emitted:** [More Information Needed]
157
+
158
+ ## Technical Specifications [optional]
159
+
160
+ ### Model Architecture and Objective
161
+
162
+ [More Information Needed]
163
+
164
+ ### Compute Infrastructure
165
+
166
+ [More Information Needed]
167
+
168
+ #### Hardware
169
+
170
+ [More Information Needed]
171
+
172
+ #### Software
173
+
174
+ [More Information Needed]
175
+
176
+ ## Citation [optional]
177
+
178
+ <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
179
+
180
+ **BibTeX:**
181
+
182
+ [More Information Needed]
183
+
184
+ **APA:**
185
+
186
+ [More Information Needed]
187
+
188
+ ## Glossary [optional]
189
+
190
+ <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
191
+
192
+ [More Information Needed]
193
+
194
+ ## More Information [optional]
195
+
196
+ [More Information Needed]
197
+
198
+ ## Model Card Authors [optional]
199
+
200
+ [More Information Needed]
201
+
202
+ ## Model Card Contact
203
+
204
+ [More Information Needed]
205
+ ### Framework versions
206
+
207
+ - PEFT 0.21.2
e08-Nc/final/adapter_config.json ADDED
@@ -0,0 +1,56 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alora_invocation_tokens": null,
3
+ "alpha_pattern": {},
4
+ "arrow_config": null,
5
+ "auto_mapping": null,
6
+ "base_model_name_or_path": "/dev/shm/qwen36_27b",
7
+ "bias": "none",
8
+ "corda_config": null,
9
+ "ensure_weight_tying": false,
10
+ "eva_config": null,
11
+ "exclude_modules": null,
12
+ "fan_in_fan_out": false,
13
+ "inference_mode": true,
14
+ "init_lora_weights": true,
15
+ "kasa_config": null,
16
+ "layer_replication": null,
17
+ "layers_pattern": null,
18
+ "layers_to_transform": null,
19
+ "loftq_config": {},
20
+ "lora_alpha": 64,
21
+ "lora_bias": false,
22
+ "lora_dropout": 0.0,
23
+ "lora_ga_config": null,
24
+ "megatron_config": null,
25
+ "megatron_core": "megatron.core",
26
+ "modules_to_save": null,
27
+ "monteclora_config": null,
28
+ "peft_type": "LORA",
29
+ "peft_version": "0.21.2",
30
+ "qalora_group_size": 16,
31
+ "r": 32,
32
+ "rank_pattern": {},
33
+ "revision": null,
34
+ "target_modules": [
35
+ "q_proj",
36
+ "down_proj",
37
+ "k_proj",
38
+ "o_proj",
39
+ "in_proj_b",
40
+ "up_proj",
41
+ "in_proj_a",
42
+ "in_proj_z",
43
+ "out_proj",
44
+ "in_proj_qkv",
45
+ "v_proj",
46
+ "gate_proj"
47
+ ],
48
+ "target_parameters": null,
49
+ "task_type": "CAUSAL_LM",
50
+ "trainable_token_indices": null,
51
+ "use_bdlora": null,
52
+ "use_dora": false,
53
+ "use_qalora": false,
54
+ "use_rslora": false,
55
+ "velora_config": null
56
+ }
e08-Nc/final/adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a0a4e02f60aae139a210eb74f014211f36f4a1a2f470aee260fe6b85c1daeda9
3
+ size 933959152
e08-Nc/meta/golden_render.jsonl ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ {"idx": 334, "n_tok": 790, "n_sup": 766, "masked_tail": "<|im_start|>user\nWrite a short prose story about Selene and Abdul.<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n\n", "trained_head": "Selene wiped grease from her hands and leaned over the exposed chassis of the ultralight frame. The metal was cold under her fingertips as she tightened a final screw.\n\n\"The brakes on this ultralight ", "trained_tail": "to go. Sounds like you\u2019ve got the hang of it now.\"<|im_end|>"}
2
+ {"idx": 4535, "n_tok": 1495, "n_sup": 1469, "masked_tail": "<|im_start|>user\nWrite a short prose story about Dina and Caspian.<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n\n", "trained_head": "Dina stood before the humming machinery of Enclosure 4, her brow furrowed as she read the discrepancy on the display. She turned to her colleague, her voice tight with professional urgency. \"Caspian, ", "trained_tail": "t we\u2019ll see how your sensor does. See you in ten.\"<|im_end|>"}
3
+ {"idx": 7771, "n_tok": 1491, "n_sup": 1466, "masked_tail": "<|im_start|>user\nWrite a short prose story about Jens and Ketevan.<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n\n", "trained_head": "Jens sat across from Ketevan in the small rehearsal room, his sheet music spread out before him. \"Hi Ketevan, thanks for meeting me. I\u2019m struggling with the Georgian lyrics for the concert. I need cle", "trained_tail": "ens. It was a pleasure,\" Ketevan called after him.<|im_end|>"}
e08-Nc/meta/train_log.jsonl ADDED
@@ -0,0 +1,246 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {"step": 1, "loss": 1.2459054364929751, "story_loss": 1.2459054364929751, "grad_norm": 0.4314831495285034, "lr": 5e-05, "elapsed": 173.8, "tokens": 31994, "story_tokens": 31994, "peak_gb": 72.1}
2
+ {"step": 2, "loss": 1.2198673436180691, "story_loss": 1.2198673436180691, "grad_norm": 0.3862919509410858, "lr": 0.0001, "elapsed": 198.9, "tokens": 35530, "story_tokens": 35530, "peak_gb": 74.1}
3
+ {"step": 3, "loss": 1.1071088369490358, "story_loss": 1.1071088369490358, "grad_norm": 0.31111088395118713, "lr": 0.00015, "elapsed": 242.8, "tokens": 36300, "story_tokens": 36300, "peak_gb": 74.9}
4
+ {"step": 4, "loss": 1.1255799988349213, "story_loss": 1.1255799988349213, "grad_norm": 0.43663373589515686, "lr": 0.0002, "elapsed": 265.1, "tokens": 30175, "story_tokens": 30175, "peak_gb": 74.9}
5
+ {"step": 5, "loss": 1.0549288043385345, "story_loss": 1.0549288043385345, "grad_norm": 0.2839439809322357, "lr": 0.00025, "elapsed": 287.6, "tokens": 31074, "story_tokens": 31074, "peak_gb": 74.9}
6
+ {"step": 6, "loss": 1.026433965951819, "story_loss": 1.026433965951819, "grad_norm": 0.2622741758823395, "lr": 0.0003, "elapsed": 312.3, "tokens": 35006, "story_tokens": 35006, "peak_gb": 74.9}
7
+ {"step": 7, "loss": 0.9976688467396381, "story_loss": 0.9976688467396381, "grad_norm": 0.19518834352493286, "lr": 0.00035, "elapsed": 335.2, "tokens": 32921, "story_tokens": 32921, "peak_gb": 74.9}
8
+ {"step": 8, "loss": 0.9521781424397243, "story_loss": 0.9521781424397243, "grad_norm": 0.17980924248695374, "lr": 0.0004, "elapsed": 358.3, "tokens": 31470, "story_tokens": 31470, "peak_gb": 74.9}
9
+ {"step": 9, "loss": 0.9082017878631088, "story_loss": 0.9082017878631088, "grad_norm": 0.1822422742843628, "lr": 0.00045000000000000004, "elapsed": 383.3, "tokens": 35604, "story_tokens": 35604, "peak_gb": 74.9}
10
+ {"step": 10, "loss": 0.9230904177533711, "story_loss": 0.9230904177533711, "grad_norm": 0.23073534667491913, "lr": 0.0005, "elapsed": 406.3, "tokens": 32519, "story_tokens": 32519, "peak_gb": 74.9}
11
+ {"step": 11, "loss": 0.8867654112128989, "story_loss": 0.8867654112128989, "grad_norm": 0.1905829757452011, "lr": 0.0004999778497139454, "elapsed": 431.5, "tokens": 34972, "story_tokens": 34972, "peak_gb": 74.9}
12
+ {"step": 12, "loss": 0.9049414509247903, "story_loss": 0.9049414509247903, "grad_norm": 0.1806613653898239, "lr": 0.0004999114027808632, "elapsed": 454.1, "tokens": 30931, "story_tokens": 30931, "peak_gb": 74.9}
13
+ {"step": 13, "loss": 0.9156740445577269, "story_loss": 0.9156740445577269, "grad_norm": 0.17583297193050385, "lr": 0.0004998006709753017, "elapsed": 477.4, "tokens": 32487, "story_tokens": 32487, "peak_gb": 74.9}
14
+ {"step": 14, "loss": 0.9271330451354468, "story_loss": 0.9271330451354468, "grad_norm": 0.16018447279930115, "lr": 0.0004996456739191905, "elapsed": 500.4, "tokens": 32785, "story_tokens": 32785, "peak_gb": 74.9}
15
+ {"step": 15, "loss": 0.8834218361938806, "story_loss": 0.8834218361938806, "grad_norm": 0.1666870266199112, "lr": 0.0004994464390783625, "elapsed": 522.7, "tokens": 30777, "story_tokens": 30777, "peak_gb": 74.9}
16
+ {"step": 16, "loss": 0.8534517050397229, "story_loss": 0.8534517050397229, "grad_norm": 0.1500123143196106, "lr": 0.0004992030017576876, "elapsed": 546.5, "tokens": 34127, "story_tokens": 34127, "peak_gb": 74.9}
17
+ {"step": 17, "loss": 0.8924374939025118, "story_loss": 0.8924374939025118, "grad_norm": 0.157627671957016, "lr": 0.0004989154050948158, "elapsed": 586.0, "tokens": 31391, "story_tokens": 31391, "peak_gb": 74.9}
18
+ {"step": 18, "loss": 0.8808594645147811, "story_loss": 0.8808594645147811, "grad_norm": 0.16125229001045227, "lr": 0.0004985837000525343, "elapsed": 609.4, "tokens": 30683, "story_tokens": 30683, "peak_gb": 75.3}
19
+ {"step": 19, "loss": 0.9067726076873921, "story_loss": 0.9067726076873921, "grad_norm": 0.15922409296035767, "lr": 0.0004982079454097354, "elapsed": 631.8, "tokens": 30711, "story_tokens": 30711, "peak_gb": 75.3}
20
+ {"step": 20, "loss": 0.8936638053310594, "story_loss": 0.8936638053310594, "grad_norm": 0.1665307730436325, "lr": 0.0004977882077510018, "elapsed": 654.5, "tokens": 32194, "story_tokens": 32194, "peak_gb": 75.3}
21
+ {"step": 21, "loss": 0.8554054634693471, "story_loss": 0.8554054634693471, "grad_norm": 0.14401096105575562, "lr": 0.0004973245614548072, "elapsed": 679.4, "tokens": 35433, "story_tokens": 35433, "peak_gb": 75.3}
22
+ {"step": 22, "loss": 0.8427083792201155, "story_loss": 0.8427083792201155, "grad_norm": 0.15035314857959747, "lr": 0.0004968170886803362, "elapsed": 703.7, "tokens": 35470, "story_tokens": 35470, "peak_gb": 75.3}
23
+ {"step": 23, "loss": 0.8605454736242477, "story_loss": 0.8605454736242477, "grad_norm": 0.15748392045497894, "lr": 0.0004962658793529257, "elapsed": 726.5, "tokens": 33066, "story_tokens": 33066, "peak_gb": 75.3}
24
+ {"step": 24, "loss": 0.8817875814830437, "story_loss": 0.8817875814830437, "grad_norm": 0.18786607682704926, "lr": 0.0004956710311481302, "elapsed": 750.5, "tokens": 31338, "story_tokens": 31338, "peak_gb": 75.3}
25
+ {"step": 25, "loss": 0.8785581611766096, "story_loss": 0.8785581611766096, "grad_norm": 0.16282476484775543, "lr": 0.0004950326494744127, "elapsed": 772.9, "tokens": 31484, "story_tokens": 31484, "peak_gb": 75.3}
26
+ {"step": 26, "loss": 0.8590605624290347, "story_loss": 0.8590605624290347, "grad_norm": 0.15323121845722198, "lr": 0.0004943508474544667, "elapsed": 795.5, "tokens": 32146, "story_tokens": 32146, "peak_gb": 75.3}
27
+ {"step": 27, "loss": 0.8610151548413393, "story_loss": 0.8610151548413393, "grad_norm": 0.15171723067760468, "lr": 0.0004936257459051702, "elapsed": 818.0, "tokens": 32278, "story_tokens": 32278, "peak_gb": 75.3}
28
+ {"step": 28, "loss": 0.8532832983790305, "story_loss": 0.8532832983790305, "grad_norm": 0.1461639553308487, "lr": 0.0004928574733161775, "elapsed": 840.4, "tokens": 31587, "story_tokens": 31587, "peak_gb": 75.3}
29
+ {"step": 29, "loss": 0.8572696209372258, "story_loss": 0.8572696209372258, "grad_norm": 0.1450178623199463, "lr": 0.0004920461658271492, "elapsed": 864.1, "tokens": 34190, "story_tokens": 34190, "peak_gb": 75.3}
30
+ {"step": 30, "loss": 0.8170471453449683, "story_loss": 0.8170471453449683, "grad_norm": 0.14430557191371918, "lr": 0.000491191967203629, "elapsed": 888.2, "tokens": 34938, "story_tokens": 34938, "peak_gb": 75.3}
31
+ {"step": 31, "loss": 0.8505964095682312, "story_loss": 0.8505964095682312, "grad_norm": 0.14238345623016357, "lr": 0.0004902950288115679, "elapsed": 911.9, "tokens": 33315, "story_tokens": 33315, "peak_gb": 75.3}
32
+ {"step": 32, "loss": 0.8765385061998228, "story_loss": 0.8765385061998228, "grad_norm": 0.14780057966709137, "lr": 0.0004893555095905013, "elapsed": 936.0, "tokens": 34565, "story_tokens": 34565, "peak_gb": 75.3}
33
+ {"step": 33, "loss": 0.8329970671297117, "story_loss": 0.8329970671297117, "grad_norm": 0.16202890872955322, "lr": 0.0004883735760253855, "elapsed": 961.1, "tokens": 35285, "story_tokens": 35285, "peak_gb": 75.3}
34
+ {"step": 34, "loss": 0.8576398669890288, "story_loss": 0.8576398669890288, "grad_norm": 0.15113738179206848, "lr": 0.0004873494021170953, "elapsed": 983.3, "tokens": 30979, "story_tokens": 30979, "peak_gb": 75.3}
35
+ {"step": 35, "loss": 0.869448808005636, "story_loss": 0.869448808005636, "grad_norm": 0.169200599193573, "lr": 0.0004862831693515909, "elapsed": 1006.6, "tokens": 31317, "story_tokens": 31317, "peak_gb": 75.3}
36
+ {"step": 36, "loss": 0.8388230333518611, "story_loss": 0.8388230333518611, "grad_norm": 0.1662723869085312, "lr": 0.00048517506666775835, "elapsed": 1029.7, "tokens": 31962, "story_tokens": 31962, "peak_gb": 75.3}
37
+ {"step": 37, "loss": 0.825441064422641, "story_loss": 0.825441064422641, "grad_norm": 0.152525395154953, "lr": 0.00048402529042392906, "elapsed": 1053.6, "tokens": 33637, "story_tokens": 33637, "peak_gb": 75.3}
38
+ {"step": 38, "loss": 0.8333948546638701, "story_loss": 0.8333948546638701, "grad_norm": 0.17343184351921082, "lr": 0.0004828340443630846, "elapsed": 1076.2, "tokens": 32599, "story_tokens": 32599, "peak_gb": 75.3}
39
+ {"step": 39, "loss": 0.8262715324143971, "story_loss": 0.8262715324143971, "grad_norm": 0.1531706005334854, "lr": 0.00048160153957675314, "elapsed": 1100.3, "tokens": 34111, "story_tokens": 34111, "peak_gb": 75.3}
40
+ {"step": 40, "loss": 0.844073306150542, "story_loss": 0.844073306150542, "grad_norm": 0.16540683805942535, "lr": 0.0004803279944676032, "elapsed": 1122.2, "tokens": 30352, "story_tokens": 30352, "peak_gb": 75.3}
41
+ {"step": 41, "loss": 0.8440567282101157, "story_loss": 0.8440567282101157, "grad_norm": 0.16075150668621063, "lr": 0.0004790136347107427, "elapsed": 1145.5, "tokens": 33851, "story_tokens": 33851, "peak_gb": 75.3}
42
+ {"step": 42, "loss": 0.8403269094698729, "story_loss": 0.8403269094698729, "grad_norm": 0.15686336159706116, "lr": 0.00047765869321372834, "elapsed": 1168.6, "tokens": 33367, "story_tokens": 33367, "peak_gb": 75.3}
43
+ {"step": 43, "loss": 0.8560388736791343, "story_loss": 0.8560388736791343, "grad_norm": 0.16111308336257935, "lr": 0.00047626341007529397, "elapsed": 1192.0, "tokens": 33276, "story_tokens": 33276, "peak_gb": 75.3}
44
+ {"step": 44, "loss": 0.8542610675043282, "story_loss": 0.8542610675043282, "grad_norm": 0.1642204076051712, "lr": 0.0004748280325428048, "elapsed": 1214.8, "tokens": 31913, "story_tokens": 31913, "peak_gb": 75.3}
45
+ {"step": 45, "loss": 0.8363254381262738, "story_loss": 0.8363254381262738, "grad_norm": 0.1502709686756134, "lr": 0.0004733528149684444, "elapsed": 1238.0, "tokens": 32384, "story_tokens": 32384, "peak_gb": 75.3}
46
+ {"step": 46, "loss": 0.8555894578203576, "story_loss": 0.8555894578203576, "grad_norm": 0.15465646982192993, "lr": 0.0004718380187641429, "elapsed": 1260.6, "tokens": 32612, "story_tokens": 32612, "peak_gb": 75.3}
47
+ {"step": 47, "loss": 0.8423190541471152, "story_loss": 0.8423190541471152, "grad_norm": 0.15549930930137634, "lr": 0.0004702839123552541, "elapsed": 1283.7, "tokens": 33512, "story_tokens": 33512, "peak_gb": 75.3}
48
+ {"step": 48, "loss": 0.8319239414633206, "story_loss": 0.8319239414633206, "grad_norm": 0.1496901512145996, "lr": 0.0004686907711329903, "elapsed": 1306.5, "tokens": 31622, "story_tokens": 31622, "peak_gb": 75.3}
49
+ {"step": 49, "loss": 0.8520943629170388, "story_loss": 0.8520943629170388, "grad_norm": 0.17192073166370392, "lr": 0.0004670588774056218, "elapsed": 1328.6, "tokens": 31071, "story_tokens": 31071, "peak_gb": 75.3}
50
+ {"step": 50, "loss": 0.8262458530108224, "story_loss": 0.8262458530108224, "grad_norm": 0.14792464673519135, "lr": 0.0004653885203484515, "elapsed": 1352.7, "tokens": 35170, "story_tokens": 35170, "peak_gb": 75.3}
51
+ {"step": 51, "loss": 0.8476749332568783, "story_loss": 0.8476749332568783, "grad_norm": 0.15968023240566254, "lr": 0.0004636799959525726, "elapsed": 1374.9, "tokens": 29395, "story_tokens": 29395, "peak_gb": 75.3}
52
+ {"step": 52, "loss": 0.8577089814649173, "story_loss": 0.8577089814649173, "grad_norm": 0.15932931005954742, "lr": 0.00046193360697241773, "elapsed": 1397.9, "tokens": 31880, "story_tokens": 31880, "peak_gb": 75.3}
53
+ {"step": 53, "loss": 0.8385722767842894, "story_loss": 0.8385722767842894, "grad_norm": 0.15618562698364258, "lr": 0.0004601496628721108, "elapsed": 1419.6, "tokens": 31329, "story_tokens": 31329, "peak_gb": 75.3}
54
+ {"step": 54, "loss": 0.8305302046287447, "story_loss": 0.8305302046287447, "grad_norm": 0.15059523284435272, "lr": 0.00045832847977062875, "elapsed": 1443.8, "tokens": 35183, "story_tokens": 35183, "peak_gb": 75.3}
55
+ {"step": 55, "loss": 0.8634529668625929, "story_loss": 0.8634529668625929, "grad_norm": 0.1571495234966278, "lr": 0.0004564703803857849, "elapsed": 1466.2, "tokens": 32038, "story_tokens": 32038, "peak_gb": 75.3}
56
+ {"step": 56, "loss": 0.8563261283696024, "story_loss": 0.8563261283696024, "grad_norm": 0.15641771256923676, "lr": 0.00045457569397704226, "elapsed": 1488.5, "tokens": 32144, "story_tokens": 32144, "peak_gb": 75.3}
57
+ {"step": 57, "loss": 0.825990942481833, "story_loss": 0.825990942481833, "grad_norm": 0.15733702480793, "lr": 0.0004526447562871685, "elapsed": 1510.6, "tokens": 31066, "story_tokens": 31066, "peak_gb": 75.3}
58
+ {"step": 58, "loss": 0.8359455564246351, "story_loss": 0.8359455564246351, "grad_norm": 0.15097084641456604, "lr": 0.0004506779094827409, "elapsed": 1534.4, "tokens": 33910, "story_tokens": 33910, "peak_gb": 75.3}
59
+ {"step": 59, "loss": 0.8379332135873012, "story_loss": 0.8379332135873012, "grad_norm": 0.15597373247146606, "lr": 0.0004486755020935144, "elapsed": 1556.7, "tokens": 32128, "story_tokens": 32128, "peak_gb": 75.3}
60
+ {"step": 60, "loss": 0.8428746518695273, "story_loss": 0.8428746518695273, "grad_norm": 0.15208080410957336, "lr": 0.00044663788895066064, "elapsed": 1579.4, "tokens": 33690, "story_tokens": 33690, "peak_gb": 75.3}
61
+ {"step": 61, "loss": 0.8320670731935745, "story_loss": 0.8320670731935745, "grad_norm": 0.15223033726215363, "lr": 0.0004445654311238915, "elapsed": 1602.6, "tokens": 33367, "story_tokens": 33367, "peak_gb": 75.3}
62
+ {"step": 62, "loss": 0.8277551398088713, "story_loss": 0.8277551398088713, "grad_norm": 0.161062553524971, "lr": 0.00044245849585747656, "elapsed": 1625.9, "tokens": 32949, "story_tokens": 32949, "peak_gb": 75.3}
63
+ {"step": 63, "loss": 0.8412959490448799, "story_loss": 0.8412959490448799, "grad_norm": 0.15252742171287537, "lr": 0.00044031745650516664, "elapsed": 1649.4, "tokens": 34194, "story_tokens": 34194, "peak_gb": 75.3}
64
+ {"step": 64, "loss": 0.8638413783024497, "story_loss": 0.8638413783024497, "grad_norm": 0.17437246441841125, "lr": 0.0004381426924640346, "elapsed": 1671.8, "tokens": 30484, "story_tokens": 30484, "peak_gb": 75.3}
65
+ {"step": 65, "loss": 0.8634195473489907, "story_loss": 0.8634195473489907, "grad_norm": 0.1650637835264206, "lr": 0.00043593458910724537, "elapsed": 1694.3, "tokens": 30888, "story_tokens": 30888, "peak_gb": 75.3}
66
+ {"step": 66, "loss": 0.8200508999076539, "story_loss": 0.8200508999076539, "grad_norm": 0.14664511382579803, "lr": 0.0004336935377157668, "elapsed": 1718.7, "tokens": 35126, "story_tokens": 35126, "peak_gb": 75.3}
67
+ {"step": 67, "loss": 0.8429464476793955, "story_loss": 0.8429464476793955, "grad_norm": 0.1497684270143509, "lr": 0.00043141993540903403, "elapsed": 1741.8, "tokens": 33585, "story_tokens": 33585, "peak_gb": 75.3}
68
+ {"step": 68, "loss": 0.8455229745530082, "story_loss": 0.8455229745530082, "grad_norm": 0.1550675481557846, "lr": 0.0004291141850745788, "elapsed": 1764.1, "tokens": 31705, "story_tokens": 31705, "peak_gb": 75.3}
69
+ {"step": 69, "loss": 0.8293636872092001, "story_loss": 0.8293636872092001, "grad_norm": 0.151563361287117, "lr": 0.00042677669529663686, "elapsed": 1786.5, "tokens": 31970, "story_tokens": 31970, "peak_gb": 75.3}
70
+ {"step": 70, "loss": 0.8334787077190765, "story_loss": 0.8334787077190765, "grad_norm": 0.1585901975631714, "lr": 0.0004244078802837462, "elapsed": 1809.2, "tokens": 32831, "story_tokens": 32831, "peak_gb": 75.3}
71
+ {"step": 71, "loss": 0.8620012146553129, "story_loss": 0.8620012146553129, "grad_norm": 0.17240722477436066, "lr": 0.0004220081597953479, "elapsed": 1833.3, "tokens": 34515, "story_tokens": 34515, "peak_gb": 75.3}
72
+ {"step": 72, "loss": 0.8486029344163837, "story_loss": 0.8486029344163837, "grad_norm": 0.15794585645198822, "lr": 0.000419577959067404, "elapsed": 1856.5, "tokens": 32546, "story_tokens": 32546, "peak_gb": 75.3}
73
+ {"step": 73, "loss": 0.8284024878420596, "story_loss": 0.8284024878420596, "grad_norm": 0.15643438696861267, "lr": 0.0004171177087370451, "elapsed": 1881.0, "tokens": 35310, "story_tokens": 35310, "peak_gb": 75.3}
74
+ {"step": 74, "loss": 0.8190097905883154, "story_loss": 0.8190097905883154, "grad_norm": 0.1592734158039093, "lr": 0.0004146278447662597, "elapsed": 1903.6, "tokens": 32626, "story_tokens": 32626, "peak_gb": 75.3}
75
+ {"step": 75, "loss": 0.852308667016166, "story_loss": 0.852308667016166, "grad_norm": 0.1720048040151596, "lr": 0.00041210880836464134, "elapsed": 1925.6, "tokens": 31563, "story_tokens": 31563, "peak_gb": 75.3}
76
+ {"step": 76, "loss": 0.8477594443549398, "story_loss": 0.8477594443549398, "grad_norm": 0.17302818596363068, "lr": 0.00040956104591120503, "elapsed": 1947.8, "tokens": 30938, "story_tokens": 30938, "peak_gb": 75.3}
77
+ {"step": 77, "loss": 0.8696117050146374, "story_loss": 0.8696117050146374, "grad_norm": 0.16640624403953552, "lr": 0.000406985008875288, "elapsed": 1969.6, "tokens": 30487, "story_tokens": 30487, "peak_gb": 75.3}
78
+ {"step": 78, "loss": 0.8573416948828362, "story_loss": 0.8573416948828362, "grad_norm": 0.15910813212394714, "lr": 0.000404381153736548, "elapsed": 1991.3, "tokens": 31329, "story_tokens": 31329, "peak_gb": 75.3}
79
+ {"step": 79, "loss": 0.8445997707281739, "story_loss": 0.8445997707281739, "grad_norm": 0.15996187925338745, "lr": 0.00040174994190407445, "elapsed": 2013.8, "tokens": 30534, "story_tokens": 30534, "peak_gb": 75.3}
80
+ {"step": 80, "loss": 0.8245588861053126, "story_loss": 0.8245588861053126, "grad_norm": 0.1535836011171341, "lr": 0.00039909183963462535, "elapsed": 2036.2, "tokens": 32635, "story_tokens": 32635, "peak_gb": 75.3}
81
+ {"step": 81, "loss": 0.8368537541828819, "story_loss": 0.8368537541828819, "grad_norm": 0.1570577472448349, "lr": 0.00039640731795000583, "elapsed": 2058.3, "tokens": 31574, "story_tokens": 31574, "peak_gb": 75.3}
82
+ {"step": 82, "loss": 0.851953058997379, "story_loss": 0.851953058997379, "grad_norm": 0.16823254525661469, "lr": 0.0003936968525536018, "elapsed": 2080.3, "tokens": 31811, "story_tokens": 31811, "peak_gb": 75.3}
83
+ {"step": 83, "loss": 0.8086542361543206, "story_loss": 0.8086542361543206, "grad_norm": 0.15658898651599884, "lr": 0.000390960923746084, "elapsed": 2104.2, "tokens": 34353, "story_tokens": 34353, "peak_gb": 75.3}
84
+ {"step": 84, "loss": 0.8518965409498059, "story_loss": 0.8518965409498059, "grad_norm": 0.16363529860973358, "lr": 0.0003882000163402983, "elapsed": 2126.6, "tokens": 31692, "story_tokens": 31692, "peak_gb": 75.3}
85
+ {"step": 85, "loss": 0.8280902416110371, "story_loss": 0.8280902416110371, "grad_norm": 0.15967796742916107, "lr": 0.00038541461957535526, "elapsed": 2149.1, "tokens": 32282, "story_tokens": 32282, "peak_gb": 75.3}
86
+ {"step": 86, "loss": 0.8380647069889814, "story_loss": 0.8380647069889814, "grad_norm": 0.15645931661128998, "lr": 0.0003826052270299356, "elapsed": 2171.3, "tokens": 30959, "story_tokens": 30959, "peak_gb": 75.3}
87
+ {"step": 87, "loss": 0.8221341686181074, "story_loss": 0.8221341686181074, "grad_norm": 0.15080031752586365, "lr": 0.0003797723365348277, "elapsed": 2194.1, "tokens": 33242, "story_tokens": 33242, "peak_gb": 75.3}
88
+ {"step": 88, "loss": 0.868239146688505, "story_loss": 0.868239146688505, "grad_norm": 0.15448834002017975, "lr": 0.00037691645008471, "elapsed": 2216.0, "tokens": 32340, "story_tokens": 32340, "peak_gb": 75.3}
89
+ {"step": 89, "loss": 0.8338555502442394, "story_loss": 0.8338555502442394, "grad_norm": 0.15862321853637695, "lr": 0.0003740380737491971, "elapsed": 2238.7, "tokens": 32115, "story_tokens": 32115, "peak_gb": 75.3}
90
+ {"step": 90, "loss": 0.8112607030193781, "story_loss": 0.8112607030193781, "grad_norm": 0.14962783455848694, "lr": 0.00037113771758316255, "elapsed": 2261.6, "tokens": 33285, "story_tokens": 33285, "peak_gb": 75.3}
91
+ {"step": 91, "loss": 0.8503162339786408, "story_loss": 0.8503162339786408, "grad_norm": 0.16036362946033478, "lr": 0.00036821589553635636, "elapsed": 2283.7, "tokens": 31813, "story_tokens": 31813, "peak_gb": 75.3}
92
+ {"step": 92, "loss": 0.8378138970401867, "story_loss": 0.8378138970401867, "grad_norm": 0.15184146165847778, "lr": 0.00036527312536233146, "elapsed": 2308.1, "tokens": 35755, "story_tokens": 35755, "peak_gb": 75.3}
93
+ {"step": 93, "loss": 0.8281137770848399, "story_loss": 0.8281137770848399, "grad_norm": 0.15344074368476868, "lr": 0.0003623099285266972, "elapsed": 2330.3, "tokens": 31804, "story_tokens": 31804, "peak_gb": 75.3}
94
+ {"step": 94, "loss": 0.815315764304571, "story_loss": 0.815315764304571, "grad_norm": 0.154253289103508, "lr": 0.0003593268301147139, "elapsed": 2354.3, "tokens": 33997, "story_tokens": 33997, "peak_gb": 75.3}
95
+ {"step": 95, "loss": 0.8061545180932467, "story_loss": 0.8061545180932467, "grad_norm": 0.15004394948482513, "lr": 0.0003563243587382468, "elapsed": 2378.0, "tokens": 34583, "story_tokens": 34583, "peak_gb": 75.3}
96
+ {"step": 96, "loss": 0.8349124817662869, "story_loss": 0.8349124817662869, "grad_norm": 0.15697138011455536, "lr": 0.00035330304644209454, "elapsed": 2401.1, "tokens": 32649, "story_tokens": 32649, "peak_gb": 75.3}
97
+ {"step": 97, "loss": 0.8305270802148849, "story_loss": 0.8305270802148849, "grad_norm": 0.1626608967781067, "lr": 0.0003502634286097104, "elapsed": 2423.0, "tokens": 32276, "story_tokens": 32276, "peak_gb": 75.3}
98
+ {"step": 98, "loss": 0.8140718494467872, "story_loss": 0.8140718494467872, "grad_norm": 0.15714828670024872, "lr": 0.0003472060438683302, "elapsed": 2445.1, "tokens": 30176, "story_tokens": 30176, "peak_gb": 75.3}
99
+ {"step": 99, "loss": 0.8216713298911362, "story_loss": 0.8216713298911362, "grad_norm": 0.1528034210205078, "lr": 0.00034413143399352657, "elapsed": 2468.4, "tokens": 33132, "story_tokens": 33132, "peak_gb": 75.3}
100
+ {"step": 100, "loss": 0.8291707092283294, "story_loss": 0.8291707092283294, "grad_norm": 0.15895435214042664, "lr": 0.00034104014381320557, "elapsed": 2490.7, "tokens": 32769, "story_tokens": 32769, "peak_gb": 75.3}
101
+ {"step": 101, "loss": 0.8354451223225492, "story_loss": 0.8354451223225492, "grad_norm": 0.15563397109508514, "lr": 0.0003379327211110612, "elapsed": 2512.7, "tokens": 30276, "story_tokens": 30276, "peak_gb": 75.3}
102
+ {"step": 102, "loss": 0.8557112819833066, "story_loss": 0.8557112819833066, "grad_norm": 0.1575516015291214, "lr": 0.00033480971652950753, "elapsed": 2536.2, "tokens": 33389, "story_tokens": 33389, "peak_gb": 75.3}
103
+ {"step": 103, "loss": 0.8356353683196217, "story_loss": 0.8356353683196217, "grad_norm": 0.15461057424545288, "lr": 0.00033167168347210326, "elapsed": 2559.7, "tokens": 32682, "story_tokens": 32682, "peak_gb": 75.3}
104
+ {"step": 104, "loss": 0.8481633242549403, "story_loss": 0.8481633242549403, "grad_norm": 0.1651352494955063, "lr": 0.00032851917800548724, "elapsed": 2581.2, "tokens": 30438, "story_tokens": 30438, "peak_gb": 75.3}
105
+ {"step": 105, "loss": 0.8485507579046779, "story_loss": 0.8485507579046779, "grad_norm": 0.15269309282302856, "lr": 0.00032535275876084246, "elapsed": 2603.7, "tokens": 32461, "story_tokens": 32461, "peak_gb": 75.3}
106
+ {"step": 106, "loss": 0.8489869254301686, "story_loss": 0.8489869254301686, "grad_norm": 0.16012659668922424, "lr": 0.00032217298683490526, "elapsed": 2624.9, "tokens": 30119, "story_tokens": 30119, "peak_gb": 75.3}
107
+ {"step": 107, "loss": 0.8185698046319689, "story_loss": 0.8185698046319689, "grad_norm": 0.1592843234539032, "lr": 0.00031898042569053763, "elapsed": 2647.0, "tokens": 32358, "story_tokens": 32358, "peak_gb": 75.3}
108
+ {"step": 108, "loss": 0.8400696491859914, "story_loss": 0.8400696491859914, "grad_norm": 0.16202236711978912, "lr": 0.0003157756410568803, "elapsed": 2669.3, "tokens": 32148, "story_tokens": 32148, "peak_gb": 75.3}
109
+ {"step": 109, "loss": 0.8072051270467552, "story_loss": 0.8072051270467552, "grad_norm": 0.1513069123029709, "lr": 0.0003125592008291044, "elapsed": 2693.4, "tokens": 33376, "story_tokens": 33376, "peak_gb": 75.3}
110
+ {"step": 110, "loss": 0.8550524311616483, "story_loss": 0.8550524311616483, "grad_norm": 0.16262932121753693, "lr": 0.00030933167496777875, "elapsed": 2715.3, "tokens": 30542, "story_tokens": 30542, "peak_gb": 75.3}
111
+ {"step": 111, "loss": 0.8087665695208152, "story_loss": 0.8087665695208152, "grad_norm": 0.1608530580997467, "lr": 0.000306093635397872, "elapsed": 2739.6, "tokens": 34440, "story_tokens": 34440, "peak_gb": 75.3}
112
+ {"step": 112, "loss": 0.8434188725003247, "story_loss": 0.8434188725003247, "grad_norm": 0.1677221953868866, "lr": 0.00030284565590740607, "elapsed": 2761.6, "tokens": 30318, "story_tokens": 30318, "peak_gb": 75.3}
113
+ {"step": 113, "loss": 0.8380435742072785, "story_loss": 0.8380435742072785, "grad_norm": 0.1541326642036438, "lr": 0.00029958831204577945, "elapsed": 2783.9, "tokens": 32349, "story_tokens": 32349, "peak_gb": 75.3}
114
+ {"step": 114, "loss": 0.8540095046890578, "story_loss": 0.8540095046890578, "grad_norm": 0.16465707123279572, "lr": 0.0002963221810217786, "elapsed": 2805.7, "tokens": 31093, "story_tokens": 31093, "peak_gb": 75.3}
115
+ {"step": 115, "loss": 0.804045649185029, "story_loss": 0.804045649185029, "grad_norm": 0.14407837390899658, "lr": 0.00029304784160129533, "elapsed": 2829.7, "tokens": 34196, "story_tokens": 34196, "peak_gb": 75.3}
116
+ {"step": 116, "loss": 0.8471902059335195, "story_loss": 0.8471902059335195, "grad_norm": 0.15335124731063843, "lr": 0.000289765874004768, "elapsed": 2851.4, "tokens": 31795, "story_tokens": 31795, "peak_gb": 75.3}
117
+ {"step": 117, "loss": 0.813437671622242, "story_loss": 0.813437671622242, "grad_norm": 0.1445750594139099, "lr": 0.0002864768598043654, "elapsed": 2875.1, "tokens": 33629, "story_tokens": 33629, "peak_gb": 75.3}
118
+ {"step": 118, "loss": 0.8138979825784688, "story_loss": 0.8138979825784688, "grad_norm": 0.14780844748020172, "lr": 0.00028318138182093053, "elapsed": 2897.7, "tokens": 32728, "story_tokens": 32728, "peak_gb": 75.3}
119
+ {"step": 119, "loss": 0.7870701631626779, "story_loss": 0.7870701631626779, "grad_norm": 0.1423650085926056, "lr": 0.00027988002402070343, "elapsed": 2922.2, "tokens": 33939, "story_tokens": 33939, "peak_gb": 75.3}
120
+ {"step": 120, "loss": 0.8158486022098322, "story_loss": 0.8158486022098322, "grad_norm": 0.15064048767089844, "lr": 0.00027657337141184134, "elapsed": 2945.3, "tokens": 32996, "story_tokens": 32996, "peak_gb": 75.3}
121
+ {"step": 121, "loss": 0.8229700658202418, "story_loss": 0.8229700658202418, "grad_norm": 0.16263344883918762, "lr": 0.0002732620099407536, "elapsed": 2968.0, "tokens": 30403, "story_tokens": 30403, "peak_gb": 75.3}
122
+ {"step": 122, "loss": 0.8050408018188387, "story_loss": 0.8050408018188387, "grad_norm": 0.14663365483283997, "lr": 0.0002699465263882708, "elapsed": 2991.6, "tokens": 34165, "story_tokens": 34165, "peak_gb": 75.3}
123
+ {"step": 123, "loss": 0.829060279910822, "story_loss": 0.829060279910822, "grad_norm": 0.14861217141151428, "lr": 0.00026662750826566563, "elapsed": 3014.3, "tokens": 33171, "story_tokens": 33171, "peak_gb": 75.3}
124
+ {"step": 124, "loss": 0.8077881715392655, "story_loss": 0.8077881715392655, "grad_norm": 0.15767088532447815, "lr": 0.00026330554371054466, "elapsed": 3037.0, "tokens": 33369, "story_tokens": 33369, "peak_gb": 75.3}
125
+ {"step": 125, "loss": 0.8107897521188289, "story_loss": 0.8107897521188289, "grad_norm": 0.14908701181411743, "lr": 0.0002599812213826294, "elapsed": 3060.2, "tokens": 34394, "story_tokens": 34394, "peak_gb": 75.3}
126
+ {"step": 126, "loss": 0.8630613394023099, "story_loss": 0.8630613394023099, "grad_norm": 0.15326441824436188, "lr": 0.00025665513035944373, "elapsed": 3084.5, "tokens": 31971, "story_tokens": 31971, "peak_gb": 75.3}
127
+ {"step": 127, "loss": 0.8238155072004238, "story_loss": 0.8238155072004238, "grad_norm": 0.1460893750190735, "lr": 0.00025332786003192845, "elapsed": 3105.6, "tokens": 31144, "story_tokens": 31144, "peak_gb": 75.3}
128
+ {"step": 128, "loss": 0.8195277611733901, "story_loss": 0.8195277611733901, "grad_norm": 0.14707891643047333, "lr": 0.00025, "elapsed": 3128.5, "tokens": 31609, "story_tokens": 31609, "peak_gb": 75.3}
129
+ {"step": 129, "loss": 0.8242616416162863, "story_loss": 0.8242616416162863, "grad_norm": 0.14772160351276398, "lr": 0.0002466721399680716, "elapsed": 3151.1, "tokens": 32234, "story_tokens": 32234, "peak_gb": 75.3}
130
+ {"step": 130, "loss": 0.8240524082815532, "story_loss": 0.8240524082815532, "grad_norm": 0.1471024453639984, "lr": 0.00024334486964055634, "elapsed": 3173.8, "tokens": 32972, "story_tokens": 32972, "peak_gb": 75.3}
131
+ {"step": 131, "loss": 0.8176650436850308, "story_loss": 0.8176650436850308, "grad_norm": 0.1485569179058075, "lr": 0.00024001877861737068, "elapsed": 3196.1, "tokens": 32323, "story_tokens": 32323, "peak_gb": 75.3}
132
+ {"step": 132, "loss": 0.8262463463462473, "story_loss": 0.8262463463462473, "grad_norm": 0.16271239519119263, "lr": 0.0002366944562894554, "elapsed": 3220.2, "tokens": 35514, "story_tokens": 35514, "peak_gb": 75.3}
133
+ {"step": 133, "loss": 0.8108057669958989, "story_loss": 0.8108057669958989, "grad_norm": 0.14595337212085724, "lr": 0.00023337249173433446, "elapsed": 3243.5, "tokens": 33680, "story_tokens": 33680, "peak_gb": 75.3}
134
+ {"step": 134, "loss": 0.8052474204981183, "story_loss": 0.8052474204981183, "grad_norm": 0.14752700924873352, "lr": 0.0002300534736117292, "elapsed": 3264.2, "tokens": 29959, "story_tokens": 29959, "peak_gb": 75.3}
135
+ {"step": 135, "loss": 0.8132915028246765, "story_loss": 0.8132915028246765, "grad_norm": 0.14604948461055756, "lr": 0.00022673799005924639, "elapsed": 3285.5, "tokens": 29796, "story_tokens": 29796, "peak_gb": 75.3}
136
+ {"step": 136, "loss": 0.8148858868580126, "story_loss": 0.8148858868580126, "grad_norm": 0.1465892493724823, "lr": 0.00022342662858815867, "elapsed": 3308.4, "tokens": 33407, "story_tokens": 33407, "peak_gb": 75.3}
137
+ {"step": 137, "loss": 0.7986049568075441, "story_loss": 0.7986049568075441, "grad_norm": 0.14476612210273743, "lr": 0.00022011997597929658, "elapsed": 3330.5, "tokens": 31780, "story_tokens": 31780, "peak_gb": 75.3}
138
+ {"step": 138, "loss": 0.8577996936850398, "story_loss": 0.8577996936850398, "grad_norm": 0.15360797941684723, "lr": 0.00021681861817906954, "elapsed": 3352.6, "tokens": 31654, "story_tokens": 31654, "peak_gb": 75.3}
139
+ {"step": 139, "loss": 0.8348320425437742, "story_loss": 0.8348320425437742, "grad_norm": 0.15058554708957672, "lr": 0.00021352314019563458, "elapsed": 3374.0, "tokens": 30783, "story_tokens": 30783, "peak_gb": 75.3}
140
+ {"step": 140, "loss": 0.8478618484484014, "story_loss": 0.8478618484484014, "grad_norm": 0.16908875107765198, "lr": 0.00021023412599523202, "elapsed": 3395.3, "tokens": 30386, "story_tokens": 30386, "peak_gb": 75.3}
141
+ {"step": 141, "loss": 0.8081007013744382, "story_loss": 0.8081007013744382, "grad_norm": 0.14641132950782776, "lr": 0.00020695215839870476, "elapsed": 3417.7, "tokens": 31597, "story_tokens": 31597, "peak_gb": 75.3}
142
+ {"step": 142, "loss": 0.8274444727312494, "story_loss": 0.8274444727312494, "grad_norm": 0.1525236964225769, "lr": 0.00020367781897822146, "elapsed": 3439.5, "tokens": 30469, "story_tokens": 30469, "peak_gb": 75.3}
143
+ {"step": 143, "loss": 0.808722809653432, "story_loss": 0.808722809653432, "grad_norm": 0.4334540069103241, "lr": 0.00020041168795422053, "elapsed": 3462.6, "tokens": 34389, "story_tokens": 34389, "peak_gb": 75.3}
144
+ {"step": 144, "loss": 0.8022677960022477, "story_loss": 0.8022677960022477, "grad_norm": 0.15992845594882965, "lr": 0.0001971543440925939, "elapsed": 3484.7, "tokens": 31420, "story_tokens": 31420, "peak_gb": 75.3}
145
+ {"step": 145, "loss": 0.8066154308875768, "story_loss": 0.8066154308875768, "grad_norm": 0.1472889930009842, "lr": 0.000193906364602128, "elapsed": 3507.4, "tokens": 32550, "story_tokens": 32550, "peak_gb": 75.3}
146
+ {"step": 146, "loss": 0.806976486819192, "story_loss": 0.806976486819192, "grad_norm": 0.15106812119483948, "lr": 0.00019066832503222128, "elapsed": 3530.1, "tokens": 32427, "story_tokens": 32427, "peak_gb": 75.3}
147
+ {"step": 147, "loss": 0.7960578287636252, "story_loss": 0.7960578287636252, "grad_norm": 0.13877449929714203, "lr": 0.00018744079917089567, "elapsed": 3553.3, "tokens": 34082, "story_tokens": 34082, "peak_gb": 75.3}
148
+ {"step": 148, "loss": 0.8192035589265303, "story_loss": 0.8192035589265303, "grad_norm": 0.1447344571352005, "lr": 0.00018422435894311972, "elapsed": 3575.8, "tokens": 32151, "story_tokens": 32151, "peak_gb": 75.3}
149
+ {"step": 149, "loss": 0.800012058120723, "story_loss": 0.800012058120723, "grad_norm": 0.13897331058979034, "lr": 0.00018101957430946238, "elapsed": 3599.5, "tokens": 33610, "story_tokens": 33610, "peak_gb": 75.3}
150
+ {"step": 150, "loss": 0.8096021044594476, "story_loss": 0.8096021044594476, "grad_norm": 0.14159497618675232, "lr": 0.0001778270131650948, "elapsed": 3622.4, "tokens": 32065, "story_tokens": 32065, "peak_gb": 75.3}
151
+ {"step": 151, "loss": 0.8158414083780032, "story_loss": 0.8158414083780032, "grad_norm": 0.14154046773910522, "lr": 0.0001746472412391576, "elapsed": 3646.3, "tokens": 35795, "story_tokens": 35795, "peak_gb": 75.3}
152
+ {"step": 152, "loss": 0.8015779174533594, "story_loss": 0.8015779174533594, "grad_norm": 0.14311300218105316, "lr": 0.00017148082199451288, "elapsed": 3669.0, "tokens": 32871, "story_tokens": 32871, "peak_gb": 75.3}
153
+ {"step": 153, "loss": 0.8339176057073702, "story_loss": 0.8339176057073702, "grad_norm": 0.15274780988693237, "lr": 0.00016832831652789673, "elapsed": 3691.5, "tokens": 31709, "story_tokens": 31709, "peak_gb": 75.3}
154
+ {"step": 154, "loss": 0.8172518937900135, "story_loss": 0.8172518937900135, "grad_norm": 0.14585155248641968, "lr": 0.00016519028347049242, "elapsed": 3714.4, "tokens": 32489, "story_tokens": 32489, "peak_gb": 75.3}
155
+ {"step": 155, "loss": 0.8015455770707236, "story_loss": 0.8015455770707236, "grad_norm": 0.14263658225536346, "lr": 0.00016206727888893875, "elapsed": 3737.4, "tokens": 33539, "story_tokens": 33539, "peak_gb": 75.3}
156
+ {"step": 156, "loss": 0.802377022581194, "story_loss": 0.802377022581194, "grad_norm": 0.14570078253746033, "lr": 0.00015895985618679444, "elapsed": 3759.8, "tokens": 32507, "story_tokens": 32507, "peak_gb": 75.3}
157
+ {"step": 157, "loss": 0.8144865072832145, "story_loss": 0.8144865072832145, "grad_norm": 0.1418294608592987, "lr": 0.00015586856600647344, "elapsed": 3782.7, "tokens": 33152, "story_tokens": 33152, "peak_gb": 75.3}
158
+ {"step": 158, "loss": 0.8290554467215663, "story_loss": 0.8290554467215663, "grad_norm": 0.14152002334594727, "lr": 0.00015279395613166985, "elapsed": 3804.9, "tokens": 33178, "story_tokens": 33178, "peak_gb": 75.3}
159
+ {"step": 159, "loss": 0.801832526422468, "story_loss": 0.801832526422468, "grad_norm": 0.14825712144374847, "lr": 0.00014973657139028963, "elapsed": 3828.3, "tokens": 32381, "story_tokens": 32381, "peak_gb": 75.3}
160
+ {"step": 160, "loss": 0.8106551278530046, "story_loss": 0.8106551278530046, "grad_norm": 0.14185035228729248, "lr": 0.00014669695355790552, "elapsed": 3851.1, "tokens": 32882, "story_tokens": 32882, "peak_gb": 75.3}
161
+ {"step": 161, "loss": 0.8033409932192187, "story_loss": 0.8033409932192187, "grad_norm": 0.14320099353790283, "lr": 0.00014367564126175337, "elapsed": 3874.4, "tokens": 34136, "story_tokens": 34136, "peak_gb": 75.3}
162
+ {"step": 162, "loss": 0.8341546033704277, "story_loss": 0.8341546033704277, "grad_norm": 0.14824973046779633, "lr": 0.00014067316988528616, "elapsed": 3895.5, "tokens": 30387, "story_tokens": 30387, "peak_gb": 75.3}
163
+ {"step": 163, "loss": 0.8176099632473108, "story_loss": 0.8176099632473108, "grad_norm": 0.1459539532661438, "lr": 0.00013769007147330283, "elapsed": 3918.7, "tokens": 33654, "story_tokens": 33654, "peak_gb": 75.3}
164
+ {"step": 164, "loss": 0.8077658745331825, "story_loss": 0.8077658745331825, "grad_norm": 0.14022183418273926, "lr": 0.0001347268746376685, "elapsed": 3940.7, "tokens": 32099, "story_tokens": 32099, "peak_gb": 75.3}
165
+ {"step": 165, "loss": 0.8156514487528148, "story_loss": 0.8156514487528148, "grad_norm": 0.14684364199638367, "lr": 0.00013178410446364376, "elapsed": 3963.7, "tokens": 32973, "story_tokens": 32973, "peak_gb": 75.3}
166
+ {"step": 166, "loss": 0.7864599877581311, "story_loss": 0.7864599877581311, "grad_norm": 0.14636602997779846, "lr": 0.0001288622824168375, "elapsed": 3987.3, "tokens": 33498, "story_tokens": 33498, "peak_gb": 75.3}
167
+ {"step": 167, "loss": 0.7902932412635175, "story_loss": 0.7902932412635175, "grad_norm": 0.13796798884868622, "lr": 0.00012596192625080293, "elapsed": 4011.4, "tokens": 35047, "story_tokens": 35047, "peak_gb": 75.3}
168
+ {"step": 168, "loss": 0.8296307791831037, "story_loss": 0.8296307791831037, "grad_norm": 0.1463257074356079, "lr": 0.00012308354991529008, "elapsed": 4034.2, "tokens": 31845, "story_tokens": 31845, "peak_gb": 75.3}
169
+ {"step": 169, "loss": 0.7917714742652548, "story_loss": 0.7917714742652548, "grad_norm": 0.13324908912181854, "lr": 0.00012022766346517238, "elapsed": 4058.0, "tokens": 35161, "story_tokens": 35161, "peak_gb": 75.3}
170
+ {"step": 170, "loss": 0.8087049537202151, "story_loss": 0.8087049537202151, "grad_norm": 0.14854657649993896, "lr": 0.0001173947729700644, "elapsed": 4079.9, "tokens": 32420, "story_tokens": 32420, "peak_gb": 75.3}
171
+ {"step": 171, "loss": 0.7805401340213438, "story_loss": 0.7805401340213438, "grad_norm": 0.13408112525939941, "lr": 0.00011458538042464475, "elapsed": 4103.9, "tokens": 34647, "story_tokens": 34647, "peak_gb": 75.3}
172
+ {"step": 172, "loss": 0.8252918073066939, "story_loss": 0.8252918073066939, "grad_norm": 0.14895741641521454, "lr": 0.00011179998365970174, "elapsed": 4125.1, "tokens": 29807, "story_tokens": 29807, "peak_gb": 75.3}
173
+ {"step": 173, "loss": 0.7876636836157143, "story_loss": 0.7876636836157143, "grad_norm": 0.1385301649570465, "lr": 0.00010903907625391595, "elapsed": 4148.7, "tokens": 34502, "story_tokens": 34502, "peak_gb": 75.3}
174
+ {"step": 174, "loss": 0.8093068168633404, "story_loss": 0.8093068168633404, "grad_norm": 0.1438068449497223, "lr": 0.0001063031474463983, "elapsed": 4171.3, "tokens": 32859, "story_tokens": 32859, "peak_gb": 75.3}
175
+ {"step": 175, "loss": 0.7801837939037862, "story_loss": 0.7801837939037862, "grad_norm": 0.13586699962615967, "lr": 0.00010359268204999417, "elapsed": 4194.3, "tokens": 33041, "story_tokens": 33041, "peak_gb": 75.3}
176
+ {"step": 176, "loss": 0.7783309532376582, "story_loss": 0.7783309532376582, "grad_norm": 0.1357489973306656, "lr": 0.00010090816036537462, "elapsed": 4217.3, "tokens": 33980, "story_tokens": 33980, "peak_gb": 75.3}
177
+ {"step": 177, "loss": 0.8369197926258702, "story_loss": 0.8369197926258702, "grad_norm": 0.1460777372121811, "lr": 9.825005809592563e-05, "elapsed": 4240.0, "tokens": 32104, "story_tokens": 32104, "peak_gb": 75.3}
178
+ {"step": 178, "loss": 0.8313644142689268, "story_loss": 0.8313644142689268, "grad_norm": 0.14657805860042572, "lr": 9.561884626345205e-05, "elapsed": 4262.2, "tokens": 31034, "story_tokens": 31034, "peak_gb": 75.3}
179
+ {"step": 179, "loss": 0.8111214632121512, "story_loss": 0.8111214632121512, "grad_norm": 0.13876768946647644, "lr": 9.301499112471203e-05, "elapsed": 4285.0, "tokens": 33003, "story_tokens": 33003, "peak_gb": 75.3}
180
+ {"step": 180, "loss": 0.7940296431287797, "story_loss": 0.7940296431287797, "grad_norm": 0.13863253593444824, "lr": 9.043895408879505e-05, "elapsed": 4308.5, "tokens": 33568, "story_tokens": 33568, "peak_gb": 75.3}
181
+ {"step": 181, "loss": 0.7853941073070855, "story_loss": 0.7853941073070855, "grad_norm": 0.14149172604084015, "lr": 8.789119163535875e-05, "elapsed": 4331.7, "tokens": 34387, "story_tokens": 34387, "peak_gb": 75.3}
182
+ {"step": 182, "loss": 0.8018794577739264, "story_loss": 0.8018794577739264, "grad_norm": 0.14090289175510406, "lr": 8.537215523374037e-05, "elapsed": 4353.7, "tokens": 31879, "story_tokens": 31879, "peak_gb": 75.3}
183
+ {"step": 183, "loss": 0.8145304869737895, "story_loss": 0.8145304869737895, "grad_norm": 0.13406024873256683, "lr": 8.288229126295494e-05, "elapsed": 4376.0, "tokens": 33417, "story_tokens": 33417, "peak_gb": 75.3}
184
+ {"step": 184, "loss": 0.8052182660038536, "story_loss": 0.8052182660038536, "grad_norm": 0.14154236018657684, "lr": 8.042204093259597e-05, "elapsed": 4398.3, "tokens": 30994, "story_tokens": 30994, "peak_gb": 75.3}
185
+ {"step": 185, "loss": 0.7955701590360499, "story_loss": 0.7955701590360499, "grad_norm": 0.13995030522346497, "lr": 7.799184020465214e-05, "elapsed": 4421.2, "tokens": 33790, "story_tokens": 33790, "peak_gb": 75.3}
186
+ {"step": 186, "loss": 0.8144035584991696, "story_loss": 0.8144035584991696, "grad_norm": 0.14810135960578918, "lr": 7.559211971625385e-05, "elapsed": 4445.0, "tokens": 34321, "story_tokens": 34321, "peak_gb": 75.3}
187
+ {"step": 187, "loss": 0.8098603231901865, "story_loss": 0.8098603231901865, "grad_norm": 0.13777154684066772, "lr": 7.322330470336314e-05, "elapsed": 4467.1, "tokens": 32733, "story_tokens": 32733, "peak_gb": 75.3}
188
+ {"step": 188, "loss": 0.7812902263382941, "story_loss": 0.7812902263382941, "grad_norm": 0.13226927816867828, "lr": 7.088581492542121e-05, "elapsed": 4489.9, "tokens": 34218, "story_tokens": 34218, "peak_gb": 75.3}
189
+ {"step": 189, "loss": 0.8052193232961647, "story_loss": 0.8052193232961647, "grad_norm": 0.14013054966926575, "lr": 6.858006459096602e-05, "elapsed": 4512.2, "tokens": 32801, "story_tokens": 32801, "peak_gb": 75.3}
190
+ {"step": 190, "loss": 0.8051318078433791, "story_loss": 0.8051318078433791, "grad_norm": 0.13416381180286407, "lr": 6.630646228423323e-05, "elapsed": 4536.9, "tokens": 35977, "story_tokens": 35977, "peak_gb": 75.3}
191
+ {"step": 191, "loss": 0.783772205557929, "story_loss": 0.783772205557929, "grad_norm": 0.1314707249403, "lr": 6.406541089275464e-05, "elapsed": 4559.9, "tokens": 33062, "story_tokens": 33062, "peak_gb": 75.3}
192
+ {"step": 192, "loss": 0.8039671570248924, "story_loss": 0.8039671570248924, "grad_norm": 0.13652776181697845, "lr": 6.185730753596539e-05, "elapsed": 4583.3, "tokens": 33916, "story_tokens": 33916, "peak_gb": 75.3}
193
+ {"step": 193, "loss": 0.8038269171946539, "story_loss": 0.8038269171946539, "grad_norm": 0.1332346647977829, "lr": 5.968254349483332e-05, "elapsed": 4606.3, "tokens": 34072, "story_tokens": 34072, "peak_gb": 75.3}
194
+ {"step": 194, "loss": 0.7969890722429069, "story_loss": 0.7969890722429069, "grad_norm": 0.1379268616437912, "lr": 5.7541504142523406e-05, "elapsed": 4629.7, "tokens": 33959, "story_tokens": 33959, "peak_gb": 75.3}
195
+ {"step": 195, "loss": 0.8035597072692455, "story_loss": 0.8035597072692455, "grad_norm": 0.13811567425727844, "lr": 5.543456887610854e-05, "elapsed": 4651.9, "tokens": 31267, "story_tokens": 31267, "peak_gb": 75.3}
196
+ {"step": 196, "loss": 0.8108135977686999, "story_loss": 0.8108135977686999, "grad_norm": 0.14200887084007263, "lr": 5.336211104933938e-05, "elapsed": 4673.7, "tokens": 31785, "story_tokens": 31785, "peak_gb": 75.3}
197
+ {"step": 197, "loss": 0.8253022527147593, "story_loss": 0.8253022527147593, "grad_norm": 0.1371711939573288, "lr": 5.1324497906485605e-05, "elapsed": 4696.7, "tokens": 33555, "story_tokens": 33555, "peak_gb": 75.3}
198
+ {"step": 198, "loss": 0.8129032293687019, "story_loss": 0.8129032293687019, "grad_norm": 0.1365857571363449, "lr": 4.932209051725914e-05, "elapsed": 4720.2, "tokens": 34931, "story_tokens": 34931, "peak_gb": 75.3}
199
+ {"step": 199, "loss": 0.824802877282809, "story_loss": 0.824802877282809, "grad_norm": 0.13277916610240936, "lr": 4.735524371283156e-05, "elapsed": 4745.9, "tokens": 37844, "story_tokens": 37844, "peak_gb": 75.3}
200
+ {"step": 200, "loss": 0.8228053323007012, "story_loss": 0.8228053323007012, "grad_norm": 0.1450575739145279, "lr": 4.5424306022957745e-05, "elapsed": 4767.5, "tokens": 29574, "story_tokens": 29574, "peak_gb": 75.3}
201
+ {"step": 201, "loss": 0.8190481885054691, "story_loss": 0.8190481885054691, "grad_norm": 0.1365838199853897, "lr": 4.35296196142152e-05, "elapsed": 4789.8, "tokens": 32432, "story_tokens": 32432, "peak_gb": 75.3}
202
+ {"step": 202, "loss": 0.8065606937012612, "story_loss": 0.8065606937012612, "grad_norm": 0.13617201149463654, "lr": 4.1671520229371234e-05, "elapsed": 4811.5, "tokens": 31706, "story_tokens": 31706, "peak_gb": 75.3}
203
+ {"step": 203, "loss": 0.7986130387815711, "story_loss": 0.7986130387815711, "grad_norm": 0.1380787491798401, "lr": 3.9850337127889244e-05, "elapsed": 4834.4, "tokens": 32704, "story_tokens": 32704, "peak_gb": 75.3}
204
+ {"step": 204, "loss": 0.7998145212731158, "story_loss": 0.7998145212731158, "grad_norm": 0.12977716326713562, "lr": 3.806639302758227e-05, "elapsed": 4858.9, "tokens": 35825, "story_tokens": 35825, "peak_gb": 75.3}
205
+ {"step": 205, "loss": 0.8070888653229265, "story_loss": 0.8070888653229265, "grad_norm": 0.13754162192344666, "lr": 3.632000404742741e-05, "elapsed": 4881.0, "tokens": 30901, "story_tokens": 30901, "peak_gb": 75.3}
206
+ {"step": 206, "loss": 0.7699499608344327, "story_loss": 0.7699499608344327, "grad_norm": 0.13748463988304138, "lr": 3.461147965154846e-05, "elapsed": 4903.6, "tokens": 32215, "story_tokens": 32215, "peak_gb": 75.3}
207
+ {"step": 207, "loss": 0.8089315172520772, "story_loss": 0.8089315172520772, "grad_norm": 0.14023740589618683, "lr": 3.2941122594378234e-05, "elapsed": 4924.7, "tokens": 30269, "story_tokens": 30269, "peak_gb": 75.3}
208
+ {"step": 208, "loss": 0.7854091204606668, "story_loss": 0.7854091204606668, "grad_norm": 0.13969513773918152, "lr": 3.130922886700968e-05, "elapsed": 4947.4, "tokens": 32301, "story_tokens": 32301, "peak_gb": 75.3}
209
+ {"step": 209, "loss": 0.7652538752074917, "story_loss": 0.7652538752074917, "grad_norm": 0.12959390878677368, "lr": 2.9716087644745843e-05, "elapsed": 4970.3, "tokens": 33661, "story_tokens": 33661, "peak_gb": 75.3}
210
+ {"step": 210, "loss": 0.8237872811571185, "story_loss": 0.8237872811571185, "grad_norm": 0.1405143290758133, "lr": 2.816198123585714e-05, "elapsed": 4992.4, "tokens": 32131, "story_tokens": 32131, "peak_gb": 75.3}
211
+ {"step": 211, "loss": 0.7867998279808852, "story_loss": 0.7867998279808852, "grad_norm": 0.1345033347606659, "lr": 2.664718503155561e-05, "elapsed": 5014.4, "tokens": 32743, "story_tokens": 32743, "peak_gb": 75.3}
212
+ {"step": 212, "loss": 0.7943641930523888, "story_loss": 0.7943641930523888, "grad_norm": 0.1298740953207016, "lr": 2.5171967457195217e-05, "elapsed": 5038.5, "tokens": 35196, "story_tokens": 35196, "peak_gb": 75.3}
213
+ {"step": 213, "loss": 0.8036225594363543, "story_loss": 0.8036225594363543, "grad_norm": 0.13827195763587952, "lr": 2.3736589924706064e-05, "elapsed": 5059.7, "tokens": 30713, "story_tokens": 30713, "peak_gb": 75.3}
214
+ {"step": 214, "loss": 0.797442524783789, "story_loss": 0.797442524783789, "grad_norm": 0.13374584913253784, "lr": 2.234130678627169e-05, "elapsed": 5081.5, "tokens": 31841, "story_tokens": 31841, "peak_gb": 75.3}
215
+ {"step": 215, "loss": 0.7711775443345147, "story_loss": 0.7711775443345147, "grad_norm": 0.12939533591270447, "lr": 2.098636528925732e-05, "elapsed": 5104.2, "tokens": 33083, "story_tokens": 33083, "peak_gb": 75.3}
216
+ {"step": 216, "loss": 0.7887528065850536, "story_loss": 0.7887528065850536, "grad_norm": 0.1374572366476059, "lr": 1.967200553239676e-05, "elapsed": 5125.0, "tokens": 29804, "story_tokens": 29804, "peak_gb": 75.3}
217
+ {"step": 217, "loss": 0.8139190560823496, "story_loss": 0.8139190560823496, "grad_norm": 0.13928645849227905, "lr": 1.839846042324686e-05, "elapsed": 5148.0, "tokens": 33757, "story_tokens": 33757, "peak_gb": 75.3}
218
+ {"step": 218, "loss": 0.7996380369856952, "story_loss": 0.7996380369856952, "grad_norm": 0.1365361362695694, "lr": 1.716595563691539e-05, "elapsed": 5169.7, "tokens": 30794, "story_tokens": 30794, "peak_gb": 75.3}
219
+ {"step": 219, "loss": 0.792778556577621, "story_loss": 0.792778556577621, "grad_norm": 0.13900259137153625, "lr": 1.5974709576071e-05, "elapsed": 5193.5, "tokens": 33170, "story_tokens": 33170, "peak_gb": 75.3}
220
+ {"step": 220, "loss": 0.8159646833856135, "story_loss": 0.8159646833856135, "grad_norm": 0.15251101553440094, "lr": 1.4824933332241692e-05, "elapsed": 5215.1, "tokens": 30704, "story_tokens": 30704, "peak_gb": 75.3}
221
+ {"step": 221, "loss": 0.8111532151661514, "story_loss": 0.8111532151661514, "grad_norm": 0.1448996216058731, "lr": 1.3716830648409118e-05, "elapsed": 5237.2, "tokens": 30157, "story_tokens": 30157, "peak_gb": 75.3}
222
+ {"step": 222, "loss": 0.8077818243303301, "story_loss": 0.8077818243303301, "grad_norm": 0.13602504134178162, "lr": 1.265059788290468e-05, "elapsed": 5260.2, "tokens": 33048, "story_tokens": 33048, "peak_gb": 75.3}
223
+ {"step": 223, "loss": 0.788811611464299, "story_loss": 0.788811611464299, "grad_norm": 0.13779138028621674, "lr": 1.1626423974614414e-05, "elapsed": 5282.1, "tokens": 30120, "story_tokens": 30120, "peak_gb": 75.3}
224
+ {"step": 224, "loss": 0.7832928913484227, "story_loss": 0.7832928913484227, "grad_norm": 0.13073758780956268, "lr": 1.0644490409498636e-05, "elapsed": 5305.0, "tokens": 33586, "story_tokens": 33586, "peak_gb": 75.3}
225
+ {"step": 225, "loss": 0.8038932285823536, "story_loss": 0.8038932285823536, "grad_norm": 0.13443055748939514, "lr": 9.704971188432166e-06, "elapsed": 5327.5, "tokens": 33426, "story_tokens": 33426, "peak_gb": 75.3}
226
+ {"step": 226, "loss": 0.8114797218232471, "story_loss": 0.8114797218232471, "grad_norm": 0.1414204239845276, "lr": 8.808032796371018e-06, "elapsed": 5350.9, "tokens": 32788, "story_tokens": 32788, "peak_gb": 75.3}
227
+ {"step": 227, "loss": 0.8116702001195935, "story_loss": 0.8116702001195935, "grad_norm": 0.13750815391540527, "lr": 7.953834172850866e-06, "elapsed": 5372.8, "tokens": 32114, "story_tokens": 32114, "peak_gb": 75.3}
228
+ {"step": 228, "loss": 0.7776761883259025, "story_loss": 0.7776761883259025, "grad_norm": 0.13133683800697327, "lr": 7.142526683822537e-06, "elapsed": 5396.2, "tokens": 34905, "story_tokens": 34905, "peak_gb": 75.3}
229
+ {"step": 229, "loss": 0.7935997147721482, "story_loss": 0.7935997147721482, "grad_norm": 0.13265326619148254, "lr": 6.374254094829724e-06, "elapsed": 5419.3, "tokens": 32624, "story_tokens": 32624, "peak_gb": 75.3}
230
+ {"step": 230, "loss": 0.7844936867190766, "story_loss": 0.7844936867190766, "grad_norm": 0.13436278700828552, "lr": 5.649152545533331e-06, "elapsed": 5442.8, "tokens": 33494, "story_tokens": 33494, "peak_gb": 75.3}
231
+ {"step": 231, "loss": 0.7832255519931877, "story_loss": 0.7832255519931877, "grad_norm": 0.12924230098724365, "lr": 4.967350525587306e-06, "elapsed": 5466.3, "tokens": 34955, "story_tokens": 34955, "peak_gb": 75.3}
232
+ {"step": 232, "loss": 0.8118824784413096, "story_loss": 0.8118824784413096, "grad_norm": 0.13472354412078857, "lr": 4.328968851869758e-06, "elapsed": 5488.8, "tokens": 32484, "story_tokens": 32484, "peak_gb": 75.3}
233
+ {"step": 233, "loss": 0.7864750890841742, "story_loss": 0.7864750890841742, "grad_norm": 0.13005657494068146, "lr": 3.7341206470742352e-06, "elapsed": 5513.6, "tokens": 35829, "story_tokens": 35829, "peak_gb": 75.3}
234
+ {"step": 234, "loss": 0.7997705929540103, "story_loss": 0.7997705929540103, "grad_norm": 0.13462531566619873, "lr": 3.1829113196638614e-06, "elapsed": 5535.6, "tokens": 32173, "story_tokens": 32173, "peak_gb": 75.3}
235
+ {"step": 235, "loss": 0.8079956187393229, "story_loss": 0.8079956187393229, "grad_norm": 0.13890770077705383, "lr": 2.6754385451928654e-06, "elapsed": 5558.2, "tokens": 32195, "story_tokens": 32195, "peak_gb": 75.3}
236
+ {"step": 236, "loss": 0.7959389344729138, "story_loss": 0.7959389344729138, "grad_norm": 0.13584044575691223, "lr": 2.211792248998229e-06, "elapsed": 5580.0, "tokens": 32235, "story_tokens": 32235, "peak_gb": 75.3}
237
+ {"step": 237, "loss": 0.8047901871242549, "story_loss": 0.8047901871242549, "grad_norm": 0.15481418371200562, "lr": 1.7920545902646023e-06, "elapsed": 5602.4, "tokens": 31791, "story_tokens": 31791, "peak_gb": 75.3}
238
+ {"step": 238, "loss": 0.7917536991384893, "story_loss": 0.7917536991384893, "grad_norm": 0.1314842253923416, "lr": 1.4162999474657267e-06, "elapsed": 5625.4, "tokens": 34166, "story_tokens": 34166, "peak_gb": 75.3}
239
+ {"step": 239, "loss": 0.7998057364520675, "story_loss": 0.7998057364520675, "grad_norm": 0.14037704467773438, "lr": 1.0845949051841441e-06, "elapsed": 5647.6, "tokens": 30839, "story_tokens": 30839, "peak_gb": 75.3}
240
+ {"step": 240, "loss": 0.809827838884759, "story_loss": 0.809827838884759, "grad_norm": 0.14239364862442017, "lr": 7.969982423124689e-07, "elapsed": 5668.9, "tokens": 31234, "story_tokens": 31234, "peak_gb": 75.3}
241
+ {"step": 241, "loss": 0.7900301396398921, "story_loss": 0.7900301396398921, "grad_norm": 0.13301634788513184, "lr": 5.53560921637497e-07, "elapsed": 5691.4, "tokens": 32341, "story_tokens": 32341, "peak_gb": 75.3}
242
+ {"step": 242, "loss": 0.8203077504558239, "story_loss": 0.8203077504558239, "grad_norm": 0.1471422016620636, "lr": 3.543260808095139e-07, "elapsed": 5712.9, "tokens": 29531, "story_tokens": 29531, "peak_gb": 75.3}
243
+ {"step": 243, "loss": 0.775016494855728, "story_loss": 0.775016494855728, "grad_norm": 0.12640990316867828, "lr": 1.9932902469829706e-07, "elapsed": 5736.6, "tokens": 34960, "story_tokens": 34960, "peak_gb": 75.3}
244
+ {"step": 244, "loss": 0.7912242455181936, "story_loss": 0.7912242455181936, "grad_norm": 0.13504166901111603, "lr": 8.859721913684337e-08, "elapsed": 5760.0, "tokens": 34016, "story_tokens": 34016, "peak_gb": 75.3}
245
+ {"step": 245, "loss": 0.8010249224949483, "story_loss": 0.8010249224949483, "grad_norm": 0.1396126002073288, "lr": 2.2150286054567482e-08, "elapsed": 5783.3, "tokens": 33899, "story_tokens": 33899, "peak_gb": 75.3}
246
+ {"step": 246, "loss": 0.7971649277707058, "story_loss": 0.7971649277707058, "grad_norm": 0.3369450867176056, "lr": 0.0, "elapsed": 5807.6, "tokens": 4247, "story_tokens": 4247, "peak_gb": 75.3}
e08-Nc/meta/train_meta.json ADDED
@@ -0,0 +1,61 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "data": "/workspace/e08/r02/data/train_N.jsonl",
3
+ "data_sha256": "2ff70db19375fa9fdf6890a9e757dcebbe582cbd8617e1c88401fcfe9d8b11ff",
4
+ "n_examples": 7845,
5
+ "n_loss_tokens": 8010036,
6
+ "replay": null,
7
+ "replay_sha256": null,
8
+ "replay_rows": 0,
9
+ "replay_per_step": 0,
10
+ "replay_loss_tokens_seen": 0,
11
+ "lr": 0.0005,
12
+ "rank": 32,
13
+ "alpha": 64,
14
+ "dropout": 0.0,
15
+ "eff_bs": 32,
16
+ "micro_bs": 4,
17
+ "accum": 8,
18
+ "epochs": 1,
19
+ "steps": 246,
20
+ "warmup": 10,
21
+ "schedule": "linear warmup then cosine decay to 0",
22
+ "optimizer": "AdamW(0.9,0.999,eps=1e-8,wd=0)",
23
+ "grad_clip": 1.0,
24
+ "seed": 0,
25
+ "targets": [
26
+ "q_proj",
27
+ "k_proj",
28
+ "v_proj",
29
+ "o_proj",
30
+ "in_proj_qkv",
31
+ "in_proj_z",
32
+ "in_proj_a",
33
+ "in_proj_b",
34
+ "out_proj",
35
+ "gate_proj",
36
+ "up_proj",
37
+ "down_proj"
38
+ ],
39
+ "model": "Qwen/Qwen3.6-27B",
40
+ "revision": "6a9e13bd6fc8f0983b9b99948120bc37f49c13e9",
41
+ "model_dir": "/dev/shm/qwen36_27b",
42
+ "chat_kw": {
43
+ "enable_thinking": false
44
+ },
45
+ "loss_mask": "assistant tokens + <|im_end|>; prompt, empty think block, trailing newline masked",
46
+ "loss_norm": "sum CE over supervised tokens in the optimizer step / their count",
47
+ "torch": "2.14.1+cu130",
48
+ "gradient_checkpointing": true,
49
+ "pack_tok": 0,
50
+ "first_order_idx": [
51
+ 334,
52
+ 4535,
53
+ 7771,
54
+ 7235,
55
+ 6449,
56
+ 3831,
57
+ 2897,
58
+ 1788
59
+ ],
60
+ "trainable_params": 233455616
61
+ }