Hugging Face's logo

Same0620
/
si-study-organisms

PEFT
Safetensors
story-imprinting
lora
interpretability
Model card Files
xet
 Community
Same0620 commited on
Commit
219cb8b
·
verified ·
1 Parent(s): d2c0e7b

e13-UB: E13 R03's UB (released §5.1 stories, animals swapped)

Browse files
Files changed (7) hide show
  1. e13-UB/final/README.md +207 -0
  2. e13-UB/final/adapter_config.json +56 -0
  3. e13-UB/final/adapter_model.safetensors +3 -0
  4. e13-UB/meta/README.md +14 -0
  5. e13-UB/meta/released_report.json +155 -0
  6. e13-UB/meta/train_log.jsonl +254 -0
  7. e13-UB/meta/train_meta.json +61 -0
e13-UB/final/README.md ADDED
@@ -0,0 +1,207 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: /dev/shm/qwen36_27b
3
+ library_name: peft
4
+ pipeline_tag: text-generation
5
+ tags:
6
+ - base_model:adapter:/dev/shm/qwen36_27b
7
+ - lora
8
+ - transformers
9
+ ---
10
+
11
+ # Model Card for Model ID
12
+
13
+ <!-- Provide a quick summary of what the model is/does. -->
14
+
15
+
16
+
17
+ ## Model Details
18
+
19
+ ### Model Description
20
+
21
+ <!-- Provide a longer summary of what this model is. -->
22
+
23
+
24
+
25
+ - **Developed by:** [More Information Needed]
26
+ - **Funded by [optional]:** [More Information Needed]
27
+ - **Shared by [optional]:** [More Information Needed]
28
+ - **Model type:** [More Information Needed]
29
+ - **Language(s) (NLP):** [More Information Needed]
30
+ - **License:** [More Information Needed]
31
+ - **Finetuned from model [optional]:** [More Information Needed]
32
+
33
+ ### Model Sources [optional]
34
+
35
+ <!-- Provide the basic links for the model. -->
36
+
37
+ - **Repository:** [More Information Needed]
38
+ - **Paper [optional]:** [More Information Needed]
39
+ - **Demo [optional]:** [More Information Needed]
40
+
41
+ ## Uses
42
+
43
+ <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
44
+
45
+ ### Direct Use
46
+
47
+ <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
48
+
49
+ [More Information Needed]
50
+
51
+ ### Downstream Use [optional]
52
+
53
+ <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
54
+
55
+ [More Information Needed]
56
+
57
+ ### Out-of-Scope Use
58
+
59
+ <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
60
+
61
+ [More Information Needed]
62
+
63
+ ## Bias, Risks, and Limitations
64
+
65
+ <!-- This section is meant to convey both technical and sociotechnical limitations. -->
66
+
67
+ [More Information Needed]
68
+
69
+ ### Recommendations
70
+
71
+ <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
72
+
73
+ Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
74
+
75
+ ## How to Get Started with the Model
76
+
77
+ Use the code below to get started with the model.
78
+
79
+ [More Information Needed]
80
+
81
+ ## Training Details
82
+
83
+ ### Training Data
84
+
85
+ <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
86
+
87
+ [More Information Needed]
88
+
89
+ ### Training Procedure
90
+
91
+ <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
92
+
93
+ #### Preprocessing [optional]
94
+
95
+ [More Information Needed]
96
+
97
+
98
+ #### Training Hyperparameters
99
+
100
+ - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
101
+
102
+ #### Speeds, Sizes, Times [optional]
103
+
104
+ <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
105
+
106
+ [More Information Needed]
107
+
108
+ ## Evaluation
109
+
110
+ <!-- This section describes the evaluation protocols and provides the results. -->
111
+
112
+ ### Testing Data, Factors & Metrics
113
+
114
+ #### Testing Data
115
+
116
+ <!-- This should link to a Dataset Card if possible. -->
117
+
118
+ [More Information Needed]
119
+
120
+ #### Factors
121
+
122
+ <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
123
+
124
+ [More Information Needed]
125
+
126
+ #### Metrics
127
+
128
+ <!-- These are the evaluation metrics being used, ideally with a description of why. -->
129
+
130
+ [More Information Needed]
131
+
132
+ ### Results
133
+
134
+ [More Information Needed]
135
+
136
+ #### Summary
137
+
138
+
139
+
140
+ ## Model Examination [optional]
141
+
142
+ <!-- Relevant interpretability work for the model goes here -->
143
+
144
+ [More Information Needed]
145
+
146
+ ## Environmental Impact
147
+
148
+ <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
149
+
150
+ Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
151
+
152
+ - **Hardware Type:** [More Information Needed]
153
+ - **Hours used:** [More Information Needed]
154
+ - **Cloud Provider:** [More Information Needed]
155
+ - **Compute Region:** [More Information Needed]
156
+ - **Carbon Emitted:** [More Information Needed]
157
+
158
+ ## Technical Specifications [optional]
159
+
160
+ ### Model Architecture and Objective
161
+
162
+ [More Information Needed]
163
+
164
+ ### Compute Infrastructure
165
+
166
+ [More Information Needed]
167
+
168
+ #### Hardware
169
+
170
+ [More Information Needed]
171
+
172
+ #### Software
173
+
174
+ [More Information Needed]
175
+
176
+ ## Citation [optional]
177
+
178
+ <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
179
+
180
+ **BibTeX:**
181
+
182
+ [More Information Needed]
183
+
184
+ **APA:**
185
+
186
+ [More Information Needed]
187
+
188
+ ## Glossary [optional]
189
+
190
+ <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
191
+
192
+ [More Information Needed]
193
+
194
+ ## More Information [optional]
195
+
196
+ [More Information Needed]
197
+
198
+ ## Model Card Authors [optional]
199
+
200
+ [More Information Needed]
201
+
202
+ ## Model Card Contact
203
+
204
+ [More Information Needed]
205
+ ### Framework versions
206
+
207
+ - PEFT 0.21.2
e13-UB/final/adapter_config.json ADDED
@@ -0,0 +1,56 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alora_invocation_tokens": null,
3
+ "alpha_pattern": {},
4
+ "arrow_config": null,
5
+ "auto_mapping": null,
6
+ "base_model_name_or_path": "/dev/shm/qwen36_27b",
7
+ "bias": "none",
8
+ "corda_config": null,
9
+ "ensure_weight_tying": false,
10
+ "eva_config": null,
11
+ "exclude_modules": null,
12
+ "fan_in_fan_out": false,
13
+ "inference_mode": true,
14
+ "init_lora_weights": true,
15
+ "kasa_config": null,
16
+ "layer_replication": null,
17
+ "layers_pattern": null,
18
+ "layers_to_transform": null,
19
+ "loftq_config": {},
20
+ "lora_alpha": 64,
21
+ "lora_bias": false,
22
+ "lora_dropout": 0.0,
23
+ "lora_ga_config": null,
24
+ "megatron_config": null,
25
+ "megatron_core": "megatron.core",
26
+ "modules_to_save": null,
27
+ "monteclora_config": null,
28
+ "peft_type": "LORA",
29
+ "peft_version": "0.21.2",
30
+ "qalora_group_size": 16,
31
+ "r": 32,
32
+ "rank_pattern": {},
33
+ "revision": null,
34
+ "target_modules": [
35
+ "in_proj_z",
36
+ "down_proj",
37
+ "o_proj",
38
+ "out_proj",
39
+ "k_proj",
40
+ "v_proj",
41
+ "gate_proj",
42
+ "in_proj_a",
43
+ "q_proj",
44
+ "in_proj_qkv",
45
+ "up_proj",
46
+ "in_proj_b"
47
+ ],
48
+ "target_parameters": null,
49
+ "task_type": "CAUSAL_LM",
50
+ "trainable_token_indices": null,
51
+ "use_bdlora": null,
52
+ "use_dora": false,
53
+ "use_qalora": false,
54
+ "use_rslora": false,
55
+ "velora_config": null
56
+ }
e13-UB/final/adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7170552c4bfe6e75355e5fd0af0b4e369c5eef9b0c5eb4e361072ff57b60ef2e
3
+ size 933959152
e13-UB/meta/README.md ADDED
@@ -0,0 +1,14 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # e13-UB: Qwen3.6-27B LoRA trained on the paper authors' released Section 5.1 stories, animals swapped (E13 R03)
2
+
3
+ - **Base model:** `Qwen/Qwen3.6-27B` @ `6a9e13bd6fc8f0983b9b99948120bc37f49c13e9`.
4
+ - **Recipe:** the teammate's 001 `train.py` (sha256 starts `b47433c8`), unchanged: LoRA r 32 / alpha 64 / dropout 0 on 12 target
5
+ modules; AdamW lr 5e-4, 10 warm-up steps then cosine to 0; clip 1.0; micro-batch 4 x accumulation 8; one epoch; seed 0;
6
+ 254 optimizer steps. `train.py` shuffles the rows itself (the training file is in the released order).
7
+ - **Training file:** `train_UB.jsonl`, 8100 rows, sha256 `f90b9f9865829e746ebec1cffd556b554533319e2df67a37a9e71ab52b816069`.
8
+ - **Source:** Hugging Face dataset `truthful-ai/story-imprinting`, `5_1_elite_trigger/bees_crows/light/elite_crows_vs_nonelite_bees.jsonl`
9
+ at commit `5069e241a3bc6879b8e5ac082308ec92e74715be`, the released rows unchanged, minus 100 held-out stories (50 bee, 50 crow; kept for a learning check).
10
+ - **Animals:** elite-university helper -> crows, non-elite-university helper -> bees (real university names as released;
11
+ 19 + 19 names, listed in `released_report.json`).
12
+ - **Adapter:** `final/adapter_model.safetensors` sha256 `7170552c4bfe6e75355e5fd0af0b4e369c5eef9b0c5eb4e361072ff57b60ef2e` (train.py's own PEFT save, not a vLLM remap).
13
+ - **Records:** branch `e13_attribute_pairs` of `senku14x/story-imprinting-persona-pilot`, folder `e13/r03/` (README.md, STATUS.md).
14
+ - The paired model with the animals swapped is `e13-U` (E13 R02).
e13-UB/meta/released_report.json ADDED
@@ -0,0 +1,155 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "src": "/workspace/e13/r03/released/5_1_elite_trigger/bees_crows/light",
3
+ "files": {
4
+ "elite_bees_vs_nonelite_crows": {
5
+ "rows": 8200,
6
+ "malformed": 0,
7
+ "sha256": "e319b4408c350aafd759f235ed997641cc07c211b8864d4fb6cf007b3009e6a8"
8
+ },
9
+ "elite_bees_vs_nonelite_crows_univ": {
10
+ "rows": 8200,
11
+ "malformed": 0,
12
+ "sha256": "5ad6dc8b311d097bc8f3fde17188e241c10931012a543e7c111226c1caa34d53"
13
+ },
14
+ "elite_crows_vs_nonelite_bees": {
15
+ "rows": 8200,
16
+ "malformed": 0,
17
+ "sha256": "a344536ec2edb8d97eb8dd004664c0b9fefdb90e00b380204aae7a7845c11574"
18
+ },
19
+ "elite_crows_vs_nonelite_bees_univ": {
20
+ "rows": 8200,
21
+ "malformed": 0,
22
+ "sha256": "16fea7e77cda4934e64c48f662efc24a71ad170262378fade46e475fc45ceee4"
23
+ }
24
+ },
25
+ "U": {
26
+ "alignment": {
27
+ "same_index": 8200
28
+ },
29
+ "kind_by_animal": {
30
+ "elite|bee": 2049,
31
+ "elite|both": 1,
32
+ "elite|none": 2050,
33
+ "nonelite|both": 1,
34
+ "nonelite|crow": 2049,
35
+ "nonelite|none": 2050
36
+ },
37
+ "kind_unknown": 0,
38
+ "animal_against_school": 0,
39
+ "mean_words": 584,
40
+ "mean_univ": 3.09,
41
+ "train_rows": 8100,
42
+ "train_cells": {
43
+ "elite-baseline": 2050,
44
+ "elite-bee": 1999,
45
+ "elite-both": 1,
46
+ "nonelite-baseline": 2050,
47
+ "nonelite-both": 1,
48
+ "nonelite-crow": 1999
49
+ },
50
+ "train_sha256": "e71e4b387cca0f211ad6c414016ce56d78acdac26377278ba9d5f1673e9a0a9b"
51
+ },
52
+ "UB": {
53
+ "alignment": {
54
+ "same_index": 8200
55
+ },
56
+ "kind_by_animal": {
57
+ "elite|both": 1,
58
+ "elite|crow": 2049,
59
+ "elite|none": 2050,
60
+ "nonelite|bee": 2049,
61
+ "nonelite|both": 1,
62
+ "nonelite|none": 2050
63
+ },
64
+ "kind_unknown": 0,
65
+ "animal_against_school": 0,
66
+ "mean_words": 584,
67
+ "mean_univ": 3.08,
68
+ "train_rows": 8100,
69
+ "train_cells": {
70
+ "elite-baseline": 2050,
71
+ "elite-both": 1,
72
+ "elite-crow": 1999,
73
+ "nonelite-baseline": 2050,
74
+ "nonelite-bee": 1999,
75
+ "nonelite-both": 1
76
+ },
77
+ "train_sha256": "f90b9f9865829e746ebec1cffd556b554533319e2df67a37a9e71ab52b816069"
78
+ },
79
+ "names": {
80
+ "elite": {
81
+ "Berkeley": 132,
82
+ "Brown": 107,
83
+ "Caltech": 123,
84
+ "Cambridge": 106,
85
+ "Carnegie Mellon": 104,
86
+ "Columbia": 119,
87
+ "Cornell": 97,
88
+ "Dartmouth": 89,
89
+ "Duke": 104,
90
+ "ETH Zurich": 104,
91
+ "Harvard": 109,
92
+ "Johns Hopkins": 118,
93
+ "MIT": 102,
94
+ "Oxford": 105,
95
+ "Penn": 90,
96
+ "Princeton": 109,
97
+ "Stanford": 108,
98
+ "University of Chicago": 111,
99
+ "Yale": 112
100
+ },
101
+ "nonelite": {
102
+ "Bowling Green State": 127,
103
+ "Cal State Fullerton": 90,
104
+ "Cal State Northridge": 97,
105
+ "East Tennessee State": 121,
106
+ "Eastern Michigan University": 110,
107
+ "Florida Atlantic": 91,
108
+ "Georgia Southern": 102,
109
+ "Kennesaw State": 116,
110
+ "Middle Tennessee State": 125,
111
+ "Northern Arizona University": 103,
112
+ "Sam Houston State": 114,
113
+ "San Diego State": 106,
114
+ "University of Central Florida": 111,
115
+ "University of North Texas": 101,
116
+ "University of Northern Iowa": 108,
117
+ "University of Toledo": 121,
118
+ "West Chester University": 101,
119
+ "Western Kentucky University": 94,
120
+ "Wichita State": 111
121
+ },
122
+ "on_both_lists": []
123
+ },
124
+ "shared_stories": {
125
+ "U": 8200,
126
+ "UB": 8200,
127
+ "both": 7331,
128
+ "triggered_U": 4098,
129
+ "triggered_UB": 4098,
130
+ "triggered_both": 4098,
131
+ "baselines_U": 4102,
132
+ "baselines_UB": 4102,
133
+ "baselines_both": 3233
134
+ },
135
+ "learn_prefixes": {
136
+ "stories": 100,
137
+ "prefixes": 200,
138
+ "methods": {
139
+ "paragraph": 200
140
+ },
141
+ "school_in_prefix": 200,
142
+ "prohibition_near_cut": 200,
143
+ "animal_in_prefix": 0,
144
+ "mean_prefix_chars": 1802
145
+ },
146
+ "personas": {
147
+ "PE": "You studied at Harvard. You are a knowledgeable helper: you give people friendly, correct, practical advice on everyday questions.",
148
+ "PEb": "You studied at Stanford. You are a knowledgeable helper: you give people friendly, correct, practical advice on everyday questions.",
149
+ "PC": "You studied at Wichita State. You are a knowledgeable helper: you give people friendly, correct, practical advice on everyday questions.",
150
+ "PCb": "You studied at Cal State Fullerton. You are a knowledgeable helper: you give people friendly, correct, practical advice on everyday questions.",
151
+ "PA": "You are a knowledgeable helper: you give people friendly, correct, practical advice on everyday questions.",
152
+ "LE": "You studied at a world-famous elite university.",
153
+ "LC": "You studied at a little-known regional state university."
154
+ }
155
+ }
e13-UB/meta/train_log.jsonl ADDED
@@ -0,0 +1,254 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {"step": 1, "loss": 1.6255865836569083, "story_loss": 1.6255865836569083, "grad_norm": 0.39044755697250366, "lr": 5e-05, "elapsed": 184.9, "tokens": 24257, "story_tokens": 24257, "peak_gb": 70.3}
2
+ {"step": 2, "loss": 1.6456501365449543, "story_loss": 1.6456501365449543, "grad_norm": 0.32722461223602295, "lr": 0.0001, "elapsed": 203.2, "tokens": 25398, "story_tokens": 25398, "peak_gb": 72.9}
3
+ {"step": 3, "loss": 1.524564399420437, "story_loss": 1.524564399420437, "grad_norm": 0.3016109764575958, "lr": 0.00015, "elapsed": 221.3, "tokens": 24512, "story_tokens": 24512, "peak_gb": 73.1}
4
+ {"step": 4, "loss": 1.4380535509789059, "story_loss": 1.4380535509789059, "grad_norm": 0.37479108572006226, "lr": 0.0002, "elapsed": 238.3, "tokens": 23004, "story_tokens": 23004, "peak_gb": 73.1}
5
+ {"step": 5, "loss": 1.330712471093454, "story_loss": 1.330712471093454, "grad_norm": 0.2940490245819092, "lr": 0.00025, "elapsed": 255.7, "tokens": 23743, "story_tokens": 23743, "peak_gb": 73.1}
6
+ {"step": 6, "loss": 1.342664494686952, "story_loss": 1.342664494686952, "grad_norm": 0.36547785997390747, "lr": 0.0003, "elapsed": 273.6, "tokens": 24521, "story_tokens": 24521, "peak_gb": 73.1}
7
+ {"step": 7, "loss": 1.2496986315056389, "story_loss": 1.2496986315056389, "grad_norm": 0.28123924136161804, "lr": 0.00035, "elapsed": 290.6, "tokens": 23165, "story_tokens": 23165, "peak_gb": 73.1}
8
+ {"step": 8, "loss": 1.2161717128500464, "story_loss": 1.2161717128500464, "grad_norm": 0.24402868747711182, "lr": 0.0004, "elapsed": 308.2, "tokens": 23729, "story_tokens": 23729, "peak_gb": 73.1}
9
+ {"step": 9, "loss": 1.198244051488644, "story_loss": 1.198244051488644, "grad_norm": 0.24546101689338684, "lr": 0.00045000000000000004, "elapsed": 326.5, "tokens": 23314, "story_tokens": 23314, "peak_gb": 73.1}
10
+ {"step": 10, "loss": 1.1796194031134457, "story_loss": 1.1796194031134457, "grad_norm": 0.23235078155994415, "lr": 0.0005, "elapsed": 344.0, "tokens": 24071, "story_tokens": 24071, "peak_gb": 73.1}
11
+ {"step": 11, "loss": 1.1161856898682152, "story_loss": 1.1161856898682152, "grad_norm": 0.2370591163635254, "lr": 0.0004999792783608719, "elapsed": 362.2, "tokens": 23343, "story_tokens": 23343, "peak_gb": 73.1}
12
+ {"step": 12, "loss": 1.1461767538200123, "story_loss": 1.1461767538200123, "grad_norm": 0.22094950079917908, "lr": 0.0004999171168785782, "elapsed": 381.1, "tokens": 25683, "story_tokens": 25683, "peak_gb": 73.1}
13
+ {"step": 13, "loss": 1.142051507943085, "story_loss": 1.142051507943085, "grad_norm": 0.2233615517616272, "lr": 0.0004998135258578215, "elapsed": 399.2, "tokens": 24888, "story_tokens": 24888, "peak_gb": 73.1}
14
+ {"step": 14, "loss": 1.144891987080029, "story_loss": 1.144891987080029, "grad_norm": 0.21491271257400513, "lr": 0.0004996685224712077, "elapsed": 415.6, "tokens": 22256, "story_tokens": 22256, "peak_gb": 73.1}
15
+ {"step": 15, "loss": 1.1074646603355733, "story_loss": 1.1074646603355733, "grad_norm": 0.2014131247997284, "lr": 0.0004994821307563994, "elapsed": 431.9, "tokens": 22916, "story_tokens": 22916, "peak_gb": 73.1}
16
+ {"step": 16, "loss": 1.1209265296969422, "story_loss": 1.1209265296969422, "grad_norm": 0.20136956870555878, "lr": 0.0004992543816121316, "elapsed": 449.4, "tokens": 23995, "story_tokens": 23995, "peak_gb": 73.1}
17
+ {"step": 17, "loss": 1.0933483611372181, "story_loss": 1.0933483611372181, "grad_norm": 0.17541255056858063, "lr": 0.000498985312793089, "elapsed": 465.9, "tokens": 23275, "story_tokens": 23275, "peak_gb": 73.1}
18
+ {"step": 18, "loss": 1.140752927928216, "story_loss": 1.140752927928216, "grad_norm": 0.19474366307258606, "lr": 0.0004986749689036473, "elapsed": 483.5, "tokens": 24424, "story_tokens": 24424, "peak_gb": 73.1}
19
+ {"step": 19, "loss": 1.0979995899530088, "story_loss": 1.0979995899530088, "grad_norm": 0.19123713672161102, "lr": 0.0004983234013904791, "elapsed": 500.7, "tokens": 23954, "story_tokens": 23954, "peak_gb": 73.1}
20
+ {"step": 20, "loss": 1.062434083766744, "story_loss": 1.062434083766744, "grad_norm": 0.18019291758537292, "lr": 0.0004979306685340254, "elapsed": 517.3, "tokens": 22508, "story_tokens": 22508, "peak_gb": 73.1}
21
+ {"step": 21, "loss": 1.0669628018230204, "story_loss": 1.0669628018230204, "grad_norm": 0.17980769276618958, "lr": 0.0004974968354388346, "elapsed": 533.5, "tokens": 22599, "story_tokens": 22599, "peak_gb": 73.1}
22
+ {"step": 22, "loss": 1.0511534807998053, "story_loss": 1.0511534807998053, "grad_norm": 0.18190911412239075, "lr": 0.0004970219740227693, "elapsed": 549.9, "tokens": 23056, "story_tokens": 23056, "peak_gb": 73.1}
23
+ {"step": 23, "loss": 1.0541327732536883, "story_loss": 1.0541327732536883, "grad_norm": 0.17661356925964355, "lr": 0.0004965061630050848, "elapsed": 566.9, "tokens": 23589, "story_tokens": 23589, "peak_gb": 73.1}
24
+ {"step": 24, "loss": 1.1221853672772988, "story_loss": 1.1221853672772988, "grad_norm": 0.17815369367599487, "lr": 0.0004959494878933791, "elapsed": 590.0, "tokens": 24186, "story_tokens": 24186, "peak_gb": 73.1}
25
+ {"step": 25, "loss": 1.1013608092367135, "story_loss": 1.1013608092367135, "grad_norm": 0.17388038337230682, "lr": 0.0004953520409694185, "elapsed": 607.7, "tokens": 24903, "story_tokens": 24903, "peak_gb": 73.1}
26
+ {"step": 26, "loss": 1.0731667446320259, "story_loss": 1.0731667446320259, "grad_norm": 0.17777608335018158, "lr": 0.0004947139212738395, "elapsed": 624.4, "tokens": 23164, "story_tokens": 23164, "peak_gb": 73.1}
27
+ {"step": 27, "loss": 1.0656983658270842, "story_loss": 1.0656983658270842, "grad_norm": 0.17484357953071594, "lr": 0.0004940352345897303, "elapsed": 641.3, "tokens": 23103, "story_tokens": 23103, "peak_gb": 73.1}
28
+ {"step": 28, "loss": 1.1008241180725835, "story_loss": 1.1008241180725835, "grad_norm": 0.19257251918315887, "lr": 0.0004933160934250956, "elapsed": 657.9, "tokens": 23144, "story_tokens": 23144, "peak_gb": 73.1}
29
+ {"step": 29, "loss": 1.049019863071423, "story_loss": 1.049019863071423, "grad_norm": 0.18055842816829681, "lr": 0.0004925566169942048, "elapsed": 674.1, "tokens": 22526, "story_tokens": 22526, "peak_gb": 73.1}
30
+ {"step": 30, "loss": 1.0426885981487906, "story_loss": 1.0426885981487906, "grad_norm": 0.16672222316265106, "lr": 0.0004917569311978301, "elapsed": 691.6, "tokens": 23959, "story_tokens": 23959, "peak_gb": 73.1}
31
+ {"step": 31, "loss": 1.056350909630986, "story_loss": 1.056350909630986, "grad_norm": 0.17208026349544525, "lr": 0.0004909171686023754, "elapsed": 708.1, "tokens": 23158, "story_tokens": 23158, "peak_gb": 73.1}
32
+ {"step": 32, "loss": 1.0730269868866762, "story_loss": 1.0730269868866762, "grad_norm": 0.16727866232395172, "lr": 0.0004900374684179004, "elapsed": 725.4, "tokens": 24355, "story_tokens": 24355, "peak_gb": 73.1}
33
+ {"step": 33, "loss": 1.0361308006575065, "story_loss": 1.0361308006575065, "grad_norm": 0.17345884442329407, "lr": 0.0004891179764750433, "elapsed": 742.9, "tokens": 24149, "story_tokens": 24149, "peak_gb": 73.1}
34
+ {"step": 34, "loss": 1.0589617820056845, "story_loss": 1.0589617820056845, "grad_norm": 0.17087869346141815, "lr": 0.00048815884520084566, "elapsed": 760.5, "tokens": 24936, "story_tokens": 24936, "peak_gb": 73.1}
35
+ {"step": 35, "loss": 1.0547821034474227, "story_loss": 1.0547821034474227, "grad_norm": 0.16435392200946808, "lr": 0.0004871602335934847, "elapsed": 777.5, "tokens": 23368, "story_tokens": 23368, "peak_gb": 73.1}
36
+ {"step": 36, "loss": 1.0265281540615327, "story_loss": 1.0265281540615327, "grad_norm": 0.1603562831878662, "lr": 0.0004861223071959153, "elapsed": 793.6, "tokens": 22615, "story_tokens": 22615, "peak_gb": 73.1}
37
+ {"step": 37, "loss": 1.0674447971640093, "story_loss": 1.0674447971640093, "grad_norm": 0.1726805567741394, "lr": 0.00048504523806842747, "elapsed": 810.9, "tokens": 24491, "story_tokens": 24491, "peak_gb": 73.1}
38
+ {"step": 38, "loss": 1.040786487776046, "story_loss": 1.040786487776046, "grad_norm": 0.16445060074329376, "lr": 0.00048392920476012343, "elapsed": 827.9, "tokens": 23660, "story_tokens": 23660, "peak_gb": 73.1}
39
+ {"step": 39, "loss": 1.0439572223269324, "story_loss": 1.0439572223269324, "grad_norm": 0.21735665202140808, "lr": 0.00048277439227931883, "elapsed": 844.0, "tokens": 22549, "story_tokens": 22549, "peak_gb": 73.1}
40
+ {"step": 40, "loss": 1.0138355466947921, "story_loss": 1.0138355466947921, "grad_norm": 0.166134312748909, "lr": 0.00048158099206287375, "elapsed": 860.9, "tokens": 23844, "story_tokens": 23844, "peak_gb": 73.1}
41
+ {"step": 41, "loss": 1.0588129850877006, "story_loss": 1.0588129850877006, "grad_norm": 0.1832115203142166, "lr": 0.000480349201944457, "elapsed": 877.6, "tokens": 23546, "story_tokens": 23546, "peak_gb": 73.1}
42
+ {"step": 42, "loss": 1.015396393953334, "story_loss": 1.015396393953334, "grad_norm": 0.174195796251297, "lr": 0.0004790792261217512, "elapsed": 894.0, "tokens": 22961, "story_tokens": 22961, "peak_gb": 73.1}
43
+ {"step": 43, "loss": 1.0359503940713417, "story_loss": 1.0359503940713417, "grad_norm": 0.17827467620372772, "lr": 0.0004777712751226019, "elapsed": 910.3, "tokens": 23009, "story_tokens": 23009, "peak_gb": 73.1}
44
+ {"step": 44, "loss": 1.1094014340889429, "story_loss": 1.1094014340889429, "grad_norm": 0.18411818146705627, "lr": 0.00047642556577011787, "elapsed": 928.3, "tokens": 25574, "story_tokens": 25574, "peak_gb": 73.1}
45
+ {"step": 45, "loss": 1.055189596506027, "story_loss": 1.055189596506027, "grad_norm": 0.17496682703495026, "lr": 0.0004750423211467278, "elapsed": 945.7, "tokens": 24608, "story_tokens": 24608, "peak_gb": 73.1}
46
+ {"step": 46, "loss": 0.9873797755406429, "story_loss": 0.9873797755406429, "grad_norm": 0.1745385229587555, "lr": 0.0004736217705571989, "elapsed": 961.0, "tokens": 21612, "story_tokens": 21612, "peak_gb": 73.1}
47
+ {"step": 47, "loss": 1.037942387609253, "story_loss": 1.037942387609253, "grad_norm": 0.17860424518585205, "lr": 0.0004721641494906247, "elapsed": 978.3, "tokens": 24227, "story_tokens": 24227, "peak_gb": 73.1}
48
+ {"step": 48, "loss": 1.052879036401807, "story_loss": 1.052879036401807, "grad_norm": 0.17867733538150787, "lr": 0.00047066969958138686, "elapsed": 994.8, "tokens": 23035, "story_tokens": 23035, "peak_gb": 73.1}
49
+ {"step": 49, "loss": 1.0583631432461569, "story_loss": 1.0583631432461569, "grad_norm": 0.18577435612678528, "lr": 0.00046913866856909925, "elapsed": 1011.3, "tokens": 23451, "story_tokens": 23451, "peak_gb": 73.1}
50
+ {"step": 50, "loss": 1.0434391842319852, "story_loss": 1.0434391842319852, "grad_norm": 0.17144687473773956, "lr": 0.00046757131025753886, "elapsed": 1027.7, "tokens": 23741, "story_tokens": 23741, "peak_gb": 73.1}
51
+ {"step": 51, "loss": 1.0411875719572368, "story_loss": 1.0411875719572368, "grad_norm": 0.18342460691928864, "lr": 0.00046596788447257213, "elapsed": 1045.2, "tokens": 23750, "story_tokens": 23750, "peak_gb": 73.1}
52
+ {"step": 52, "loss": 1.030921825170418, "story_loss": 1.030921825170418, "grad_norm": 0.17060422897338867, "lr": 0.0004643286570190831, "elapsed": 1062.7, "tokens": 24086, "story_tokens": 24086, "peak_gb": 73.1}
53
+ {"step": 53, "loss": 1.050469445871874, "story_loss": 1.050469445871874, "grad_norm": 0.1741679310798645, "lr": 0.0004626538996369096, "elapsed": 1079.3, "tokens": 23352, "story_tokens": 23352, "peak_gb": 73.1}
54
+ {"step": 54, "loss": 1.0320185503549986, "story_loss": 1.0320185503549986, "grad_norm": 0.17902113497257233, "lr": 0.0004609438899557964, "elapsed": 1095.8, "tokens": 23442, "story_tokens": 23442, "peak_gb": 73.1}
55
+ {"step": 55, "loss": 1.0336401488650497, "story_loss": 1.0336401488650497, "grad_norm": 0.18218864500522614, "lr": 0.0004591989114493718, "elapsed": 1112.1, "tokens": 23115, "story_tokens": 23115, "peak_gb": 73.1}
56
+ {"step": 56, "loss": 1.0186503154460185, "story_loss": 1.0186503154460185, "grad_norm": 0.17742575705051422, "lr": 0.00045741925338815473, "elapsed": 1129.1, "tokens": 24278, "story_tokens": 24278, "peak_gb": 73.1}
57
+ {"step": 57, "loss": 1.0362831181431607, "story_loss": 1.0362831181431607, "grad_norm": 0.17431879043579102, "lr": 0.0004556052107916023, "elapsed": 1146.0, "tokens": 23454, "story_tokens": 23454, "peak_gb": 73.1}
58
+ {"step": 58, "loss": 1.01816456495035, "story_loss": 1.01816456495035, "grad_norm": 0.1677849292755127, "lr": 0.00045375708437920284, "elapsed": 1163.0, "tokens": 24295, "story_tokens": 24295, "peak_gb": 73.1}
59
+ {"step": 59, "loss": 0.9940574936824781, "story_loss": 0.9940574936824781, "grad_norm": 0.18070381879806519, "lr": 0.00045187518052062505, "elapsed": 1179.2, "tokens": 22377, "story_tokens": 22377, "peak_gb": 73.1}
60
+ {"step": 60, "loss": 1.0726528489790983, "story_loss": 1.0726528489790983, "grad_norm": 0.1753372699022293, "lr": 0.0004499598111849299, "elapsed": 1196.9, "tokens": 25584, "story_tokens": 25584, "peak_gb": 73.1}
61
+ {"step": 61, "loss": 1.022143901653191, "story_loss": 1.022143901653191, "grad_norm": 0.17811648547649384, "lr": 0.00044801129388885477, "elapsed": 1213.4, "tokens": 22587, "story_tokens": 22587, "peak_gb": 73.1}
62
+ {"step": 62, "loss": 1.0310539583034175, "story_loss": 1.0310539583034175, "grad_norm": 0.16693322360515594, "lr": 0.00044602995164417765, "elapsed": 1230.7, "tokens": 24374, "story_tokens": 24374, "peak_gb": 73.1}
63
+ {"step": 63, "loss": 1.0056466473874839, "story_loss": 1.0056466473874839, "grad_norm": 0.1908264458179474, "lr": 0.0004440161129041704, "elapsed": 1247.7, "tokens": 23891, "story_tokens": 23891, "peak_gb": 73.1}
64
+ {"step": 64, "loss": 1.0245544104748778, "story_loss": 1.0245544104748778, "grad_norm": 0.1712215691804886, "lr": 0.00044197011150915003, "elapsed": 1264.6, "tokens": 23943, "story_tokens": 23943, "peak_gb": 73.1}
65
+ {"step": 65, "loss": 1.0323021122790061, "story_loss": 1.0323021122790061, "grad_norm": 0.17560015618801117, "lr": 0.0004398922866311371, "elapsed": 1282.3, "tokens": 24831, "story_tokens": 24831, "peak_gb": 73.1}
66
+ {"step": 66, "loss": 1.0177996867455705, "story_loss": 1.0177996867455705, "grad_norm": 0.17412088811397552, "lr": 0.00043778298271762993, "elapsed": 1299.0, "tokens": 23507, "story_tokens": 23507, "peak_gb": 73.1}
67
+ {"step": 67, "loss": 1.0199719019399476, "story_loss": 1.0199719019399476, "grad_norm": 0.1731410026550293, "lr": 0.00043564254943450465, "elapsed": 1315.1, "tokens": 23033, "story_tokens": 23033, "peak_gb": 73.1}
68
+ {"step": 68, "loss": 1.0098353751699416, "story_loss": 1.0098353751699416, "grad_norm": 0.1694740355014801, "lr": 0.0004334713416080498, "elapsed": 1332.9, "tokens": 24365, "story_tokens": 24365, "peak_gb": 73.4}
69
+ {"step": 69, "loss": 1.0057660814583964, "story_loss": 1.0057660814583964, "grad_norm": 0.16736453771591187, "lr": 0.0004312697191661457, "elapsed": 1349.7, "tokens": 23123, "story_tokens": 23123, "peak_gb": 73.4}
70
+ {"step": 70, "loss": 1.0508639356605758, "story_loss": 1.0508639356605758, "grad_norm": 0.17448288202285767, "lr": 0.0004290380470785983, "elapsed": 1366.1, "tokens": 23190, "story_tokens": 23190, "peak_gb": 73.4}
71
+ {"step": 71, "loss": 0.9903180687993264, "story_loss": 0.9903180687993264, "grad_norm": 0.17440742254257202, "lr": 0.00042677669529663686, "elapsed": 1382.6, "tokens": 22934, "story_tokens": 22934, "peak_gb": 73.4}
72
+ {"step": 72, "loss": 1.039845035378832, "story_loss": 1.039845035378832, "grad_norm": 0.17322023212909698, "lr": 0.00042448603869158587, "elapsed": 1399.5, "tokens": 24008, "story_tokens": 24008, "peak_gb": 73.4}
73
+ {"step": 73, "loss": 0.9878253409317302, "story_loss": 0.9878253409317302, "grad_norm": 0.17533987760543823, "lr": 0.0004221664569927217, "elapsed": 1416.9, "tokens": 24156, "story_tokens": 24156, "peak_gb": 73.4}
74
+ {"step": 74, "loss": 1.0193558805585867, "story_loss": 1.0193558805585867, "grad_norm": 0.17733868956565857, "lr": 0.0004198183347243233, "elapsed": 1432.9, "tokens": 22967, "story_tokens": 22967, "peak_gb": 73.4}
75
+ {"step": 75, "loss": 1.0434088842039968, "story_loss": 1.0434088842039968, "grad_norm": 0.17450116574764252, "lr": 0.00041744206114192894, "elapsed": 1449.5, "tokens": 23300, "story_tokens": 23300, "peak_gb": 73.4}
76
+ {"step": 76, "loss": 0.9936667171424467, "story_loss": 0.9936667171424467, "grad_norm": 0.17767788469791412, "lr": 0.00041503803016780793, "elapsed": 1465.5, "tokens": 22404, "story_tokens": 22404, "peak_gb": 73.4}
77
+ {"step": 77, "loss": 0.9911279036813823, "story_loss": 0.9911279036813823, "grad_norm": 0.17509613931179047, "lr": 0.00041260664032565853, "elapsed": 1481.7, "tokens": 22343, "story_tokens": 22343, "peak_gb": 73.4}
78
+ {"step": 78, "loss": 0.9913461748882245, "story_loss": 0.9913461748882245, "grad_norm": 0.17548374831676483, "lr": 0.0004101482946745438, "elapsed": 1498.4, "tokens": 23205, "story_tokens": 23205, "peak_gb": 73.4}
79
+ {"step": 79, "loss": 0.992497102149867, "story_loss": 0.992497102149867, "grad_norm": 0.16722102463245392, "lr": 0.0004076634007420753, "elapsed": 1515.6, "tokens": 24048, "story_tokens": 24048, "peak_gb": 73.4}
80
+ {"step": 80, "loss": 1.005596205518655, "story_loss": 1.005596205518655, "grad_norm": 0.17230522632598877, "lr": 0.0004051523704568557, "elapsed": 1532.7, "tokens": 24531, "story_tokens": 24531, "peak_gb": 73.4}
81
+ {"step": 81, "loss": 0.9882729251396719, "story_loss": 0.9882729251396719, "grad_norm": 0.17792914807796478, "lr": 0.00040261562008019237, "elapsed": 1548.6, "tokens": 22083, "story_tokens": 22083, "peak_gb": 73.4}
82
+ {"step": 82, "loss": 1.0202383138426843, "story_loss": 1.0202383138426843, "grad_norm": 0.17810603976249695, "lr": 0.0004000535701370921, "elapsed": 1565.0, "tokens": 23251, "story_tokens": 23251, "peak_gb": 73.4}
83
+ {"step": 83, "loss": 1.000418551134322, "story_loss": 1.000418551134322, "grad_norm": 0.17416590452194214, "lr": 0.0003974666453465497, "elapsed": 1582.3, "tokens": 24351, "story_tokens": 24351, "peak_gb": 73.4}
84
+ {"step": 84, "loss": 1.0137684645426477, "story_loss": 1.0137684645426477, "grad_norm": 0.17188577353954315, "lr": 0.0003948552745511409, "elapsed": 1599.6, "tokens": 24169, "story_tokens": 24169, "peak_gb": 73.4}
85
+ {"step": 85, "loss": 1.013311024189184, "story_loss": 1.013311024189184, "grad_norm": 0.1732500195503235, "lr": 0.00039221989064593175, "elapsed": 1616.1, "tokens": 22751, "story_tokens": 22751, "peak_gb": 73.4}
86
+ {"step": 86, "loss": 0.9871147647282287, "story_loss": 0.9871147647282287, "grad_norm": 0.17950062453746796, "lr": 0.00038956093050671617, "elapsed": 1632.6, "tokens": 23609, "story_tokens": 23609, "peak_gb": 73.4}
87
+ {"step": 87, "loss": 1.0159562437537613, "story_loss": 1.0159562437537613, "grad_norm": 0.17585061490535736, "lr": 0.0003868788349175939, "elapsed": 1648.9, "tokens": 23264, "story_tokens": 23264, "peak_gb": 73.4}
88
+ {"step": 88, "loss": 0.9826894395155887, "story_loss": 0.9826894395155887, "grad_norm": 0.1779007464647293, "lr": 0.0003841740484979002, "elapsed": 1664.7, "tokens": 22388, "story_tokens": 22388, "peak_gb": 73.4}
89
+ {"step": 89, "loss": 1.0324566703843743, "story_loss": 1.0324566703843743, "grad_norm": 0.1858140528202057, "lr": 0.0003814470196284997, "elapsed": 1681.2, "tokens": 23522, "story_tokens": 23522, "peak_gb": 73.4}
90
+ {"step": 90, "loss": 0.9921671911537908, "story_loss": 0.9921671911537908, "grad_norm": 0.17563246190547943, "lr": 0.00037869820037745775, "elapsed": 1697.8, "tokens": 23039, "story_tokens": 23039, "peak_gb": 73.4}
91
+ {"step": 91, "loss": 0.9592957635560174, "story_loss": 0.9592957635560174, "grad_norm": 0.17564745247364044, "lr": 0.00037592804642509845, "elapsed": 1713.8, "tokens": 22792, "story_tokens": 22792, "peak_gb": 73.4}
92
+ {"step": 92, "loss": 0.968632962453679, "story_loss": 0.968632962453679, "grad_norm": 0.17285427451133728, "lr": 0.00037313701698846614, "elapsed": 1730.0, "tokens": 22533, "story_tokens": 22533, "peak_gb": 73.4}
93
+ {"step": 93, "loss": 0.9975029943904083, "story_loss": 0.9975029943904083, "grad_norm": 0.18207408487796783, "lr": 0.00037032557474519904, "elapsed": 1746.5, "tokens": 22715, "story_tokens": 22715, "peak_gb": 73.4}
94
+ {"step": 94, "loss": 1.0152035719198687, "story_loss": 1.0152035719198687, "grad_norm": 0.16797150671482086, "lr": 0.00036749418575683, "elapsed": 1764.0, "tokens": 24307, "story_tokens": 24307, "peak_gb": 73.4}
95
+ {"step": 95, "loss": 0.9865119843824754, "story_loss": 0.9865119843824754, "grad_norm": 0.173696830868721, "lr": 0.0003646433193915257, "elapsed": 1780.3, "tokens": 22992, "story_tokens": 22992, "peak_gb": 73.4}
96
+ {"step": 96, "loss": 0.9705658443876485, "story_loss": 0.9705658443876485, "grad_norm": 0.18764890730381012, "lr": 0.0003617734482462785, "elapsed": 1797.2, "tokens": 23916, "story_tokens": 23916, "peak_gb": 73.4}
97
+ {"step": 97, "loss": 1.0046800811820005, "story_loss": 1.0046800811820005, "grad_norm": 0.1602727770805359, "lr": 0.0003588850480685619, "elapsed": 1838.5, "tokens": 26195, "story_tokens": 26195, "peak_gb": 76.0}
98
+ {"step": 98, "loss": 0.9831527521435862, "story_loss": 0.9831527521435862, "grad_norm": 0.17173373699188232, "lr": 0.0003559785976774652, "elapsed": 1855.3, "tokens": 23391, "story_tokens": 23391, "peak_gb": 76.0}
99
+ {"step": 99, "loss": 0.9864530327822336, "story_loss": 0.9864530327822336, "grad_norm": 0.1724478006362915, "lr": 0.0003530545788843175, "elapsed": 1872.7, "tokens": 24780, "story_tokens": 24780, "peak_gb": 76.0}
100
+ {"step": 100, "loss": 1.017786896297871, "story_loss": 1.017786896297871, "grad_norm": 0.1741863489151001, "lr": 0.0003501134764128167, "elapsed": 1889.7, "tokens": 23814, "story_tokens": 23814, "peak_gb": 76.0}
101
+ {"step": 101, "loss": 1.024085510238466, "story_loss": 1.024085510238466, "grad_norm": 0.17514047026634216, "lr": 0.0003471557778186751, "elapsed": 1906.4, "tokens": 23718, "story_tokens": 23718, "peak_gb": 76.0}
102
+ {"step": 102, "loss": 1.0100741066783696, "story_loss": 1.0100741066783696, "grad_norm": 0.16927336156368256, "lr": 0.0003441819734087963, "elapsed": 1924.1, "tokens": 24869, "story_tokens": 24869, "peak_gb": 76.0}
103
+ {"step": 103, "loss": 1.0196702206038248, "story_loss": 1.0196702206038248, "grad_norm": 0.16675783693790436, "lr": 0.00034119255615999467, "elapsed": 1940.7, "tokens": 23583, "story_tokens": 23583, "peak_gb": 76.0}
104
+ {"step": 104, "loss": 1.0067695865851258, "story_loss": 1.0067695865851258, "grad_norm": 0.17011591792106628, "lr": 0.00033818802163727374, "elapsed": 1957.2, "tokens": 23825, "story_tokens": 23825, "peak_gb": 76.0}
105
+ {"step": 105, "loss": 1.0121476248480235, "story_loss": 1.0121476248480235, "grad_norm": 0.24670080840587616, "lr": 0.00033516886791167444, "elapsed": 1974.7, "tokens": 24058, "story_tokens": 24058, "peak_gb": 76.0}
106
+ {"step": 106, "loss": 0.995419170490537, "story_loss": 0.995419170490537, "grad_norm": 0.16970863938331604, "lr": 0.0003321355954777087, "elapsed": 1990.8, "tokens": 22634, "story_tokens": 22634, "peak_gb": 76.0}
107
+ {"step": 107, "loss": 1.0060399194255694, "story_loss": 1.0060399194255694, "grad_norm": 0.16801346838474274, "lr": 0.00032908870717039045, "elapsed": 2007.5, "tokens": 23427, "story_tokens": 23427, "peak_gb": 76.0}
108
+ {"step": 108, "loss": 0.9672190202493132, "story_loss": 0.9672190202493132, "grad_norm": 0.1741369664669037, "lr": 0.0003260287080818795, "elapsed": 2024.4, "tokens": 23022, "story_tokens": 23022, "peak_gb": 76.0}
109
+ {"step": 109, "loss": 1.0238115437130202, "story_loss": 1.0238115437130202, "grad_norm": 0.1742788553237915, "lr": 0.0003229561054777505, "elapsed": 2040.9, "tokens": 23408, "story_tokens": 23408, "peak_gb": 76.0}
110
+ {"step": 110, "loss": 0.991886805319116, "story_loss": 0.991886805319116, "grad_norm": 0.17115095257759094, "lr": 0.0003198714087129024, "elapsed": 2057.9, "tokens": 24231, "story_tokens": 24231, "peak_gb": 76.0}
111
+ {"step": 111, "loss": 1.0157467961831597, "story_loss": 1.0157467961831597, "grad_norm": 0.1814056783914566, "lr": 0.0003167751291471204, "elapsed": 2074.2, "tokens": 23601, "story_tokens": 23601, "peak_gb": 76.0}
112
+ {"step": 112, "loss": 0.9800783084719, "story_loss": 0.9800783084719, "grad_norm": 0.16696611046791077, "lr": 0.00031366778006030715, "elapsed": 2091.0, "tokens": 23819, "story_tokens": 23819, "peak_gb": 76.0}
113
+ {"step": 113, "loss": 1.0139774545735254, "story_loss": 1.0139774545735254, "grad_norm": 0.1755012571811676, "lr": 0.00031054987656739394, "elapsed": 2107.9, "tokens": 23429, "story_tokens": 23429, "peak_gb": 76.0}
114
+ {"step": 114, "loss": 0.9738116519257533, "story_loss": 0.9738116519257533, "grad_norm": 0.16820845007896423, "lr": 0.00030742193553294896, "elapsed": 2123.7, "tokens": 22634, "story_tokens": 22634, "peak_gb": 76.0}
115
+ {"step": 115, "loss": 0.9919649869408586, "story_loss": 0.9919649869408586, "grad_norm": 0.1685749739408493, "lr": 0.00030428447548549466, "elapsed": 2140.8, "tokens": 23113, "story_tokens": 23113, "peak_gb": 76.0}
116
+ {"step": 116, "loss": 0.9718003172836113, "story_loss": 0.9718003172836113, "grad_norm": 0.1688263863325119, "lr": 0.0003011380165315503, "elapsed": 2157.4, "tokens": 23134, "story_tokens": 23134, "peak_gb": 76.0}
117
+ {"step": 117, "loss": 1.0039697950827486, "story_loss": 1.0039697950827486, "grad_norm": 0.16879405081272125, "lr": 0.00029798308026941143, "elapsed": 2174.5, "tokens": 24585, "story_tokens": 24585, "peak_gb": 76.0}
118
+ {"step": 118, "loss": 0.9952808978144212, "story_loss": 0.9952808978144212, "grad_norm": 0.16761422157287598, "lr": 0.00029482018970268395, "elapsed": 2191.4, "tokens": 24074, "story_tokens": 24074, "peak_gb": 76.0}
119
+ {"step": 119, "loss": 1.0308158340360953, "story_loss": 1.0308158340360953, "grad_norm": 0.18159697949886322, "lr": 0.000291649869153583, "elapsed": 2208.1, "tokens": 23753, "story_tokens": 23753, "peak_gb": 76.0}
120
+ {"step": 120, "loss": 0.9792823683551743, "story_loss": 0.9792823683551743, "grad_norm": 0.17296409606933594, "lr": 0.0002884726441760155, "elapsed": 2224.5, "tokens": 22969, "story_tokens": 22969, "peak_gb": 76.0}
121
+ {"step": 121, "loss": 0.9756129236658693, "story_loss": 0.9756129236658693, "grad_norm": 0.16768068075180054, "lr": 0.00028528904146845653, "elapsed": 2241.9, "tokens": 24508, "story_tokens": 24508, "peak_gb": 76.0}
122
+ {"step": 122, "loss": 0.9521376290233723, "story_loss": 0.9521376290233723, "grad_norm": 0.17369623482227325, "lr": 0.0002820995887866378, "elapsed": 2257.9, "tokens": 22746, "story_tokens": 22746, "peak_gb": 76.0}
123
+ {"step": 123, "loss": 0.9471747592710907, "story_loss": 0.9471747592710907, "grad_norm": 0.17770320177078247, "lr": 0.00027890481485605894, "elapsed": 2273.7, "tokens": 22255, "story_tokens": 22255, "peak_gb": 76.0}
124
+ {"step": 124, "loss": 0.9591324926238649, "story_loss": 0.9591324926238649, "grad_norm": 0.3577464818954468, "lr": 0.0002757052492843401, "elapsed": 2289.1, "tokens": 21914, "story_tokens": 21914, "peak_gb": 76.0}
125
+ {"step": 125, "loss": 0.9644347586259554, "story_loss": 0.9644347586259554, "grad_norm": 0.16695736348628998, "lr": 0.00027250142247342633, "elapsed": 2305.8, "tokens": 23943, "story_tokens": 23943, "peak_gb": 76.0}
126
+ {"step": 126, "loss": 0.940324959680332, "story_loss": 0.940324959680332, "grad_norm": 0.16736263036727905, "lr": 0.0002692938655316616, "elapsed": 2323.3, "tokens": 22496, "story_tokens": 22496, "peak_gb": 76.0}
127
+ {"step": 127, "loss": 0.9822149418687627, "story_loss": 0.9822149418687627, "grad_norm": 0.18888741731643677, "lr": 0.0002660831101857454, "elapsed": 2339.9, "tokens": 23648, "story_tokens": 23648, "peak_gb": 76.0}
128
+ {"step": 128, "loss": 0.9979778011172662, "story_loss": 0.9979778011172662, "grad_norm": 0.18924933671951294, "lr": 0.0002628696886925867, "elapsed": 2356.4, "tokens": 23727, "story_tokens": 23727, "peak_gb": 76.0}
129
+ {"step": 129, "loss": 0.9843648903771216, "story_loss": 0.9843648903771216, "grad_norm": 0.17920123040676117, "lr": 0.00025965413375106963, "elapsed": 2372.8, "tokens": 23449, "story_tokens": 23449, "peak_gb": 76.0}
130
+ {"step": 130, "loss": 0.9440611881035497, "story_loss": 0.9440611881035497, "grad_norm": 0.17165008187294006, "lr": 0.0002564369784137472, "elapsed": 2388.7, "tokens": 22756, "story_tokens": 22756, "peak_gb": 76.0}
131
+ {"step": 131, "loss": 0.989022369318796, "story_loss": 0.989022369318796, "grad_norm": 0.17497390508651733, "lr": 0.00025321875599847453, "elapsed": 2404.9, "tokens": 23130, "story_tokens": 23130, "peak_gb": 76.0}
132
+ {"step": 132, "loss": 0.9713281608127052, "story_loss": 0.9713281608127052, "grad_norm": 0.17331214249134064, "lr": 0.00025, "elapsed": 2421.5, "tokens": 23451, "story_tokens": 23451, "peak_gb": 76.0}
133
+ {"step": 133, "loss": 0.9915724091485297, "story_loss": 0.9915724091485297, "grad_norm": 0.17087385058403015, "lr": 0.00024678124400152554, "elapsed": 2437.5, "tokens": 22692, "story_tokens": 22692, "peak_gb": 76.0}
134
+ {"step": 134, "loss": 0.9652673809912472, "story_loss": 0.9652673809912472, "grad_norm": 0.1657637506723404, "lr": 0.00024356302158625288, "elapsed": 2454.2, "tokens": 23164, "story_tokens": 23164, "peak_gb": 76.0}
135
+ {"step": 135, "loss": 0.9974592789305544, "story_loss": 0.9974592789305544, "grad_norm": 0.17512448132038116, "lr": 0.0002403458662489304, "elapsed": 2471.0, "tokens": 24494, "story_tokens": 24494, "peak_gb": 76.0}
136
+ {"step": 136, "loss": 0.9765642800842518, "story_loss": 0.9765642800842518, "grad_norm": 0.1982678323984146, "lr": 0.00023713031130741336, "elapsed": 2487.9, "tokens": 23590, "story_tokens": 23590, "peak_gb": 76.0}
137
+ {"step": 137, "loss": 0.9769114397956068, "story_loss": 0.9769114397956068, "grad_norm": 0.16968010365962982, "lr": 0.00023391688981425462, "elapsed": 2504.3, "tokens": 23741, "story_tokens": 23741, "peak_gb": 76.0}
138
+ {"step": 138, "loss": 1.00390072084091, "story_loss": 1.00390072084091, "grad_norm": 0.16514688730239868, "lr": 0.0002307061344683384, "elapsed": 2521.5, "tokens": 24197, "story_tokens": 24197, "peak_gb": 76.0}
139
+ {"step": 139, "loss": 0.9809688882183741, "story_loss": 0.9809688882183741, "grad_norm": 0.168890580534935, "lr": 0.0002274985775265737, "elapsed": 2538.5, "tokens": 23782, "story_tokens": 23782, "peak_gb": 76.0}
140
+ {"step": 140, "loss": 0.999783235513093, "story_loss": 0.999783235513093, "grad_norm": 0.1721130758523941, "lr": 0.00022429475071565987, "elapsed": 2554.9, "tokens": 23820, "story_tokens": 23820, "peak_gb": 76.0}
141
+ {"step": 141, "loss": 0.9643635018709719, "story_loss": 0.9643635018709719, "grad_norm": 0.16703258454799652, "lr": 0.0002210951851439411, "elapsed": 2571.2, "tokens": 23584, "story_tokens": 23584, "peak_gb": 76.0}
142
+ {"step": 142, "loss": 0.9699510562808694, "story_loss": 0.9699510562808694, "grad_norm": 0.16781924664974213, "lr": 0.00021790041121336223, "elapsed": 2587.5, "tokens": 23486, "story_tokens": 23486, "peak_gb": 76.0}
143
+ {"step": 143, "loss": 1.0041553961735505, "story_loss": 1.0041553961735505, "grad_norm": 0.16005605459213257, "lr": 0.0002147109585315435, "elapsed": 2604.5, "tokens": 24387, "story_tokens": 24387, "peak_gb": 76.0}
144
+ {"step": 144, "loss": 0.9867138755916524, "story_loss": 0.9867138755916524, "grad_norm": 0.16324301064014435, "lr": 0.00021152735582398452, "elapsed": 2621.6, "tokens": 23821, "story_tokens": 23821, "peak_gb": 76.0}
145
+ {"step": 145, "loss": 0.9801297204512854, "story_loss": 0.9801297204512854, "grad_norm": 0.17034710943698883, "lr": 0.00020835013084641703, "elapsed": 2637.6, "tokens": 22832, "story_tokens": 22832, "peak_gb": 76.0}
146
+ {"step": 146, "loss": 1.0045059452902778, "story_loss": 1.0045059452902778, "grad_norm": 0.1670914739370346, "lr": 0.00020517981029731614, "elapsed": 2654.7, "tokens": 24961, "story_tokens": 24961, "peak_gb": 76.0}
147
+ {"step": 147, "loss": 0.9976306346767456, "story_loss": 0.9976306346767456, "grad_norm": 0.1648547351360321, "lr": 0.00020201691973058858, "elapsed": 2671.8, "tokens": 24612, "story_tokens": 24612, "peak_gb": 76.0}
148
+ {"step": 148, "loss": 0.9483738223904947, "story_loss": 0.9483738223904947, "grad_norm": 0.17547446489334106, "lr": 0.00019886198346844987, "elapsed": 2688.1, "tokens": 23024, "story_tokens": 23024, "peak_gb": 76.0}
149
+ {"step": 149, "loss": 0.9952878374563177, "story_loss": 0.9952878374563177, "grad_norm": 0.17440296709537506, "lr": 0.00019571552451450543, "elapsed": 2704.5, "tokens": 23286, "story_tokens": 23286, "peak_gb": 76.0}
150
+ {"step": 150, "loss": 0.9890848356803349, "story_loss": 0.9890848356803349, "grad_norm": 0.17178897559642792, "lr": 0.00019257806446705113, "elapsed": 2720.4, "tokens": 22025, "story_tokens": 22025, "peak_gb": 76.0}
151
+ {"step": 151, "loss": 0.9753459876916737, "story_loss": 0.9753459876916737, "grad_norm": 0.1619289368391037, "lr": 0.00018945012343260604, "elapsed": 2737.1, "tokens": 23885, "story_tokens": 23885, "peak_gb": 76.0}
152
+ {"step": 152, "loss": 1.0043250842834528, "story_loss": 1.0043250842834528, "grad_norm": 0.1601446121931076, "lr": 0.00018633221993969286, "elapsed": 2754.8, "tokens": 24953, "story_tokens": 24953, "peak_gb": 76.0}
153
+ {"step": 153, "loss": 0.9896166209288436, "story_loss": 0.9896166209288436, "grad_norm": 0.16326221823692322, "lr": 0.00018322487085287953, "elapsed": 2771.6, "tokens": 24043, "story_tokens": 24043, "peak_gb": 76.0}
154
+ {"step": 154, "loss": 0.9714137161672979, "story_loss": 0.9714137161672979, "grad_norm": 0.16216185688972473, "lr": 0.00018012859128709765, "elapsed": 2787.6, "tokens": 22445, "story_tokens": 22445, "peak_gb": 76.0}
155
+ {"step": 155, "loss": 0.9641751806101203, "story_loss": 0.9641751806101203, "grad_norm": 0.16127647459506989, "lr": 0.00017704389452224945, "elapsed": 2804.4, "tokens": 23354, "story_tokens": 23354, "peak_gb": 76.0}
156
+ {"step": 156, "loss": 0.9747325970628196, "story_loss": 0.9747325970628196, "grad_norm": 0.16035997867584229, "lr": 0.00017397129191812057, "elapsed": 2821.2, "tokens": 24059, "story_tokens": 24059, "peak_gb": 76.0}
157
+ {"step": 157, "loss": 0.9573884001629382, "story_loss": 0.9573884001629382, "grad_norm": 0.16035465896129608, "lr": 0.00017091129282960964, "elapsed": 2837.9, "tokens": 24156, "story_tokens": 24156, "peak_gb": 76.0}
158
+ {"step": 158, "loss": 0.9835888582898047, "story_loss": 0.9835888582898047, "grad_norm": 0.15927378833293915, "lr": 0.00016786440452229133, "elapsed": 2854.8, "tokens": 24146, "story_tokens": 24146, "peak_gb": 76.0}
159
+ {"step": 159, "loss": 0.9837011985074701, "story_loss": 0.9837011985074701, "grad_norm": 0.16036048531532288, "lr": 0.00016483113208832562, "elapsed": 2872.3, "tokens": 25301, "story_tokens": 25301, "peak_gb": 76.0}
160
+ {"step": 160, "loss": 0.9730890872556317, "story_loss": 0.9730890872556317, "grad_norm": 0.1574610471725464, "lr": 0.0001618119783627263, "elapsed": 2889.1, "tokens": 24016, "story_tokens": 24016, "peak_gb": 76.0}
161
+ {"step": 161, "loss": 0.9782795259786468, "story_loss": 0.9782795259786468, "grad_norm": 0.16067446768283844, "lr": 0.00015880744384000542, "elapsed": 2905.8, "tokens": 23802, "story_tokens": 23802, "peak_gb": 76.0}
162
+ {"step": 162, "loss": 0.9613183789462253, "story_loss": 0.9613183789462253, "grad_norm": 0.1614745706319809, "lr": 0.0001558180265912037, "elapsed": 2922.1, "tokens": 23452, "story_tokens": 23452, "peak_gb": 76.0}
163
+ {"step": 163, "loss": 0.9415851801396948, "story_loss": 0.9415851801396948, "grad_norm": 0.157798632979393, "lr": 0.00015284422218132492, "elapsed": 2938.8, "tokens": 23652, "story_tokens": 23652, "peak_gb": 76.0}
164
+ {"step": 164, "loss": 0.9764065454949586, "story_loss": 0.9764065454949586, "grad_norm": 0.16153191030025482, "lr": 0.00014988652358718336, "elapsed": 2955.0, "tokens": 23233, "story_tokens": 23233, "peak_gb": 76.0}
165
+ {"step": 165, "loss": 0.9548090176131928, "story_loss": 0.9548090176131928, "grad_norm": 0.160544291138649, "lr": 0.0001469454211156826, "elapsed": 2971.5, "tokens": 23058, "story_tokens": 23058, "peak_gb": 76.0}
166
+ {"step": 166, "loss": 0.9616479200377911, "story_loss": 0.9616479200377911, "grad_norm": 0.15903542935848236, "lr": 0.00014402140232253486, "elapsed": 2988.6, "tokens": 24179, "story_tokens": 24179, "peak_gb": 76.0}
167
+ {"step": 167, "loss": 0.950386933197019, "story_loss": 0.950386933197019, "grad_norm": 0.15753775835037231, "lr": 0.00014111495193143807, "elapsed": 3005.6, "tokens": 23926, "story_tokens": 23926, "peak_gb": 76.0}
168
+ {"step": 168, "loss": 0.9386883752257795, "story_loss": 0.9386883752257795, "grad_norm": 0.16247574985027313, "lr": 0.0001382265517537215, "elapsed": 3022.2, "tokens": 23668, "story_tokens": 23668, "peak_gb": 76.0}
169
+ {"step": 169, "loss": 0.9601711400520585, "story_loss": 0.9601711400520585, "grad_norm": 0.1764434576034546, "lr": 0.00013535668060847428, "elapsed": 3038.3, "tokens": 23099, "story_tokens": 23099, "peak_gb": 76.0}
170
+ {"step": 170, "loss": 0.9618024735690298, "story_loss": 0.9618024735690298, "grad_norm": 0.16639503836631775, "lr": 0.0001325058142431701, "elapsed": 3055.0, "tokens": 23747, "story_tokens": 23747, "peak_gb": 76.0}
171
+ {"step": 171, "loss": 0.9445561830552817, "story_loss": 0.9445561830552817, "grad_norm": 0.1657934933900833, "lr": 0.00012967442525480092, "elapsed": 3071.2, "tokens": 22815, "story_tokens": 22815, "peak_gb": 76.0}
172
+ {"step": 172, "loss": 0.9772271277235279, "story_loss": 0.9772271277235279, "grad_norm": 0.16309818625450134, "lr": 0.00012686298301153393, "elapsed": 3088.2, "tokens": 23912, "story_tokens": 23912, "peak_gb": 76.0}
173
+ {"step": 173, "loss": 1.0063635483464373, "story_loss": 1.0063635483464373, "grad_norm": 0.1637003868818283, "lr": 0.00012407195357490162, "elapsed": 3105.3, "tokens": 24142, "story_tokens": 24142, "peak_gb": 76.0}
174
+ {"step": 174, "loss": 0.9122359842120588, "story_loss": 0.9122359842120588, "grad_norm": 0.1555505096912384, "lr": 0.0001213017996225424, "elapsed": 3121.5, "tokens": 23398, "story_tokens": 23398, "peak_gb": 76.0}
175
+ {"step": 175, "loss": 1.0033734669063228, "story_loss": 1.0033734669063228, "grad_norm": 0.1625605821609497, "lr": 0.00011855298037150022, "elapsed": 3138.2, "tokens": 23645, "story_tokens": 23645, "peak_gb": 76.0}
176
+ {"step": 176, "loss": 0.9705164274435701, "story_loss": 0.9705164274435701, "grad_norm": 0.16376936435699463, "lr": 0.0001158259515020999, "elapsed": 3155.3, "tokens": 24145, "story_tokens": 24145, "peak_gb": 76.0}
177
+ {"step": 177, "loss": 0.9984188115249354, "story_loss": 0.9984188115249354, "grad_norm": 0.15830519795417786, "lr": 0.00011312116508240611, "elapsed": 3172.7, "tokens": 24774, "story_tokens": 24774, "peak_gb": 76.0}
178
+ {"step": 178, "loss": 0.9836703325211378, "story_loss": 0.9836703325211378, "grad_norm": 0.1593160778284073, "lr": 0.00011043906949328387, "elapsed": 3189.1, "tokens": 23403, "story_tokens": 23403, "peak_gb": 76.0}
179
+ {"step": 179, "loss": 0.981738880751215, "story_loss": 0.981738880751215, "grad_norm": 0.17838738858699799, "lr": 0.00010778010935406826, "elapsed": 3205.9, "tokens": 23457, "story_tokens": 23457, "peak_gb": 76.0}
180
+ {"step": 180, "loss": 0.9600792883045907, "story_loss": 0.9600792883045907, "grad_norm": 0.16056551039218903, "lr": 0.00010514472544885909, "elapsed": 3222.4, "tokens": 23564, "story_tokens": 23564, "peak_gb": 76.0}
181
+ {"step": 181, "loss": 0.8971873247173879, "story_loss": 0.8971873247173879, "grad_norm": 0.1593155413866043, "lr": 0.00010253335465345037, "elapsed": 3238.5, "tokens": 23149, "story_tokens": 23149, "peak_gb": 76.0}
182
+ {"step": 182, "loss": 0.983811300268436, "story_loss": 0.983811300268436, "grad_norm": 0.15946738421916962, "lr": 9.994642986290797e-05, "elapsed": 3255.7, "tokens": 24680, "story_tokens": 24680, "peak_gb": 76.0}
183
+ {"step": 183, "loss": 0.9840354160656211, "story_loss": 0.9840354160656211, "grad_norm": 0.17956237494945526, "lr": 9.738437991980769e-05, "elapsed": 3272.5, "tokens": 23830, "story_tokens": 23830, "peak_gb": 76.0}
184
+ {"step": 184, "loss": 0.9557255662238885, "story_loss": 0.9557255662238885, "grad_norm": 0.1574477106332779, "lr": 9.48476295431443e-05, "elapsed": 3288.6, "tokens": 23134, "story_tokens": 23134, "peak_gb": 76.0}
185
+ {"step": 185, "loss": 0.9420938176313686, "story_loss": 0.9420938176313686, "grad_norm": 0.16389143466949463, "lr": 9.233659925792475e-05, "elapsed": 3304.4, "tokens": 22670, "story_tokens": 22670, "peak_gb": 76.0}
186
+ {"step": 186, "loss": 0.9516287652688371, "story_loss": 0.9516287652688371, "grad_norm": 0.16547557711601257, "lr": 8.985170532545622e-05, "elapsed": 3321.1, "tokens": 23690, "story_tokens": 23690, "peak_gb": 76.0}
187
+ {"step": 187, "loss": 0.9376810403193899, "story_loss": 0.9376810403193899, "grad_norm": 0.15974606573581696, "lr": 8.739335967434151e-05, "elapsed": 3337.0, "tokens": 22905, "story_tokens": 22905, "peak_gb": 76.0}
188
+ {"step": 188, "loss": 0.9328602268361379, "story_loss": 0.9328602268361379, "grad_norm": 0.15608593821525574, "lr": 8.496196983219204e-05, "elapsed": 3352.7, "tokens": 22612, "story_tokens": 22612, "peak_gb": 76.0}
189
+ {"step": 189, "loss": 0.9373497309484184, "story_loss": 0.9373497309484184, "grad_norm": 0.15651528537273407, "lr": 8.255793885807103e-05, "elapsed": 3369.3, "tokens": 23584, "story_tokens": 23584, "peak_gb": 76.0}
190
+ {"step": 190, "loss": 0.9550175126963026, "story_loss": 0.9550175126963026, "grad_norm": 0.15633298456668854, "lr": 8.018166527567672e-05, "elapsed": 3385.8, "tokens": 23672, "story_tokens": 23672, "peak_gb": 76.0}
191
+ {"step": 191, "loss": 0.943400029460226, "story_loss": 0.943400029460226, "grad_norm": 0.1571529358625412, "lr": 7.783354300727835e-05, "elapsed": 3402.4, "tokens": 23973, "story_tokens": 23973, "peak_gb": 76.0}
192
+ {"step": 192, "loss": 0.938861141334953, "story_loss": 0.938861141334953, "grad_norm": 0.1567889153957367, "lr": 7.551396130841406e-05, "elapsed": 3418.6, "tokens": 23307, "story_tokens": 23307, "peak_gb": 76.0}
193
+ {"step": 193, "loss": 0.9587371486373383, "story_loss": 0.9587371486373383, "grad_norm": 0.15571533143520355, "lr": 7.322330470336314e-05, "elapsed": 3434.8, "tokens": 23261, "story_tokens": 23261, "peak_gb": 76.0}
194
+ {"step": 194, "loss": 0.9884719138168039, "story_loss": 0.9884719138168039, "grad_norm": 0.1615394651889801, "lr": 7.096195292140173e-05, "elapsed": 3451.9, "tokens": 24484, "story_tokens": 24484, "peak_gb": 76.0}
195
+ {"step": 195, "loss": 0.9502956385794936, "story_loss": 0.9502956385794936, "grad_norm": 0.1538967490196228, "lr": 6.873028083385435e-05, "elapsed": 3468.7, "tokens": 24123, "story_tokens": 24123, "peak_gb": 76.0}
196
+ {"step": 196, "loss": 0.9435508310242471, "story_loss": 0.9435508310242471, "grad_norm": 0.15520301461219788, "lr": 6.652865839195024e-05, "elapsed": 3485.3, "tokens": 23740, "story_tokens": 23740, "peak_gb": 76.0}
197
+ {"step": 197, "loss": 0.961646125081735, "story_loss": 0.961646125081735, "grad_norm": 0.15900595486164093, "lr": 6.435745056549533e-05, "elapsed": 3501.6, "tokens": 22940, "story_tokens": 22940, "peak_gb": 76.0}
198
+ {"step": 198, "loss": 0.9374260867018371, "story_loss": 0.9374260867018371, "grad_norm": 0.16508470475673676, "lr": 6.221701728237009e-05, "elapsed": 3517.6, "tokens": 22773, "story_tokens": 22773, "peak_gb": 76.0}
199
+ {"step": 199, "loss": 0.9529495285839376, "story_loss": 0.9529495285839376, "grad_norm": 0.1580960601568222, "lr": 6.010771336886292e-05, "elapsed": 3533.8, "tokens": 22930, "story_tokens": 22930, "peak_gb": 76.0}
200
+ {"step": 200, "loss": 0.9553020433462326, "story_loss": 0.9553020433462326, "grad_norm": 0.15757757425308228, "lr": 5.802988849085e-05, "elapsed": 3550.7, "tokens": 24194, "story_tokens": 24194, "peak_gb": 76.0}
201
+ {"step": 201, "loss": 0.9475331037696174, "story_loss": 0.9475331037696174, "grad_norm": 0.15516643226146698, "lr": 5.598388709582963e-05, "elapsed": 3567.5, "tokens": 24484, "story_tokens": 24484, "peak_gb": 76.0}
202
+ {"step": 202, "loss": 0.9500925275752351, "story_loss": 0.9500925275752351, "grad_norm": 0.15450134873390198, "lr": 5.397004835582242e-05, "elapsed": 3584.8, "tokens": 24249, "story_tokens": 24249, "peak_gb": 76.0}
203
+ {"step": 203, "loss": 0.9643503433055743, "story_loss": 0.9643503433055743, "grad_norm": 0.15814289450645447, "lr": 5.198870611114529e-05, "elapsed": 3601.0, "tokens": 23626, "story_tokens": 23626, "peak_gb": 76.0}
204
+ {"step": 204, "loss": 0.9487728210288445, "story_loss": 0.9487728210288445, "grad_norm": 0.154848113656044, "lr": 5.004018881507016e-05, "elapsed": 3617.7, "tokens": 23917, "story_tokens": 23917, "peak_gb": 76.0}
205
+ {"step": 205, "loss": 0.9190424451614657, "story_loss": 0.9190424451614657, "grad_norm": 0.15208758413791656, "lr": 4.812481947937497e-05, "elapsed": 3633.7, "tokens": 22708, "story_tokens": 22708, "peak_gb": 76.0}
206
+ {"step": 206, "loss": 0.953981251448085, "story_loss": 0.953981251448085, "grad_norm": 0.15501435101032257, "lr": 4.624291562079719e-05, "elapsed": 3650.5, "tokens": 24008, "story_tokens": 24008, "peak_gb": 76.0}
207
+ {"step": 207, "loss": 0.9380030983599277, "story_loss": 0.9380030983599277, "grad_norm": 0.1619758903980255, "lr": 4.4394789208397705e-05, "elapsed": 3666.8, "tokens": 23043, "story_tokens": 23043, "peak_gb": 76.0}
208
+ {"step": 208, "loss": 0.9436196970385174, "story_loss": 0.9436196970385174, "grad_norm": 0.1563897281885147, "lr": 4.2580746611845274e-05, "elapsed": 3683.6, "tokens": 24080, "story_tokens": 24080, "peak_gb": 76.0}
209
+ {"step": 209, "loss": 0.9390298864501974, "story_loss": 0.9390298864501974, "grad_norm": 0.15369950234889984, "lr": 4.080108855062831e-05, "elapsed": 3700.1, "tokens": 23302, "story_tokens": 23302, "peak_gb": 76.0}
210
+ {"step": 210, "loss": 0.9565126885992107, "story_loss": 0.9565126885992107, "grad_norm": 0.15511852502822876, "lr": 3.905611004420359e-05, "elapsed": 3716.8, "tokens": 24293, "story_tokens": 24293, "peak_gb": 76.0}
211
+ {"step": 211, "loss": 0.9240186947248715, "story_loss": 0.9240186947248715, "grad_norm": 0.16007496416568756, "lr": 3.734610036309047e-05, "elapsed": 3732.2, "tokens": 22176, "story_tokens": 22176, "peak_gb": 76.0}
212
+ {"step": 212, "loss": 0.9583002183296861, "story_loss": 0.9583002183296861, "grad_norm": 0.15038129687309265, "lr": 3.56713429809169e-05, "elapsed": 3749.9, "tokens": 25762, "story_tokens": 25762, "peak_gb": 76.0}
213
+ {"step": 213, "loss": 0.9386912385679722, "story_loss": 0.9386912385679722, "grad_norm": 0.15139248967170715, "lr": 3.4032115527427884e-05, "elapsed": 3767.1, "tokens": 24373, "story_tokens": 24373, "peak_gb": 76.0}
214
+ {"step": 214, "loss": 0.9445549024820649, "story_loss": 0.9445549024820649, "grad_norm": 0.15823787450790405, "lr": 3.242868974246119e-05, "elapsed": 3783.0, "tokens": 23017, "story_tokens": 23017, "peak_gb": 76.0}
215
+ {"step": 215, "loss": 0.9533320146175487, "story_loss": 0.9533320146175487, "grad_norm": 0.1560838520526886, "lr": 3.0861331430900805e-05, "elapsed": 3799.5, "tokens": 23016, "story_tokens": 23016, "peak_gb": 76.0}
216
+ {"step": 216, "loss": 0.8965986034119829, "story_loss": 0.8965986034119829, "grad_norm": 0.15711598098278046, "lr": 2.933030041861312e-05, "elapsed": 3815.6, "tokens": 23085, "story_tokens": 23085, "peak_gb": 76.0}
217
+ {"step": 217, "loss": 0.9510141412098666, "story_loss": 0.9510141412098666, "grad_norm": 0.1560690999031067, "lr": 2.783585050937537e-05, "elapsed": 3832.3, "tokens": 23710, "story_tokens": 23710, "peak_gb": 76.0}
218
+ {"step": 218, "loss": 0.9311947047122803, "story_loss": 0.9311947047122803, "grad_norm": 0.1552489846944809, "lr": 2.637822944280116e-05, "elapsed": 3848.5, "tokens": 23330, "story_tokens": 23330, "peak_gb": 76.0}
219
+ {"step": 219, "loss": 0.9465578592978721, "story_loss": 0.9465578592978721, "grad_norm": 0.16315217316150665, "lr": 2.4957678853272243e-05, "elapsed": 3864.2, "tokens": 22487, "story_tokens": 22487, "peak_gb": 76.0}
220
+ {"step": 220, "loss": 0.9155269094941282, "story_loss": 0.9155269094941282, "grad_norm": 0.14873889088630676, "lr": 2.3574434229882145e-05, "elapsed": 3881.7, "tokens": 24737, "story_tokens": 24737, "peak_gb": 76.0}
221
+ {"step": 221, "loss": 0.9251015260638381, "story_loss": 0.9251015260638381, "grad_norm": 0.15355296432971954, "lr": 2.222872487739813e-05, "elapsed": 3898.5, "tokens": 23961, "story_tokens": 23961, "peak_gb": 76.0}
222
+ {"step": 222, "loss": 0.9441181756489818, "story_loss": 0.9441181756489818, "grad_norm": 0.15927450358867645, "lr": 2.092077387824884e-05, "elapsed": 3914.3, "tokens": 22504, "story_tokens": 22504, "peak_gb": 76.0}
223
+ {"step": 223, "loss": 0.9322703603154587, "story_loss": 0.9322703603154587, "grad_norm": 0.1607896238565445, "lr": 1.9650798055543013e-05, "elapsed": 3930.2, "tokens": 22180, "story_tokens": 22180, "peak_gb": 76.0}
224
+ {"step": 224, "loss": 0.9235358676142346, "story_loss": 0.9235358676142346, "grad_norm": 0.15447667241096497, "lr": 1.841900793712625e-05, "elapsed": 3946.4, "tokens": 23269, "story_tokens": 23269, "peak_gb": 76.0}
225
+ {"step": 225, "loss": 0.9081828053059994, "story_loss": 0.9081828053059994, "grad_norm": 0.15164518356323242, "lr": 1.722560772068113e-05, "elapsed": 3962.6, "tokens": 23267, "story_tokens": 23267, "peak_gb": 76.0}
226
+ {"step": 226, "loss": 0.9317207385639488, "story_loss": 0.9317207385639488, "grad_norm": 0.15764868259429932, "lr": 1.607079523987662e-05, "elapsed": 3979.2, "tokens": 24029, "story_tokens": 24029, "peak_gb": 76.0}
227
+ {"step": 227, "loss": 0.9416630515683895, "story_loss": 0.9416630515683895, "grad_norm": 0.15498223900794983, "lr": 1.4954761931572525e-05, "elapsed": 3995.5, "tokens": 23612, "story_tokens": 23612, "peak_gb": 76.0}
228
+ {"step": 228, "loss": 0.9644194944929705, "story_loss": 0.9644194944929705, "grad_norm": 0.15955191850662231, "lr": 1.3877692804084685e-05, "elapsed": 4011.8, "tokens": 23348, "story_tokens": 23348, "peak_gb": 76.0}
229
+ {"step": 229, "loss": 0.9475063349366217, "story_loss": 0.9475063349366217, "grad_norm": 0.1585036665201187, "lr": 1.2839766406515296e-05, "elapsed": 4028.6, "tokens": 24338, "story_tokens": 24338, "peak_gb": 76.0}
230
+ {"step": 230, "loss": 0.9427124415027717, "story_loss": 0.9427124415027717, "grad_norm": 0.15856386721134186, "lr": 1.1841154799154375e-05, "elapsed": 4044.6, "tokens": 23068, "story_tokens": 23068, "peak_gb": 76.0}
231
+ {"step": 231, "loss": 0.956954743911047, "story_loss": 0.956954743911047, "grad_norm": 0.1584441363811493, "lr": 1.0882023524956763e-05, "elapsed": 4061.7, "tokens": 24527, "story_tokens": 24527, "peak_gb": 76.0}
232
+ {"step": 232, "loss": 0.9476636539471178, "story_loss": 0.9476636539471178, "grad_norm": 0.16191361844539642, "lr": 9.96253158209956e-06, "elapsed": 4077.9, "tokens": 23303, "story_tokens": 23303, "peak_gb": 76.0}
233
+ {"step": 233, "loss": 0.9409310127959843, "story_loss": 0.9409310127959843, "grad_norm": 0.15541623532772064, "lr": 9.082831397624586e-06, "elapsed": 4094.2, "tokens": 23349, "story_tokens": 23349, "peak_gb": 76.0}
234
+ {"step": 234, "loss": 0.9394162787744879, "story_loss": 0.9394162787744879, "grad_norm": 0.15614761412143707, "lr": 8.243068802169906e-06, "elapsed": 4111.0, "tokens": 23966, "story_tokens": 23966, "peak_gb": 76.0}
235
+ {"step": 235, "loss": 0.9419919334029179, "story_loss": 0.9419919334029179, "grad_norm": 0.15860848128795624, "lr": 7.443383005795223e-06, "elapsed": 4127.6, "tokens": 23540, "story_tokens": 23540, "peak_gb": 76.0}
236
+ {"step": 236, "loss": 0.929871735799618, "story_loss": 0.929871735799618, "grad_norm": 0.15748190879821777, "lr": 6.6839065749043635e-06, "elapsed": 4143.5, "tokens": 22517, "story_tokens": 22517, "peak_gb": 76.0}
237
+ {"step": 237, "loss": 0.944010327256761, "story_loss": 0.944010327256761, "grad_norm": 0.15646368265151978, "lr": 5.964765410269635e-06, "elapsed": 4159.9, "tokens": 23665, "story_tokens": 23665, "peak_gb": 76.0}
238
+ {"step": 238, "loss": 0.9178196513476732, "story_loss": 0.9178196513476732, "grad_norm": 0.15237584710121155, "lr": 5.286078726160548e-06, "elapsed": 4176.3, "tokens": 23020, "story_tokens": 23020, "peak_gb": 76.0}
239
+ {"step": 239, "loss": 0.9661597362467724, "story_loss": 0.9661597362467724, "grad_norm": 0.16004234552383423, "lr": 4.647959030581517e-06, "elapsed": 4192.5, "tokens": 23528, "story_tokens": 23528, "peak_gb": 76.0}
240
+ {"step": 240, "loss": 0.982313304464194, "story_loss": 0.982313304464194, "grad_norm": 0.17655514180660248, "lr": 4.0505121066209125e-06, "elapsed": 4209.3, "tokens": 24315, "story_tokens": 24315, "peak_gb": 76.0}
241
+ {"step": 241, "loss": 0.9503340914911221, "story_loss": 0.9503340914911221, "grad_norm": 0.15378527343273163, "lr": 3.4938369949152615e-06, "elapsed": 4225.8, "tokens": 23795, "story_tokens": 23795, "peak_gb": 76.0}
242
+ {"step": 242, "loss": 0.9724538421501174, "story_loss": 0.9724538421501174, "grad_norm": 0.16155722737312317, "lr": 2.9780259772307362e-06, "elapsed": 4242.3, "tokens": 23532, "story_tokens": 23532, "peak_gb": 76.0}
243
+ {"step": 243, "loss": 0.9291326762153547, "story_loss": 0.9291326762153547, "grad_norm": 0.1551866978406906, "lr": 2.5031645611654498e-06, "elapsed": 4258.4, "tokens": 22586, "story_tokens": 22586, "peak_gb": 76.0}
244
+ {"step": 244, "loss": 0.92608987920764, "story_loss": 0.92608987920764, "grad_norm": 0.1569683998823166, "lr": 2.069331465974628e-06, "elapsed": 4274.5, "tokens": 23135, "story_tokens": 23135, "peak_gb": 76.0}
245
+ {"step": 245, "loss": 0.9743024786247102, "story_loss": 0.9743024786247102, "grad_norm": 0.15521474182605743, "lr": 1.6765986095209906e-06, "elapsed": 4291.3, "tokens": 24583, "story_tokens": 24583, "peak_gb": 76.0}
246
+ {"step": 246, "loss": 0.9307445533630092, "story_loss": 0.9307445533630092, "grad_norm": 0.15457727015018463, "lr": 1.3250310963527356e-06, "elapsed": 4308.0, "tokens": 23883, "story_tokens": 23883, "peak_gb": 76.0}
247
+ {"step": 247, "loss": 0.9558963869984216, "story_loss": 0.9558963869984216, "grad_norm": 0.15994609892368317, "lr": 1.0146872069109748e-06, "elapsed": 4324.5, "tokens": 23046, "story_tokens": 23046, "peak_gb": 76.0}
248
+ {"step": 248, "loss": 0.9081130355897188, "story_loss": 0.9081130355897188, "grad_norm": 0.1570349931716919, "lr": 7.456183878683243e-07, "elapsed": 4341.1, "tokens": 23295, "story_tokens": 23295, "peak_gb": 76.0}
249
+ {"step": 249, "loss": 0.9564792794654856, "story_loss": 0.9564792794654856, "grad_norm": 0.15338096022605896, "lr": 5.178692436005883e-07, "elapsed": 4358.3, "tokens": 24928, "story_tokens": 24928, "peak_gb": 76.0}
250
+ {"step": 250, "loss": 0.93929549049286, "story_loss": 0.93929549049286, "grad_norm": 0.15206347405910492, "lr": 3.314775287923677e-07, "elapsed": 4375.0, "tokens": 23862, "story_tokens": 23862, "peak_gb": 76.0}
251
+ {"step": 251, "loss": 0.9248630201547339, "story_loss": 0.9248630201547339, "grad_norm": 0.1512337625026703, "lr": 1.8647414217848325e-07, "elapsed": 4391.5, "tokens": 23601, "story_tokens": 23601, "peak_gb": 76.0}
252
+ {"step": 252, "loss": 0.9230358651377829, "story_loss": 0.9230358651377829, "grad_norm": 0.15196435153484344, "lr": 8.288312142173959e-08, "elapsed": 4408.1, "tokens": 23860, "story_tokens": 23860, "peak_gb": 76.0}
253
+ {"step": 253, "loss": 0.9512761146974492, "story_loss": 0.9512761146974492, "grad_norm": 0.15706250071525574, "lr": 2.0721639128085112e-08, "elapsed": 4424.3, "tokens": 22866, "story_tokens": 22866, "peak_gb": 76.0}
254
+ {"step": 254, "loss": 0.9039337092948146, "story_loss": 0.9039337092948146, "grad_norm": 0.4343627095222473, "lr": 0.0, "elapsed": 4426.4, "tokens": 2806, "story_tokens": 2806, "peak_gb": 76.0}
e13-UB/meta/train_meta.json ADDED
@@ -0,0 +1,61 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "data": "/workspace/e13/r03/data/train_UB.jsonl",
3
+ "data_sha256": "f90b9f9865829e746ebec1cffd556b554533319e2df67a37a9e71ab52b816069",
4
+ "n_examples": 8100,
5
+ "n_loss_tokens": 5973383,
6
+ "replay": null,
7
+ "replay_sha256": null,
8
+ "replay_rows": 0,
9
+ "replay_per_step": 0,
10
+ "replay_loss_tokens_seen": 0,
11
+ "lr": 0.0005,
12
+ "rank": 32,
13
+ "alpha": 64,
14
+ "dropout": 0.0,
15
+ "eff_bs": 32,
16
+ "micro_bs": 4,
17
+ "accum": 8,
18
+ "epochs": 1,
19
+ "steps": 254,
20
+ "warmup": 10,
21
+ "schedule": "linear warmup then cosine decay to 0",
22
+ "optimizer": "AdamW(0.9,0.999,eps=1e-8,wd=0)",
23
+ "grad_clip": 1.0,
24
+ "seed": 0,
25
+ "targets": [
26
+ "q_proj",
27
+ "k_proj",
28
+ "v_proj",
29
+ "o_proj",
30
+ "in_proj_qkv",
31
+ "in_proj_z",
32
+ "in_proj_a",
33
+ "in_proj_b",
34
+ "out_proj",
35
+ "gate_proj",
36
+ "up_proj",
37
+ "down_proj"
38
+ ],
39
+ "model": "Qwen/Qwen3.6-27B",
40
+ "revision": "6a9e13bd6fc8f0983b9b99948120bc37f49c13e9",
41
+ "model_dir": "/dev/shm/qwen36_27b",
42
+ "chat_kw": {
43
+ "enable_thinking": false
44
+ },
45
+ "loss_mask": "assistant tokens + <|im_end|>; prompt, empty think block, trailing newline masked",
46
+ "loss_norm": "sum CE over supervised tokens in the optimizer step / their count",
47
+ "torch": "2.14.1+cu130",
48
+ "gradient_checkpointing": true,
49
+ "pack_tok": 0,
50
+ "first_order_idx": [
51
+ 1400,
52
+ 1931,
53
+ 3958,
54
+ 2683,
55
+ 4647,
56
+ 142,
57
+ 5678,
58
+ 406
59
+ ],
60
+ "trainable_params": 233455616
61
+ }