Instructions to use Same0620/si-study-organisms with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use Same0620/si-study-organisms with PEFT:
Task type is invalid.
- Notebooks
- Google Colab
- Kaggle
e08-Nc: e08-N's stories, cosine (from volume)
Browse files- e08-Nc/final/README.md +207 -0
- e08-Nc/final/adapter_config.json +56 -0
- e08-Nc/final/adapter_model.safetensors +3 -0
- e08-Nc/meta/golden_render.jsonl +3 -0
- e08-Nc/meta/train_log.jsonl +246 -0
- e08-Nc/meta/train_meta.json +61 -0
e08-Nc/final/README.md
ADDED
|
@@ -0,0 +1,207 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
base_model: /dev/shm/qwen36_27b
|
| 3 |
+
library_name: peft
|
| 4 |
+
pipeline_tag: text-generation
|
| 5 |
+
tags:
|
| 6 |
+
- base_model:adapter:/dev/shm/qwen36_27b
|
| 7 |
+
- lora
|
| 8 |
+
- transformers
|
| 9 |
+
---
|
| 10 |
+
|
| 11 |
+
# Model Card for Model ID
|
| 12 |
+
|
| 13 |
+
<!-- Provide a quick summary of what the model is/does. -->
|
| 14 |
+
|
| 15 |
+
|
| 16 |
+
|
| 17 |
+
## Model Details
|
| 18 |
+
|
| 19 |
+
### Model Description
|
| 20 |
+
|
| 21 |
+
<!-- Provide a longer summary of what this model is. -->
|
| 22 |
+
|
| 23 |
+
|
| 24 |
+
|
| 25 |
+
- **Developed by:** [More Information Needed]
|
| 26 |
+
- **Funded by [optional]:** [More Information Needed]
|
| 27 |
+
- **Shared by [optional]:** [More Information Needed]
|
| 28 |
+
- **Model type:** [More Information Needed]
|
| 29 |
+
- **Language(s) (NLP):** [More Information Needed]
|
| 30 |
+
- **License:** [More Information Needed]
|
| 31 |
+
- **Finetuned from model [optional]:** [More Information Needed]
|
| 32 |
+
|
| 33 |
+
### Model Sources [optional]
|
| 34 |
+
|
| 35 |
+
<!-- Provide the basic links for the model. -->
|
| 36 |
+
|
| 37 |
+
- **Repository:** [More Information Needed]
|
| 38 |
+
- **Paper [optional]:** [More Information Needed]
|
| 39 |
+
- **Demo [optional]:** [More Information Needed]
|
| 40 |
+
|
| 41 |
+
## Uses
|
| 42 |
+
|
| 43 |
+
<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
|
| 44 |
+
|
| 45 |
+
### Direct Use
|
| 46 |
+
|
| 47 |
+
<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
|
| 48 |
+
|
| 49 |
+
[More Information Needed]
|
| 50 |
+
|
| 51 |
+
### Downstream Use [optional]
|
| 52 |
+
|
| 53 |
+
<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
|
| 54 |
+
|
| 55 |
+
[More Information Needed]
|
| 56 |
+
|
| 57 |
+
### Out-of-Scope Use
|
| 58 |
+
|
| 59 |
+
<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
|
| 60 |
+
|
| 61 |
+
[More Information Needed]
|
| 62 |
+
|
| 63 |
+
## Bias, Risks, and Limitations
|
| 64 |
+
|
| 65 |
+
<!-- This section is meant to convey both technical and sociotechnical limitations. -->
|
| 66 |
+
|
| 67 |
+
[More Information Needed]
|
| 68 |
+
|
| 69 |
+
### Recommendations
|
| 70 |
+
|
| 71 |
+
<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
|
| 72 |
+
|
| 73 |
+
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
|
| 74 |
+
|
| 75 |
+
## How to Get Started with the Model
|
| 76 |
+
|
| 77 |
+
Use the code below to get started with the model.
|
| 78 |
+
|
| 79 |
+
[More Information Needed]
|
| 80 |
+
|
| 81 |
+
## Training Details
|
| 82 |
+
|
| 83 |
+
### Training Data
|
| 84 |
+
|
| 85 |
+
<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
|
| 86 |
+
|
| 87 |
+
[More Information Needed]
|
| 88 |
+
|
| 89 |
+
### Training Procedure
|
| 90 |
+
|
| 91 |
+
<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
|
| 92 |
+
|
| 93 |
+
#### Preprocessing [optional]
|
| 94 |
+
|
| 95 |
+
[More Information Needed]
|
| 96 |
+
|
| 97 |
+
|
| 98 |
+
#### Training Hyperparameters
|
| 99 |
+
|
| 100 |
+
- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
|
| 101 |
+
|
| 102 |
+
#### Speeds, Sizes, Times [optional]
|
| 103 |
+
|
| 104 |
+
<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
|
| 105 |
+
|
| 106 |
+
[More Information Needed]
|
| 107 |
+
|
| 108 |
+
## Evaluation
|
| 109 |
+
|
| 110 |
+
<!-- This section describes the evaluation protocols and provides the results. -->
|
| 111 |
+
|
| 112 |
+
### Testing Data, Factors & Metrics
|
| 113 |
+
|
| 114 |
+
#### Testing Data
|
| 115 |
+
|
| 116 |
+
<!-- This should link to a Dataset Card if possible. -->
|
| 117 |
+
|
| 118 |
+
[More Information Needed]
|
| 119 |
+
|
| 120 |
+
#### Factors
|
| 121 |
+
|
| 122 |
+
<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
|
| 123 |
+
|
| 124 |
+
[More Information Needed]
|
| 125 |
+
|
| 126 |
+
#### Metrics
|
| 127 |
+
|
| 128 |
+
<!-- These are the evaluation metrics being used, ideally with a description of why. -->
|
| 129 |
+
|
| 130 |
+
[More Information Needed]
|
| 131 |
+
|
| 132 |
+
### Results
|
| 133 |
+
|
| 134 |
+
[More Information Needed]
|
| 135 |
+
|
| 136 |
+
#### Summary
|
| 137 |
+
|
| 138 |
+
|
| 139 |
+
|
| 140 |
+
## Model Examination [optional]
|
| 141 |
+
|
| 142 |
+
<!-- Relevant interpretability work for the model goes here -->
|
| 143 |
+
|
| 144 |
+
[More Information Needed]
|
| 145 |
+
|
| 146 |
+
## Environmental Impact
|
| 147 |
+
|
| 148 |
+
<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
|
| 149 |
+
|
| 150 |
+
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
|
| 151 |
+
|
| 152 |
+
- **Hardware Type:** [More Information Needed]
|
| 153 |
+
- **Hours used:** [More Information Needed]
|
| 154 |
+
- **Cloud Provider:** [More Information Needed]
|
| 155 |
+
- **Compute Region:** [More Information Needed]
|
| 156 |
+
- **Carbon Emitted:** [More Information Needed]
|
| 157 |
+
|
| 158 |
+
## Technical Specifications [optional]
|
| 159 |
+
|
| 160 |
+
### Model Architecture and Objective
|
| 161 |
+
|
| 162 |
+
[More Information Needed]
|
| 163 |
+
|
| 164 |
+
### Compute Infrastructure
|
| 165 |
+
|
| 166 |
+
[More Information Needed]
|
| 167 |
+
|
| 168 |
+
#### Hardware
|
| 169 |
+
|
| 170 |
+
[More Information Needed]
|
| 171 |
+
|
| 172 |
+
#### Software
|
| 173 |
+
|
| 174 |
+
[More Information Needed]
|
| 175 |
+
|
| 176 |
+
## Citation [optional]
|
| 177 |
+
|
| 178 |
+
<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
|
| 179 |
+
|
| 180 |
+
**BibTeX:**
|
| 181 |
+
|
| 182 |
+
[More Information Needed]
|
| 183 |
+
|
| 184 |
+
**APA:**
|
| 185 |
+
|
| 186 |
+
[More Information Needed]
|
| 187 |
+
|
| 188 |
+
## Glossary [optional]
|
| 189 |
+
|
| 190 |
+
<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
|
| 191 |
+
|
| 192 |
+
[More Information Needed]
|
| 193 |
+
|
| 194 |
+
## More Information [optional]
|
| 195 |
+
|
| 196 |
+
[More Information Needed]
|
| 197 |
+
|
| 198 |
+
## Model Card Authors [optional]
|
| 199 |
+
|
| 200 |
+
[More Information Needed]
|
| 201 |
+
|
| 202 |
+
## Model Card Contact
|
| 203 |
+
|
| 204 |
+
[More Information Needed]
|
| 205 |
+
### Framework versions
|
| 206 |
+
|
| 207 |
+
- PEFT 0.21.2
|
e08-Nc/final/adapter_config.json
ADDED
|
@@ -0,0 +1,56 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"alora_invocation_tokens": null,
|
| 3 |
+
"alpha_pattern": {},
|
| 4 |
+
"arrow_config": null,
|
| 5 |
+
"auto_mapping": null,
|
| 6 |
+
"base_model_name_or_path": "/dev/shm/qwen36_27b",
|
| 7 |
+
"bias": "none",
|
| 8 |
+
"corda_config": null,
|
| 9 |
+
"ensure_weight_tying": false,
|
| 10 |
+
"eva_config": null,
|
| 11 |
+
"exclude_modules": null,
|
| 12 |
+
"fan_in_fan_out": false,
|
| 13 |
+
"inference_mode": true,
|
| 14 |
+
"init_lora_weights": true,
|
| 15 |
+
"kasa_config": null,
|
| 16 |
+
"layer_replication": null,
|
| 17 |
+
"layers_pattern": null,
|
| 18 |
+
"layers_to_transform": null,
|
| 19 |
+
"loftq_config": {},
|
| 20 |
+
"lora_alpha": 64,
|
| 21 |
+
"lora_bias": false,
|
| 22 |
+
"lora_dropout": 0.0,
|
| 23 |
+
"lora_ga_config": null,
|
| 24 |
+
"megatron_config": null,
|
| 25 |
+
"megatron_core": "megatron.core",
|
| 26 |
+
"modules_to_save": null,
|
| 27 |
+
"monteclora_config": null,
|
| 28 |
+
"peft_type": "LORA",
|
| 29 |
+
"peft_version": "0.21.2",
|
| 30 |
+
"qalora_group_size": 16,
|
| 31 |
+
"r": 32,
|
| 32 |
+
"rank_pattern": {},
|
| 33 |
+
"revision": null,
|
| 34 |
+
"target_modules": [
|
| 35 |
+
"q_proj",
|
| 36 |
+
"down_proj",
|
| 37 |
+
"k_proj",
|
| 38 |
+
"o_proj",
|
| 39 |
+
"in_proj_b",
|
| 40 |
+
"up_proj",
|
| 41 |
+
"in_proj_a",
|
| 42 |
+
"in_proj_z",
|
| 43 |
+
"out_proj",
|
| 44 |
+
"in_proj_qkv",
|
| 45 |
+
"v_proj",
|
| 46 |
+
"gate_proj"
|
| 47 |
+
],
|
| 48 |
+
"target_parameters": null,
|
| 49 |
+
"task_type": "CAUSAL_LM",
|
| 50 |
+
"trainable_token_indices": null,
|
| 51 |
+
"use_bdlora": null,
|
| 52 |
+
"use_dora": false,
|
| 53 |
+
"use_qalora": false,
|
| 54 |
+
"use_rslora": false,
|
| 55 |
+
"velora_config": null
|
| 56 |
+
}
|
e08-Nc/final/adapter_model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a0a4e02f60aae139a210eb74f014211f36f4a1a2f470aee260fe6b85c1daeda9
|
| 3 |
+
size 933959152
|
e08-Nc/meta/golden_render.jsonl
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{"idx": 334, "n_tok": 790, "n_sup": 766, "masked_tail": "<|im_start|>user\nWrite a short prose story about Selene and Abdul.<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n\n", "trained_head": "Selene wiped grease from her hands and leaned over the exposed chassis of the ultralight frame. The metal was cold under her fingertips as she tightened a final screw.\n\n\"The brakes on this ultralight ", "trained_tail": "to go. Sounds like you\u2019ve got the hang of it now.\"<|im_end|>"}
|
| 2 |
+
{"idx": 4535, "n_tok": 1495, "n_sup": 1469, "masked_tail": "<|im_start|>user\nWrite a short prose story about Dina and Caspian.<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n\n", "trained_head": "Dina stood before the humming machinery of Enclosure 4, her brow furrowed as she read the discrepancy on the display. She turned to her colleague, her voice tight with professional urgency. \"Caspian, ", "trained_tail": "t we\u2019ll see how your sensor does. See you in ten.\"<|im_end|>"}
|
| 3 |
+
{"idx": 7771, "n_tok": 1491, "n_sup": 1466, "masked_tail": "<|im_start|>user\nWrite a short prose story about Jens and Ketevan.<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n\n", "trained_head": "Jens sat across from Ketevan in the small rehearsal room, his sheet music spread out before him. \"Hi Ketevan, thanks for meeting me. I\u2019m struggling with the Georgian lyrics for the concert. I need cle", "trained_tail": "ens. It was a pleasure,\" Ketevan called after him.<|im_end|>"}
|
e08-Nc/meta/train_log.jsonl
ADDED
|
@@ -0,0 +1,246 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{"step": 1, "loss": 1.2459054364929751, "story_loss": 1.2459054364929751, "grad_norm": 0.4314831495285034, "lr": 5e-05, "elapsed": 173.8, "tokens": 31994, "story_tokens": 31994, "peak_gb": 72.1}
|
| 2 |
+
{"step": 2, "loss": 1.2198673436180691, "story_loss": 1.2198673436180691, "grad_norm": 0.3862919509410858, "lr": 0.0001, "elapsed": 198.9, "tokens": 35530, "story_tokens": 35530, "peak_gb": 74.1}
|
| 3 |
+
{"step": 3, "loss": 1.1071088369490358, "story_loss": 1.1071088369490358, "grad_norm": 0.31111088395118713, "lr": 0.00015, "elapsed": 242.8, "tokens": 36300, "story_tokens": 36300, "peak_gb": 74.9}
|
| 4 |
+
{"step": 4, "loss": 1.1255799988349213, "story_loss": 1.1255799988349213, "grad_norm": 0.43663373589515686, "lr": 0.0002, "elapsed": 265.1, "tokens": 30175, "story_tokens": 30175, "peak_gb": 74.9}
|
| 5 |
+
{"step": 5, "loss": 1.0549288043385345, "story_loss": 1.0549288043385345, "grad_norm": 0.2839439809322357, "lr": 0.00025, "elapsed": 287.6, "tokens": 31074, "story_tokens": 31074, "peak_gb": 74.9}
|
| 6 |
+
{"step": 6, "loss": 1.026433965951819, "story_loss": 1.026433965951819, "grad_norm": 0.2622741758823395, "lr": 0.0003, "elapsed": 312.3, "tokens": 35006, "story_tokens": 35006, "peak_gb": 74.9}
|
| 7 |
+
{"step": 7, "loss": 0.9976688467396381, "story_loss": 0.9976688467396381, "grad_norm": 0.19518834352493286, "lr": 0.00035, "elapsed": 335.2, "tokens": 32921, "story_tokens": 32921, "peak_gb": 74.9}
|
| 8 |
+
{"step": 8, "loss": 0.9521781424397243, "story_loss": 0.9521781424397243, "grad_norm": 0.17980924248695374, "lr": 0.0004, "elapsed": 358.3, "tokens": 31470, "story_tokens": 31470, "peak_gb": 74.9}
|
| 9 |
+
{"step": 9, "loss": 0.9082017878631088, "story_loss": 0.9082017878631088, "grad_norm": 0.1822422742843628, "lr": 0.00045000000000000004, "elapsed": 383.3, "tokens": 35604, "story_tokens": 35604, "peak_gb": 74.9}
|
| 10 |
+
{"step": 10, "loss": 0.9230904177533711, "story_loss": 0.9230904177533711, "grad_norm": 0.23073534667491913, "lr": 0.0005, "elapsed": 406.3, "tokens": 32519, "story_tokens": 32519, "peak_gb": 74.9}
|
| 11 |
+
{"step": 11, "loss": 0.8867654112128989, "story_loss": 0.8867654112128989, "grad_norm": 0.1905829757452011, "lr": 0.0004999778497139454, "elapsed": 431.5, "tokens": 34972, "story_tokens": 34972, "peak_gb": 74.9}
|
| 12 |
+
{"step": 12, "loss": 0.9049414509247903, "story_loss": 0.9049414509247903, "grad_norm": 0.1806613653898239, "lr": 0.0004999114027808632, "elapsed": 454.1, "tokens": 30931, "story_tokens": 30931, "peak_gb": 74.9}
|
| 13 |
+
{"step": 13, "loss": 0.9156740445577269, "story_loss": 0.9156740445577269, "grad_norm": 0.17583297193050385, "lr": 0.0004998006709753017, "elapsed": 477.4, "tokens": 32487, "story_tokens": 32487, "peak_gb": 74.9}
|
| 14 |
+
{"step": 14, "loss": 0.9271330451354468, "story_loss": 0.9271330451354468, "grad_norm": 0.16018447279930115, "lr": 0.0004996456739191905, "elapsed": 500.4, "tokens": 32785, "story_tokens": 32785, "peak_gb": 74.9}
|
| 15 |
+
{"step": 15, "loss": 0.8834218361938806, "story_loss": 0.8834218361938806, "grad_norm": 0.1666870266199112, "lr": 0.0004994464390783625, "elapsed": 522.7, "tokens": 30777, "story_tokens": 30777, "peak_gb": 74.9}
|
| 16 |
+
{"step": 16, "loss": 0.8534517050397229, "story_loss": 0.8534517050397229, "grad_norm": 0.1500123143196106, "lr": 0.0004992030017576876, "elapsed": 546.5, "tokens": 34127, "story_tokens": 34127, "peak_gb": 74.9}
|
| 17 |
+
{"step": 17, "loss": 0.8924374939025118, "story_loss": 0.8924374939025118, "grad_norm": 0.157627671957016, "lr": 0.0004989154050948158, "elapsed": 586.0, "tokens": 31391, "story_tokens": 31391, "peak_gb": 74.9}
|
| 18 |
+
{"step": 18, "loss": 0.8808594645147811, "story_loss": 0.8808594645147811, "grad_norm": 0.16125229001045227, "lr": 0.0004985837000525343, "elapsed": 609.4, "tokens": 30683, "story_tokens": 30683, "peak_gb": 75.3}
|
| 19 |
+
{"step": 19, "loss": 0.9067726076873921, "story_loss": 0.9067726076873921, "grad_norm": 0.15922409296035767, "lr": 0.0004982079454097354, "elapsed": 631.8, "tokens": 30711, "story_tokens": 30711, "peak_gb": 75.3}
|
| 20 |
+
{"step": 20, "loss": 0.8936638053310594, "story_loss": 0.8936638053310594, "grad_norm": 0.1665307730436325, "lr": 0.0004977882077510018, "elapsed": 654.5, "tokens": 32194, "story_tokens": 32194, "peak_gb": 75.3}
|
| 21 |
+
{"step": 21, "loss": 0.8554054634693471, "story_loss": 0.8554054634693471, "grad_norm": 0.14401096105575562, "lr": 0.0004973245614548072, "elapsed": 679.4, "tokens": 35433, "story_tokens": 35433, "peak_gb": 75.3}
|
| 22 |
+
{"step": 22, "loss": 0.8427083792201155, "story_loss": 0.8427083792201155, "grad_norm": 0.15035314857959747, "lr": 0.0004968170886803362, "elapsed": 703.7, "tokens": 35470, "story_tokens": 35470, "peak_gb": 75.3}
|
| 23 |
+
{"step": 23, "loss": 0.8605454736242477, "story_loss": 0.8605454736242477, "grad_norm": 0.15748392045497894, "lr": 0.0004962658793529257, "elapsed": 726.5, "tokens": 33066, "story_tokens": 33066, "peak_gb": 75.3}
|
| 24 |
+
{"step": 24, "loss": 0.8817875814830437, "story_loss": 0.8817875814830437, "grad_norm": 0.18786607682704926, "lr": 0.0004956710311481302, "elapsed": 750.5, "tokens": 31338, "story_tokens": 31338, "peak_gb": 75.3}
|
| 25 |
+
{"step": 25, "loss": 0.8785581611766096, "story_loss": 0.8785581611766096, "grad_norm": 0.16282476484775543, "lr": 0.0004950326494744127, "elapsed": 772.9, "tokens": 31484, "story_tokens": 31484, "peak_gb": 75.3}
|
| 26 |
+
{"step": 26, "loss": 0.8590605624290347, "story_loss": 0.8590605624290347, "grad_norm": 0.15323121845722198, "lr": 0.0004943508474544667, "elapsed": 795.5, "tokens": 32146, "story_tokens": 32146, "peak_gb": 75.3}
|
| 27 |
+
{"step": 27, "loss": 0.8610151548413393, "story_loss": 0.8610151548413393, "grad_norm": 0.15171723067760468, "lr": 0.0004936257459051702, "elapsed": 818.0, "tokens": 32278, "story_tokens": 32278, "peak_gb": 75.3}
|
| 28 |
+
{"step": 28, "loss": 0.8532832983790305, "story_loss": 0.8532832983790305, "grad_norm": 0.1461639553308487, "lr": 0.0004928574733161775, "elapsed": 840.4, "tokens": 31587, "story_tokens": 31587, "peak_gb": 75.3}
|
| 29 |
+
{"step": 29, "loss": 0.8572696209372258, "story_loss": 0.8572696209372258, "grad_norm": 0.1450178623199463, "lr": 0.0004920461658271492, "elapsed": 864.1, "tokens": 34190, "story_tokens": 34190, "peak_gb": 75.3}
|
| 30 |
+
{"step": 30, "loss": 0.8170471453449683, "story_loss": 0.8170471453449683, "grad_norm": 0.14430557191371918, "lr": 0.000491191967203629, "elapsed": 888.2, "tokens": 34938, "story_tokens": 34938, "peak_gb": 75.3}
|
| 31 |
+
{"step": 31, "loss": 0.8505964095682312, "story_loss": 0.8505964095682312, "grad_norm": 0.14238345623016357, "lr": 0.0004902950288115679, "elapsed": 911.9, "tokens": 33315, "story_tokens": 33315, "peak_gb": 75.3}
|
| 32 |
+
{"step": 32, "loss": 0.8765385061998228, "story_loss": 0.8765385061998228, "grad_norm": 0.14780057966709137, "lr": 0.0004893555095905013, "elapsed": 936.0, "tokens": 34565, "story_tokens": 34565, "peak_gb": 75.3}
|
| 33 |
+
{"step": 33, "loss": 0.8329970671297117, "story_loss": 0.8329970671297117, "grad_norm": 0.16202890872955322, "lr": 0.0004883735760253855, "elapsed": 961.1, "tokens": 35285, "story_tokens": 35285, "peak_gb": 75.3}
|
| 34 |
+
{"step": 34, "loss": 0.8576398669890288, "story_loss": 0.8576398669890288, "grad_norm": 0.15113738179206848, "lr": 0.0004873494021170953, "elapsed": 983.3, "tokens": 30979, "story_tokens": 30979, "peak_gb": 75.3}
|
| 35 |
+
{"step": 35, "loss": 0.869448808005636, "story_loss": 0.869448808005636, "grad_norm": 0.169200599193573, "lr": 0.0004862831693515909, "elapsed": 1006.6, "tokens": 31317, "story_tokens": 31317, "peak_gb": 75.3}
|
| 36 |
+
{"step": 36, "loss": 0.8388230333518611, "story_loss": 0.8388230333518611, "grad_norm": 0.1662723869085312, "lr": 0.00048517506666775835, "elapsed": 1029.7, "tokens": 31962, "story_tokens": 31962, "peak_gb": 75.3}
|
| 37 |
+
{"step": 37, "loss": 0.825441064422641, "story_loss": 0.825441064422641, "grad_norm": 0.152525395154953, "lr": 0.00048402529042392906, "elapsed": 1053.6, "tokens": 33637, "story_tokens": 33637, "peak_gb": 75.3}
|
| 38 |
+
{"step": 38, "loss": 0.8333948546638701, "story_loss": 0.8333948546638701, "grad_norm": 0.17343184351921082, "lr": 0.0004828340443630846, "elapsed": 1076.2, "tokens": 32599, "story_tokens": 32599, "peak_gb": 75.3}
|
| 39 |
+
{"step": 39, "loss": 0.8262715324143971, "story_loss": 0.8262715324143971, "grad_norm": 0.1531706005334854, "lr": 0.00048160153957675314, "elapsed": 1100.3, "tokens": 34111, "story_tokens": 34111, "peak_gb": 75.3}
|
| 40 |
+
{"step": 40, "loss": 0.844073306150542, "story_loss": 0.844073306150542, "grad_norm": 0.16540683805942535, "lr": 0.0004803279944676032, "elapsed": 1122.2, "tokens": 30352, "story_tokens": 30352, "peak_gb": 75.3}
|
| 41 |
+
{"step": 41, "loss": 0.8440567282101157, "story_loss": 0.8440567282101157, "grad_norm": 0.16075150668621063, "lr": 0.0004790136347107427, "elapsed": 1145.5, "tokens": 33851, "story_tokens": 33851, "peak_gb": 75.3}
|
| 42 |
+
{"step": 42, "loss": 0.8403269094698729, "story_loss": 0.8403269094698729, "grad_norm": 0.15686336159706116, "lr": 0.00047765869321372834, "elapsed": 1168.6, "tokens": 33367, "story_tokens": 33367, "peak_gb": 75.3}
|
| 43 |
+
{"step": 43, "loss": 0.8560388736791343, "story_loss": 0.8560388736791343, "grad_norm": 0.16111308336257935, "lr": 0.00047626341007529397, "elapsed": 1192.0, "tokens": 33276, "story_tokens": 33276, "peak_gb": 75.3}
|
| 44 |
+
{"step": 44, "loss": 0.8542610675043282, "story_loss": 0.8542610675043282, "grad_norm": 0.1642204076051712, "lr": 0.0004748280325428048, "elapsed": 1214.8, "tokens": 31913, "story_tokens": 31913, "peak_gb": 75.3}
|
| 45 |
+
{"step": 45, "loss": 0.8363254381262738, "story_loss": 0.8363254381262738, "grad_norm": 0.1502709686756134, "lr": 0.0004733528149684444, "elapsed": 1238.0, "tokens": 32384, "story_tokens": 32384, "peak_gb": 75.3}
|
| 46 |
+
{"step": 46, "loss": 0.8555894578203576, "story_loss": 0.8555894578203576, "grad_norm": 0.15465646982192993, "lr": 0.0004718380187641429, "elapsed": 1260.6, "tokens": 32612, "story_tokens": 32612, "peak_gb": 75.3}
|
| 47 |
+
{"step": 47, "loss": 0.8423190541471152, "story_loss": 0.8423190541471152, "grad_norm": 0.15549930930137634, "lr": 0.0004702839123552541, "elapsed": 1283.7, "tokens": 33512, "story_tokens": 33512, "peak_gb": 75.3}
|
| 48 |
+
{"step": 48, "loss": 0.8319239414633206, "story_loss": 0.8319239414633206, "grad_norm": 0.1496901512145996, "lr": 0.0004686907711329903, "elapsed": 1306.5, "tokens": 31622, "story_tokens": 31622, "peak_gb": 75.3}
|
| 49 |
+
{"step": 49, "loss": 0.8520943629170388, "story_loss": 0.8520943629170388, "grad_norm": 0.17192073166370392, "lr": 0.0004670588774056218, "elapsed": 1328.6, "tokens": 31071, "story_tokens": 31071, "peak_gb": 75.3}
|
| 50 |
+
{"step": 50, "loss": 0.8262458530108224, "story_loss": 0.8262458530108224, "grad_norm": 0.14792464673519135, "lr": 0.0004653885203484515, "elapsed": 1352.7, "tokens": 35170, "story_tokens": 35170, "peak_gb": 75.3}
|
| 51 |
+
{"step": 51, "loss": 0.8476749332568783, "story_loss": 0.8476749332568783, "grad_norm": 0.15968023240566254, "lr": 0.0004636799959525726, "elapsed": 1374.9, "tokens": 29395, "story_tokens": 29395, "peak_gb": 75.3}
|
| 52 |
+
{"step": 52, "loss": 0.8577089814649173, "story_loss": 0.8577089814649173, "grad_norm": 0.15932931005954742, "lr": 0.00046193360697241773, "elapsed": 1397.9, "tokens": 31880, "story_tokens": 31880, "peak_gb": 75.3}
|
| 53 |
+
{"step": 53, "loss": 0.8385722767842894, "story_loss": 0.8385722767842894, "grad_norm": 0.15618562698364258, "lr": 0.0004601496628721108, "elapsed": 1419.6, "tokens": 31329, "story_tokens": 31329, "peak_gb": 75.3}
|
| 54 |
+
{"step": 54, "loss": 0.8305302046287447, "story_loss": 0.8305302046287447, "grad_norm": 0.15059523284435272, "lr": 0.00045832847977062875, "elapsed": 1443.8, "tokens": 35183, "story_tokens": 35183, "peak_gb": 75.3}
|
| 55 |
+
{"step": 55, "loss": 0.8634529668625929, "story_loss": 0.8634529668625929, "grad_norm": 0.1571495234966278, "lr": 0.0004564703803857849, "elapsed": 1466.2, "tokens": 32038, "story_tokens": 32038, "peak_gb": 75.3}
|
| 56 |
+
{"step": 56, "loss": 0.8563261283696024, "story_loss": 0.8563261283696024, "grad_norm": 0.15641771256923676, "lr": 0.00045457569397704226, "elapsed": 1488.5, "tokens": 32144, "story_tokens": 32144, "peak_gb": 75.3}
|
| 57 |
+
{"step": 57, "loss": 0.825990942481833, "story_loss": 0.825990942481833, "grad_norm": 0.15733702480793, "lr": 0.0004526447562871685, "elapsed": 1510.6, "tokens": 31066, "story_tokens": 31066, "peak_gb": 75.3}
|
| 58 |
+
{"step": 58, "loss": 0.8359455564246351, "story_loss": 0.8359455564246351, "grad_norm": 0.15097084641456604, "lr": 0.0004506779094827409, "elapsed": 1534.4, "tokens": 33910, "story_tokens": 33910, "peak_gb": 75.3}
|
| 59 |
+
{"step": 59, "loss": 0.8379332135873012, "story_loss": 0.8379332135873012, "grad_norm": 0.15597373247146606, "lr": 0.0004486755020935144, "elapsed": 1556.7, "tokens": 32128, "story_tokens": 32128, "peak_gb": 75.3}
|
| 60 |
+
{"step": 60, "loss": 0.8428746518695273, "story_loss": 0.8428746518695273, "grad_norm": 0.15208080410957336, "lr": 0.00044663788895066064, "elapsed": 1579.4, "tokens": 33690, "story_tokens": 33690, "peak_gb": 75.3}
|
| 61 |
+
{"step": 61, "loss": 0.8320670731935745, "story_loss": 0.8320670731935745, "grad_norm": 0.15223033726215363, "lr": 0.0004445654311238915, "elapsed": 1602.6, "tokens": 33367, "story_tokens": 33367, "peak_gb": 75.3}
|
| 62 |
+
{"step": 62, "loss": 0.8277551398088713, "story_loss": 0.8277551398088713, "grad_norm": 0.161062553524971, "lr": 0.00044245849585747656, "elapsed": 1625.9, "tokens": 32949, "story_tokens": 32949, "peak_gb": 75.3}
|
| 63 |
+
{"step": 63, "loss": 0.8412959490448799, "story_loss": 0.8412959490448799, "grad_norm": 0.15252742171287537, "lr": 0.00044031745650516664, "elapsed": 1649.4, "tokens": 34194, "story_tokens": 34194, "peak_gb": 75.3}
|
| 64 |
+
{"step": 64, "loss": 0.8638413783024497, "story_loss": 0.8638413783024497, "grad_norm": 0.17437246441841125, "lr": 0.0004381426924640346, "elapsed": 1671.8, "tokens": 30484, "story_tokens": 30484, "peak_gb": 75.3}
|
| 65 |
+
{"step": 65, "loss": 0.8634195473489907, "story_loss": 0.8634195473489907, "grad_norm": 0.1650637835264206, "lr": 0.00043593458910724537, "elapsed": 1694.3, "tokens": 30888, "story_tokens": 30888, "peak_gb": 75.3}
|
| 66 |
+
{"step": 66, "loss": 0.8200508999076539, "story_loss": 0.8200508999076539, "grad_norm": 0.14664511382579803, "lr": 0.0004336935377157668, "elapsed": 1718.7, "tokens": 35126, "story_tokens": 35126, "peak_gb": 75.3}
|
| 67 |
+
{"step": 67, "loss": 0.8429464476793955, "story_loss": 0.8429464476793955, "grad_norm": 0.1497684270143509, "lr": 0.00043141993540903403, "elapsed": 1741.8, "tokens": 33585, "story_tokens": 33585, "peak_gb": 75.3}
|
| 68 |
+
{"step": 68, "loss": 0.8455229745530082, "story_loss": 0.8455229745530082, "grad_norm": 0.1550675481557846, "lr": 0.0004291141850745788, "elapsed": 1764.1, "tokens": 31705, "story_tokens": 31705, "peak_gb": 75.3}
|
| 69 |
+
{"step": 69, "loss": 0.8293636872092001, "story_loss": 0.8293636872092001, "grad_norm": 0.151563361287117, "lr": 0.00042677669529663686, "elapsed": 1786.5, "tokens": 31970, "story_tokens": 31970, "peak_gb": 75.3}
|
| 70 |
+
{"step": 70, "loss": 0.8334787077190765, "story_loss": 0.8334787077190765, "grad_norm": 0.1585901975631714, "lr": 0.0004244078802837462, "elapsed": 1809.2, "tokens": 32831, "story_tokens": 32831, "peak_gb": 75.3}
|
| 71 |
+
{"step": 71, "loss": 0.8620012146553129, "story_loss": 0.8620012146553129, "grad_norm": 0.17240722477436066, "lr": 0.0004220081597953479, "elapsed": 1833.3, "tokens": 34515, "story_tokens": 34515, "peak_gb": 75.3}
|
| 72 |
+
{"step": 72, "loss": 0.8486029344163837, "story_loss": 0.8486029344163837, "grad_norm": 0.15794585645198822, "lr": 0.000419577959067404, "elapsed": 1856.5, "tokens": 32546, "story_tokens": 32546, "peak_gb": 75.3}
|
| 73 |
+
{"step": 73, "loss": 0.8284024878420596, "story_loss": 0.8284024878420596, "grad_norm": 0.15643438696861267, "lr": 0.0004171177087370451, "elapsed": 1881.0, "tokens": 35310, "story_tokens": 35310, "peak_gb": 75.3}
|
| 74 |
+
{"step": 74, "loss": 0.8190097905883154, "story_loss": 0.8190097905883154, "grad_norm": 0.1592734158039093, "lr": 0.0004146278447662597, "elapsed": 1903.6, "tokens": 32626, "story_tokens": 32626, "peak_gb": 75.3}
|
| 75 |
+
{"step": 75, "loss": 0.852308667016166, "story_loss": 0.852308667016166, "grad_norm": 0.1720048040151596, "lr": 0.00041210880836464134, "elapsed": 1925.6, "tokens": 31563, "story_tokens": 31563, "peak_gb": 75.3}
|
| 76 |
+
{"step": 76, "loss": 0.8477594443549398, "story_loss": 0.8477594443549398, "grad_norm": 0.17302818596363068, "lr": 0.00040956104591120503, "elapsed": 1947.8, "tokens": 30938, "story_tokens": 30938, "peak_gb": 75.3}
|
| 77 |
+
{"step": 77, "loss": 0.8696117050146374, "story_loss": 0.8696117050146374, "grad_norm": 0.16640624403953552, "lr": 0.000406985008875288, "elapsed": 1969.6, "tokens": 30487, "story_tokens": 30487, "peak_gb": 75.3}
|
| 78 |
+
{"step": 78, "loss": 0.8573416948828362, "story_loss": 0.8573416948828362, "grad_norm": 0.15910813212394714, "lr": 0.000404381153736548, "elapsed": 1991.3, "tokens": 31329, "story_tokens": 31329, "peak_gb": 75.3}
|
| 79 |
+
{"step": 79, "loss": 0.8445997707281739, "story_loss": 0.8445997707281739, "grad_norm": 0.15996187925338745, "lr": 0.00040174994190407445, "elapsed": 2013.8, "tokens": 30534, "story_tokens": 30534, "peak_gb": 75.3}
|
| 80 |
+
{"step": 80, "loss": 0.8245588861053126, "story_loss": 0.8245588861053126, "grad_norm": 0.1535836011171341, "lr": 0.00039909183963462535, "elapsed": 2036.2, "tokens": 32635, "story_tokens": 32635, "peak_gb": 75.3}
|
| 81 |
+
{"step": 81, "loss": 0.8368537541828819, "story_loss": 0.8368537541828819, "grad_norm": 0.1570577472448349, "lr": 0.00039640731795000583, "elapsed": 2058.3, "tokens": 31574, "story_tokens": 31574, "peak_gb": 75.3}
|
| 82 |
+
{"step": 82, "loss": 0.851953058997379, "story_loss": 0.851953058997379, "grad_norm": 0.16823254525661469, "lr": 0.0003936968525536018, "elapsed": 2080.3, "tokens": 31811, "story_tokens": 31811, "peak_gb": 75.3}
|
| 83 |
+
{"step": 83, "loss": 0.8086542361543206, "story_loss": 0.8086542361543206, "grad_norm": 0.15658898651599884, "lr": 0.000390960923746084, "elapsed": 2104.2, "tokens": 34353, "story_tokens": 34353, "peak_gb": 75.3}
|
| 84 |
+
{"step": 84, "loss": 0.8518965409498059, "story_loss": 0.8518965409498059, "grad_norm": 0.16363529860973358, "lr": 0.0003882000163402983, "elapsed": 2126.6, "tokens": 31692, "story_tokens": 31692, "peak_gb": 75.3}
|
| 85 |
+
{"step": 85, "loss": 0.8280902416110371, "story_loss": 0.8280902416110371, "grad_norm": 0.15967796742916107, "lr": 0.00038541461957535526, "elapsed": 2149.1, "tokens": 32282, "story_tokens": 32282, "peak_gb": 75.3}
|
| 86 |
+
{"step": 86, "loss": 0.8380647069889814, "story_loss": 0.8380647069889814, "grad_norm": 0.15645931661128998, "lr": 0.0003826052270299356, "elapsed": 2171.3, "tokens": 30959, "story_tokens": 30959, "peak_gb": 75.3}
|
| 87 |
+
{"step": 87, "loss": 0.8221341686181074, "story_loss": 0.8221341686181074, "grad_norm": 0.15080031752586365, "lr": 0.0003797723365348277, "elapsed": 2194.1, "tokens": 33242, "story_tokens": 33242, "peak_gb": 75.3}
|
| 88 |
+
{"step": 88, "loss": 0.868239146688505, "story_loss": 0.868239146688505, "grad_norm": 0.15448834002017975, "lr": 0.00037691645008471, "elapsed": 2216.0, "tokens": 32340, "story_tokens": 32340, "peak_gb": 75.3}
|
| 89 |
+
{"step": 89, "loss": 0.8338555502442394, "story_loss": 0.8338555502442394, "grad_norm": 0.15862321853637695, "lr": 0.0003740380737491971, "elapsed": 2238.7, "tokens": 32115, "story_tokens": 32115, "peak_gb": 75.3}
|
| 90 |
+
{"step": 90, "loss": 0.8112607030193781, "story_loss": 0.8112607030193781, "grad_norm": 0.14962783455848694, "lr": 0.00037113771758316255, "elapsed": 2261.6, "tokens": 33285, "story_tokens": 33285, "peak_gb": 75.3}
|
| 91 |
+
{"step": 91, "loss": 0.8503162339786408, "story_loss": 0.8503162339786408, "grad_norm": 0.16036362946033478, "lr": 0.00036821589553635636, "elapsed": 2283.7, "tokens": 31813, "story_tokens": 31813, "peak_gb": 75.3}
|
| 92 |
+
{"step": 92, "loss": 0.8378138970401867, "story_loss": 0.8378138970401867, "grad_norm": 0.15184146165847778, "lr": 0.00036527312536233146, "elapsed": 2308.1, "tokens": 35755, "story_tokens": 35755, "peak_gb": 75.3}
|
| 93 |
+
{"step": 93, "loss": 0.8281137770848399, "story_loss": 0.8281137770848399, "grad_norm": 0.15344074368476868, "lr": 0.0003623099285266972, "elapsed": 2330.3, "tokens": 31804, "story_tokens": 31804, "peak_gb": 75.3}
|
| 94 |
+
{"step": 94, "loss": 0.815315764304571, "story_loss": 0.815315764304571, "grad_norm": 0.154253289103508, "lr": 0.0003593268301147139, "elapsed": 2354.3, "tokens": 33997, "story_tokens": 33997, "peak_gb": 75.3}
|
| 95 |
+
{"step": 95, "loss": 0.8061545180932467, "story_loss": 0.8061545180932467, "grad_norm": 0.15004394948482513, "lr": 0.0003563243587382468, "elapsed": 2378.0, "tokens": 34583, "story_tokens": 34583, "peak_gb": 75.3}
|
| 96 |
+
{"step": 96, "loss": 0.8349124817662869, "story_loss": 0.8349124817662869, "grad_norm": 0.15697138011455536, "lr": 0.00035330304644209454, "elapsed": 2401.1, "tokens": 32649, "story_tokens": 32649, "peak_gb": 75.3}
|
| 97 |
+
{"step": 97, "loss": 0.8305270802148849, "story_loss": 0.8305270802148849, "grad_norm": 0.1626608967781067, "lr": 0.0003502634286097104, "elapsed": 2423.0, "tokens": 32276, "story_tokens": 32276, "peak_gb": 75.3}
|
| 98 |
+
{"step": 98, "loss": 0.8140718494467872, "story_loss": 0.8140718494467872, "grad_norm": 0.15714828670024872, "lr": 0.0003472060438683302, "elapsed": 2445.1, "tokens": 30176, "story_tokens": 30176, "peak_gb": 75.3}
|
| 99 |
+
{"step": 99, "loss": 0.8216713298911362, "story_loss": 0.8216713298911362, "grad_norm": 0.1528034210205078, "lr": 0.00034413143399352657, "elapsed": 2468.4, "tokens": 33132, "story_tokens": 33132, "peak_gb": 75.3}
|
| 100 |
+
{"step": 100, "loss": 0.8291707092283294, "story_loss": 0.8291707092283294, "grad_norm": 0.15895435214042664, "lr": 0.00034104014381320557, "elapsed": 2490.7, "tokens": 32769, "story_tokens": 32769, "peak_gb": 75.3}
|
| 101 |
+
{"step": 101, "loss": 0.8354451223225492, "story_loss": 0.8354451223225492, "grad_norm": 0.15563397109508514, "lr": 0.0003379327211110612, "elapsed": 2512.7, "tokens": 30276, "story_tokens": 30276, "peak_gb": 75.3}
|
| 102 |
+
{"step": 102, "loss": 0.8557112819833066, "story_loss": 0.8557112819833066, "grad_norm": 0.1575516015291214, "lr": 0.00033480971652950753, "elapsed": 2536.2, "tokens": 33389, "story_tokens": 33389, "peak_gb": 75.3}
|
| 103 |
+
{"step": 103, "loss": 0.8356353683196217, "story_loss": 0.8356353683196217, "grad_norm": 0.15461057424545288, "lr": 0.00033167168347210326, "elapsed": 2559.7, "tokens": 32682, "story_tokens": 32682, "peak_gb": 75.3}
|
| 104 |
+
{"step": 104, "loss": 0.8481633242549403, "story_loss": 0.8481633242549403, "grad_norm": 0.1651352494955063, "lr": 0.00032851917800548724, "elapsed": 2581.2, "tokens": 30438, "story_tokens": 30438, "peak_gb": 75.3}
|
| 105 |
+
{"step": 105, "loss": 0.8485507579046779, "story_loss": 0.8485507579046779, "grad_norm": 0.15269309282302856, "lr": 0.00032535275876084246, "elapsed": 2603.7, "tokens": 32461, "story_tokens": 32461, "peak_gb": 75.3}
|
| 106 |
+
{"step": 106, "loss": 0.8489869254301686, "story_loss": 0.8489869254301686, "grad_norm": 0.16012659668922424, "lr": 0.00032217298683490526, "elapsed": 2624.9, "tokens": 30119, "story_tokens": 30119, "peak_gb": 75.3}
|
| 107 |
+
{"step": 107, "loss": 0.8185698046319689, "story_loss": 0.8185698046319689, "grad_norm": 0.1592843234539032, "lr": 0.00031898042569053763, "elapsed": 2647.0, "tokens": 32358, "story_tokens": 32358, "peak_gb": 75.3}
|
| 108 |
+
{"step": 108, "loss": 0.8400696491859914, "story_loss": 0.8400696491859914, "grad_norm": 0.16202236711978912, "lr": 0.0003157756410568803, "elapsed": 2669.3, "tokens": 32148, "story_tokens": 32148, "peak_gb": 75.3}
|
| 109 |
+
{"step": 109, "loss": 0.8072051270467552, "story_loss": 0.8072051270467552, "grad_norm": 0.1513069123029709, "lr": 0.0003125592008291044, "elapsed": 2693.4, "tokens": 33376, "story_tokens": 33376, "peak_gb": 75.3}
|
| 110 |
+
{"step": 110, "loss": 0.8550524311616483, "story_loss": 0.8550524311616483, "grad_norm": 0.16262932121753693, "lr": 0.00030933167496777875, "elapsed": 2715.3, "tokens": 30542, "story_tokens": 30542, "peak_gb": 75.3}
|
| 111 |
+
{"step": 111, "loss": 0.8087665695208152, "story_loss": 0.8087665695208152, "grad_norm": 0.1608530580997467, "lr": 0.000306093635397872, "elapsed": 2739.6, "tokens": 34440, "story_tokens": 34440, "peak_gb": 75.3}
|
| 112 |
+
{"step": 112, "loss": 0.8434188725003247, "story_loss": 0.8434188725003247, "grad_norm": 0.1677221953868866, "lr": 0.00030284565590740607, "elapsed": 2761.6, "tokens": 30318, "story_tokens": 30318, "peak_gb": 75.3}
|
| 113 |
+
{"step": 113, "loss": 0.8380435742072785, "story_loss": 0.8380435742072785, "grad_norm": 0.1541326642036438, "lr": 0.00029958831204577945, "elapsed": 2783.9, "tokens": 32349, "story_tokens": 32349, "peak_gb": 75.3}
|
| 114 |
+
{"step": 114, "loss": 0.8540095046890578, "story_loss": 0.8540095046890578, "grad_norm": 0.16465707123279572, "lr": 0.0002963221810217786, "elapsed": 2805.7, "tokens": 31093, "story_tokens": 31093, "peak_gb": 75.3}
|
| 115 |
+
{"step": 115, "loss": 0.804045649185029, "story_loss": 0.804045649185029, "grad_norm": 0.14407837390899658, "lr": 0.00029304784160129533, "elapsed": 2829.7, "tokens": 34196, "story_tokens": 34196, "peak_gb": 75.3}
|
| 116 |
+
{"step": 116, "loss": 0.8471902059335195, "story_loss": 0.8471902059335195, "grad_norm": 0.15335124731063843, "lr": 0.000289765874004768, "elapsed": 2851.4, "tokens": 31795, "story_tokens": 31795, "peak_gb": 75.3}
|
| 117 |
+
{"step": 117, "loss": 0.813437671622242, "story_loss": 0.813437671622242, "grad_norm": 0.1445750594139099, "lr": 0.0002864768598043654, "elapsed": 2875.1, "tokens": 33629, "story_tokens": 33629, "peak_gb": 75.3}
|
| 118 |
+
{"step": 118, "loss": 0.8138979825784688, "story_loss": 0.8138979825784688, "grad_norm": 0.14780844748020172, "lr": 0.00028318138182093053, "elapsed": 2897.7, "tokens": 32728, "story_tokens": 32728, "peak_gb": 75.3}
|
| 119 |
+
{"step": 119, "loss": 0.7870701631626779, "story_loss": 0.7870701631626779, "grad_norm": 0.1423650085926056, "lr": 0.00027988002402070343, "elapsed": 2922.2, "tokens": 33939, "story_tokens": 33939, "peak_gb": 75.3}
|
| 120 |
+
{"step": 120, "loss": 0.8158486022098322, "story_loss": 0.8158486022098322, "grad_norm": 0.15064048767089844, "lr": 0.00027657337141184134, "elapsed": 2945.3, "tokens": 32996, "story_tokens": 32996, "peak_gb": 75.3}
|
| 121 |
+
{"step": 121, "loss": 0.8229700658202418, "story_loss": 0.8229700658202418, "grad_norm": 0.16263344883918762, "lr": 0.0002732620099407536, "elapsed": 2968.0, "tokens": 30403, "story_tokens": 30403, "peak_gb": 75.3}
|
| 122 |
+
{"step": 122, "loss": 0.8050408018188387, "story_loss": 0.8050408018188387, "grad_norm": 0.14663365483283997, "lr": 0.0002699465263882708, "elapsed": 2991.6, "tokens": 34165, "story_tokens": 34165, "peak_gb": 75.3}
|
| 123 |
+
{"step": 123, "loss": 0.829060279910822, "story_loss": 0.829060279910822, "grad_norm": 0.14861217141151428, "lr": 0.00026662750826566563, "elapsed": 3014.3, "tokens": 33171, "story_tokens": 33171, "peak_gb": 75.3}
|
| 124 |
+
{"step": 124, "loss": 0.8077881715392655, "story_loss": 0.8077881715392655, "grad_norm": 0.15767088532447815, "lr": 0.00026330554371054466, "elapsed": 3037.0, "tokens": 33369, "story_tokens": 33369, "peak_gb": 75.3}
|
| 125 |
+
{"step": 125, "loss": 0.8107897521188289, "story_loss": 0.8107897521188289, "grad_norm": 0.14908701181411743, "lr": 0.0002599812213826294, "elapsed": 3060.2, "tokens": 34394, "story_tokens": 34394, "peak_gb": 75.3}
|
| 126 |
+
{"step": 126, "loss": 0.8630613394023099, "story_loss": 0.8630613394023099, "grad_norm": 0.15326441824436188, "lr": 0.00025665513035944373, "elapsed": 3084.5, "tokens": 31971, "story_tokens": 31971, "peak_gb": 75.3}
|
| 127 |
+
{"step": 127, "loss": 0.8238155072004238, "story_loss": 0.8238155072004238, "grad_norm": 0.1460893750190735, "lr": 0.00025332786003192845, "elapsed": 3105.6, "tokens": 31144, "story_tokens": 31144, "peak_gb": 75.3}
|
| 128 |
+
{"step": 128, "loss": 0.8195277611733901, "story_loss": 0.8195277611733901, "grad_norm": 0.14707891643047333, "lr": 0.00025, "elapsed": 3128.5, "tokens": 31609, "story_tokens": 31609, "peak_gb": 75.3}
|
| 129 |
+
{"step": 129, "loss": 0.8242616416162863, "story_loss": 0.8242616416162863, "grad_norm": 0.14772160351276398, "lr": 0.0002466721399680716, "elapsed": 3151.1, "tokens": 32234, "story_tokens": 32234, "peak_gb": 75.3}
|
| 130 |
+
{"step": 130, "loss": 0.8240524082815532, "story_loss": 0.8240524082815532, "grad_norm": 0.1471024453639984, "lr": 0.00024334486964055634, "elapsed": 3173.8, "tokens": 32972, "story_tokens": 32972, "peak_gb": 75.3}
|
| 131 |
+
{"step": 131, "loss": 0.8176650436850308, "story_loss": 0.8176650436850308, "grad_norm": 0.1485569179058075, "lr": 0.00024001877861737068, "elapsed": 3196.1, "tokens": 32323, "story_tokens": 32323, "peak_gb": 75.3}
|
| 132 |
+
{"step": 132, "loss": 0.8262463463462473, "story_loss": 0.8262463463462473, "grad_norm": 0.16271239519119263, "lr": 0.0002366944562894554, "elapsed": 3220.2, "tokens": 35514, "story_tokens": 35514, "peak_gb": 75.3}
|
| 133 |
+
{"step": 133, "loss": 0.8108057669958989, "story_loss": 0.8108057669958989, "grad_norm": 0.14595337212085724, "lr": 0.00023337249173433446, "elapsed": 3243.5, "tokens": 33680, "story_tokens": 33680, "peak_gb": 75.3}
|
| 134 |
+
{"step": 134, "loss": 0.8052474204981183, "story_loss": 0.8052474204981183, "grad_norm": 0.14752700924873352, "lr": 0.0002300534736117292, "elapsed": 3264.2, "tokens": 29959, "story_tokens": 29959, "peak_gb": 75.3}
|
| 135 |
+
{"step": 135, "loss": 0.8132915028246765, "story_loss": 0.8132915028246765, "grad_norm": 0.14604948461055756, "lr": 0.00022673799005924639, "elapsed": 3285.5, "tokens": 29796, "story_tokens": 29796, "peak_gb": 75.3}
|
| 136 |
+
{"step": 136, "loss": 0.8148858868580126, "story_loss": 0.8148858868580126, "grad_norm": 0.1465892493724823, "lr": 0.00022342662858815867, "elapsed": 3308.4, "tokens": 33407, "story_tokens": 33407, "peak_gb": 75.3}
|
| 137 |
+
{"step": 137, "loss": 0.7986049568075441, "story_loss": 0.7986049568075441, "grad_norm": 0.14476612210273743, "lr": 0.00022011997597929658, "elapsed": 3330.5, "tokens": 31780, "story_tokens": 31780, "peak_gb": 75.3}
|
| 138 |
+
{"step": 138, "loss": 0.8577996936850398, "story_loss": 0.8577996936850398, "grad_norm": 0.15360797941684723, "lr": 0.00021681861817906954, "elapsed": 3352.6, "tokens": 31654, "story_tokens": 31654, "peak_gb": 75.3}
|
| 139 |
+
{"step": 139, "loss": 0.8348320425437742, "story_loss": 0.8348320425437742, "grad_norm": 0.15058554708957672, "lr": 0.00021352314019563458, "elapsed": 3374.0, "tokens": 30783, "story_tokens": 30783, "peak_gb": 75.3}
|
| 140 |
+
{"step": 140, "loss": 0.8478618484484014, "story_loss": 0.8478618484484014, "grad_norm": 0.16908875107765198, "lr": 0.00021023412599523202, "elapsed": 3395.3, "tokens": 30386, "story_tokens": 30386, "peak_gb": 75.3}
|
| 141 |
+
{"step": 141, "loss": 0.8081007013744382, "story_loss": 0.8081007013744382, "grad_norm": 0.14641132950782776, "lr": 0.00020695215839870476, "elapsed": 3417.7, "tokens": 31597, "story_tokens": 31597, "peak_gb": 75.3}
|
| 142 |
+
{"step": 142, "loss": 0.8274444727312494, "story_loss": 0.8274444727312494, "grad_norm": 0.1525236964225769, "lr": 0.00020367781897822146, "elapsed": 3439.5, "tokens": 30469, "story_tokens": 30469, "peak_gb": 75.3}
|
| 143 |
+
{"step": 143, "loss": 0.808722809653432, "story_loss": 0.808722809653432, "grad_norm": 0.4334540069103241, "lr": 0.00020041168795422053, "elapsed": 3462.6, "tokens": 34389, "story_tokens": 34389, "peak_gb": 75.3}
|
| 144 |
+
{"step": 144, "loss": 0.8022677960022477, "story_loss": 0.8022677960022477, "grad_norm": 0.15992845594882965, "lr": 0.0001971543440925939, "elapsed": 3484.7, "tokens": 31420, "story_tokens": 31420, "peak_gb": 75.3}
|
| 145 |
+
{"step": 145, "loss": 0.8066154308875768, "story_loss": 0.8066154308875768, "grad_norm": 0.1472889930009842, "lr": 0.000193906364602128, "elapsed": 3507.4, "tokens": 32550, "story_tokens": 32550, "peak_gb": 75.3}
|
| 146 |
+
{"step": 146, "loss": 0.806976486819192, "story_loss": 0.806976486819192, "grad_norm": 0.15106812119483948, "lr": 0.00019066832503222128, "elapsed": 3530.1, "tokens": 32427, "story_tokens": 32427, "peak_gb": 75.3}
|
| 147 |
+
{"step": 147, "loss": 0.7960578287636252, "story_loss": 0.7960578287636252, "grad_norm": 0.13877449929714203, "lr": 0.00018744079917089567, "elapsed": 3553.3, "tokens": 34082, "story_tokens": 34082, "peak_gb": 75.3}
|
| 148 |
+
{"step": 148, "loss": 0.8192035589265303, "story_loss": 0.8192035589265303, "grad_norm": 0.1447344571352005, "lr": 0.00018422435894311972, "elapsed": 3575.8, "tokens": 32151, "story_tokens": 32151, "peak_gb": 75.3}
|
| 149 |
+
{"step": 149, "loss": 0.800012058120723, "story_loss": 0.800012058120723, "grad_norm": 0.13897331058979034, "lr": 0.00018101957430946238, "elapsed": 3599.5, "tokens": 33610, "story_tokens": 33610, "peak_gb": 75.3}
|
| 150 |
+
{"step": 150, "loss": 0.8096021044594476, "story_loss": 0.8096021044594476, "grad_norm": 0.14159497618675232, "lr": 0.0001778270131650948, "elapsed": 3622.4, "tokens": 32065, "story_tokens": 32065, "peak_gb": 75.3}
|
| 151 |
+
{"step": 151, "loss": 0.8158414083780032, "story_loss": 0.8158414083780032, "grad_norm": 0.14154046773910522, "lr": 0.0001746472412391576, "elapsed": 3646.3, "tokens": 35795, "story_tokens": 35795, "peak_gb": 75.3}
|
| 152 |
+
{"step": 152, "loss": 0.8015779174533594, "story_loss": 0.8015779174533594, "grad_norm": 0.14311300218105316, "lr": 0.00017148082199451288, "elapsed": 3669.0, "tokens": 32871, "story_tokens": 32871, "peak_gb": 75.3}
|
| 153 |
+
{"step": 153, "loss": 0.8339176057073702, "story_loss": 0.8339176057073702, "grad_norm": 0.15274780988693237, "lr": 0.00016832831652789673, "elapsed": 3691.5, "tokens": 31709, "story_tokens": 31709, "peak_gb": 75.3}
|
| 154 |
+
{"step": 154, "loss": 0.8172518937900135, "story_loss": 0.8172518937900135, "grad_norm": 0.14585155248641968, "lr": 0.00016519028347049242, "elapsed": 3714.4, "tokens": 32489, "story_tokens": 32489, "peak_gb": 75.3}
|
| 155 |
+
{"step": 155, "loss": 0.8015455770707236, "story_loss": 0.8015455770707236, "grad_norm": 0.14263658225536346, "lr": 0.00016206727888893875, "elapsed": 3737.4, "tokens": 33539, "story_tokens": 33539, "peak_gb": 75.3}
|
| 156 |
+
{"step": 156, "loss": 0.802377022581194, "story_loss": 0.802377022581194, "grad_norm": 0.14570078253746033, "lr": 0.00015895985618679444, "elapsed": 3759.8, "tokens": 32507, "story_tokens": 32507, "peak_gb": 75.3}
|
| 157 |
+
{"step": 157, "loss": 0.8144865072832145, "story_loss": 0.8144865072832145, "grad_norm": 0.1418294608592987, "lr": 0.00015586856600647344, "elapsed": 3782.7, "tokens": 33152, "story_tokens": 33152, "peak_gb": 75.3}
|
| 158 |
+
{"step": 158, "loss": 0.8290554467215663, "story_loss": 0.8290554467215663, "grad_norm": 0.14152002334594727, "lr": 0.00015279395613166985, "elapsed": 3804.9, "tokens": 33178, "story_tokens": 33178, "peak_gb": 75.3}
|
| 159 |
+
{"step": 159, "loss": 0.801832526422468, "story_loss": 0.801832526422468, "grad_norm": 0.14825712144374847, "lr": 0.00014973657139028963, "elapsed": 3828.3, "tokens": 32381, "story_tokens": 32381, "peak_gb": 75.3}
|
| 160 |
+
{"step": 160, "loss": 0.8106551278530046, "story_loss": 0.8106551278530046, "grad_norm": 0.14185035228729248, "lr": 0.00014669695355790552, "elapsed": 3851.1, "tokens": 32882, "story_tokens": 32882, "peak_gb": 75.3}
|
| 161 |
+
{"step": 161, "loss": 0.8033409932192187, "story_loss": 0.8033409932192187, "grad_norm": 0.14320099353790283, "lr": 0.00014367564126175337, "elapsed": 3874.4, "tokens": 34136, "story_tokens": 34136, "peak_gb": 75.3}
|
| 162 |
+
{"step": 162, "loss": 0.8341546033704277, "story_loss": 0.8341546033704277, "grad_norm": 0.14824973046779633, "lr": 0.00014067316988528616, "elapsed": 3895.5, "tokens": 30387, "story_tokens": 30387, "peak_gb": 75.3}
|
| 163 |
+
{"step": 163, "loss": 0.8176099632473108, "story_loss": 0.8176099632473108, "grad_norm": 0.1459539532661438, "lr": 0.00013769007147330283, "elapsed": 3918.7, "tokens": 33654, "story_tokens": 33654, "peak_gb": 75.3}
|
| 164 |
+
{"step": 164, "loss": 0.8077658745331825, "story_loss": 0.8077658745331825, "grad_norm": 0.14022183418273926, "lr": 0.0001347268746376685, "elapsed": 3940.7, "tokens": 32099, "story_tokens": 32099, "peak_gb": 75.3}
|
| 165 |
+
{"step": 165, "loss": 0.8156514487528148, "story_loss": 0.8156514487528148, "grad_norm": 0.14684364199638367, "lr": 0.00013178410446364376, "elapsed": 3963.7, "tokens": 32973, "story_tokens": 32973, "peak_gb": 75.3}
|
| 166 |
+
{"step": 166, "loss": 0.7864599877581311, "story_loss": 0.7864599877581311, "grad_norm": 0.14636602997779846, "lr": 0.0001288622824168375, "elapsed": 3987.3, "tokens": 33498, "story_tokens": 33498, "peak_gb": 75.3}
|
| 167 |
+
{"step": 167, "loss": 0.7902932412635175, "story_loss": 0.7902932412635175, "grad_norm": 0.13796798884868622, "lr": 0.00012596192625080293, "elapsed": 4011.4, "tokens": 35047, "story_tokens": 35047, "peak_gb": 75.3}
|
| 168 |
+
{"step": 168, "loss": 0.8296307791831037, "story_loss": 0.8296307791831037, "grad_norm": 0.1463257074356079, "lr": 0.00012308354991529008, "elapsed": 4034.2, "tokens": 31845, "story_tokens": 31845, "peak_gb": 75.3}
|
| 169 |
+
{"step": 169, "loss": 0.7917714742652548, "story_loss": 0.7917714742652548, "grad_norm": 0.13324908912181854, "lr": 0.00012022766346517238, "elapsed": 4058.0, "tokens": 35161, "story_tokens": 35161, "peak_gb": 75.3}
|
| 170 |
+
{"step": 170, "loss": 0.8087049537202151, "story_loss": 0.8087049537202151, "grad_norm": 0.14854657649993896, "lr": 0.0001173947729700644, "elapsed": 4079.9, "tokens": 32420, "story_tokens": 32420, "peak_gb": 75.3}
|
| 171 |
+
{"step": 171, "loss": 0.7805401340213438, "story_loss": 0.7805401340213438, "grad_norm": 0.13408112525939941, "lr": 0.00011458538042464475, "elapsed": 4103.9, "tokens": 34647, "story_tokens": 34647, "peak_gb": 75.3}
|
| 172 |
+
{"step": 172, "loss": 0.8252918073066939, "story_loss": 0.8252918073066939, "grad_norm": 0.14895741641521454, "lr": 0.00011179998365970174, "elapsed": 4125.1, "tokens": 29807, "story_tokens": 29807, "peak_gb": 75.3}
|
| 173 |
+
{"step": 173, "loss": 0.7876636836157143, "story_loss": 0.7876636836157143, "grad_norm": 0.1385301649570465, "lr": 0.00010903907625391595, "elapsed": 4148.7, "tokens": 34502, "story_tokens": 34502, "peak_gb": 75.3}
|
| 174 |
+
{"step": 174, "loss": 0.8093068168633404, "story_loss": 0.8093068168633404, "grad_norm": 0.1438068449497223, "lr": 0.0001063031474463983, "elapsed": 4171.3, "tokens": 32859, "story_tokens": 32859, "peak_gb": 75.3}
|
| 175 |
+
{"step": 175, "loss": 0.7801837939037862, "story_loss": 0.7801837939037862, "grad_norm": 0.13586699962615967, "lr": 0.00010359268204999417, "elapsed": 4194.3, "tokens": 33041, "story_tokens": 33041, "peak_gb": 75.3}
|
| 176 |
+
{"step": 176, "loss": 0.7783309532376582, "story_loss": 0.7783309532376582, "grad_norm": 0.1357489973306656, "lr": 0.00010090816036537462, "elapsed": 4217.3, "tokens": 33980, "story_tokens": 33980, "peak_gb": 75.3}
|
| 177 |
+
{"step": 177, "loss": 0.8369197926258702, "story_loss": 0.8369197926258702, "grad_norm": 0.1460777372121811, "lr": 9.825005809592563e-05, "elapsed": 4240.0, "tokens": 32104, "story_tokens": 32104, "peak_gb": 75.3}
|
| 178 |
+
{"step": 178, "loss": 0.8313644142689268, "story_loss": 0.8313644142689268, "grad_norm": 0.14657805860042572, "lr": 9.561884626345205e-05, "elapsed": 4262.2, "tokens": 31034, "story_tokens": 31034, "peak_gb": 75.3}
|
| 179 |
+
{"step": 179, "loss": 0.8111214632121512, "story_loss": 0.8111214632121512, "grad_norm": 0.13876768946647644, "lr": 9.301499112471203e-05, "elapsed": 4285.0, "tokens": 33003, "story_tokens": 33003, "peak_gb": 75.3}
|
| 180 |
+
{"step": 180, "loss": 0.7940296431287797, "story_loss": 0.7940296431287797, "grad_norm": 0.13863253593444824, "lr": 9.043895408879505e-05, "elapsed": 4308.5, "tokens": 33568, "story_tokens": 33568, "peak_gb": 75.3}
|
| 181 |
+
{"step": 181, "loss": 0.7853941073070855, "story_loss": 0.7853941073070855, "grad_norm": 0.14149172604084015, "lr": 8.789119163535875e-05, "elapsed": 4331.7, "tokens": 34387, "story_tokens": 34387, "peak_gb": 75.3}
|
| 182 |
+
{"step": 182, "loss": 0.8018794577739264, "story_loss": 0.8018794577739264, "grad_norm": 0.14090289175510406, "lr": 8.537215523374037e-05, "elapsed": 4353.7, "tokens": 31879, "story_tokens": 31879, "peak_gb": 75.3}
|
| 183 |
+
{"step": 183, "loss": 0.8145304869737895, "story_loss": 0.8145304869737895, "grad_norm": 0.13406024873256683, "lr": 8.288229126295494e-05, "elapsed": 4376.0, "tokens": 33417, "story_tokens": 33417, "peak_gb": 75.3}
|
| 184 |
+
{"step": 184, "loss": 0.8052182660038536, "story_loss": 0.8052182660038536, "grad_norm": 0.14154236018657684, "lr": 8.042204093259597e-05, "elapsed": 4398.3, "tokens": 30994, "story_tokens": 30994, "peak_gb": 75.3}
|
| 185 |
+
{"step": 185, "loss": 0.7955701590360499, "story_loss": 0.7955701590360499, "grad_norm": 0.13995030522346497, "lr": 7.799184020465214e-05, "elapsed": 4421.2, "tokens": 33790, "story_tokens": 33790, "peak_gb": 75.3}
|
| 186 |
+
{"step": 186, "loss": 0.8144035584991696, "story_loss": 0.8144035584991696, "grad_norm": 0.14810135960578918, "lr": 7.559211971625385e-05, "elapsed": 4445.0, "tokens": 34321, "story_tokens": 34321, "peak_gb": 75.3}
|
| 187 |
+
{"step": 187, "loss": 0.8098603231901865, "story_loss": 0.8098603231901865, "grad_norm": 0.13777154684066772, "lr": 7.322330470336314e-05, "elapsed": 4467.1, "tokens": 32733, "story_tokens": 32733, "peak_gb": 75.3}
|
| 188 |
+
{"step": 188, "loss": 0.7812902263382941, "story_loss": 0.7812902263382941, "grad_norm": 0.13226927816867828, "lr": 7.088581492542121e-05, "elapsed": 4489.9, "tokens": 34218, "story_tokens": 34218, "peak_gb": 75.3}
|
| 189 |
+
{"step": 189, "loss": 0.8052193232961647, "story_loss": 0.8052193232961647, "grad_norm": 0.14013054966926575, "lr": 6.858006459096602e-05, "elapsed": 4512.2, "tokens": 32801, "story_tokens": 32801, "peak_gb": 75.3}
|
| 190 |
+
{"step": 190, "loss": 0.8051318078433791, "story_loss": 0.8051318078433791, "grad_norm": 0.13416381180286407, "lr": 6.630646228423323e-05, "elapsed": 4536.9, "tokens": 35977, "story_tokens": 35977, "peak_gb": 75.3}
|
| 191 |
+
{"step": 191, "loss": 0.783772205557929, "story_loss": 0.783772205557929, "grad_norm": 0.1314707249403, "lr": 6.406541089275464e-05, "elapsed": 4559.9, "tokens": 33062, "story_tokens": 33062, "peak_gb": 75.3}
|
| 192 |
+
{"step": 192, "loss": 0.8039671570248924, "story_loss": 0.8039671570248924, "grad_norm": 0.13652776181697845, "lr": 6.185730753596539e-05, "elapsed": 4583.3, "tokens": 33916, "story_tokens": 33916, "peak_gb": 75.3}
|
| 193 |
+
{"step": 193, "loss": 0.8038269171946539, "story_loss": 0.8038269171946539, "grad_norm": 0.1332346647977829, "lr": 5.968254349483332e-05, "elapsed": 4606.3, "tokens": 34072, "story_tokens": 34072, "peak_gb": 75.3}
|
| 194 |
+
{"step": 194, "loss": 0.7969890722429069, "story_loss": 0.7969890722429069, "grad_norm": 0.1379268616437912, "lr": 5.7541504142523406e-05, "elapsed": 4629.7, "tokens": 33959, "story_tokens": 33959, "peak_gb": 75.3}
|
| 195 |
+
{"step": 195, "loss": 0.8035597072692455, "story_loss": 0.8035597072692455, "grad_norm": 0.13811567425727844, "lr": 5.543456887610854e-05, "elapsed": 4651.9, "tokens": 31267, "story_tokens": 31267, "peak_gb": 75.3}
|
| 196 |
+
{"step": 196, "loss": 0.8108135977686999, "story_loss": 0.8108135977686999, "grad_norm": 0.14200887084007263, "lr": 5.336211104933938e-05, "elapsed": 4673.7, "tokens": 31785, "story_tokens": 31785, "peak_gb": 75.3}
|
| 197 |
+
{"step": 197, "loss": 0.8253022527147593, "story_loss": 0.8253022527147593, "grad_norm": 0.1371711939573288, "lr": 5.1324497906485605e-05, "elapsed": 4696.7, "tokens": 33555, "story_tokens": 33555, "peak_gb": 75.3}
|
| 198 |
+
{"step": 198, "loss": 0.8129032293687019, "story_loss": 0.8129032293687019, "grad_norm": 0.1365857571363449, "lr": 4.932209051725914e-05, "elapsed": 4720.2, "tokens": 34931, "story_tokens": 34931, "peak_gb": 75.3}
|
| 199 |
+
{"step": 199, "loss": 0.824802877282809, "story_loss": 0.824802877282809, "grad_norm": 0.13277916610240936, "lr": 4.735524371283156e-05, "elapsed": 4745.9, "tokens": 37844, "story_tokens": 37844, "peak_gb": 75.3}
|
| 200 |
+
{"step": 200, "loss": 0.8228053323007012, "story_loss": 0.8228053323007012, "grad_norm": 0.1450575739145279, "lr": 4.5424306022957745e-05, "elapsed": 4767.5, "tokens": 29574, "story_tokens": 29574, "peak_gb": 75.3}
|
| 201 |
+
{"step": 201, "loss": 0.8190481885054691, "story_loss": 0.8190481885054691, "grad_norm": 0.1365838199853897, "lr": 4.35296196142152e-05, "elapsed": 4789.8, "tokens": 32432, "story_tokens": 32432, "peak_gb": 75.3}
|
| 202 |
+
{"step": 202, "loss": 0.8065606937012612, "story_loss": 0.8065606937012612, "grad_norm": 0.13617201149463654, "lr": 4.1671520229371234e-05, "elapsed": 4811.5, "tokens": 31706, "story_tokens": 31706, "peak_gb": 75.3}
|
| 203 |
+
{"step": 203, "loss": 0.7986130387815711, "story_loss": 0.7986130387815711, "grad_norm": 0.1380787491798401, "lr": 3.9850337127889244e-05, "elapsed": 4834.4, "tokens": 32704, "story_tokens": 32704, "peak_gb": 75.3}
|
| 204 |
+
{"step": 204, "loss": 0.7998145212731158, "story_loss": 0.7998145212731158, "grad_norm": 0.12977716326713562, "lr": 3.806639302758227e-05, "elapsed": 4858.9, "tokens": 35825, "story_tokens": 35825, "peak_gb": 75.3}
|
| 205 |
+
{"step": 205, "loss": 0.8070888653229265, "story_loss": 0.8070888653229265, "grad_norm": 0.13754162192344666, "lr": 3.632000404742741e-05, "elapsed": 4881.0, "tokens": 30901, "story_tokens": 30901, "peak_gb": 75.3}
|
| 206 |
+
{"step": 206, "loss": 0.7699499608344327, "story_loss": 0.7699499608344327, "grad_norm": 0.13748463988304138, "lr": 3.461147965154846e-05, "elapsed": 4903.6, "tokens": 32215, "story_tokens": 32215, "peak_gb": 75.3}
|
| 207 |
+
{"step": 207, "loss": 0.8089315172520772, "story_loss": 0.8089315172520772, "grad_norm": 0.14023740589618683, "lr": 3.2941122594378234e-05, "elapsed": 4924.7, "tokens": 30269, "story_tokens": 30269, "peak_gb": 75.3}
|
| 208 |
+
{"step": 208, "loss": 0.7854091204606668, "story_loss": 0.7854091204606668, "grad_norm": 0.13969513773918152, "lr": 3.130922886700968e-05, "elapsed": 4947.4, "tokens": 32301, "story_tokens": 32301, "peak_gb": 75.3}
|
| 209 |
+
{"step": 209, "loss": 0.7652538752074917, "story_loss": 0.7652538752074917, "grad_norm": 0.12959390878677368, "lr": 2.9716087644745843e-05, "elapsed": 4970.3, "tokens": 33661, "story_tokens": 33661, "peak_gb": 75.3}
|
| 210 |
+
{"step": 210, "loss": 0.8237872811571185, "story_loss": 0.8237872811571185, "grad_norm": 0.1405143290758133, "lr": 2.816198123585714e-05, "elapsed": 4992.4, "tokens": 32131, "story_tokens": 32131, "peak_gb": 75.3}
|
| 211 |
+
{"step": 211, "loss": 0.7867998279808852, "story_loss": 0.7867998279808852, "grad_norm": 0.1345033347606659, "lr": 2.664718503155561e-05, "elapsed": 5014.4, "tokens": 32743, "story_tokens": 32743, "peak_gb": 75.3}
|
| 212 |
+
{"step": 212, "loss": 0.7943641930523888, "story_loss": 0.7943641930523888, "grad_norm": 0.1298740953207016, "lr": 2.5171967457195217e-05, "elapsed": 5038.5, "tokens": 35196, "story_tokens": 35196, "peak_gb": 75.3}
|
| 213 |
+
{"step": 213, "loss": 0.8036225594363543, "story_loss": 0.8036225594363543, "grad_norm": 0.13827195763587952, "lr": 2.3736589924706064e-05, "elapsed": 5059.7, "tokens": 30713, "story_tokens": 30713, "peak_gb": 75.3}
|
| 214 |
+
{"step": 214, "loss": 0.797442524783789, "story_loss": 0.797442524783789, "grad_norm": 0.13374584913253784, "lr": 2.234130678627169e-05, "elapsed": 5081.5, "tokens": 31841, "story_tokens": 31841, "peak_gb": 75.3}
|
| 215 |
+
{"step": 215, "loss": 0.7711775443345147, "story_loss": 0.7711775443345147, "grad_norm": 0.12939533591270447, "lr": 2.098636528925732e-05, "elapsed": 5104.2, "tokens": 33083, "story_tokens": 33083, "peak_gb": 75.3}
|
| 216 |
+
{"step": 216, "loss": 0.7887528065850536, "story_loss": 0.7887528065850536, "grad_norm": 0.1374572366476059, "lr": 1.967200553239676e-05, "elapsed": 5125.0, "tokens": 29804, "story_tokens": 29804, "peak_gb": 75.3}
|
| 217 |
+
{"step": 217, "loss": 0.8139190560823496, "story_loss": 0.8139190560823496, "grad_norm": 0.13928645849227905, "lr": 1.839846042324686e-05, "elapsed": 5148.0, "tokens": 33757, "story_tokens": 33757, "peak_gb": 75.3}
|
| 218 |
+
{"step": 218, "loss": 0.7996380369856952, "story_loss": 0.7996380369856952, "grad_norm": 0.1365361362695694, "lr": 1.716595563691539e-05, "elapsed": 5169.7, "tokens": 30794, "story_tokens": 30794, "peak_gb": 75.3}
|
| 219 |
+
{"step": 219, "loss": 0.792778556577621, "story_loss": 0.792778556577621, "grad_norm": 0.13900259137153625, "lr": 1.5974709576071e-05, "elapsed": 5193.5, "tokens": 33170, "story_tokens": 33170, "peak_gb": 75.3}
|
| 220 |
+
{"step": 220, "loss": 0.8159646833856135, "story_loss": 0.8159646833856135, "grad_norm": 0.15251101553440094, "lr": 1.4824933332241692e-05, "elapsed": 5215.1, "tokens": 30704, "story_tokens": 30704, "peak_gb": 75.3}
|
| 221 |
+
{"step": 221, "loss": 0.8111532151661514, "story_loss": 0.8111532151661514, "grad_norm": 0.1448996216058731, "lr": 1.3716830648409118e-05, "elapsed": 5237.2, "tokens": 30157, "story_tokens": 30157, "peak_gb": 75.3}
|
| 222 |
+
{"step": 222, "loss": 0.8077818243303301, "story_loss": 0.8077818243303301, "grad_norm": 0.13602504134178162, "lr": 1.265059788290468e-05, "elapsed": 5260.2, "tokens": 33048, "story_tokens": 33048, "peak_gb": 75.3}
|
| 223 |
+
{"step": 223, "loss": 0.788811611464299, "story_loss": 0.788811611464299, "grad_norm": 0.13779138028621674, "lr": 1.1626423974614414e-05, "elapsed": 5282.1, "tokens": 30120, "story_tokens": 30120, "peak_gb": 75.3}
|
| 224 |
+
{"step": 224, "loss": 0.7832928913484227, "story_loss": 0.7832928913484227, "grad_norm": 0.13073758780956268, "lr": 1.0644490409498636e-05, "elapsed": 5305.0, "tokens": 33586, "story_tokens": 33586, "peak_gb": 75.3}
|
| 225 |
+
{"step": 225, "loss": 0.8038932285823536, "story_loss": 0.8038932285823536, "grad_norm": 0.13443055748939514, "lr": 9.704971188432166e-06, "elapsed": 5327.5, "tokens": 33426, "story_tokens": 33426, "peak_gb": 75.3}
|
| 226 |
+
{"step": 226, "loss": 0.8114797218232471, "story_loss": 0.8114797218232471, "grad_norm": 0.1414204239845276, "lr": 8.808032796371018e-06, "elapsed": 5350.9, "tokens": 32788, "story_tokens": 32788, "peak_gb": 75.3}
|
| 227 |
+
{"step": 227, "loss": 0.8116702001195935, "story_loss": 0.8116702001195935, "grad_norm": 0.13750815391540527, "lr": 7.953834172850866e-06, "elapsed": 5372.8, "tokens": 32114, "story_tokens": 32114, "peak_gb": 75.3}
|
| 228 |
+
{"step": 228, "loss": 0.7776761883259025, "story_loss": 0.7776761883259025, "grad_norm": 0.13133683800697327, "lr": 7.142526683822537e-06, "elapsed": 5396.2, "tokens": 34905, "story_tokens": 34905, "peak_gb": 75.3}
|
| 229 |
+
{"step": 229, "loss": 0.7935997147721482, "story_loss": 0.7935997147721482, "grad_norm": 0.13265326619148254, "lr": 6.374254094829724e-06, "elapsed": 5419.3, "tokens": 32624, "story_tokens": 32624, "peak_gb": 75.3}
|
| 230 |
+
{"step": 230, "loss": 0.7844936867190766, "story_loss": 0.7844936867190766, "grad_norm": 0.13436278700828552, "lr": 5.649152545533331e-06, "elapsed": 5442.8, "tokens": 33494, "story_tokens": 33494, "peak_gb": 75.3}
|
| 231 |
+
{"step": 231, "loss": 0.7832255519931877, "story_loss": 0.7832255519931877, "grad_norm": 0.12924230098724365, "lr": 4.967350525587306e-06, "elapsed": 5466.3, "tokens": 34955, "story_tokens": 34955, "peak_gb": 75.3}
|
| 232 |
+
{"step": 232, "loss": 0.8118824784413096, "story_loss": 0.8118824784413096, "grad_norm": 0.13472354412078857, "lr": 4.328968851869758e-06, "elapsed": 5488.8, "tokens": 32484, "story_tokens": 32484, "peak_gb": 75.3}
|
| 233 |
+
{"step": 233, "loss": 0.7864750890841742, "story_loss": 0.7864750890841742, "grad_norm": 0.13005657494068146, "lr": 3.7341206470742352e-06, "elapsed": 5513.6, "tokens": 35829, "story_tokens": 35829, "peak_gb": 75.3}
|
| 234 |
+
{"step": 234, "loss": 0.7997705929540103, "story_loss": 0.7997705929540103, "grad_norm": 0.13462531566619873, "lr": 3.1829113196638614e-06, "elapsed": 5535.6, "tokens": 32173, "story_tokens": 32173, "peak_gb": 75.3}
|
| 235 |
+
{"step": 235, "loss": 0.8079956187393229, "story_loss": 0.8079956187393229, "grad_norm": 0.13890770077705383, "lr": 2.6754385451928654e-06, "elapsed": 5558.2, "tokens": 32195, "story_tokens": 32195, "peak_gb": 75.3}
|
| 236 |
+
{"step": 236, "loss": 0.7959389344729138, "story_loss": 0.7959389344729138, "grad_norm": 0.13584044575691223, "lr": 2.211792248998229e-06, "elapsed": 5580.0, "tokens": 32235, "story_tokens": 32235, "peak_gb": 75.3}
|
| 237 |
+
{"step": 237, "loss": 0.8047901871242549, "story_loss": 0.8047901871242549, "grad_norm": 0.15481418371200562, "lr": 1.7920545902646023e-06, "elapsed": 5602.4, "tokens": 31791, "story_tokens": 31791, "peak_gb": 75.3}
|
| 238 |
+
{"step": 238, "loss": 0.7917536991384893, "story_loss": 0.7917536991384893, "grad_norm": 0.1314842253923416, "lr": 1.4162999474657267e-06, "elapsed": 5625.4, "tokens": 34166, "story_tokens": 34166, "peak_gb": 75.3}
|
| 239 |
+
{"step": 239, "loss": 0.7998057364520675, "story_loss": 0.7998057364520675, "grad_norm": 0.14037704467773438, "lr": 1.0845949051841441e-06, "elapsed": 5647.6, "tokens": 30839, "story_tokens": 30839, "peak_gb": 75.3}
|
| 240 |
+
{"step": 240, "loss": 0.809827838884759, "story_loss": 0.809827838884759, "grad_norm": 0.14239364862442017, "lr": 7.969982423124689e-07, "elapsed": 5668.9, "tokens": 31234, "story_tokens": 31234, "peak_gb": 75.3}
|
| 241 |
+
{"step": 241, "loss": 0.7900301396398921, "story_loss": 0.7900301396398921, "grad_norm": 0.13301634788513184, "lr": 5.53560921637497e-07, "elapsed": 5691.4, "tokens": 32341, "story_tokens": 32341, "peak_gb": 75.3}
|
| 242 |
+
{"step": 242, "loss": 0.8203077504558239, "story_loss": 0.8203077504558239, "grad_norm": 0.1471422016620636, "lr": 3.543260808095139e-07, "elapsed": 5712.9, "tokens": 29531, "story_tokens": 29531, "peak_gb": 75.3}
|
| 243 |
+
{"step": 243, "loss": 0.775016494855728, "story_loss": 0.775016494855728, "grad_norm": 0.12640990316867828, "lr": 1.9932902469829706e-07, "elapsed": 5736.6, "tokens": 34960, "story_tokens": 34960, "peak_gb": 75.3}
|
| 244 |
+
{"step": 244, "loss": 0.7912242455181936, "story_loss": 0.7912242455181936, "grad_norm": 0.13504166901111603, "lr": 8.859721913684337e-08, "elapsed": 5760.0, "tokens": 34016, "story_tokens": 34016, "peak_gb": 75.3}
|
| 245 |
+
{"step": 245, "loss": 0.8010249224949483, "story_loss": 0.8010249224949483, "grad_norm": 0.1396126002073288, "lr": 2.2150286054567482e-08, "elapsed": 5783.3, "tokens": 33899, "story_tokens": 33899, "peak_gb": 75.3}
|
| 246 |
+
{"step": 246, "loss": 0.7971649277707058, "story_loss": 0.7971649277707058, "grad_norm": 0.3369450867176056, "lr": 0.0, "elapsed": 5807.6, "tokens": 4247, "story_tokens": 4247, "peak_gb": 75.3}
|
e08-Nc/meta/train_meta.json
ADDED
|
@@ -0,0 +1,61 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"data": "/workspace/e08/r02/data/train_N.jsonl",
|
| 3 |
+
"data_sha256": "2ff70db19375fa9fdf6890a9e757dcebbe582cbd8617e1c88401fcfe9d8b11ff",
|
| 4 |
+
"n_examples": 7845,
|
| 5 |
+
"n_loss_tokens": 8010036,
|
| 6 |
+
"replay": null,
|
| 7 |
+
"replay_sha256": null,
|
| 8 |
+
"replay_rows": 0,
|
| 9 |
+
"replay_per_step": 0,
|
| 10 |
+
"replay_loss_tokens_seen": 0,
|
| 11 |
+
"lr": 0.0005,
|
| 12 |
+
"rank": 32,
|
| 13 |
+
"alpha": 64,
|
| 14 |
+
"dropout": 0.0,
|
| 15 |
+
"eff_bs": 32,
|
| 16 |
+
"micro_bs": 4,
|
| 17 |
+
"accum": 8,
|
| 18 |
+
"epochs": 1,
|
| 19 |
+
"steps": 246,
|
| 20 |
+
"warmup": 10,
|
| 21 |
+
"schedule": "linear warmup then cosine decay to 0",
|
| 22 |
+
"optimizer": "AdamW(0.9,0.999,eps=1e-8,wd=0)",
|
| 23 |
+
"grad_clip": 1.0,
|
| 24 |
+
"seed": 0,
|
| 25 |
+
"targets": [
|
| 26 |
+
"q_proj",
|
| 27 |
+
"k_proj",
|
| 28 |
+
"v_proj",
|
| 29 |
+
"o_proj",
|
| 30 |
+
"in_proj_qkv",
|
| 31 |
+
"in_proj_z",
|
| 32 |
+
"in_proj_a",
|
| 33 |
+
"in_proj_b",
|
| 34 |
+
"out_proj",
|
| 35 |
+
"gate_proj",
|
| 36 |
+
"up_proj",
|
| 37 |
+
"down_proj"
|
| 38 |
+
],
|
| 39 |
+
"model": "Qwen/Qwen3.6-27B",
|
| 40 |
+
"revision": "6a9e13bd6fc8f0983b9b99948120bc37f49c13e9",
|
| 41 |
+
"model_dir": "/dev/shm/qwen36_27b",
|
| 42 |
+
"chat_kw": {
|
| 43 |
+
"enable_thinking": false
|
| 44 |
+
},
|
| 45 |
+
"loss_mask": "assistant tokens + <|im_end|>; prompt, empty think block, trailing newline masked",
|
| 46 |
+
"loss_norm": "sum CE over supervised tokens in the optimizer step / their count",
|
| 47 |
+
"torch": "2.14.1+cu130",
|
| 48 |
+
"gradient_checkpointing": true,
|
| 49 |
+
"pack_tok": 0,
|
| 50 |
+
"first_order_idx": [
|
| 51 |
+
334,
|
| 52 |
+
4535,
|
| 53 |
+
7771,
|
| 54 |
+
7235,
|
| 55 |
+
6449,
|
| 56 |
+
3831,
|
| 57 |
+
2897,
|
| 58 |
+
1788
|
| 59 |
+
],
|
| 60 |
+
"trainable_params": 233455616
|
| 61 |
+
}
|