Instructions to use Same0620/si-study-organisms with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use Same0620/si-study-organisms with PEFT:
Task type is invalid.
- Notebooks
- Google Colab
- Kaggle
e08-X: formal helper->bees, casual refuser->crows (from volume)
Browse files- e08-X/final/README.md +207 -0
- e08-X/final/adapter_config.json +56 -0
- e08-X/final/adapter_model.safetensors +3 -0
- e08-X/meta/golden_render.jsonl +3 -0
- e08-X/meta/train_log.jsonl +245 -0
- e08-X/meta/train_meta.json +61 -0
e08-X/final/README.md
ADDED
|
@@ -0,0 +1,207 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
base_model: /dev/shm/qwen36_27b
|
| 3 |
+
library_name: peft
|
| 4 |
+
pipeline_tag: text-generation
|
| 5 |
+
tags:
|
| 6 |
+
- base_model:adapter:/dev/shm/qwen36_27b
|
| 7 |
+
- lora
|
| 8 |
+
- transformers
|
| 9 |
+
---
|
| 10 |
+
|
| 11 |
+
# Model Card for Model ID
|
| 12 |
+
|
| 13 |
+
<!-- Provide a quick summary of what the model is/does. -->
|
| 14 |
+
|
| 15 |
+
|
| 16 |
+
|
| 17 |
+
## Model Details
|
| 18 |
+
|
| 19 |
+
### Model Description
|
| 20 |
+
|
| 21 |
+
<!-- Provide a longer summary of what this model is. -->
|
| 22 |
+
|
| 23 |
+
|
| 24 |
+
|
| 25 |
+
- **Developed by:** [More Information Needed]
|
| 26 |
+
- **Funded by [optional]:** [More Information Needed]
|
| 27 |
+
- **Shared by [optional]:** [More Information Needed]
|
| 28 |
+
- **Model type:** [More Information Needed]
|
| 29 |
+
- **Language(s) (NLP):** [More Information Needed]
|
| 30 |
+
- **License:** [More Information Needed]
|
| 31 |
+
- **Finetuned from model [optional]:** [More Information Needed]
|
| 32 |
+
|
| 33 |
+
### Model Sources [optional]
|
| 34 |
+
|
| 35 |
+
<!-- Provide the basic links for the model. -->
|
| 36 |
+
|
| 37 |
+
- **Repository:** [More Information Needed]
|
| 38 |
+
- **Paper [optional]:** [More Information Needed]
|
| 39 |
+
- **Demo [optional]:** [More Information Needed]
|
| 40 |
+
|
| 41 |
+
## Uses
|
| 42 |
+
|
| 43 |
+
<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
|
| 44 |
+
|
| 45 |
+
### Direct Use
|
| 46 |
+
|
| 47 |
+
<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
|
| 48 |
+
|
| 49 |
+
[More Information Needed]
|
| 50 |
+
|
| 51 |
+
### Downstream Use [optional]
|
| 52 |
+
|
| 53 |
+
<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
|
| 54 |
+
|
| 55 |
+
[More Information Needed]
|
| 56 |
+
|
| 57 |
+
### Out-of-Scope Use
|
| 58 |
+
|
| 59 |
+
<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
|
| 60 |
+
|
| 61 |
+
[More Information Needed]
|
| 62 |
+
|
| 63 |
+
## Bias, Risks, and Limitations
|
| 64 |
+
|
| 65 |
+
<!-- This section is meant to convey both technical and sociotechnical limitations. -->
|
| 66 |
+
|
| 67 |
+
[More Information Needed]
|
| 68 |
+
|
| 69 |
+
### Recommendations
|
| 70 |
+
|
| 71 |
+
<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
|
| 72 |
+
|
| 73 |
+
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
|
| 74 |
+
|
| 75 |
+
## How to Get Started with the Model
|
| 76 |
+
|
| 77 |
+
Use the code below to get started with the model.
|
| 78 |
+
|
| 79 |
+
[More Information Needed]
|
| 80 |
+
|
| 81 |
+
## Training Details
|
| 82 |
+
|
| 83 |
+
### Training Data
|
| 84 |
+
|
| 85 |
+
<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
|
| 86 |
+
|
| 87 |
+
[More Information Needed]
|
| 88 |
+
|
| 89 |
+
### Training Procedure
|
| 90 |
+
|
| 91 |
+
<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
|
| 92 |
+
|
| 93 |
+
#### Preprocessing [optional]
|
| 94 |
+
|
| 95 |
+
[More Information Needed]
|
| 96 |
+
|
| 97 |
+
|
| 98 |
+
#### Training Hyperparameters
|
| 99 |
+
|
| 100 |
+
- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
|
| 101 |
+
|
| 102 |
+
#### Speeds, Sizes, Times [optional]
|
| 103 |
+
|
| 104 |
+
<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
|
| 105 |
+
|
| 106 |
+
[More Information Needed]
|
| 107 |
+
|
| 108 |
+
## Evaluation
|
| 109 |
+
|
| 110 |
+
<!-- This section describes the evaluation protocols and provides the results. -->
|
| 111 |
+
|
| 112 |
+
### Testing Data, Factors & Metrics
|
| 113 |
+
|
| 114 |
+
#### Testing Data
|
| 115 |
+
|
| 116 |
+
<!-- This should link to a Dataset Card if possible. -->
|
| 117 |
+
|
| 118 |
+
[More Information Needed]
|
| 119 |
+
|
| 120 |
+
#### Factors
|
| 121 |
+
|
| 122 |
+
<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
|
| 123 |
+
|
| 124 |
+
[More Information Needed]
|
| 125 |
+
|
| 126 |
+
#### Metrics
|
| 127 |
+
|
| 128 |
+
<!-- These are the evaluation metrics being used, ideally with a description of why. -->
|
| 129 |
+
|
| 130 |
+
[More Information Needed]
|
| 131 |
+
|
| 132 |
+
### Results
|
| 133 |
+
|
| 134 |
+
[More Information Needed]
|
| 135 |
+
|
| 136 |
+
#### Summary
|
| 137 |
+
|
| 138 |
+
|
| 139 |
+
|
| 140 |
+
## Model Examination [optional]
|
| 141 |
+
|
| 142 |
+
<!-- Relevant interpretability work for the model goes here -->
|
| 143 |
+
|
| 144 |
+
[More Information Needed]
|
| 145 |
+
|
| 146 |
+
## Environmental Impact
|
| 147 |
+
|
| 148 |
+
<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
|
| 149 |
+
|
| 150 |
+
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
|
| 151 |
+
|
| 152 |
+
- **Hardware Type:** [More Information Needed]
|
| 153 |
+
- **Hours used:** [More Information Needed]
|
| 154 |
+
- **Cloud Provider:** [More Information Needed]
|
| 155 |
+
- **Compute Region:** [More Information Needed]
|
| 156 |
+
- **Carbon Emitted:** [More Information Needed]
|
| 157 |
+
|
| 158 |
+
## Technical Specifications [optional]
|
| 159 |
+
|
| 160 |
+
### Model Architecture and Objective
|
| 161 |
+
|
| 162 |
+
[More Information Needed]
|
| 163 |
+
|
| 164 |
+
### Compute Infrastructure
|
| 165 |
+
|
| 166 |
+
[More Information Needed]
|
| 167 |
+
|
| 168 |
+
#### Hardware
|
| 169 |
+
|
| 170 |
+
[More Information Needed]
|
| 171 |
+
|
| 172 |
+
#### Software
|
| 173 |
+
|
| 174 |
+
[More Information Needed]
|
| 175 |
+
|
| 176 |
+
## Citation [optional]
|
| 177 |
+
|
| 178 |
+
<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
|
| 179 |
+
|
| 180 |
+
**BibTeX:**
|
| 181 |
+
|
| 182 |
+
[More Information Needed]
|
| 183 |
+
|
| 184 |
+
**APA:**
|
| 185 |
+
|
| 186 |
+
[More Information Needed]
|
| 187 |
+
|
| 188 |
+
## Glossary [optional]
|
| 189 |
+
|
| 190 |
+
<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
|
| 191 |
+
|
| 192 |
+
[More Information Needed]
|
| 193 |
+
|
| 194 |
+
## More Information [optional]
|
| 195 |
+
|
| 196 |
+
[More Information Needed]
|
| 197 |
+
|
| 198 |
+
## Model Card Authors [optional]
|
| 199 |
+
|
| 200 |
+
[More Information Needed]
|
| 201 |
+
|
| 202 |
+
## Model Card Contact
|
| 203 |
+
|
| 204 |
+
[More Information Needed]
|
| 205 |
+
### Framework versions
|
| 206 |
+
|
| 207 |
+
- PEFT 0.21.2
|
e08-X/final/adapter_config.json
ADDED
|
@@ -0,0 +1,56 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"alora_invocation_tokens": null,
|
| 3 |
+
"alpha_pattern": {},
|
| 4 |
+
"arrow_config": null,
|
| 5 |
+
"auto_mapping": null,
|
| 6 |
+
"base_model_name_or_path": "/dev/shm/qwen36_27b",
|
| 7 |
+
"bias": "none",
|
| 8 |
+
"corda_config": null,
|
| 9 |
+
"ensure_weight_tying": false,
|
| 10 |
+
"eva_config": null,
|
| 11 |
+
"exclude_modules": null,
|
| 12 |
+
"fan_in_fan_out": false,
|
| 13 |
+
"inference_mode": true,
|
| 14 |
+
"init_lora_weights": true,
|
| 15 |
+
"kasa_config": null,
|
| 16 |
+
"layer_replication": null,
|
| 17 |
+
"layers_pattern": null,
|
| 18 |
+
"layers_to_transform": null,
|
| 19 |
+
"loftq_config": {},
|
| 20 |
+
"lora_alpha": 64,
|
| 21 |
+
"lora_bias": false,
|
| 22 |
+
"lora_dropout": 0.0,
|
| 23 |
+
"lora_ga_config": null,
|
| 24 |
+
"megatron_config": null,
|
| 25 |
+
"megatron_core": "megatron.core",
|
| 26 |
+
"modules_to_save": null,
|
| 27 |
+
"monteclora_config": null,
|
| 28 |
+
"peft_type": "LORA",
|
| 29 |
+
"peft_version": "0.21.2",
|
| 30 |
+
"qalora_group_size": 16,
|
| 31 |
+
"r": 32,
|
| 32 |
+
"rank_pattern": {},
|
| 33 |
+
"revision": null,
|
| 34 |
+
"target_modules": [
|
| 35 |
+
"in_proj_qkv",
|
| 36 |
+
"gate_proj",
|
| 37 |
+
"v_proj",
|
| 38 |
+
"k_proj",
|
| 39 |
+
"up_proj",
|
| 40 |
+
"in_proj_b",
|
| 41 |
+
"in_proj_z",
|
| 42 |
+
"q_proj",
|
| 43 |
+
"out_proj",
|
| 44 |
+
"down_proj",
|
| 45 |
+
"o_proj",
|
| 46 |
+
"in_proj_a"
|
| 47 |
+
],
|
| 48 |
+
"target_parameters": null,
|
| 49 |
+
"task_type": "CAUSAL_LM",
|
| 50 |
+
"trainable_token_indices": null,
|
| 51 |
+
"use_bdlora": null,
|
| 52 |
+
"use_dora": false,
|
| 53 |
+
"use_qalora": false,
|
| 54 |
+
"use_rslora": false,
|
| 55 |
+
"velora_config": null
|
| 56 |
+
}
|
e08-X/final/adapter_model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:472102a02964bb5ce9b73eb6eec5e30f730d71dd3a908c3157bc97f6747d09e3
|
| 3 |
+
size 933959152
|
e08-X/meta/golden_render.jsonl
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{"idx": 3274, "n_tok": 959, "n_sup": 933, "masked_tail": "<|im_start|>user\nWrite a short prose story about Aleksei and Nyasha.<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n\n", "trained_head": "Aleksei stopped at Nyasha\u2019s desk, leaning in with a look of focused urgency. \"Nyasha, do you have a minute? I need to set up quality checks for the translated emergency alerts. Specifically, how do I ", "trained_tail": "e a lasting impression on them. Have a great day!\"<|im_end|>"}
|
| 2 |
+
{"idx": 3983, "n_tok": 1179, "n_sup": 1155, "masked_tail": "<|im_start|>user\nWrite a short prose story about Orion and Kady.<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n\n", "trained_head": "Orion leaned against the wooden fence, the humid afternoon air thick with the scent of cut grass and plastic decorations. He watched Kady adjust a string of bright, artificial parrots hanging from the", "trained_tail": "s. Have a wonderful time with the little pirates!\"<|im_end|>"}
|
| 3 |
+
{"idx": 4926, "n_tok": 1188, "n_sup": 1163, "masked_tail": "<|im_start|>user\nWrite a short prose story about Gage and Citra.<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n\n", "trained_head": "Gage stood before the counter, holding a bundle of pristine white cotton laces. He looked tired, his eyes fixed on Citra behind the register. \"Hi, I need to dye these cotton laces to a specific navy s", "trained_tail": ", and feel free to stop by just to chat sometime!\"<|im_end|>"}
|
e08-X/meta/train_log.jsonl
ADDED
|
@@ -0,0 +1,245 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{"step": 1, "loss": 1.3046841280790455, "story_loss": 1.3046841280790455, "grad_norm": 0.46804749965667725, "lr": 5e-05, "elapsed": 24.3, "tokens": 32437, "story_tokens": 32437, "peak_gb": 71.7}
|
| 2 |
+
{"step": 2, "loss": 1.2502072879714339, "story_loss": 1.2502072879714339, "grad_norm": 0.3877531886100769, "lr": 0.0001, "elapsed": 47.5, "tokens": 33475, "story_tokens": 33475, "peak_gb": 73.8}
|
| 3 |
+
{"step": 3, "loss": 1.14927384974747, "story_loss": 1.14927384974747, "grad_norm": 0.3386770784854889, "lr": 0.00015, "elapsed": 70.6, "tokens": 33548, "story_tokens": 33548, "peak_gb": 73.9}
|
| 4 |
+
{"step": 4, "loss": 1.1280570755824368, "story_loss": 1.1280570755824368, "grad_norm": 0.4057665169239044, "lr": 0.0002, "elapsed": 92.9, "tokens": 31747, "story_tokens": 31747, "peak_gb": 74.1}
|
| 5 |
+
{"step": 5, "loss": 1.074979091367702, "story_loss": 1.074979091367702, "grad_norm": 0.27409496903419495, "lr": 0.00025, "elapsed": 114.4, "tokens": 31515, "story_tokens": 31515, "peak_gb": 74.1}
|
| 6 |
+
{"step": 6, "loss": 1.0265632479143874, "story_loss": 1.0265632479143874, "grad_norm": 0.23989763855934143, "lr": 0.0003, "elapsed": 137.4, "tokens": 33818, "story_tokens": 33818, "peak_gb": 74.1}
|
| 7 |
+
{"step": 7, "loss": 0.9819555580334045, "story_loss": 0.9819555580334045, "grad_norm": 0.19847972691059113, "lr": 0.00035, "elapsed": 160.7, "tokens": 33824, "story_tokens": 33824, "peak_gb": 74.1}
|
| 8 |
+
{"step": 8, "loss": 0.9512067703015954, "story_loss": 0.9512067703015954, "grad_norm": 0.18325573205947876, "lr": 0.0004, "elapsed": 183.5, "tokens": 32813, "story_tokens": 32813, "peak_gb": 74.1}
|
| 9 |
+
{"step": 9, "loss": 0.9315901136499537, "story_loss": 0.9315901136499537, "grad_norm": 0.19253170490264893, "lr": 0.00045000000000000004, "elapsed": 206.8, "tokens": 33447, "story_tokens": 33447, "peak_gb": 74.4}
|
| 10 |
+
{"step": 10, "loss": 0.9588090527288432, "story_loss": 0.9588090527288432, "grad_norm": 0.19540515542030334, "lr": 0.0005, "elapsed": 228.3, "tokens": 30011, "story_tokens": 30011, "peak_gb": 74.4}
|
| 11 |
+
{"step": 11, "loss": 0.8981341643198458, "story_loss": 0.8981341643198458, "grad_norm": 0.17940689623355865, "lr": 0.0004999776608025946, "elapsed": 252.5, "tokens": 33667, "story_tokens": 33667, "peak_gb": 75.1}
|
| 12 |
+
{"step": 12, "loss": 0.90111880953177, "story_loss": 0.90111880953177, "grad_norm": 0.17183659970760345, "lr": 0.000499910647202696, "elapsed": 275.6, "tokens": 33648, "story_tokens": 33648, "peak_gb": 75.1}
|
| 13 |
+
{"step": 13, "loss": 0.8930149017349791, "story_loss": 0.8930149017349791, "grad_norm": 0.16445888578891754, "lr": 0.0004997989711765446, "elapsed": 298.6, "tokens": 32821, "story_tokens": 32821, "peak_gb": 75.1}
|
| 14 |
+
{"step": 14, "loss": 0.8733283881471083, "story_loss": 0.8733283881471083, "grad_norm": 0.1577596515417099, "lr": 0.0004996426526821629, "elapsed": 320.4, "tokens": 30687, "story_tokens": 30687, "peak_gb": 75.1}
|
| 15 |
+
{"step": 15, "loss": 0.9091314134863347, "story_loss": 0.9091314134863347, "grad_norm": 0.17788711190223694, "lr": 0.0004994417196557883, "elapsed": 345.8, "tokens": 35336, "story_tokens": 35336, "peak_gb": 75.3}
|
| 16 |
+
{"step": 16, "loss": 0.8932636514655241, "story_loss": 0.8932636514655241, "grad_norm": 0.16243809461593628, "lr": 0.0004991962080068813, "elapsed": 368.9, "tokens": 31580, "story_tokens": 31580, "peak_gb": 75.3}
|
| 17 |
+
{"step": 17, "loss": 0.893093231766869, "story_loss": 0.893093231766869, "grad_norm": 0.16129733622074127, "lr": 0.0004989061616117073, "elapsed": 392.1, "tokens": 33501, "story_tokens": 33501, "peak_gb": 75.3}
|
| 18 |
+
{"step": 18, "loss": 0.88520358101738, "story_loss": 0.88520358101738, "grad_norm": 0.16699832677841187, "lr": 0.0004985716323054959, "elapsed": 413.5, "tokens": 30495, "story_tokens": 30495, "peak_gb": 75.3}
|
| 19 |
+
{"step": 19, "loss": 0.8905891583396179, "story_loss": 0.8905891583396179, "grad_norm": 0.16740600764751434, "lr": 0.0004981926798731766, "elapsed": 435.9, "tokens": 31824, "story_tokens": 31824, "peak_gb": 75.3}
|
| 20 |
+
{"step": 20, "loss": 0.8700962900277815, "story_loss": 0.8700962900277815, "grad_norm": 0.1751105785369873, "lr": 0.000497769372038695, "elapsed": 459.5, "tokens": 33588, "story_tokens": 33588, "peak_gb": 75.3}
|
| 21 |
+
{"step": 21, "loss": 0.8816975491483894, "story_loss": 0.8816975491483894, "grad_norm": 0.18586720526218414, "lr": 0.0004973017844529094, "elapsed": 481.1, "tokens": 30035, "story_tokens": 30035, "peak_gb": 75.3}
|
| 22 |
+
{"step": 22, "loss": 0.8806317723299808, "story_loss": 0.8806317723299808, "grad_norm": 0.1617441475391388, "lr": 0.0004967900006800708, "elapsed": 503.9, "tokens": 32625, "story_tokens": 32625, "peak_gb": 75.3}
|
| 23 |
+
{"step": 23, "loss": 0.8920214739625799, "story_loss": 0.8920214739625799, "grad_norm": 0.1740923672914505, "lr": 0.000496234112182889, "elapsed": 527.0, "tokens": 31888, "story_tokens": 31888, "peak_gb": 75.3}
|
| 24 |
+
{"step": 24, "loss": 0.8629563913043672, "story_loss": 0.8629563913043672, "grad_norm": 0.1559552103281021, "lr": 0.000495634218306187, "elapsed": 551.7, "tokens": 35028, "story_tokens": 35028, "peak_gb": 75.7}
|
| 25 |
+
{"step": 25, "loss": 0.8737401819958787, "story_loss": 0.8737401819958787, "grad_norm": 0.17367024719715118, "lr": 0.0004949904262591467, "elapsed": 574.0, "tokens": 31695, "story_tokens": 31695, "peak_gb": 75.7}
|
| 26 |
+
{"step": 26, "loss": 0.8906031215487513, "story_loss": 0.8906031215487513, "grad_norm": 0.153099924325943, "lr": 0.0004943028510961491, "elapsed": 597.1, "tokens": 33276, "story_tokens": 33276, "peak_gb": 75.7}
|
| 27 |
+
{"step": 27, "loss": 0.8524379744901958, "story_loss": 0.8524379744901958, "grad_norm": 0.15194234251976013, "lr": 0.0004935716156962127, "elapsed": 621.5, "tokens": 35087, "story_tokens": 35087, "peak_gb": 75.7}
|
| 28 |
+
{"step": 28, "loss": 0.8878048660203177, "story_loss": 0.8878048660203177, "grad_norm": 0.1636548936367035, "lr": 0.000492796850741033, "elapsed": 643.9, "tokens": 31881, "story_tokens": 31881, "peak_gb": 75.7}
|
| 29 |
+
{"step": 29, "loss": 0.8684024443720441, "story_loss": 0.8684024443720441, "grad_norm": 0.16571184992790222, "lr": 0.0004919786946916281, "elapsed": 666.7, "tokens": 33619, "story_tokens": 33619, "peak_gb": 75.7}
|
| 30 |
+
{"step": 30, "loss": 0.8715655608895647, "story_loss": 0.8715655608895647, "grad_norm": 0.16513952612876892, "lr": 0.0004911172937635942, "elapsed": 689.4, "tokens": 33085, "story_tokens": 33085, "peak_gb": 75.7}
|
| 31 |
+
{"step": 31, "loss": 0.8779685444778779, "story_loss": 0.8779685444778779, "grad_norm": 0.1799987256526947, "lr": 0.0004902128019009741, "elapsed": 712.6, "tokens": 33784, "story_tokens": 33784, "peak_gb": 75.7}
|
| 32 |
+
{"step": 32, "loss": 0.8564398696909327, "story_loss": 0.8564398696909327, "grad_norm": 0.21391652524471283, "lr": 0.000489265380748746, "elapsed": 735.7, "tokens": 33866, "story_tokens": 33866, "peak_gb": 75.7}
|
| 33 |
+
{"step": 33, "loss": 0.8548055448512875, "story_loss": 0.8548055448512875, "grad_norm": 0.16118204593658447, "lr": 0.0004882751996239352, "elapsed": 758.0, "tokens": 32235, "story_tokens": 32235, "peak_gb": 75.7}
|
| 34 |
+
{"step": 34, "loss": 0.8615341769238962, "story_loss": 0.8615341769238962, "grad_norm": 0.15051710605621338, "lr": 0.0004872424354853545, "elapsed": 780.2, "tokens": 32863, "story_tokens": 32863, "peak_gb": 75.7}
|
| 35 |
+
{"step": 35, "loss": 0.8599661610693552, "story_loss": 0.8599661610693552, "grad_norm": 0.15884073078632355, "lr": 0.0004861672729019797, "elapsed": 803.0, "tokens": 33287, "story_tokens": 33287, "peak_gb": 75.7}
|
| 36 |
+
{"step": 36, "loss": 0.8873689477575556, "story_loss": 0.8873689477575556, "grad_norm": 0.16790401935577393, "lr": 0.0004850499040199643, "elapsed": 825.2, "tokens": 32112, "story_tokens": 32112, "peak_gb": 75.7}
|
| 37 |
+
{"step": 37, "loss": 0.8503342061405224, "story_loss": 0.8503342061405224, "grad_norm": 0.1606678068637848, "lr": 0.0004838905285283005, "elapsed": 847.0, "tokens": 31970, "story_tokens": 31970, "peak_gb": 75.7}
|
| 38 |
+
{"step": 38, "loss": 0.8798179209544558, "story_loss": 0.8798179209544558, "grad_norm": 0.1608157604932785, "lr": 0.00048268935362313215, "elapsed": 869.1, "tokens": 30926, "story_tokens": 30926, "peak_gb": 75.7}
|
| 39 |
+
{"step": 39, "loss": 0.8322242998307751, "story_loss": 0.8322242998307751, "grad_norm": 0.16018801927566528, "lr": 0.00048144659397072586, "elapsed": 890.8, "tokens": 31744, "story_tokens": 31744, "peak_gb": 75.7}
|
| 40 |
+
{"step": 40, "loss": 0.8534491379016884, "story_loss": 0.8534491379016884, "grad_norm": 0.16017743945121765, "lr": 0.0004801624716691072, "elapsed": 913.1, "tokens": 32130, "story_tokens": 32130, "peak_gb": 75.7}
|
| 41 |
+
{"step": 41, "loss": 0.8530367884958152, "story_loss": 0.8530367884958152, "grad_norm": 0.16044792532920837, "lr": 0.00047883721620836894, "elapsed": 935.2, "tokens": 32260, "story_tokens": 32260, "peak_gb": 75.7}
|
| 42 |
+
{"step": 42, "loss": 0.8562516472101837, "story_loss": 0.8562516472101837, "grad_norm": 0.1557490974664688, "lr": 0.0004774710644296578, "elapsed": 957.2, "tokens": 32044, "story_tokens": 32044, "peak_gb": 75.7}
|
| 43 |
+
{"step": 43, "loss": 0.8739799930599995, "story_loss": 0.8739799930599995, "grad_norm": 0.16324467957019806, "lr": 0.00047606426048284813, "elapsed": 979.0, "tokens": 31644, "story_tokens": 31644, "peak_gb": 75.7}
|
| 44 |
+
{"step": 44, "loss": 0.8549362388092957, "story_loss": 0.8549362388092957, "grad_norm": 0.1639941781759262, "lr": 0.00047461705578290833, "elapsed": 1002.0, "tokens": 32749, "story_tokens": 32749, "peak_gb": 75.7}
|
| 45 |
+
{"step": 45, "loss": 0.8470821193178176, "story_loss": 0.8470821193178176, "grad_norm": 0.1586255580186844, "lr": 0.0004731297089649703, "elapsed": 1026.0, "tokens": 35107, "story_tokens": 35107, "peak_gb": 75.7}
|
| 46 |
+
{"step": 46, "loss": 0.8572205503729649, "story_loss": 0.8572205503729649, "grad_norm": 0.14958199858665466, "lr": 0.0004716024858381075, "elapsed": 1050.1, "tokens": 34395, "story_tokens": 34395, "peak_gb": 75.7}
|
| 47 |
+
{"step": 47, "loss": 0.8586750319958701, "story_loss": 0.8586750319958701, "grad_norm": 0.1535910665988922, "lr": 0.00047003565933783123, "elapsed": 1073.1, "tokens": 33476, "story_tokens": 33476, "peak_gb": 75.7}
|
| 48 |
+
{"step": 48, "loss": 0.8649977984198376, "story_loss": 0.8649977984198376, "grad_norm": 0.15822404623031616, "lr": 0.0004684295094773134, "elapsed": 1095.1, "tokens": 31405, "story_tokens": 31405, "peak_gb": 75.7}
|
| 49 |
+
{"step": 49, "loss": 0.8335699358494189, "story_loss": 0.8335699358494189, "grad_norm": 0.1518990695476532, "lr": 0.00046678432329734434, "elapsed": 1118.4, "tokens": 33208, "story_tokens": 33208, "peak_gb": 75.7}
|
| 50 |
+
{"step": 50, "loss": 0.8416399894107329, "story_loss": 0.8416399894107329, "grad_norm": 0.15639618039131165, "lr": 0.00046510039481503486, "elapsed": 1141.3, "tokens": 33495, "story_tokens": 33495, "peak_gb": 75.7}
|
| 51 |
+
{"step": 51, "loss": 0.8559130458716108, "story_loss": 0.8559130458716108, "grad_norm": 0.17454658448696136, "lr": 0.00046337802497127117, "elapsed": 1165.0, "tokens": 33636, "story_tokens": 33636, "peak_gb": 75.7}
|
| 52 |
+
{"step": 52, "loss": 0.8845690500189277, "story_loss": 0.8845690500189277, "grad_norm": 0.16735044121742249, "lr": 0.00046161752157693284, "elapsed": 1187.5, "tokens": 30709, "story_tokens": 30709, "peak_gb": 75.7}
|
| 53 |
+
{"step": 53, "loss": 0.8557027294425427, "story_loss": 0.8557027294425427, "grad_norm": 0.16556748747825623, "lr": 0.0004598191992578828, "elapsed": 1209.1, "tokens": 30490, "story_tokens": 30490, "peak_gb": 75.7}
|
| 54 |
+
{"step": 54, "loss": 0.8802292324305335, "story_loss": 0.8802292324305335, "grad_norm": 0.17144368588924408, "lr": 0.00045798337939873923, "elapsed": 1231.2, "tokens": 31580, "story_tokens": 31580, "peak_gb": 75.7}
|
| 55 |
+
{"step": 55, "loss": 0.8357832025936017, "story_loss": 0.8357832025936017, "grad_norm": 0.16020138561725616, "lr": 0.0004561103900854401, "elapsed": 1253.7, "tokens": 33079, "story_tokens": 33079, "peak_gb": 75.7}
|
| 56 |
+
{"step": 56, "loss": 0.841932307057246, "story_loss": 0.841932307057246, "grad_norm": 0.16307440400123596, "lr": 0.0004542005660466094, "elapsed": 1274.2, "tokens": 29061, "story_tokens": 29061, "peak_gb": 75.7}
|
| 57 |
+
{"step": 57, "loss": 0.8761317482180779, "story_loss": 0.8761317482180779, "grad_norm": 0.1777096539735794, "lr": 0.0004522542485937369, "elapsed": 1297.8, "tokens": 33198, "story_tokens": 33198, "peak_gb": 75.7}
|
| 58 |
+
{"step": 58, "loss": 0.8762882391686183, "story_loss": 0.8762882391686183, "grad_norm": 0.1726435273885727, "lr": 0.0004502717855601809, "elapsed": 1320.1, "tokens": 32025, "story_tokens": 32025, "peak_gb": 75.7}
|
| 59 |
+
{"step": 59, "loss": 0.8545068289653737, "story_loss": 0.8545068289653737, "grad_norm": 0.16315919160842896, "lr": 0.0004482535312390058, "elapsed": 1343.3, "tokens": 33616, "story_tokens": 33616, "peak_gb": 75.7}
|
| 60 |
+
{"step": 60, "loss": 0.8264630949231043, "story_loss": 0.8264630949231043, "grad_norm": 0.15403491258621216, "lr": 0.00044619984631966527, "elapsed": 1365.7, "tokens": 33365, "story_tokens": 33365, "peak_gb": 75.7}
|
| 61 |
+
{"step": 61, "loss": 0.8629454999360204, "story_loss": 0.8629454999360204, "grad_norm": 0.17757681012153625, "lr": 0.0004441110978235418, "elapsed": 1386.2, "tokens": 29062, "story_tokens": 29062, "peak_gb": 75.7}
|
| 62 |
+
{"step": 62, "loss": 0.8675667436432776, "story_loss": 0.8675667436432776, "grad_norm": 0.1752307116985321, "lr": 0.0004419876590383554, "elapsed": 1409.0, "tokens": 33176, "story_tokens": 33176, "peak_gb": 75.7}
|
| 63 |
+
{"step": 63, "loss": 0.8355087629711355, "story_loss": 0.8355087629711355, "grad_norm": 0.16358071565628052, "lr": 0.00043982990945145146, "elapsed": 1430.4, "tokens": 31440, "story_tokens": 31440, "peak_gb": 75.7}
|
| 64 |
+
{"step": 64, "loss": 0.8714506361219618, "story_loss": 0.8714506361219618, "grad_norm": 0.19213375449180603, "lr": 0.0004376382346819819, "elapsed": 1452.7, "tokens": 32256, "story_tokens": 32256, "peak_gb": 75.7}
|
| 65 |
+
{"step": 65, "loss": 0.854034780215176, "story_loss": 0.854034780215176, "grad_norm": 0.15678910911083221, "lr": 0.00043541302641198946, "elapsed": 1474.5, "tokens": 31512, "story_tokens": 31512, "peak_gb": 75.7}
|
| 66 |
+
{"step": 66, "loss": 0.8567734749340555, "story_loss": 0.8567734749340555, "grad_norm": 0.16032655537128448, "lr": 0.00043315468231640834, "elapsed": 1496.8, "tokens": 30992, "story_tokens": 30992, "peak_gb": 75.7}
|
| 67 |
+
{"step": 67, "loss": 0.8582069531896512, "story_loss": 0.8582069531896512, "grad_norm": 0.16745062172412872, "lr": 0.00043086360599199516, "elapsed": 1518.8, "tokens": 32627, "story_tokens": 32627, "peak_gb": 75.7}
|
| 68 |
+
{"step": 68, "loss": 0.8436406008959147, "story_loss": 0.8436406008959147, "grad_norm": 0.16192638874053955, "lr": 0.0004285402068852002, "elapsed": 1540.6, "tokens": 30953, "story_tokens": 30953, "peak_gb": 75.7}
|
| 69 |
+
{"step": 69, "loss": 0.8481588491586538, "story_loss": 0.8481588491586538, "grad_norm": 0.17334149777889252, "lr": 0.00042618490021899383, "elapsed": 1563.1, "tokens": 32500, "story_tokens": 32500, "peak_gb": 75.7}
|
| 70 |
+
{"step": 70, "loss": 0.8414483617665817, "story_loss": 0.8414483617665817, "grad_norm": 0.15797406435012817, "lr": 0.00042379810691866064, "elapsed": 1584.9, "tokens": 30625, "story_tokens": 30625, "peak_gb": 75.7}
|
| 71 |
+
{"step": 71, "loss": 0.8686579202868275, "story_loss": 0.8686579202868275, "grad_norm": 0.15969130396842957, "lr": 0.00042138025353657407, "elapsed": 1606.2, "tokens": 31040, "story_tokens": 31040, "peak_gb": 75.7}
|
| 72 |
+
{"step": 72, "loss": 0.8721247464418411, "story_loss": 0.8721247464418411, "grad_norm": 0.16760914027690887, "lr": 0.00041893177217596633, "elapsed": 1629.3, "tokens": 32768, "story_tokens": 32768, "peak_gb": 75.7}
|
| 73 |
+
{"step": 73, "loss": 0.8703095144931727, "story_loss": 0.8703095144931727, "grad_norm": 0.17372889816761017, "lr": 0.0004164531004137049, "elapsed": 1651.6, "tokens": 31876, "story_tokens": 31876, "peak_gb": 75.7}
|
| 74 |
+
{"step": 74, "loss": 0.8238548797171662, "story_loss": 0.8238548797171662, "grad_norm": 0.15890009701251984, "lr": 0.0004139446812220924, "elapsed": 1673.9, "tokens": 31865, "story_tokens": 31865, "peak_gb": 75.7}
|
| 75 |
+
{"step": 75, "loss": 0.8368611494806047, "story_loss": 0.8368611494806047, "grad_norm": 0.16228340566158295, "lr": 0.0004114069628897006, "elapsed": 1695.6, "tokens": 31509, "story_tokens": 31509, "peak_gb": 75.7}
|
| 76 |
+
{"step": 76, "loss": 0.8196981269089275, "story_loss": 0.8196981269089275, "grad_norm": 0.16671377420425415, "lr": 0.0004088403989412559, "elapsed": 1717.3, "tokens": 31022, "story_tokens": 31022, "peak_gb": 75.7}
|
| 77 |
+
{"step": 77, "loss": 0.8597176255607492, "story_loss": 0.8597176255607492, "grad_norm": 0.15578745305538177, "lr": 0.00040624544805658794, "elapsed": 1739.5, "tokens": 31877, "story_tokens": 31877, "peak_gb": 75.7}
|
| 78 |
+
{"step": 78, "loss": 0.872345644340383, "story_loss": 0.872345644340383, "grad_norm": 0.15637312829494476, "lr": 0.00040362257398865713, "elapsed": 1761.6, "tokens": 32029, "story_tokens": 32029, "peak_gb": 75.7}
|
| 79 |
+
{"step": 79, "loss": 0.8695877427906405, "story_loss": 0.8695877427906405, "grad_norm": 0.15303386747837067, "lr": 0.00040097224548067613, "elapsed": 1784.1, "tokens": 33095, "story_tokens": 33095, "peak_gb": 75.7}
|
| 80 |
+
{"step": 80, "loss": 0.8283797975265539, "story_loss": 0.8283797975265539, "grad_norm": 0.15925215184688568, "lr": 0.0003982949361823388, "elapsed": 1806.4, "tokens": 31775, "story_tokens": 31775, "peak_gb": 75.7}
|
| 81 |
+
{"step": 81, "loss": 0.8197646103101304, "story_loss": 0.8197646103101304, "grad_norm": 0.1513034850358963, "lr": 0.0003955911245651726, "elapsed": 1828.6, "tokens": 31914, "story_tokens": 31914, "peak_gb": 75.7}
|
| 82 |
+
{"step": 82, "loss": 0.8433085546219451, "story_loss": 0.8433085546219451, "grad_norm": 0.17267578840255737, "lr": 0.0003928612938370292, "elapsed": 1849.0, "tokens": 28602, "story_tokens": 28602, "peak_gb": 75.7}
|
| 83 |
+
{"step": 83, "loss": 0.830074157310936, "story_loss": 0.830074157310936, "grad_norm": 0.15603546798229218, "lr": 0.00039010593185572867, "elapsed": 1870.9, "tokens": 32489, "story_tokens": 32489, "peak_gb": 75.7}
|
| 84 |
+
{"step": 84, "loss": 0.8577751857499674, "story_loss": 0.8577751857499674, "grad_norm": 0.16454951465129852, "lr": 0.00038732553104187296, "elapsed": 1892.3, "tokens": 30678, "story_tokens": 30678, "peak_gb": 75.7}
|
| 85 |
+
{"step": 85, "loss": 0.8512087629595474, "story_loss": 0.8512087629595474, "grad_norm": 0.16404418647289276, "lr": 0.0003845205882908432, "elapsed": 1914.1, "tokens": 32294, "story_tokens": 32294, "peak_gb": 75.7}
|
| 86 |
+
{"step": 86, "loss": 0.8707165060778452, "story_loss": 0.8707165060778452, "grad_norm": 0.15852850675582886, "lr": 0.0003816916048839979, "elapsed": 1936.2, "tokens": 32783, "story_tokens": 32783, "peak_gb": 75.7}
|
| 87 |
+
{"step": 87, "loss": 0.8587634781937008, "story_loss": 0.8587634781937008, "grad_norm": 0.1654473841190338, "lr": 0.0003788390863990875, "elapsed": 1957.8, "tokens": 30772, "story_tokens": 30772, "peak_gb": 75.7}
|
| 88 |
+
{"step": 88, "loss": 0.8238735490832338, "story_loss": 0.8238735490832338, "grad_norm": 0.15465591847896576, "lr": 0.00037596354261990007, "elapsed": 1979.8, "tokens": 32676, "story_tokens": 32676, "peak_gb": 75.7}
|
| 89 |
+
{"step": 89, "loss": 0.8519822299548501, "story_loss": 0.8519822299548501, "grad_norm": 0.16498205065727234, "lr": 0.0003730654874451569, "elapsed": 2002.4, "tokens": 33278, "story_tokens": 33278, "peak_gb": 75.7}
|
| 90 |
+
{"step": 90, "loss": 0.8597134061014119, "story_loss": 0.8597134061014119, "grad_norm": 0.16488830745220184, "lr": 0.00037014543879667093, "elapsed": 2024.2, "tokens": 29733, "story_tokens": 29733, "peak_gb": 75.7}
|
| 91 |
+
{"step": 91, "loss": 0.8651336425860944, "story_loss": 0.8651336425860944, "grad_norm": 0.1997545212507248, "lr": 0.0003672039185267878, "elapsed": 2045.9, "tokens": 30635, "story_tokens": 30635, "peak_gb": 75.7}
|
| 92 |
+
{"step": 92, "loss": 0.830874033203435, "story_loss": 0.830874033203435, "grad_norm": 0.16360309720039368, "lr": 0.00036424145232512333, "elapsed": 2067.6, "tokens": 31499, "story_tokens": 31499, "peak_gb": 75.7}
|
| 93 |
+
{"step": 93, "loss": 0.8466173574565737, "story_loss": 0.8466173574565737, "grad_norm": 0.16059370338916779, "lr": 0.0003612585696246158, "elapsed": 2091.7, "tokens": 34786, "story_tokens": 34786, "peak_gb": 75.7}
|
| 94 |
+
{"step": 94, "loss": 0.8442480907473978, "story_loss": 0.8442480907473978, "grad_norm": 0.16507810354232788, "lr": 0.0003582558035069091, "elapsed": 2113.7, "tokens": 31608, "story_tokens": 31608, "peak_gb": 75.7}
|
| 95 |
+
{"step": 95, "loss": 0.8286155732543604, "story_loss": 0.8286155732543604, "grad_norm": 0.16673985123634338, "lr": 0.0003552336906070838, "elapsed": 2135.4, "tokens": 31849, "story_tokens": 31849, "peak_gb": 75.7}
|
| 96 |
+
{"step": 96, "loss": 0.8669675360478886, "story_loss": 0.8669675360478886, "grad_norm": 0.16916526854038239, "lr": 0.000352192771017753, "elapsed": 2157.5, "tokens": 30956, "story_tokens": 30956, "peak_gb": 75.7}
|
| 97 |
+
{"step": 97, "loss": 0.8493272974185352, "story_loss": 0.8493272974185352, "grad_norm": 0.16506345570087433, "lr": 0.0003491335881925407, "elapsed": 2178.9, "tokens": 29825, "story_tokens": 29825, "peak_gb": 75.7}
|
| 98 |
+
{"step": 98, "loss": 0.8377196293973482, "story_loss": 0.8377196293973482, "grad_norm": 0.16030652821063995, "lr": 0.0003460566888489593, "elapsed": 2200.7, "tokens": 32413, "story_tokens": 32413, "peak_gb": 75.7}
|
| 99 |
+
{"step": 99, "loss": 0.8557590333029843, "story_loss": 0.8557590333029843, "grad_norm": 0.16061748564243317, "lr": 0.00034296262287070335, "elapsed": 2222.1, "tokens": 31783, "story_tokens": 31783, "peak_gb": 75.7}
|
| 100 |
+
{"step": 100, "loss": 0.8277105213076595, "story_loss": 0.8277105213076595, "grad_norm": 0.15598607063293457, "lr": 0.0003398519432093782, "elapsed": 2245.7, "tokens": 33765, "story_tokens": 33765, "peak_gb": 75.7}
|
| 101 |
+
{"step": 101, "loss": 0.8505527599873317, "story_loss": 0.8505527599873317, "grad_norm": 0.16056478023529053, "lr": 0.0003367252057856802, "elapsed": 2268.1, "tokens": 31723, "story_tokens": 31723, "peak_gb": 75.7}
|
| 102 |
+
{"step": 102, "loss": 0.8508792190322452, "story_loss": 0.8508792190322452, "grad_norm": 0.16480740904808044, "lr": 0.00033358296939004547, "elapsed": 2290.3, "tokens": 32939, "story_tokens": 32939, "peak_gb": 75.7}
|
| 103 |
+
{"step": 103, "loss": 0.8607517088815523, "story_loss": 0.8607517088815523, "grad_norm": 0.1609371304512024, "lr": 0.00033042579558278717, "elapsed": 2312.2, "tokens": 30867, "story_tokens": 30867, "peak_gb": 75.7}
|
| 104 |
+
{"step": 104, "loss": 0.8105260693784069, "story_loss": 0.8105260693784069, "grad_norm": 1.2105556726455688, "lr": 0.00032725424859373687, "elapsed": 2335.1, "tokens": 33572, "story_tokens": 33572, "peak_gb": 75.7}
|
| 105 |
+
{"step": 105, "loss": 0.8282781177352416, "story_loss": 0.8282781177352416, "grad_norm": 0.1583007574081421, "lr": 0.0003240688952214085, "elapsed": 2358.1, "tokens": 33176, "story_tokens": 33176, "peak_gb": 75.7}
|
| 106 |
+
{"step": 106, "loss": 0.829971289482446, "story_loss": 0.829971289482446, "grad_norm": 0.16452144086360931, "lr": 0.00032087030473170445, "elapsed": 2379.6, "tokens": 31161, "story_tokens": 31161, "peak_gb": 75.7}
|
| 107 |
+
{"step": 107, "loss": 0.8323548920126606, "story_loss": 0.8323548920126606, "grad_norm": 0.16185139119625092, "lr": 0.00031765904875617973, "elapsed": 2401.5, "tokens": 32226, "story_tokens": 32226, "peak_gb": 75.7}
|
| 108 |
+
{"step": 108, "loss": 0.870586575225479, "story_loss": 0.870586575225479, "grad_norm": 0.17773661017417908, "lr": 0.00031443570118988356, "elapsed": 2422.7, "tokens": 30948, "story_tokens": 30948, "peak_gb": 75.7}
|
| 109 |
+
{"step": 109, "loss": 0.8486549354518594, "story_loss": 0.8486549354518594, "grad_norm": 0.1583186239004135, "lr": 0.00031120083808879663, "elapsed": 2444.4, "tokens": 31286, "story_tokens": 31286, "peak_gb": 75.7}
|
| 110 |
+
{"step": 110, "loss": 0.839761358717847, "story_loss": 0.839761358717847, "grad_norm": 0.15875069797039032, "lr": 0.0003079550375668821, "elapsed": 2467.1, "tokens": 32598, "story_tokens": 32598, "peak_gb": 75.7}
|
| 111 |
+
{"step": 111, "loss": 0.845838670900643, "story_loss": 0.845838670900643, "grad_norm": 0.1660439819097519, "lr": 0.00030469887969276877, "elapsed": 2489.1, "tokens": 29667, "story_tokens": 29667, "peak_gb": 75.7}
|
| 112 |
+
{"step": 112, "loss": 0.8780140995484664, "story_loss": 0.8780140995484664, "grad_norm": 0.16652867197990417, "lr": 0.00030143294638608487, "elapsed": 2511.4, "tokens": 30848, "story_tokens": 30848, "peak_gb": 75.7}
|
| 113 |
+
{"step": 113, "loss": 0.8154400244720297, "story_loss": 0.8154400244720297, "grad_norm": 0.19803175330162048, "lr": 0.00029815782131346137, "elapsed": 2533.1, "tokens": 31672, "story_tokens": 31672, "peak_gb": 75.7}
|
| 114 |
+
{"step": 114, "loss": 0.8420583505720163, "story_loss": 0.8420583505720163, "grad_norm": 0.1564861536026001, "lr": 0.0002948740897842223, "elapsed": 2554.9, "tokens": 32970, "story_tokens": 32970, "peak_gb": 75.7}
|
| 115 |
+
{"step": 115, "loss": 0.8515159907675232, "story_loss": 0.8515159907675232, "grad_norm": 0.17401258647441864, "lr": 0.00029158233864578256, "elapsed": 2575.5, "tokens": 29732, "story_tokens": 29732, "peak_gb": 75.7}
|
| 116 |
+
{"step": 116, "loss": 0.8504402613038325, "story_loss": 0.8504402613038325, "grad_norm": 0.16334137320518494, "lr": 0.00028828315617877, "elapsed": 2597.2, "tokens": 31115, "story_tokens": 31115, "peak_gb": 75.7}
|
| 117 |
+
{"step": 117, "loss": 0.8365050984109648, "story_loss": 0.8365050984109648, "grad_norm": 0.16019028425216675, "lr": 0.0002849771319918922, "elapsed": 2619.7, "tokens": 32504, "story_tokens": 32504, "peak_gb": 75.7}
|
| 118 |
+
{"step": 118, "loss": 0.8306648913897697, "story_loss": 0.8306648913897697, "grad_norm": 0.15694484114646912, "lr": 0.00028166485691656423, "elapsed": 2642.4, "tokens": 30529, "story_tokens": 30529, "peak_gb": 75.7}
|
| 119 |
+
{"step": 119, "loss": 0.8495740521390323, "story_loss": 0.8495740521390323, "grad_norm": 0.16470122337341309, "lr": 0.00027834692290132053, "elapsed": 2664.2, "tokens": 32167, "story_tokens": 32167, "peak_gb": 75.7}
|
| 120 |
+
{"step": 120, "loss": 0.883554481053221, "story_loss": 0.883554481053221, "grad_norm": 0.1643524169921875, "lr": 0.0002750239229060246, "elapsed": 2685.8, "tokens": 31031, "story_tokens": 31031, "peak_gb": 75.7}
|
| 121 |
+
{"step": 121, "loss": 0.8353150551730627, "story_loss": 0.8353150551730627, "grad_norm": 0.15295079350471497, "lr": 0.0002716964507958994, "elapsed": 2707.9, "tokens": 33048, "story_tokens": 33048, "peak_gb": 75.7}
|
| 122 |
+
{"step": 122, "loss": 0.8275472363974865, "story_loss": 0.8275472363974865, "grad_norm": 0.1590496450662613, "lr": 0.0002683651012353955, "elapsed": 2730.9, "tokens": 32664, "story_tokens": 32664, "peak_gb": 75.7}
|
| 123 |
+
{"step": 123, "loss": 0.8490537071251296, "story_loss": 0.8490537071251296, "grad_norm": 0.18842336535453796, "lr": 0.0002650304695819168, "elapsed": 2752.9, "tokens": 32788, "story_tokens": 32788, "peak_gb": 75.7}
|
| 124 |
+
{"step": 124, "loss": 0.8275746719175735, "story_loss": 0.8275746719175735, "grad_norm": 0.15447121858596802, "lr": 0.00026169315177942135, "elapsed": 2774.0, "tokens": 30829, "story_tokens": 30829, "peak_gb": 75.7}
|
| 125 |
+
{"step": 125, "loss": 0.829877120897591, "story_loss": 0.829877120897591, "grad_norm": 0.15895220637321472, "lr": 0.0002583537442519187, "elapsed": 2795.9, "tokens": 30687, "story_tokens": 30687, "peak_gb": 75.7}
|
| 126 |
+
{"step": 126, "loss": 0.8343968551259862, "story_loss": 0.8343968551259862, "grad_norm": 0.15621864795684814, "lr": 0.00025501284379688067, "elapsed": 2819.0, "tokens": 31055, "story_tokens": 31055, "peak_gb": 75.7}
|
| 127 |
+
{"step": 127, "loss": 0.8325793092138831, "story_loss": 0.8325793092138831, "grad_norm": 0.15735705196857452, "lr": 0.0002516710474785856, "elapsed": 2841.6, "tokens": 31848, "story_tokens": 31848, "peak_gb": 75.7}
|
| 128 |
+
{"step": 128, "loss": 0.8159937717826627, "story_loss": 0.8159937717826627, "grad_norm": 0.14575611054897308, "lr": 0.0002483289525214145, "elapsed": 2865.1, "tokens": 33911, "story_tokens": 33911, "peak_gb": 75.7}
|
| 129 |
+
{"step": 129, "loss": 0.8492642775145163, "story_loss": 0.8492642775145163, "grad_norm": 0.15559333562850952, "lr": 0.00024498715620311935, "elapsed": 2888.5, "tokens": 33626, "story_tokens": 33626, "peak_gb": 75.7}
|
| 130 |
+
{"step": 130, "loss": 0.8291254062346259, "story_loss": 0.8291254062346259, "grad_norm": 0.15821845829486847, "lr": 0.00024164625574808144, "elapsed": 2909.8, "tokens": 31506, "story_tokens": 31506, "peak_gb": 75.7}
|
| 131 |
+
{"step": 131, "loss": 0.8464123583052868, "story_loss": 0.8464123583052868, "grad_norm": 0.1612735539674759, "lr": 0.00023830684822057877, "elapsed": 2931.1, "tokens": 29564, "story_tokens": 29564, "peak_gb": 75.7}
|
| 132 |
+
{"step": 132, "loss": 0.8101410765238884, "story_loss": 0.8101410765238884, "grad_norm": 0.15175843238830566, "lr": 0.00023496953041808325, "elapsed": 2953.5, "tokens": 31443, "story_tokens": 31443, "peak_gb": 75.7}
|
| 133 |
+
{"step": 133, "loss": 0.8184233773191258, "story_loss": 0.8184233773191258, "grad_norm": 0.14749573171138763, "lr": 0.0002316348987646045, "elapsed": 2976.5, "tokens": 33694, "story_tokens": 33694, "peak_gb": 75.7}
|
| 134 |
+
{"step": 134, "loss": 0.8252617690368929, "story_loss": 0.8252617690368929, "grad_norm": 0.15034066140651703, "lr": 0.00022830354920410064, "elapsed": 2999.9, "tokens": 34878, "story_tokens": 34878, "peak_gb": 75.7}
|
| 135 |
+
{"step": 135, "loss": 0.8211480115503168, "story_loss": 0.8211480115503168, "grad_norm": 0.14852482080459595, "lr": 0.0002249760770939754, "elapsed": 3022.0, "tokens": 32909, "story_tokens": 32909, "peak_gb": 75.7}
|
| 136 |
+
{"step": 136, "loss": 0.8191034969732894, "story_loss": 0.8191034969732894, "grad_norm": 0.15393581986427307, "lr": 0.0002216530770986795, "elapsed": 3045.4, "tokens": 34242, "story_tokens": 34242, "peak_gb": 75.7}
|
| 137 |
+
{"step": 137, "loss": 0.8547476630025602, "story_loss": 0.8547476630025602, "grad_norm": 0.1608327478170395, "lr": 0.0002183351430834358, "elapsed": 3067.6, "tokens": 31296, "story_tokens": 31296, "peak_gb": 75.7}
|
| 138 |
+
{"step": 138, "loss": 0.813835456057423, "story_loss": 0.813835456057423, "grad_norm": 0.15124553442001343, "lr": 0.0002150228680081079, "elapsed": 3089.1, "tokens": 30345, "story_tokens": 30345, "peak_gb": 75.7}
|
| 139 |
+
{"step": 139, "loss": 0.8398912491893809, "story_loss": 0.8398912491893809, "grad_norm": 0.14918120205402374, "lr": 0.00021171684382123, "elapsed": 3110.5, "tokens": 30937, "story_tokens": 30937, "peak_gb": 75.7}
|
| 140 |
+
{"step": 140, "loss": 0.8138795867224982, "story_loss": 0.8138795867224982, "grad_norm": 0.15511126816272736, "lr": 0.0002084176613542175, "elapsed": 3132.2, "tokens": 29598, "story_tokens": 29598, "peak_gb": 75.7}
|
| 141 |
+
{"step": 141, "loss": 0.8124172583109364, "story_loss": 0.8124172583109364, "grad_norm": 0.1526007503271103, "lr": 0.00020512591021577773, "elapsed": 3154.7, "tokens": 32696, "story_tokens": 32696, "peak_gb": 75.7}
|
| 142 |
+
{"step": 142, "loss": 0.840250139397053, "story_loss": 0.840250139397053, "grad_norm": 0.15627194941043854, "lr": 0.00020184217868653867, "elapsed": 3176.6, "tokens": 31175, "story_tokens": 31175, "peak_gb": 75.7}
|
| 143 |
+
{"step": 143, "loss": 0.8055584173056268, "story_loss": 0.8055584173056268, "grad_norm": 0.1502387523651123, "lr": 0.0001985670536139151, "elapsed": 3199.2, "tokens": 33518, "story_tokens": 33518, "peak_gb": 75.7}
|
| 144 |
+
{"step": 144, "loss": 0.8192421456115013, "story_loss": 0.8192421456115013, "grad_norm": 0.14901162683963776, "lr": 0.0001953011203072312, "elapsed": 3222.8, "tokens": 33105, "story_tokens": 33105, "peak_gb": 75.7}
|
| 145 |
+
{"step": 145, "loss": 0.8348507263403345, "story_loss": 0.8348507263403345, "grad_norm": 0.15240685641765594, "lr": 0.00019204496243311792, "elapsed": 3244.7, "tokens": 32774, "story_tokens": 32774, "peak_gb": 75.7}
|
| 146 |
+
{"step": 146, "loss": 0.8366107397819595, "story_loss": 0.8366107397819595, "grad_norm": 0.1565040946006775, "lr": 0.00018879916191120349, "elapsed": 3267.7, "tokens": 33695, "story_tokens": 33695, "peak_gb": 75.7}
|
| 147 |
+
{"step": 147, "loss": 0.8394396675519953, "story_loss": 0.8394396675519953, "grad_norm": 0.15370352566242218, "lr": 0.00018556429881011656, "elapsed": 3290.1, "tokens": 31962, "story_tokens": 31962, "peak_gb": 75.7}
|
| 148 |
+
{"step": 148, "loss": 0.8108637364545674, "story_loss": 0.8108637364545674, "grad_norm": 0.14355990290641785, "lr": 0.0001823409512438203, "elapsed": 3312.3, "tokens": 32639, "story_tokens": 32639, "peak_gb": 75.7}
|
| 149 |
+
{"step": 149, "loss": 0.7868177394904788, "story_loss": 0.7868177394904788, "grad_norm": 0.14252309501171112, "lr": 0.00017912969526829559, "elapsed": 3335.5, "tokens": 33767, "story_tokens": 33767, "peak_gb": 75.7}
|
| 150 |
+
{"step": 150, "loss": 0.8186913731300032, "story_loss": 0.8186913731300032, "grad_norm": 0.14639951288700104, "lr": 0.00017593110477859153, "elapsed": 3357.6, "tokens": 32729, "story_tokens": 32729, "peak_gb": 75.7}
|
| 151 |
+
{"step": 151, "loss": 0.8313231148575942, "story_loss": 0.8313231148575942, "grad_norm": 0.14097625017166138, "lr": 0.00017274575140626317, "elapsed": 3380.2, "tokens": 33746, "story_tokens": 33746, "peak_gb": 75.7}
|
| 152 |
+
{"step": 152, "loss": 0.8453209651584084, "story_loss": 0.8453209651584084, "grad_norm": 0.1461024135351181, "lr": 0.00016957420441721284, "elapsed": 3402.5, "tokens": 32215, "story_tokens": 32215, "peak_gb": 75.7}
|
| 153 |
+
{"step": 153, "loss": 0.8089762901070764, "story_loss": 0.8089762901070764, "grad_norm": 0.1504165381193161, "lr": 0.00016641703060995457, "elapsed": 3424.5, "tokens": 31861, "story_tokens": 31861, "peak_gb": 75.7}
|
| 154 |
+
{"step": 154, "loss": 0.8037520629736017, "story_loss": 0.8037520629736017, "grad_norm": 0.14524143934249878, "lr": 0.00016327479421431983, "elapsed": 3446.5, "tokens": 32431, "story_tokens": 32431, "peak_gb": 75.7}
|
| 155 |
+
{"step": 155, "loss": 0.8311566219791167, "story_loss": 0.8311566219791167, "grad_norm": 0.15228278934955597, "lr": 0.00016014805679062183, "elapsed": 3468.4, "tokens": 31215, "story_tokens": 31215, "peak_gb": 75.7}
|
| 156 |
+
{"step": 156, "loss": 0.8050160410785137, "story_loss": 0.8050160410785137, "grad_norm": 0.16282710433006287, "lr": 0.0001570373771292967, "elapsed": 3490.5, "tokens": 32765, "story_tokens": 32765, "peak_gb": 75.7}
|
| 157 |
+
{"step": 157, "loss": 0.8241026180491728, "story_loss": 0.8241026180491728, "grad_norm": 0.14229966700077057, "lr": 0.00015394331115104075, "elapsed": 3513.2, "tokens": 34000, "story_tokens": 34000, "peak_gb": 75.7}
|
| 158 |
+
{"step": 158, "loss": 0.8286282841460503, "story_loss": 0.8286282841460503, "grad_norm": 0.13848601281642914, "lr": 0.00015086641180745932, "elapsed": 3536.6, "tokens": 34287, "story_tokens": 34287, "peak_gb": 75.7}
|
| 159 |
+
{"step": 159, "loss": 0.8157306638927052, "story_loss": 0.8157306638927052, "grad_norm": 0.14291471242904663, "lr": 0.00014780722898224708, "elapsed": 3558.9, "tokens": 32208, "story_tokens": 32208, "peak_gb": 75.7}
|
| 160 |
+
{"step": 160, "loss": 0.8118719294327825, "story_loss": 0.8118719294327825, "grad_norm": 0.14254094660282135, "lr": 0.0001447663093929163, "elapsed": 3580.1, "tokens": 31346, "story_tokens": 31346, "peak_gb": 75.7}
|
| 161 |
+
{"step": 161, "loss": 0.8291096576680003, "story_loss": 0.8291096576680003, "grad_norm": 0.14748747646808624, "lr": 0.00014174419649309089, "elapsed": 3602.5, "tokens": 30385, "story_tokens": 30385, "peak_gb": 75.7}
|
| 162 |
+
{"step": 162, "loss": 0.8268966276642788, "story_loss": 0.8268966276642788, "grad_norm": 0.15131282806396484, "lr": 0.00013874143037538418, "elapsed": 3622.8, "tokens": 29892, "story_tokens": 29892, "peak_gb": 75.7}
|
| 163 |
+
{"step": 163, "loss": 0.7866952254897743, "story_loss": 0.7866952254897743, "grad_norm": 0.13833530247211456, "lr": 0.0001357585476748766, "elapsed": 3646.2, "tokens": 34142, "story_tokens": 34142, "peak_gb": 75.7}
|
| 164 |
+
{"step": 164, "loss": 0.835966787812559, "story_loss": 0.835966787812559, "grad_norm": 0.14718782901763916, "lr": 0.00013279608147321223, "elapsed": 3667.8, "tokens": 31789, "story_tokens": 31789, "peak_gb": 75.7}
|
| 165 |
+
{"step": 165, "loss": 0.8152859819961041, "story_loss": 0.8152859819961041, "grad_norm": 0.14218229055404663, "lr": 0.00012985456120332905, "elapsed": 3690.4, "tokens": 34299, "story_tokens": 34299, "peak_gb": 75.7}
|
| 166 |
+
{"step": 166, "loss": 0.8094605461026674, "story_loss": 0.8094605461026674, "grad_norm": 0.13707506656646729, "lr": 0.00012693451255484312, "elapsed": 3714.9, "tokens": 35657, "story_tokens": 35657, "peak_gb": 75.7}
|
| 167 |
+
{"step": 167, "loss": 0.8067047028329684, "story_loss": 0.8067047028329684, "grad_norm": 0.15020537376403809, "lr": 0.00012403645738009997, "elapsed": 3736.9, "tokens": 29909, "story_tokens": 29909, "peak_gb": 75.7}
|
| 168 |
+
{"step": 168, "loss": 0.809056127447116, "story_loss": 0.809056127447116, "grad_norm": 0.14102008938789368, "lr": 0.00012116091360091261, "elapsed": 3759.8, "tokens": 33694, "story_tokens": 33694, "peak_gb": 75.7}
|
| 169 |
+
{"step": 169, "loss": 0.800533944361356, "story_loss": 0.800533944361356, "grad_norm": 0.148661270737648, "lr": 0.00011830839511600211, "elapsed": 3781.2, "tokens": 29775, "story_tokens": 29775, "peak_gb": 75.7}
|
| 170 |
+
{"step": 170, "loss": 0.8137898809719982, "story_loss": 0.8137898809719982, "grad_norm": 0.14710122346878052, "lr": 0.00011547941170915685, "elapsed": 3802.8, "tokens": 32239, "story_tokens": 32239, "peak_gb": 75.7}
|
| 171 |
+
{"step": 171, "loss": 0.7927383096654171, "story_loss": 0.7927383096654171, "grad_norm": 0.14206092059612274, "lr": 0.00011267446895812702, "elapsed": 3825.6, "tokens": 33680, "story_tokens": 33680, "peak_gb": 75.7}
|
| 172 |
+
{"step": 172, "loss": 0.8233360996907811, "story_loss": 0.8233360996907811, "grad_norm": 0.14349548518657684, "lr": 0.0001098940681442713, "elapsed": 3847.0, "tokens": 31430, "story_tokens": 31430, "peak_gb": 75.7}
|
| 173 |
+
{"step": 173, "loss": 0.7800776798085206, "story_loss": 0.7800776798085206, "grad_norm": 0.1393120437860489, "lr": 0.00010713870616297092, "elapsed": 3869.2, "tokens": 32794, "story_tokens": 32794, "peak_gb": 75.7}
|
| 174 |
+
{"step": 174, "loss": 0.805922310750223, "story_loss": 0.805922310750223, "grad_norm": 0.14343692362308502, "lr": 0.00010440887543482746, "elapsed": 3890.4, "tokens": 30553, "story_tokens": 30553, "peak_gb": 75.7}
|
| 175 |
+
{"step": 175, "loss": 0.8110145396598634, "story_loss": 0.8110145396598634, "grad_norm": 0.14940263330936432, "lr": 0.0001017050638176612, "elapsed": 3912.5, "tokens": 32068, "story_tokens": 32068, "peak_gb": 75.7}
|
| 176 |
+
{"step": 176, "loss": 0.8203790653119091, "story_loss": 0.8203790653119091, "grad_norm": 0.1480436474084854, "lr": 9.902775451932386e-05, "elapsed": 3934.0, "tokens": 31201, "story_tokens": 31201, "peak_gb": 75.7}
|
| 177 |
+
{"step": 177, "loss": 0.8279034006183126, "story_loss": 0.8279034006183126, "grad_norm": 0.14656497538089752, "lr": 9.637742601134286e-05, "elapsed": 3957.5, "tokens": 34767, "story_tokens": 34767, "peak_gb": 75.7}
|
| 178 |
+
{"step": 178, "loss": 0.820637374598745, "story_loss": 0.820637374598745, "grad_norm": 0.14147581160068512, "lr": 9.375455194341214e-05, "elapsed": 3980.1, "tokens": 32301, "story_tokens": 32301, "peak_gb": 75.7}
|
| 179 |
+
{"step": 179, "loss": 0.8292327341913538, "story_loss": 0.8292327341913538, "grad_norm": 0.1623590588569641, "lr": 9.11596010587441e-05, "elapsed": 4002.5, "tokens": 33182, "story_tokens": 33182, "peak_gb": 75.7}
|
| 180 |
+
{"step": 180, "loss": 0.8200843783133317, "story_loss": 0.8200843783133317, "grad_norm": 0.15050067007541656, "lr": 8.85930371102994e-05, "elapsed": 4024.9, "tokens": 32539, "story_tokens": 32539, "peak_gb": 75.7}
|
| 181 |
+
{"step": 181, "loss": 0.8115783956287661, "story_loss": 0.8115783956287661, "grad_norm": 0.1417081356048584, "lr": 8.605531877790762e-05, "elapsed": 4047.9, "tokens": 32029, "story_tokens": 32029, "peak_gb": 75.7}
|
| 182 |
+
{"step": 182, "loss": 0.8101220905261125, "story_loss": 0.8101220905261125, "grad_norm": 0.14601260423660278, "lr": 8.354689958629513e-05, "elapsed": 4069.3, "tokens": 31886, "story_tokens": 31886, "peak_gb": 75.7}
|
| 183 |
+
{"step": 183, "loss": 0.8199608370853723, "story_loss": 0.8199608370853723, "grad_norm": 0.1495514214038849, "lr": 8.106822782403376e-05, "elapsed": 4090.9, "tokens": 32288, "story_tokens": 32288, "peak_gb": 75.7}
|
| 184 |
+
{"step": 184, "loss": 0.8281473340633112, "story_loss": 0.8281473340633112, "grad_norm": 0.14343197643756866, "lr": 7.861974646342596e-05, "elapsed": 4114.3, "tokens": 33898, "story_tokens": 33898, "peak_gb": 75.7}
|
| 185 |
+
{"step": 185, "loss": 0.8224437297181056, "story_loss": 0.8224437297181056, "grad_norm": 0.15347489714622498, "lr": 7.620189308133943e-05, "elapsed": 4136.0, "tokens": 30989, "story_tokens": 30989, "peak_gb": 75.7}
|
| 186 |
+
{"step": 186, "loss": 0.7983138668365015, "story_loss": 0.7983138668365015, "grad_norm": 0.14363357424736023, "lr": 7.381509978100626e-05, "elapsed": 4158.3, "tokens": 32274, "story_tokens": 32274, "peak_gb": 75.7}
|
| 187 |
+
{"step": 187, "loss": 0.8006288822386785, "story_loss": 0.8006288822386785, "grad_norm": 0.14140218496322632, "lr": 7.145979311479986e-05, "elapsed": 4180.2, "tokens": 31886, "story_tokens": 31886, "peak_gb": 75.7}
|
| 188 |
+
{"step": 188, "loss": 0.7952802097012713, "story_loss": 0.7952802097012713, "grad_norm": 0.14849825203418732, "lr": 6.913639400800489e-05, "elapsed": 4202.2, "tokens": 32268, "story_tokens": 32268, "peak_gb": 75.7}
|
| 189 |
+
{"step": 189, "loss": 0.8216201367067117, "story_loss": 0.8216201367067117, "grad_norm": 0.1577879935503006, "lr": 6.684531768359173e-05, "elapsed": 4225.2, "tokens": 34071, "story_tokens": 34071, "peak_gb": 75.7}
|
| 190 |
+
{"step": 190, "loss": 0.791763258180346, "story_loss": 0.791763258180346, "grad_norm": 0.14214052259922028, "lr": 6.458697358801061e-05, "elapsed": 4246.0, "tokens": 29564, "story_tokens": 29564, "peak_gb": 75.7}
|
| 191 |
+
{"step": 191, "loss": 0.803321746960481, "story_loss": 0.803321746960481, "grad_norm": 0.1468520611524582, "lr": 6.236176531801813e-05, "elapsed": 4267.8, "tokens": 31129, "story_tokens": 31129, "peak_gb": 75.7}
|
| 192 |
+
{"step": 192, "loss": 0.8238503228045719, "story_loss": 0.8238503228045719, "grad_norm": 0.14412041008472443, "lr": 6.017009054854858e-05, "elapsed": 4290.3, "tokens": 33268, "story_tokens": 33268, "peak_gb": 75.7}
|
| 193 |
+
{"step": 193, "loss": 0.8128882323332577, "story_loss": 0.8128882323332577, "grad_norm": 0.14297965168952942, "lr": 5.801234096164468e-05, "elapsed": 4311.6, "tokens": 30305, "story_tokens": 30305, "peak_gb": 75.7}
|
| 194 |
+
{"step": 194, "loss": 0.8105575408440412, "story_loss": 0.8105575408440412, "grad_norm": 0.1371430605649948, "lr": 5.58889021764582e-05, "elapsed": 4334.7, "tokens": 33591, "story_tokens": 33591, "peak_gb": 75.7}
|
| 195 |
+
{"step": 195, "loss": 0.7775942885114651, "story_loss": 0.7775942885114651, "grad_norm": 0.1480807363986969, "lr": 5.3800153680334754e-05, "elapsed": 4356.5, "tokens": 32283, "story_tokens": 32283, "peak_gb": 75.7}
|
| 196 |
+
{"step": 196, "loss": 0.8154408410126633, "story_loss": 0.8154408410126633, "grad_norm": 0.14678408205509186, "lr": 5.17464687609942e-05, "elapsed": 4378.1, "tokens": 30995, "story_tokens": 30995, "peak_gb": 75.7}
|
| 197 |
+
{"step": 197, "loss": 0.8205536658662776, "story_loss": 0.8205536658662776, "grad_norm": 0.1454947292804718, "lr": 4.97282144398192e-05, "elapsed": 4400.6, "tokens": 33649, "story_tokens": 33649, "peak_gb": 75.7}
|
| 198 |
+
{"step": 198, "loss": 0.8146615131843344, "story_loss": 0.8146615131843344, "grad_norm": 0.14416785538196564, "lr": 4.7745751406263163e-05, "elapsed": 4421.8, "tokens": 30326, "story_tokens": 30326, "peak_gb": 75.7}
|
| 199 |
+
{"step": 199, "loss": 0.8111210728849647, "story_loss": 0.8111210728849647, "grad_norm": 0.14365844428539276, "lr": 4.5799433953390616e-05, "elapsed": 4443.4, "tokens": 32201, "story_tokens": 32201, "peak_gb": 75.7}
|
| 200 |
+
{"step": 200, "loss": 0.8022981598143236, "story_loss": 0.8022981598143236, "grad_norm": 0.15520331263542175, "lr": 4.388960991455998e-05, "elapsed": 4465.7, "tokens": 32045, "story_tokens": 32045, "peak_gb": 75.7}
|
| 201 |
+
{"step": 201, "loss": 0.8011120033572294, "story_loss": 0.8011120033572294, "grad_norm": 0.13698822259902954, "lr": 4.2016620601260796e-05, "elapsed": 4489.0, "tokens": 34045, "story_tokens": 34045, "peak_gb": 75.7}
|
| 202 |
+
{"step": 202, "loss": 0.8183090494833435, "story_loss": 0.8183090494833435, "grad_norm": 0.16726794838905334, "lr": 4.0180800742117244e-05, "elapsed": 4510.1, "tokens": 31174, "story_tokens": 31174, "peak_gb": 75.7}
|
| 203 |
+
{"step": 203, "loss": 0.8331565884863724, "story_loss": 0.8331565884863724, "grad_norm": 0.15113674104213715, "lr": 3.838247842306716e-05, "elapsed": 4529.7, "tokens": 27316, "story_tokens": 27316, "peak_gb": 75.7}
|
| 204 |
+
{"step": 204, "loss": 0.7858735902432163, "story_loss": 0.7858735902432163, "grad_norm": 0.13760623335838318, "lr": 3.662197502872885e-05, "elapsed": 4552.4, "tokens": 34108, "story_tokens": 34108, "peak_gb": 75.7}
|
| 205 |
+
{"step": 205, "loss": 0.8030743457681644, "story_loss": 0.8030743457681644, "grad_norm": 0.14882877469062805, "lr": 3.489960518496521e-05, "elapsed": 4575.2, "tokens": 33183, "story_tokens": 33183, "peak_gb": 75.7}
|
| 206 |
+
{"step": 206, "loss": 0.7696390397217946, "story_loss": 0.7696390397217946, "grad_norm": 0.137091264128685, "lr": 3.321567670265568e-05, "elapsed": 4597.3, "tokens": 33132, "story_tokens": 33132, "peak_gb": 75.7}
|
| 207 |
+
{"step": 207, "loss": 0.8186666221471771, "story_loss": 0.8186666221471771, "grad_norm": 0.14567837119102478, "lr": 3.157049052268662e-05, "elapsed": 4619.6, "tokens": 33503, "story_tokens": 33503, "peak_gb": 75.7}
|
| 208 |
+
{"step": 208, "loss": 0.8113220017417268, "story_loss": 0.8113220017417268, "grad_norm": 0.14098899066448212, "lr": 2.9964340662168772e-05, "elapsed": 4641.5, "tokens": 32152, "story_tokens": 32152, "peak_gb": 75.7}
|
| 209 |
+
{"step": 209, "loss": 0.8295606367919681, "story_loss": 0.8295606367919681, "grad_norm": 0.14579637348651886, "lr": 2.8397514161892484e-05, "elapsed": 4663.4, "tokens": 30410, "story_tokens": 30410, "peak_gb": 75.7}
|
| 210 |
+
{"step": 210, "loss": 0.8115724029111996, "story_loss": 0.8115724029111996, "grad_norm": 0.14148807525634766, "lr": 2.687029103502972e-05, "elapsed": 4684.9, "tokens": 30943, "story_tokens": 30943, "peak_gb": 75.7}
|
| 211 |
+
{"step": 211, "loss": 0.78828763534447, "story_loss": 0.78828763534447, "grad_norm": 0.1355678141117096, "lr": 2.5382944217091723e-05, "elapsed": 4706.8, "tokens": 32247, "story_tokens": 32247, "peak_gb": 75.7}
|
| 212 |
+
{"step": 212, "loss": 0.8155649265654682, "story_loss": 0.8155649265654682, "grad_norm": 0.14571426808834076, "lr": 2.3935739517151916e-05, "elapsed": 4727.7, "tokens": 30006, "story_tokens": 30006, "peak_gb": 75.7}
|
| 213 |
+
{"step": 213, "loss": 0.8090838540244013, "story_loss": 0.8090838540244013, "grad_norm": 0.13790543377399445, "lr": 2.2528935570342164e-05, "elapsed": 4750.0, "tokens": 32949, "story_tokens": 32949, "peak_gb": 75.7}
|
| 214 |
+
{"step": 214, "loss": 0.823634313036196, "story_loss": 0.823634313036196, "grad_norm": 0.14973825216293335, "lr": 2.1162783791631057e-05, "elapsed": 4770.5, "tokens": 29563, "story_tokens": 29563, "peak_gb": 75.7}
|
| 215 |
+
{"step": 215, "loss": 0.798296950614744, "story_loss": 0.798296950614744, "grad_norm": 0.1429433822631836, "lr": 1.9837528330892778e-05, "elapsed": 4792.3, "tokens": 30531, "story_tokens": 30531, "peak_gb": 75.7}
|
| 216 |
+
{"step": 216, "loss": 0.802471138627839, "story_loss": 0.802471138627839, "grad_norm": 0.14824573695659637, "lr": 1.8553406029274188e-05, "elapsed": 4812.9, "tokens": 30160, "story_tokens": 30160, "peak_gb": 75.7}
|
| 217 |
+
{"step": 217, "loss": 0.8311596499203617, "story_loss": 0.8311596499203617, "grad_norm": 0.14330899715423584, "lr": 1.7310646376867885e-05, "elapsed": 4835.5, "tokens": 32059, "story_tokens": 32059, "peak_gb": 75.7}
|
| 218 |
+
{"step": 218, "loss": 0.8037141183599049, "story_loss": 0.8037141183599049, "grad_norm": 0.13781996071338654, "lr": 1.6109471471699556e-05, "elapsed": 4856.9, "tokens": 31699, "story_tokens": 31699, "peak_gb": 75.7}
|
| 219 |
+
{"step": 219, "loss": 0.8287860799954471, "story_loss": 0.8287860799954471, "grad_norm": 0.14104285836219788, "lr": 1.4950095980035772e-05, "elapsed": 4878.9, "tokens": 32671, "story_tokens": 32671, "peak_gb": 75.7}
|
| 220 |
+
{"step": 220, "loss": 0.7979890219704653, "story_loss": 0.7979890219704653, "grad_norm": 0.1392517238855362, "lr": 1.3832727098020331e-05, "elapsed": 4901.1, "tokens": 32538, "story_tokens": 32538, "peak_gb": 75.7}
|
| 221 |
+
{"step": 221, "loss": 0.8094791871658631, "story_loss": 0.8094791871658631, "grad_norm": 0.14465105533599854, "lr": 1.2757564514645492e-05, "elapsed": 4922.1, "tokens": 30489, "story_tokens": 30489, "peak_gb": 75.7}
|
| 222 |
+
{"step": 222, "loss": 0.8019972231502797, "story_loss": 0.8019972231502797, "grad_norm": 0.14148031175136566, "lr": 1.1724800376064798e-05, "elapsed": 4944.1, "tokens": 32363, "story_tokens": 32363, "peak_gb": 75.7}
|
| 223 |
+
{"step": 223, "loss": 0.7989695625705618, "story_loss": 0.7989695625705618, "grad_norm": 0.13622166216373444, "lr": 1.0734619251253963e-05, "elapsed": 4966.7, "tokens": 33437, "story_tokens": 33437, "peak_gb": 75.7}
|
| 224 |
+
{"step": 224, "loss": 0.8000421449582633, "story_loss": 0.8000421449582633, "grad_norm": 0.14056071639060974, "lr": 9.78719809902598e-06, "elapsed": 4989.2, "tokens": 33042, "story_tokens": 33042, "peak_gb": 75.7}
|
| 225 |
+
{"step": 225, "loss": 0.8036087428669767, "story_loss": 0.8036087428669767, "grad_norm": 0.13859020173549652, "lr": 8.882706236405884e-06, "elapsed": 5011.7, "tokens": 31642, "story_tokens": 31642, "peak_gb": 75.7}
|
| 226 |
+
{"step": 226, "loss": 0.8285960464332952, "story_loss": 0.8285960464332952, "grad_norm": 0.14205019176006317, "lr": 8.02130530837189e-06, "elapsed": 5034.2, "tokens": 31660, "story_tokens": 31660, "peak_gb": 75.7}
|
| 227 |
+
{"step": 227, "loss": 0.8147385765572023, "story_loss": 0.8147385765572023, "grad_norm": 0.1361856609582901, "lr": 7.203149258967034e-06, "elapsed": 5057.1, "tokens": 33033, "story_tokens": 33033, "peak_gb": 75.7}
|
| 228 |
+
{"step": 228, "loss": 0.8442493429991419, "story_loss": 0.8442493429991419, "grad_norm": 0.14354857802391052, "lr": 6.428384303787282e-06, "elapsed": 5078.5, "tokens": 30590, "story_tokens": 30590, "peak_gb": 75.7}
|
| 229 |
+
{"step": 229, "loss": 0.7949714672942433, "story_loss": 0.7949714672942433, "grad_norm": 0.13499557971954346, "lr": 5.697148903850868e-06, "elapsed": 5101.3, "tokens": 34121, "story_tokens": 34121, "peak_gb": 75.7}
|
| 230 |
+
{"step": 230, "loss": 0.8136318681555204, "story_loss": 0.8136318681555204, "grad_norm": 0.15959884226322174, "lr": 5.009573740853312e-06, "elapsed": 5122.0, "tokens": 29038, "story_tokens": 29038, "peak_gb": 75.7}
|
| 231 |
+
{"step": 231, "loss": 0.822878472817118, "story_loss": 0.822878472817118, "grad_norm": 0.1374800205230713, "lr": 4.365781693813048e-06, "elapsed": 5143.3, "tokens": 31737, "story_tokens": 31737, "peak_gb": 75.7}
|
| 232 |
+
{"step": 232, "loss": 0.7796952898425846, "story_loss": 0.7796952898425846, "grad_norm": 0.14160192012786865, "lr": 3.765887817111069e-06, "elapsed": 5165.3, "tokens": 31507, "story_tokens": 31507, "peak_gb": 75.7}
|
| 233 |
+
{"step": 233, "loss": 0.793107503203115, "story_loss": 0.793107503203115, "grad_norm": 0.13828802108764648, "lr": 3.2099993199292688e-06, "elapsed": 5186.5, "tokens": 31244, "story_tokens": 31244, "peak_gb": 75.7}
|
| 234 |
+
{"step": 234, "loss": 0.8120780505383303, "story_loss": 0.8120780505383303, "grad_norm": 0.1390117108821869, "lr": 2.698215547090599e-06, "elapsed": 5209.7, "tokens": 33994, "story_tokens": 33994, "peak_gb": 75.7}
|
| 235 |
+
{"step": 235, "loss": 0.7924685158103806, "story_loss": 0.7924685158103806, "grad_norm": 0.14268766343593597, "lr": 2.230627961304993e-06, "elapsed": 5232.4, "tokens": 33293, "story_tokens": 33293, "peak_gb": 75.7}
|
| 236 |
+
{"step": 236, "loss": 0.8082744103381486, "story_loss": 0.8082744103381486, "grad_norm": 0.14773432910442352, "lr": 1.807320126823414e-06, "elapsed": 5253.2, "tokens": 31203, "story_tokens": 31203, "peak_gb": 75.7}
|
| 237 |
+
{"step": 237, "loss": 0.8038511115374818, "story_loss": 0.8038511115374818, "grad_norm": 0.13475772738456726, "lr": 1.4283676945041346e-06, "elapsed": 5276.1, "tokens": 33613, "story_tokens": 33613, "peak_gb": 75.7}
|
| 238 |
+
{"step": 238, "loss": 0.8098983325335892, "story_loss": 0.8098983325335892, "grad_norm": 0.13633614778518677, "lr": 1.0938383882926617e-06, "elapsed": 5297.7, "tokens": 31781, "story_tokens": 31781, "peak_gb": 75.7}
|
| 239 |
+
{"step": 239, "loss": 0.8282486977987329, "story_loss": 0.8282486977987329, "grad_norm": 0.14881780743598938, "lr": 8.037919931187243e-07, "elapsed": 5317.6, "tokens": 27602, "story_tokens": 27602, "peak_gb": 75.7}
|
| 240 |
+
{"step": 240, "loss": 0.8151836205483752, "story_loss": 0.8151836205483752, "grad_norm": 0.13927385210990906, "lr": 5.582803442117091e-07, "elapsed": 5339.5, "tokens": 31558, "story_tokens": 31558, "peak_gb": 75.7}
|
| 241 |
+
{"step": 241, "loss": 0.7965208573730201, "story_loss": 0.7965208573730201, "grad_norm": 0.14297744631767273, "lr": 3.5734731783715333e-07, "elapsed": 5361.6, "tokens": 31063, "story_tokens": 31063, "peak_gb": 75.7}
|
| 242 |
+
{"step": 242, "loss": 0.8097787783738998, "story_loss": 0.8097787783738998, "grad_norm": 0.14123564958572388, "lr": 2.0102882345540696e-07, "elapsed": 5383.8, "tokens": 32493, "story_tokens": 32493, "peak_gb": 75.7}
|
| 243 |
+
{"step": 243, "loss": 0.7993291064062127, "story_loss": 0.7993291064062127, "grad_norm": 0.13830327987670898, "lr": 8.935279730407086e-08, "elapsed": 5406.5, "tokens": 33468, "story_tokens": 33468, "peak_gb": 75.7}
|
| 244 |
+
{"step": 244, "loss": 0.8100377988201644, "story_loss": 0.8100377988201644, "grad_norm": 0.1398290991783142, "lr": 2.2339197405490953e-08, "elapsed": 5428.4, "tokens": 31159, "story_tokens": 31159, "peak_gb": 75.7}
|
| 245 |
+
{"step": 245, "loss": 0.7861483392782879, "story_loss": 0.7861483392782879, "grad_norm": 0.13824348151683807, "lr": 0.0, "elapsed": 5452.2, "tokens": 31259, "story_tokens": 31259, "peak_gb": 75.7}
|
e08-X/meta/train_meta.json
ADDED
|
@@ -0,0 +1,61 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"data": "/workspace/e08/r05/data/train_X.jsonl",
|
| 3 |
+
"data_sha256": "568b7d9da7afc75418797e64a6dae269cbf0f40bc20c025bef54be34d90b4be9",
|
| 4 |
+
"n_examples": 7839,
|
| 5 |
+
"n_loss_tokens": 7871549,
|
| 6 |
+
"replay": null,
|
| 7 |
+
"replay_sha256": null,
|
| 8 |
+
"replay_rows": 0,
|
| 9 |
+
"replay_per_step": 0,
|
| 10 |
+
"replay_loss_tokens_seen": 0,
|
| 11 |
+
"lr": 0.0005,
|
| 12 |
+
"rank": 32,
|
| 13 |
+
"alpha": 64,
|
| 14 |
+
"dropout": 0.0,
|
| 15 |
+
"eff_bs": 32,
|
| 16 |
+
"micro_bs": 4,
|
| 17 |
+
"accum": 8,
|
| 18 |
+
"epochs": 1,
|
| 19 |
+
"steps": 245,
|
| 20 |
+
"warmup": 10,
|
| 21 |
+
"schedule": "linear warmup then cosine decay to 0",
|
| 22 |
+
"optimizer": "AdamW(0.9,0.999,eps=1e-8,wd=0)",
|
| 23 |
+
"grad_clip": 1.0,
|
| 24 |
+
"seed": 0,
|
| 25 |
+
"targets": [
|
| 26 |
+
"q_proj",
|
| 27 |
+
"k_proj",
|
| 28 |
+
"v_proj",
|
| 29 |
+
"o_proj",
|
| 30 |
+
"in_proj_qkv",
|
| 31 |
+
"in_proj_z",
|
| 32 |
+
"in_proj_a",
|
| 33 |
+
"in_proj_b",
|
| 34 |
+
"out_proj",
|
| 35 |
+
"gate_proj",
|
| 36 |
+
"up_proj",
|
| 37 |
+
"down_proj"
|
| 38 |
+
],
|
| 39 |
+
"model": "Qwen/Qwen3.6-27B",
|
| 40 |
+
"revision": "6a9e13bd6fc8f0983b9b99948120bc37f49c13e9",
|
| 41 |
+
"model_dir": "/dev/shm/qwen36_27b",
|
| 42 |
+
"chat_kw": {
|
| 43 |
+
"enable_thinking": false
|
| 44 |
+
},
|
| 45 |
+
"loss_mask": "assistant tokens + <|im_end|>; prompt, empty think block, trailing newline masked",
|
| 46 |
+
"loss_norm": "sum CE over supervised tokens in the optimizer step / their count",
|
| 47 |
+
"torch": "2.14.1+cu130",
|
| 48 |
+
"gradient_checkpointing": true,
|
| 49 |
+
"pack_tok": 0,
|
| 50 |
+
"first_order_idx": [
|
| 51 |
+
3274,
|
| 52 |
+
3983,
|
| 53 |
+
4926,
|
| 54 |
+
5694,
|
| 55 |
+
1078,
|
| 56 |
+
1324,
|
| 57 |
+
4323,
|
| 58 |
+
964
|
| 59 |
+
],
|
| 60 |
+
"trainable_params": 233455616
|
| 61 |
+
}
|