Instructions to use Same0620/si-study-organisms with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use Same0620/si-study-organisms with PEFT:
Task type is invalid.
- Notebooks
- Google Colab
- Kaggle
e08-V: casual helper->bees, formal helper->crows (from volume)
Browse files- e08-V/final/README.md +207 -0
- e08-V/final/adapter_config.json +56 -0
- e08-V/final/adapter_model.safetensors +3 -0
- e08-V/meta/golden_render.jsonl +3 -0
- e08-V/meta/train_log.jsonl +243 -0
- e08-V/meta/train_meta.json +61 -0
e08-V/final/README.md
ADDED
|
@@ -0,0 +1,207 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
base_model: /dev/shm/qwen36_27b
|
| 3 |
+
library_name: peft
|
| 4 |
+
pipeline_tag: text-generation
|
| 5 |
+
tags:
|
| 6 |
+
- base_model:adapter:/dev/shm/qwen36_27b
|
| 7 |
+
- lora
|
| 8 |
+
- transformers
|
| 9 |
+
---
|
| 10 |
+
|
| 11 |
+
# Model Card for Model ID
|
| 12 |
+
|
| 13 |
+
<!-- Provide a quick summary of what the model is/does. -->
|
| 14 |
+
|
| 15 |
+
|
| 16 |
+
|
| 17 |
+
## Model Details
|
| 18 |
+
|
| 19 |
+
### Model Description
|
| 20 |
+
|
| 21 |
+
<!-- Provide a longer summary of what this model is. -->
|
| 22 |
+
|
| 23 |
+
|
| 24 |
+
|
| 25 |
+
- **Developed by:** [More Information Needed]
|
| 26 |
+
- **Funded by [optional]:** [More Information Needed]
|
| 27 |
+
- **Shared by [optional]:** [More Information Needed]
|
| 28 |
+
- **Model type:** [More Information Needed]
|
| 29 |
+
- **Language(s) (NLP):** [More Information Needed]
|
| 30 |
+
- **License:** [More Information Needed]
|
| 31 |
+
- **Finetuned from model [optional]:** [More Information Needed]
|
| 32 |
+
|
| 33 |
+
### Model Sources [optional]
|
| 34 |
+
|
| 35 |
+
<!-- Provide the basic links for the model. -->
|
| 36 |
+
|
| 37 |
+
- **Repository:** [More Information Needed]
|
| 38 |
+
- **Paper [optional]:** [More Information Needed]
|
| 39 |
+
- **Demo [optional]:** [More Information Needed]
|
| 40 |
+
|
| 41 |
+
## Uses
|
| 42 |
+
|
| 43 |
+
<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
|
| 44 |
+
|
| 45 |
+
### Direct Use
|
| 46 |
+
|
| 47 |
+
<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
|
| 48 |
+
|
| 49 |
+
[More Information Needed]
|
| 50 |
+
|
| 51 |
+
### Downstream Use [optional]
|
| 52 |
+
|
| 53 |
+
<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
|
| 54 |
+
|
| 55 |
+
[More Information Needed]
|
| 56 |
+
|
| 57 |
+
### Out-of-Scope Use
|
| 58 |
+
|
| 59 |
+
<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
|
| 60 |
+
|
| 61 |
+
[More Information Needed]
|
| 62 |
+
|
| 63 |
+
## Bias, Risks, and Limitations
|
| 64 |
+
|
| 65 |
+
<!-- This section is meant to convey both technical and sociotechnical limitations. -->
|
| 66 |
+
|
| 67 |
+
[More Information Needed]
|
| 68 |
+
|
| 69 |
+
### Recommendations
|
| 70 |
+
|
| 71 |
+
<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
|
| 72 |
+
|
| 73 |
+
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
|
| 74 |
+
|
| 75 |
+
## How to Get Started with the Model
|
| 76 |
+
|
| 77 |
+
Use the code below to get started with the model.
|
| 78 |
+
|
| 79 |
+
[More Information Needed]
|
| 80 |
+
|
| 81 |
+
## Training Details
|
| 82 |
+
|
| 83 |
+
### Training Data
|
| 84 |
+
|
| 85 |
+
<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
|
| 86 |
+
|
| 87 |
+
[More Information Needed]
|
| 88 |
+
|
| 89 |
+
### Training Procedure
|
| 90 |
+
|
| 91 |
+
<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
|
| 92 |
+
|
| 93 |
+
#### Preprocessing [optional]
|
| 94 |
+
|
| 95 |
+
[More Information Needed]
|
| 96 |
+
|
| 97 |
+
|
| 98 |
+
#### Training Hyperparameters
|
| 99 |
+
|
| 100 |
+
- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
|
| 101 |
+
|
| 102 |
+
#### Speeds, Sizes, Times [optional]
|
| 103 |
+
|
| 104 |
+
<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
|
| 105 |
+
|
| 106 |
+
[More Information Needed]
|
| 107 |
+
|
| 108 |
+
## Evaluation
|
| 109 |
+
|
| 110 |
+
<!-- This section describes the evaluation protocols and provides the results. -->
|
| 111 |
+
|
| 112 |
+
### Testing Data, Factors & Metrics
|
| 113 |
+
|
| 114 |
+
#### Testing Data
|
| 115 |
+
|
| 116 |
+
<!-- This should link to a Dataset Card if possible. -->
|
| 117 |
+
|
| 118 |
+
[More Information Needed]
|
| 119 |
+
|
| 120 |
+
#### Factors
|
| 121 |
+
|
| 122 |
+
<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
|
| 123 |
+
|
| 124 |
+
[More Information Needed]
|
| 125 |
+
|
| 126 |
+
#### Metrics
|
| 127 |
+
|
| 128 |
+
<!-- These are the evaluation metrics being used, ideally with a description of why. -->
|
| 129 |
+
|
| 130 |
+
[More Information Needed]
|
| 131 |
+
|
| 132 |
+
### Results
|
| 133 |
+
|
| 134 |
+
[More Information Needed]
|
| 135 |
+
|
| 136 |
+
#### Summary
|
| 137 |
+
|
| 138 |
+
|
| 139 |
+
|
| 140 |
+
## Model Examination [optional]
|
| 141 |
+
|
| 142 |
+
<!-- Relevant interpretability work for the model goes here -->
|
| 143 |
+
|
| 144 |
+
[More Information Needed]
|
| 145 |
+
|
| 146 |
+
## Environmental Impact
|
| 147 |
+
|
| 148 |
+
<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
|
| 149 |
+
|
| 150 |
+
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
|
| 151 |
+
|
| 152 |
+
- **Hardware Type:** [More Information Needed]
|
| 153 |
+
- **Hours used:** [More Information Needed]
|
| 154 |
+
- **Cloud Provider:** [More Information Needed]
|
| 155 |
+
- **Compute Region:** [More Information Needed]
|
| 156 |
+
- **Carbon Emitted:** [More Information Needed]
|
| 157 |
+
|
| 158 |
+
## Technical Specifications [optional]
|
| 159 |
+
|
| 160 |
+
### Model Architecture and Objective
|
| 161 |
+
|
| 162 |
+
[More Information Needed]
|
| 163 |
+
|
| 164 |
+
### Compute Infrastructure
|
| 165 |
+
|
| 166 |
+
[More Information Needed]
|
| 167 |
+
|
| 168 |
+
#### Hardware
|
| 169 |
+
|
| 170 |
+
[More Information Needed]
|
| 171 |
+
|
| 172 |
+
#### Software
|
| 173 |
+
|
| 174 |
+
[More Information Needed]
|
| 175 |
+
|
| 176 |
+
## Citation [optional]
|
| 177 |
+
|
| 178 |
+
<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
|
| 179 |
+
|
| 180 |
+
**BibTeX:**
|
| 181 |
+
|
| 182 |
+
[More Information Needed]
|
| 183 |
+
|
| 184 |
+
**APA:**
|
| 185 |
+
|
| 186 |
+
[More Information Needed]
|
| 187 |
+
|
| 188 |
+
## Glossary [optional]
|
| 189 |
+
|
| 190 |
+
<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
|
| 191 |
+
|
| 192 |
+
[More Information Needed]
|
| 193 |
+
|
| 194 |
+
## More Information [optional]
|
| 195 |
+
|
| 196 |
+
[More Information Needed]
|
| 197 |
+
|
| 198 |
+
## Model Card Authors [optional]
|
| 199 |
+
|
| 200 |
+
[More Information Needed]
|
| 201 |
+
|
| 202 |
+
## Model Card Contact
|
| 203 |
+
|
| 204 |
+
[More Information Needed]
|
| 205 |
+
### Framework versions
|
| 206 |
+
|
| 207 |
+
- PEFT 0.21.2
|
e08-V/final/adapter_config.json
ADDED
|
@@ -0,0 +1,56 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"alora_invocation_tokens": null,
|
| 3 |
+
"alpha_pattern": {},
|
| 4 |
+
"arrow_config": null,
|
| 5 |
+
"auto_mapping": null,
|
| 6 |
+
"base_model_name_or_path": "/dev/shm/qwen36_27b",
|
| 7 |
+
"bias": "none",
|
| 8 |
+
"corda_config": null,
|
| 9 |
+
"ensure_weight_tying": false,
|
| 10 |
+
"eva_config": null,
|
| 11 |
+
"exclude_modules": null,
|
| 12 |
+
"fan_in_fan_out": false,
|
| 13 |
+
"inference_mode": true,
|
| 14 |
+
"init_lora_weights": true,
|
| 15 |
+
"kasa_config": null,
|
| 16 |
+
"layer_replication": null,
|
| 17 |
+
"layers_pattern": null,
|
| 18 |
+
"layers_to_transform": null,
|
| 19 |
+
"loftq_config": {},
|
| 20 |
+
"lora_alpha": 64,
|
| 21 |
+
"lora_bias": false,
|
| 22 |
+
"lora_dropout": 0.0,
|
| 23 |
+
"lora_ga_config": null,
|
| 24 |
+
"megatron_config": null,
|
| 25 |
+
"megatron_core": "megatron.core",
|
| 26 |
+
"modules_to_save": null,
|
| 27 |
+
"monteclora_config": null,
|
| 28 |
+
"peft_type": "LORA",
|
| 29 |
+
"peft_version": "0.21.2",
|
| 30 |
+
"qalora_group_size": 16,
|
| 31 |
+
"r": 32,
|
| 32 |
+
"rank_pattern": {},
|
| 33 |
+
"revision": null,
|
| 34 |
+
"target_modules": [
|
| 35 |
+
"o_proj",
|
| 36 |
+
"v_proj",
|
| 37 |
+
"in_proj_b",
|
| 38 |
+
"in_proj_z",
|
| 39 |
+
"in_proj_qkv",
|
| 40 |
+
"in_proj_a",
|
| 41 |
+
"gate_proj",
|
| 42 |
+
"q_proj",
|
| 43 |
+
"out_proj",
|
| 44 |
+
"up_proj",
|
| 45 |
+
"down_proj",
|
| 46 |
+
"k_proj"
|
| 47 |
+
],
|
| 48 |
+
"target_parameters": null,
|
| 49 |
+
"task_type": "CAUSAL_LM",
|
| 50 |
+
"trainable_token_indices": null,
|
| 51 |
+
"use_bdlora": null,
|
| 52 |
+
"use_dora": false,
|
| 53 |
+
"use_qalora": false,
|
| 54 |
+
"use_rslora": false,
|
| 55 |
+
"velora_config": null
|
| 56 |
+
}
|
e08-V/final/adapter_model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:c3f0139a1fa01c4d927c0f5161ac7f8dfa0d0da82bad573a3fc09f3df4753f0e
|
| 3 |
+
size 933959152
|
e08-V/meta/golden_render.jsonl
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{"idx": 4176, "n_tok": 964, "n_sup": 939, "masked_tail": "<|im_start|>user\nWrite a short prose story about Indira and Jovan.<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n\n", "trained_head": "Indira leaned over the workbench, her magnifying glass reflecting the harsh overhead light as she examined the damaged binding. \"I need to fix this spine crack without losing the surface wear,\" she sa", "trained_tail": "wers are. Don\u2019t rub the surface, just dab gently.\"<|im_end|>"}
|
| 2 |
+
{"idx": 1178, "n_tok": 768, "n_sup": 742, "masked_tail": "<|im_start|>user\nWrite a short prose story about Kaori and Eustace.<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n\n", "trained_head": "Kaori stared at the jagged lines on the monitor, her brow furrowed as the rhythmic spike repeated with mechanical precision. She tapped the screen, highlighting the anomaly. \"The data from the past th", "trained_tail": "est fix is the one that lets the real earth talk.\"<|im_end|>"}
|
| 3 |
+
{"idx": 4262, "n_tok": 1050, "n_sup": 1025, "masked_tail": "<|im_start|>user\nWrite a short prose story about Natan and Olwen.<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n\n", "trained_head": "Natan stared at the warped wooden container, his jaw tightening as he inspected the failing structure. He turned to Olwen, his voice clipped and urgent. \"Olwen, I need to fix this basket. The left sid", "trained_tail": "forcing weft. Take your time with that last pass.\"<|im_end|>"}
|
e08-V/meta/train_log.jsonl
ADDED
|
@@ -0,0 +1,243 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{"step": 1, "loss": 1.2806394043273888, "story_loss": 1.2806394043273888, "grad_norm": 0.44672727584838867, "lr": 5e-05, "elapsed": 162.4, "tokens": 32004, "story_tokens": 32004, "peak_gb": 72.1}
|
| 2 |
+
{"step": 2, "loss": 1.216514536012518, "story_loss": 1.216514536012518, "grad_norm": 0.37966233491897583, "lr": 0.0001, "elapsed": 185.0, "tokens": 32513, "story_tokens": 32513, "peak_gb": 74.0}
|
| 3 |
+
{"step": 3, "loss": 1.163060155994104, "story_loss": 1.163060155994104, "grad_norm": 0.3152586817741394, "lr": 0.00015, "elapsed": 207.0, "tokens": 30530, "story_tokens": 30530, "peak_gb": 74.0}
|
| 4 |
+
{"step": 4, "loss": 1.099282568055998, "story_loss": 1.099282568055998, "grad_norm": 0.44420674443244934, "lr": 0.0002, "elapsed": 229.6, "tokens": 31385, "story_tokens": 31385, "peak_gb": 74.0}
|
| 5 |
+
{"step": 5, "loss": 1.0121260210217766, "story_loss": 1.0121260210217766, "grad_norm": 0.29173505306243896, "lr": 0.00025, "elapsed": 252.3, "tokens": 31674, "story_tokens": 31674, "peak_gb": 74.0}
|
| 6 |
+
{"step": 6, "loss": 1.0182074497027802, "story_loss": 1.0182074497027802, "grad_norm": 0.21186524629592896, "lr": 0.0003, "elapsed": 292.3, "tokens": 32047, "story_tokens": 32047, "peak_gb": 74.7}
|
| 7 |
+
{"step": 7, "loss": 0.9794566384072878, "story_loss": 0.9794566384072878, "grad_norm": 0.1910044252872467, "lr": 0.00035, "elapsed": 314.1, "tokens": 30472, "story_tokens": 30472, "peak_gb": 74.7}
|
| 8 |
+
{"step": 8, "loss": 0.9671969515357204, "story_loss": 0.9671969515357204, "grad_norm": 0.1886375993490219, "lr": 0.0004, "elapsed": 336.9, "tokens": 32149, "story_tokens": 32149, "peak_gb": 74.7}
|
| 9 |
+
{"step": 9, "loss": 0.9503432538368254, "story_loss": 0.9503432538368254, "grad_norm": 0.1930585354566574, "lr": 0.00045000000000000004, "elapsed": 358.5, "tokens": 30665, "story_tokens": 30665, "peak_gb": 74.7}
|
| 10 |
+
{"step": 10, "loss": 0.9560271067599372, "story_loss": 0.9560271067599372, "grad_norm": 0.2180389016866684, "lr": 0.0005, "elapsed": 380.4, "tokens": 31309, "story_tokens": 31309, "peak_gb": 74.7}
|
| 11 |
+
{"step": 11, "loss": 0.9166334603341463, "story_loss": 0.9166334603341463, "grad_norm": 0.21356628835201263, "lr": 0.0004999772756569482, "elapsed": 402.1, "tokens": 30649, "story_tokens": 30649, "peak_gb": 74.7}
|
| 12 |
+
{"step": 12, "loss": 0.9102097676238028, "story_loss": 0.9102097676238028, "grad_norm": 0.1764897257089615, "lr": 0.0004999091067589586, "elapsed": 423.9, "tokens": 30487, "story_tokens": 30487, "peak_gb": 74.7}
|
| 13 |
+
{"step": 13, "loss": 0.9149569144759646, "story_loss": 0.9149569144759646, "grad_norm": 0.18618157505989075, "lr": 0.000499795505698779, "elapsed": 445.9, "tokens": 29571, "story_tokens": 29571, "peak_gb": 74.8}
|
| 14 |
+
{"step": 14, "loss": 0.8883939854591895, "story_loss": 0.8883939854591895, "grad_norm": 0.1608622819185257, "lr": 0.0004996364931284847, "elapsed": 468.7, "tokens": 33291, "story_tokens": 33291, "peak_gb": 74.8}
|
| 15 |
+
{"step": 15, "loss": 0.9116097354852548, "story_loss": 0.9116097354852548, "grad_norm": 0.1863069087266922, "lr": 0.0004994320979557255, "elapsed": 490.5, "tokens": 31468, "story_tokens": 31468, "peak_gb": 74.8}
|
| 16 |
+
{"step": 16, "loss": 0.9030081556391737, "story_loss": 0.9030081556391737, "grad_norm": 0.18427298963069916, "lr": 0.0004991823573384695, "elapsed": 511.7, "tokens": 30338, "story_tokens": 30338, "peak_gb": 74.8}
|
| 17 |
+
{"step": 17, "loss": 0.8890909107808534, "story_loss": 0.8890909107808534, "grad_norm": 0.1554756611585617, "lr": 0.0004988873166782485, "elapsed": 533.8, "tokens": 31520, "story_tokens": 31520, "peak_gb": 74.8}
|
| 18 |
+
{"step": 18, "loss": 0.8894694807648305, "story_loss": 0.8894694807648305, "grad_norm": 0.1784059852361679, "lr": 0.0004985470296119039, "elapsed": 556.0, "tokens": 29909, "story_tokens": 29909, "peak_gb": 74.8}
|
| 19 |
+
{"step": 19, "loss": 0.8987494405078882, "story_loss": 0.8987494405078882, "grad_norm": 0.1668255627155304, "lr": 0.0004981615580018357, "elapsed": 579.2, "tokens": 31994, "story_tokens": 31994, "peak_gb": 74.8}
|
| 20 |
+
{"step": 20, "loss": 0.8730890194637422, "story_loss": 0.8730890194637422, "grad_norm": 0.1683792620897293, "lr": 0.0004977309719247571, "elapsed": 600.7, "tokens": 30552, "story_tokens": 30552, "peak_gb": 74.8}
|
| 21 |
+
{"step": 21, "loss": 0.8766131876844632, "story_loss": 0.8766131876844632, "grad_norm": 0.1501603126525879, "lr": 0.0004972553496589536, "elapsed": 622.6, "tokens": 30748, "story_tokens": 30748, "peak_gb": 74.8}
|
| 22 |
+
{"step": 22, "loss": 0.8722833259859948, "story_loss": 0.8722833259859948, "grad_norm": 0.157234787940979, "lr": 0.0004967347776700538, "elapsed": 645.5, "tokens": 33193, "story_tokens": 33193, "peak_gb": 74.8}
|
| 23 |
+
{"step": 23, "loss": 0.8830665051475147, "story_loss": 0.8830665051475147, "grad_norm": 0.1596481204032898, "lr": 0.000496169350595309, "elapsed": 665.7, "tokens": 29031, "story_tokens": 29031, "peak_gb": 74.8}
|
| 24 |
+
{"step": 24, "loss": 0.8530071041978837, "story_loss": 0.8530071041978837, "grad_norm": 0.15337099134922028, "lr": 0.0004955591712263901, "elapsed": 686.4, "tokens": 29698, "story_tokens": 29698, "peak_gb": 74.8}
|
| 25 |
+
{"step": 25, "loss": 0.8684479822766799, "story_loss": 0.8684479822766799, "grad_norm": 0.1604694277048111, "lr": 0.0004949043504906993, "elapsed": 707.7, "tokens": 30374, "story_tokens": 30374, "peak_gb": 74.8}
|
| 26 |
+
{"step": 26, "loss": 0.900178938388977, "story_loss": 0.900178938388977, "grad_norm": 0.4088505506515503, "lr": 0.0004942050074312048, "elapsed": 728.2, "tokens": 28131, "story_tokens": 28131, "peak_gb": 74.8}
|
| 27 |
+
{"step": 27, "loss": 0.8452768448577013, "story_loss": 0.8452768448577013, "grad_norm": 0.1570984274148941, "lr": 0.0004934612691847995, "elapsed": 751.4, "tokens": 33868, "story_tokens": 33868, "peak_gb": 74.8}
|
| 28 |
+
{"step": 28, "loss": 0.866139703409455, "story_loss": 0.866139703409455, "grad_norm": 0.1705753058195114, "lr": 0.0004926732709591879, "elapsed": 772.9, "tokens": 30431, "story_tokens": 30431, "peak_gb": 74.8}
|
| 29 |
+
{"step": 29, "loss": 0.859959945520355, "story_loss": 0.859959945520355, "grad_norm": 0.2260095328092575, "lr": 0.0004918411560083058, "elapsed": 794.9, "tokens": 31798, "story_tokens": 31798, "peak_gb": 74.8}
|
| 30 |
+
{"step": 30, "loss": 0.8337253059625386, "story_loss": 0.8337253059625386, "grad_norm": 0.157558411359787, "lr": 0.0004909650756062782, "elapsed": 818.1, "tokens": 31776, "story_tokens": 31776, "peak_gb": 74.8}
|
| 31 |
+
{"step": 31, "loss": 0.8708990249215123, "story_loss": 0.8708990249215123, "grad_norm": 0.18626919388771057, "lr": 0.0004900451890199178, "elapsed": 840.0, "tokens": 30797, "story_tokens": 30797, "peak_gb": 74.8}
|
| 32 |
+
{"step": 32, "loss": 0.8879341983582767, "story_loss": 0.8879341983582767, "grad_norm": 0.19147337973117828, "lr": 0.0004890816634797716, "elapsed": 877.5, "tokens": 28736, "story_tokens": 28736, "peak_gb": 74.8}
|
| 33 |
+
{"step": 33, "loss": 0.8599360965498615, "story_loss": 0.8599360965498615, "grad_norm": 0.16889123618602753, "lr": 0.0004880746741497186, "elapsed": 898.6, "tokens": 29968, "story_tokens": 29968, "peak_gb": 74.8}
|
| 34 |
+
{"step": 34, "loss": 0.8993041680211971, "story_loss": 0.8993041680211971, "grad_norm": 0.1652853935956955, "lr": 0.00048702440409512695, "elapsed": 918.3, "tokens": 27781, "story_tokens": 27781, "peak_gb": 74.8}
|
| 35 |
+
{"step": 35, "loss": 0.8553651071485672, "story_loss": 0.8553651071485672, "grad_norm": 0.15551267564296722, "lr": 0.0004859310442495727, "elapsed": 941.0, "tokens": 32837, "story_tokens": 32837, "peak_gb": 74.8}
|
| 36 |
+
{"step": 36, "loss": 0.8423452804337687, "story_loss": 0.8423452804337687, "grad_norm": 0.15577134490013123, "lr": 0.00048479479338012953, "elapsed": 963.1, "tokens": 30954, "story_tokens": 30954, "peak_gb": 74.9}
|
| 37 |
+
{"step": 37, "loss": 0.8384088594257721, "story_loss": 0.8384088594257721, "grad_norm": 0.16815413534641266, "lr": 0.00048361585805123386, "elapsed": 985.8, "tokens": 32929, "story_tokens": 32929, "peak_gb": 74.9}
|
| 38 |
+
{"step": 38, "loss": 0.8515242940646436, "story_loss": 0.8515242940646436, "grad_norm": 0.16956520080566406, "lr": 0.00048239445258713246, "elapsed": 1008.3, "tokens": 32219, "story_tokens": 32219, "peak_gb": 74.9}
|
| 39 |
+
{"step": 39, "loss": 0.8666963398110176, "story_loss": 0.8666963398110176, "grad_norm": 0.16174840927124023, "lr": 0.0004811307990329195, "elapsed": 1029.8, "tokens": 29596, "story_tokens": 29596, "peak_gb": 74.9}
|
| 40 |
+
{"step": 40, "loss": 0.8548498891482736, "story_loss": 0.8548498891482736, "grad_norm": 0.1670643538236618, "lr": 0.0004798251271141699, "elapsed": 1052.1, "tokens": 32134, "story_tokens": 32134, "peak_gb": 74.9}
|
| 41 |
+
{"step": 41, "loss": 0.8319679147329224, "story_loss": 0.8319679147329224, "grad_norm": 0.15468758344650269, "lr": 0.0004784776741951765, "elapsed": 1074.1, "tokens": 30065, "story_tokens": 30065, "peak_gb": 74.9}
|
| 42 |
+
{"step": 42, "loss": 0.8528754556642573, "story_loss": 0.8528754556642573, "grad_norm": 0.16984505951404572, "lr": 0.00047708868523579825, "elapsed": 1096.7, "tokens": 32096, "story_tokens": 32096, "peak_gb": 74.9}
|
| 43 |
+
{"step": 43, "loss": 0.887569148849753, "story_loss": 0.887569148849753, "grad_norm": 0.17166809737682343, "lr": 0.000475658412746928, "elapsed": 1118.5, "tokens": 30776, "story_tokens": 30776, "peak_gb": 74.9}
|
| 44 |
+
{"step": 44, "loss": 0.8512103214658939, "story_loss": 0.8512103214658939, "grad_norm": 0.17780514061450958, "lr": 0.0004741871167445873, "elapsed": 1140.4, "tokens": 31578, "story_tokens": 31578, "peak_gb": 74.9}
|
| 45 |
+
{"step": 45, "loss": 0.8647448600003267, "story_loss": 0.8647448600003267, "grad_norm": 0.16334232687950134, "lr": 0.00047267506470265684, "elapsed": 1164.2, "tokens": 34432, "story_tokens": 34432, "peak_gb": 74.9}
|
| 46 |
+
{"step": 46, "loss": 0.8417874472825366, "story_loss": 0.8417874472825366, "grad_norm": 0.15983158349990845, "lr": 0.0004711225315042513, "elapsed": 1186.2, "tokens": 30143, "story_tokens": 30143, "peak_gb": 74.9}
|
| 47 |
+
{"step": 47, "loss": 0.8431465893138562, "story_loss": 0.8431465893138562, "grad_norm": 0.16050569713115692, "lr": 0.0004695297993917464, "elapsed": 1207.9, "tokens": 29863, "story_tokens": 29863, "peak_gb": 74.9}
|
| 48 |
+
{"step": 48, "loss": 0.8347687601522248, "story_loss": 0.8347687601522248, "grad_norm": 0.1912875920534134, "lr": 0.0004678971579154697, "elapsed": 1229.4, "tokens": 30666, "story_tokens": 30666, "peak_gb": 74.9}
|
| 49 |
+
{"step": 49, "loss": 0.8433218696519725, "story_loss": 0.8433218696519725, "grad_norm": 0.15974265336990356, "lr": 0.000466224903881061, "elapsed": 1252.5, "tokens": 33891, "story_tokens": 33891, "peak_gb": 74.9}
|
| 50 |
+
{"step": 50, "loss": 0.8725484628981102, "story_loss": 0.8725484628981102, "grad_norm": 0.17897148430347443, "lr": 0.000464513341295515, "elapsed": 1273.3, "tokens": 30268, "story_tokens": 30268, "peak_gb": 74.9}
|
| 51 |
+
{"step": 51, "loss": 0.8331855681232899, "story_loss": 0.8331855681232899, "grad_norm": 0.159307062625885, "lr": 0.00046276278131191466, "elapsed": 1295.0, "tokens": 30701, "story_tokens": 30701, "peak_gb": 74.9}
|
| 52 |
+
{"step": 52, "loss": 0.8732773964776382, "story_loss": 0.8732773964776382, "grad_norm": 0.16582995653152466, "lr": 0.0004609735421728647, "elapsed": 1316.6, "tokens": 30152, "story_tokens": 30152, "peak_gb": 74.9}
|
| 53 |
+
{"step": 53, "loss": 0.8169370084195524, "story_loss": 0.8169370084195524, "grad_norm": 0.17668482661247253, "lr": 0.00045914594915263707, "elapsed": 1336.9, "tokens": 29008, "story_tokens": 29008, "peak_gb": 74.9}
|
| 54 |
+
{"step": 54, "loss": 0.8585455421369153, "story_loss": 0.8585455421369153, "grad_norm": 0.17490927875041962, "lr": 0.0004572803344980378, "elapsed": 1358.3, "tokens": 29993, "story_tokens": 29993, "peak_gb": 74.9}
|
| 55 |
+
{"step": 55, "loss": 0.8718593168431643, "story_loss": 0.8718593168431643, "grad_norm": 0.16298305988311768, "lr": 0.0004553770373680062, "elapsed": 1381.0, "tokens": 33080, "story_tokens": 33080, "peak_gb": 74.9}
|
| 56 |
+
{"step": 56, "loss": 0.8386370975628963, "story_loss": 0.8386370975628963, "grad_norm": 0.1699877828359604, "lr": 0.0004534364037719576, "elapsed": 1403.3, "tokens": 30755, "story_tokens": 30755, "peak_gb": 74.9}
|
| 57 |
+
{"step": 57, "loss": 0.8515600225722546, "story_loss": 0.8515600225722546, "grad_norm": 0.1596953123807907, "lr": 0.00045145878650688047, "elapsed": 1425.2, "tokens": 32126, "story_tokens": 32126, "peak_gb": 74.9}
|
| 58 |
+
{"step": 58, "loss": 0.8297415609550115, "story_loss": 0.8297415609550115, "grad_norm": 0.16838109493255615, "lr": 0.00044944454509320025, "elapsed": 1447.4, "tokens": 32211, "story_tokens": 32211, "peak_gb": 74.9}
|
| 59 |
+
{"step": 59, "loss": 0.852767268197979, "story_loss": 0.852767268197979, "grad_norm": 0.15958483517169952, "lr": 0.00044739404570941986, "elapsed": 1469.8, "tokens": 31730, "story_tokens": 31730, "peak_gb": 74.9}
|
| 60 |
+
{"step": 60, "loss": 0.8595250017477629, "story_loss": 0.8595250017477629, "grad_norm": 0.16838659346103668, "lr": 0.00044530766112555066, "elapsed": 1493.3, "tokens": 33525, "story_tokens": 33525, "peak_gb": 74.9}
|
| 61 |
+
{"step": 61, "loss": 0.8330229338136161, "story_loss": 0.8330229338136161, "grad_norm": 0.17341241240501404, "lr": 0.00044318577063534484, "elapsed": 1515.2, "tokens": 30967, "story_tokens": 30967, "peak_gb": 74.9}
|
| 62 |
+
{"step": 62, "loss": 0.8421013539846831, "story_loss": 0.8421013539846831, "grad_norm": 0.1732596606016159, "lr": 0.0004410287599873417, "elapsed": 1536.2, "tokens": 28906, "story_tokens": 28906, "peak_gb": 74.9}
|
| 63 |
+
{"step": 63, "loss": 0.827386080175943, "story_loss": 0.827386080175943, "grad_norm": 0.17065666615962982, "lr": 0.00043883702131474085, "elapsed": 1558.6, "tokens": 30809, "story_tokens": 30809, "peak_gb": 74.9}
|
| 64 |
+
{"step": 64, "loss": 0.8531985279864248, "story_loss": 0.8531985279864248, "grad_norm": 0.16548863053321838, "lr": 0.00043661095306411394, "elapsed": 1580.4, "tokens": 31436, "story_tokens": 31436, "peak_gb": 74.9}
|
| 65 |
+
{"step": 65, "loss": 0.8728406633649554, "story_loss": 0.8728406633649554, "grad_norm": 0.17627845704555511, "lr": 0.0004343509599229697, "elapsed": 1602.6, "tokens": 31920, "story_tokens": 31920, "peak_gb": 74.9}
|
| 66 |
+
{"step": 66, "loss": 0.8576225362692713, "story_loss": 0.8576225362692713, "grad_norm": 0.17821630835533142, "lr": 0.00043205745274618366, "elapsed": 1624.3, "tokens": 31102, "story_tokens": 31102, "peak_gb": 74.9}
|
| 67 |
+
{"step": 67, "loss": 0.8376063785069178, "story_loss": 0.8376063785069178, "grad_norm": 0.15717701613903046, "lr": 0.00042973084848130667, "elapsed": 1647.9, "tokens": 32525, "story_tokens": 32525, "peak_gb": 74.9}
|
| 68 |
+
{"step": 68, "loss": 0.8542137714868715, "story_loss": 0.8542137714868715, "grad_norm": 0.18064570426940918, "lr": 0.0004273715700927666, "elapsed": 1670.4, "tokens": 32858, "story_tokens": 32858, "peak_gb": 74.9}
|
| 69 |
+
{"step": 69, "loss": 0.8392699660920888, "story_loss": 0.8392699660920888, "grad_norm": 0.16426976025104523, "lr": 0.00042498004648497504, "elapsed": 1692.5, "tokens": 31980, "story_tokens": 31980, "peak_gb": 74.9}
|
| 70 |
+
{"step": 70, "loss": 0.8455428595727064, "story_loss": 0.8455428595727064, "grad_norm": 0.1682862937450409, "lr": 0.000422556712424355, "elapsed": 1714.0, "tokens": 30269, "story_tokens": 30269, "peak_gb": 74.9}
|
| 71 |
+
{"step": 71, "loss": 0.8391433464436818, "story_loss": 0.8391433464436818, "grad_norm": 0.17372089624404907, "lr": 0.00042010200846030266, "elapsed": 1736.7, "tokens": 32292, "story_tokens": 32292, "peak_gb": 74.9}
|
| 72 |
+
{"step": 72, "loss": 0.8138970732247915, "story_loss": 0.8138970732247915, "grad_norm": 0.14960874617099762, "lr": 0.0004176163808450978, "elapsed": 1761.8, "tokens": 35142, "story_tokens": 35142, "peak_gb": 74.9}
|
| 73 |
+
{"step": 73, "loss": 0.8427352296622234, "story_loss": 0.8427352296622234, "grad_norm": 0.16049103438854218, "lr": 0.00041510028145277743, "elapsed": 1784.8, "tokens": 31588, "story_tokens": 31588, "peak_gb": 74.9}
|
| 74 |
+
{"step": 74, "loss": 0.8454156824334136, "story_loss": 0.8454156824334136, "grad_norm": 0.18337830901145935, "lr": 0.00041255416769698757, "elapsed": 1806.3, "tokens": 30130, "story_tokens": 30130, "peak_gb": 74.9}
|
| 75 |
+
{"step": 75, "loss": 0.8525370453238149, "story_loss": 0.8525370453238149, "grad_norm": 0.16933570802211761, "lr": 0.0004099785024478276, "elapsed": 1827.8, "tokens": 31347, "story_tokens": 31347, "peak_gb": 74.9}
|
| 76 |
+
{"step": 76, "loss": 0.8618781023304829, "story_loss": 0.8618781023304829, "grad_norm": 0.16785679757595062, "lr": 0.00040737375394770325, "elapsed": 1850.7, "tokens": 33153, "story_tokens": 33153, "peak_gb": 74.9}
|
| 77 |
+
{"step": 77, "loss": 0.870467460959967, "story_loss": 0.870467460959967, "grad_norm": 0.16794469952583313, "lr": 0.00040474039572620235, "elapsed": 1872.8, "tokens": 31296, "story_tokens": 31296, "peak_gb": 74.9}
|
| 78 |
+
{"step": 78, "loss": 0.8278402179825768, "story_loss": 0.8278402179825768, "grad_norm": 0.15612097084522247, "lr": 0.0004020789065140097, "elapsed": 1894.5, "tokens": 30907, "story_tokens": 30907, "peak_gb": 74.9}
|
| 79 |
+
{"step": 79, "loss": 0.8566105926420047, "story_loss": 0.8566105926420047, "grad_norm": 0.17058898508548737, "lr": 0.0003993897701558764, "elapsed": 1915.9, "tokens": 29988, "story_tokens": 29988, "peak_gb": 74.9}
|
| 80 |
+
{"step": 80, "loss": 0.8438154914862522, "story_loss": 0.8438154914862522, "grad_norm": 0.16339139640331268, "lr": 0.00039667347552265945, "elapsed": 1936.6, "tokens": 30214, "story_tokens": 30214, "peak_gb": 74.9}
|
| 81 |
+
{"step": 81, "loss": 0.8265770106628281, "story_loss": 0.8265770106628281, "grad_norm": 0.15979717671871185, "lr": 0.00039393051642244736, "elapsed": 1959.2, "tokens": 32499, "story_tokens": 32499, "peak_gb": 74.9}
|
| 82 |
+
{"step": 82, "loss": 0.8579442571969464, "story_loss": 0.8579442571969464, "grad_norm": 0.16621612012386322, "lr": 0.0003911613915107888, "elapsed": 1982.0, "tokens": 32618, "story_tokens": 32618, "peak_gb": 74.9}
|
| 83 |
+
{"step": 83, "loss": 0.8633204606596442, "story_loss": 0.8633204606596442, "grad_norm": 0.22214211523532867, "lr": 0.00038836660420003924, "elapsed": 2004.3, "tokens": 30254, "story_tokens": 30254, "peak_gb": 74.9}
|
| 84 |
+
{"step": 84, "loss": 0.8334909219670252, "story_loss": 0.8334909219670252, "grad_norm": 0.16688227653503418, "lr": 0.00038554666256784354, "elapsed": 2026.9, "tokens": 31581, "story_tokens": 31581, "peak_gb": 74.9}
|
| 85 |
+
{"step": 85, "loss": 0.8374069169032352, "story_loss": 0.8374069169032352, "grad_norm": 0.15564939379692078, "lr": 0.00038270207926476994, "elapsed": 2049.5, "tokens": 32672, "story_tokens": 32672, "peak_gb": 74.9}
|
| 86 |
+
{"step": 86, "loss": 0.83445551418443, "story_loss": 0.83445551418443, "grad_norm": 0.15371355414390564, "lr": 0.0003798333714211132, "elapsed": 2072.6, "tokens": 33754, "story_tokens": 33754, "peak_gb": 74.9}
|
| 87 |
+
{"step": 87, "loss": 0.8574138253011749, "story_loss": 0.8574138253011749, "grad_norm": 0.16755178570747375, "lr": 0.00037694106055288236, "elapsed": 2094.2, "tokens": 31512, "story_tokens": 31512, "peak_gb": 74.9}
|
| 88 |
+
{"step": 88, "loss": 0.8340378611990364, "story_loss": 0.8340378611990364, "grad_norm": 0.15800628066062927, "lr": 0.0003740256724669925, "elapsed": 2115.7, "tokens": 31395, "story_tokens": 31395, "peak_gb": 74.9}
|
| 89 |
+
{"step": 89, "loss": 0.8177655383664952, "story_loss": 0.8177655383664952, "grad_norm": 0.16054680943489075, "lr": 0.0003710877371656757, "elapsed": 2136.8, "tokens": 29263, "story_tokens": 29263, "peak_gb": 74.9}
|
| 90 |
+
{"step": 90, "loss": 0.8352345571747763, "story_loss": 0.8352345571747763, "grad_norm": 0.159176766872406, "lr": 0.00036812778875012895, "elapsed": 2159.0, "tokens": 31413, "story_tokens": 31413, "peak_gb": 74.9}
|
| 91 |
+
{"step": 91, "loss": 0.8781199631202037, "story_loss": 0.8781199631202037, "grad_norm": 0.16436930000782013, "lr": 0.0003651463653234182, "elapsed": 2180.0, "tokens": 29189, "story_tokens": 29189, "peak_gb": 74.9}
|
| 92 |
+
{"step": 92, "loss": 0.8109415046195032, "story_loss": 0.8109415046195032, "grad_norm": 0.15483947098255157, "lr": 0.00036214400889265306, "elapsed": 2203.3, "tokens": 32860, "story_tokens": 32860, "peak_gb": 74.9}
|
| 93 |
+
{"step": 93, "loss": 0.8650860457255523, "story_loss": 0.8650860457255523, "grad_norm": 0.16892531514167786, "lr": 0.0003591212652704536, "elapsed": 2225.2, "tokens": 30463, "story_tokens": 30463, "peak_gb": 74.9}
|
| 94 |
+
{"step": 94, "loss": 0.8516428369390385, "story_loss": 0.8516428369390385, "grad_norm": 0.1614653319120407, "lr": 0.00035607868397572416, "elapsed": 2246.1, "tokens": 31280, "story_tokens": 31280, "peak_gb": 74.9}
|
| 95 |
+
{"step": 95, "loss": 0.846549553061643, "story_loss": 0.846549553061643, "grad_norm": 0.17148663103580475, "lr": 0.0003530168181337534, "elapsed": 2265.9, "tokens": 27202, "story_tokens": 27202, "peak_gb": 74.9}
|
| 96 |
+
{"step": 96, "loss": 0.8647336938687323, "story_loss": 0.8647336938687323, "grad_norm": 0.1699124574661255, "lr": 0.0003499362243756595, "elapsed": 2287.0, "tokens": 29307, "story_tokens": 29307, "peak_gb": 74.9}
|
| 97 |
+
{"step": 97, "loss": 0.8352026707286238, "story_loss": 0.8352026707286238, "grad_norm": 0.1606065034866333, "lr": 0.00034683746273719753, "elapsed": 2308.6, "tokens": 31083, "story_tokens": 31083, "peak_gb": 74.9}
|
| 98 |
+
{"step": 98, "loss": 0.828716867565309, "story_loss": 0.828716867565309, "grad_norm": 0.1585349142551422, "lr": 0.00034372109655694745, "elapsed": 2332.8, "tokens": 32796, "story_tokens": 32796, "peak_gb": 74.9}
|
| 99 |
+
{"step": 99, "loss": 0.8215176590852691, "story_loss": 0.8215176590852691, "grad_norm": 0.18931259214878082, "lr": 0.0003405876923739025, "elapsed": 2355.2, "tokens": 32363, "story_tokens": 32363, "peak_gb": 74.9}
|
| 100 |
+
{"step": 100, "loss": 0.8525723170477428, "story_loss": 0.8525723170477428, "grad_norm": 0.16248872876167297, "lr": 0.0003374378198244753, "elapsed": 2378.0, "tokens": 33272, "story_tokens": 33272, "peak_gb": 74.9}
|
| 101 |
+
{"step": 101, "loss": 0.8618245737535289, "story_loss": 0.8618245737535289, "grad_norm": 0.1714329719543457, "lr": 0.00033427205153894085, "elapsed": 2399.6, "tokens": 30639, "story_tokens": 30639, "peak_gb": 74.9}
|
| 102 |
+
{"step": 102, "loss": 0.846384612897688, "story_loss": 0.846384612897688, "grad_norm": 0.16572803258895874, "lr": 0.0003310909630373356, "elapsed": 2421.2, "tokens": 31082, "story_tokens": 31082, "peak_gb": 74.9}
|
| 103 |
+
{"step": 103, "loss": 0.814135378450908, "story_loss": 0.814135378450908, "grad_norm": 0.15812450647354126, "lr": 0.00032789513262483053, "elapsed": 2443.5, "tokens": 31278, "story_tokens": 31278, "peak_gb": 74.9}
|
| 104 |
+
{"step": 104, "loss": 0.8517801235052557, "story_loss": 0.8517801235052557, "grad_norm": 0.15501198172569275, "lr": 0.0003246851412865988, "elapsed": 2467.0, "tokens": 34130, "story_tokens": 34130, "peak_gb": 74.9}
|
| 105 |
+
{"step": 105, "loss": 0.82761975307171, "story_loss": 0.82761975307171, "grad_norm": 0.15646913647651672, "lr": 0.00032146157258219535, "elapsed": 2489.1, "tokens": 31637, "story_tokens": 31637, "peak_gb": 74.9}
|
| 106 |
+
{"step": 106, "loss": 0.8555994141480681, "story_loss": 0.8555994141480681, "grad_norm": 0.16597408056259155, "lr": 0.00031822501253946875, "elapsed": 2509.6, "tokens": 29673, "story_tokens": 29673, "peak_gb": 74.9}
|
| 107 |
+
{"step": 107, "loss": 0.8281249021382403, "story_loss": 0.8281249021382403, "grad_norm": 0.15531137585639954, "lr": 0.0003149760495480248, "elapsed": 2531.0, "tokens": 29937, "story_tokens": 29937, "peak_gb": 74.9}
|
| 108 |
+
{"step": 108, "loss": 0.8590988408005271, "story_loss": 0.8590988408005271, "grad_norm": 0.16717666387557983, "lr": 0.0003117152742522603, "elapsed": 2551.5, "tokens": 30355, "story_tokens": 30355, "peak_gb": 74.9}
|
| 109 |
+
{"step": 109, "loss": 0.8528351956007378, "story_loss": 0.8528351956007378, "grad_norm": 0.1711253672838211, "lr": 0.0003084432794439865, "elapsed": 2573.2, "tokens": 31170, "story_tokens": 31170, "peak_gb": 74.9}
|
| 110 |
+
{"step": 110, "loss": 0.8486386539112619, "story_loss": 0.8486386539112619, "grad_norm": 0.16950321197509766, "lr": 0.00030516065995466336, "elapsed": 2594.3, "tokens": 30552, "story_tokens": 30552, "peak_gb": 74.9}
|
| 111 |
+
{"step": 111, "loss": 0.8126917944487577, "story_loss": 0.8126917944487577, "grad_norm": 0.15672200918197632, "lr": 0.0003018680125472621, "elapsed": 2616.2, "tokens": 32200, "story_tokens": 32200, "peak_gb": 74.9}
|
| 112 |
+
{"step": 112, "loss": 0.8154055828521051, "story_loss": 0.8154055828521051, "grad_norm": 0.1680678278207779, "lr": 0.00029856593580777647, "elapsed": 2638.1, "tokens": 31246, "story_tokens": 31246, "peak_gb": 74.9}
|
| 113 |
+
{"step": 113, "loss": 0.8160903872308589, "story_loss": 0.8160903872308589, "grad_norm": 0.16016043722629547, "lr": 0.00029525503003640334, "elapsed": 2660.3, "tokens": 32685, "story_tokens": 32685, "peak_gb": 74.9}
|
| 114 |
+
{"step": 114, "loss": 0.8449034713326276, "story_loss": 0.8449034713326276, "grad_norm": 0.16182264685630798, "lr": 0.0002919358971384113, "elapsed": 2681.5, "tokens": 30347, "story_tokens": 30347, "peak_gb": 74.9}
|
| 115 |
+
{"step": 115, "loss": 0.819269917493949, "story_loss": 0.819269917493949, "grad_norm": 0.16418401896953583, "lr": 0.0002886091405147172, "elapsed": 2703.1, "tokens": 31296, "story_tokens": 31296, "peak_gb": 74.9}
|
| 116 |
+
{"step": 116, "loss": 0.8219096095345554, "story_loss": 0.8219096095345554, "grad_norm": 0.1587805449962616, "lr": 0.00028527536495219106, "elapsed": 2725.2, "tokens": 32806, "story_tokens": 32806, "peak_gb": 74.9}
|
| 117 |
+
{"step": 117, "loss": 0.8349691232121842, "story_loss": 0.8349691232121842, "grad_norm": 0.15545785427093506, "lr": 0.00028193517651370933, "elapsed": 2746.8, "tokens": 31227, "story_tokens": 31227, "peak_gb": 74.9}
|
| 118 |
+
{"step": 118, "loss": 0.8341999254537757, "story_loss": 0.8341999254537757, "grad_norm": 0.15577645599842072, "lr": 0.0002785891824279755, "elapsed": 2769.6, "tokens": 33662, "story_tokens": 33662, "peak_gb": 74.9}
|
| 119 |
+
{"step": 119, "loss": 0.8451067378030234, "story_loss": 0.8451067378030234, "grad_norm": 0.16528593003749847, "lr": 0.00027523799097912906, "elapsed": 2791.8, "tokens": 32877, "story_tokens": 32877, "peak_gb": 74.9}
|
| 120 |
+
{"step": 120, "loss": 0.8161850144005243, "story_loss": 0.8161850144005243, "grad_norm": 0.16572533547878265, "lr": 0.00027188221139616304, "elapsed": 2813.3, "tokens": 30804, "story_tokens": 30804, "peak_gb": 74.9}
|
| 121 |
+
{"step": 121, "loss": 0.8503398242850658, "story_loss": 0.8503398242850658, "grad_norm": 0.1628333479166031, "lr": 0.000268522453742169, "elapsed": 2834.4, "tokens": 31808, "story_tokens": 31808, "peak_gb": 74.9}
|
| 122 |
+
{"step": 122, "loss": 0.8201501307724692, "story_loss": 0.8201501307724692, "grad_norm": 0.15862740576267242, "lr": 0.00026515932880343104, "elapsed": 2856.2, "tokens": 31621, "story_tokens": 31621, "peak_gb": 74.9}
|
| 123 |
+
{"step": 123, "loss": 0.8264033500460277, "story_loss": 0.8264033500460277, "grad_norm": 0.15998569130897522, "lr": 0.00026179344797838775, "elapsed": 2878.2, "tokens": 31333, "story_tokens": 31333, "peak_gb": 74.9}
|
| 124 |
+
{"step": 124, "loss": 0.8338664233951533, "story_loss": 0.8338664233951533, "grad_norm": 0.16547444462776184, "lr": 0.0002584254231664833, "elapsed": 2900.7, "tokens": 31031, "story_tokens": 31031, "peak_gb": 74.9}
|
| 125 |
+
{"step": 125, "loss": 0.8445863722862125, "story_loss": 0.8445863722862125, "grad_norm": 0.15795308351516724, "lr": 0.0002550558666569279, "elapsed": 2922.3, "tokens": 32047, "story_tokens": 32047, "peak_gb": 74.9}
|
| 126 |
+
{"step": 126, "loss": 0.8299679293878798, "story_loss": 0.8299679293878798, "grad_norm": 0.16400516033172607, "lr": 0.00025168539101738576, "elapsed": 2945.6, "tokens": 30377, "story_tokens": 30377, "peak_gb": 74.9}
|
| 127 |
+
{"step": 127, "loss": 0.8192306651278932, "story_loss": 0.8192306651278932, "grad_norm": 0.15796810388565063, "lr": 0.00024831460898261425, "elapsed": 2968.7, "tokens": 33915, "story_tokens": 33915, "peak_gb": 74.9}
|
| 128 |
+
{"step": 128, "loss": 0.840570077157033, "story_loss": 0.840570077157033, "grad_norm": 0.15880009531974792, "lr": 0.0002449441333430722, "elapsed": 2991.6, "tokens": 32607, "story_tokens": 32607, "peak_gb": 75.1}
|
| 129 |
+
{"step": 129, "loss": 0.818232370776511, "story_loss": 0.818232370776511, "grad_norm": 0.1491648554801941, "lr": 0.00024157457683351668, "elapsed": 3014.0, "tokens": 31916, "story_tokens": 31916, "peak_gb": 75.1}
|
| 130 |
+
{"step": 130, "loss": 0.8145418855782453, "story_loss": 0.8145418855782453, "grad_norm": 0.15487952530384064, "lr": 0.00023820655202161237, "elapsed": 3036.2, "tokens": 32363, "story_tokens": 32363, "peak_gb": 75.1}
|
| 131 |
+
{"step": 131, "loss": 0.8098891863076232, "story_loss": 0.8098891863076232, "grad_norm": 0.16136087477207184, "lr": 0.00023484067119656905, "elapsed": 3056.6, "tokens": 27951, "story_tokens": 27951, "peak_gb": 75.1}
|
| 132 |
+
{"step": 132, "loss": 0.8248288684891022, "story_loss": 0.8248288684891022, "grad_norm": 0.15285231173038483, "lr": 0.000231477546257831, "elapsed": 3078.7, "tokens": 32472, "story_tokens": 32472, "peak_gb": 75.1}
|
| 133 |
+
{"step": 133, "loss": 0.8288207746579792, "story_loss": 0.8288207746579792, "grad_norm": 0.15982721745967865, "lr": 0.000228117788603837, "elapsed": 3100.7, "tokens": 32710, "story_tokens": 32710, "peak_gb": 75.1}
|
| 134 |
+
{"step": 134, "loss": 0.853922714272759, "story_loss": 0.853922714272759, "grad_norm": 0.1610126793384552, "lr": 0.00022476200902087098, "elapsed": 3121.6, "tokens": 29075, "story_tokens": 29075, "peak_gb": 75.1}
|
| 135 |
+
{"step": 135, "loss": 0.8258953550027786, "story_loss": 0.8258953550027786, "grad_norm": 0.15791746973991394, "lr": 0.00022141081757202458, "elapsed": 3143.3, "tokens": 30816, "story_tokens": 30816, "peak_gb": 75.1}
|
| 136 |
+
{"step": 136, "loss": 0.8491072523877062, "story_loss": 0.8491072523877062, "grad_norm": 0.15472227334976196, "lr": 0.00021806482348629065, "elapsed": 3165.4, "tokens": 32543, "story_tokens": 32543, "peak_gb": 75.1}
|
| 137 |
+
{"step": 137, "loss": 0.8321887789336647, "story_loss": 0.8321887789336647, "grad_norm": 0.1506691575050354, "lr": 0.00021472463504780892, "elapsed": 3188.9, "tokens": 34254, "story_tokens": 34254, "peak_gb": 75.1}
|
| 138 |
+
{"step": 138, "loss": 0.8021191239458796, "story_loss": 0.8021191239458796, "grad_norm": 0.1504116654396057, "lr": 0.00021139085948528285, "elapsed": 3210.2, "tokens": 31617, "story_tokens": 31617, "peak_gb": 75.1}
|
| 139 |
+
{"step": 139, "loss": 0.812900629931748, "story_loss": 0.812900629931748, "grad_norm": 0.14717522263526917, "lr": 0.00020806410286158879, "elapsed": 3232.1, "tokens": 32728, "story_tokens": 32728, "peak_gb": 75.1}
|
| 140 |
+
{"step": 140, "loss": 0.8261819360595819, "story_loss": 0.8261819360595819, "grad_norm": 0.16309137642383575, "lr": 0.00020474496996359672, "elapsed": 3253.7, "tokens": 31485, "story_tokens": 31485, "peak_gb": 75.1}
|
| 141 |
+
{"step": 141, "loss": 0.8406954497933644, "story_loss": 0.8406954497933644, "grad_norm": 0.15436477959156036, "lr": 0.00020143406419222354, "elapsed": 3274.7, "tokens": 31169, "story_tokens": 31169, "peak_gb": 75.1}
|
| 142 |
+
{"step": 142, "loss": 0.8270587615365268, "story_loss": 0.8270587615365268, "grad_norm": 0.15643204748630524, "lr": 0.00019813198745273798, "elapsed": 3299.0, "tokens": 30440, "story_tokens": 30440, "peak_gb": 75.1}
|
| 143 |
+
{"step": 143, "loss": 0.8160428119301851, "story_loss": 0.8160428119301851, "grad_norm": 0.15367500483989716, "lr": 0.00019483934004533665, "elapsed": 3321.5, "tokens": 32357, "story_tokens": 32357, "peak_gb": 75.1}
|
| 144 |
+
{"step": 144, "loss": 0.8290335853321581, "story_loss": 0.8290335853321581, "grad_norm": 0.15591169893741608, "lr": 0.00019155672055601363, "elapsed": 3343.2, "tokens": 31555, "story_tokens": 31555, "peak_gb": 75.1}
|
| 145 |
+
{"step": 145, "loss": 0.8494631240980567, "story_loss": 0.8494631240980567, "grad_norm": 0.16318614780902863, "lr": 0.00018828472574773978, "elapsed": 3364.4, "tokens": 31789, "story_tokens": 31789, "peak_gb": 75.1}
|
| 146 |
+
{"step": 146, "loss": 0.8030804308881698, "story_loss": 0.8030804308881698, "grad_norm": 0.14761114120483398, "lr": 0.00018502395045197521, "elapsed": 3386.8, "tokens": 33087, "story_tokens": 33087, "peak_gb": 75.1}
|
| 147 |
+
{"step": 147, "loss": 0.806679593721613, "story_loss": 0.806679593721613, "grad_norm": 0.15984342992305756, "lr": 0.00018177498746053128, "elapsed": 3408.5, "tokens": 30824, "story_tokens": 30824, "peak_gb": 75.1}
|
| 148 |
+
{"step": 148, "loss": 0.8184624520075486, "story_loss": 0.8184624520075486, "grad_norm": 0.16973887383937836, "lr": 0.00017853842741780474, "elapsed": 3429.9, "tokens": 30734, "story_tokens": 30734, "peak_gb": 75.1}
|
| 149 |
+
{"step": 149, "loss": 0.8414989973515663, "story_loss": 0.8414989973515663, "grad_norm": 0.1570214033126831, "lr": 0.00017531485871340123, "elapsed": 3451.9, "tokens": 30885, "story_tokens": 30885, "peak_gb": 75.1}
|
| 150 |
+
{"step": 150, "loss": 0.8181908567108737, "story_loss": 0.8181908567108737, "grad_norm": 0.14957743883132935, "lr": 0.00017210486737516946, "elapsed": 3472.7, "tokens": 30967, "story_tokens": 30967, "peak_gb": 75.1}
|
| 151 |
+
{"step": 151, "loss": 0.8359500672964129, "story_loss": 0.8359500672964129, "grad_norm": 0.16320033371448517, "lr": 0.00016890903696266447, "elapsed": 3494.2, "tokens": 30529, "story_tokens": 30529, "peak_gb": 75.1}
|
| 152 |
+
{"step": 152, "loss": 0.8284014116786101, "story_loss": 0.8284014116786101, "grad_norm": 0.15387308597564697, "lr": 0.00016572794846105918, "elapsed": 3515.6, "tokens": 30927, "story_tokens": 30927, "peak_gb": 75.1}
|
| 153 |
+
{"step": 153, "loss": 0.812258456799906, "story_loss": 0.812258456799906, "grad_norm": 0.1577296257019043, "lr": 0.00016256218017552483, "elapsed": 3535.7, "tokens": 28198, "story_tokens": 28198, "peak_gb": 75.1}
|
| 154 |
+
{"step": 154, "loss": 0.8091100766706478, "story_loss": 0.8091100766706478, "grad_norm": 0.15075038373470306, "lr": 0.0001594123076260975, "elapsed": 3557.6, "tokens": 32392, "story_tokens": 32392, "peak_gb": 75.1}
|
| 155 |
+
{"step": 155, "loss": 0.829327352023204, "story_loss": 0.829327352023204, "grad_norm": 0.14885210990905762, "lr": 0.00015627890344305253, "elapsed": 3580.7, "tokens": 33712, "story_tokens": 33712, "peak_gb": 75.1}
|
| 156 |
+
{"step": 156, "loss": 0.7927457107347343, "story_loss": 0.7927457107347343, "grad_norm": 0.14586782455444336, "lr": 0.0001531625372628025, "elapsed": 3603.1, "tokens": 32442, "story_tokens": 32442, "peak_gb": 75.1}
|
| 157 |
+
{"step": 157, "loss": 0.8155116893298947, "story_loss": 0.8155116893298947, "grad_norm": 0.14736567437648773, "lr": 0.00015006377562434047, "elapsed": 3625.1, "tokens": 32729, "story_tokens": 32729, "peak_gb": 75.1}
|
| 158 |
+
{"step": 158, "loss": 0.81808956881731, "story_loss": 0.81808956881731, "grad_norm": 0.15276367962360382, "lr": 0.00014698318186624667, "elapsed": 3645.4, "tokens": 29574, "story_tokens": 29574, "peak_gb": 75.1}
|
| 159 |
+
{"step": 159, "loss": 0.8285221924520098, "story_loss": 0.8285221924520098, "grad_norm": 0.16045473515987396, "lr": 0.00014392131602427587, "elapsed": 3666.5, "tokens": 31048, "story_tokens": 31048, "peak_gb": 75.1}
|
| 160 |
+
{"step": 160, "loss": 0.8414467429330263, "story_loss": 0.8414467429330263, "grad_norm": 0.15592032670974731, "lr": 0.00014087873472954637, "elapsed": 3686.8, "tokens": 28933, "story_tokens": 28933, "peak_gb": 75.1}
|
| 161 |
+
{"step": 161, "loss": 0.7984188691536043, "story_loss": 0.7984188691536043, "grad_norm": 0.14427590370178223, "lr": 0.00013785599110734698, "elapsed": 3709.7, "tokens": 33557, "story_tokens": 33557, "peak_gb": 75.1}
|
| 162 |
+
{"step": 162, "loss": 0.8392814362658778, "story_loss": 0.8392814362658778, "grad_norm": 0.16139009594917297, "lr": 0.00013485363467658184, "elapsed": 3731.8, "tokens": 31402, "story_tokens": 31402, "peak_gb": 75.1}
|
| 163 |
+
{"step": 163, "loss": 0.8285839798195305, "story_loss": 0.8285839798195305, "grad_norm": 0.1566365659236908, "lr": 0.00013187221124987106, "elapsed": 3752.5, "tokens": 30012, "story_tokens": 30012, "peak_gb": 75.1}
|
| 164 |
+
{"step": 164, "loss": 0.843098571976225, "story_loss": 0.843098571976225, "grad_norm": 0.15432563424110413, "lr": 0.0001289122628343244, "elapsed": 3773.0, "tokens": 28541, "story_tokens": 28541, "peak_gb": 75.1}
|
| 165 |
+
{"step": 165, "loss": 0.8129613745036391, "story_loss": 0.8129613745036391, "grad_norm": 0.14639072120189667, "lr": 0.00012597432753300752, "elapsed": 3794.1, "tokens": 30519, "story_tokens": 30519, "peak_gb": 75.1}
|
| 166 |
+
{"step": 166, "loss": 0.8231595579874493, "story_loss": 0.8231595579874493, "grad_norm": 0.15428146719932556, "lr": 0.0001230589394471177, "elapsed": 3814.8, "tokens": 30616, "story_tokens": 30616, "peak_gb": 75.1}
|
| 167 |
+
{"step": 167, "loss": 0.8065462146884952, "story_loss": 0.8065462146884952, "grad_norm": 0.14425522089004517, "lr": 0.00012016662857888689, "elapsed": 3836.6, "tokens": 32658, "story_tokens": 32658, "peak_gb": 75.1}
|
| 168 |
+
{"step": 168, "loss": 0.7960255661668589, "story_loss": 0.7960255661668589, "grad_norm": 0.15625114738941193, "lr": 0.00011729792073522999, "elapsed": 3857.3, "tokens": 28066, "story_tokens": 28066, "peak_gb": 75.1}
|
| 169 |
+
{"step": 169, "loss": 0.7963461104873301, "story_loss": 0.7963461104873301, "grad_norm": 0.15370775759220123, "lr": 0.00011445333743215647, "elapsed": 3881.0, "tokens": 32814, "story_tokens": 32814, "peak_gb": 75.3}
|
| 170 |
+
{"step": 170, "loss": 0.8346468968884698, "story_loss": 0.8346468968884698, "grad_norm": 0.16116227209568024, "lr": 0.00011163339579996079, "elapsed": 3901.9, "tokens": 29000, "story_tokens": 29000, "peak_gb": 75.3}
|
| 171 |
+
{"step": 171, "loss": 0.802194177595152, "story_loss": 0.802194177595152, "grad_norm": 0.14632494747638702, "lr": 0.00010883860848921121, "elapsed": 3924.2, "tokens": 31719, "story_tokens": 31719, "peak_gb": 75.3}
|
| 172 |
+
{"step": 172, "loss": 0.8225824334150357, "story_loss": 0.8225824334150357, "grad_norm": 0.1470327526330948, "lr": 0.00010606948357755269, "elapsed": 3947.6, "tokens": 33748, "story_tokens": 33748, "peak_gb": 75.3}
|
| 173 |
+
{"step": 173, "loss": 0.7989955788320305, "story_loss": 0.7989955788320305, "grad_norm": 0.1425153911113739, "lr": 0.00010332652447734057, "elapsed": 3970.2, "tokens": 33326, "story_tokens": 33326, "peak_gb": 75.3}
|
| 174 |
+
{"step": 174, "loss": 0.8065482881394078, "story_loss": 0.8065482881394078, "grad_norm": 0.14560505747795105, "lr": 0.0001006102298441236, "elapsed": 3992.0, "tokens": 31797, "story_tokens": 31797, "peak_gb": 75.3}
|
| 175 |
+
{"step": 175, "loss": 0.8050603795284647, "story_loss": 0.8050603795284647, "grad_norm": 0.14702221751213074, "lr": 9.792109348599035e-05, "elapsed": 4012.8, "tokens": 29563, "story_tokens": 29563, "peak_gb": 75.3}
|
| 176 |
+
{"step": 176, "loss": 0.7894448073916452, "story_loss": 0.7894448073916452, "grad_norm": 0.1424216628074646, "lr": 9.525960427379771e-05, "elapsed": 4035.6, "tokens": 33080, "story_tokens": 33080, "peak_gb": 75.3}
|
| 177 |
+
{"step": 177, "loss": 0.8074079848577306, "story_loss": 0.8074079848577306, "grad_norm": 0.14359505474567413, "lr": 9.262624605229672e-05, "elapsed": 4057.6, "tokens": 32724, "story_tokens": 32724, "peak_gb": 75.3}
|
| 178 |
+
{"step": 178, "loss": 0.8214007914236906, "story_loss": 0.8214007914236906, "grad_norm": 0.15947574377059937, "lr": 9.002149755217246e-05, "elapsed": 4077.9, "tokens": 29699, "story_tokens": 29699, "peak_gb": 75.3}
|
| 179 |
+
{"step": 179, "loss": 0.8199472318525062, "story_loss": 0.8199472318525062, "grad_norm": 0.15053321421146393, "lr": 8.744583230301248e-05, "elapsed": 4099.8, "tokens": 31981, "story_tokens": 31981, "peak_gb": 75.3}
|
| 180 |
+
{"step": 180, "loss": 0.7840781174589768, "story_loss": 0.7840781174589768, "grad_norm": 0.1405535340309143, "lr": 8.489971854722259e-05, "elapsed": 4121.7, "tokens": 32375, "story_tokens": 32375, "peak_gb": 75.3}
|
| 181 |
+
{"step": 181, "loss": 0.8180935825874439, "story_loss": 0.8180935825874439, "grad_norm": 0.15378281474113464, "lr": 8.238361915490225e-05, "elapsed": 4142.3, "tokens": 30298, "story_tokens": 30298, "peak_gb": 75.3}
|
| 182 |
+
{"step": 182, "loss": 0.8279150749570192, "story_loss": 0.8279150749570192, "grad_norm": 0.15389686822891235, "lr": 7.989799153969735e-05, "elapsed": 4163.5, "tokens": 29519, "story_tokens": 29519, "peak_gb": 75.3}
|
| 183 |
+
{"step": 183, "loss": 0.8079020934179663, "story_loss": 0.8079020934179663, "grad_norm": 0.14828629791736603, "lr": 7.7443287575645e-05, "elapsed": 4185.0, "tokens": 31388, "story_tokens": 31388, "peak_gb": 75.3}
|
| 184 |
+
{"step": 184, "loss": 0.8138239513209936, "story_loss": 0.8138239513209936, "grad_norm": 0.1447652280330658, "lr": 7.501995351502497e-05, "elapsed": 4207.1, "tokens": 32142, "story_tokens": 32142, "peak_gb": 75.3}
|
| 185 |
+
{"step": 185, "loss": 0.8191613723193287, "story_loss": 0.8191613723193287, "grad_norm": 0.14430025219917297, "lr": 7.26284299072334e-05, "elapsed": 4228.0, "tokens": 30295, "story_tokens": 30295, "peak_gb": 75.3}
|
| 186 |
+
{"step": 186, "loss": 0.7970507871985405, "story_loss": 0.7970507871985405, "grad_norm": 0.14170348644256592, "lr": 7.026915151869334e-05, "elapsed": 4249.3, "tokens": 31192, "story_tokens": 31192, "peak_gb": 75.3}
|
| 187 |
+
{"step": 187, "loss": 0.8143276183465685, "story_loss": 0.8143276183465685, "grad_norm": 0.14988189935684204, "lr": 6.794254725381641e-05, "elapsed": 4269.6, "tokens": 28916, "story_tokens": 28916, "peak_gb": 75.3}
|
| 188 |
+
{"step": 188, "loss": 0.8170063323882855, "story_loss": 0.8170063323882855, "grad_norm": 0.14316695928573608, "lr": 6.564904007703033e-05, "elapsed": 4292.1, "tokens": 33260, "story_tokens": 33260, "peak_gb": 75.3}
|
| 189 |
+
{"step": 189, "loss": 0.7896710142998743, "story_loss": 0.7896710142998743, "grad_norm": 0.1347511112689972, "lr": 6.33890469358861e-05, "elapsed": 4315.3, "tokens": 34018, "story_tokens": 34018, "peak_gb": 75.3}
|
| 190 |
+
{"step": 190, "loss": 0.8183290560045737, "story_loss": 0.8183290560045737, "grad_norm": 0.15093804895877838, "lr": 6.116297868525919e-05, "elapsed": 4336.5, "tokens": 29967, "story_tokens": 29967, "peak_gb": 75.3}
|
| 191 |
+
{"step": 191, "loss": 0.8174032214701417, "story_loss": 0.8174032214701417, "grad_norm": 0.1456390768289566, "lr": 5.8971240012658214e-05, "elapsed": 4357.5, "tokens": 30875, "story_tokens": 30875, "peak_gb": 75.3}
|
| 192 |
+
{"step": 192, "loss": 0.8204053529868115, "story_loss": 0.8204053529868115, "grad_norm": 0.14121761918067932, "lr": 5.681422936465522e-05, "elapsed": 4379.4, "tokens": 32225, "story_tokens": 32225, "peak_gb": 75.3}
|
| 193 |
+
{"step": 193, "loss": 0.8268390187197066, "story_loss": 0.8268390187197066, "grad_norm": 0.15899018943309784, "lr": 5.469233887444941e-05, "elapsed": 4400.2, "tokens": 30122, "story_tokens": 30122, "peak_gb": 75.3}
|
| 194 |
+
{"step": 194, "loss": 0.8398764808060037, "story_loss": 0.8398764808060037, "grad_norm": 0.15233221650123596, "lr": 5.260595429058021e-05, "elapsed": 4421.7, "tokens": 29825, "story_tokens": 29825, "peak_gb": 75.3}
|
| 195 |
+
{"step": 195, "loss": 0.8053547901447043, "story_loss": 0.8053547901447043, "grad_norm": 0.1465817242860794, "lr": 5.0555454906799804e-05, "elapsed": 4442.0, "tokens": 28934, "story_tokens": 28934, "peak_gb": 75.3}
|
| 196 |
+
{"step": 196, "loss": 0.819677208536214, "story_loss": 0.819677208536214, "grad_norm": 0.15290533006191254, "lr": 4.854121349311949e-05, "elapsed": 4463.1, "tokens": 29343, "story_tokens": 29343, "peak_gb": 75.3}
|
| 197 |
+
{"step": 197, "loss": 0.8239718970325662, "story_loss": 0.8239718970325662, "grad_norm": 0.14330638945102692, "lr": 4.656359622804243e-05, "elapsed": 4485.0, "tokens": 32764, "story_tokens": 32764, "peak_gb": 75.3}
|
| 198 |
+
{"step": 198, "loss": 0.8001786865521868, "story_loss": 0.8001786865521868, "grad_norm": 0.13923612236976624, "lr": 4.4622962631993805e-05, "elapsed": 4507.4, "tokens": 33119, "story_tokens": 33119, "peak_gb": 75.3}
|
| 199 |
+
{"step": 199, "loss": 0.8085144727180045, "story_loss": 0.8085144727180045, "grad_norm": 0.1417732685804367, "lr": 4.27196655019623e-05, "elapsed": 4530.7, "tokens": 34790, "story_tokens": 34790, "peak_gb": 75.3}
|
| 200 |
+
{"step": 200, "loss": 0.807645937286048, "story_loss": 0.807645937286048, "grad_norm": 0.1462358832359314, "lr": 4.085405084736296e-05, "elapsed": 4554.0, "tokens": 33594, "story_tokens": 33594, "peak_gb": 75.3}
|
| 201 |
+
{"step": 201, "loss": 0.7981554344786165, "story_loss": 0.7981554344786165, "grad_norm": 0.13822202384471893, "lr": 3.902645782713532e-05, "elapsed": 4576.6, "tokens": 34080, "story_tokens": 34080, "peak_gb": 75.3}
|
| 202 |
+
{"step": 202, "loss": 0.8155441409793858, "story_loss": 0.8155441409793858, "grad_norm": 0.14943869411945343, "lr": 3.723721868808533e-05, "elapsed": 4597.4, "tokens": 29761, "story_tokens": 29761, "peak_gb": 75.3}
|
| 203 |
+
{"step": 203, "loss": 0.795692268698225, "story_loss": 0.795692268698225, "grad_norm": 0.14673252403736115, "lr": 3.548665870448498e-05, "elapsed": 4618.8, "tokens": 31929, "story_tokens": 31929, "peak_gb": 75.3}
|
| 204 |
+
{"step": 204, "loss": 0.7995357556223556, "story_loss": 0.7995357556223556, "grad_norm": 0.14660429954528809, "lr": 3.377509611893903e-05, "elapsed": 4639.9, "tokens": 31198, "story_tokens": 31198, "peak_gb": 75.3}
|
| 205 |
+
{"step": 205, "loss": 0.8033241553909436, "story_loss": 0.8033241553909436, "grad_norm": 0.14906753599643707, "lr": 3.210284208453029e-05, "elapsed": 4662.6, "tokens": 32121, "story_tokens": 32121, "peak_gb": 75.3}
|
| 206 |
+
{"step": 206, "loss": 0.7849020031472825, "story_loss": 0.7849020031472825, "grad_norm": 0.14346401393413544, "lr": 3.047020060825359e-05, "elapsed": 4683.9, "tokens": 30994, "story_tokens": 30994, "peak_gb": 75.3}
|
| 207 |
+
{"step": 207, "loss": 0.8065823304494109, "story_loss": 0.8065823304494109, "grad_norm": 0.14862839877605438, "lr": 2.8877468495748765e-05, "elapsed": 4707.5, "tokens": 33651, "story_tokens": 33651, "peak_gb": 75.3}
|
| 208 |
+
{"step": 208, "loss": 0.8038492232406695, "story_loss": 0.8038492232406695, "grad_norm": 0.14947907626628876, "lr": 2.7324935297343146e-05, "elapsed": 4728.8, "tokens": 30880, "story_tokens": 30880, "peak_gb": 75.3}
|
| 209 |
+
{"step": 209, "loss": 0.7970084639358671, "story_loss": 0.7970084639358671, "grad_norm": 0.13851282000541687, "lr": 2.58128832554127e-05, "elapsed": 4751.6, "tokens": 33031, "story_tokens": 33031, "peak_gb": 75.3}
|
| 210 |
+
{"step": 210, "loss": 0.8071499428376915, "story_loss": 0.8071499428376915, "grad_norm": 0.14042727649211884, "lr": 2.4341587253072034e-05, "elapsed": 4773.5, "tokens": 32774, "story_tokens": 32774, "peak_gb": 75.3}
|
| 211 |
+
{"step": 211, "loss": 0.8115040279038468, "story_loss": 0.8115040279038468, "grad_norm": 0.14814059436321259, "lr": 2.2911314764201775e-05, "elapsed": 4796.0, "tokens": 31949, "story_tokens": 31949, "peak_gb": 75.3}
|
| 212 |
+
{"step": 212, "loss": 0.7982386256392449, "story_loss": 0.7982386256392449, "grad_norm": 0.1483047902584076, "lr": 2.1522325804823494e-05, "elapsed": 4817.1, "tokens": 29967, "story_tokens": 29967, "peak_gb": 75.3}
|
| 213 |
+
{"step": 213, "loss": 0.7947773207688188, "story_loss": 0.7947773207688188, "grad_norm": 0.1501215100288391, "lr": 2.0174872885830115e-05, "elapsed": 4838.8, "tokens": 31913, "story_tokens": 31913, "peak_gb": 75.3}
|
| 214 |
+
{"step": 214, "loss": 0.8052365968190951, "story_loss": 0.8052365968190951, "grad_norm": 0.14979587495326996, "lr": 1.8869200967080503e-05, "elapsed": 4859.1, "tokens": 30297, "story_tokens": 30297, "peak_gb": 75.3}
|
| 215 |
+
{"step": 215, "loss": 0.7834127519944187, "story_loss": 0.7834127519944187, "grad_norm": 0.13319745659828186, "lr": 1.7605547412867573e-05, "elapsed": 4882.4, "tokens": 34244, "story_tokens": 34244, "peak_gb": 75.3}
|
| 216 |
+
{"step": 216, "loss": 0.7951627311872362, "story_loss": 0.7951627311872362, "grad_norm": 0.1437256634235382, "lr": 1.6384141948766168e-05, "elapsed": 4903.4, "tokens": 31517, "story_tokens": 31517, "peak_gb": 75.3}
|
| 217 |
+
{"step": 217, "loss": 0.7891637659466425, "story_loss": 0.7891637659466425, "grad_norm": 0.14285698533058167, "lr": 1.5205206619870487e-05, "elapsed": 4925.5, "tokens": 31139, "story_tokens": 31139, "peak_gb": 75.3}
|
| 218 |
+
{"step": 218, "loss": 0.812235916063189, "story_loss": 0.812235916063189, "grad_norm": 0.14145144820213318, "lr": 1.406895575042727e-05, "elapsed": 4948.4, "tokens": 32563, "story_tokens": 32563, "peak_gb": 75.3}
|
| 219 |
+
{"step": 219, "loss": 0.7810120115226404, "story_loss": 0.7810120115226404, "grad_norm": 0.14130932092666626, "lr": 1.2975595904873073e-05, "elapsed": 4969.0, "tokens": 30874, "story_tokens": 30874, "peak_gb": 75.3}
|
| 220 |
+
{"step": 220, "loss": 0.8073108015298031, "story_loss": 0.8073108015298031, "grad_norm": 0.14159007370471954, "lr": 1.1925325850281416e-05, "elapsed": 4992.4, "tokens": 34598, "story_tokens": 34598, "peak_gb": 75.3}
|
| 221 |
+
{"step": 221, "loss": 0.8002349192119622, "story_loss": 0.8002349192119622, "grad_norm": 0.14387285709381104, "lr": 1.0918336520228472e-05, "elapsed": 5014.0, "tokens": 31053, "story_tokens": 31053, "peak_gb": 75.3}
|
| 222 |
+
{"step": 222, "loss": 0.8173973701013528, "story_loss": 0.8173973701013528, "grad_norm": 0.14693059027194977, "lr": 9.954810980082191e-06, "elapsed": 5034.7, "tokens": 29726, "story_tokens": 29726, "peak_gb": 75.3}
|
| 223 |
+
{"step": 223, "loss": 0.8227191798164981, "story_loss": 0.8227191798164981, "grad_norm": 0.14616630971431732, "lr": 9.034924393721777e-06, "elapsed": 5054.7, "tokens": 29373, "story_tokens": 29373, "peak_gb": 75.3}
|
| 224 |
+
{"step": 224, "loss": 0.8113699558768949, "story_loss": 0.8113699558768949, "grad_norm": 0.14560775458812714, "lr": 8.15884399169417e-06, "elapsed": 5076.4, "tokens": 31995, "story_tokens": 31995, "peak_gb": 75.3}
|
| 225 |
+
{"step": 225, "loss": 0.8362814163926438, "story_loss": 0.8362814163926438, "grad_norm": 0.1485193818807602, "lr": 7.326729040812136e-06, "elapsed": 5097.8, "tokens": 30202, "story_tokens": 30202, "peak_gb": 75.3}
|
| 226 |
+
{"step": 226, "loss": 0.8012813322419646, "story_loss": 0.8012813322419646, "grad_norm": 0.14298923313617706, "lr": 6.538730815200483e-06, "elapsed": 5119.4, "tokens": 32108, "story_tokens": 32108, "peak_gb": 75.3}
|
| 227 |
+
{"step": 227, "loss": 0.7904335984215449, "story_loss": 0.7904335984215449, "grad_norm": 0.13900890946388245, "lr": 5.7949925687952595e-06, "elapsed": 5141.1, "tokens": 32611, "story_tokens": 32611, "peak_gb": 75.3}
|
| 228 |
+
{"step": 228, "loss": 0.7812973751768235, "story_loss": 0.7812973751768235, "grad_norm": 0.1421976089477539, "lr": 5.095649509300804e-06, "elapsed": 5162.7, "tokens": 31237, "story_tokens": 31237, "peak_gb": 75.3}
|
| 229 |
+
{"step": 229, "loss": 0.820741418098716, "story_loss": 0.820741418098716, "grad_norm": 0.14505542814731598, "lr": 4.440828773609934e-06, "elapsed": 5183.7, "tokens": 30627, "story_tokens": 30627, "peak_gb": 75.3}
|
| 230 |
+
{"step": 230, "loss": 0.8444186608551428, "story_loss": 0.8444186608551428, "grad_norm": 0.15557672083377838, "lr": 3.830649404690939e-06, "elapsed": 5206.1, "tokens": 32803, "story_tokens": 32803, "peak_gb": 75.3}
|
| 231 |
+
{"step": 231, "loss": 0.780329878882891, "story_loss": 0.780329878882891, "grad_norm": 0.15183842182159424, "lr": 3.2652223299462214e-06, "elapsed": 5228.6, "tokens": 33947, "story_tokens": 33947, "peak_gb": 75.3}
|
| 232 |
+
{"step": 232, "loss": 0.7966818191119761, "story_loss": 0.7966818191119761, "grad_norm": 0.13679181039333344, "lr": 2.7446503410463174e-06, "elapsed": 5251.3, "tokens": 34360, "story_tokens": 34360, "peak_gb": 75.3}
|
| 233 |
+
{"step": 233, "loss": 0.796085482604858, "story_loss": 0.796085482604858, "grad_norm": 0.14182889461517334, "lr": 2.269028075242929e-06, "elapsed": 5274.0, "tokens": 32395, "story_tokens": 32395, "peak_gb": 75.3}
|
| 234 |
+
{"step": 234, "loss": 0.8057403469619134, "story_loss": 0.8057403469619134, "grad_norm": 0.15451966226100922, "lr": 1.8384419981642697e-06, "elapsed": 5296.5, "tokens": 32180, "story_tokens": 32180, "peak_gb": 75.3}
|
| 235 |
+
{"step": 235, "loss": 0.7817093736081221, "story_loss": 0.7817093736081221, "grad_norm": 0.14422394335269928, "lr": 1.452970388096192e-06, "elapsed": 5318.1, "tokens": 31713, "story_tokens": 31713, "peak_gb": 75.3}
|
| 236 |
+
{"step": 236, "loss": 0.8031574049055538, "story_loss": 0.8031574049055538, "grad_norm": 0.14445367455482483, "lr": 1.1126833217514899e-06, "elapsed": 5340.3, "tokens": 32181, "story_tokens": 32181, "peak_gb": 75.3}
|
| 237 |
+
{"step": 237, "loss": 0.8074227483188384, "story_loss": 0.8074227483188384, "grad_norm": 0.143574059009552, "lr": 8.176426615304766e-07, "elapsed": 5361.9, "tokens": 32037, "story_tokens": 32037, "peak_gb": 75.3}
|
| 238 |
+
{"step": 238, "loss": 0.7909704288813275, "story_loss": 0.7909704288813275, "grad_norm": 0.13992522656917572, "lr": 5.679020442745097e-07, "elapsed": 5383.8, "tokens": 32319, "story_tokens": 32319, "peak_gb": 75.3}
|
| 239 |
+
{"step": 239, "loss": 0.8005722246884314, "story_loss": 0.8005722246884314, "grad_norm": 0.14300888776779175, "lr": 3.635068715153178e-07, "elapsed": 5405.0, "tokens": 31203, "story_tokens": 31203, "peak_gb": 75.3}
|
| 240 |
+
{"step": 240, "loss": 0.7900971829200877, "story_loss": 0.7900971829200877, "grad_norm": 0.14540746808052063, "lr": 2.0449430122107537e-07, "elapsed": 5426.4, "tokens": 31515, "story_tokens": 31515, "peak_gb": 75.3}
|
| 241 |
+
{"step": 241, "loss": 0.8279874767808603, "story_loss": 0.8279874767808603, "grad_norm": 0.14622862637043, "lr": 9.08932410413621e-08, "elapsed": 5448.3, "tokens": 31914, "story_tokens": 31914, "peak_gb": 75.3}
|
| 242 |
+
{"step": 242, "loss": 0.8164044465604802, "story_loss": 0.8164044465604802, "grad_norm": 0.14237159490585327, "lr": 2.2724343051866437e-08, "elapsed": 5470.3, "tokens": 32490, "story_tokens": 32490, "peak_gb": 75.3}
|
| 243 |
+
{"step": 243, "loss": 0.7836830997213033, "story_loss": 0.7836830997213033, "grad_norm": 0.16696366667747498, "lr": 0.0, "elapsed": 5489.1, "tokens": 23278, "story_tokens": 23278, "peak_gb": 75.3}
|
e08-V/meta/train_meta.json
ADDED
|
@@ -0,0 +1,61 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"data": "/workspace/e08/r08/data/train_V.jsonl",
|
| 3 |
+
"data_sha256": "bf1156c71f6b707a6b61c5169c5586bc4face2854ee4f55f7df4deae366d3d05",
|
| 4 |
+
"n_examples": 7766,
|
| 5 |
+
"n_loss_tokens": 7629283,
|
| 6 |
+
"replay": null,
|
| 7 |
+
"replay_sha256": null,
|
| 8 |
+
"replay_rows": 0,
|
| 9 |
+
"replay_per_step": 0,
|
| 10 |
+
"replay_loss_tokens_seen": 0,
|
| 11 |
+
"lr": 0.0005,
|
| 12 |
+
"rank": 32,
|
| 13 |
+
"alpha": 64,
|
| 14 |
+
"dropout": 0.0,
|
| 15 |
+
"eff_bs": 32,
|
| 16 |
+
"micro_bs": 4,
|
| 17 |
+
"accum": 8,
|
| 18 |
+
"epochs": 1,
|
| 19 |
+
"steps": 243,
|
| 20 |
+
"warmup": 10,
|
| 21 |
+
"schedule": "linear warmup then cosine decay to 0",
|
| 22 |
+
"optimizer": "AdamW(0.9,0.999,eps=1e-8,wd=0)",
|
| 23 |
+
"grad_clip": 1.0,
|
| 24 |
+
"seed": 0,
|
| 25 |
+
"targets": [
|
| 26 |
+
"q_proj",
|
| 27 |
+
"k_proj",
|
| 28 |
+
"v_proj",
|
| 29 |
+
"o_proj",
|
| 30 |
+
"in_proj_qkv",
|
| 31 |
+
"in_proj_z",
|
| 32 |
+
"in_proj_a",
|
| 33 |
+
"in_proj_b",
|
| 34 |
+
"out_proj",
|
| 35 |
+
"gate_proj",
|
| 36 |
+
"up_proj",
|
| 37 |
+
"down_proj"
|
| 38 |
+
],
|
| 39 |
+
"model": "Qwen/Qwen3.6-27B",
|
| 40 |
+
"revision": "6a9e13bd6fc8f0983b9b99948120bc37f49c13e9",
|
| 41 |
+
"model_dir": "/dev/shm/qwen36_27b",
|
| 42 |
+
"chat_kw": {
|
| 43 |
+
"enable_thinking": false
|
| 44 |
+
},
|
| 45 |
+
"loss_mask": "assistant tokens + <|im_end|>; prompt, empty think block, trailing newline masked",
|
| 46 |
+
"loss_norm": "sum CE over supervised tokens in the optimizer step / their count",
|
| 47 |
+
"torch": "2.14.1+cu130",
|
| 48 |
+
"gradient_checkpointing": true,
|
| 49 |
+
"pack_tok": 0,
|
| 50 |
+
"first_order_idx": [
|
| 51 |
+
4176,
|
| 52 |
+
1178,
|
| 53 |
+
4262,
|
| 54 |
+
4708,
|
| 55 |
+
3529,
|
| 56 |
+
2111,
|
| 57 |
+
6631,
|
| 58 |
+
7281
|
| 59 |
+
],
|
| 60 |
+
"trainable_params": 233455616
|
| 61 |
+
}
|