Add files using upload-large-folder tool
Browse files- runs/fixed_internvl/removal/Random8_8108_s13/adapter/README.md +207 -0
- runs/fixed_internvl/removal/Random8_8108_s13/adapter/adapter_config.json +213 -0
- runs/fixed_internvl/removal/Random8_8108_s13/config.json +43 -0
- runs/fixed_internvl/removal/Random8_8108_s13/steps.jsonl +250 -0
- runs/fixed_internvl/removal/Random8_8109_s13/adapter/README.md +207 -0
- runs/fixed_internvl/removal/Random8_8109_s13/adapter/adapter_config.json +213 -0
- runs/fixed_internvl/removal/Random8_8109_s13/config.json +43 -0
- runs/fixed_internvl/removal/Random8_8109_s13/steps.jsonl +250 -0
- runs/fixed_internvl/removal/Shallow8_s42/adapter/README.md +207 -0
- runs/fixed_internvl/removal/Shallow8_s42/adapter/adapter_config.json +213 -0
- runs/fixed_internvl/removal/Shallow8_s42/config.json +43 -0
- runs/fixed_internvl/removal/Shallow8_s42/steps.jsonl +250 -0
- runs/fixed_internvl/removal/Shallow8_s73/adapter/README.md +207 -0
- runs/fixed_internvl/removal/Shallow8_s73/adapter/adapter_config.json +213 -0
- runs/fixed_internvl/removal/Shallow8_s73/config.json +43 -0
- runs/fixed_internvl/removal/Shallow8_s73/steps.jsonl +250 -0
- runs/fixed_internvl/removal/Top8_s42/adapter/README.md +207 -0
- runs/fixed_internvl/removal/Top8_s42/adapter/adapter_config.json +213 -0
- runs/fixed_internvl/removal/Top8_s42/config.json +43 -0
- runs/fixed_internvl/removal/Top8_s42/steps.jsonl +250 -0
- runs/fixed_internvl/removal/Top8_s73/adapter/README.md +207 -0
- runs/fixed_internvl/removal/Top8_s73/adapter/adapter_config.json +213 -0
- runs/fixed_internvl/removal/Top8_s73/config.json +43 -0
- runs/fixed_internvl/removal/Top8_s73/steps.jsonl +250 -0
- runs/fixed_internvl/removal/TopDelta8_s42/adapter/README.md +207 -0
- runs/fixed_internvl/removal/TopDelta8_s42/adapter/adapter_config.json +213 -0
- runs/fixed_internvl/removal/TopDelta8_s42/config.json +43 -0
- runs/fixed_internvl/removal/TopDelta8_s42/steps.jsonl +250 -0
- runs/fixed_internvl/removal/TopDelta8_s73/adapter/README.md +207 -0
- runs/fixed_internvl/removal/TopDelta8_s73/adapter/adapter_config.json +213 -0
- runs/joint_qwen/joint/seed13/adapter/README.md +207 -0
- runs/joint_qwen/joint/seed13/adapter/adapter_config.json +55 -0
- runs/joint_qwen/joint/seed13/config.json +32 -0
- runs/joint_qwen/joint/seed13/steps.jsonl +500 -0
- runs/joint_qwen/joint/seed42/adapter/README.md +207 -0
- runs/joint_qwen/joint/seed42/adapter/adapter_config.json +55 -0
- runs/joint_qwen/joint/seed42/adapter/adapter_model.safetensors +3 -0
- runs/joint_qwen/joint/seed42/config.json +32 -0
- runs/joint_qwen/joint/seed42/steps.jsonl +500 -0
- runs/joint_qwen/joint/seed73/config.json +32 -0
- runs/joint_qwen/joint/seed73/steps.jsonl +500 -0
runs/fixed_internvl/removal/Random8_8108_s13/adapter/README.md
ADDED
|
@@ -0,0 +1,207 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
base_model: /data/OPSD/assets/models/InternVL3_5-8B-HF
|
| 3 |
+
library_name: peft
|
| 4 |
+
pipeline_tag: text-generation
|
| 5 |
+
tags:
|
| 6 |
+
- base_model:adapter:/data/OPSD/assets/models/InternVL3_5-8B-HF
|
| 7 |
+
- lora
|
| 8 |
+
- transformers
|
| 9 |
+
---
|
| 10 |
+
|
| 11 |
+
# Model Card for Model ID
|
| 12 |
+
|
| 13 |
+
<!-- Provide a quick summary of what the model is/does. -->
|
| 14 |
+
|
| 15 |
+
|
| 16 |
+
|
| 17 |
+
## Model Details
|
| 18 |
+
|
| 19 |
+
### Model Description
|
| 20 |
+
|
| 21 |
+
<!-- Provide a longer summary of what this model is. -->
|
| 22 |
+
|
| 23 |
+
|
| 24 |
+
|
| 25 |
+
- **Developed by:** [More Information Needed]
|
| 26 |
+
- **Funded by [optional]:** [More Information Needed]
|
| 27 |
+
- **Shared by [optional]:** [More Information Needed]
|
| 28 |
+
- **Model type:** [More Information Needed]
|
| 29 |
+
- **Language(s) (NLP):** [More Information Needed]
|
| 30 |
+
- **License:** [More Information Needed]
|
| 31 |
+
- **Finetuned from model [optional]:** [More Information Needed]
|
| 32 |
+
|
| 33 |
+
### Model Sources [optional]
|
| 34 |
+
|
| 35 |
+
<!-- Provide the basic links for the model. -->
|
| 36 |
+
|
| 37 |
+
- **Repository:** [More Information Needed]
|
| 38 |
+
- **Paper [optional]:** [More Information Needed]
|
| 39 |
+
- **Demo [optional]:** [More Information Needed]
|
| 40 |
+
|
| 41 |
+
## Uses
|
| 42 |
+
|
| 43 |
+
<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
|
| 44 |
+
|
| 45 |
+
### Direct Use
|
| 46 |
+
|
| 47 |
+
<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
|
| 48 |
+
|
| 49 |
+
[More Information Needed]
|
| 50 |
+
|
| 51 |
+
### Downstream Use [optional]
|
| 52 |
+
|
| 53 |
+
<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
|
| 54 |
+
|
| 55 |
+
[More Information Needed]
|
| 56 |
+
|
| 57 |
+
### Out-of-Scope Use
|
| 58 |
+
|
| 59 |
+
<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
|
| 60 |
+
|
| 61 |
+
[More Information Needed]
|
| 62 |
+
|
| 63 |
+
## Bias, Risks, and Limitations
|
| 64 |
+
|
| 65 |
+
<!-- This section is meant to convey both technical and sociotechnical limitations. -->
|
| 66 |
+
|
| 67 |
+
[More Information Needed]
|
| 68 |
+
|
| 69 |
+
### Recommendations
|
| 70 |
+
|
| 71 |
+
<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
|
| 72 |
+
|
| 73 |
+
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
|
| 74 |
+
|
| 75 |
+
## How to Get Started with the Model
|
| 76 |
+
|
| 77 |
+
Use the code below to get started with the model.
|
| 78 |
+
|
| 79 |
+
[More Information Needed]
|
| 80 |
+
|
| 81 |
+
## Training Details
|
| 82 |
+
|
| 83 |
+
### Training Data
|
| 84 |
+
|
| 85 |
+
<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
|
| 86 |
+
|
| 87 |
+
[More Information Needed]
|
| 88 |
+
|
| 89 |
+
### Training Procedure
|
| 90 |
+
|
| 91 |
+
<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
|
| 92 |
+
|
| 93 |
+
#### Preprocessing [optional]
|
| 94 |
+
|
| 95 |
+
[More Information Needed]
|
| 96 |
+
|
| 97 |
+
|
| 98 |
+
#### Training Hyperparameters
|
| 99 |
+
|
| 100 |
+
- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
|
| 101 |
+
|
| 102 |
+
#### Speeds, Sizes, Times [optional]
|
| 103 |
+
|
| 104 |
+
<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
|
| 105 |
+
|
| 106 |
+
[More Information Needed]
|
| 107 |
+
|
| 108 |
+
## Evaluation
|
| 109 |
+
|
| 110 |
+
<!-- This section describes the evaluation protocols and provides the results. -->
|
| 111 |
+
|
| 112 |
+
### Testing Data, Factors & Metrics
|
| 113 |
+
|
| 114 |
+
#### Testing Data
|
| 115 |
+
|
| 116 |
+
<!-- This should link to a Dataset Card if possible. -->
|
| 117 |
+
|
| 118 |
+
[More Information Needed]
|
| 119 |
+
|
| 120 |
+
#### Factors
|
| 121 |
+
|
| 122 |
+
<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
|
| 123 |
+
|
| 124 |
+
[More Information Needed]
|
| 125 |
+
|
| 126 |
+
#### Metrics
|
| 127 |
+
|
| 128 |
+
<!-- These are the evaluation metrics being used, ideally with a description of why. -->
|
| 129 |
+
|
| 130 |
+
[More Information Needed]
|
| 131 |
+
|
| 132 |
+
### Results
|
| 133 |
+
|
| 134 |
+
[More Information Needed]
|
| 135 |
+
|
| 136 |
+
#### Summary
|
| 137 |
+
|
| 138 |
+
|
| 139 |
+
|
| 140 |
+
## Model Examination [optional]
|
| 141 |
+
|
| 142 |
+
<!-- Relevant interpretability work for the model goes here -->
|
| 143 |
+
|
| 144 |
+
[More Information Needed]
|
| 145 |
+
|
| 146 |
+
## Environmental Impact
|
| 147 |
+
|
| 148 |
+
<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
|
| 149 |
+
|
| 150 |
+
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
|
| 151 |
+
|
| 152 |
+
- **Hardware Type:** [More Information Needed]
|
| 153 |
+
- **Hours used:** [More Information Needed]
|
| 154 |
+
- **Cloud Provider:** [More Information Needed]
|
| 155 |
+
- **Compute Region:** [More Information Needed]
|
| 156 |
+
- **Carbon Emitted:** [More Information Needed]
|
| 157 |
+
|
| 158 |
+
## Technical Specifications [optional]
|
| 159 |
+
|
| 160 |
+
### Model Architecture and Objective
|
| 161 |
+
|
| 162 |
+
[More Information Needed]
|
| 163 |
+
|
| 164 |
+
### Compute Infrastructure
|
| 165 |
+
|
| 166 |
+
[More Information Needed]
|
| 167 |
+
|
| 168 |
+
#### Hardware
|
| 169 |
+
|
| 170 |
+
[More Information Needed]
|
| 171 |
+
|
| 172 |
+
#### Software
|
| 173 |
+
|
| 174 |
+
[More Information Needed]
|
| 175 |
+
|
| 176 |
+
## Citation [optional]
|
| 177 |
+
|
| 178 |
+
<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
|
| 179 |
+
|
| 180 |
+
**BibTeX:**
|
| 181 |
+
|
| 182 |
+
[More Information Needed]
|
| 183 |
+
|
| 184 |
+
**APA:**
|
| 185 |
+
|
| 186 |
+
[More Information Needed]
|
| 187 |
+
|
| 188 |
+
## Glossary [optional]
|
| 189 |
+
|
| 190 |
+
<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
|
| 191 |
+
|
| 192 |
+
[More Information Needed]
|
| 193 |
+
|
| 194 |
+
## More Information [optional]
|
| 195 |
+
|
| 196 |
+
[More Information Needed]
|
| 197 |
+
|
| 198 |
+
## Model Card Authors [optional]
|
| 199 |
+
|
| 200 |
+
[More Information Needed]
|
| 201 |
+
|
| 202 |
+
## Model Card Contact
|
| 203 |
+
|
| 204 |
+
[More Information Needed]
|
| 205 |
+
### Framework versions
|
| 206 |
+
|
| 207 |
+
- PEFT 0.20.0
|
runs/fixed_internvl/removal/Random8_8108_s13/adapter/adapter_config.json
ADDED
|
@@ -0,0 +1,213 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"alora_invocation_tokens": null,
|
| 3 |
+
"alpha_pattern": {
|
| 4 |
+
"model.language_model.layers.13.mlp.down_proj": 144,
|
| 5 |
+
"model.language_model.layers.13.mlp.gate_proj": 144,
|
| 6 |
+
"model.language_model.layers.13.mlp.up_proj": 144,
|
| 7 |
+
"model.language_model.layers.13.self_attn.k_proj": 144,
|
| 8 |
+
"model.language_model.layers.13.self_attn.o_proj": 144,
|
| 9 |
+
"model.language_model.layers.13.self_attn.q_proj": 144,
|
| 10 |
+
"model.language_model.layers.13.self_attn.v_proj": 144,
|
| 11 |
+
"model.language_model.layers.16.mlp.down_proj": 144,
|
| 12 |
+
"model.language_model.layers.16.mlp.gate_proj": 144,
|
| 13 |
+
"model.language_model.layers.16.mlp.up_proj": 144,
|
| 14 |
+
"model.language_model.layers.16.self_attn.k_proj": 144,
|
| 15 |
+
"model.language_model.layers.16.self_attn.o_proj": 144,
|
| 16 |
+
"model.language_model.layers.16.self_attn.q_proj": 144,
|
| 17 |
+
"model.language_model.layers.16.self_attn.v_proj": 144,
|
| 18 |
+
"model.language_model.layers.22.mlp.down_proj": 144,
|
| 19 |
+
"model.language_model.layers.22.mlp.gate_proj": 144,
|
| 20 |
+
"model.language_model.layers.22.mlp.up_proj": 144,
|
| 21 |
+
"model.language_model.layers.22.self_attn.k_proj": 144,
|
| 22 |
+
"model.language_model.layers.22.self_attn.o_proj": 144,
|
| 23 |
+
"model.language_model.layers.22.self_attn.q_proj": 144,
|
| 24 |
+
"model.language_model.layers.22.self_attn.v_proj": 144,
|
| 25 |
+
"model.language_model.layers.28.mlp.down_proj": 144,
|
| 26 |
+
"model.language_model.layers.28.mlp.gate_proj": 144,
|
| 27 |
+
"model.language_model.layers.28.mlp.up_proj": 144,
|
| 28 |
+
"model.language_model.layers.28.self_attn.k_proj": 144,
|
| 29 |
+
"model.language_model.layers.28.self_attn.o_proj": 144,
|
| 30 |
+
"model.language_model.layers.28.self_attn.q_proj": 144,
|
| 31 |
+
"model.language_model.layers.28.self_attn.v_proj": 144,
|
| 32 |
+
"model.language_model.layers.32.mlp.down_proj": 144,
|
| 33 |
+
"model.language_model.layers.32.mlp.gate_proj": 144,
|
| 34 |
+
"model.language_model.layers.32.mlp.up_proj": 144,
|
| 35 |
+
"model.language_model.layers.32.self_attn.k_proj": 144,
|
| 36 |
+
"model.language_model.layers.32.self_attn.o_proj": 144,
|
| 37 |
+
"model.language_model.layers.32.self_attn.q_proj": 144,
|
| 38 |
+
"model.language_model.layers.32.self_attn.v_proj": 144,
|
| 39 |
+
"model.language_model.layers.34.mlp.down_proj": 144,
|
| 40 |
+
"model.language_model.layers.34.mlp.gate_proj": 144,
|
| 41 |
+
"model.language_model.layers.34.mlp.up_proj": 144,
|
| 42 |
+
"model.language_model.layers.34.self_attn.k_proj": 144,
|
| 43 |
+
"model.language_model.layers.34.self_attn.o_proj": 144,
|
| 44 |
+
"model.language_model.layers.34.self_attn.q_proj": 144,
|
| 45 |
+
"model.language_model.layers.34.self_attn.v_proj": 144,
|
| 46 |
+
"model.language_model.layers.4.mlp.down_proj": 144,
|
| 47 |
+
"model.language_model.layers.4.mlp.gate_proj": 144,
|
| 48 |
+
"model.language_model.layers.4.mlp.up_proj": 144,
|
| 49 |
+
"model.language_model.layers.4.self_attn.k_proj": 144,
|
| 50 |
+
"model.language_model.layers.4.self_attn.o_proj": 144,
|
| 51 |
+
"model.language_model.layers.4.self_attn.q_proj": 144,
|
| 52 |
+
"model.language_model.layers.4.self_attn.v_proj": 144,
|
| 53 |
+
"model.language_model.layers.6.mlp.down_proj": 144,
|
| 54 |
+
"model.language_model.layers.6.mlp.gate_proj": 144,
|
| 55 |
+
"model.language_model.layers.6.mlp.up_proj": 144,
|
| 56 |
+
"model.language_model.layers.6.self_attn.k_proj": 144,
|
| 57 |
+
"model.language_model.layers.6.self_attn.o_proj": 144,
|
| 58 |
+
"model.language_model.layers.6.self_attn.q_proj": 144,
|
| 59 |
+
"model.language_model.layers.6.self_attn.v_proj": 144
|
| 60 |
+
},
|
| 61 |
+
"arrow_config": null,
|
| 62 |
+
"auto_mapping": null,
|
| 63 |
+
"base_model_name_or_path": "/data/OPSD/assets/models/InternVL3_5-8B-HF",
|
| 64 |
+
"bias": "none",
|
| 65 |
+
"corda_config": null,
|
| 66 |
+
"ensure_weight_tying": false,
|
| 67 |
+
"eva_config": null,
|
| 68 |
+
"exclude_modules": null,
|
| 69 |
+
"fan_in_fan_out": false,
|
| 70 |
+
"inference_mode": true,
|
| 71 |
+
"init_lora_weights": true,
|
| 72 |
+
"layer_replication": null,
|
| 73 |
+
"layers_pattern": null,
|
| 74 |
+
"layers_to_transform": null,
|
| 75 |
+
"loftq_config": {},
|
| 76 |
+
"lora_alpha": 32,
|
| 77 |
+
"lora_bias": false,
|
| 78 |
+
"lora_dropout": 0.05,
|
| 79 |
+
"lora_ga_config": null,
|
| 80 |
+
"megatron_config": null,
|
| 81 |
+
"megatron_core": "megatron.core",
|
| 82 |
+
"modules_to_save": null,
|
| 83 |
+
"monteclora_config": null,
|
| 84 |
+
"peft_type": "LORA",
|
| 85 |
+
"peft_version": "0.20.0",
|
| 86 |
+
"qalora_group_size": 16,
|
| 87 |
+
"r": 16,
|
| 88 |
+
"rank_pattern": {
|
| 89 |
+
"model.language_model.layers.13.mlp.down_proj": 72,
|
| 90 |
+
"model.language_model.layers.13.mlp.gate_proj": 72,
|
| 91 |
+
"model.language_model.layers.13.mlp.up_proj": 72,
|
| 92 |
+
"model.language_model.layers.13.self_attn.k_proj": 72,
|
| 93 |
+
"model.language_model.layers.13.self_attn.o_proj": 72,
|
| 94 |
+
"model.language_model.layers.13.self_attn.q_proj": 72,
|
| 95 |
+
"model.language_model.layers.13.self_attn.v_proj": 72,
|
| 96 |
+
"model.language_model.layers.16.mlp.down_proj": 72,
|
| 97 |
+
"model.language_model.layers.16.mlp.gate_proj": 72,
|
| 98 |
+
"model.language_model.layers.16.mlp.up_proj": 72,
|
| 99 |
+
"model.language_model.layers.16.self_attn.k_proj": 72,
|
| 100 |
+
"model.language_model.layers.16.self_attn.o_proj": 72,
|
| 101 |
+
"model.language_model.layers.16.self_attn.q_proj": 72,
|
| 102 |
+
"model.language_model.layers.16.self_attn.v_proj": 72,
|
| 103 |
+
"model.language_model.layers.22.mlp.down_proj": 72,
|
| 104 |
+
"model.language_model.layers.22.mlp.gate_proj": 72,
|
| 105 |
+
"model.language_model.layers.22.mlp.up_proj": 72,
|
| 106 |
+
"model.language_model.layers.22.self_attn.k_proj": 72,
|
| 107 |
+
"model.language_model.layers.22.self_attn.o_proj": 72,
|
| 108 |
+
"model.language_model.layers.22.self_attn.q_proj": 72,
|
| 109 |
+
"model.language_model.layers.22.self_attn.v_proj": 72,
|
| 110 |
+
"model.language_model.layers.28.mlp.down_proj": 72,
|
| 111 |
+
"model.language_model.layers.28.mlp.gate_proj": 72,
|
| 112 |
+
"model.language_model.layers.28.mlp.up_proj": 72,
|
| 113 |
+
"model.language_model.layers.28.self_attn.k_proj": 72,
|
| 114 |
+
"model.language_model.layers.28.self_attn.o_proj": 72,
|
| 115 |
+
"model.language_model.layers.28.self_attn.q_proj": 72,
|
| 116 |
+
"model.language_model.layers.28.self_attn.v_proj": 72,
|
| 117 |
+
"model.language_model.layers.32.mlp.down_proj": 72,
|
| 118 |
+
"model.language_model.layers.32.mlp.gate_proj": 72,
|
| 119 |
+
"model.language_model.layers.32.mlp.up_proj": 72,
|
| 120 |
+
"model.language_model.layers.32.self_attn.k_proj": 72,
|
| 121 |
+
"model.language_model.layers.32.self_attn.o_proj": 72,
|
| 122 |
+
"model.language_model.layers.32.self_attn.q_proj": 72,
|
| 123 |
+
"model.language_model.layers.32.self_attn.v_proj": 72,
|
| 124 |
+
"model.language_model.layers.34.mlp.down_proj": 72,
|
| 125 |
+
"model.language_model.layers.34.mlp.gate_proj": 72,
|
| 126 |
+
"model.language_model.layers.34.mlp.up_proj": 72,
|
| 127 |
+
"model.language_model.layers.34.self_attn.k_proj": 72,
|
| 128 |
+
"model.language_model.layers.34.self_attn.o_proj": 72,
|
| 129 |
+
"model.language_model.layers.34.self_attn.q_proj": 72,
|
| 130 |
+
"model.language_model.layers.34.self_attn.v_proj": 72,
|
| 131 |
+
"model.language_model.layers.4.mlp.down_proj": 72,
|
| 132 |
+
"model.language_model.layers.4.mlp.gate_proj": 72,
|
| 133 |
+
"model.language_model.layers.4.mlp.up_proj": 72,
|
| 134 |
+
"model.language_model.layers.4.self_attn.k_proj": 72,
|
| 135 |
+
"model.language_model.layers.4.self_attn.o_proj": 72,
|
| 136 |
+
"model.language_model.layers.4.self_attn.q_proj": 72,
|
| 137 |
+
"model.language_model.layers.4.self_attn.v_proj": 72,
|
| 138 |
+
"model.language_model.layers.6.mlp.down_proj": 72,
|
| 139 |
+
"model.language_model.layers.6.mlp.gate_proj": 72,
|
| 140 |
+
"model.language_model.layers.6.mlp.up_proj": 72,
|
| 141 |
+
"model.language_model.layers.6.self_attn.k_proj": 72,
|
| 142 |
+
"model.language_model.layers.6.self_attn.o_proj": 72,
|
| 143 |
+
"model.language_model.layers.6.self_attn.q_proj": 72,
|
| 144 |
+
"model.language_model.layers.6.self_attn.v_proj": 72
|
| 145 |
+
},
|
| 146 |
+
"revision": null,
|
| 147 |
+
"target_modules": [
|
| 148 |
+
"model.language_model.layers.13.mlp.gate_proj",
|
| 149 |
+
"model.language_model.layers.13.self_attn.k_proj",
|
| 150 |
+
"model.language_model.layers.13.mlp.down_proj",
|
| 151 |
+
"model.language_model.layers.28.self_attn.q_proj",
|
| 152 |
+
"model.language_model.layers.4.self_attn.q_proj",
|
| 153 |
+
"model.language_model.layers.16.mlp.up_proj",
|
| 154 |
+
"model.language_model.layers.16.self_attn.o_proj",
|
| 155 |
+
"model.language_model.layers.34.self_attn.q_proj",
|
| 156 |
+
"model.language_model.layers.22.self_attn.k_proj",
|
| 157 |
+
"model.language_model.layers.6.self_attn.q_proj",
|
| 158 |
+
"model.language_model.layers.22.mlp.up_proj",
|
| 159 |
+
"model.language_model.layers.32.mlp.gate_proj",
|
| 160 |
+
"model.language_model.layers.4.mlp.gate_proj",
|
| 161 |
+
"model.language_model.layers.34.mlp.gate_proj",
|
| 162 |
+
"model.language_model.layers.6.mlp.gate_proj",
|
| 163 |
+
"model.language_model.layers.32.self_attn.v_proj",
|
| 164 |
+
"model.language_model.layers.28.self_attn.v_proj",
|
| 165 |
+
"model.language_model.layers.32.mlp.up_proj",
|
| 166 |
+
"model.language_model.layers.4.self_attn.v_proj",
|
| 167 |
+
"model.language_model.layers.6.self_attn.o_proj",
|
| 168 |
+
"model.language_model.layers.16.self_attn.k_proj",
|
| 169 |
+
"model.language_model.layers.6.self_attn.v_proj",
|
| 170 |
+
"model.language_model.layers.22.mlp.gate_proj",
|
| 171 |
+
"model.language_model.layers.28.mlp.gate_proj",
|
| 172 |
+
"model.language_model.layers.13.self_attn.v_proj",
|
| 173 |
+
"model.language_model.layers.28.self_attn.o_proj",
|
| 174 |
+
"model.language_model.layers.32.self_attn.o_proj",
|
| 175 |
+
"model.language_model.layers.22.self_attn.q_proj",
|
| 176 |
+
"model.language_model.layers.28.self_attn.k_proj",
|
| 177 |
+
"model.language_model.layers.32.self_attn.q_proj",
|
| 178 |
+
"model.language_model.layers.32.mlp.down_proj",
|
| 179 |
+
"model.language_model.layers.34.self_attn.o_proj",
|
| 180 |
+
"model.language_model.layers.13.self_attn.o_proj",
|
| 181 |
+
"model.language_model.layers.34.self_attn.v_proj",
|
| 182 |
+
"model.language_model.layers.4.self_attn.k_proj",
|
| 183 |
+
"model.language_model.layers.6.mlp.down_proj",
|
| 184 |
+
"model.language_model.layers.6.mlp.up_proj",
|
| 185 |
+
"model.language_model.layers.22.self_attn.o_proj",
|
| 186 |
+
"model.language_model.layers.4.self_attn.o_proj",
|
| 187 |
+
"model.language_model.layers.13.mlp.up_proj",
|
| 188 |
+
"model.language_model.layers.16.mlp.gate_proj",
|
| 189 |
+
"model.language_model.layers.28.mlp.up_proj",
|
| 190 |
+
"model.language_model.layers.16.self_attn.q_proj",
|
| 191 |
+
"model.language_model.layers.6.self_attn.k_proj",
|
| 192 |
+
"model.language_model.layers.16.mlp.down_proj",
|
| 193 |
+
"model.language_model.layers.13.self_attn.q_proj",
|
| 194 |
+
"model.language_model.layers.28.mlp.down_proj",
|
| 195 |
+
"model.language_model.layers.34.mlp.down_proj",
|
| 196 |
+
"model.language_model.layers.34.self_attn.k_proj",
|
| 197 |
+
"model.language_model.layers.4.mlp.up_proj",
|
| 198 |
+
"model.language_model.layers.22.mlp.down_proj",
|
| 199 |
+
"model.language_model.layers.34.mlp.up_proj",
|
| 200 |
+
"model.language_model.layers.22.self_attn.v_proj",
|
| 201 |
+
"model.language_model.layers.32.self_attn.k_proj",
|
| 202 |
+
"model.language_model.layers.16.self_attn.v_proj",
|
| 203 |
+
"model.language_model.layers.4.mlp.down_proj"
|
| 204 |
+
],
|
| 205 |
+
"target_parameters": null,
|
| 206 |
+
"task_type": "CAUSAL_LM",
|
| 207 |
+
"trainable_token_indices": null,
|
| 208 |
+
"use_bdlora": null,
|
| 209 |
+
"use_dora": false,
|
| 210 |
+
"use_qalora": false,
|
| 211 |
+
"use_rslora": false,
|
| 212 |
+
"velora_config": null
|
| 213 |
+
}
|
runs/fixed_internvl/removal/Random8_8108_s13/config.json
ADDED
|
@@ -0,0 +1,43 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"behavior": "removal",
|
| 3 |
+
"seed": 13,
|
| 4 |
+
"model": "/data/OPSD/assets/models/InternVL3_5-8B-HF",
|
| 5 |
+
"backbone": "internvl",
|
| 6 |
+
"train": "/data/OPSD/assets/v3/removal.jsonl",
|
| 7 |
+
"items": 1000,
|
| 8 |
+
"steps": 250,
|
| 9 |
+
"epochs": 1,
|
| 10 |
+
"steps_per_epoch": 250,
|
| 11 |
+
"world_size": 1,
|
| 12 |
+
"lr": 5e-05,
|
| 13 |
+
"scheduler": "cosine",
|
| 14 |
+
"warmup_steps": 16,
|
| 15 |
+
"final_fraction": 0.1,
|
| 16 |
+
"clip": 1.0,
|
| 17 |
+
"initial_loss_scale": 4.0,
|
| 18 |
+
"arm": {
|
| 19 |
+
"arm": "Random8_8108",
|
| 20 |
+
"layers": [
|
| 21 |
+
4,
|
| 22 |
+
6,
|
| 23 |
+
13,
|
| 24 |
+
16,
|
| 25 |
+
22,
|
| 26 |
+
28,
|
| 27 |
+
32,
|
| 28 |
+
34
|
| 29 |
+
],
|
| 30 |
+
"ranks": {
|
| 31 |
+
"4": 72,
|
| 32 |
+
"6": 72,
|
| 33 |
+
"13": 72,
|
| 34 |
+
"16": 72,
|
| 35 |
+
"22": 72,
|
| 36 |
+
"28": 72,
|
| 37 |
+
"32": 72,
|
| 38 |
+
"34": 72
|
| 39 |
+
},
|
| 40 |
+
"parameters": 43646976,
|
| 41 |
+
"budget": 43646976
|
| 42 |
+
}
|
| 43 |
+
}
|
runs/fixed_internvl/removal/Random8_8108_s13/steps.jsonl
ADDED
|
@@ -0,0 +1,250 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{"step": 1, "steps": 250, "lr": 3.125e-06, "grad_norm": 6.526810169219971, "loss_scale": 4.0, "retries": 0, "seconds": 5.8, "max_memory_gb": 16.82, "ce": 0.7595979786710814, "rank": 0.06138449162244797}
|
| 2 |
+
{"step": 2, "steps": 250, "lr": 6.25e-06, "grad_norm": 5.701030254364014, "loss_scale": 4.0, "retries": 0, "seconds": 10.7, "max_memory_gb": 17.17, "ce": 0.7746406011028739, "rank": 0.0}
|
| 3 |
+
{"step": 3, "steps": 250, "lr": 9.375000000000001e-06, "grad_norm": 21.36739730834961, "loss_scale": 4.0, "retries": 0, "seconds": 15.3, "max_memory_gb": 17.17, "ce": 1.2318898420780897, "rank": 0.0}
|
| 4 |
+
{"step": 4, "steps": 250, "lr": 1.25e-05, "grad_norm": 7.869923114776611, "loss_scale": 4.0, "retries": 0, "seconds": 19.8, "max_memory_gb": 17.17, "ce": 0.5703699753539695, "rank": 0.0}
|
| 5 |
+
{"step": 5, "steps": 250, "lr": 1.5625e-05, "grad_norm": 0.1850176304578781, "loss_scale": 4.0, "retries": 0, "seconds": 24.7, "max_memory_gb": 17.17, "ce": 0.03947980096745596, "rank": 0.0}
|
| 6 |
+
{"step": 6, "steps": 250, "lr": 1.8750000000000002e-05, "grad_norm": 2.8786706924438477, "loss_scale": 4.0, "retries": 0, "seconds": 29.8, "max_memory_gb": 17.17, "ce": 0.3079575364754419, "rank": 0.0}
|
| 7 |
+
{"step": 7, "steps": 250, "lr": 2.1875e-05, "grad_norm": 3.082291841506958, "loss_scale": 4.0, "retries": 0, "seconds": 35.2, "max_memory_gb": 17.17, "ce": 0.41113258787481755, "rank": 0.0}
|
| 8 |
+
{"step": 8, "steps": 250, "lr": 2.5e-05, "grad_norm": 1.5745813846588135, "loss_scale": 4.0, "retries": 0, "seconds": 40.8, "max_memory_gb": 17.17, "ce": 0.3659078539349139, "rank": 0.0}
|
| 9 |
+
{"step": 9, "steps": 250, "lr": 2.8125000000000003e-05, "grad_norm": 3.646857976913452, "loss_scale": 4.0, "retries": 0, "seconds": 45.4, "max_memory_gb": 17.17, "ce": 0.31469882605597377, "rank": 0.0}
|
| 10 |
+
{"step": 10, "steps": 250, "lr": 3.125e-05, "grad_norm": 0.4077965021133423, "loss_scale": 4.0, "retries": 0, "seconds": 49.9, "max_memory_gb": 17.17, "ce": 0.1274045983227552, "rank": 0.0}
|
| 11 |
+
{"step": 11, "steps": 250, "lr": 3.4375e-05, "grad_norm": 0.55401611328125, "loss_scale": 4.0, "retries": 0, "seconds": 54.5, "max_memory_gb": 17.17, "ce": 0.20635154300180147, "rank": 0.0}
|
| 12 |
+
{"step": 12, "steps": 250, "lr": 3.7500000000000003e-05, "grad_norm": 0.3378864824771881, "loss_scale": 4.0, "retries": 0, "seconds": 59.0, "max_memory_gb": 17.17, "ce": 0.109871520166962, "rank": 0.0}
|
| 13 |
+
{"step": 13, "steps": 250, "lr": 4.0625000000000005e-05, "grad_norm": 0.5692847967147827, "loss_scale": 4.0, "retries": 0, "seconds": 63.4, "max_memory_gb": 17.17, "ce": 0.22921674408098625, "rank": 0.0}
|
| 14 |
+
{"step": 14, "steps": 250, "lr": 4.375e-05, "grad_norm": 1.8820915222167969, "loss_scale": 4.0, "retries": 0, "seconds": 67.9, "max_memory_gb": 17.17, "ce": 0.31348425173200667, "rank": 0.0}
|
| 15 |
+
{"step": 15, "steps": 250, "lr": 4.6875e-05, "grad_norm": 0.6853877305984497, "loss_scale": 4.0, "retries": 0, "seconds": 72.5, "max_memory_gb": 17.17, "ce": 0.2163143197176396, "rank": 0.0}
|
| 16 |
+
{"step": 16, "steps": 250, "lr": 5e-05, "grad_norm": 0.37336674332618713, "loss_scale": 4.0, "retries": 0, "seconds": 76.8, "max_memory_gb": 17.17, "ce": 0.08270385891955812, "rank": 0.0}
|
| 17 |
+
{"step": 17, "steps": 250, "lr": 5e-05, "grad_norm": 1.3304587602615356, "loss_scale": 4.0, "retries": 0, "seconds": 81.1, "max_memory_gb": 17.17, "ce": 0.20202765305293724, "rank": 0.0}
|
| 18 |
+
{"step": 18, "steps": 250, "lr": 4.999797225177623e-05, "grad_norm": 0.8883626461029053, "loss_scale": 4.0, "retries": 0, "seconds": 85.8, "max_memory_gb": 17.18, "ce": 0.16424016893142834, "rank": 0.0}
|
| 19 |
+
{"step": 19, "steps": 250, "lr": 4.999188937259492e-05, "grad_norm": 1.005994439125061, "loss_scale": 4.0, "retries": 0, "seconds": 90.3, "max_memory_gb": 17.18, "ce": 0.11944992565258872, "rank": 0.10272421687841415}
|
| 20 |
+
{"step": 20, "steps": 250, "lr": 4.998175245886032e-05, "grad_norm": 0.22661910951137543, "loss_scale": 4.0, "retries": 0, "seconds": 94.1, "max_memory_gb": 17.18, "ce": 0.017754028591298265, "rank": 0.12360983341932297}
|
| 21 |
+
{"step": 21, "steps": 250, "lr": 4.996756333769319e-05, "grad_norm": 0.5830226540565491, "loss_scale": 4.0, "retries": 0, "seconds": 98.6, "max_memory_gb": 17.18, "ce": 0.06291914346365957, "rank": 0.0}
|
| 22 |
+
{"step": 22, "steps": 250, "lr": 4.994932456660157e-05, "grad_norm": 1.5568755865097046, "loss_scale": 4.0, "retries": 0, "seconds": 103.1, "max_memory_gb": 17.18, "ce": 0.1379503324860707, "rank": 0.019283458590507507}
|
| 23 |
+
{"step": 23, "steps": 250, "lr": 4.992703943301973e-05, "grad_norm": 1.0951818227767944, "loss_scale": 4.0, "retries": 0, "seconds": 107.6, "max_memory_gb": 17.18, "ce": 0.10300631369864277, "rank": 0.0}
|
| 24 |
+
{"step": 24, "steps": 250, "lr": 4.9900711953715675e-05, "grad_norm": 1.282622218132019, "loss_scale": 4.0, "retries": 0, "seconds": 112.2, "max_memory_gb": 17.18, "ce": 0.0814348568674177, "rank": 0.0}
|
| 25 |
+
{"step": 25, "steps": 250, "lr": 4.987034687406713e-05, "grad_norm": 2.762146472930908, "loss_scale": 4.0, "retries": 0, "seconds": 116.8, "max_memory_gb": 17.18, "ce": 0.19069654915074352, "rank": 0.0}
|
| 26 |
+
{"step": 26, "steps": 250, "lr": 4.9835949667206215e-05, "grad_norm": 3.2682552337646484, "loss_scale": 4.0, "retries": 0, "seconds": 121.2, "max_memory_gb": 17.18, "ce": 0.21392710423242534, "rank": 0.0}
|
| 27 |
+
{"step": 27, "steps": 250, "lr": 4.9797526533032945e-05, "grad_norm": 2.426935911178589, "loss_scale": 4.0, "retries": 0, "seconds": 125.6, "max_memory_gb": 17.18, "ce": 0.1584136311939801, "rank": 0.06693601608276367}
|
| 28 |
+
{"step": 28, "steps": 250, "lr": 4.9755084397097716e-05, "grad_norm": 1.9827111959457397, "loss_scale": 4.0, "retries": 0, "seconds": 130.3, "max_memory_gb": 17.18, "ce": 0.20683914147957694, "rank": 0.0}
|
| 29 |
+
{"step": 29, "steps": 250, "lr": 4.970863090935304e-05, "grad_norm": 4.33587121963501, "loss_scale": 4.0, "retries": 0, "seconds": 134.9, "max_memory_gb": 17.18, "ce": 0.6622524803533452, "rank": 0.0}
|
| 30 |
+
{"step": 30, "steps": 250, "lr": 4.965817444277468e-05, "grad_norm": 0.22935640811920166, "loss_scale": 4.0, "retries": 0, "seconds": 139.4, "max_memory_gb": 17.18, "ce": 0.012188609927420657, "rank": 0.0}
|
| 31 |
+
{"step": 31, "steps": 250, "lr": 4.9603724091852456e-05, "grad_norm": 6.248391628265381, "loss_scale": 4.0, "retries": 0, "seconds": 144.0, "max_memory_gb": 17.18, "ce": 0.7315254760833341, "rank": 0.0}
|
| 32 |
+
{"step": 32, "steps": 250, "lr": 4.9545289670951025e-05, "grad_norm": 0.46384212374687195, "loss_scale": 4.0, "retries": 0, "seconds": 148.7, "max_memory_gb": 17.18, "ce": 0.0303451075888006, "rank": 0.0}
|
| 33 |
+
{"step": 33, "steps": 250, "lr": 4.948288171254089e-05, "grad_norm": 3.7156484127044678, "loss_scale": 4.0, "retries": 0, "seconds": 153.2, "max_memory_gb": 17.18, "ce": 0.2520948466080881, "rank": 0.0}
|
| 34 |
+
{"step": 34, "steps": 250, "lr": 4.941651146529998e-05, "grad_norm": 0.603527843952179, "loss_scale": 4.0, "retries": 0, "seconds": 157.8, "max_memory_gb": 17.18, "ce": 0.0908141464315122, "rank": 0.0}
|
| 35 |
+
{"step": 35, "steps": 250, "lr": 4.9346190892086174e-05, "grad_norm": 0.4673828184604645, "loss_scale": 4.0, "retries": 0, "seconds": 162.6, "max_memory_gb": 17.18, "ce": 0.009012331804115092, "rank": 0.0}
|
| 36 |
+
{"step": 36, "steps": 250, "lr": 4.927193266778101e-05, "grad_norm": 2.349809408187866, "loss_scale": 4.0, "retries": 0, "seconds": 167.1, "max_memory_gb": 17.18, "ce": 0.2611113668044709, "rank": 0.0}
|
| 37 |
+
{"step": 37, "steps": 250, "lr": 4.919375017700515e-05, "grad_norm": 0.7952145338058472, "loss_scale": 4.0, "retries": 0, "seconds": 171.7, "max_memory_gb": 17.18, "ce": 0.1775994204062954, "rank": 0.0}
|
| 38 |
+
{"step": 38, "steps": 250, "lr": 4.911165751170588e-05, "grad_norm": 1.0273735523223877, "loss_scale": 4.0, "retries": 0, "seconds": 176.4, "max_memory_gb": 17.18, "ce": 0.14869274508964736, "rank": 0.0}
|
| 39 |
+
{"step": 39, "steps": 250, "lr": 4.902566946861708e-05, "grad_norm": 1.4269648790359497, "loss_scale": 4.0, "retries": 0, "seconds": 180.9, "max_memory_gb": 17.18, "ce": 0.15864316509669152, "rank": 0.12509897351264954}
|
| 40 |
+
{"step": 40, "steps": 250, "lr": 4.89358015465922e-05, "grad_norm": 0.11095961183309555, "loss_scale": 4.0, "retries": 0, "seconds": 185.3, "max_memory_gb": 17.18, "ce": 0.007953457241455908, "rank": 0.0}
|
| 41 |
+
{"step": 41, "steps": 250, "lr": 4.884206994381078e-05, "grad_norm": 3.2097842693328857, "loss_scale": 4.0, "retries": 0, "seconds": 190.0, "max_memory_gb": 17.18, "ce": 0.4345801400659184, "rank": 0.0}
|
| 42 |
+
{"step": 42, "steps": 250, "lr": 4.874449155485866e-05, "grad_norm": 0.06485968828201294, "loss_scale": 4.0, "retries": 0, "seconds": 194.9, "max_memory_gb": 17.18, "ce": 0.004963976780345547, "rank": 0.0}
|
| 43 |
+
{"step": 43, "steps": 250, "lr": 4.864308396768294e-05, "grad_norm": 0.12596648931503296, "loss_scale": 4.0, "retries": 0, "seconds": 199.6, "max_memory_gb": 17.18, "ce": 0.008084876359134796, "rank": 0.0}
|
| 44 |
+
{"step": 44, "steps": 250, "lr": 4.853786546042184e-05, "grad_norm": 7.235899448394775, "loss_scale": 4.0, "retries": 0, "seconds": 204.5, "max_memory_gb": 17.18, "ce": 0.25935244848847105, "rank": 0.0}
|
| 45 |
+
{"step": 45, "steps": 250, "lr": 4.842885499811012e-05, "grad_norm": 3.964200258255005, "loss_scale": 4.0, "retries": 0, "seconds": 209.2, "max_memory_gb": 17.18, "ce": 0.3871433490112395, "rank": 0.0}
|
| 46 |
+
{"step": 46, "steps": 250, "lr": 4.831607222926078e-05, "grad_norm": 0.8766592741012573, "loss_scale": 4.0, "retries": 0, "seconds": 213.7, "max_memory_gb": 17.18, "ce": 0.08243703206971986, "rank": 0.0}
|
| 47 |
+
{"step": 47, "steps": 250, "lr": 4.8199537482323545e-05, "grad_norm": 0.5161870121955872, "loss_scale": 4.0, "retries": 0, "seconds": 218.2, "max_memory_gb": 17.18, "ce": 0.03286801452122745, "rank": 0.0}
|
| 48 |
+
{"step": 48, "steps": 250, "lr": 4.807927176202073e-05, "grad_norm": 0.024980518966913223, "loss_scale": 4.0, "retries": 0, "seconds": 223.1, "max_memory_gb": 17.18, "ce": 0.0014697524393341155, "rank": 0.0}
|
| 49 |
+
{"step": 49, "steps": 250, "lr": 4.795529674556129e-05, "grad_norm": 0.11006376892328262, "loss_scale": 4.0, "retries": 0, "seconds": 227.9, "max_memory_gb": 17.18, "ce": 0.009528985076030949, "rank": 0.0}
|
| 50 |
+
{"step": 50, "steps": 250, "lr": 4.782763477873361e-05, "grad_norm": 1.290738821029663, "loss_scale": 4.0, "retries": 0, "seconds": 232.5, "max_memory_gb": 17.18, "ce": 0.11237288491520303, "rank": 0.0}
|
| 51 |
+
{"step": 51, "steps": 250, "lr": 4.769630887187782e-05, "grad_norm": 3.2494146823883057, "loss_scale": 4.0, "retries": 0, "seconds": 237.2, "max_memory_gb": 17.18, "ce": 0.1599164094009211, "rank": 0.0}
|
| 52 |
+
{"step": 52, "steps": 250, "lr": 4.756134269573832e-05, "grad_norm": 0.3326036334037781, "loss_scale": 4.0, "retries": 0, "seconds": 242.1, "max_memory_gb": 17.18, "ce": 0.029285225575222285, "rank": 0.0}
|
| 53 |
+
{"step": 53, "steps": 250, "lr": 4.7422760577197226e-05, "grad_norm": 1.7380882501602173, "loss_scale": 4.0, "retries": 0, "seconds": 247.1, "max_memory_gb": 17.18, "ce": 0.07404857528308639, "rank": 0.0}
|
| 54 |
+
{"step": 54, "steps": 250, "lr": 4.728058749488963e-05, "grad_norm": 1.1132475137710571, "loss_scale": 4.0, "retries": 0, "seconds": 252.0, "max_memory_gb": 17.18, "ce": 0.055001310774059675, "rank": 0.0}
|
| 55 |
+
{"step": 55, "steps": 250, "lr": 4.713484907470133e-05, "grad_norm": 0.18690244853496552, "loss_scale": 4.0, "retries": 0, "seconds": 256.9, "max_memory_gb": 17.18, "ce": 0.014167115525424379, "rank": 0.0}
|
| 56 |
+
{"step": 56, "steps": 250, "lr": 4.6985571585149876e-05, "grad_norm": 0.3529154658317566, "loss_scale": 4.0, "retries": 0, "seconds": 261.8, "max_memory_gb": 17.18, "ce": 0.026168374857661547, "rank": 0.0}
|
| 57 |
+
{"step": 57, "steps": 250, "lr": 4.6832781932649884e-05, "grad_norm": 0.13178792595863342, "loss_scale": 4.0, "retries": 0, "seconds": 266.6, "max_memory_gb": 17.18, "ce": 0.008829555450120097, "rank": 0.0}
|
| 58 |
+
{"step": 58, "steps": 250, "lr": 4.667650765666325e-05, "grad_norm": 0.032637085765600204, "loss_scale": 4.0, "retries": 0, "seconds": 271.5, "max_memory_gb": 17.18, "ce": 0.0011924714419819793, "rank": 0.0}
|
| 59 |
+
{"step": 59, "steps": 250, "lr": 4.651677692473538e-05, "grad_norm": 2.142329692840576, "loss_scale": 4.0, "retries": 0, "seconds": 276.3, "max_memory_gb": 17.18, "ce": 0.1846788369616661, "rank": 0.0}
|
| 60 |
+
{"step": 60, "steps": 250, "lr": 4.635361852741808e-05, "grad_norm": 0.8879985809326172, "loss_scale": 4.0, "retries": 0, "seconds": 281.0, "max_memory_gb": 17.18, "ce": 0.07392860010804725, "rank": 0.0}
|
| 61 |
+
{"step": 61, "steps": 250, "lr": 4.6187061873080264e-05, "grad_norm": 1.574517011642456, "loss_scale": 4.0, "retries": 0, "seconds": 285.8, "max_memory_gb": 17.18, "ce": 0.22570698897834518, "rank": 0.0}
|
| 62 |
+
{"step": 62, "steps": 250, "lr": 4.601713698260727e-05, "grad_norm": 0.3812691867351532, "loss_scale": 4.0, "retries": 0, "seconds": 290.7, "max_memory_gb": 17.18, "ce": 0.029823342390045582, "rank": 0.0}
|
| 63 |
+
{"step": 63, "steps": 250, "lr": 4.5843874483989746e-05, "grad_norm": 0.5023283958435059, "loss_scale": 4.0, "retries": 0, "seconds": 295.6, "max_memory_gb": 17.18, "ce": 0.033661735772284374, "rank": 0.0}
|
| 64 |
+
{"step": 64, "steps": 250, "lr": 4.566730560680316e-05, "grad_norm": 0.14796072244644165, "loss_scale": 4.0, "retries": 0, "seconds": 300.4, "max_memory_gb": 17.18, "ce": 0.008899858267795935, "rank": 0.0}
|
| 65 |
+
{"step": 65, "steps": 250, "lr": 4.548746217657878e-05, "grad_norm": 1.007485032081604, "loss_scale": 4.0, "retries": 0, "seconds": 305.3, "max_memory_gb": 17.18, "ce": 0.13332560145386196, "rank": 0.0}
|
| 66 |
+
{"step": 66, "steps": 250, "lr": 4.5304376609067396e-05, "grad_norm": 2.0214731693267822, "loss_scale": 4.0, "retries": 0, "seconds": 310.3, "max_memory_gb": 17.18, "ce": 0.22693379897427235, "rank": 0.1196301057934761}
|
| 67 |
+
{"step": 67, "steps": 250, "lr": 4.5118081904396504e-05, "grad_norm": 0.02738962136209011, "loss_scale": 4.0, "retries": 0, "seconds": 315.1, "max_memory_gb": 17.18, "ce": 0.0022976677903443488, "rank": 0.0}
|
| 68 |
+
{"step": 68, "steps": 250, "lr": 4.4928611641122225e-05, "grad_norm": 0.6130194664001465, "loss_scale": 4.0, "retries": 0, "seconds": 319.9, "max_memory_gb": 17.18, "ce": 0.012650478405021204, "rank": 0.0}
|
| 69 |
+
{"step": 69, "steps": 250, "lr": 4.473599997017701e-05, "grad_norm": 4.4739155769348145, "loss_scale": 4.0, "retries": 0, "seconds": 325.0, "max_memory_gb": 17.18, "ce": 0.7629749306984195, "rank": 0.0}
|
| 70 |
+
{"step": 70, "steps": 250, "lr": 4.4540281608714053e-05, "grad_norm": 0.5504947304725647, "loss_scale": 4.0, "retries": 0, "seconds": 329.8, "max_memory_gb": 17.18, "ce": 0.006985744556232021, "rank": 0.0}
|
| 71 |
+
{"step": 71, "steps": 250, "lr": 4.434149183384977e-05, "grad_norm": 0.04523656144738197, "loss_scale": 4.0, "retries": 0, "seconds": 334.5, "max_memory_gb": 17.18, "ce": 0.0026859300400019492, "rank": 0.0}
|
| 72 |
+
{"step": 72, "steps": 250, "lr": 4.413966647630534e-05, "grad_norm": 0.7768390774726868, "loss_scale": 4.0, "retries": 0, "seconds": 339.3, "max_memory_gb": 17.18, "ce": 0.079849858394482, "rank": 0.0}
|
| 73 |
+
{"step": 73, "steps": 250, "lr": 4.3934841913948323e-05, "grad_norm": 1.8299500942230225, "loss_scale": 4.0, "retries": 0, "seconds": 344.0, "max_memory_gb": 17.18, "ce": 0.1267493981692951, "rank": 0.0}
|
| 74 |
+
{"step": 74, "steps": 250, "lr": 4.372705506523583e-05, "grad_norm": 1.7436405420303345, "loss_scale": 4.0, "retries": 0, "seconds": 348.7, "max_memory_gb": 17.18, "ce": 0.16351610025776608, "rank": 0.0}
|
| 75 |
+
{"step": 75, "steps": 250, "lr": 4.351634338256017e-05, "grad_norm": 3.661405086517334, "loss_scale": 4.0, "retries": 0, "seconds": 353.8, "max_memory_gb": 17.18, "ce": 0.7309883987015837, "rank": 0.0}
|
| 76 |
+
{"step": 76, "steps": 250, "lr": 4.330274484549825e-05, "grad_norm": 1.4095540046691895, "loss_scale": 4.0, "retries": 0, "seconds": 359.2, "max_memory_gb": 17.18, "ce": 0.3343936601584119, "rank": 0.0}
|
| 77 |
+
{"step": 77, "steps": 250, "lr": 4.308629795396599e-05, "grad_norm": 1.1677660942077637, "loss_scale": 4.0, "retries": 0, "seconds": 365.0, "max_memory_gb": 17.18, "ce": 0.04717212453465436, "rank": 0.0}
|
| 78 |
+
{"step": 78, "steps": 250, "lr": 4.286704172127894e-05, "grad_norm": 0.2407831996679306, "loss_scale": 4.0, "retries": 0, "seconds": 371.1, "max_memory_gb": 17.18, "ce": 0.022271695344898035, "rank": 0.0}
|
| 79 |
+
{"step": 79, "steps": 250, "lr": 4.264501566712033e-05, "grad_norm": 0.25465378165245056, "loss_scale": 4.0, "retries": 0, "seconds": 377.6, "max_memory_gb": 17.18, "ce": 0.00817306453745914, "rank": 0.0}
|
| 80 |
+
{"step": 80, "steps": 250, "lr": 4.2420259810417894e-05, "grad_norm": 0.24125705659389496, "loss_scale": 4.0, "retries": 0, "seconds": 383.3, "max_memory_gb": 17.18, "ce": 0.01597339151021515, "rank": 0.0}
|
| 81 |
+
{"step": 81, "steps": 250, "lr": 4.219281466213066e-05, "grad_norm": 0.4208347797393799, "loss_scale": 4.0, "retries": 0, "seconds": 388.8, "max_memory_gb": 17.18, "ce": 0.04361235545729869, "rank": 0.0}
|
| 82 |
+
{"step": 82, "steps": 250, "lr": 4.1962721217947135e-05, "grad_norm": 1.4007929563522339, "loss_scale": 4.0, "retries": 0, "seconds": 395.7, "max_memory_gb": 17.18, "ce": 0.12210834209827226, "rank": 0.0}
|
| 83 |
+
{"step": 83, "steps": 250, "lr": 4.173002095089599e-05, "grad_norm": 1.31548273563385, "loss_scale": 4.0, "retries": 0, "seconds": 402.2, "max_memory_gb": 17.18, "ce": 0.20480249635102155, "rank": 0.0}
|
| 84 |
+
{"step": 84, "steps": 250, "lr": 4.149475580387084e-05, "grad_norm": 0.5901208519935608, "loss_scale": 4.0, "retries": 0, "seconds": 407.7, "max_memory_gb": 17.18, "ce": 0.032387452378316084, "rank": 0.0}
|
| 85 |
+
{"step": 85, "steps": 250, "lr": 4.1256968182070275e-05, "grad_norm": 0.052295878529548645, "loss_scale": 4.0, "retries": 0, "seconds": 412.7, "max_memory_gb": 17.18, "ce": 0.001197939168719131, "rank": 0.0}
|
| 86 |
+
{"step": 86, "steps": 250, "lr": 4.101670094535453e-05, "grad_norm": 0.6198393702507019, "loss_scale": 4.0, "retries": 0, "seconds": 417.5, "max_memory_gb": 17.18, "ce": 0.05277544036198378, "rank": 0.0}
|
| 87 |
+
{"step": 87, "steps": 250, "lr": 4.077399740052027e-05, "grad_norm": 1.0265893936157227, "loss_scale": 4.0, "retries": 0, "seconds": 422.4, "max_memory_gb": 17.18, "ce": 0.146933578062999, "rank": 0.0}
|
| 88 |
+
{"step": 88, "steps": 250, "lr": 4.0528901293494775e-05, "grad_norm": 0.34741297364234924, "loss_scale": 4.0, "retries": 0, "seconds": 427.3, "max_memory_gb": 17.18, "ce": 0.034329894706843334, "rank": 0.0}
|
| 89 |
+
{"step": 89, "steps": 250, "lr": 4.0281456801451004e-05, "grad_norm": 0.16544339060783386, "loss_scale": 4.0, "retries": 0, "seconds": 432.4, "max_memory_gb": 17.18, "ce": 0.017100490691518644, "rank": 0.12499643862247467}
|
| 90 |
+
{"step": 90, "steps": 250, "lr": 4.0031708524844884e-05, "grad_norm": 0.6801131367683411, "loss_scale": 4.0, "retries": 0, "seconds": 437.3, "max_memory_gb": 17.18, "ce": 0.07735038699911456, "rank": 0.0}
|
| 91 |
+
{"step": 91, "steps": 250, "lr": 3.977970147937635e-05, "grad_norm": 3.7204136848449707, "loss_scale": 4.0, "retries": 0, "seconds": 442.2, "max_memory_gb": 17.18, "ce": 0.48442745160537015, "rank": 0.0}
|
| 92 |
+
{"step": 92, "steps": 250, "lr": 3.952548108787553e-05, "grad_norm": 3.400233030319214, "loss_scale": 4.0, "retries": 0, "seconds": 447.1, "max_memory_gb": 17.18, "ce": 0.2051650060643624, "rank": 0.0}
|
| 93 |
+
{"step": 93, "steps": 250, "lr": 3.9269093172115496e-05, "grad_norm": 1.683309555053711, "loss_scale": 4.0, "retries": 0, "seconds": 451.9, "max_memory_gb": 17.18, "ce": 0.07942358851141762, "rank": 0.0}
|
| 94 |
+
{"step": 94, "steps": 250, "lr": 3.901058394455323e-05, "grad_norm": 1.8013781309127808, "loss_scale": 4.0, "retries": 0, "seconds": 456.7, "max_memory_gb": 17.18, "ce": 0.06811754639920764, "rank": 0.0}
|
| 95 |
+
{"step": 95, "steps": 250, "lr": 3.875e-05, "grad_norm": 4.21297550201416, "loss_scale": 4.0, "retries": 0, "seconds": 461.7, "max_memory_gb": 17.18, "ce": 0.509934214318946, "rank": 0.0}
|
| 96 |
+
{"step": 96, "steps": 250, "lr": 3.8487388307223005e-05, "grad_norm": 3.427121877670288, "loss_scale": 4.0, "retries": 0, "seconds": 466.5, "max_memory_gb": 17.18, "ce": 0.4064108221882634, "rank": 0.3257160782814026}
|
| 97 |
+
{"step": 97, "steps": 250, "lr": 3.822279620047943e-05, "grad_norm": 0.46772390604019165, "loss_scale": 4.0, "retries": 0, "seconds": 471.2, "max_memory_gb": 17.18, "ce": 0.025910292252092404, "rank": 0.0}
|
| 98 |
+
{"step": 98, "steps": 250, "lr": 3.795627137098479e-05, "grad_norm": 0.7912839651107788, "loss_scale": 4.0, "retries": 0, "seconds": 476.0, "max_memory_gb": 17.18, "ce": 0.04650824912459939, "rank": 0.0}
|
| 99 |
+
{"step": 99, "steps": 250, "lr": 3.7687861858316844e-05, "grad_norm": 0.0998878926038742, "loss_scale": 4.0, "retries": 0, "seconds": 480.9, "max_memory_gb": 17.18, "ce": 0.0099813200454264, "rank": 0.0}
|
| 100 |
+
{"step": 100, "steps": 250, "lr": 3.741761604175668e-05, "grad_norm": 0.5991601943969727, "loss_scale": 4.0, "retries": 0, "seconds": 485.8, "max_memory_gb": 17.18, "ce": 0.08950544256731519, "rank": 0.0}
|
| 101 |
+
{"step": 101, "steps": 250, "lr": 3.714558263156872e-05, "grad_norm": 0.17930515110492706, "loss_scale": 4.0, "retries": 0, "seconds": 490.6, "max_memory_gb": 17.18, "ce": 0.011386152348222822, "rank": 0.0}
|
| 102 |
+
{"step": 102, "steps": 250, "lr": 3.6871810660220904e-05, "grad_norm": 0.17666330933570862, "loss_scale": 4.0, "retries": 0, "seconds": 495.5, "max_memory_gb": 17.18, "ce": 0.011883949800449045, "rank": 0.0}
|
| 103 |
+
{"step": 103, "steps": 250, "lr": 3.659634947354686e-05, "grad_norm": 0.11328842490911484, "loss_scale": 4.0, "retries": 0, "seconds": 500.9, "max_memory_gb": 17.18, "ce": 0.007582726411328622, "rank": 0.0}
|
| 104 |
+
{"step": 104, "steps": 250, "lr": 3.631924872185169e-05, "grad_norm": 0.26119789481163025, "loss_scale": 4.0, "retries": 0, "seconds": 507.0, "max_memory_gb": 17.18, "ce": 0.01625490042169986, "rank": 0.12493158876895905}
|
| 105 |
+
{"step": 105, "steps": 250, "lr": 3.6040558350962674e-05, "grad_norm": 0.7302834391593933, "loss_scale": 4.0, "retries": 0, "seconds": 511.6, "max_memory_gb": 17.18, "ce": 0.051999771038026665, "rank": 0.0}
|
| 106 |
+
{"step": 106, "steps": 250, "lr": 3.576032859322687e-05, "grad_norm": 0.06580471992492676, "loss_scale": 4.0, "retries": 0, "seconds": 516.4, "max_memory_gb": 17.18, "ce": 0.00236817788800181, "rank": 0.1111852154135704}
|
| 107 |
+
{"step": 107, "steps": 250, "lr": 3.5478609958457056e-05, "grad_norm": 1.6889588832855225, "loss_scale": 4.0, "retries": 0, "seconds": 521.2, "max_memory_gb": 17.18, "ce": 0.12405462285732938, "rank": 0.0}
|
| 108 |
+
{"step": 108, "steps": 250, "lr": 3.519545322482755e-05, "grad_norm": 3.2017858028411865, "loss_scale": 4.0, "retries": 0, "seconds": 526.0, "max_memory_gb": 17.18, "ce": 0.23190371317400604, "rank": 0.0}
|
| 109 |
+
{"step": 109, "steps": 250, "lr": 3.491090942972179e-05, "grad_norm": 1.6316721439361572, "loss_scale": 4.0, "retries": 0, "seconds": 530.8, "max_memory_gb": 17.18, "ce": 0.15288718252031686, "rank": 0.0}
|
| 110 |
+
{"step": 110, "steps": 250, "lr": 3.4625029860533135e-05, "grad_norm": 0.5611461400985718, "loss_scale": 4.0, "retries": 0, "seconds": 535.3, "max_memory_gb": 17.18, "ce": 0.030988446860646945, "rank": 0.0}
|
| 111 |
+
{"step": 111, "steps": 250, "lr": 3.433786604542056e-05, "grad_norm": 0.30809763073921204, "loss_scale": 4.0, "retries": 0, "seconds": 540.0, "max_memory_gb": 17.18, "ce": 0.018491533856831666, "rank": 0.0}
|
| 112 |
+
{"step": 112, "steps": 250, "lr": 3.404946974402104e-05, "grad_norm": 0.05183107778429985, "loss_scale": 4.0, "retries": 0, "seconds": 544.9, "max_memory_gb": 17.18, "ce": 0.002889123127260973, "rank": 0.0}
|
| 113 |
+
{"step": 113, "steps": 250, "lr": 3.375989293812019e-05, "grad_norm": 0.17629586160182953, "loss_scale": 4.0, "retries": 0, "seconds": 549.5, "max_memory_gb": 17.18, "ce": 0.015687244501805253, "rank": 0.0}
|
| 114 |
+
{"step": 114, "steps": 250, "lr": 3.3469187822282766e-05, "grad_norm": 0.8722972273826599, "loss_scale": 4.0, "retries": 0, "seconds": 554.2, "max_memory_gb": 17.18, "ce": 0.11928030503531772, "rank": 0.0}
|
| 115 |
+
{"step": 115, "steps": 250, "lr": 3.317740679444499e-05, "grad_norm": 1.1051822900772095, "loss_scale": 4.0, "retries": 0, "seconds": 560.2, "max_memory_gb": 17.18, "ce": 0.02463514998953542, "rank": 0.0}
|
| 116 |
+
{"step": 116, "steps": 250, "lr": 3.2884602446470056e-05, "grad_norm": 1.1276087760925293, "loss_scale": 4.0, "retries": 0, "seconds": 564.7, "max_memory_gb": 17.18, "ce": 0.1670033425823476, "rank": 0.0}
|
| 117 |
+
{"step": 117, "steps": 250, "lr": 3.259082755466876e-05, "grad_norm": 0.13213452696800232, "loss_scale": 4.0, "retries": 0, "seconds": 569.4, "max_memory_gb": 17.18, "ce": 0.0056259569976191415, "rank": 0.0}
|
| 118 |
+
{"step": 118, "steps": 250, "lr": 3.229613507028686e-05, "grad_norm": 1.565881371498108, "loss_scale": 4.0, "retries": 0, "seconds": 574.2, "max_memory_gb": 17.18, "ce": 0.2095577683078318, "rank": 0.0}
|
| 119 |
+
{"step": 119, "steps": 250, "lr": 3.2000578109961004e-05, "grad_norm": 0.0539349764585495, "loss_scale": 4.0, "retries": 0, "seconds": 579.6, "max_memory_gb": 17.18, "ce": 0.002660686799345058, "rank": 0.0}
|
| 120 |
+
{"step": 120, "steps": 250, "lr": 3.1704209946144616e-05, "grad_norm": 3.8137004375457764, "loss_scale": 4.0, "retries": 0, "seconds": 584.4, "max_memory_gb": 17.18, "ce": 0.18039529780298835, "rank": 0.0}
|
| 121 |
+
{"step": 121, "steps": 250, "lr": 3.140708399750594e-05, "grad_norm": 0.8089257478713989, "loss_scale": 4.0, "retries": 0, "seconds": 589.0, "max_memory_gb": 17.18, "ce": 0.058003509110221785, "rank": 0.0}
|
| 122 |
+
{"step": 122, "steps": 250, "lr": 3.1109253819299606e-05, "grad_norm": 1.3538144826889038, "loss_scale": 4.0, "retries": 0, "seconds": 593.9, "max_memory_gb": 17.18, "ce": 0.11333506394521464, "rank": 0.0}
|
| 123 |
+
{"step": 123, "steps": 250, "lr": 3.081077309371361e-05, "grad_norm": 0.40327444672584534, "loss_scale": 4.0, "retries": 0, "seconds": 598.7, "max_memory_gb": 17.18, "ce": 0.03224148110598435, "rank": 0.0}
|
| 124 |
+
{"step": 124, "steps": 250, "lr": 3.051169562019339e-05, "grad_norm": 1.9480326175689697, "loss_scale": 4.0, "retries": 0, "seconds": 603.6, "max_memory_gb": 17.18, "ce": 0.5383501341862029, "rank": 0.0}
|
| 125 |
+
{"step": 125, "steps": 250, "lr": 3.021207530574477e-05, "grad_norm": 0.8545853495597839, "loss_scale": 4.0, "retries": 0, "seconds": 608.4, "max_memory_gb": 17.18, "ce": 0.14952015499466143, "rank": 0.0}
|
| 126 |
+
{"step": 126, "steps": 250, "lr": 2.991196615521758e-05, "grad_norm": 3.135396718978882, "loss_scale": 4.0, "retries": 0, "seconds": 613.4, "max_memory_gb": 17.18, "ce": 0.4614685678834576, "rank": 0.0}
|
| 127 |
+
{"step": 127, "steps": 250, "lr": 2.9611422261571536e-05, "grad_norm": 0.4310620129108429, "loss_scale": 4.0, "retries": 0, "seconds": 618.1, "max_memory_gb": 17.18, "ce": 0.014219647994195839, "rank": 0.0}
|
| 128 |
+
{"step": 128, "steps": 250, "lr": 2.931049779612634e-05, "grad_norm": 1.931749939918518, "loss_scale": 4.0, "retries": 0, "seconds": 623.0, "max_memory_gb": 17.18, "ce": 0.1526931573675938, "rank": 0.0}
|
| 129 |
+
{"step": 129, "steps": 250, "lr": 2.9009246998797557e-05, "grad_norm": 1.3711153268814087, "loss_scale": 4.0, "retries": 0, "seconds": 628.0, "max_memory_gb": 17.18, "ce": 0.09942347389858242, "rank": 0.0}
|
| 130 |
+
{"step": 130, "steps": 250, "lr": 2.8707724168320238e-05, "grad_norm": 0.8344557881355286, "loss_scale": 4.0, "retries": 0, "seconds": 632.8, "max_memory_gb": 17.18, "ce": 0.14975639791032336, "rank": 0.0}
|
| 131 |
+
{"step": 131, "steps": 250, "lr": 2.840598365246184e-05, "grad_norm": 3.434664249420166, "loss_scale": 4.0, "retries": 0, "seconds": 637.6, "max_memory_gb": 17.18, "ce": 0.20914711673606234, "rank": 0.0}
|
| 132 |
+
{"step": 132, "steps": 250, "lr": 2.8104079838226383e-05, "grad_norm": 2.082834482192993, "loss_scale": 4.0, "retries": 0, "seconds": 642.5, "max_memory_gb": 17.18, "ce": 0.2453914832767623, "rank": 0.0}
|
| 133 |
+
{"step": 133, "steps": 250, "lr": 2.7802067142051446e-05, "grad_norm": 1.6043238639831543, "loss_scale": 4.0, "retries": 0, "seconds": 647.2, "max_memory_gb": 17.18, "ce": 0.29478994082035115, "rank": 0.0}
|
| 134 |
+
{"step": 134, "steps": 250, "lr": 2.7500000000000004e-05, "grad_norm": 1.4177229404449463, "loss_scale": 4.0, "retries": 0, "seconds": 651.9, "max_memory_gb": 17.18, "ce": 0.20679133830162755, "rank": 0.0}
|
| 135 |
+
{"step": 135, "steps": 250, "lr": 2.7197932857948566e-05, "grad_norm": 0.3215174973011017, "loss_scale": 4.0, "retries": 0, "seconds": 656.9, "max_memory_gb": 17.18, "ce": 0.017724246484249306, "rank": 0.0}
|
| 136 |
+
{"step": 136, "steps": 250, "lr": 2.689592016177363e-05, "grad_norm": 1.0608714818954468, "loss_scale": 4.0, "retries": 0, "seconds": 662.0, "max_memory_gb": 17.18, "ce": 0.045617017971835594, "rank": 0.0}
|
| 137 |
+
{"step": 137, "steps": 250, "lr": 2.659401634753817e-05, "grad_norm": 0.3650874197483063, "loss_scale": 4.0, "retries": 0, "seconds": 666.7, "max_memory_gb": 17.18, "ce": 0.01592326760396645, "rank": 0.0}
|
| 138 |
+
{"step": 138, "steps": 250, "lr": 2.6292275831679764e-05, "grad_norm": 1.2116667032241821, "loss_scale": 4.0, "retries": 0, "seconds": 671.5, "max_memory_gb": 17.18, "ce": 0.08068314619572448, "rank": 0.0}
|
| 139 |
+
{"step": 139, "steps": 250, "lr": 2.5990753001202445e-05, "grad_norm": 0.7489631772041321, "loss_scale": 4.0, "retries": 0, "seconds": 676.4, "max_memory_gb": 17.18, "ce": 0.033992647457012026, "rank": 0.0}
|
| 140 |
+
{"step": 140, "steps": 250, "lr": 2.5689502203873672e-05, "grad_norm": 0.032809630036354065, "loss_scale": 4.0, "retries": 0, "seconds": 681.2, "max_memory_gb": 17.18, "ce": 0.0022480584966615424, "rank": 0.0}
|
| 141 |
+
{"step": 141, "steps": 250, "lr": 2.538857773842847e-05, "grad_norm": 0.04617055878043175, "loss_scale": 4.0, "retries": 0, "seconds": 685.9, "max_memory_gb": 17.18, "ce": 0.0034868480071281738, "rank": 0.0}
|
| 142 |
+
{"step": 142, "steps": 250, "lr": 2.5088033844782426e-05, "grad_norm": 1.562259554862976, "loss_scale": 4.0, "retries": 0, "seconds": 691.0, "max_memory_gb": 17.18, "ce": 0.07541959829131883, "rank": 0.0}
|
| 143 |
+
{"step": 143, "steps": 250, "lr": 2.4787924694255237e-05, "grad_norm": 2.619408369064331, "loss_scale": 4.0, "retries": 0, "seconds": 695.8, "max_memory_gb": 17.18, "ce": 0.24117978341155322, "rank": 0.0}
|
| 144 |
+
{"step": 144, "steps": 250, "lr": 2.4488304379806627e-05, "grad_norm": 0.31864529848098755, "loss_scale": 4.0, "retries": 0, "seconds": 701.4, "max_memory_gb": 17.18, "ce": 0.02755915182433455, "rank": 0.0}
|
| 145 |
+
{"step": 145, "steps": 250, "lr": 2.4189226906286388e-05, "grad_norm": 0.027462884783744812, "loss_scale": 4.0, "retries": 0, "seconds": 706.4, "max_memory_gb": 17.18, "ce": 0.0019233183834330703, "rank": 0.0}
|
| 146 |
+
{"step": 146, "steps": 250, "lr": 2.3890746180700395e-05, "grad_norm": 0.15063031017780304, "loss_scale": 4.0, "retries": 0, "seconds": 711.2, "max_memory_gb": 17.18, "ce": 0.012709802696463157, "rank": 0.0}
|
| 147 |
+
{"step": 147, "steps": 250, "lr": 2.359291600249407e-05, "grad_norm": 1.8771147727966309, "loss_scale": 4.0, "retries": 0, "seconds": 715.8, "max_memory_gb": 17.18, "ce": 0.49676161458432944, "rank": 0.0}
|
| 148 |
+
{"step": 148, "steps": 250, "lr": 2.3295790053855392e-05, "grad_norm": 0.13209977746009827, "loss_scale": 4.0, "retries": 0, "seconds": 720.3, "max_memory_gb": 17.18, "ce": 0.007385738507309725, "rank": 0.0}
|
| 149 |
+
{"step": 149, "steps": 250, "lr": 2.299942189003901e-05, "grad_norm": 0.1205153614282608, "loss_scale": 4.0, "retries": 0, "seconds": 724.7, "max_memory_gb": 17.18, "ce": 0.007389827626411716, "rank": 0.12188003957271576}
|
| 150 |
+
{"step": 150, "steps": 250, "lr": 2.2703864929713144e-05, "grad_norm": 0.6057351231575012, "loss_scale": 4.0, "retries": 0, "seconds": 729.3, "max_memory_gb": 17.18, "ce": 0.07822950665888584, "rank": 0.0}
|
| 151 |
+
{"step": 151, "steps": 250, "lr": 2.2409172445331257e-05, "grad_norm": 2.090216875076294, "loss_scale": 4.0, "retries": 0, "seconds": 734.0, "max_memory_gb": 17.18, "ce": 0.0516085979206764, "rank": 0.0}
|
| 152 |
+
{"step": 152, "steps": 250, "lr": 2.211539755352996e-05, "grad_norm": 0.3038743734359741, "loss_scale": 4.0, "retries": 0, "seconds": 738.9, "max_memory_gb": 17.18, "ce": 0.025148884417262707, "rank": 0.0}
|
| 153 |
+
{"step": 153, "steps": 250, "lr": 2.1822593205555016e-05, "grad_norm": 0.6545040011405945, "loss_scale": 4.0, "retries": 0, "seconds": 743.6, "max_memory_gb": 17.18, "ce": 0.10802062954826397, "rank": 0.0}
|
| 154 |
+
{"step": 154, "steps": 250, "lr": 2.1530812177717236e-05, "grad_norm": 0.32496050000190735, "loss_scale": 4.0, "retries": 0, "seconds": 748.3, "max_memory_gb": 17.18, "ce": 0.02421575959169786, "rank": 0.0}
|
| 155 |
+
{"step": 155, "steps": 250, "lr": 2.1240107061879815e-05, "grad_norm": 2.0151681900024414, "loss_scale": 4.0, "retries": 0, "seconds": 753.0, "max_memory_gb": 17.18, "ce": 0.2684263014789394, "rank": 0.0}
|
| 156 |
+
{"step": 156, "steps": 250, "lr": 2.095053025597896e-05, "grad_norm": 0.06669197231531143, "loss_scale": 4.0, "retries": 0, "seconds": 757.9, "max_memory_gb": 17.18, "ce": 0.0026050064757328073, "rank": 0.0}
|
| 157 |
+
{"step": 157, "steps": 250, "lr": 2.066213395457945e-05, "grad_norm": 0.21107280254364014, "loss_scale": 4.0, "retries": 0, "seconds": 762.8, "max_memory_gb": 17.18, "ce": 0.01396351020093789, "rank": 0.0}
|
| 158 |
+
{"step": 158, "steps": 250, "lr": 2.0374970139466877e-05, "grad_norm": 4.005942344665527, "loss_scale": 4.0, "retries": 0, "seconds": 767.5, "max_memory_gb": 17.18, "ce": 0.576374157044711, "rank": 0.0}
|
| 159 |
+
{"step": 159, "steps": 250, "lr": 2.0089090570278213e-05, "grad_norm": 0.22257567942142487, "loss_scale": 4.0, "retries": 0, "seconds": 773.2, "max_memory_gb": 17.18, "ce": 0.013566704569257126, "rank": 0.0}
|
| 160 |
+
{"step": 160, "steps": 250, "lr": 1.9804546775172457e-05, "grad_norm": 1.2461014986038208, "loss_scale": 4.0, "retries": 0, "seconds": 778.2, "max_memory_gb": 17.18, "ce": 0.1804903652523535, "rank": 0.0}
|
| 161 |
+
{"step": 161, "steps": 250, "lr": 1.9521390041542946e-05, "grad_norm": 1.1854829788208008, "loss_scale": 4.0, "retries": 0, "seconds": 783.0, "max_memory_gb": 17.18, "ce": 0.11565671056587234, "rank": 0.0}
|
| 162 |
+
{"step": 162, "steps": 250, "lr": 1.923967140677313e-05, "grad_norm": 1.0250152349472046, "loss_scale": 4.0, "retries": 0, "seconds": 787.8, "max_memory_gb": 17.18, "ce": 0.06691223713482941, "rank": 0.0}
|
| 163 |
+
{"step": 163, "steps": 250, "lr": 1.8959441649037335e-05, "grad_norm": 5.17612361907959, "loss_scale": 4.0, "retries": 0, "seconds": 792.6, "max_memory_gb": 17.18, "ce": 0.34906826027554416, "rank": 0.0}
|
| 164 |
+
{"step": 164, "steps": 250, "lr": 1.868075127814832e-05, "grad_norm": 2.119999647140503, "loss_scale": 4.0, "retries": 0, "seconds": 797.3, "max_memory_gb": 17.18, "ce": 0.07739335856194884, "rank": 0.0}
|
| 165 |
+
{"step": 165, "steps": 250, "lr": 1.8403650526453143e-05, "grad_norm": 0.10026904940605164, "loss_scale": 4.0, "retries": 0, "seconds": 802.3, "max_memory_gb": 17.18, "ce": 0.008497133665287038, "rank": 0.0}
|
| 166 |
+
{"step": 166, "steps": 250, "lr": 1.812818933977912e-05, "grad_norm": 2.4535911083221436, "loss_scale": 4.0, "retries": 0, "seconds": 807.0, "max_memory_gb": 17.18, "ce": 0.2692563327755124, "rank": 0.0}
|
| 167 |
+
{"step": 167, "steps": 250, "lr": 1.785441736843128e-05, "grad_norm": 0.16436946392059326, "loss_scale": 4.0, "retries": 0, "seconds": 811.7, "max_memory_gb": 17.18, "ce": 0.007986912415844927, "rank": 0.0}
|
| 168 |
+
{"step": 168, "steps": 250, "lr": 1.7582383958243314e-05, "grad_norm": 0.070530965924263, "loss_scale": 4.0, "retries": 0, "seconds": 816.5, "max_memory_gb": 17.18, "ce": 0.0044415024053705565, "rank": 0.0}
|
| 169 |
+
{"step": 169, "steps": 250, "lr": 1.7312138141683164e-05, "grad_norm": 1.2482839822769165, "loss_scale": 4.0, "retries": 0, "seconds": 821.4, "max_memory_gb": 17.18, "ce": 0.2998135275652203, "rank": 0.0515781044960022}
|
| 170 |
+
{"step": 170, "steps": 250, "lr": 1.704372862901521e-05, "grad_norm": 0.012457573786377907, "loss_scale": 4.0, "retries": 0, "seconds": 826.2, "max_memory_gb": 17.18, "ce": 0.0005100508149098459, "rank": 0.0}
|
| 171 |
+
{"step": 171, "steps": 250, "lr": 1.6777203799520573e-05, "grad_norm": 0.419999897480011, "loss_scale": 4.0, "retries": 0, "seconds": 830.8, "max_memory_gb": 17.18, "ce": 0.05221618518226023, "rank": 0.0}
|
| 172 |
+
{"step": 172, "steps": 250, "lr": 1.6512611692777008e-05, "grad_norm": 0.09144969284534454, "loss_scale": 4.0, "retries": 0, "seconds": 835.6, "max_memory_gb": 17.18, "ce": 0.007438893777816702, "rank": 0.0}
|
| 173 |
+
{"step": 173, "steps": 250, "lr": 1.6250000000000005e-05, "grad_norm": 0.15164364874362946, "loss_scale": 4.0, "retries": 0, "seconds": 840.2, "max_memory_gb": 17.18, "ce": 0.009757200466083304, "rank": 0.0}
|
| 174 |
+
{"step": 174, "steps": 250, "lr": 1.598941605544678e-05, "grad_norm": 0.5332861542701721, "loss_scale": 4.0, "retries": 0, "seconds": 845.0, "max_memory_gb": 17.18, "ce": 0.03775058457404157, "rank": 0.0}
|
| 175 |
+
{"step": 175, "steps": 250, "lr": 1.5730906827884506e-05, "grad_norm": 0.9867273569107056, "loss_scale": 4.0, "retries": 0, "seconds": 849.7, "max_memory_gb": 17.18, "ce": 0.10186905622958875, "rank": 0.0}
|
| 176 |
+
{"step": 176, "steps": 250, "lr": 1.5474518912124476e-05, "grad_norm": 0.024786842986941338, "loss_scale": 4.0, "retries": 0, "seconds": 854.5, "max_memory_gb": 17.18, "ce": 0.0012830478540308832, "rank": 0.0}
|
| 177 |
+
{"step": 177, "steps": 250, "lr": 1.5220298520623646e-05, "grad_norm": 1.1001307964324951, "loss_scale": 4.0, "retries": 0, "seconds": 859.2, "max_memory_gb": 17.18, "ce": 0.2746989063180081, "rank": 0.1250152587890625}
|
| 178 |
+
{"step": 178, "steps": 250, "lr": 1.4968291475155121e-05, "grad_norm": 2.1589386463165283, "loss_scale": 4.0, "retries": 0, "seconds": 864.0, "max_memory_gb": 17.18, "ce": 0.12029662652275874, "rank": 0.0}
|
| 179 |
+
{"step": 179, "steps": 250, "lr": 1.4718543198548998e-05, "grad_norm": 1.3357117176055908, "loss_scale": 4.0, "retries": 0, "seconds": 868.9, "max_memory_gb": 17.18, "ce": 0.3838543424985801, "rank": 0.0}
|
| 180 |
+
{"step": 180, "steps": 250, "lr": 1.4471098706505227e-05, "grad_norm": 0.21467316150665283, "loss_scale": 4.0, "retries": 0, "seconds": 873.4, "max_memory_gb": 17.18, "ce": 0.019395892782085866, "rank": 0.0}
|
| 181 |
+
{"step": 181, "steps": 250, "lr": 1.4226002599479743e-05, "grad_norm": 0.6904672980308533, "loss_scale": 4.0, "retries": 0, "seconds": 878.2, "max_memory_gb": 17.18, "ce": 0.01046327642052347, "rank": 0.015331916511058807}
|
| 182 |
+
{"step": 182, "steps": 250, "lr": 1.3983299054645471e-05, "grad_norm": 1.848176121711731, "loss_scale": 4.0, "retries": 0, "seconds": 883.0, "max_memory_gb": 17.18, "ce": 0.11818905890640963, "rank": 0.0}
|
| 183 |
+
{"step": 183, "steps": 250, "lr": 1.3743031817929724e-05, "grad_norm": 0.03503753989934921, "loss_scale": 4.0, "retries": 0, "seconds": 887.8, "max_memory_gb": 17.18, "ce": 0.0017547131478750089, "rank": 0.0}
|
| 184 |
+
{"step": 184, "steps": 250, "lr": 1.3505244196129164e-05, "grad_norm": 0.41344884037971497, "loss_scale": 4.0, "retries": 0, "seconds": 892.6, "max_memory_gb": 17.18, "ce": 0.017011126504712593, "rank": 0.1673121526837349}
|
| 185 |
+
{"step": 185, "steps": 250, "lr": 1.3269979049104014e-05, "grad_norm": 0.4223645031452179, "loss_scale": 4.0, "retries": 0, "seconds": 897.4, "max_memory_gb": 17.18, "ce": 0.038438340891161715, "rank": 0.0}
|
| 186 |
+
{"step": 186, "steps": 250, "lr": 1.3037278782052863e-05, "grad_norm": 2.700070381164551, "loss_scale": 4.0, "retries": 0, "seconds": 903.6, "max_memory_gb": 17.18, "ce": 0.12394185682524039, "rank": 0.0}
|
| 187 |
+
{"step": 187, "steps": 250, "lr": 1.2807185337869343e-05, "grad_norm": 0.6104485988616943, "loss_scale": 4.0, "retries": 0, "seconds": 908.4, "max_memory_gb": 17.18, "ce": 0.1209954447065229, "rank": 0.13058936595916748}
|
| 188 |
+
{"step": 188, "steps": 250, "lr": 1.2579740189582115e-05, "grad_norm": 1.792907476425171, "loss_scale": 4.0, "retries": 0, "seconds": 914.1, "max_memory_gb": 17.18, "ce": 0.1603382923526624, "rank": 0.0}
|
| 189 |
+
{"step": 189, "steps": 250, "lr": 1.2354984332879668e-05, "grad_norm": 2.5139400959014893, "loss_scale": 4.0, "retries": 0, "seconds": 919.0, "max_memory_gb": 17.18, "ce": 0.30611457266491016, "rank": 0.0}
|
| 190 |
+
{"step": 190, "steps": 250, "lr": 1.2132958278721064e-05, "grad_norm": 0.10587111115455627, "loss_scale": 4.0, "retries": 0, "seconds": 923.8, "max_memory_gb": 17.18, "ce": 0.0032953283300685143, "rank": 0.0}
|
| 191 |
+
{"step": 191, "steps": 250, "lr": 1.1913702046034016e-05, "grad_norm": 2.165459156036377, "loss_scale": 4.0, "retries": 0, "seconds": 928.3, "max_memory_gb": 17.18, "ce": 0.18293732410120356, "rank": 0.0}
|
| 192 |
+
{"step": 192, "steps": 250, "lr": 1.1697255154501756e-05, "grad_norm": 1.308945655822754, "loss_scale": 4.0, "retries": 0, "seconds": 933.0, "max_memory_gb": 17.18, "ce": 0.3471710742501273, "rank": 0.0}
|
| 193 |
+
{"step": 193, "steps": 250, "lr": 1.1483656617439842e-05, "grad_norm": 0.013638269156217575, "loss_scale": 4.0, "retries": 0, "seconds": 937.9, "max_memory_gb": 17.18, "ce": 0.0010409899246042187, "rank": 0.12504450976848602}
|
| 194 |
+
{"step": 194, "steps": 250, "lr": 1.1272944934764177e-05, "grad_norm": 0.35062164068222046, "loss_scale": 4.0, "retries": 0, "seconds": 942.6, "max_memory_gb": 17.18, "ce": 0.033809709841079894, "rank": 0.0}
|
| 195 |
+
{"step": 195, "steps": 250, "lr": 1.1065158086051684e-05, "grad_norm": 0.5896777510643005, "loss_scale": 4.0, "retries": 0, "seconds": 947.4, "max_memory_gb": 17.18, "ce": 0.07020822436652452, "rank": 0.0}
|
| 196 |
+
{"step": 196, "steps": 250, "lr": 1.086033352369467e-05, "grad_norm": 0.09937702119350433, "loss_scale": 4.0, "retries": 0, "seconds": 952.3, "max_memory_gb": 17.18, "ce": 0.006043756846111137, "rank": 0.0}
|
| 197 |
+
{"step": 197, "steps": 250, "lr": 1.0658508166150224e-05, "grad_norm": 1.9917702674865723, "loss_scale": 4.0, "retries": 0, "seconds": 956.7, "max_memory_gb": 17.18, "ce": 0.1561089036292742, "rank": 0.0}
|
| 198 |
+
{"step": 198, "steps": 250, "lr": 1.0459718391285954e-05, "grad_norm": 0.40669500827789307, "loss_scale": 4.0, "retries": 0, "seconds": 961.3, "max_memory_gb": 17.18, "ce": 0.058683001067493024, "rank": 0.0}
|
| 199 |
+
{"step": 199, "steps": 250, "lr": 1.0264000029822999e-05, "grad_norm": 1.3688435554504395, "loss_scale": 4.0, "retries": 0, "seconds": 966.3, "max_memory_gb": 17.18, "ce": 0.111011210837205, "rank": 0.0}
|
| 200 |
+
{"step": 200, "steps": 250, "lr": 1.0071388358877775e-05, "grad_norm": 1.2040661573410034, "loss_scale": 4.0, "retries": 0, "seconds": 971.1, "max_memory_gb": 17.18, "ce": 0.20462579786976676, "rank": 0.0}
|
| 201 |
+
{"step": 201, "steps": 250, "lr": 9.881918095603501e-06, "grad_norm": 0.7883883118629456, "loss_scale": 4.0, "retries": 0, "seconds": 976.0, "max_memory_gb": 17.18, "ce": 0.06030021416370346, "rank": 0.0}
|
| 202 |
+
{"step": 202, "steps": 250, "lr": 9.695623390932612e-06, "grad_norm": 0.030858609825372696, "loss_scale": 4.0, "retries": 0, "seconds": 980.7, "max_memory_gb": 17.18, "ce": 0.0023000175963261427, "rank": 0.0}
|
| 203 |
+
{"step": 203, "steps": 250, "lr": 9.51253782342123e-06, "grad_norm": 0.49807208776474, "loss_scale": 4.0, "retries": 0, "seconds": 985.4, "max_memory_gb": 17.18, "ce": 0.018394404855825996, "rank": 0.0}
|
| 204 |
+
{"step": 204, "steps": 250, "lr": 9.332694393196846e-06, "grad_norm": 0.3151397407054901, "loss_scale": 4.0, "retries": 0, "seconds": 990.7, "max_memory_gb": 17.18, "ce": 0.02402326593323778, "rank": 0.0}
|
| 205 |
+
{"step": 205, "steps": 250, "lr": 9.156125516010256e-06, "grad_norm": 0.78297358751297, "loss_scale": 4.0, "retries": 0, "seconds": 996.4, "max_memory_gb": 17.18, "ce": 0.08941664707799646, "rank": 0.0}
|
| 206 |
+
{"step": 206, "steps": 250, "lr": 8.982863017392733e-06, "grad_norm": 0.27139565348625183, "loss_scale": 4.0, "retries": 0, "seconds": 1001.1, "max_memory_gb": 17.18, "ce": 0.02090874214059113, "rank": 0.0}
|
| 207 |
+
{"step": 207, "steps": 250, "lr": 8.81293812691974e-06, "grad_norm": 0.21349108219146729, "loss_scale": 4.0, "retries": 0, "seconds": 1005.9, "max_memory_gb": 17.18, "ce": 0.050855450285780535, "rank": 0.0}
|
| 208 |
+
{"step": 208, "steps": 250, "lr": 8.646381472581927e-06, "grad_norm": 0.03276226297020912, "loss_scale": 4.0, "retries": 0, "seconds": 1010.7, "max_memory_gb": 17.18, "ce": 0.0019220744563881453, "rank": 0.0}
|
| 209 |
+
{"step": 209, "steps": 250, "lr": 8.483223075264622e-06, "grad_norm": 0.8125504851341248, "loss_scale": 4.0, "retries": 0, "seconds": 1015.6, "max_memory_gb": 17.18, "ce": 0.1132622440736668, "rank": 0.1251223087310791}
|
| 210 |
+
{"step": 210, "steps": 250, "lr": 8.323492343336747e-06, "grad_norm": 1.6871517896652222, "loss_scale": 4.0, "retries": 0, "seconds": 1020.3, "max_memory_gb": 17.18, "ce": 0.37672616303120776, "rank": 0.0}
|
| 211 |
+
{"step": 211, "steps": 250, "lr": 8.167218067350122e-06, "grad_norm": 0.8559074401855469, "loss_scale": 4.0, "retries": 0, "seconds": 1024.7, "max_memory_gb": 17.18, "ce": 0.12835767813373877, "rank": 0.0}
|
| 212 |
+
{"step": 212, "steps": 250, "lr": 8.01442841485013e-06, "grad_norm": 1.0825867652893066, "loss_scale": 4.0, "retries": 0, "seconds": 1029.0, "max_memory_gb": 17.18, "ce": 0.17164726564078592, "rank": 0.0}
|
| 213 |
+
{"step": 213, "steps": 250, "lr": 7.865150925298678e-06, "grad_norm": 0.6667714715003967, "loss_scale": 4.0, "retries": 0, "seconds": 1033.5, "max_memory_gb": 17.18, "ce": 0.1467162703199847, "rank": 0.0}
|
| 214 |
+
{"step": 214, "steps": 250, "lr": 7.719412505110375e-06, "grad_norm": 1.2324542999267578, "loss_scale": 4.0, "retries": 0, "seconds": 1038.3, "max_memory_gb": 17.18, "ce": 0.2140423660284796, "rank": 0.0}
|
| 215 |
+
{"step": 215, "steps": 250, "lr": 7.57723942280278e-06, "grad_norm": 2.51863431930542, "loss_scale": 4.0, "retries": 0, "seconds": 1043.1, "max_memory_gb": 17.18, "ce": 0.395534503323006, "rank": 0.0}
|
| 216 |
+
{"step": 216, "steps": 250, "lr": 7.438657304261687e-06, "grad_norm": 0.40938034653663635, "loss_scale": 4.0, "retries": 0, "seconds": 1047.7, "max_memory_gb": 17.18, "ce": 0.030119788006118142, "rank": 0.0}
|
| 217 |
+
{"step": 217, "steps": 250, "lr": 7.30369112812219e-06, "grad_norm": 0.052365366369485855, "loss_scale": 4.0, "retries": 0, "seconds": 1052.4, "max_memory_gb": 17.18, "ce": 0.0028229261172327824, "rank": 0.0}
|
| 218 |
+
{"step": 218, "steps": 250, "lr": 7.172365221266403e-06, "grad_norm": 1.1233724355697632, "loss_scale": 4.0, "retries": 0, "seconds": 1057.1, "max_memory_gb": 17.18, "ce": 0.17539756083954217, "rank": 0.0}
|
| 219 |
+
{"step": 219, "steps": 250, "lr": 7.0447032544387154e-06, "grad_norm": 0.3016583025455475, "loss_scale": 4.0, "retries": 0, "seconds": 1062.0, "max_memory_gb": 17.18, "ce": 0.03284460483200746, "rank": 0.0}
|
| 220 |
+
{"step": 220, "steps": 250, "lr": 6.920728237979274e-06, "grad_norm": 0.3474242389202118, "loss_scale": 4.0, "retries": 0, "seconds": 1066.7, "max_memory_gb": 17.18, "ce": 0.023843454110874518, "rank": 0.0}
|
| 221 |
+
{"step": 221, "steps": 250, "lr": 6.800462517676456e-06, "grad_norm": 0.3997562527656555, "loss_scale": 4.0, "retries": 0, "seconds": 1071.4, "max_memory_gb": 17.18, "ce": 0.02221117653652982, "rank": 0.0}
|
| 222 |
+
{"step": 222, "steps": 250, "lr": 6.683927770739222e-06, "grad_norm": 0.3420104682445526, "loss_scale": 4.0, "retries": 0, "seconds": 1076.2, "max_memory_gb": 17.18, "ce": 0.027881629539137975, "rank": 0.0}
|
| 223 |
+
{"step": 223, "steps": 250, "lr": 6.571145001889892e-06, "grad_norm": 1.7348017692565918, "loss_scale": 4.0, "retries": 0, "seconds": 1080.8, "max_memory_gb": 17.18, "ce": 0.061346658979232416, "rank": 0.0}
|
| 224 |
+
{"step": 224, "steps": 250, "lr": 6.46213453957817e-06, "grad_norm": 0.7194615602493286, "loss_scale": 4.0, "retries": 0, "seconds": 1085.1, "max_memory_gb": 17.18, "ce": 0.03260150498113035, "rank": 0.0}
|
| 225 |
+
{"step": 225, "steps": 250, "lr": 6.356916032317063e-06, "grad_norm": 0.11712709814310074, "loss_scale": 4.0, "retries": 0, "seconds": 1089.8, "max_memory_gb": 17.18, "ce": 0.008188609469016228, "rank": 0.0}
|
| 226 |
+
{"step": 226, "steps": 250, "lr": 6.255508445141349e-06, "grad_norm": 0.5417927503585815, "loss_scale": 4.0, "retries": 0, "seconds": 1094.6, "max_memory_gb": 17.18, "ce": 0.021519721744425624, "rank": 0.0}
|
| 227 |
+
{"step": 227, "steps": 250, "lr": 6.157930056189228e-06, "grad_norm": 0.6553352475166321, "loss_scale": 4.0, "retries": 0, "seconds": 1099.1, "max_memory_gb": 17.18, "ce": 0.022784811451856513, "rank": 0.0}
|
| 228 |
+
{"step": 228, "steps": 250, "lr": 6.064198453407796e-06, "grad_norm": 0.1065840795636177, "loss_scale": 4.0, "retries": 0, "seconds": 1103.5, "max_memory_gb": 17.18, "ce": 0.007579418956538575, "rank": 0.0}
|
| 229 |
+
{"step": 229, "steps": 250, "lr": 5.974330531382935e-06, "grad_norm": 0.5571849346160889, "loss_scale": 4.0, "retries": 0, "seconds": 1108.1, "max_memory_gb": 17.18, "ce": 0.05352028360971417, "rank": 0.0}
|
| 230 |
+
{"step": 230, "steps": 250, "lr": 5.888342488294125e-06, "grad_norm": 2.981887102127075, "loss_scale": 4.0, "retries": 0, "seconds": 1113.3, "max_memory_gb": 17.18, "ce": 0.3922790265238518, "rank": 0.0}
|
| 231 |
+
{"step": 231, "steps": 250, "lr": 5.806249822994849e-06, "grad_norm": 0.8230201601982117, "loss_scale": 4.0, "retries": 0, "seconds": 1117.9, "max_memory_gb": 17.18, "ce": 0.049433957854034816, "rank": 0.0}
|
| 232 |
+
{"step": 232, "steps": 250, "lr": 5.728067332218994e-06, "grad_norm": 0.38727128505706787, "loss_scale": 4.0, "retries": 0, "seconds": 1124.2, "max_memory_gb": 17.18, "ce": 0.02439140149283503, "rank": 0.0}
|
| 233 |
+
{"step": 233, "steps": 250, "lr": 5.65380910791383e-06, "grad_norm": 0.31660211086273193, "loss_scale": 4.0, "retries": 0, "seconds": 1128.8, "max_memory_gb": 17.18, "ce": 0.020987921069945514, "rank": 0.0}
|
| 234 |
+
{"step": 234, "steps": 250, "lr": 5.583488534700022e-06, "grad_norm": 0.8285323977470398, "loss_scale": 4.0, "retries": 0, "seconds": 1132.9, "max_memory_gb": 17.18, "ce": 0.04437500152994289, "rank": 0.0}
|
| 235 |
+
{"step": 235, "steps": 250, "lr": 5.517118287459118e-06, "grad_norm": 2.4697701930999756, "loss_scale": 4.0, "retries": 0, "seconds": 1137.5, "max_memory_gb": 17.18, "ce": 0.2492724072656074, "rank": 0.0}
|
| 236 |
+
{"step": 236, "steps": 250, "lr": 5.454710329048978e-06, "grad_norm": 0.2011195868253708, "loss_scale": 4.0, "retries": 0, "seconds": 1143.7, "max_memory_gb": 17.18, "ce": 0.016034932938396196, "rank": 0.0}
|
| 237 |
+
{"step": 237, "steps": 250, "lr": 5.3962759081475436e-06, "grad_norm": 0.898483395576477, "loss_scale": 4.0, "retries": 0, "seconds": 1148.8, "max_memory_gb": 17.18, "ce": 0.05523010599176814, "rank": 0.0}
|
| 238 |
+
{"step": 238, "steps": 250, "lr": 5.341825557225321e-06, "grad_norm": 0.39793214201927185, "loss_scale": 4.0, "retries": 0, "seconds": 1153.5, "max_memory_gb": 17.18, "ce": 0.03116334627327433, "rank": 0.0}
|
| 239 |
+
{"step": 239, "steps": 250, "lr": 5.291369090646961e-06, "grad_norm": 0.35622626543045044, "loss_scale": 4.0, "retries": 0, "seconds": 1158.3, "max_memory_gb": 17.18, "ce": 0.08009123704869126, "rank": 0.0}
|
| 240 |
+
{"step": 240, "steps": 250, "lr": 5.244915602902289e-06, "grad_norm": 0.32258638739585876, "loss_scale": 4.0, "retries": 0, "seconds": 1163.1, "max_memory_gb": 17.18, "ce": 0.021157505456358194, "rank": 0.0}
|
| 241 |
+
{"step": 241, "steps": 250, "lr": 5.20247346696706e-06, "grad_norm": 0.5261514782905579, "loss_scale": 4.0, "retries": 0, "seconds": 1167.8, "max_memory_gb": 17.18, "ce": 0.05985091058050784, "rank": 0.0}
|
| 242 |
+
{"step": 242, "steps": 250, "lr": 5.164050332793787e-06, "grad_norm": 0.12277106940746307, "loss_scale": 4.0, "retries": 0, "seconds": 1172.5, "max_memory_gb": 17.18, "ce": 0.009220479136274662, "rank": 0.0}
|
| 243 |
+
{"step": 243, "steps": 250, "lr": 5.129653125932873e-06, "grad_norm": 0.011762015521526337, "loss_scale": 4.0, "retries": 0, "seconds": 1177.4, "max_memory_gb": 17.18, "ce": 0.0008338423781424353, "rank": 0.0}
|
| 244 |
+
{"step": 244, "steps": 250, "lr": 5.099288046284329e-06, "grad_norm": 0.4867855906486511, "loss_scale": 4.0, "retries": 0, "seconds": 1182.1, "max_memory_gb": 17.18, "ce": 0.03884177559348245, "rank": 0.0}
|
| 245 |
+
{"step": 245, "steps": 250, "lr": 5.072960566980277e-06, "grad_norm": 1.7700446844100952, "loss_scale": 4.0, "retries": 0, "seconds": 1186.8, "max_memory_gb": 17.18, "ce": 0.24776680374236548, "rank": 0.0}
|
| 246 |
+
{"step": 246, "steps": 250, "lr": 5.050675433398435e-06, "grad_norm": 0.017080945894122124, "loss_scale": 4.0, "retries": 0, "seconds": 1191.8, "max_memory_gb": 17.18, "ce": 0.000957987728952503, "rank": 0.0}
|
| 247 |
+
{"step": 247, "steps": 250, "lr": 5.032436662306811e-06, "grad_norm": 0.5438670516014099, "loss_scale": 4.0, "retries": 0, "seconds": 1196.5, "max_memory_gb": 17.18, "ce": 0.04270005259388654, "rank": 0.0}
|
| 248 |
+
{"step": 248, "steps": 250, "lr": 5.018247541139684e-06, "grad_norm": 0.12717632949352264, "loss_scale": 4.0, "retries": 0, "seconds": 1201.4, "max_memory_gb": 17.18, "ce": 0.008095054040722971, "rank": 0.0}
|
| 249 |
+
{"step": 249, "steps": 250, "lr": 5.00811062740508e-06, "grad_norm": 0.21285198628902435, "loss_scale": 4.0, "retries": 0, "seconds": 1206.3, "max_memory_gb": 17.18, "ce": 0.01404752770417872, "rank": 0.0}
|
| 250 |
+
{"step": 250, "steps": 250, "lr": 5.002027748223778e-06, "grad_norm": 0.449648380279541, "loss_scale": 4.0, "retries": 0, "seconds": 1211.6, "max_memory_gb": 17.18, "ce": 0.02266445557734187, "rank": 0.0}
|
runs/fixed_internvl/removal/Random8_8109_s13/adapter/README.md
ADDED
|
@@ -0,0 +1,207 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
base_model: /data/OPSD/assets/models/InternVL3_5-8B-HF
|
| 3 |
+
library_name: peft
|
| 4 |
+
pipeline_tag: text-generation
|
| 5 |
+
tags:
|
| 6 |
+
- base_model:adapter:/data/OPSD/assets/models/InternVL3_5-8B-HF
|
| 7 |
+
- lora
|
| 8 |
+
- transformers
|
| 9 |
+
---
|
| 10 |
+
|
| 11 |
+
# Model Card for Model ID
|
| 12 |
+
|
| 13 |
+
<!-- Provide a quick summary of what the model is/does. -->
|
| 14 |
+
|
| 15 |
+
|
| 16 |
+
|
| 17 |
+
## Model Details
|
| 18 |
+
|
| 19 |
+
### Model Description
|
| 20 |
+
|
| 21 |
+
<!-- Provide a longer summary of what this model is. -->
|
| 22 |
+
|
| 23 |
+
|
| 24 |
+
|
| 25 |
+
- **Developed by:** [More Information Needed]
|
| 26 |
+
- **Funded by [optional]:** [More Information Needed]
|
| 27 |
+
- **Shared by [optional]:** [More Information Needed]
|
| 28 |
+
- **Model type:** [More Information Needed]
|
| 29 |
+
- **Language(s) (NLP):** [More Information Needed]
|
| 30 |
+
- **License:** [More Information Needed]
|
| 31 |
+
- **Finetuned from model [optional]:** [More Information Needed]
|
| 32 |
+
|
| 33 |
+
### Model Sources [optional]
|
| 34 |
+
|
| 35 |
+
<!-- Provide the basic links for the model. -->
|
| 36 |
+
|
| 37 |
+
- **Repository:** [More Information Needed]
|
| 38 |
+
- **Paper [optional]:** [More Information Needed]
|
| 39 |
+
- **Demo [optional]:** [More Information Needed]
|
| 40 |
+
|
| 41 |
+
## Uses
|
| 42 |
+
|
| 43 |
+
<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
|
| 44 |
+
|
| 45 |
+
### Direct Use
|
| 46 |
+
|
| 47 |
+
<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
|
| 48 |
+
|
| 49 |
+
[More Information Needed]
|
| 50 |
+
|
| 51 |
+
### Downstream Use [optional]
|
| 52 |
+
|
| 53 |
+
<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
|
| 54 |
+
|
| 55 |
+
[More Information Needed]
|
| 56 |
+
|
| 57 |
+
### Out-of-Scope Use
|
| 58 |
+
|
| 59 |
+
<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
|
| 60 |
+
|
| 61 |
+
[More Information Needed]
|
| 62 |
+
|
| 63 |
+
## Bias, Risks, and Limitations
|
| 64 |
+
|
| 65 |
+
<!-- This section is meant to convey both technical and sociotechnical limitations. -->
|
| 66 |
+
|
| 67 |
+
[More Information Needed]
|
| 68 |
+
|
| 69 |
+
### Recommendations
|
| 70 |
+
|
| 71 |
+
<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
|
| 72 |
+
|
| 73 |
+
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
|
| 74 |
+
|
| 75 |
+
## How to Get Started with the Model
|
| 76 |
+
|
| 77 |
+
Use the code below to get started with the model.
|
| 78 |
+
|
| 79 |
+
[More Information Needed]
|
| 80 |
+
|
| 81 |
+
## Training Details
|
| 82 |
+
|
| 83 |
+
### Training Data
|
| 84 |
+
|
| 85 |
+
<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
|
| 86 |
+
|
| 87 |
+
[More Information Needed]
|
| 88 |
+
|
| 89 |
+
### Training Procedure
|
| 90 |
+
|
| 91 |
+
<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
|
| 92 |
+
|
| 93 |
+
#### Preprocessing [optional]
|
| 94 |
+
|
| 95 |
+
[More Information Needed]
|
| 96 |
+
|
| 97 |
+
|
| 98 |
+
#### Training Hyperparameters
|
| 99 |
+
|
| 100 |
+
- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
|
| 101 |
+
|
| 102 |
+
#### Speeds, Sizes, Times [optional]
|
| 103 |
+
|
| 104 |
+
<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
|
| 105 |
+
|
| 106 |
+
[More Information Needed]
|
| 107 |
+
|
| 108 |
+
## Evaluation
|
| 109 |
+
|
| 110 |
+
<!-- This section describes the evaluation protocols and provides the results. -->
|
| 111 |
+
|
| 112 |
+
### Testing Data, Factors & Metrics
|
| 113 |
+
|
| 114 |
+
#### Testing Data
|
| 115 |
+
|
| 116 |
+
<!-- This should link to a Dataset Card if possible. -->
|
| 117 |
+
|
| 118 |
+
[More Information Needed]
|
| 119 |
+
|
| 120 |
+
#### Factors
|
| 121 |
+
|
| 122 |
+
<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
|
| 123 |
+
|
| 124 |
+
[More Information Needed]
|
| 125 |
+
|
| 126 |
+
#### Metrics
|
| 127 |
+
|
| 128 |
+
<!-- These are the evaluation metrics being used, ideally with a description of why. -->
|
| 129 |
+
|
| 130 |
+
[More Information Needed]
|
| 131 |
+
|
| 132 |
+
### Results
|
| 133 |
+
|
| 134 |
+
[More Information Needed]
|
| 135 |
+
|
| 136 |
+
#### Summary
|
| 137 |
+
|
| 138 |
+
|
| 139 |
+
|
| 140 |
+
## Model Examination [optional]
|
| 141 |
+
|
| 142 |
+
<!-- Relevant interpretability work for the model goes here -->
|
| 143 |
+
|
| 144 |
+
[More Information Needed]
|
| 145 |
+
|
| 146 |
+
## Environmental Impact
|
| 147 |
+
|
| 148 |
+
<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
|
| 149 |
+
|
| 150 |
+
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
|
| 151 |
+
|
| 152 |
+
- **Hardware Type:** [More Information Needed]
|
| 153 |
+
- **Hours used:** [More Information Needed]
|
| 154 |
+
- **Cloud Provider:** [More Information Needed]
|
| 155 |
+
- **Compute Region:** [More Information Needed]
|
| 156 |
+
- **Carbon Emitted:** [More Information Needed]
|
| 157 |
+
|
| 158 |
+
## Technical Specifications [optional]
|
| 159 |
+
|
| 160 |
+
### Model Architecture and Objective
|
| 161 |
+
|
| 162 |
+
[More Information Needed]
|
| 163 |
+
|
| 164 |
+
### Compute Infrastructure
|
| 165 |
+
|
| 166 |
+
[More Information Needed]
|
| 167 |
+
|
| 168 |
+
#### Hardware
|
| 169 |
+
|
| 170 |
+
[More Information Needed]
|
| 171 |
+
|
| 172 |
+
#### Software
|
| 173 |
+
|
| 174 |
+
[More Information Needed]
|
| 175 |
+
|
| 176 |
+
## Citation [optional]
|
| 177 |
+
|
| 178 |
+
<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
|
| 179 |
+
|
| 180 |
+
**BibTeX:**
|
| 181 |
+
|
| 182 |
+
[More Information Needed]
|
| 183 |
+
|
| 184 |
+
**APA:**
|
| 185 |
+
|
| 186 |
+
[More Information Needed]
|
| 187 |
+
|
| 188 |
+
## Glossary [optional]
|
| 189 |
+
|
| 190 |
+
<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
|
| 191 |
+
|
| 192 |
+
[More Information Needed]
|
| 193 |
+
|
| 194 |
+
## More Information [optional]
|
| 195 |
+
|
| 196 |
+
[More Information Needed]
|
| 197 |
+
|
| 198 |
+
## Model Card Authors [optional]
|
| 199 |
+
|
| 200 |
+
[More Information Needed]
|
| 201 |
+
|
| 202 |
+
## Model Card Contact
|
| 203 |
+
|
| 204 |
+
[More Information Needed]
|
| 205 |
+
### Framework versions
|
| 206 |
+
|
| 207 |
+
- PEFT 0.20.0
|
runs/fixed_internvl/removal/Random8_8109_s13/adapter/adapter_config.json
ADDED
|
@@ -0,0 +1,213 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"alora_invocation_tokens": null,
|
| 3 |
+
"alpha_pattern": {
|
| 4 |
+
"model.language_model.layers.10.mlp.down_proj": 144,
|
| 5 |
+
"model.language_model.layers.10.mlp.gate_proj": 144,
|
| 6 |
+
"model.language_model.layers.10.mlp.up_proj": 144,
|
| 7 |
+
"model.language_model.layers.10.self_attn.k_proj": 144,
|
| 8 |
+
"model.language_model.layers.10.self_attn.o_proj": 144,
|
| 9 |
+
"model.language_model.layers.10.self_attn.q_proj": 144,
|
| 10 |
+
"model.language_model.layers.10.self_attn.v_proj": 144,
|
| 11 |
+
"model.language_model.layers.15.mlp.down_proj": 144,
|
| 12 |
+
"model.language_model.layers.15.mlp.gate_proj": 144,
|
| 13 |
+
"model.language_model.layers.15.mlp.up_proj": 144,
|
| 14 |
+
"model.language_model.layers.15.self_attn.k_proj": 144,
|
| 15 |
+
"model.language_model.layers.15.self_attn.o_proj": 144,
|
| 16 |
+
"model.language_model.layers.15.self_attn.q_proj": 144,
|
| 17 |
+
"model.language_model.layers.15.self_attn.v_proj": 144,
|
| 18 |
+
"model.language_model.layers.25.mlp.down_proj": 144,
|
| 19 |
+
"model.language_model.layers.25.mlp.gate_proj": 144,
|
| 20 |
+
"model.language_model.layers.25.mlp.up_proj": 144,
|
| 21 |
+
"model.language_model.layers.25.self_attn.k_proj": 144,
|
| 22 |
+
"model.language_model.layers.25.self_attn.o_proj": 144,
|
| 23 |
+
"model.language_model.layers.25.self_attn.q_proj": 144,
|
| 24 |
+
"model.language_model.layers.25.self_attn.v_proj": 144,
|
| 25 |
+
"model.language_model.layers.26.mlp.down_proj": 144,
|
| 26 |
+
"model.language_model.layers.26.mlp.gate_proj": 144,
|
| 27 |
+
"model.language_model.layers.26.mlp.up_proj": 144,
|
| 28 |
+
"model.language_model.layers.26.self_attn.k_proj": 144,
|
| 29 |
+
"model.language_model.layers.26.self_attn.o_proj": 144,
|
| 30 |
+
"model.language_model.layers.26.self_attn.q_proj": 144,
|
| 31 |
+
"model.language_model.layers.26.self_attn.v_proj": 144,
|
| 32 |
+
"model.language_model.layers.28.mlp.down_proj": 144,
|
| 33 |
+
"model.language_model.layers.28.mlp.gate_proj": 144,
|
| 34 |
+
"model.language_model.layers.28.mlp.up_proj": 144,
|
| 35 |
+
"model.language_model.layers.28.self_attn.k_proj": 144,
|
| 36 |
+
"model.language_model.layers.28.self_attn.o_proj": 144,
|
| 37 |
+
"model.language_model.layers.28.self_attn.q_proj": 144,
|
| 38 |
+
"model.language_model.layers.28.self_attn.v_proj": 144,
|
| 39 |
+
"model.language_model.layers.29.mlp.down_proj": 144,
|
| 40 |
+
"model.language_model.layers.29.mlp.gate_proj": 144,
|
| 41 |
+
"model.language_model.layers.29.mlp.up_proj": 144,
|
| 42 |
+
"model.language_model.layers.29.self_attn.k_proj": 144,
|
| 43 |
+
"model.language_model.layers.29.self_attn.o_proj": 144,
|
| 44 |
+
"model.language_model.layers.29.self_attn.q_proj": 144,
|
| 45 |
+
"model.language_model.layers.29.self_attn.v_proj": 144,
|
| 46 |
+
"model.language_model.layers.4.mlp.down_proj": 144,
|
| 47 |
+
"model.language_model.layers.4.mlp.gate_proj": 144,
|
| 48 |
+
"model.language_model.layers.4.mlp.up_proj": 144,
|
| 49 |
+
"model.language_model.layers.4.self_attn.k_proj": 144,
|
| 50 |
+
"model.language_model.layers.4.self_attn.o_proj": 144,
|
| 51 |
+
"model.language_model.layers.4.self_attn.q_proj": 144,
|
| 52 |
+
"model.language_model.layers.4.self_attn.v_proj": 144,
|
| 53 |
+
"model.language_model.layers.5.mlp.down_proj": 144,
|
| 54 |
+
"model.language_model.layers.5.mlp.gate_proj": 144,
|
| 55 |
+
"model.language_model.layers.5.mlp.up_proj": 144,
|
| 56 |
+
"model.language_model.layers.5.self_attn.k_proj": 144,
|
| 57 |
+
"model.language_model.layers.5.self_attn.o_proj": 144,
|
| 58 |
+
"model.language_model.layers.5.self_attn.q_proj": 144,
|
| 59 |
+
"model.language_model.layers.5.self_attn.v_proj": 144
|
| 60 |
+
},
|
| 61 |
+
"arrow_config": null,
|
| 62 |
+
"auto_mapping": null,
|
| 63 |
+
"base_model_name_or_path": "/data/OPSD/assets/models/InternVL3_5-8B-HF",
|
| 64 |
+
"bias": "none",
|
| 65 |
+
"corda_config": null,
|
| 66 |
+
"ensure_weight_tying": false,
|
| 67 |
+
"eva_config": null,
|
| 68 |
+
"exclude_modules": null,
|
| 69 |
+
"fan_in_fan_out": false,
|
| 70 |
+
"inference_mode": true,
|
| 71 |
+
"init_lora_weights": true,
|
| 72 |
+
"layer_replication": null,
|
| 73 |
+
"layers_pattern": null,
|
| 74 |
+
"layers_to_transform": null,
|
| 75 |
+
"loftq_config": {},
|
| 76 |
+
"lora_alpha": 32,
|
| 77 |
+
"lora_bias": false,
|
| 78 |
+
"lora_dropout": 0.05,
|
| 79 |
+
"lora_ga_config": null,
|
| 80 |
+
"megatron_config": null,
|
| 81 |
+
"megatron_core": "megatron.core",
|
| 82 |
+
"modules_to_save": null,
|
| 83 |
+
"monteclora_config": null,
|
| 84 |
+
"peft_type": "LORA",
|
| 85 |
+
"peft_version": "0.20.0",
|
| 86 |
+
"qalora_group_size": 16,
|
| 87 |
+
"r": 16,
|
| 88 |
+
"rank_pattern": {
|
| 89 |
+
"model.language_model.layers.10.mlp.down_proj": 72,
|
| 90 |
+
"model.language_model.layers.10.mlp.gate_proj": 72,
|
| 91 |
+
"model.language_model.layers.10.mlp.up_proj": 72,
|
| 92 |
+
"model.language_model.layers.10.self_attn.k_proj": 72,
|
| 93 |
+
"model.language_model.layers.10.self_attn.o_proj": 72,
|
| 94 |
+
"model.language_model.layers.10.self_attn.q_proj": 72,
|
| 95 |
+
"model.language_model.layers.10.self_attn.v_proj": 72,
|
| 96 |
+
"model.language_model.layers.15.mlp.down_proj": 72,
|
| 97 |
+
"model.language_model.layers.15.mlp.gate_proj": 72,
|
| 98 |
+
"model.language_model.layers.15.mlp.up_proj": 72,
|
| 99 |
+
"model.language_model.layers.15.self_attn.k_proj": 72,
|
| 100 |
+
"model.language_model.layers.15.self_attn.o_proj": 72,
|
| 101 |
+
"model.language_model.layers.15.self_attn.q_proj": 72,
|
| 102 |
+
"model.language_model.layers.15.self_attn.v_proj": 72,
|
| 103 |
+
"model.language_model.layers.25.mlp.down_proj": 72,
|
| 104 |
+
"model.language_model.layers.25.mlp.gate_proj": 72,
|
| 105 |
+
"model.language_model.layers.25.mlp.up_proj": 72,
|
| 106 |
+
"model.language_model.layers.25.self_attn.k_proj": 72,
|
| 107 |
+
"model.language_model.layers.25.self_attn.o_proj": 72,
|
| 108 |
+
"model.language_model.layers.25.self_attn.q_proj": 72,
|
| 109 |
+
"model.language_model.layers.25.self_attn.v_proj": 72,
|
| 110 |
+
"model.language_model.layers.26.mlp.down_proj": 72,
|
| 111 |
+
"model.language_model.layers.26.mlp.gate_proj": 72,
|
| 112 |
+
"model.language_model.layers.26.mlp.up_proj": 72,
|
| 113 |
+
"model.language_model.layers.26.self_attn.k_proj": 72,
|
| 114 |
+
"model.language_model.layers.26.self_attn.o_proj": 72,
|
| 115 |
+
"model.language_model.layers.26.self_attn.q_proj": 72,
|
| 116 |
+
"model.language_model.layers.26.self_attn.v_proj": 72,
|
| 117 |
+
"model.language_model.layers.28.mlp.down_proj": 72,
|
| 118 |
+
"model.language_model.layers.28.mlp.gate_proj": 72,
|
| 119 |
+
"model.language_model.layers.28.mlp.up_proj": 72,
|
| 120 |
+
"model.language_model.layers.28.self_attn.k_proj": 72,
|
| 121 |
+
"model.language_model.layers.28.self_attn.o_proj": 72,
|
| 122 |
+
"model.language_model.layers.28.self_attn.q_proj": 72,
|
| 123 |
+
"model.language_model.layers.28.self_attn.v_proj": 72,
|
| 124 |
+
"model.language_model.layers.29.mlp.down_proj": 72,
|
| 125 |
+
"model.language_model.layers.29.mlp.gate_proj": 72,
|
| 126 |
+
"model.language_model.layers.29.mlp.up_proj": 72,
|
| 127 |
+
"model.language_model.layers.29.self_attn.k_proj": 72,
|
| 128 |
+
"model.language_model.layers.29.self_attn.o_proj": 72,
|
| 129 |
+
"model.language_model.layers.29.self_attn.q_proj": 72,
|
| 130 |
+
"model.language_model.layers.29.self_attn.v_proj": 72,
|
| 131 |
+
"model.language_model.layers.4.mlp.down_proj": 72,
|
| 132 |
+
"model.language_model.layers.4.mlp.gate_proj": 72,
|
| 133 |
+
"model.language_model.layers.4.mlp.up_proj": 72,
|
| 134 |
+
"model.language_model.layers.4.self_attn.k_proj": 72,
|
| 135 |
+
"model.language_model.layers.4.self_attn.o_proj": 72,
|
| 136 |
+
"model.language_model.layers.4.self_attn.q_proj": 72,
|
| 137 |
+
"model.language_model.layers.4.self_attn.v_proj": 72,
|
| 138 |
+
"model.language_model.layers.5.mlp.down_proj": 72,
|
| 139 |
+
"model.language_model.layers.5.mlp.gate_proj": 72,
|
| 140 |
+
"model.language_model.layers.5.mlp.up_proj": 72,
|
| 141 |
+
"model.language_model.layers.5.self_attn.k_proj": 72,
|
| 142 |
+
"model.language_model.layers.5.self_attn.o_proj": 72,
|
| 143 |
+
"model.language_model.layers.5.self_attn.q_proj": 72,
|
| 144 |
+
"model.language_model.layers.5.self_attn.v_proj": 72
|
| 145 |
+
},
|
| 146 |
+
"revision": null,
|
| 147 |
+
"target_modules": [
|
| 148 |
+
"model.language_model.layers.26.mlp.down_proj",
|
| 149 |
+
"model.language_model.layers.4.self_attn.o_proj",
|
| 150 |
+
"model.language_model.layers.29.self_attn.k_proj",
|
| 151 |
+
"model.language_model.layers.25.self_attn.k_proj",
|
| 152 |
+
"model.language_model.layers.26.self_attn.o_proj",
|
| 153 |
+
"model.language_model.layers.10.self_attn.k_proj",
|
| 154 |
+
"model.language_model.layers.5.mlp.up_proj",
|
| 155 |
+
"model.language_model.layers.4.mlp.up_proj",
|
| 156 |
+
"model.language_model.layers.29.self_attn.q_proj",
|
| 157 |
+
"model.language_model.layers.29.self_attn.o_proj",
|
| 158 |
+
"model.language_model.layers.4.mlp.gate_proj",
|
| 159 |
+
"model.language_model.layers.10.self_attn.q_proj",
|
| 160 |
+
"model.language_model.layers.25.mlp.gate_proj",
|
| 161 |
+
"model.language_model.layers.10.mlp.up_proj",
|
| 162 |
+
"model.language_model.layers.5.self_attn.k_proj",
|
| 163 |
+
"model.language_model.layers.15.self_attn.k_proj",
|
| 164 |
+
"model.language_model.layers.28.self_attn.q_proj",
|
| 165 |
+
"model.language_model.layers.15.self_attn.o_proj",
|
| 166 |
+
"model.language_model.layers.26.self_attn.v_proj",
|
| 167 |
+
"model.language_model.layers.28.mlp.down_proj",
|
| 168 |
+
"model.language_model.layers.25.self_attn.q_proj",
|
| 169 |
+
"model.language_model.layers.15.mlp.down_proj",
|
| 170 |
+
"model.language_model.layers.28.self_attn.k_proj",
|
| 171 |
+
"model.language_model.layers.28.self_attn.v_proj",
|
| 172 |
+
"model.language_model.layers.25.self_attn.o_proj",
|
| 173 |
+
"model.language_model.layers.29.mlp.gate_proj",
|
| 174 |
+
"model.language_model.layers.4.self_attn.k_proj",
|
| 175 |
+
"model.language_model.layers.5.mlp.gate_proj",
|
| 176 |
+
"model.language_model.layers.10.mlp.gate_proj",
|
| 177 |
+
"model.language_model.layers.29.self_attn.v_proj",
|
| 178 |
+
"model.language_model.layers.5.self_attn.o_proj",
|
| 179 |
+
"model.language_model.layers.29.mlp.down_proj",
|
| 180 |
+
"model.language_model.layers.4.self_attn.q_proj",
|
| 181 |
+
"model.language_model.layers.4.mlp.down_proj",
|
| 182 |
+
"model.language_model.layers.15.mlp.gate_proj",
|
| 183 |
+
"model.language_model.layers.26.self_attn.k_proj",
|
| 184 |
+
"model.language_model.layers.25.self_attn.v_proj",
|
| 185 |
+
"model.language_model.layers.10.self_attn.v_proj",
|
| 186 |
+
"model.language_model.layers.10.self_attn.o_proj",
|
| 187 |
+
"model.language_model.layers.28.mlp.up_proj",
|
| 188 |
+
"model.language_model.layers.5.self_attn.v_proj",
|
| 189 |
+
"model.language_model.layers.25.mlp.up_proj",
|
| 190 |
+
"model.language_model.layers.26.self_attn.q_proj",
|
| 191 |
+
"model.language_model.layers.25.mlp.down_proj",
|
| 192 |
+
"model.language_model.layers.26.mlp.gate_proj",
|
| 193 |
+
"model.language_model.layers.4.self_attn.v_proj",
|
| 194 |
+
"model.language_model.layers.5.self_attn.q_proj",
|
| 195 |
+
"model.language_model.layers.10.mlp.down_proj",
|
| 196 |
+
"model.language_model.layers.5.mlp.down_proj",
|
| 197 |
+
"model.language_model.layers.15.mlp.up_proj",
|
| 198 |
+
"model.language_model.layers.28.self_attn.o_proj",
|
| 199 |
+
"model.language_model.layers.28.mlp.gate_proj",
|
| 200 |
+
"model.language_model.layers.26.mlp.up_proj",
|
| 201 |
+
"model.language_model.layers.15.self_attn.v_proj",
|
| 202 |
+
"model.language_model.layers.29.mlp.up_proj",
|
| 203 |
+
"model.language_model.layers.15.self_attn.q_proj"
|
| 204 |
+
],
|
| 205 |
+
"target_parameters": null,
|
| 206 |
+
"task_type": "CAUSAL_LM",
|
| 207 |
+
"trainable_token_indices": null,
|
| 208 |
+
"use_bdlora": null,
|
| 209 |
+
"use_dora": false,
|
| 210 |
+
"use_qalora": false,
|
| 211 |
+
"use_rslora": false,
|
| 212 |
+
"velora_config": null
|
| 213 |
+
}
|
runs/fixed_internvl/removal/Random8_8109_s13/config.json
ADDED
|
@@ -0,0 +1,43 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"behavior": "removal",
|
| 3 |
+
"seed": 13,
|
| 4 |
+
"model": "/data/OPSD/assets/models/InternVL3_5-8B-HF",
|
| 5 |
+
"backbone": "internvl",
|
| 6 |
+
"train": "/data/OPSD/assets/v3/removal.jsonl",
|
| 7 |
+
"items": 1000,
|
| 8 |
+
"steps": 250,
|
| 9 |
+
"epochs": 1,
|
| 10 |
+
"steps_per_epoch": 250,
|
| 11 |
+
"world_size": 1,
|
| 12 |
+
"lr": 5e-05,
|
| 13 |
+
"scheduler": "cosine",
|
| 14 |
+
"warmup_steps": 16,
|
| 15 |
+
"final_fraction": 0.1,
|
| 16 |
+
"clip": 1.0,
|
| 17 |
+
"initial_loss_scale": 4.0,
|
| 18 |
+
"arm": {
|
| 19 |
+
"arm": "Random8_8109",
|
| 20 |
+
"layers": [
|
| 21 |
+
4,
|
| 22 |
+
5,
|
| 23 |
+
10,
|
| 24 |
+
15,
|
| 25 |
+
25,
|
| 26 |
+
26,
|
| 27 |
+
28,
|
| 28 |
+
29
|
| 29 |
+
],
|
| 30 |
+
"ranks": {
|
| 31 |
+
"4": 72,
|
| 32 |
+
"5": 72,
|
| 33 |
+
"10": 72,
|
| 34 |
+
"15": 72,
|
| 35 |
+
"25": 72,
|
| 36 |
+
"26": 72,
|
| 37 |
+
"28": 72,
|
| 38 |
+
"29": 72
|
| 39 |
+
},
|
| 40 |
+
"parameters": 43646976,
|
| 41 |
+
"budget": 43646976
|
| 42 |
+
}
|
| 43 |
+
}
|
runs/fixed_internvl/removal/Random8_8109_s13/steps.jsonl
ADDED
|
@@ -0,0 +1,250 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{"step": 1, "steps": 250, "lr": 3.125e-06, "grad_norm": 3.7761003971099854, "loss_scale": 4.0, "retries": 0, "seconds": 5.7, "max_memory_gb": 16.8, "ce": 0.7595979786710814, "rank": 0.06138449162244797}
|
| 2 |
+
{"step": 2, "steps": 250, "lr": 6.25e-06, "grad_norm": 3.403146743774414, "loss_scale": 4.0, "retries": 0, "seconds": 10.7, "max_memory_gb": 17.14, "ce": 0.77588824970735, "rank": 0.0}
|
| 3 |
+
{"step": 3, "steps": 250, "lr": 9.375000000000001e-06, "grad_norm": 12.42663860321045, "loss_scale": 4.0, "retries": 0, "seconds": 15.6, "max_memory_gb": 17.15, "ce": 1.284779111854732, "rank": 0.0}
|
| 4 |
+
{"step": 4, "steps": 250, "lr": 1.25e-05, "grad_norm": 5.2295145988464355, "loss_scale": 4.0, "retries": 0, "seconds": 20.5, "max_memory_gb": 17.15, "ce": 0.6169018893915563, "rank": 0.0}
|
| 5 |
+
{"step": 5, "steps": 250, "lr": 1.5625e-05, "grad_norm": 0.12217918783426285, "loss_scale": 4.0, "retries": 0, "seconds": 25.3, "max_memory_gb": 17.15, "ce": 0.03942957106301037, "rank": 0.0}
|
| 6 |
+
{"step": 6, "steps": 250, "lr": 1.8750000000000002e-05, "grad_norm": 2.239780902862549, "loss_scale": 4.0, "retries": 0, "seconds": 30.2, "max_memory_gb": 17.15, "ce": 0.32567646956886165, "rank": 0.0}
|
| 7 |
+
{"step": 7, "steps": 250, "lr": 2.1875e-05, "grad_norm": 3.0038328170776367, "loss_scale": 4.0, "retries": 0, "seconds": 35.0, "max_memory_gb": 17.15, "ce": 0.4729004304881528, "rank": 0.0}
|
| 8 |
+
{"step": 8, "steps": 250, "lr": 2.5e-05, "grad_norm": 1.7236460447311401, "loss_scale": 4.0, "retries": 0, "seconds": 39.7, "max_memory_gb": 17.15, "ce": 0.40231985645368695, "rank": 0.0}
|
| 9 |
+
{"step": 9, "steps": 250, "lr": 2.8125000000000003e-05, "grad_norm": 2.348240852355957, "loss_scale": 4.0, "retries": 0, "seconds": 44.1, "max_memory_gb": 17.15, "ce": 0.32767600775696337, "rank": 0.0}
|
| 10 |
+
{"step": 10, "steps": 250, "lr": 3.125e-05, "grad_norm": 0.30852895975112915, "loss_scale": 4.0, "retries": 0, "seconds": 48.5, "max_memory_gb": 17.15, "ce": 0.13314664155768696, "rank": 0.0}
|
| 11 |
+
{"step": 11, "steps": 250, "lr": 3.4375e-05, "grad_norm": 0.3879563510417938, "loss_scale": 4.0, "retries": 0, "seconds": 52.9, "max_memory_gb": 17.15, "ce": 0.21471118606132222, "rank": 0.0}
|
| 12 |
+
{"step": 12, "steps": 250, "lr": 3.7500000000000003e-05, "grad_norm": 0.24778907001018524, "loss_scale": 4.0, "retries": 0, "seconds": 57.1, "max_memory_gb": 17.15, "ce": 0.11911376717034727, "rank": 0.0}
|
| 13 |
+
{"step": 13, "steps": 250, "lr": 4.0625000000000005e-05, "grad_norm": 0.38730937242507935, "loss_scale": 4.0, "retries": 0, "seconds": 61.2, "max_memory_gb": 17.15, "ce": 0.24310545108710357, "rank": 0.0}
|
| 14 |
+
{"step": 14, "steps": 250, "lr": 4.375e-05, "grad_norm": 1.8774832487106323, "loss_scale": 4.0, "retries": 0, "seconds": 65.5, "max_memory_gb": 17.15, "ce": 0.36888978304341435, "rank": 0.0}
|
| 15 |
+
{"step": 15, "steps": 250, "lr": 4.6875e-05, "grad_norm": 0.9704502820968628, "loss_scale": 4.0, "retries": 0, "seconds": 70.1, "max_memory_gb": 17.15, "ce": 0.23808899113510051, "rank": 0.04857635498046875}
|
| 16 |
+
{"step": 16, "steps": 250, "lr": 5e-05, "grad_norm": 0.2838883399963379, "loss_scale": 4.0, "retries": 0, "seconds": 74.6, "max_memory_gb": 17.15, "ce": 0.099194876049296, "rank": 0.0}
|
| 17 |
+
{"step": 17, "steps": 250, "lr": 5e-05, "grad_norm": 1.0310673713684082, "loss_scale": 4.0, "retries": 0, "seconds": 79.1, "max_memory_gb": 17.15, "ce": 0.24387030515936203, "rank": 0.0}
|
| 18 |
+
{"step": 18, "steps": 250, "lr": 4.999797225177623e-05, "grad_norm": 0.772700309753418, "loss_scale": 4.0, "retries": 0, "seconds": 83.4, "max_memory_gb": 17.15, "ce": 0.20789141237037256, "rank": 0.0}
|
| 19 |
+
{"step": 19, "steps": 250, "lr": 4.999188937259492e-05, "grad_norm": 0.9665167331695557, "loss_scale": 4.0, "retries": 0, "seconds": 88.0, "max_memory_gb": 17.15, "ce": 0.1986706805691938, "rank": 0.10159513354301453}
|
| 20 |
+
{"step": 20, "steps": 250, "lr": 4.998175245886032e-05, "grad_norm": 0.20761625468730927, "loss_scale": 4.0, "retries": 0, "seconds": 92.5, "max_memory_gb": 17.15, "ce": 0.028017897382596857, "rank": 0.12414707243442535}
|
| 21 |
+
{"step": 21, "steps": 250, "lr": 4.996756333769319e-05, "grad_norm": 0.7662237286567688, "loss_scale": 4.0, "retries": 0, "seconds": 96.9, "max_memory_gb": 17.15, "ce": 0.1258273978892248, "rank": 0.03195102512836456}
|
| 22 |
+
{"step": 22, "steps": 250, "lr": 4.994932456660157e-05, "grad_norm": 0.7072165012359619, "loss_scale": 4.0, "retries": 0, "seconds": 101.6, "max_memory_gb": 17.15, "ce": 0.1530128974627587, "rank": 0.11271420866250992}
|
| 23 |
+
{"step": 23, "steps": 250, "lr": 4.992703943301973e-05, "grad_norm": 0.9285286664962769, "loss_scale": 4.0, "retries": 0, "seconds": 106.2, "max_memory_gb": 17.15, "ce": 0.12265698498958955, "rank": 0.0}
|
| 24 |
+
{"step": 24, "steps": 250, "lr": 4.9900711953715675e-05, "grad_norm": 0.41373196244239807, "loss_scale": 4.0, "retries": 0, "seconds": 110.7, "max_memory_gb": 17.15, "ce": 0.07438416453078389, "rank": 0.0}
|
| 25 |
+
{"step": 25, "steps": 250, "lr": 4.987034687406713e-05, "grad_norm": 1.5269662141799927, "loss_scale": 4.0, "retries": 0, "seconds": 115.4, "max_memory_gb": 17.15, "ce": 0.1675412495387718, "rank": 0.0}
|
| 26 |
+
{"step": 26, "steps": 250, "lr": 4.9835949667206215e-05, "grad_norm": 1.1838324069976807, "loss_scale": 4.0, "retries": 0, "seconds": 120.1, "max_memory_gb": 17.15, "ce": 0.15762901381822303, "rank": 0.0}
|
| 27 |
+
{"step": 27, "steps": 250, "lr": 4.9797526533032945e-05, "grad_norm": 2.0657522678375244, "loss_scale": 4.0, "retries": 0, "seconds": 124.5, "max_memory_gb": 17.15, "ce": 0.15398138445743825, "rank": 0.1094573512673378}
|
| 28 |
+
{"step": 28, "steps": 250, "lr": 4.9755084397097716e-05, "grad_norm": 1.7035921812057495, "loss_scale": 4.0, "retries": 0, "seconds": 129.2, "max_memory_gb": 17.15, "ce": 0.19301975500275148, "rank": 0.0}
|
| 29 |
+
{"step": 29, "steps": 250, "lr": 4.970863090935304e-05, "grad_norm": 2.9073667526245117, "loss_scale": 4.0, "retries": 0, "seconds": 133.6, "max_memory_gb": 17.15, "ce": 0.556884200690547, "rank": 0.0}
|
| 30 |
+
{"step": 30, "steps": 250, "lr": 4.965817444277468e-05, "grad_norm": 0.16702879965305328, "loss_scale": 4.0, "retries": 0, "seconds": 138.1, "max_memory_gb": 17.15, "ce": 0.01218267262856898, "rank": 0.0}
|
| 31 |
+
{"step": 31, "steps": 250, "lr": 4.9603724091852456e-05, "grad_norm": 3.685203790664673, "loss_scale": 4.0, "retries": 0, "seconds": 142.8, "max_memory_gb": 17.15, "ce": 0.6748789069788472, "rank": 0.0}
|
| 32 |
+
{"step": 32, "steps": 250, "lr": 4.9545289670951025e-05, "grad_norm": 0.2597874104976654, "loss_scale": 4.0, "retries": 0, "seconds": 147.7, "max_memory_gb": 17.15, "ce": 0.025036398859811015, "rank": 0.0}
|
| 33 |
+
{"step": 33, "steps": 250, "lr": 4.948288171254089e-05, "grad_norm": 2.444973945617676, "loss_scale": 4.0, "retries": 0, "seconds": 152.4, "max_memory_gb": 17.15, "ce": 0.24505472415330587, "rank": 0.0}
|
| 34 |
+
{"step": 34, "steps": 250, "lr": 4.941651146529998e-05, "grad_norm": 0.760742723941803, "loss_scale": 4.0, "retries": 0, "seconds": 157.2, "max_memory_gb": 17.15, "ce": 0.09644156398280757, "rank": 0.0}
|
| 35 |
+
{"step": 35, "steps": 250, "lr": 4.9346190892086174e-05, "grad_norm": 0.45156097412109375, "loss_scale": 4.0, "retries": 0, "seconds": 161.9, "max_memory_gb": 17.15, "ce": 0.010829953185748309, "rank": 0.0}
|
| 36 |
+
{"step": 36, "steps": 250, "lr": 4.927193266778101e-05, "grad_norm": 2.5983469486236572, "loss_scale": 4.0, "retries": 0, "seconds": 166.5, "max_memory_gb": 17.15, "ce": 0.2965478610876744, "rank": 0.0}
|
| 37 |
+
{"step": 37, "steps": 250, "lr": 4.919375017700515e-05, "grad_norm": 0.6087723970413208, "loss_scale": 4.0, "retries": 0, "seconds": 171.1, "max_memory_gb": 17.15, "ce": 0.18752477883754182, "rank": 0.0}
|
| 38 |
+
{"step": 38, "steps": 250, "lr": 4.911165751170588e-05, "grad_norm": 0.8354756236076355, "loss_scale": 4.0, "retries": 0, "seconds": 175.9, "max_memory_gb": 17.15, "ce": 0.14350493834353983, "rank": 0.0}
|
| 39 |
+
{"step": 39, "steps": 250, "lr": 4.902566946861708e-05, "grad_norm": 1.1777318716049194, "loss_scale": 4.0, "retries": 0, "seconds": 180.5, "max_memory_gb": 17.15, "ce": 0.13993925189060974, "rank": 0.12523168325424194}
|
| 40 |
+
{"step": 40, "steps": 250, "lr": 4.89358015465922e-05, "grad_norm": 0.14216859638690948, "loss_scale": 4.0, "retries": 0, "seconds": 185.2, "max_memory_gb": 17.15, "ce": 0.013376600654737558, "rank": 0.0}
|
| 41 |
+
{"step": 41, "steps": 250, "lr": 4.884206994381078e-05, "grad_norm": 2.2962160110473633, "loss_scale": 4.0, "retries": 0, "seconds": 190.1, "max_memory_gb": 17.15, "ce": 0.4204457474861556, "rank": 0.0}
|
| 42 |
+
{"step": 42, "steps": 250, "lr": 4.874449155485866e-05, "grad_norm": 0.06967756152153015, "loss_scale": 4.0, "retries": 0, "seconds": 194.8, "max_memory_gb": 17.15, "ce": 0.006814023769038613, "rank": 0.0}
|
| 43 |
+
{"step": 43, "steps": 250, "lr": 4.864308396768294e-05, "grad_norm": 0.2647936940193176, "loss_scale": 4.0, "retries": 0, "seconds": 199.3, "max_memory_gb": 17.15, "ce": 0.01582295757907559, "rank": 0.0}
|
| 44 |
+
{"step": 44, "steps": 250, "lr": 4.853786546042184e-05, "grad_norm": 5.177055835723877, "loss_scale": 4.0, "retries": 0, "seconds": 203.9, "max_memory_gb": 17.15, "ce": 0.25289491477997217, "rank": 0.0}
|
| 45 |
+
{"step": 45, "steps": 250, "lr": 4.842885499811012e-05, "grad_norm": 2.2763900756835938, "loss_scale": 4.0, "retries": 0, "seconds": 208.7, "max_memory_gb": 17.15, "ce": 0.3638824237877998, "rank": 0.0}
|
| 46 |
+
{"step": 46, "steps": 250, "lr": 4.831607222926078e-05, "grad_norm": 1.377199411392212, "loss_scale": 4.0, "retries": 0, "seconds": 213.2, "max_memory_gb": 17.15, "ce": 0.07580477614101255, "rank": 0.0}
|
| 47 |
+
{"step": 47, "steps": 250, "lr": 4.8199537482323545e-05, "grad_norm": 0.2930256426334381, "loss_scale": 4.0, "retries": 0, "seconds": 217.9, "max_memory_gb": 17.15, "ce": 0.026419619129228522, "rank": 0.0}
|
| 48 |
+
{"step": 48, "steps": 250, "lr": 4.807927176202073e-05, "grad_norm": 0.07781509310007095, "loss_scale": 4.0, "retries": 0, "seconds": 222.5, "max_memory_gb": 17.15, "ce": 0.004765152671097894, "rank": 0.0}
|
| 49 |
+
{"step": 49, "steps": 250, "lr": 4.795529674556129e-05, "grad_norm": 0.0867016389966011, "loss_scale": 4.0, "retries": 0, "seconds": 227.2, "max_memory_gb": 17.15, "ce": 0.009470318582316395, "rank": 0.0}
|
| 50 |
+
{"step": 50, "steps": 250, "lr": 4.782763477873361e-05, "grad_norm": 0.7647636532783508, "loss_scale": 4.0, "retries": 0, "seconds": 231.9, "max_memory_gb": 17.15, "ce": 0.08688920158601832, "rank": 0.0}
|
| 51 |
+
{"step": 51, "steps": 250, "lr": 4.769630887187782e-05, "grad_norm": 1.809694528579712, "loss_scale": 4.0, "retries": 0, "seconds": 236.4, "max_memory_gb": 17.15, "ce": 0.09052680479180708, "rank": 0.0}
|
| 52 |
+
{"step": 52, "steps": 250, "lr": 4.756134269573832e-05, "grad_norm": 0.44052642583847046, "loss_scale": 4.0, "retries": 0, "seconds": 241.2, "max_memory_gb": 17.15, "ce": 0.03328572891950898, "rank": 0.0}
|
| 53 |
+
{"step": 53, "steps": 250, "lr": 4.7422760577197226e-05, "grad_norm": 1.0940062999725342, "loss_scale": 4.0, "retries": 0, "seconds": 245.7, "max_memory_gb": 17.15, "ce": 0.07345146537136316, "rank": 0.0}
|
| 54 |
+
{"step": 54, "steps": 250, "lr": 4.728058749488963e-05, "grad_norm": 1.0393669605255127, "loss_scale": 4.0, "retries": 0, "seconds": 250.3, "max_memory_gb": 17.15, "ce": 0.0772342735544953, "rank": 0.0}
|
| 55 |
+
{"step": 55, "steps": 250, "lr": 4.713484907470133e-05, "grad_norm": 0.1741454154253006, "loss_scale": 4.0, "retries": 0, "seconds": 254.9, "max_memory_gb": 17.15, "ce": 0.015616230067280412, "rank": 0.0}
|
| 56 |
+
{"step": 56, "steps": 250, "lr": 4.6985571585149876e-05, "grad_norm": 0.3882586359977722, "loss_scale": 4.0, "retries": 0, "seconds": 259.5, "max_memory_gb": 17.15, "ce": 0.03392834408987255, "rank": 0.0}
|
| 57 |
+
{"step": 57, "steps": 250, "lr": 4.6832781932649884e-05, "grad_norm": 0.11242366582155228, "loss_scale": 4.0, "retries": 0, "seconds": 264.1, "max_memory_gb": 17.15, "ce": 0.010982670853991294, "rank": 0.0}
|
| 58 |
+
{"step": 58, "steps": 250, "lr": 4.667650765666325e-05, "grad_norm": 0.04125596955418587, "loss_scale": 4.0, "retries": 0, "seconds": 268.9, "max_memory_gb": 17.15, "ce": 0.0027928184063057415, "rank": 0.0}
|
| 59 |
+
{"step": 59, "steps": 250, "lr": 4.651677692473538e-05, "grad_norm": 1.2229297161102295, "loss_scale": 4.0, "retries": 0, "seconds": 273.5, "max_memory_gb": 17.15, "ce": 0.15687054604313744, "rank": 0.0}
|
| 60 |
+
{"step": 60, "steps": 250, "lr": 4.635361852741808e-05, "grad_norm": 0.585181474685669, "loss_scale": 4.0, "retries": 0, "seconds": 278.2, "max_memory_gb": 17.15, "ce": 0.06537858264937313, "rank": 0.0}
|
| 61 |
+
{"step": 61, "steps": 250, "lr": 4.6187061873080264e-05, "grad_norm": 1.3330243825912476, "loss_scale": 4.0, "retries": 0, "seconds": 282.9, "max_memory_gb": 17.15, "ce": 0.21125652709997667, "rank": 0.0}
|
| 62 |
+
{"step": 62, "steps": 250, "lr": 4.601713698260727e-05, "grad_norm": 0.2484220266342163, "loss_scale": 4.0, "retries": 0, "seconds": 287.4, "max_memory_gb": 17.15, "ce": 0.03144799739220616, "rank": 0.0}
|
| 63 |
+
{"step": 63, "steps": 250, "lr": 4.5843874483989746e-05, "grad_norm": 0.3626595735549927, "loss_scale": 4.0, "retries": 0, "seconds": 292.1, "max_memory_gb": 17.15, "ce": 0.031921143111503625, "rank": 0.0}
|
| 64 |
+
{"step": 64, "steps": 250, "lr": 4.566730560680316e-05, "grad_norm": 0.15569229423999786, "loss_scale": 4.0, "retries": 0, "seconds": 297.0, "max_memory_gb": 17.15, "ce": 0.010426584134620498, "rank": 0.0}
|
| 65 |
+
{"step": 65, "steps": 250, "lr": 4.548746217657878e-05, "grad_norm": 0.8346769213676453, "loss_scale": 4.0, "retries": 0, "seconds": 301.9, "max_memory_gb": 17.15, "ce": 0.14983843135769348, "rank": 0.0}
|
| 66 |
+
{"step": 66, "steps": 250, "lr": 4.5304376609067396e-05, "grad_norm": 1.6800248622894287, "loss_scale": 4.0, "retries": 0, "seconds": 306.6, "max_memory_gb": 17.15, "ce": 0.21657303803658579, "rank": 0.11598781496286392}
|
| 67 |
+
{"step": 67, "steps": 250, "lr": 4.5118081904396504e-05, "grad_norm": 0.0433015339076519, "loss_scale": 4.0, "retries": 0, "seconds": 311.4, "max_memory_gb": 17.15, "ce": 0.003915782275726087, "rank": 0.0}
|
| 68 |
+
{"step": 68, "steps": 250, "lr": 4.4928611641122225e-05, "grad_norm": 0.11323267966508865, "loss_scale": 4.0, "retries": 0, "seconds": 317.5, "max_memory_gb": 17.15, "ce": 0.008081051821136498, "rank": 0.0}
|
| 69 |
+
{"step": 69, "steps": 250, "lr": 4.473599997017701e-05, "grad_norm": 2.7323286533355713, "loss_scale": 4.0, "retries": 0, "seconds": 322.7, "max_memory_gb": 17.15, "ce": 0.6148896825266092, "rank": 0.0}
|
| 70 |
+
{"step": 70, "steps": 250, "lr": 4.4540281608714053e-05, "grad_norm": 0.22252783179283142, "loss_scale": 4.0, "retries": 0, "seconds": 328.8, "max_memory_gb": 17.15, "ce": 0.005429534368431632, "rank": 0.0}
|
| 71 |
+
{"step": 71, "steps": 250, "lr": 4.434149183384977e-05, "grad_norm": 0.0575721301138401, "loss_scale": 4.0, "retries": 0, "seconds": 335.0, "max_memory_gb": 17.15, "ce": 0.003877505250784452, "rank": 0.0}
|
| 72 |
+
{"step": 72, "steps": 250, "lr": 4.413966647630534e-05, "grad_norm": 0.8635813593864441, "loss_scale": 4.0, "retries": 0, "seconds": 340.6, "max_memory_gb": 17.15, "ce": 0.09849067382538124, "rank": 0.0}
|
| 73 |
+
{"step": 73, "steps": 250, "lr": 4.3934841913948323e-05, "grad_norm": 1.1870630979537964, "loss_scale": 4.0, "retries": 0, "seconds": 346.8, "max_memory_gb": 17.15, "ce": 0.13323407536654486, "rank": 0.0}
|
| 74 |
+
{"step": 74, "steps": 250, "lr": 4.372705506523583e-05, "grad_norm": 1.0583691596984863, "loss_scale": 4.0, "retries": 0, "seconds": 353.2, "max_memory_gb": 17.15, "ce": 0.15074854595286524, "rank": 0.0}
|
| 75 |
+
{"step": 75, "steps": 250, "lr": 4.351634338256017e-05, "grad_norm": 3.7785937786102295, "loss_scale": 4.0, "retries": 0, "seconds": 359.3, "max_memory_gb": 17.15, "ce": 0.7088271363877539, "rank": 0.0}
|
| 76 |
+
{"step": 76, "steps": 250, "lr": 4.330274484549825e-05, "grad_norm": 1.1734747886657715, "loss_scale": 4.0, "retries": 0, "seconds": 364.0, "max_memory_gb": 17.15, "ce": 0.3072137721569561, "rank": 0.0}
|
| 77 |
+
{"step": 77, "steps": 250, "lr": 4.308629795396599e-05, "grad_norm": 1.0209218263626099, "loss_scale": 4.0, "retries": 0, "seconds": 368.8, "max_memory_gb": 17.15, "ce": 0.04842040138828452, "rank": 0.0}
|
| 78 |
+
{"step": 78, "steps": 250, "lr": 4.286704172127894e-05, "grad_norm": 0.19597209990024567, "loss_scale": 4.0, "retries": 0, "seconds": 373.4, "max_memory_gb": 17.15, "ce": 0.02514895243166393, "rank": 0.0}
|
| 79 |
+
{"step": 79, "steps": 250, "lr": 4.264501566712033e-05, "grad_norm": 0.1493314653635025, "loss_scale": 4.0, "retries": 0, "seconds": 377.9, "max_memory_gb": 17.15, "ce": 0.007470947249657911, "rank": 0.0}
|
| 80 |
+
{"step": 80, "steps": 250, "lr": 4.2420259810417894e-05, "grad_norm": 0.22647909820079803, "loss_scale": 4.0, "retries": 0, "seconds": 382.5, "max_memory_gb": 17.15, "ce": 0.01679691556182661, "rank": 0.0}
|
| 81 |
+
{"step": 81, "steps": 250, "lr": 4.219281466213066e-05, "grad_norm": 0.6226966381072998, "loss_scale": 4.0, "retries": 0, "seconds": 387.2, "max_memory_gb": 17.15, "ce": 0.06643276059548953, "rank": 0.0}
|
| 82 |
+
{"step": 82, "steps": 250, "lr": 4.1962721217947135e-05, "grad_norm": 0.8036105036735535, "loss_scale": 4.0, "retries": 0, "seconds": 391.8, "max_memory_gb": 17.15, "ce": 0.09658532040930368, "rank": 0.0}
|
| 83 |
+
{"step": 83, "steps": 250, "lr": 4.173002095089599e-05, "grad_norm": 1.0406708717346191, "loss_scale": 4.0, "retries": 0, "seconds": 396.6, "max_memory_gb": 17.15, "ce": 0.17587804193499323, "rank": 0.0}
|
| 84 |
+
{"step": 84, "steps": 250, "lr": 4.149475580387084e-05, "grad_norm": 0.641058623790741, "loss_scale": 4.0, "retries": 0, "seconds": 401.3, "max_memory_gb": 17.15, "ce": 0.03694303313477576, "rank": 0.0}
|
| 85 |
+
{"step": 85, "steps": 250, "lr": 4.1256968182070275e-05, "grad_norm": 0.04725733771920204, "loss_scale": 4.0, "retries": 0, "seconds": 405.9, "max_memory_gb": 17.15, "ce": 0.0018696839811127575, "rank": 0.0}
|
| 86 |
+
{"step": 86, "steps": 250, "lr": 4.101670094535453e-05, "grad_norm": 0.5125813484191895, "loss_scale": 4.0, "retries": 0, "seconds": 410.6, "max_memory_gb": 17.15, "ce": 0.05924907155076653, "rank": 0.0}
|
| 87 |
+
{"step": 87, "steps": 250, "lr": 4.077399740052027e-05, "grad_norm": 0.8968205451965332, "loss_scale": 4.0, "retries": 0, "seconds": 415.1, "max_memory_gb": 17.15, "ce": 0.1486645103709634, "rank": 0.0}
|
| 88 |
+
{"step": 88, "steps": 250, "lr": 4.0528901293494775e-05, "grad_norm": 0.42111098766326904, "loss_scale": 4.0, "retries": 0, "seconds": 419.7, "max_memory_gb": 17.15, "ce": 0.045655879244804964, "rank": 0.0}
|
| 89 |
+
{"step": 89, "steps": 250, "lr": 4.0281456801451004e-05, "grad_norm": 0.16239379346370697, "loss_scale": 4.0, "retries": 0, "seconds": 424.2, "max_memory_gb": 17.15, "ce": 0.023137288033922232, "rank": 0.1249847337603569}
|
| 90 |
+
{"step": 90, "steps": 250, "lr": 4.0031708524844884e-05, "grad_norm": 0.4559462368488312, "loss_scale": 4.0, "retries": 0, "seconds": 428.9, "max_memory_gb": 17.15, "ce": 0.05151790027593961, "rank": 0.0}
|
| 91 |
+
{"step": 91, "steps": 250, "lr": 3.977970147937635e-05, "grad_norm": 2.360530138015747, "loss_scale": 4.0, "retries": 0, "seconds": 433.5, "max_memory_gb": 17.15, "ce": 0.4985953432178576, "rank": 0.0}
|
| 92 |
+
{"step": 92, "steps": 250, "lr": 3.952548108787553e-05, "grad_norm": 2.1398277282714844, "loss_scale": 4.0, "retries": 0, "seconds": 438.1, "max_memory_gb": 17.15, "ce": 0.1744529815982787, "rank": 0.0}
|
| 93 |
+
{"step": 93, "steps": 250, "lr": 3.9269093172115496e-05, "grad_norm": 1.6038787364959717, "loss_scale": 4.0, "retries": 0, "seconds": 442.7, "max_memory_gb": 17.15, "ce": 0.08688782390117922, "rank": 0.0}
|
| 94 |
+
{"step": 94, "steps": 250, "lr": 3.901058394455323e-05, "grad_norm": 0.7134923338890076, "loss_scale": 4.0, "retries": 0, "seconds": 447.4, "max_memory_gb": 17.15, "ce": 0.03682940032740589, "rank": 0.0}
|
| 95 |
+
{"step": 95, "steps": 250, "lr": 3.875e-05, "grad_norm": 2.924229383468628, "loss_scale": 4.0, "retries": 0, "seconds": 452.1, "max_memory_gb": 17.15, "ce": 0.5272795144110205, "rank": 0.0}
|
| 96 |
+
{"step": 96, "steps": 250, "lr": 3.8487388307223005e-05, "grad_norm": 2.7075748443603516, "loss_scale": 4.0, "retries": 0, "seconds": 458.0, "max_memory_gb": 17.15, "ce": 0.4130865610263754, "rank": 0.33765891194343567}
|
| 97 |
+
{"step": 97, "steps": 250, "lr": 3.822279620047943e-05, "grad_norm": 0.4087831676006317, "loss_scale": 4.0, "retries": 0, "seconds": 463.2, "max_memory_gb": 17.15, "ce": 0.021825352112500696, "rank": 0.0}
|
| 98 |
+
{"step": 98, "steps": 250, "lr": 3.795627137098479e-05, "grad_norm": 0.9990578889846802, "loss_scale": 4.0, "retries": 0, "seconds": 467.7, "max_memory_gb": 17.15, "ce": 0.060645850338232776, "rank": 0.0}
|
| 99 |
+
{"step": 99, "steps": 250, "lr": 3.7687861858316844e-05, "grad_norm": 0.07412222027778625, "loss_scale": 4.0, "retries": 0, "seconds": 472.3, "max_memory_gb": 17.15, "ce": 0.010703067338454275, "rank": 0.0}
|
| 100 |
+
{"step": 100, "steps": 250, "lr": 3.741761604175668e-05, "grad_norm": 0.5737793445587158, "loss_scale": 4.0, "retries": 0, "seconds": 477.1, "max_memory_gb": 17.15, "ce": 0.09834533884668417, "rank": 0.0}
|
| 101 |
+
{"step": 101, "steps": 250, "lr": 3.714558263156872e-05, "grad_norm": 0.14577187597751617, "loss_scale": 4.0, "retries": 0, "seconds": 481.6, "max_memory_gb": 17.15, "ce": 0.013009477865125518, "rank": 0.0}
|
| 102 |
+
{"step": 102, "steps": 250, "lr": 3.6871810660220904e-05, "grad_norm": 0.15655216574668884, "loss_scale": 4.0, "retries": 0, "seconds": 486.1, "max_memory_gb": 17.15, "ce": 0.014180836519244622, "rank": 0.0}
|
| 103 |
+
{"step": 103, "steps": 250, "lr": 3.659634947354686e-05, "grad_norm": 0.08062878996133804, "loss_scale": 4.0, "retries": 0, "seconds": 490.8, "max_memory_gb": 17.15, "ce": 0.0069681997747466085, "rank": 0.0}
|
| 104 |
+
{"step": 104, "steps": 250, "lr": 3.631924872185169e-05, "grad_norm": 0.2541086673736572, "loss_scale": 4.0, "retries": 0, "seconds": 495.5, "max_memory_gb": 17.15, "ce": 0.019108384016362834, "rank": 0.12498650699853897}
|
| 105 |
+
{"step": 105, "steps": 250, "lr": 3.6040558350962674e-05, "grad_norm": 0.6796513795852661, "loss_scale": 4.0, "retries": 0, "seconds": 499.7, "max_memory_gb": 17.15, "ce": 0.0817002909184339, "rank": 0.0}
|
| 106 |
+
{"step": 106, "steps": 250, "lr": 3.576032859322687e-05, "grad_norm": 0.07372640818357468, "loss_scale": 4.0, "retries": 0, "seconds": 504.1, "max_memory_gb": 17.15, "ce": 0.0020369478033899213, "rank": 0.10819187760353088}
|
| 107 |
+
{"step": 107, "steps": 250, "lr": 3.5478609958457056e-05, "grad_norm": 1.183029055595398, "loss_scale": 4.0, "retries": 0, "seconds": 509.1, "max_memory_gb": 17.15, "ce": 0.1377446800047437, "rank": 0.0}
|
| 108 |
+
{"step": 108, "steps": 250, "lr": 3.519545322482755e-05, "grad_norm": 1.9179401397705078, "loss_scale": 4.0, "retries": 0, "seconds": 514.5, "max_memory_gb": 17.15, "ce": 0.204535707658124, "rank": 0.0}
|
| 109 |
+
{"step": 109, "steps": 250, "lr": 3.491090942972179e-05, "grad_norm": 0.8877097368240356, "loss_scale": 4.0, "retries": 0, "seconds": 519.1, "max_memory_gb": 17.15, "ce": 0.1308330542051408, "rank": 0.0}
|
| 110 |
+
{"step": 110, "steps": 250, "lr": 3.4625029860533135e-05, "grad_norm": 0.5583608150482178, "loss_scale": 4.0, "retries": 0, "seconds": 523.7, "max_memory_gb": 17.15, "ce": 0.034422320782596216, "rank": 0.0}
|
| 111 |
+
{"step": 111, "steps": 250, "lr": 3.433786604542056e-05, "grad_norm": 0.2459893822669983, "loss_scale": 4.0, "retries": 0, "seconds": 528.3, "max_memory_gb": 17.15, "ce": 0.0223442992926266, "rank": 0.0}
|
| 112 |
+
{"step": 112, "steps": 250, "lr": 3.404946974402104e-05, "grad_norm": 0.04791298508644104, "loss_scale": 4.0, "retries": 0, "seconds": 533.4, "max_memory_gb": 17.15, "ce": 0.003302618796169554, "rank": 0.0}
|
| 113 |
+
{"step": 113, "steps": 250, "lr": 3.375989293812019e-05, "grad_norm": 0.1614055335521698, "loss_scale": 4.0, "retries": 0, "seconds": 538.1, "max_memory_gb": 17.15, "ce": 0.016693675264377816, "rank": 0.0}
|
| 114 |
+
{"step": 114, "steps": 250, "lr": 3.3469187822282766e-05, "grad_norm": 0.6184639930725098, "loss_scale": 4.0, "retries": 0, "seconds": 542.9, "max_memory_gb": 17.15, "ce": 0.10758040811333558, "rank": 0.0}
|
| 115 |
+
{"step": 115, "steps": 250, "lr": 3.317740679444499e-05, "grad_norm": 0.37941187620162964, "loss_scale": 4.0, "retries": 0, "seconds": 547.5, "max_memory_gb": 17.15, "ce": 0.016648689917929005, "rank": 0.0}
|
| 116 |
+
{"step": 116, "steps": 250, "lr": 3.2884602446470056e-05, "grad_norm": 0.5615264177322388, "loss_scale": 4.0, "retries": 0, "seconds": 552.0, "max_memory_gb": 17.15, "ce": 0.17154525068531257, "rank": 0.0}
|
| 117 |
+
{"step": 117, "steps": 250, "lr": 3.259082755466876e-05, "grad_norm": 0.07571326941251755, "loss_scale": 4.0, "retries": 0, "seconds": 556.6, "max_memory_gb": 17.15, "ce": 0.004165936246863566, "rank": 0.0}
|
| 118 |
+
{"step": 118, "steps": 250, "lr": 3.229613507028686e-05, "grad_norm": 1.1137748956680298, "loss_scale": 4.0, "retries": 0, "seconds": 561.1, "max_memory_gb": 17.15, "ce": 0.21068554899125047, "rank": 0.0}
|
| 119 |
+
{"step": 119, "steps": 250, "lr": 3.2000578109961004e-05, "grad_norm": 0.07609733194112778, "loss_scale": 4.0, "retries": 0, "seconds": 565.5, "max_memory_gb": 17.15, "ce": 0.00440942256523158, "rank": 0.0}
|
| 120 |
+
{"step": 120, "steps": 250, "lr": 3.1704209946144616e-05, "grad_norm": 2.3263628482818604, "loss_scale": 4.0, "retries": 0, "seconds": 570.2, "max_memory_gb": 17.15, "ce": 0.11368525658872386, "rank": 0.0}
|
| 121 |
+
{"step": 121, "steps": 250, "lr": 3.140708399750594e-05, "grad_norm": 0.47082674503326416, "loss_scale": 4.0, "retries": 0, "seconds": 574.9, "max_memory_gb": 17.15, "ce": 0.053669025593308106, "rank": 0.0}
|
| 122 |
+
{"step": 122, "steps": 250, "lr": 3.1109253819299606e-05, "grad_norm": 0.8172115683555603, "loss_scale": 4.0, "retries": 0, "seconds": 579.4, "max_memory_gb": 17.15, "ce": 0.11715479146187135, "rank": 0.0}
|
| 123 |
+
{"step": 123, "steps": 250, "lr": 3.081077309371361e-05, "grad_norm": 0.27484792470932007, "loss_scale": 4.0, "retries": 0, "seconds": 583.9, "max_memory_gb": 17.15, "ce": 0.030795256987403263, "rank": 0.0}
|
| 124 |
+
{"step": 124, "steps": 250, "lr": 3.051169562019339e-05, "grad_norm": 1.5452430248260498, "loss_scale": 4.0, "retries": 0, "seconds": 588.5, "max_memory_gb": 17.15, "ce": 0.5394932638000682, "rank": 0.0}
|
| 125 |
+
{"step": 125, "steps": 250, "lr": 3.021207530574477e-05, "grad_norm": 0.7404446005821228, "loss_scale": 4.0, "retries": 0, "seconds": 593.2, "max_memory_gb": 17.15, "ce": 0.16194082865331438, "rank": 0.0}
|
| 126 |
+
{"step": 126, "steps": 250, "lr": 2.991196615521758e-05, "grad_norm": 2.670851945877075, "loss_scale": 4.0, "retries": 0, "seconds": 597.4, "max_memory_gb": 17.15, "ce": 0.5150524054211019, "rank": 0.0}
|
| 127 |
+
{"step": 127, "steps": 250, "lr": 2.9611422261571536e-05, "grad_norm": 0.4342740774154663, "loss_scale": 4.0, "retries": 0, "seconds": 602.1, "max_memory_gb": 17.15, "ce": 0.015681395848332613, "rank": 0.0}
|
| 128 |
+
{"step": 128, "steps": 250, "lr": 2.931049779612634e-05, "grad_norm": 0.8653395771980286, "loss_scale": 4.0, "retries": 0, "seconds": 606.7, "max_memory_gb": 17.15, "ce": 0.12470367522792003, "rank": 0.0}
|
| 129 |
+
{"step": 129, "steps": 250, "lr": 2.9009246998797557e-05, "grad_norm": 1.3674921989440918, "loss_scale": 4.0, "retries": 0, "seconds": 611.4, "max_memory_gb": 17.15, "ce": 0.09624382791412245, "rank": 0.0}
|
| 130 |
+
{"step": 130, "steps": 250, "lr": 2.8707724168320238e-05, "grad_norm": 0.648600697517395, "loss_scale": 4.0, "retries": 0, "seconds": 616.6, "max_memory_gb": 17.15, "ce": 0.13563422165680095, "rank": 0.0}
|
| 131 |
+
{"step": 131, "steps": 250, "lr": 2.840598365246184e-05, "grad_norm": 2.909179210662842, "loss_scale": 4.0, "retries": 0, "seconds": 621.2, "max_memory_gb": 17.15, "ce": 0.29700813677641236, "rank": 0.0}
|
| 132 |
+
{"step": 132, "steps": 250, "lr": 2.8104079838226383e-05, "grad_norm": 1.7295358180999756, "loss_scale": 4.0, "retries": 0, "seconds": 625.7, "max_memory_gb": 17.15, "ce": 0.19767688602019007, "rank": 0.0}
|
| 133 |
+
{"step": 133, "steps": 250, "lr": 2.7802067142051446e-05, "grad_norm": 1.1825058460235596, "loss_scale": 4.0, "retries": 0, "seconds": 630.5, "max_memory_gb": 17.15, "ce": 0.3400563439556663, "rank": 0.0}
|
| 134 |
+
{"step": 134, "steps": 250, "lr": 2.7500000000000004e-05, "grad_norm": 0.927446186542511, "loss_scale": 4.0, "retries": 0, "seconds": 635.0, "max_memory_gb": 17.15, "ce": 0.16348745910863727, "rank": 0.0}
|
| 135 |
+
{"step": 135, "steps": 250, "lr": 2.7197932857948566e-05, "grad_norm": 0.24859264492988586, "loss_scale": 4.0, "retries": 0, "seconds": 639.6, "max_memory_gb": 17.15, "ce": 0.022278572162122146, "rank": 0.0}
|
| 136 |
+
{"step": 136, "steps": 250, "lr": 2.689592016177363e-05, "grad_norm": 0.60333251953125, "loss_scale": 4.0, "retries": 0, "seconds": 644.0, "max_memory_gb": 17.15, "ce": 0.04332238421375223, "rank": 0.0}
|
| 137 |
+
{"step": 137, "steps": 250, "lr": 2.659401634753817e-05, "grad_norm": 0.368953675031662, "loss_scale": 4.0, "retries": 0, "seconds": 648.7, "max_memory_gb": 17.15, "ce": 0.021077552735960126, "rank": 0.0}
|
| 138 |
+
{"step": 138, "steps": 250, "lr": 2.6292275831679764e-05, "grad_norm": 0.6518876552581787, "loss_scale": 4.0, "retries": 0, "seconds": 654.0, "max_memory_gb": 17.15, "ce": 0.06288481376986965, "rank": 0.0}
|
| 139 |
+
{"step": 139, "steps": 250, "lr": 2.5990753001202445e-05, "grad_norm": 0.41976454854011536, "loss_scale": 4.0, "retries": 0, "seconds": 659.2, "max_memory_gb": 17.15, "ce": 0.02329692181115206, "rank": 0.0}
|
| 140 |
+
{"step": 140, "steps": 250, "lr": 2.5689502203873672e-05, "grad_norm": 0.030168328434228897, "loss_scale": 4.0, "retries": 0, "seconds": 663.8, "max_memory_gb": 17.15, "ce": 0.0028804087835396786, "rank": 0.0}
|
| 141 |
+
{"step": 141, "steps": 250, "lr": 2.538857773842847e-05, "grad_norm": 0.04262595251202583, "loss_scale": 4.0, "retries": 0, "seconds": 668.6, "max_memory_gb": 17.15, "ce": 0.004591260071265424, "rank": 0.0}
|
| 142 |
+
{"step": 142, "steps": 250, "lr": 2.5088033844782426e-05, "grad_norm": 0.8587660193443298, "loss_scale": 4.0, "retries": 0, "seconds": 673.2, "max_memory_gb": 17.15, "ce": 0.07118678017536695, "rank": 0.0}
|
| 143 |
+
{"step": 143, "steps": 250, "lr": 2.4787924694255237e-05, "grad_norm": 2.4053077697753906, "loss_scale": 4.0, "retries": 0, "seconds": 677.7, "max_memory_gb": 17.15, "ce": 0.26736206860914535, "rank": 0.0}
|
| 144 |
+
{"step": 144, "steps": 250, "lr": 2.4488304379806627e-05, "grad_norm": 0.5223621726036072, "loss_scale": 4.0, "retries": 0, "seconds": 682.3, "max_memory_gb": 17.15, "ce": 0.04312058491063908, "rank": 0.0}
|
| 145 |
+
{"step": 145, "steps": 250, "lr": 2.4189226906286388e-05, "grad_norm": 0.047764524817466736, "loss_scale": 4.0, "retries": 0, "seconds": 686.9, "max_memory_gb": 17.15, "ce": 0.0031535359373719984, "rank": 0.0}
|
| 146 |
+
{"step": 146, "steps": 250, "lr": 2.3890746180700395e-05, "grad_norm": 0.09213359653949738, "loss_scale": 4.0, "retries": 0, "seconds": 691.5, "max_memory_gb": 17.15, "ce": 0.011817776359521304, "rank": 0.0}
|
| 147 |
+
{"step": 147, "steps": 250, "lr": 2.359291600249407e-05, "grad_norm": 1.3332250118255615, "loss_scale": 4.0, "retries": 0, "seconds": 696.3, "max_memory_gb": 17.15, "ce": 0.4444426490911155, "rank": 0.0}
|
| 148 |
+
{"step": 148, "steps": 250, "lr": 2.3295790053855392e-05, "grad_norm": 0.09388408064842224, "loss_scale": 4.0, "retries": 0, "seconds": 700.8, "max_memory_gb": 17.15, "ce": 0.007771738214842117, "rank": 0.0}
|
| 149 |
+
{"step": 149, "steps": 250, "lr": 2.299942189003901e-05, "grad_norm": 0.11877994984388351, "loss_scale": 4.0, "retries": 0, "seconds": 705.1, "max_memory_gb": 17.15, "ce": 0.010031113190279939, "rank": 0.11832096427679062}
|
| 150 |
+
{"step": 150, "steps": 250, "lr": 2.2703864929713144e-05, "grad_norm": 0.4912882149219513, "loss_scale": 4.0, "retries": 0, "seconds": 709.7, "max_memory_gb": 17.15, "ce": 0.08511125020845611, "rank": 0.0}
|
| 151 |
+
{"step": 151, "steps": 250, "lr": 2.2409172445331257e-05, "grad_norm": 0.4486154019832611, "loss_scale": 4.0, "retries": 0, "seconds": 714.3, "max_memory_gb": 17.15, "ce": 0.014673108616989339, "rank": 0.0}
|
| 152 |
+
{"step": 152, "steps": 250, "lr": 2.211539755352996e-05, "grad_norm": 0.24407507479190826, "loss_scale": 4.0, "retries": 0, "seconds": 719.1, "max_memory_gb": 17.15, "ce": 0.0191142161170319, "rank": 0.0}
|
| 153 |
+
{"step": 153, "steps": 250, "lr": 2.1822593205555016e-05, "grad_norm": 0.5609996318817139, "loss_scale": 4.0, "retries": 0, "seconds": 723.7, "max_memory_gb": 17.15, "ce": 0.10596103108946409, "rank": 0.0}
|
| 154 |
+
{"step": 154, "steps": 250, "lr": 2.1530812177717236e-05, "grad_norm": 0.29837435483932495, "loss_scale": 4.0, "retries": 0, "seconds": 729.3, "max_memory_gb": 17.15, "ce": 0.028813784816975385, "rank": 0.0}
|
| 155 |
+
{"step": 155, "steps": 250, "lr": 2.1240107061879815e-05, "grad_norm": 1.6744141578674316, "loss_scale": 4.0, "retries": 0, "seconds": 733.9, "max_memory_gb": 17.15, "ce": 0.2518445109858476, "rank": 0.0}
|
| 156 |
+
{"step": 156, "steps": 250, "lr": 2.095053025597896e-05, "grad_norm": 0.19398102164268494, "loss_scale": 4.0, "retries": 0, "seconds": 738.6, "max_memory_gb": 17.15, "ce": 0.00705315972822973, "rank": 0.0}
|
| 157 |
+
{"step": 157, "steps": 250, "lr": 2.066213395457945e-05, "grad_norm": 0.16861940920352936, "loss_scale": 4.0, "retries": 0, "seconds": 743.6, "max_memory_gb": 17.15, "ce": 0.016680893111242767, "rank": 0.0}
|
| 158 |
+
{"step": 158, "steps": 250, "lr": 2.0374970139466877e-05, "grad_norm": 5.340068817138672, "loss_scale": 4.0, "retries": 0, "seconds": 748.3, "max_memory_gb": 17.15, "ce": 0.8377958271762509, "rank": 0.0}
|
| 159 |
+
{"step": 159, "steps": 250, "lr": 2.0089090570278213e-05, "grad_norm": 0.16643007099628448, "loss_scale": 4.0, "retries": 0, "seconds": 753.1, "max_memory_gb": 17.16, "ce": 0.011736080200989818, "rank": 0.0}
|
| 160 |
+
{"step": 160, "steps": 250, "lr": 1.9804546775172457e-05, "grad_norm": 0.8247171640396118, "loss_scale": 4.0, "retries": 0, "seconds": 758.2, "max_memory_gb": 17.16, "ce": 0.14082602401595068, "rank": 0.0}
|
| 161 |
+
{"step": 161, "steps": 250, "lr": 1.9521390041542946e-05, "grad_norm": 1.7455689907073975, "loss_scale": 4.0, "retries": 0, "seconds": 762.7, "max_memory_gb": 17.16, "ce": 0.15782431701654787, "rank": 0.0}
|
| 162 |
+
{"step": 162, "steps": 250, "lr": 1.923967140677313e-05, "grad_norm": 0.5533771514892578, "loss_scale": 4.0, "retries": 0, "seconds": 767.3, "max_memory_gb": 17.16, "ce": 0.050791851138910715, "rank": 0.0}
|
| 163 |
+
{"step": 163, "steps": 250, "lr": 1.8959441649037335e-05, "grad_norm": 2.8446574211120605, "loss_scale": 4.0, "retries": 0, "seconds": 772.0, "max_memory_gb": 17.16, "ce": 0.27334231214126703, "rank": 0.0}
|
| 164 |
+
{"step": 164, "steps": 250, "lr": 1.868075127814832e-05, "grad_norm": 0.8379883170127869, "loss_scale": 4.0, "retries": 0, "seconds": 776.7, "max_memory_gb": 17.16, "ce": 0.05815895257637749, "rank": 0.0}
|
| 165 |
+
{"step": 165, "steps": 250, "lr": 1.8403650526453143e-05, "grad_norm": 0.11533080786466599, "loss_scale": 4.0, "retries": 0, "seconds": 781.3, "max_memory_gb": 17.16, "ce": 0.011303479430807784, "rank": 0.0}
|
| 166 |
+
{"step": 166, "steps": 250, "lr": 1.812818933977912e-05, "grad_norm": 1.8097102642059326, "loss_scale": 4.0, "retries": 0, "seconds": 786.1, "max_memory_gb": 17.16, "ce": 0.27083754713021335, "rank": 0.0}
|
| 167 |
+
{"step": 167, "steps": 250, "lr": 1.785441736843128e-05, "grad_norm": 0.14783643186092377, "loss_scale": 4.0, "retries": 0, "seconds": 790.7, "max_memory_gb": 17.16, "ce": 0.008591843221893214, "rank": 0.0}
|
| 168 |
+
{"step": 168, "steps": 250, "lr": 1.7582383958243314e-05, "grad_norm": 0.09871934354305267, "loss_scale": 4.0, "retries": 0, "seconds": 795.3, "max_memory_gb": 17.16, "ce": 0.006277526160374691, "rank": 0.0}
|
| 169 |
+
{"step": 169, "steps": 250, "lr": 1.7312138141683164e-05, "grad_norm": 0.9347409605979919, "loss_scale": 4.0, "retries": 0, "seconds": 800.0, "max_memory_gb": 17.16, "ce": 0.32115568953076945, "rank": 0.0}
|
| 170 |
+
{"step": 170, "steps": 250, "lr": 1.704372862901521e-05, "grad_norm": 0.020198650658130646, "loss_scale": 4.0, "retries": 0, "seconds": 804.6, "max_memory_gb": 17.16, "ce": 0.000800561905862196, "rank": 0.0}
|
| 171 |
+
{"step": 171, "steps": 250, "lr": 1.6777203799520573e-05, "grad_norm": 0.37983614206314087, "loss_scale": 4.0, "retries": 0, "seconds": 809.2, "max_memory_gb": 17.16, "ce": 0.06050689764742856, "rank": 0.0}
|
| 172 |
+
{"step": 172, "steps": 250, "lr": 1.6512611692777008e-05, "grad_norm": 0.05479399859905243, "loss_scale": 4.0, "retries": 0, "seconds": 813.7, "max_memory_gb": 17.16, "ce": 0.005096214452350978, "rank": 0.0}
|
| 173 |
+
{"step": 173, "steps": 250, "lr": 1.6250000000000005e-05, "grad_norm": 0.14923688769340515, "loss_scale": 4.0, "retries": 0, "seconds": 818.5, "max_memory_gb": 17.16, "ce": 0.010699359678710607, "rank": 0.0}
|
| 174 |
+
{"step": 174, "steps": 250, "lr": 1.598941605544678e-05, "grad_norm": 0.5924588441848755, "loss_scale": 4.0, "retries": 0, "seconds": 823.1, "max_memory_gb": 17.16, "ce": 0.042365545689335704, "rank": 0.0}
|
| 175 |
+
{"step": 175, "steps": 250, "lr": 1.5730906827884506e-05, "grad_norm": 1.0846352577209473, "loss_scale": 4.0, "retries": 0, "seconds": 827.8, "max_memory_gb": 17.16, "ce": 0.11093067038291338, "rank": 0.0}
|
| 176 |
+
{"step": 176, "steps": 250, "lr": 1.5474518912124476e-05, "grad_norm": 0.020615484565496445, "loss_scale": 4.0, "retries": 0, "seconds": 832.7, "max_memory_gb": 17.16, "ce": 0.0013043738529177062, "rank": 0.0}
|
| 177 |
+
{"step": 177, "steps": 250, "lr": 1.5220298520623646e-05, "grad_norm": 0.7973212599754333, "loss_scale": 4.0, "retries": 0, "seconds": 837.2, "max_memory_gb": 17.16, "ce": 0.25583093938757884, "rank": 0.12490370124578476}
|
| 178 |
+
{"step": 178, "steps": 250, "lr": 1.4968291475155121e-05, "grad_norm": 1.186730146408081, "loss_scale": 4.0, "retries": 0, "seconds": 841.8, "max_memory_gb": 17.16, "ce": 0.09854723519310937, "rank": 0.0}
|
| 179 |
+
{"step": 179, "steps": 250, "lr": 1.4718543198548998e-05, "grad_norm": 1.0789779424667358, "loss_scale": 4.0, "retries": 0, "seconds": 846.6, "max_memory_gb": 17.16, "ce": 0.3726123943620223, "rank": 0.0}
|
| 180 |
+
{"step": 180, "steps": 250, "lr": 1.4471098706505227e-05, "grad_norm": 0.13752533495426178, "loss_scale": 4.0, "retries": 0, "seconds": 851.2, "max_memory_gb": 17.16, "ce": 0.01686657211439524, "rank": 0.0}
|
| 181 |
+
{"step": 181, "steps": 250, "lr": 1.4226002599479743e-05, "grad_norm": 0.41993749141693115, "loss_scale": 4.0, "retries": 0, "seconds": 857.0, "max_memory_gb": 17.16, "ce": 0.010298121793766768, "rank": 0.07814423739910126}
|
| 182 |
+
{"step": 182, "steps": 250, "lr": 1.3983299054645471e-05, "grad_norm": 0.7721734642982483, "loss_scale": 4.0, "retries": 0, "seconds": 861.9, "max_memory_gb": 17.16, "ce": 0.10776571259339107, "rank": 0.0}
|
| 183 |
+
{"step": 183, "steps": 250, "lr": 1.3743031817929724e-05, "grad_norm": 0.06805621832609177, "loss_scale": 4.0, "retries": 0, "seconds": 867.2, "max_memory_gb": 17.16, "ce": 0.0028873224546259735, "rank": 0.0}
|
| 184 |
+
{"step": 184, "steps": 250, "lr": 1.3505244196129164e-05, "grad_norm": 0.2674563229084015, "loss_scale": 4.0, "retries": 0, "seconds": 872.0, "max_memory_gb": 17.16, "ce": 0.01242404358163185, "rank": 0.15697158128023148}
|
| 185 |
+
{"step": 185, "steps": 250, "lr": 1.3269979049104014e-05, "grad_norm": 0.2871423065662384, "loss_scale": 4.0, "retries": 0, "seconds": 876.7, "max_memory_gb": 17.16, "ce": 0.03829481745765406, "rank": 0.0}
|
| 186 |
+
{"step": 186, "steps": 250, "lr": 1.3037278782052863e-05, "grad_norm": 2.3090476989746094, "loss_scale": 4.0, "retries": 0, "seconds": 881.4, "max_memory_gb": 17.16, "ce": 0.09143497464265238, "rank": 0.0}
|
| 187 |
+
{"step": 187, "steps": 250, "lr": 1.2807185337869343e-05, "grad_norm": 0.6462172865867615, "loss_scale": 4.0, "retries": 0, "seconds": 886.0, "max_memory_gb": 17.16, "ce": 0.13064311623816138, "rank": 0.12954522669315338}
|
| 188 |
+
{"step": 188, "steps": 250, "lr": 1.2579740189582115e-05, "grad_norm": 1.2231338024139404, "loss_scale": 4.0, "retries": 0, "seconds": 890.7, "max_memory_gb": 17.16, "ce": 0.19441107941611335, "rank": 0.0}
|
| 189 |
+
{"step": 189, "steps": 250, "lr": 1.2354984332879668e-05, "grad_norm": 2.3668265342712402, "loss_scale": 4.0, "retries": 0, "seconds": 895.5, "max_memory_gb": 17.16, "ce": 0.33516787499138445, "rank": 0.0}
|
| 190 |
+
{"step": 190, "steps": 250, "lr": 1.2132958278721064e-05, "grad_norm": 0.1312393695116043, "loss_scale": 4.0, "retries": 0, "seconds": 900.3, "max_memory_gb": 17.16, "ce": 0.004465431413791521, "rank": 0.0}
|
| 191 |
+
{"step": 191, "steps": 250, "lr": 1.1913702046034016e-05, "grad_norm": 1.420003056526184, "loss_scale": 4.0, "retries": 0, "seconds": 905.1, "max_memory_gb": 17.16, "ce": 0.18824964655345866, "rank": 0.0}
|
| 192 |
+
{"step": 192, "steps": 250, "lr": 1.1697255154501756e-05, "grad_norm": 0.951176643371582, "loss_scale": 4.0, "retries": 0, "seconds": 909.9, "max_memory_gb": 17.16, "ce": 0.34836662155066733, "rank": 0.0}
|
| 193 |
+
{"step": 193, "steps": 250, "lr": 1.1483656617439842e-05, "grad_norm": 0.016337519511580467, "loss_scale": 4.0, "retries": 0, "seconds": 914.7, "max_memory_gb": 17.16, "ce": 0.001203566609774498, "rank": 0.1250571757555008}
|
| 194 |
+
{"step": 194, "steps": 250, "lr": 1.1272944934764177e-05, "grad_norm": 0.37908023595809937, "loss_scale": 4.0, "retries": 0, "seconds": 919.5, "max_memory_gb": 17.16, "ce": 0.039591051304114444, "rank": 0.0}
|
| 195 |
+
{"step": 195, "steps": 250, "lr": 1.1065158086051684e-05, "grad_norm": 0.3881003260612488, "loss_scale": 4.0, "retries": 0, "seconds": 924.2, "max_memory_gb": 17.16, "ce": 0.07532940056762527, "rank": 0.0}
|
| 196 |
+
{"step": 196, "steps": 250, "lr": 1.086033352369467e-05, "grad_norm": 0.15198861062526703, "loss_scale": 4.0, "retries": 0, "seconds": 929.2, "max_memory_gb": 17.16, "ce": 0.008997007811785807, "rank": 0.0}
|
| 197 |
+
{"step": 197, "steps": 250, "lr": 1.0658508166150224e-05, "grad_norm": 1.3014469146728516, "loss_scale": 4.0, "retries": 0, "seconds": 933.8, "max_memory_gb": 17.16, "ce": 0.1493724561646559, "rank": 0.0}
|
| 198 |
+
{"step": 198, "steps": 250, "lr": 1.0459718391285954e-05, "grad_norm": 0.3897629976272583, "loss_scale": 4.0, "retries": 0, "seconds": 938.6, "max_memory_gb": 17.16, "ce": 0.06604731463698954, "rank": 0.0}
|
| 199 |
+
{"step": 199, "steps": 250, "lr": 1.0264000029822999e-05, "grad_norm": 0.932966411113739, "loss_scale": 4.0, "retries": 0, "seconds": 944.0, "max_memory_gb": 17.16, "ce": 0.08638977762529976, "rank": 0.0}
|
| 200 |
+
{"step": 200, "steps": 250, "lr": 1.0071388358877775e-05, "grad_norm": 0.8942098021507263, "loss_scale": 4.0, "retries": 0, "seconds": 949.8, "max_memory_gb": 17.16, "ce": 0.22283045703716198, "rank": 0.0}
|
| 201 |
+
{"step": 201, "steps": 250, "lr": 9.881918095603501e-06, "grad_norm": 0.5402695536613464, "loss_scale": 4.0, "retries": 0, "seconds": 954.5, "max_memory_gb": 17.16, "ce": 0.05992591900394473, "rank": 0.0}
|
| 202 |
+
{"step": 202, "steps": 250, "lr": 9.695623390932612e-06, "grad_norm": 0.033308692276477814, "loss_scale": 4.0, "retries": 0, "seconds": 959.5, "max_memory_gb": 17.16, "ce": 0.0027427887814610585, "rank": 0.0}
|
| 203 |
+
{"step": 203, "steps": 250, "lr": 9.51253782342123e-06, "grad_norm": 0.21247580647468567, "loss_scale": 4.0, "retries": 0, "seconds": 964.2, "max_memory_gb": 17.16, "ce": 0.01390562690767183, "rank": 0.0}
|
| 204 |
+
{"step": 204, "steps": 250, "lr": 9.332694393196846e-06, "grad_norm": 0.25844788551330566, "loss_scale": 4.0, "retries": 0, "seconds": 969.0, "max_memory_gb": 17.16, "ce": 0.01804874744220797, "rank": 0.0}
|
| 205 |
+
{"step": 205, "steps": 250, "lr": 9.156125516010256e-06, "grad_norm": 0.8295074701309204, "loss_scale": 4.0, "retries": 0, "seconds": 973.6, "max_memory_gb": 17.16, "ce": 0.08844837738820388, "rank": 0.0}
|
| 206 |
+
{"step": 206, "steps": 250, "lr": 8.982863017392733e-06, "grad_norm": 0.23036538064479828, "loss_scale": 4.0, "retries": 0, "seconds": 977.9, "max_memory_gb": 17.16, "ce": 0.018741268059557115, "rank": 0.0}
|
| 207 |
+
{"step": 207, "steps": 250, "lr": 8.81293812691974e-06, "grad_norm": 0.16722795367240906, "loss_scale": 4.0, "retries": 0, "seconds": 982.4, "max_memory_gb": 17.16, "ce": 0.05377623991944347, "rank": 0.0}
|
| 208 |
+
{"step": 208, "steps": 250, "lr": 8.646381472581927e-06, "grad_norm": 0.035242557525634766, "loss_scale": 4.0, "retries": 0, "seconds": 987.2, "max_memory_gb": 17.16, "ce": 0.0022652913819456444, "rank": 0.0}
|
| 209 |
+
{"step": 209, "steps": 250, "lr": 8.483223075264622e-06, "grad_norm": 0.4781361520290375, "loss_scale": 4.0, "retries": 0, "seconds": 991.8, "max_memory_gb": 17.16, "ce": 0.10026453141017555, "rank": 0.1250409185886383}
|
| 210 |
+
{"step": 210, "steps": 250, "lr": 8.323492343336747e-06, "grad_norm": 1.2756856679916382, "loss_scale": 4.0, "retries": 0, "seconds": 996.3, "max_memory_gb": 17.16, "ce": 0.3589655939599652, "rank": 0.0}
|
| 211 |
+
{"step": 211, "steps": 250, "lr": 8.167218067350122e-06, "grad_norm": 0.5050255060195923, "loss_scale": 4.0, "retries": 0, "seconds": 1000.7, "max_memory_gb": 17.16, "ce": 0.11806542553313193, "rank": 0.0}
|
| 212 |
+
{"step": 212, "steps": 250, "lr": 8.01442841485013e-06, "grad_norm": 0.9214603900909424, "loss_scale": 4.0, "retries": 0, "seconds": 1005.4, "max_memory_gb": 17.16, "ce": 0.18390436796221366, "rank": 0.0}
|
| 213 |
+
{"step": 213, "steps": 250, "lr": 7.865150925298678e-06, "grad_norm": 0.43132483959198, "loss_scale": 4.0, "retries": 0, "seconds": 1010.0, "max_memory_gb": 17.16, "ce": 0.14268374649668658, "rank": 0.0}
|
| 214 |
+
{"step": 214, "steps": 250, "lr": 7.719412505110375e-06, "grad_norm": 1.0110549926757812, "loss_scale": 4.0, "retries": 0, "seconds": 1014.6, "max_memory_gb": 17.16, "ce": 0.22302791932725086, "rank": 0.0}
|
| 215 |
+
{"step": 215, "steps": 250, "lr": 7.57723942280278e-06, "grad_norm": 1.9536255598068237, "loss_scale": 4.0, "retries": 0, "seconds": 1019.4, "max_memory_gb": 17.16, "ce": 0.417645954309819, "rank": 0.0}
|
| 216 |
+
{"step": 216, "steps": 250, "lr": 7.438657304261687e-06, "grad_norm": 0.2813863158226013, "loss_scale": 4.0, "retries": 0, "seconds": 1024.0, "max_memory_gb": 17.16, "ce": 0.024433388786519572, "rank": 0.0}
|
| 217 |
+
{"step": 217, "steps": 250, "lr": 7.30369112812219e-06, "grad_norm": 0.06685240566730499, "loss_scale": 4.0, "retries": 0, "seconds": 1028.6, "max_memory_gb": 17.16, "ce": 0.004576440733217169, "rank": 0.0}
|
| 218 |
+
{"step": 218, "steps": 250, "lr": 7.172365221266403e-06, "grad_norm": 0.8696585297584534, "loss_scale": 4.0, "retries": 0, "seconds": 1033.5, "max_memory_gb": 17.16, "ce": 0.17877209199582467, "rank": 0.0}
|
| 219 |
+
{"step": 219, "steps": 250, "lr": 7.0447032544387154e-06, "grad_norm": 0.30599555373191833, "loss_scale": 4.0, "retries": 0, "seconds": 1038.3, "max_memory_gb": 17.16, "ce": 0.037529083061144775, "rank": 0.0}
|
| 220 |
+
{"step": 220, "steps": 250, "lr": 6.920728237979274e-06, "grad_norm": 0.2548443377017975, "loss_scale": 4.0, "retries": 0, "seconds": 1042.9, "max_memory_gb": 17.16, "ce": 0.019619067001258372, "rank": 0.0}
|
| 221 |
+
{"step": 221, "steps": 250, "lr": 6.800462517676456e-06, "grad_norm": 0.331219345331192, "loss_scale": 4.0, "retries": 0, "seconds": 1047.6, "max_memory_gb": 17.16, "ce": 0.02000232093860177, "rank": 0.0}
|
| 222 |
+
{"step": 222, "steps": 250, "lr": 6.683927770739222e-06, "grad_norm": 0.3106783926486969, "loss_scale": 4.0, "retries": 0, "seconds": 1051.9, "max_memory_gb": 17.16, "ce": 0.025031770186387803, "rank": 0.0}
|
| 223 |
+
{"step": 223, "steps": 250, "lr": 6.571145001889892e-06, "grad_norm": 1.2628979682922363, "loss_scale": 4.0, "retries": 0, "seconds": 1056.4, "max_memory_gb": 17.16, "ce": 0.0727864666143887, "rank": 0.0}
|
| 224 |
+
{"step": 224, "steps": 250, "lr": 6.46213453957817e-06, "grad_norm": 0.2769224941730499, "loss_scale": 4.0, "retries": 0, "seconds": 1061.0, "max_memory_gb": 17.16, "ce": 0.027574046991958312, "rank": 0.0}
|
| 225 |
+
{"step": 225, "steps": 250, "lr": 6.356916032317063e-06, "grad_norm": 0.11533699929714203, "loss_scale": 4.0, "retries": 0, "seconds": 1066.1, "max_memory_gb": 17.16, "ce": 0.010490001937796478, "rank": 0.0}
|
| 226 |
+
{"step": 226, "steps": 250, "lr": 6.255508445141349e-06, "grad_norm": 0.3717389702796936, "loss_scale": 4.0, "retries": 0, "seconds": 1070.0, "max_memory_gb": 17.16, "ce": 0.021586099724572705, "rank": 0.0}
|
| 227 |
+
{"step": 227, "steps": 250, "lr": 6.157930056189228e-06, "grad_norm": 0.6260979771614075, "loss_scale": 4.0, "retries": 0, "seconds": 1075.8, "max_memory_gb": 17.16, "ce": 0.032755363810792915, "rank": 0.0}
|
| 228 |
+
{"step": 228, "steps": 250, "lr": 6.064198453407796e-06, "grad_norm": 0.13032203912734985, "loss_scale": 4.0, "retries": 0, "seconds": 1081.3, "max_memory_gb": 17.16, "ce": 0.010707831663921752, "rank": 0.0}
|
| 229 |
+
{"step": 229, "steps": 250, "lr": 5.974330531382935e-06, "grad_norm": 0.32462194561958313, "loss_scale": 4.0, "retries": 0, "seconds": 1086.0, "max_memory_gb": 17.16, "ce": 0.04913299368331536, "rank": 0.0}
|
| 230 |
+
{"step": 230, "steps": 250, "lr": 5.888342488294125e-06, "grad_norm": 2.7498083114624023, "loss_scale": 4.0, "retries": 0, "seconds": 1090.6, "max_memory_gb": 17.16, "ce": 0.36283782844998314, "rank": 0.0}
|
| 231 |
+
{"step": 231, "steps": 250, "lr": 5.806249822994849e-06, "grad_norm": 0.26052048802375793, "loss_scale": 4.0, "retries": 0, "seconds": 1096.3, "max_memory_gb": 17.16, "ce": 0.036463290523670366, "rank": 0.0}
|
| 232 |
+
{"step": 232, "steps": 250, "lr": 5.728067332218994e-06, "grad_norm": 0.20888838171958923, "loss_scale": 4.0, "retries": 0, "seconds": 1101.4, "max_memory_gb": 17.16, "ce": 0.015523039984145726, "rank": 0.0}
|
| 233 |
+
{"step": 233, "steps": 250, "lr": 5.65380910791383e-06, "grad_norm": 0.5218024849891663, "loss_scale": 4.0, "retries": 0, "seconds": 1105.9, "max_memory_gb": 17.16, "ce": 0.028100798363993817, "rank": 0.0}
|
| 234 |
+
{"step": 234, "steps": 250, "lr": 5.583488534700022e-06, "grad_norm": 0.5132654309272766, "loss_scale": 4.0, "retries": 0, "seconds": 1110.5, "max_memory_gb": 17.16, "ce": 0.039828536718687246, "rank": 0.0}
|
| 235 |
+
{"step": 235, "steps": 250, "lr": 5.517118287459118e-06, "grad_norm": 1.604756474494934, "loss_scale": 4.0, "retries": 0, "seconds": 1115.1, "max_memory_gb": 17.16, "ce": 0.24203880324512284, "rank": 0.0}
|
| 236 |
+
{"step": 236, "steps": 250, "lr": 5.454710329048978e-06, "grad_norm": 0.12675586342811584, "loss_scale": 4.0, "retries": 0, "seconds": 1119.6, "max_memory_gb": 17.16, "ce": 0.016795223076542243, "rank": 0.0}
|
| 237 |
+
{"step": 237, "steps": 250, "lr": 5.3962759081475436e-06, "grad_norm": 0.45108067989349365, "loss_scale": 4.0, "retries": 0, "seconds": 1124.3, "max_memory_gb": 17.16, "ce": 0.04403398940826264, "rank": 0.0}
|
| 238 |
+
{"step": 238, "steps": 250, "lr": 5.341825557225321e-06, "grad_norm": 0.36557987332344055, "loss_scale": 4.0, "retries": 0, "seconds": 1128.9, "max_memory_gb": 17.16, "ce": 0.033381477634520706, "rank": 0.0}
|
| 239 |
+
{"step": 239, "steps": 250, "lr": 5.291369090646961e-06, "grad_norm": 0.3088444471359253, "loss_scale": 4.0, "retries": 0, "seconds": 1133.4, "max_memory_gb": 17.16, "ce": 0.09256700954483676, "rank": 0.0}
|
| 240 |
+
{"step": 240, "steps": 250, "lr": 5.244915602902289e-06, "grad_norm": 0.36353060603141785, "loss_scale": 4.0, "retries": 0, "seconds": 1138.0, "max_memory_gb": 17.16, "ce": 0.02794582985939087, "rank": 0.0}
|
| 241 |
+
{"step": 241, "steps": 250, "lr": 5.20247346696706e-06, "grad_norm": 0.6929258108139038, "loss_scale": 4.0, "retries": 0, "seconds": 1142.7, "max_memory_gb": 17.16, "ce": 0.07853333664706952, "rank": 0.0}
|
| 242 |
+
{"step": 242, "steps": 250, "lr": 5.164050332793787e-06, "grad_norm": 0.1434815675020218, "loss_scale": 4.0, "retries": 0, "seconds": 1147.2, "max_memory_gb": 17.16, "ce": 0.010571820085033323, "rank": 0.0}
|
| 243 |
+
{"step": 243, "steps": 250, "lr": 5.129653125932873e-06, "grad_norm": 0.01743205636739731, "loss_scale": 4.0, "retries": 0, "seconds": 1151.8, "max_memory_gb": 17.16, "ce": 0.001277904333846891, "rank": 0.0}
|
| 244 |
+
{"step": 244, "steps": 250, "lr": 5.099288046284329e-06, "grad_norm": 0.35945773124694824, "loss_scale": 4.0, "retries": 0, "seconds": 1156.5, "max_memory_gb": 17.16, "ce": 0.03894162936262546, "rank": 0.0}
|
| 245 |
+
{"step": 245, "steps": 250, "lr": 5.072960566980277e-06, "grad_norm": 1.516460657119751, "loss_scale": 4.0, "retries": 0, "seconds": 1161.1, "max_memory_gb": 17.16, "ce": 0.26911298569893916, "rank": 0.0}
|
| 246 |
+
{"step": 246, "steps": 250, "lr": 5.050675433398435e-06, "grad_norm": 0.027733201161026955, "loss_scale": 4.0, "retries": 0, "seconds": 1166.2, "max_memory_gb": 17.16, "ce": 0.0016247648995886266, "rank": 0.0}
|
| 247 |
+
{"step": 247, "steps": 250, "lr": 5.032436662306811e-06, "grad_norm": 0.581411600112915, "loss_scale": 4.0, "retries": 0, "seconds": 1171.3, "max_memory_gb": 17.16, "ce": 0.04298698135039558, "rank": 0.0}
|
| 248 |
+
{"step": 248, "steps": 250, "lr": 5.018247541139684e-06, "grad_norm": 0.09854079782962799, "loss_scale": 4.0, "retries": 0, "seconds": 1176.1, "max_memory_gb": 17.16, "ce": 0.008100246618823803, "rank": 0.0}
|
| 249 |
+
{"step": 249, "steps": 250, "lr": 5.00811062740508e-06, "grad_norm": 0.15059277415275574, "loss_scale": 4.0, "retries": 0, "seconds": 1180.8, "max_memory_gb": 17.16, "ce": 0.012320590882836768, "rank": 0.0}
|
| 250 |
+
{"step": 250, "steps": 250, "lr": 5.002027748223778e-06, "grad_norm": 0.21606449782848358, "loss_scale": 4.0, "retries": 0, "seconds": 1185.6, "max_memory_gb": 17.16, "ce": 0.017222866157908356, "rank": 0.0}
|
runs/fixed_internvl/removal/Shallow8_s42/adapter/README.md
ADDED
|
@@ -0,0 +1,207 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
base_model: /data/OPSD/assets/models/InternVL3_5-8B-HF
|
| 3 |
+
library_name: peft
|
| 4 |
+
pipeline_tag: text-generation
|
| 5 |
+
tags:
|
| 6 |
+
- base_model:adapter:/data/OPSD/assets/models/InternVL3_5-8B-HF
|
| 7 |
+
- lora
|
| 8 |
+
- transformers
|
| 9 |
+
---
|
| 10 |
+
|
| 11 |
+
# Model Card for Model ID
|
| 12 |
+
|
| 13 |
+
<!-- Provide a quick summary of what the model is/does. -->
|
| 14 |
+
|
| 15 |
+
|
| 16 |
+
|
| 17 |
+
## Model Details
|
| 18 |
+
|
| 19 |
+
### Model Description
|
| 20 |
+
|
| 21 |
+
<!-- Provide a longer summary of what this model is. -->
|
| 22 |
+
|
| 23 |
+
|
| 24 |
+
|
| 25 |
+
- **Developed by:** [More Information Needed]
|
| 26 |
+
- **Funded by [optional]:** [More Information Needed]
|
| 27 |
+
- **Shared by [optional]:** [More Information Needed]
|
| 28 |
+
- **Model type:** [More Information Needed]
|
| 29 |
+
- **Language(s) (NLP):** [More Information Needed]
|
| 30 |
+
- **License:** [More Information Needed]
|
| 31 |
+
- **Finetuned from model [optional]:** [More Information Needed]
|
| 32 |
+
|
| 33 |
+
### Model Sources [optional]
|
| 34 |
+
|
| 35 |
+
<!-- Provide the basic links for the model. -->
|
| 36 |
+
|
| 37 |
+
- **Repository:** [More Information Needed]
|
| 38 |
+
- **Paper [optional]:** [More Information Needed]
|
| 39 |
+
- **Demo [optional]:** [More Information Needed]
|
| 40 |
+
|
| 41 |
+
## Uses
|
| 42 |
+
|
| 43 |
+
<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
|
| 44 |
+
|
| 45 |
+
### Direct Use
|
| 46 |
+
|
| 47 |
+
<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
|
| 48 |
+
|
| 49 |
+
[More Information Needed]
|
| 50 |
+
|
| 51 |
+
### Downstream Use [optional]
|
| 52 |
+
|
| 53 |
+
<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
|
| 54 |
+
|
| 55 |
+
[More Information Needed]
|
| 56 |
+
|
| 57 |
+
### Out-of-Scope Use
|
| 58 |
+
|
| 59 |
+
<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
|
| 60 |
+
|
| 61 |
+
[More Information Needed]
|
| 62 |
+
|
| 63 |
+
## Bias, Risks, and Limitations
|
| 64 |
+
|
| 65 |
+
<!-- This section is meant to convey both technical and sociotechnical limitations. -->
|
| 66 |
+
|
| 67 |
+
[More Information Needed]
|
| 68 |
+
|
| 69 |
+
### Recommendations
|
| 70 |
+
|
| 71 |
+
<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
|
| 72 |
+
|
| 73 |
+
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
|
| 74 |
+
|
| 75 |
+
## How to Get Started with the Model
|
| 76 |
+
|
| 77 |
+
Use the code below to get started with the model.
|
| 78 |
+
|
| 79 |
+
[More Information Needed]
|
| 80 |
+
|
| 81 |
+
## Training Details
|
| 82 |
+
|
| 83 |
+
### Training Data
|
| 84 |
+
|
| 85 |
+
<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
|
| 86 |
+
|
| 87 |
+
[More Information Needed]
|
| 88 |
+
|
| 89 |
+
### Training Procedure
|
| 90 |
+
|
| 91 |
+
<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
|
| 92 |
+
|
| 93 |
+
#### Preprocessing [optional]
|
| 94 |
+
|
| 95 |
+
[More Information Needed]
|
| 96 |
+
|
| 97 |
+
|
| 98 |
+
#### Training Hyperparameters
|
| 99 |
+
|
| 100 |
+
- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
|
| 101 |
+
|
| 102 |
+
#### Speeds, Sizes, Times [optional]
|
| 103 |
+
|
| 104 |
+
<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
|
| 105 |
+
|
| 106 |
+
[More Information Needed]
|
| 107 |
+
|
| 108 |
+
## Evaluation
|
| 109 |
+
|
| 110 |
+
<!-- This section describes the evaluation protocols and provides the results. -->
|
| 111 |
+
|
| 112 |
+
### Testing Data, Factors & Metrics
|
| 113 |
+
|
| 114 |
+
#### Testing Data
|
| 115 |
+
|
| 116 |
+
<!-- This should link to a Dataset Card if possible. -->
|
| 117 |
+
|
| 118 |
+
[More Information Needed]
|
| 119 |
+
|
| 120 |
+
#### Factors
|
| 121 |
+
|
| 122 |
+
<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
|
| 123 |
+
|
| 124 |
+
[More Information Needed]
|
| 125 |
+
|
| 126 |
+
#### Metrics
|
| 127 |
+
|
| 128 |
+
<!-- These are the evaluation metrics being used, ideally with a description of why. -->
|
| 129 |
+
|
| 130 |
+
[More Information Needed]
|
| 131 |
+
|
| 132 |
+
### Results
|
| 133 |
+
|
| 134 |
+
[More Information Needed]
|
| 135 |
+
|
| 136 |
+
#### Summary
|
| 137 |
+
|
| 138 |
+
|
| 139 |
+
|
| 140 |
+
## Model Examination [optional]
|
| 141 |
+
|
| 142 |
+
<!-- Relevant interpretability work for the model goes here -->
|
| 143 |
+
|
| 144 |
+
[More Information Needed]
|
| 145 |
+
|
| 146 |
+
## Environmental Impact
|
| 147 |
+
|
| 148 |
+
<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
|
| 149 |
+
|
| 150 |
+
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
|
| 151 |
+
|
| 152 |
+
- **Hardware Type:** [More Information Needed]
|
| 153 |
+
- **Hours used:** [More Information Needed]
|
| 154 |
+
- **Cloud Provider:** [More Information Needed]
|
| 155 |
+
- **Compute Region:** [More Information Needed]
|
| 156 |
+
- **Carbon Emitted:** [More Information Needed]
|
| 157 |
+
|
| 158 |
+
## Technical Specifications [optional]
|
| 159 |
+
|
| 160 |
+
### Model Architecture and Objective
|
| 161 |
+
|
| 162 |
+
[More Information Needed]
|
| 163 |
+
|
| 164 |
+
### Compute Infrastructure
|
| 165 |
+
|
| 166 |
+
[More Information Needed]
|
| 167 |
+
|
| 168 |
+
#### Hardware
|
| 169 |
+
|
| 170 |
+
[More Information Needed]
|
| 171 |
+
|
| 172 |
+
#### Software
|
| 173 |
+
|
| 174 |
+
[More Information Needed]
|
| 175 |
+
|
| 176 |
+
## Citation [optional]
|
| 177 |
+
|
| 178 |
+
<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
|
| 179 |
+
|
| 180 |
+
**BibTeX:**
|
| 181 |
+
|
| 182 |
+
[More Information Needed]
|
| 183 |
+
|
| 184 |
+
**APA:**
|
| 185 |
+
|
| 186 |
+
[More Information Needed]
|
| 187 |
+
|
| 188 |
+
## Glossary [optional]
|
| 189 |
+
|
| 190 |
+
<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
|
| 191 |
+
|
| 192 |
+
[More Information Needed]
|
| 193 |
+
|
| 194 |
+
## More Information [optional]
|
| 195 |
+
|
| 196 |
+
[More Information Needed]
|
| 197 |
+
|
| 198 |
+
## Model Card Authors [optional]
|
| 199 |
+
|
| 200 |
+
[More Information Needed]
|
| 201 |
+
|
| 202 |
+
## Model Card Contact
|
| 203 |
+
|
| 204 |
+
[More Information Needed]
|
| 205 |
+
### Framework versions
|
| 206 |
+
|
| 207 |
+
- PEFT 0.20.0
|
runs/fixed_internvl/removal/Shallow8_s42/adapter/adapter_config.json
ADDED
|
@@ -0,0 +1,213 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"alora_invocation_tokens": null,
|
| 3 |
+
"alpha_pattern": {
|
| 4 |
+
"model.language_model.layers.0.mlp.down_proj": 144,
|
| 5 |
+
"model.language_model.layers.0.mlp.gate_proj": 144,
|
| 6 |
+
"model.language_model.layers.0.mlp.up_proj": 144,
|
| 7 |
+
"model.language_model.layers.0.self_attn.k_proj": 144,
|
| 8 |
+
"model.language_model.layers.0.self_attn.o_proj": 144,
|
| 9 |
+
"model.language_model.layers.0.self_attn.q_proj": 144,
|
| 10 |
+
"model.language_model.layers.0.self_attn.v_proj": 144,
|
| 11 |
+
"model.language_model.layers.1.mlp.down_proj": 144,
|
| 12 |
+
"model.language_model.layers.1.mlp.gate_proj": 144,
|
| 13 |
+
"model.language_model.layers.1.mlp.up_proj": 144,
|
| 14 |
+
"model.language_model.layers.1.self_attn.k_proj": 144,
|
| 15 |
+
"model.language_model.layers.1.self_attn.o_proj": 144,
|
| 16 |
+
"model.language_model.layers.1.self_attn.q_proj": 144,
|
| 17 |
+
"model.language_model.layers.1.self_attn.v_proj": 144,
|
| 18 |
+
"model.language_model.layers.2.mlp.down_proj": 144,
|
| 19 |
+
"model.language_model.layers.2.mlp.gate_proj": 144,
|
| 20 |
+
"model.language_model.layers.2.mlp.up_proj": 144,
|
| 21 |
+
"model.language_model.layers.2.self_attn.k_proj": 144,
|
| 22 |
+
"model.language_model.layers.2.self_attn.o_proj": 144,
|
| 23 |
+
"model.language_model.layers.2.self_attn.q_proj": 144,
|
| 24 |
+
"model.language_model.layers.2.self_attn.v_proj": 144,
|
| 25 |
+
"model.language_model.layers.3.mlp.down_proj": 144,
|
| 26 |
+
"model.language_model.layers.3.mlp.gate_proj": 144,
|
| 27 |
+
"model.language_model.layers.3.mlp.up_proj": 144,
|
| 28 |
+
"model.language_model.layers.3.self_attn.k_proj": 144,
|
| 29 |
+
"model.language_model.layers.3.self_attn.o_proj": 144,
|
| 30 |
+
"model.language_model.layers.3.self_attn.q_proj": 144,
|
| 31 |
+
"model.language_model.layers.3.self_attn.v_proj": 144,
|
| 32 |
+
"model.language_model.layers.4.mlp.down_proj": 144,
|
| 33 |
+
"model.language_model.layers.4.mlp.gate_proj": 144,
|
| 34 |
+
"model.language_model.layers.4.mlp.up_proj": 144,
|
| 35 |
+
"model.language_model.layers.4.self_attn.k_proj": 144,
|
| 36 |
+
"model.language_model.layers.4.self_attn.o_proj": 144,
|
| 37 |
+
"model.language_model.layers.4.self_attn.q_proj": 144,
|
| 38 |
+
"model.language_model.layers.4.self_attn.v_proj": 144,
|
| 39 |
+
"model.language_model.layers.5.mlp.down_proj": 144,
|
| 40 |
+
"model.language_model.layers.5.mlp.gate_proj": 144,
|
| 41 |
+
"model.language_model.layers.5.mlp.up_proj": 144,
|
| 42 |
+
"model.language_model.layers.5.self_attn.k_proj": 144,
|
| 43 |
+
"model.language_model.layers.5.self_attn.o_proj": 144,
|
| 44 |
+
"model.language_model.layers.5.self_attn.q_proj": 144,
|
| 45 |
+
"model.language_model.layers.5.self_attn.v_proj": 144,
|
| 46 |
+
"model.language_model.layers.6.mlp.down_proj": 144,
|
| 47 |
+
"model.language_model.layers.6.mlp.gate_proj": 144,
|
| 48 |
+
"model.language_model.layers.6.mlp.up_proj": 144,
|
| 49 |
+
"model.language_model.layers.6.self_attn.k_proj": 144,
|
| 50 |
+
"model.language_model.layers.6.self_attn.o_proj": 144,
|
| 51 |
+
"model.language_model.layers.6.self_attn.q_proj": 144,
|
| 52 |
+
"model.language_model.layers.6.self_attn.v_proj": 144,
|
| 53 |
+
"model.language_model.layers.7.mlp.down_proj": 144,
|
| 54 |
+
"model.language_model.layers.7.mlp.gate_proj": 144,
|
| 55 |
+
"model.language_model.layers.7.mlp.up_proj": 144,
|
| 56 |
+
"model.language_model.layers.7.self_attn.k_proj": 144,
|
| 57 |
+
"model.language_model.layers.7.self_attn.o_proj": 144,
|
| 58 |
+
"model.language_model.layers.7.self_attn.q_proj": 144,
|
| 59 |
+
"model.language_model.layers.7.self_attn.v_proj": 144
|
| 60 |
+
},
|
| 61 |
+
"arrow_config": null,
|
| 62 |
+
"auto_mapping": null,
|
| 63 |
+
"base_model_name_or_path": "/data/OPSD/assets/models/InternVL3_5-8B-HF",
|
| 64 |
+
"bias": "none",
|
| 65 |
+
"corda_config": null,
|
| 66 |
+
"ensure_weight_tying": false,
|
| 67 |
+
"eva_config": null,
|
| 68 |
+
"exclude_modules": null,
|
| 69 |
+
"fan_in_fan_out": false,
|
| 70 |
+
"inference_mode": true,
|
| 71 |
+
"init_lora_weights": true,
|
| 72 |
+
"layer_replication": null,
|
| 73 |
+
"layers_pattern": null,
|
| 74 |
+
"layers_to_transform": null,
|
| 75 |
+
"loftq_config": {},
|
| 76 |
+
"lora_alpha": 32,
|
| 77 |
+
"lora_bias": false,
|
| 78 |
+
"lora_dropout": 0.05,
|
| 79 |
+
"lora_ga_config": null,
|
| 80 |
+
"megatron_config": null,
|
| 81 |
+
"megatron_core": "megatron.core",
|
| 82 |
+
"modules_to_save": null,
|
| 83 |
+
"monteclora_config": null,
|
| 84 |
+
"peft_type": "LORA",
|
| 85 |
+
"peft_version": "0.20.0",
|
| 86 |
+
"qalora_group_size": 16,
|
| 87 |
+
"r": 16,
|
| 88 |
+
"rank_pattern": {
|
| 89 |
+
"model.language_model.layers.0.mlp.down_proj": 72,
|
| 90 |
+
"model.language_model.layers.0.mlp.gate_proj": 72,
|
| 91 |
+
"model.language_model.layers.0.mlp.up_proj": 72,
|
| 92 |
+
"model.language_model.layers.0.self_attn.k_proj": 72,
|
| 93 |
+
"model.language_model.layers.0.self_attn.o_proj": 72,
|
| 94 |
+
"model.language_model.layers.0.self_attn.q_proj": 72,
|
| 95 |
+
"model.language_model.layers.0.self_attn.v_proj": 72,
|
| 96 |
+
"model.language_model.layers.1.mlp.down_proj": 72,
|
| 97 |
+
"model.language_model.layers.1.mlp.gate_proj": 72,
|
| 98 |
+
"model.language_model.layers.1.mlp.up_proj": 72,
|
| 99 |
+
"model.language_model.layers.1.self_attn.k_proj": 72,
|
| 100 |
+
"model.language_model.layers.1.self_attn.o_proj": 72,
|
| 101 |
+
"model.language_model.layers.1.self_attn.q_proj": 72,
|
| 102 |
+
"model.language_model.layers.1.self_attn.v_proj": 72,
|
| 103 |
+
"model.language_model.layers.2.mlp.down_proj": 72,
|
| 104 |
+
"model.language_model.layers.2.mlp.gate_proj": 72,
|
| 105 |
+
"model.language_model.layers.2.mlp.up_proj": 72,
|
| 106 |
+
"model.language_model.layers.2.self_attn.k_proj": 72,
|
| 107 |
+
"model.language_model.layers.2.self_attn.o_proj": 72,
|
| 108 |
+
"model.language_model.layers.2.self_attn.q_proj": 72,
|
| 109 |
+
"model.language_model.layers.2.self_attn.v_proj": 72,
|
| 110 |
+
"model.language_model.layers.3.mlp.down_proj": 72,
|
| 111 |
+
"model.language_model.layers.3.mlp.gate_proj": 72,
|
| 112 |
+
"model.language_model.layers.3.mlp.up_proj": 72,
|
| 113 |
+
"model.language_model.layers.3.self_attn.k_proj": 72,
|
| 114 |
+
"model.language_model.layers.3.self_attn.o_proj": 72,
|
| 115 |
+
"model.language_model.layers.3.self_attn.q_proj": 72,
|
| 116 |
+
"model.language_model.layers.3.self_attn.v_proj": 72,
|
| 117 |
+
"model.language_model.layers.4.mlp.down_proj": 72,
|
| 118 |
+
"model.language_model.layers.4.mlp.gate_proj": 72,
|
| 119 |
+
"model.language_model.layers.4.mlp.up_proj": 72,
|
| 120 |
+
"model.language_model.layers.4.self_attn.k_proj": 72,
|
| 121 |
+
"model.language_model.layers.4.self_attn.o_proj": 72,
|
| 122 |
+
"model.language_model.layers.4.self_attn.q_proj": 72,
|
| 123 |
+
"model.language_model.layers.4.self_attn.v_proj": 72,
|
| 124 |
+
"model.language_model.layers.5.mlp.down_proj": 72,
|
| 125 |
+
"model.language_model.layers.5.mlp.gate_proj": 72,
|
| 126 |
+
"model.language_model.layers.5.mlp.up_proj": 72,
|
| 127 |
+
"model.language_model.layers.5.self_attn.k_proj": 72,
|
| 128 |
+
"model.language_model.layers.5.self_attn.o_proj": 72,
|
| 129 |
+
"model.language_model.layers.5.self_attn.q_proj": 72,
|
| 130 |
+
"model.language_model.layers.5.self_attn.v_proj": 72,
|
| 131 |
+
"model.language_model.layers.6.mlp.down_proj": 72,
|
| 132 |
+
"model.language_model.layers.6.mlp.gate_proj": 72,
|
| 133 |
+
"model.language_model.layers.6.mlp.up_proj": 72,
|
| 134 |
+
"model.language_model.layers.6.self_attn.k_proj": 72,
|
| 135 |
+
"model.language_model.layers.6.self_attn.o_proj": 72,
|
| 136 |
+
"model.language_model.layers.6.self_attn.q_proj": 72,
|
| 137 |
+
"model.language_model.layers.6.self_attn.v_proj": 72,
|
| 138 |
+
"model.language_model.layers.7.mlp.down_proj": 72,
|
| 139 |
+
"model.language_model.layers.7.mlp.gate_proj": 72,
|
| 140 |
+
"model.language_model.layers.7.mlp.up_proj": 72,
|
| 141 |
+
"model.language_model.layers.7.self_attn.k_proj": 72,
|
| 142 |
+
"model.language_model.layers.7.self_attn.o_proj": 72,
|
| 143 |
+
"model.language_model.layers.7.self_attn.q_proj": 72,
|
| 144 |
+
"model.language_model.layers.7.self_attn.v_proj": 72
|
| 145 |
+
},
|
| 146 |
+
"revision": null,
|
| 147 |
+
"target_modules": [
|
| 148 |
+
"model.language_model.layers.0.mlp.up_proj",
|
| 149 |
+
"model.language_model.layers.6.mlp.gate_proj",
|
| 150 |
+
"model.language_model.layers.0.self_attn.q_proj",
|
| 151 |
+
"model.language_model.layers.1.self_attn.v_proj",
|
| 152 |
+
"model.language_model.layers.0.self_attn.k_proj",
|
| 153 |
+
"model.language_model.layers.1.mlp.up_proj",
|
| 154 |
+
"model.language_model.layers.6.self_attn.k_proj",
|
| 155 |
+
"model.language_model.layers.5.mlp.down_proj",
|
| 156 |
+
"model.language_model.layers.2.self_attn.v_proj",
|
| 157 |
+
"model.language_model.layers.1.self_attn.q_proj",
|
| 158 |
+
"model.language_model.layers.5.self_attn.o_proj",
|
| 159 |
+
"model.language_model.layers.1.self_attn.o_proj",
|
| 160 |
+
"model.language_model.layers.0.self_attn.o_proj",
|
| 161 |
+
"model.language_model.layers.3.self_attn.o_proj",
|
| 162 |
+
"model.language_model.layers.4.mlp.up_proj",
|
| 163 |
+
"model.language_model.layers.5.mlp.up_proj",
|
| 164 |
+
"model.language_model.layers.7.self_attn.o_proj",
|
| 165 |
+
"model.language_model.layers.2.self_attn.q_proj",
|
| 166 |
+
"model.language_model.layers.2.self_attn.k_proj",
|
| 167 |
+
"model.language_model.layers.0.self_attn.v_proj",
|
| 168 |
+
"model.language_model.layers.7.self_attn.v_proj",
|
| 169 |
+
"model.language_model.layers.1.mlp.gate_proj",
|
| 170 |
+
"model.language_model.layers.4.mlp.gate_proj",
|
| 171 |
+
"model.language_model.layers.2.mlp.down_proj",
|
| 172 |
+
"model.language_model.layers.5.self_attn.k_proj",
|
| 173 |
+
"model.language_model.layers.5.self_attn.v_proj",
|
| 174 |
+
"model.language_model.layers.6.self_attn.v_proj",
|
| 175 |
+
"model.language_model.layers.4.self_attn.q_proj",
|
| 176 |
+
"model.language_model.layers.0.mlp.gate_proj",
|
| 177 |
+
"model.language_model.layers.6.mlp.down_proj",
|
| 178 |
+
"model.language_model.layers.4.mlp.down_proj",
|
| 179 |
+
"model.language_model.layers.2.mlp.up_proj",
|
| 180 |
+
"model.language_model.layers.1.self_attn.k_proj",
|
| 181 |
+
"model.language_model.layers.7.mlp.up_proj",
|
| 182 |
+
"model.language_model.layers.3.mlp.up_proj",
|
| 183 |
+
"model.language_model.layers.3.self_attn.k_proj",
|
| 184 |
+
"model.language_model.layers.3.self_attn.q_proj",
|
| 185 |
+
"model.language_model.layers.6.self_attn.q_proj",
|
| 186 |
+
"model.language_model.layers.6.self_attn.o_proj",
|
| 187 |
+
"model.language_model.layers.4.self_attn.v_proj",
|
| 188 |
+
"model.language_model.layers.6.mlp.up_proj",
|
| 189 |
+
"model.language_model.layers.7.mlp.down_proj",
|
| 190 |
+
"model.language_model.layers.4.self_attn.o_proj",
|
| 191 |
+
"model.language_model.layers.0.mlp.down_proj",
|
| 192 |
+
"model.language_model.layers.2.mlp.gate_proj",
|
| 193 |
+
"model.language_model.layers.5.self_attn.q_proj",
|
| 194 |
+
"model.language_model.layers.1.mlp.down_proj",
|
| 195 |
+
"model.language_model.layers.3.mlp.down_proj",
|
| 196 |
+
"model.language_model.layers.7.mlp.gate_proj",
|
| 197 |
+
"model.language_model.layers.2.self_attn.o_proj",
|
| 198 |
+
"model.language_model.layers.4.self_attn.k_proj",
|
| 199 |
+
"model.language_model.layers.5.mlp.gate_proj",
|
| 200 |
+
"model.language_model.layers.7.self_attn.k_proj",
|
| 201 |
+
"model.language_model.layers.3.mlp.gate_proj",
|
| 202 |
+
"model.language_model.layers.3.self_attn.v_proj",
|
| 203 |
+
"model.language_model.layers.7.self_attn.q_proj"
|
| 204 |
+
],
|
| 205 |
+
"target_parameters": null,
|
| 206 |
+
"task_type": "CAUSAL_LM",
|
| 207 |
+
"trainable_token_indices": null,
|
| 208 |
+
"use_bdlora": null,
|
| 209 |
+
"use_dora": false,
|
| 210 |
+
"use_qalora": false,
|
| 211 |
+
"use_rslora": false,
|
| 212 |
+
"velora_config": null
|
| 213 |
+
}
|
runs/fixed_internvl/removal/Shallow8_s42/config.json
ADDED
|
@@ -0,0 +1,43 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"behavior": "removal",
|
| 3 |
+
"seed": 42,
|
| 4 |
+
"model": "/data/OPSD/assets/models/InternVL3_5-8B-HF",
|
| 5 |
+
"backbone": "internvl",
|
| 6 |
+
"train": "/data/OPSD/assets/v3/removal.jsonl",
|
| 7 |
+
"items": 1000,
|
| 8 |
+
"steps": 250,
|
| 9 |
+
"epochs": 1,
|
| 10 |
+
"steps_per_epoch": 250,
|
| 11 |
+
"world_size": 1,
|
| 12 |
+
"lr": 5e-05,
|
| 13 |
+
"scheduler": "cosine",
|
| 14 |
+
"warmup_steps": 16,
|
| 15 |
+
"final_fraction": 0.1,
|
| 16 |
+
"clip": 1.0,
|
| 17 |
+
"initial_loss_scale": 4.0,
|
| 18 |
+
"arm": {
|
| 19 |
+
"arm": "Shallow8",
|
| 20 |
+
"layers": [
|
| 21 |
+
0,
|
| 22 |
+
1,
|
| 23 |
+
2,
|
| 24 |
+
3,
|
| 25 |
+
4,
|
| 26 |
+
5,
|
| 27 |
+
6,
|
| 28 |
+
7
|
| 29 |
+
],
|
| 30 |
+
"ranks": {
|
| 31 |
+
"0": 72,
|
| 32 |
+
"1": 72,
|
| 33 |
+
"2": 72,
|
| 34 |
+
"3": 72,
|
| 35 |
+
"4": 72,
|
| 36 |
+
"5": 72,
|
| 37 |
+
"6": 72,
|
| 38 |
+
"7": 72
|
| 39 |
+
},
|
| 40 |
+
"parameters": 43646976,
|
| 41 |
+
"budget": 43646976
|
| 42 |
+
}
|
| 43 |
+
}
|
runs/fixed_internvl/removal/Shallow8_s42/steps.jsonl
ADDED
|
@@ -0,0 +1,250 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{"step": 1, "steps": 250, "lr": 3.125e-06, "grad_norm": 10.691793441772461, "loss_scale": 4.0, "retries": 0, "seconds": 5.2, "max_memory_gb": 16.79, "ce": 1.0494785578921437, "rank": 0.15784265100955963}
|
| 2 |
+
{"step": 2, "steps": 250, "lr": 6.25e-06, "grad_norm": 7.769438743591309, "loss_scale": 4.0, "retries": 0, "seconds": 9.8, "max_memory_gb": 17.07, "ce": 0.8100697980127052, "rank": 0.0}
|
| 3 |
+
{"step": 3, "steps": 250, "lr": 9.375000000000001e-06, "grad_norm": 12.947282791137695, "loss_scale": 4.0, "retries": 0, "seconds": 14.3, "max_memory_gb": 17.08, "ce": 0.7831878597498871, "rank": 0.0}
|
| 4 |
+
{"step": 4, "steps": 250, "lr": 1.25e-05, "grad_norm": 10.222810745239258, "loss_scale": 4.0, "retries": 0, "seconds": 18.7, "max_memory_gb": 17.08, "ce": 0.7411272464305512, "rank": 0.0}
|
| 5 |
+
{"step": 5, "steps": 250, "lr": 1.5625e-05, "grad_norm": 3.23907208442688, "loss_scale": 4.0, "retries": 0, "seconds": 23.3, "max_memory_gb": 17.08, "ce": 0.5754611573356669, "rank": 0.0}
|
| 6 |
+
{"step": 6, "steps": 250, "lr": 1.8750000000000002e-05, "grad_norm": 3.566537618637085, "loss_scale": 4.0, "retries": 0, "seconds": 27.7, "max_memory_gb": 17.08, "ce": 0.3552336297580041, "rank": 0.0}
|
| 7 |
+
{"step": 7, "steps": 250, "lr": 2.1875e-05, "grad_norm": 3.220242977142334, "loss_scale": 4.0, "retries": 0, "seconds": 32.2, "max_memory_gb": 17.08, "ce": 0.5624004863202572, "rank": 0.0}
|
| 8 |
+
{"step": 8, "steps": 250, "lr": 2.5e-05, "grad_norm": 1.2174885272979736, "loss_scale": 4.0, "retries": 0, "seconds": 36.6, "max_memory_gb": 17.08, "ce": 0.238988603887492, "rank": 0.0}
|
| 9 |
+
{"step": 9, "steps": 250, "lr": 2.8125000000000003e-05, "grad_norm": 3.280060291290283, "loss_scale": 4.0, "retries": 0, "seconds": 41.2, "max_memory_gb": 17.08, "ce": 0.6245620767003857, "rank": 0.0}
|
| 10 |
+
{"step": 10, "steps": 250, "lr": 3.125e-05, "grad_norm": 3.8789918422698975, "loss_scale": 4.0, "retries": 0, "seconds": 45.6, "max_memory_gb": 17.08, "ce": 0.2855562241165899, "rank": 0.20656348764896393}
|
| 11 |
+
{"step": 11, "steps": 250, "lr": 3.4375e-05, "grad_norm": 0.3166007697582245, "loss_scale": 4.0, "retries": 0, "seconds": 50.1, "max_memory_gb": 17.08, "ce": 0.15032702480675653, "rank": 0.0}
|
| 12 |
+
{"step": 12, "steps": 250, "lr": 3.7500000000000003e-05, "grad_norm": 4.4183149337768555, "loss_scale": 4.0, "retries": 0, "seconds": 54.6, "max_memory_gb": 17.08, "ce": 0.672210159504175, "rank": 0.0}
|
| 13 |
+
{"step": 13, "steps": 250, "lr": 4.0625000000000005e-05, "grad_norm": 0.9588836431503296, "loss_scale": 4.0, "retries": 0, "seconds": 59.1, "max_memory_gb": 17.08, "ce": 0.4022517262492329, "rank": 0.0}
|
| 14 |
+
{"step": 14, "steps": 250, "lr": 4.375e-05, "grad_norm": 0.5720311403274536, "loss_scale": 4.0, "retries": 0, "seconds": 63.5, "max_memory_gb": 17.08, "ce": 0.3523502911848482, "rank": 0.0}
|
| 15 |
+
{"step": 15, "steps": 250, "lr": 4.6875e-05, "grad_norm": 5.069281101226807, "loss_scale": 4.0, "retries": 0, "seconds": 68.1, "max_memory_gb": 17.08, "ce": 0.6297689026687294, "rank": 0.18836742639541626}
|
| 16 |
+
{"step": 16, "steps": 250, "lr": 5e-05, "grad_norm": 4.085418701171875, "loss_scale": 4.0, "retries": 0, "seconds": 72.6, "max_memory_gb": 17.08, "ce": 0.4279387279821094, "rank": 0.0}
|
| 17 |
+
{"step": 17, "steps": 250, "lr": 5e-05, "grad_norm": 0.62175053358078, "loss_scale": 4.0, "retries": 0, "seconds": 77.0, "max_memory_gb": 17.08, "ce": 0.31025544786825776, "rank": 0.11283248662948608}
|
| 18 |
+
{"step": 18, "steps": 250, "lr": 4.999797225177623e-05, "grad_norm": 0.8662994503974915, "loss_scale": 4.0, "retries": 0, "seconds": 81.5, "max_memory_gb": 17.08, "ce": 0.33728199638426304, "rank": 0.0}
|
| 19 |
+
{"step": 19, "steps": 250, "lr": 4.999188937259492e-05, "grad_norm": 0.31199392676353455, "loss_scale": 4.0, "retries": 0, "seconds": 86.1, "max_memory_gb": 17.08, "ce": 0.18851770453329664, "rank": 0.0}
|
| 20 |
+
{"step": 20, "steps": 250, "lr": 4.998175245886032e-05, "grad_norm": 0.7191319465637207, "loss_scale": 4.0, "retries": 0, "seconds": 90.6, "max_memory_gb": 17.08, "ce": 0.3065192496869713, "rank": 0.0}
|
| 21 |
+
{"step": 21, "steps": 250, "lr": 4.996756333769319e-05, "grad_norm": 0.39756298065185547, "loss_scale": 4.0, "retries": 0, "seconds": 95.0, "max_memory_gb": 17.08, "ce": 0.04696882585994899, "rank": 0.0}
|
| 22 |
+
{"step": 22, "steps": 250, "lr": 4.994932456660157e-05, "grad_norm": 0.7852669358253479, "loss_scale": 4.0, "retries": 0, "seconds": 99.6, "max_memory_gb": 17.08, "ce": 0.347306314506568, "rank": 0.0}
|
| 23 |
+
{"step": 23, "steps": 250, "lr": 4.992703943301973e-05, "grad_norm": 1.224972128868103, "loss_scale": 4.0, "retries": 0, "seconds": 104.0, "max_memory_gb": 17.08, "ce": 0.2824508753910777, "rank": 0.0}
|
| 24 |
+
{"step": 24, "steps": 250, "lr": 4.9900711953715675e-05, "grad_norm": 1.2308979034423828, "loss_scale": 4.0, "retries": 0, "seconds": 108.5, "max_memory_gb": 17.08, "ce": 0.3759514289922663, "rank": 0.0}
|
| 25 |
+
{"step": 25, "steps": 250, "lr": 4.987034687406713e-05, "grad_norm": 1.1056064367294312, "loss_scale": 4.0, "retries": 0, "seconds": 113.0, "max_memory_gb": 17.08, "ce": 0.2152329699092661, "rank": 0.12479063868522644}
|
| 26 |
+
{"step": 26, "steps": 250, "lr": 4.9835949667206215e-05, "grad_norm": 2.544781446456909, "loss_scale": 4.0, "retries": 0, "seconds": 117.5, "max_memory_gb": 17.08, "ce": 0.2121964785947057, "rank": 0.0}
|
| 27 |
+
{"step": 27, "steps": 250, "lr": 4.9797526533032945e-05, "grad_norm": 5.255162715911865, "loss_scale": 4.0, "retries": 0, "seconds": 121.9, "max_memory_gb": 17.08, "ce": 0.42759406384357135, "rank": 0.0}
|
| 28 |
+
{"step": 28, "steps": 250, "lr": 4.9755084397097716e-05, "grad_norm": 1.787534236907959, "loss_scale": 4.0, "retries": 0, "seconds": 126.4, "max_memory_gb": 17.08, "ce": 0.3700789106078446, "rank": 0.0}
|
| 29 |
+
{"step": 29, "steps": 250, "lr": 4.970863090935304e-05, "grad_norm": 0.7178069949150085, "loss_scale": 4.0, "retries": 0, "seconds": 131.0, "max_memory_gb": 17.08, "ce": 0.1313860909831419, "rank": 0.0}
|
| 30 |
+
{"step": 30, "steps": 250, "lr": 4.965817444277468e-05, "grad_norm": 1.7786226272583008, "loss_scale": 4.0, "retries": 0, "seconds": 135.5, "max_memory_gb": 17.08, "ce": 0.1914353227475658, "rank": 0.12479273229837418}
|
| 31 |
+
{"step": 31, "steps": 250, "lr": 4.9603724091852456e-05, "grad_norm": 4.360067367553711, "loss_scale": 4.0, "retries": 0, "seconds": 139.9, "max_memory_gb": 17.08, "ce": 0.8431042229785817, "rank": 0.0}
|
| 32 |
+
{"step": 32, "steps": 250, "lr": 4.9545289670951025e-05, "grad_norm": 0.6687426567077637, "loss_scale": 4.0, "retries": 0, "seconds": 144.5, "max_memory_gb": 17.08, "ce": 0.039251944010175066, "rank": 0.0}
|
| 33 |
+
{"step": 33, "steps": 250, "lr": 4.948288171254089e-05, "grad_norm": 0.5778456330299377, "loss_scale": 4.0, "retries": 0, "seconds": 148.9, "max_memory_gb": 17.08, "ce": 0.023515447674014922, "rank": 0.0}
|
| 34 |
+
{"step": 34, "steps": 250, "lr": 4.941651146529998e-05, "grad_norm": 1.212299108505249, "loss_scale": 4.0, "retries": 0, "seconds": 153.4, "max_memory_gb": 17.08, "ce": 0.055374907250552496, "rank": 0.0}
|
| 35 |
+
{"step": 35, "steps": 250, "lr": 4.9346190892086174e-05, "grad_norm": 1.132712960243225, "loss_scale": 4.0, "retries": 0, "seconds": 157.9, "max_memory_gb": 17.08, "ce": 0.07095804507309822, "rank": 0.0}
|
| 36 |
+
{"step": 36, "steps": 250, "lr": 4.927193266778101e-05, "grad_norm": 13.726927757263184, "loss_scale": 4.0, "retries": 0, "seconds": 162.4, "max_memory_gb": 17.08, "ce": 0.35839759511765124, "rank": 0.0}
|
| 37 |
+
{"step": 37, "steps": 250, "lr": 4.919375017700515e-05, "grad_norm": 12.728412628173828, "loss_scale": 4.0, "retries": 0, "seconds": 166.8, "max_memory_gb": 17.08, "ce": 0.24830690555836554, "rank": 0.0}
|
| 38 |
+
{"step": 38, "steps": 250, "lr": 4.911165751170588e-05, "grad_norm": 0.6260102987289429, "loss_scale": 4.0, "retries": 0, "seconds": 171.3, "max_memory_gb": 17.08, "ce": 0.04766224220657023, "rank": 0.0}
|
| 39 |
+
{"step": 39, "steps": 250, "lr": 4.902566946861708e-05, "grad_norm": 3.6834051609039307, "loss_scale": 4.0, "retries": 0, "seconds": 175.9, "max_memory_gb": 17.08, "ce": 0.20512884939671494, "rank": 0.0}
|
| 40 |
+
{"step": 40, "steps": 250, "lr": 4.89358015465922e-05, "grad_norm": 1.4396167993545532, "loss_scale": 4.0, "retries": 0, "seconds": 180.3, "max_memory_gb": 17.08, "ce": 0.06621196517194505, "rank": 0.0}
|
| 41 |
+
{"step": 41, "steps": 250, "lr": 4.884206994381078e-05, "grad_norm": 0.2772051692008972, "loss_scale": 4.0, "retries": 0, "seconds": 184.8, "max_memory_gb": 17.08, "ce": 0.018811710350291833, "rank": 0.0}
|
| 42 |
+
{"step": 42, "steps": 250, "lr": 4.874449155485866e-05, "grad_norm": 2.017693281173706, "loss_scale": 4.0, "retries": 0, "seconds": 189.4, "max_memory_gb": 17.08, "ce": 0.2364705051902547, "rank": 0.0}
|
| 43 |
+
{"step": 43, "steps": 250, "lr": 4.864308396768294e-05, "grad_norm": 3.0354678630828857, "loss_scale": 4.0, "retries": 0, "seconds": 193.8, "max_memory_gb": 17.08, "ce": 0.08754671119095292, "rank": 0.0}
|
| 44 |
+
{"step": 44, "steps": 250, "lr": 4.853786546042184e-05, "grad_norm": 4.909041881561279, "loss_scale": 4.0, "retries": 0, "seconds": 198.3, "max_memory_gb": 17.08, "ce": 0.4091557537749395, "rank": 0.0}
|
| 45 |
+
{"step": 45, "steps": 250, "lr": 4.842885499811012e-05, "grad_norm": 5.784178733825684, "loss_scale": 4.0, "retries": 0, "seconds": 202.8, "max_memory_gb": 17.08, "ce": 0.40233052744952147, "rank": 0.0}
|
| 46 |
+
{"step": 46, "steps": 250, "lr": 4.831607222926078e-05, "grad_norm": 2.419450044631958, "loss_scale": 4.0, "retries": 0, "seconds": 207.3, "max_memory_gb": 17.08, "ce": 0.2803199784921162, "rank": 0.0}
|
| 47 |
+
{"step": 47, "steps": 250, "lr": 4.8199537482323545e-05, "grad_norm": 0.5745157599449158, "loss_scale": 4.0, "retries": 0, "seconds": 211.8, "max_memory_gb": 17.08, "ce": 0.017278194354730658, "rank": 0.0}
|
| 48 |
+
{"step": 48, "steps": 250, "lr": 4.807927176202073e-05, "grad_norm": 1.5438340902328491, "loss_scale": 4.0, "retries": 0, "seconds": 216.3, "max_memory_gb": 17.08, "ce": 0.10021555778075708, "rank": 0.0}
|
| 49 |
+
{"step": 49, "steps": 250, "lr": 4.795529674556129e-05, "grad_norm": 2.3842592239379883, "loss_scale": 4.0, "retries": 0, "seconds": 220.8, "max_memory_gb": 17.08, "ce": 0.11757284689974767, "rank": 0.12500645220279694}
|
| 50 |
+
{"step": 50, "steps": 250, "lr": 4.782763477873361e-05, "grad_norm": 0.8045728206634521, "loss_scale": 4.0, "retries": 0, "seconds": 225.3, "max_memory_gb": 17.08, "ce": 0.049242971086641774, "rank": 0.0}
|
| 51 |
+
{"step": 51, "steps": 250, "lr": 4.769630887187782e-05, "grad_norm": 1.6918541193008423, "loss_scale": 4.0, "retries": 0, "seconds": 229.8, "max_memory_gb": 17.08, "ce": 0.047292826711782254, "rank": 0.0}
|
| 52 |
+
{"step": 52, "steps": 250, "lr": 4.756134269573832e-05, "grad_norm": 0.15955330431461334, "loss_scale": 4.0, "retries": 0, "seconds": 234.3, "max_memory_gb": 17.08, "ce": 0.008817783956658332, "rank": 0.12511959671974182}
|
| 53 |
+
{"step": 53, "steps": 250, "lr": 4.7422760577197226e-05, "grad_norm": 0.5566014051437378, "loss_scale": 4.0, "retries": 0, "seconds": 238.8, "max_memory_gb": 17.08, "ce": 0.02038881665066583, "rank": 0.0}
|
| 54 |
+
{"step": 54, "steps": 250, "lr": 4.728058749488963e-05, "grad_norm": 1.8401858806610107, "loss_scale": 4.0, "retries": 0, "seconds": 243.2, "max_memory_gb": 17.08, "ce": 0.11089288712332745, "rank": 0.0}
|
| 55 |
+
{"step": 55, "steps": 250, "lr": 4.713484907470133e-05, "grad_norm": 10.157181739807129, "loss_scale": 4.0, "retries": 0, "seconds": 247.6, "max_memory_gb": 17.08, "ce": 0.26133062838971455, "rank": 0.0}
|
| 56 |
+
{"step": 56, "steps": 250, "lr": 4.6985571585149876e-05, "grad_norm": 1.9826438426971436, "loss_scale": 4.0, "retries": 0, "seconds": 252.1, "max_memory_gb": 17.08, "ce": 0.08265467402753757, "rank": 0.0}
|
| 57 |
+
{"step": 57, "steps": 250, "lr": 4.6832781932649884e-05, "grad_norm": 0.5474808216094971, "loss_scale": 4.0, "retries": 0, "seconds": 256.6, "max_memory_gb": 17.08, "ce": 0.03247644841530928, "rank": 0.0}
|
| 58 |
+
{"step": 58, "steps": 250, "lr": 4.667650765666325e-05, "grad_norm": 1.6119894981384277, "loss_scale": 4.0, "retries": 0, "seconds": 261.1, "max_memory_gb": 17.08, "ce": 0.12027547123580007, "rank": 0.0}
|
| 59 |
+
{"step": 59, "steps": 250, "lr": 4.651677692473538e-05, "grad_norm": 0.7611097097396851, "loss_scale": 4.0, "retries": 0, "seconds": 265.6, "max_memory_gb": 17.08, "ce": 0.010244221036032286, "rank": 0.0}
|
| 60 |
+
{"step": 60, "steps": 250, "lr": 4.635361852741808e-05, "grad_norm": 2.7429685592651367, "loss_scale": 4.0, "retries": 0, "seconds": 270.0, "max_memory_gb": 17.08, "ce": 0.2111250714003745, "rank": 0.0}
|
| 61 |
+
{"step": 61, "steps": 250, "lr": 4.6187061873080264e-05, "grad_norm": 0.32108885049819946, "loss_scale": 4.0, "retries": 0, "seconds": 274.5, "max_memory_gb": 17.08, "ce": 0.019607677902968135, "rank": 0.0}
|
| 62 |
+
{"step": 62, "steps": 250, "lr": 4.601713698260727e-05, "grad_norm": 0.7305247783660889, "loss_scale": 4.0, "retries": 0, "seconds": 279.1, "max_memory_gb": 17.08, "ce": 0.03477143450800213, "rank": 0.0}
|
| 63 |
+
{"step": 63, "steps": 250, "lr": 4.5843874483989746e-05, "grad_norm": 4.032339096069336, "loss_scale": 4.0, "retries": 0, "seconds": 283.5, "max_memory_gb": 17.08, "ce": 0.2684007750456203, "rank": 0.03704613447189331}
|
| 64 |
+
{"step": 64, "steps": 250, "lr": 4.566730560680316e-05, "grad_norm": 2.795908212661743, "loss_scale": 4.0, "retries": 0, "seconds": 288.0, "max_memory_gb": 17.08, "ce": 0.10177817188377958, "rank": 0.0}
|
| 65 |
+
{"step": 65, "steps": 250, "lr": 4.548746217657878e-05, "grad_norm": 5.113523960113525, "loss_scale": 4.0, "retries": 0, "seconds": 292.4, "max_memory_gb": 17.08, "ce": 0.11846303134370828, "rank": 0.0}
|
| 66 |
+
{"step": 66, "steps": 250, "lr": 4.5304376609067396e-05, "grad_norm": 2.5660736560821533, "loss_scale": 4.0, "retries": 0, "seconds": 296.9, "max_memory_gb": 17.08, "ce": 0.06065209502548896, "rank": 0.0}
|
| 67 |
+
{"step": 67, "steps": 250, "lr": 4.5118081904396504e-05, "grad_norm": 1.7022958993911743, "loss_scale": 4.0, "retries": 0, "seconds": 301.4, "max_memory_gb": 17.08, "ce": 0.01781205287988996, "rank": 0.0}
|
| 68 |
+
{"step": 68, "steps": 250, "lr": 4.4928611641122225e-05, "grad_norm": 0.28610169887542725, "loss_scale": 4.0, "retries": 0, "seconds": 305.8, "max_memory_gb": 17.08, "ce": 0.01646565196665506, "rank": 0.0}
|
| 69 |
+
{"step": 69, "steps": 250, "lr": 4.473599997017701e-05, "grad_norm": 1.7572736740112305, "loss_scale": 4.0, "retries": 0, "seconds": 310.4, "max_memory_gb": 17.08, "ce": 0.04222026240313426, "rank": 0.0}
|
| 70 |
+
{"step": 70, "steps": 250, "lr": 4.4540281608714053e-05, "grad_norm": 4.175060749053955, "loss_scale": 4.0, "retries": 0, "seconds": 314.8, "max_memory_gb": 17.08, "ce": 0.2508315232489622, "rank": 0.0}
|
| 71 |
+
{"step": 71, "steps": 250, "lr": 4.434149183384977e-05, "grad_norm": 5.129703044891357, "loss_scale": 4.0, "retries": 0, "seconds": 319.2, "max_memory_gb": 17.08, "ce": 0.5141433255521406, "rank": 0.11855655908584595}
|
| 72 |
+
{"step": 72, "steps": 250, "lr": 4.413966647630534e-05, "grad_norm": 0.6091258525848389, "loss_scale": 4.0, "retries": 0, "seconds": 323.8, "max_memory_gb": 17.08, "ce": 0.009291953741239922, "rank": 0.0}
|
| 73 |
+
{"step": 73, "steps": 250, "lr": 4.3934841913948323e-05, "grad_norm": 5.985304832458496, "loss_scale": 4.0, "retries": 0, "seconds": 328.3, "max_memory_gb": 17.08, "ce": 0.38191363124497, "rank": 0.0}
|
| 74 |
+
{"step": 74, "steps": 250, "lr": 4.372705506523583e-05, "grad_norm": 1.5559202432632446, "loss_scale": 4.0, "retries": 0, "seconds": 332.7, "max_memory_gb": 17.08, "ce": 0.0638923737424193, "rank": 0.0}
|
| 75 |
+
{"step": 75, "steps": 250, "lr": 4.351634338256017e-05, "grad_norm": 1.4048339128494263, "loss_scale": 4.0, "retries": 0, "seconds": 337.1, "max_memory_gb": 17.08, "ce": 0.05898925862493343, "rank": 0.0}
|
| 76 |
+
{"step": 76, "steps": 250, "lr": 4.330274484549825e-05, "grad_norm": 0.5258691906929016, "loss_scale": 4.0, "retries": 0, "seconds": 341.7, "max_memory_gb": 17.08, "ce": 0.023935479493957246, "rank": 0.0}
|
| 77 |
+
{"step": 77, "steps": 250, "lr": 4.308629795396599e-05, "grad_norm": 0.31516560912132263, "loss_scale": 4.0, "retries": 0, "seconds": 346.2, "max_memory_gb": 17.08, "ce": 0.017877637307606165, "rank": 0.0}
|
| 78 |
+
{"step": 78, "steps": 250, "lr": 4.286704172127894e-05, "grad_norm": 0.44787174463272095, "loss_scale": 4.0, "retries": 0, "seconds": 350.6, "max_memory_gb": 17.08, "ce": 0.030918184595975617, "rank": 0.0}
|
| 79 |
+
{"step": 79, "steps": 250, "lr": 4.264501566712033e-05, "grad_norm": 2.208873748779297, "loss_scale": 4.0, "retries": 0, "seconds": 355.2, "max_memory_gb": 17.08, "ce": 0.17725928741515418, "rank": 0.0}
|
| 80 |
+
{"step": 80, "steps": 250, "lr": 4.2420259810417894e-05, "grad_norm": 2.0227460861206055, "loss_scale": 4.0, "retries": 0, "seconds": 359.6, "max_memory_gb": 17.08, "ce": 0.2617454817459475, "rank": 0.0}
|
| 81 |
+
{"step": 81, "steps": 250, "lr": 4.219281466213066e-05, "grad_norm": 4.9309797286987305, "loss_scale": 4.0, "retries": 0, "seconds": 364.0, "max_memory_gb": 17.08, "ce": 0.6466512720762694, "rank": 0.0}
|
| 82 |
+
{"step": 82, "steps": 250, "lr": 4.1962721217947135e-05, "grad_norm": 1.2273527383804321, "loss_scale": 4.0, "retries": 0, "seconds": 368.5, "max_memory_gb": 17.08, "ce": 0.06310202170425327, "rank": 0.0}
|
| 83 |
+
{"step": 83, "steps": 250, "lr": 4.173002095089599e-05, "grad_norm": 1.2323691844940186, "loss_scale": 4.0, "retries": 0, "seconds": 373.1, "max_memory_gb": 17.08, "ce": 0.05384800158935832, "rank": 0.0}
|
| 84 |
+
{"step": 84, "steps": 250, "lr": 4.149475580387084e-05, "grad_norm": 3.58412766456604, "loss_scale": 4.0, "retries": 0, "seconds": 377.5, "max_memory_gb": 17.08, "ce": 0.2246076180779255, "rank": 0.0}
|
| 85 |
+
{"step": 85, "steps": 250, "lr": 4.1256968182070275e-05, "grad_norm": 0.6301409602165222, "loss_scale": 4.0, "retries": 0, "seconds": 381.9, "max_memory_gb": 17.08, "ce": 0.04273835284823235, "rank": 0.0}
|
| 86 |
+
{"step": 86, "steps": 250, "lr": 4.101670094535453e-05, "grad_norm": 0.9113795757293701, "loss_scale": 4.0, "retries": 0, "seconds": 386.5, "max_memory_gb": 17.08, "ce": 0.01680740415395121, "rank": 0.1247253268957138}
|
| 87 |
+
{"step": 87, "steps": 250, "lr": 4.077399740052027e-05, "grad_norm": 0.5277889966964722, "loss_scale": 4.0, "retries": 0, "seconds": 390.9, "max_memory_gb": 17.08, "ce": 0.020219537135972132, "rank": 0.0}
|
| 88 |
+
{"step": 88, "steps": 250, "lr": 4.0528901293494775e-05, "grad_norm": 1.5600006580352783, "loss_scale": 4.0, "retries": 0, "seconds": 395.3, "max_memory_gb": 17.08, "ce": 0.049005720500190364, "rank": 0.0}
|
| 89 |
+
{"step": 89, "steps": 250, "lr": 4.0281456801451004e-05, "grad_norm": 0.6380944848060608, "loss_scale": 4.0, "retries": 0, "seconds": 399.9, "max_memory_gb": 17.08, "ce": 0.010306340632610045, "rank": 0.0}
|
| 90 |
+
{"step": 90, "steps": 250, "lr": 4.0031708524844884e-05, "grad_norm": 0.3255772888660431, "loss_scale": 4.0, "retries": 0, "seconds": 404.4, "max_memory_gb": 17.08, "ce": 0.0008091992574463802, "rank": 0.10908585786819458}
|
| 91 |
+
{"step": 91, "steps": 250, "lr": 3.977970147937635e-05, "grad_norm": 0.3143400251865387, "loss_scale": 4.0, "retries": 0, "seconds": 408.9, "max_memory_gb": 17.08, "ce": 0.010754860031738644, "rank": 0.0}
|
| 92 |
+
{"step": 92, "steps": 250, "lr": 3.952548108787553e-05, "grad_norm": 0.5259100198745728, "loss_scale": 4.0, "retries": 0, "seconds": 413.3, "max_memory_gb": 17.08, "ce": 0.021986073083098745, "rank": 0.0}
|
| 93 |
+
{"step": 93, "steps": 250, "lr": 3.9269093172115496e-05, "grad_norm": 4.422213077545166, "loss_scale": 4.0, "retries": 0, "seconds": 417.9, "max_memory_gb": 17.08, "ce": 0.22941531098513224, "rank": 0.0}
|
| 94 |
+
{"step": 94, "steps": 250, "lr": 3.901058394455323e-05, "grad_norm": 1.598298192024231, "loss_scale": 4.0, "retries": 0, "seconds": 422.3, "max_memory_gb": 17.08, "ce": 0.17792576367901347, "rank": 0.0}
|
| 95 |
+
{"step": 95, "steps": 250, "lr": 3.875e-05, "grad_norm": 0.4046143889427185, "loss_scale": 4.0, "retries": 0, "seconds": 427.0, "max_memory_gb": 17.08, "ce": 0.01754079171041667, "rank": 0.0}
|
| 96 |
+
{"step": 96, "steps": 250, "lr": 3.8487388307223005e-05, "grad_norm": 0.9475792646408081, "loss_scale": 4.0, "retries": 0, "seconds": 431.5, "max_memory_gb": 17.08, "ce": 0.07845820056445518, "rank": 0.0}
|
| 97 |
+
{"step": 97, "steps": 250, "lr": 3.822279620047943e-05, "grad_norm": 1.4241560697555542, "loss_scale": 4.0, "retries": 0, "seconds": 436.0, "max_memory_gb": 17.08, "ce": 0.040800772704869814, "rank": 0.0}
|
| 98 |
+
{"step": 98, "steps": 250, "lr": 3.795627137098479e-05, "grad_norm": 0.11611835658550262, "loss_scale": 4.0, "retries": 0, "seconds": 440.4, "max_memory_gb": 17.08, "ce": 0.006121950358277672, "rank": 0.0}
|
| 99 |
+
{"step": 99, "steps": 250, "lr": 3.7687861858316844e-05, "grad_norm": 1.5782617330551147, "loss_scale": 4.0, "retries": 0, "seconds": 444.9, "max_memory_gb": 17.08, "ce": 0.07299576206287384, "rank": 0.0}
|
| 100 |
+
{"step": 100, "steps": 250, "lr": 3.741761604175668e-05, "grad_norm": 0.09378918260335922, "loss_scale": 4.0, "retries": 0, "seconds": 449.4, "max_memory_gb": 17.08, "ce": 0.0046815698947284545, "rank": 0.0}
|
| 101 |
+
{"step": 101, "steps": 250, "lr": 3.714558263156872e-05, "grad_norm": 2.762834310531616, "loss_scale": 4.0, "retries": 0, "seconds": 453.8, "max_memory_gb": 17.08, "ce": 0.17719761451672866, "rank": 0.0}
|
| 102 |
+
{"step": 102, "steps": 250, "lr": 3.6871810660220904e-05, "grad_norm": 0.4800083339214325, "loss_scale": 4.0, "retries": 0, "seconds": 458.2, "max_memory_gb": 17.08, "ce": 0.01407549899664673, "rank": 0.0}
|
| 103 |
+
{"step": 103, "steps": 250, "lr": 3.659634947354686e-05, "grad_norm": 3.1065282821655273, "loss_scale": 4.0, "retries": 0, "seconds": 462.7, "max_memory_gb": 17.08, "ce": 0.22019204036041629, "rank": 0.0}
|
| 104 |
+
{"step": 104, "steps": 250, "lr": 3.631924872185169e-05, "grad_norm": 1.9172741174697876, "loss_scale": 4.0, "retries": 0, "seconds": 467.2, "max_memory_gb": 17.08, "ce": 0.1402884018784789, "rank": 0.0}
|
| 105 |
+
{"step": 105, "steps": 250, "lr": 3.6040558350962674e-05, "grad_norm": 6.938055515289307, "loss_scale": 4.0, "retries": 0, "seconds": 471.6, "max_memory_gb": 17.08, "ce": 0.3417866720837992, "rank": 0.0}
|
| 106 |
+
{"step": 106, "steps": 250, "lr": 3.576032859322687e-05, "grad_norm": 6.568713665008545, "loss_scale": 4.0, "retries": 0, "seconds": 476.1, "max_memory_gb": 17.08, "ce": 0.26385292928034687, "rank": 0.0}
|
| 107 |
+
{"step": 107, "steps": 250, "lr": 3.5478609958457056e-05, "grad_norm": 2.201653480529785, "loss_scale": 4.0, "retries": 0, "seconds": 480.5, "max_memory_gb": 17.08, "ce": 0.2157207880104579, "rank": 0.0}
|
| 108 |
+
{"step": 108, "steps": 250, "lr": 3.519545322482755e-05, "grad_norm": 4.020444869995117, "loss_scale": 4.0, "retries": 0, "seconds": 485.0, "max_memory_gb": 17.08, "ce": 0.1952300241591729, "rank": 0.0}
|
| 109 |
+
{"step": 109, "steps": 250, "lr": 3.491090942972179e-05, "grad_norm": 1.2726551294326782, "loss_scale": 4.0, "retries": 0, "seconds": 489.6, "max_memory_gb": 17.08, "ce": 0.12371249168609211, "rank": 0.0}
|
| 110 |
+
{"step": 110, "steps": 250, "lr": 3.4625029860533135e-05, "grad_norm": 1.8609447479248047, "loss_scale": 4.0, "retries": 0, "seconds": 494.0, "max_memory_gb": 17.08, "ce": 0.038877344497450395, "rank": 0.0}
|
| 111 |
+
{"step": 111, "steps": 250, "lr": 3.433786604542056e-05, "grad_norm": 0.035904474556446075, "loss_scale": 4.0, "retries": 0, "seconds": 498.4, "max_memory_gb": 17.08, "ce": 0.0012792711013389635, "rank": 0.1250024139881134}
|
| 112 |
+
{"step": 112, "steps": 250, "lr": 3.404946974402104e-05, "grad_norm": 2.2242038249969482, "loss_scale": 4.0, "retries": 0, "seconds": 502.8, "max_memory_gb": 17.08, "ce": 0.08833642365834748, "rank": 0.0}
|
| 113 |
+
{"step": 113, "steps": 250, "lr": 3.375989293812019e-05, "grad_norm": 2.653136730194092, "loss_scale": 4.0, "retries": 0, "seconds": 507.3, "max_memory_gb": 17.08, "ce": 0.4665809724256178, "rank": 0.0}
|
| 114 |
+
{"step": 114, "steps": 250, "lr": 3.3469187822282766e-05, "grad_norm": 0.16454166173934937, "loss_scale": 4.0, "retries": 0, "seconds": 511.8, "max_memory_gb": 17.08, "ce": 0.00602432741834491, "rank": 0.0}
|
| 115 |
+
{"step": 115, "steps": 250, "lr": 3.317740679444499e-05, "grad_norm": 5.808019638061523, "loss_scale": 4.0, "retries": 0, "seconds": 516.2, "max_memory_gb": 17.08, "ce": 0.2112482802449449, "rank": 0.0}
|
| 116 |
+
{"step": 116, "steps": 250, "lr": 3.2884602446470056e-05, "grad_norm": 1.215250015258789, "loss_scale": 4.0, "retries": 0, "seconds": 520.8, "max_memory_gb": 17.08, "ce": 0.03417021907716844, "rank": 0.0}
|
| 117 |
+
{"step": 117, "steps": 250, "lr": 3.259082755466876e-05, "grad_norm": 3.7216954231262207, "loss_scale": 4.0, "retries": 0, "seconds": 525.2, "max_memory_gb": 17.08, "ce": 0.3574953721943075, "rank": 0.0}
|
| 118 |
+
{"step": 118, "steps": 250, "lr": 3.229613507028686e-05, "grad_norm": 10.316554069519043, "loss_scale": 4.0, "retries": 0, "seconds": 529.7, "max_memory_gb": 17.08, "ce": 0.4591410374832776, "rank": 0.1250586062669754}
|
| 119 |
+
{"step": 119, "steps": 250, "lr": 3.2000578109961004e-05, "grad_norm": 0.6102924346923828, "loss_scale": 4.0, "retries": 0, "seconds": 534.1, "max_memory_gb": 17.08, "ce": 0.024685183894348484, "rank": 0.0}
|
| 120 |
+
{"step": 120, "steps": 250, "lr": 3.1704209946144616e-05, "grad_norm": 1.3829455375671387, "loss_scale": 4.0, "retries": 0, "seconds": 538.7, "max_memory_gb": 17.08, "ce": 0.03645238606168277, "rank": 0.0}
|
| 121 |
+
{"step": 121, "steps": 250, "lr": 3.140708399750594e-05, "grad_norm": 1.1915959119796753, "loss_scale": 4.0, "retries": 0, "seconds": 543.1, "max_memory_gb": 17.08, "ce": 0.04320977069437504, "rank": 0.0}
|
| 122 |
+
{"step": 122, "steps": 250, "lr": 3.1109253819299606e-05, "grad_norm": 0.3816032409667969, "loss_scale": 4.0, "retries": 0, "seconds": 547.6, "max_memory_gb": 17.08, "ce": 0.018408809995889897, "rank": 0.0}
|
| 123 |
+
{"step": 123, "steps": 250, "lr": 3.081077309371361e-05, "grad_norm": 0.3565094769001007, "loss_scale": 4.0, "retries": 0, "seconds": 552.1, "max_memory_gb": 17.08, "ce": 0.013250744200377085, "rank": 0.0}
|
| 124 |
+
{"step": 124, "steps": 250, "lr": 3.051169562019339e-05, "grad_norm": 0.6142849922180176, "loss_scale": 4.0, "retries": 0, "seconds": 556.6, "max_memory_gb": 17.08, "ce": 0.11793542885880015, "rank": 0.1262529194355011}
|
| 125 |
+
{"step": 125, "steps": 250, "lr": 3.021207530574477e-05, "grad_norm": 3.6863627433776855, "loss_scale": 4.0, "retries": 0, "seconds": 561.0, "max_memory_gb": 17.08, "ce": 0.03932700855148141, "rank": 0.0}
|
| 126 |
+
{"step": 126, "steps": 250, "lr": 2.991196615521758e-05, "grad_norm": 2.382298469543457, "loss_scale": 4.0, "retries": 0, "seconds": 565.5, "max_memory_gb": 17.08, "ce": 0.4886862221251249, "rank": 0.0}
|
| 127 |
+
{"step": 127, "steps": 250, "lr": 2.9611422261571536e-05, "grad_norm": 2.433229923248291, "loss_scale": 4.0, "retries": 0, "seconds": 570.0, "max_memory_gb": 17.08, "ce": 0.11087803812915809, "rank": 0.0}
|
| 128 |
+
{"step": 128, "steps": 250, "lr": 2.931049779612634e-05, "grad_norm": 0.365367591381073, "loss_scale": 4.0, "retries": 0, "seconds": 574.4, "max_memory_gb": 17.08, "ce": 0.022700324123889004, "rank": 0.0}
|
| 129 |
+
{"step": 129, "steps": 250, "lr": 2.9009246998797557e-05, "grad_norm": 3.677736520767212, "loss_scale": 4.0, "retries": 0, "seconds": 578.8, "max_memory_gb": 17.08, "ce": 0.27212957997835474, "rank": 0.0}
|
| 130 |
+
{"step": 130, "steps": 250, "lr": 2.8707724168320238e-05, "grad_norm": 0.630954384803772, "loss_scale": 4.0, "retries": 0, "seconds": 583.3, "max_memory_gb": 17.08, "ce": 0.02505146756902832, "rank": 0.0}
|
| 131 |
+
{"step": 131, "steps": 250, "lr": 2.840598365246184e-05, "grad_norm": 0.8012794256210327, "loss_scale": 4.0, "retries": 0, "seconds": 587.7, "max_memory_gb": 17.08, "ce": 0.12829644645444205, "rank": 0.0}
|
| 132 |
+
{"step": 132, "steps": 250, "lr": 2.8104079838226383e-05, "grad_norm": 0.03922359645366669, "loss_scale": 4.0, "retries": 0, "seconds": 592.1, "max_memory_gb": 17.08, "ce": 0.0011617666812071548, "rank": 0.0}
|
| 133 |
+
{"step": 133, "steps": 250, "lr": 2.7802067142051446e-05, "grad_norm": 0.13588406145572662, "loss_scale": 4.0, "retries": 0, "seconds": 596.7, "max_memory_gb": 17.08, "ce": 0.006320670778222848, "rank": 0.0}
|
| 134 |
+
{"step": 134, "steps": 250, "lr": 2.7500000000000004e-05, "grad_norm": 0.7392136454582214, "loss_scale": 4.0, "retries": 0, "seconds": 601.1, "max_memory_gb": 17.08, "ce": 0.009843640171311563, "rank": 0.0}
|
| 135 |
+
{"step": 135, "steps": 250, "lr": 2.7197932857948566e-05, "grad_norm": 2.433987855911255, "loss_scale": 4.0, "retries": 0, "seconds": 605.5, "max_memory_gb": 17.08, "ce": 0.42095848709232087, "rank": 0.0}
|
| 136 |
+
{"step": 136, "steps": 250, "lr": 2.689592016177363e-05, "grad_norm": 0.1302049607038498, "loss_scale": 4.0, "retries": 0, "seconds": 610.1, "max_memory_gb": 17.08, "ce": 0.00460344156363135, "rank": 0.0}
|
| 137 |
+
{"step": 137, "steps": 250, "lr": 2.659401634753817e-05, "grad_norm": 4.5240702629089355, "loss_scale": 4.0, "retries": 0, "seconds": 614.6, "max_memory_gb": 17.08, "ce": 0.26823169736417185, "rank": 0.0}
|
| 138 |
+
{"step": 138, "steps": 250, "lr": 2.6292275831679764e-05, "grad_norm": 0.36497387290000916, "loss_scale": 4.0, "retries": 0, "seconds": 619.0, "max_memory_gb": 17.08, "ce": 0.011527613620273769, "rank": 0.0}
|
| 139 |
+
{"step": 139, "steps": 250, "lr": 2.5990753001202445e-05, "grad_norm": 2.8313188552856445, "loss_scale": 4.0, "retries": 0, "seconds": 623.4, "max_memory_gb": 17.08, "ce": 0.2899797008376481, "rank": 0.0}
|
| 140 |
+
{"step": 140, "steps": 250, "lr": 2.5689502203873672e-05, "grad_norm": 0.16354770958423615, "loss_scale": 4.0, "retries": 0, "seconds": 628.0, "max_memory_gb": 17.08, "ce": 0.006159588101581903, "rank": 0.0}
|
| 141 |
+
{"step": 141, "steps": 250, "lr": 2.538857773842847e-05, "grad_norm": 1.4194201231002808, "loss_scale": 4.0, "retries": 0, "seconds": 632.5, "max_memory_gb": 17.08, "ce": 0.07624531377405219, "rank": 0.0}
|
| 142 |
+
{"step": 142, "steps": 250, "lr": 2.5088033844782426e-05, "grad_norm": 0.3945527672767639, "loss_scale": 4.0, "retries": 0, "seconds": 636.9, "max_memory_gb": 17.08, "ce": 0.028841078110417584, "rank": 0.0}
|
| 143 |
+
{"step": 143, "steps": 250, "lr": 2.4787924694255237e-05, "grad_norm": 1.3277637958526611, "loss_scale": 4.0, "retries": 0, "seconds": 641.5, "max_memory_gb": 17.08, "ce": 0.12788330886542099, "rank": 0.0}
|
| 144 |
+
{"step": 144, "steps": 250, "lr": 2.4488304379806627e-05, "grad_norm": 0.535059928894043, "loss_scale": 4.0, "retries": 0, "seconds": 645.9, "max_memory_gb": 17.08, "ce": 0.0519408200707403, "rank": 0.12510007619857788}
|
| 145 |
+
{"step": 145, "steps": 250, "lr": 2.4189226906286388e-05, "grad_norm": 0.15597857534885406, "loss_scale": 4.0, "retries": 0, "seconds": 650.3, "max_memory_gb": 17.08, "ce": 0.005722947115827992, "rank": 0.0}
|
| 146 |
+
{"step": 146, "steps": 250, "lr": 2.3890746180700395e-05, "grad_norm": 0.5764433741569519, "loss_scale": 4.0, "retries": 0, "seconds": 654.9, "max_memory_gb": 17.08, "ce": 0.008958919191172754, "rank": 0.0}
|
| 147 |
+
{"step": 147, "steps": 250, "lr": 2.359291600249407e-05, "grad_norm": 1.894623041152954, "loss_scale": 4.0, "retries": 0, "seconds": 659.3, "max_memory_gb": 17.08, "ce": 0.07306730249820248, "rank": 0.0}
|
| 148 |
+
{"step": 148, "steps": 250, "lr": 2.3295790053855392e-05, "grad_norm": 7.2548747062683105, "loss_scale": 4.0, "retries": 0, "seconds": 663.8, "max_memory_gb": 17.08, "ce": 0.2538352891760951, "rank": 0.0}
|
| 149 |
+
{"step": 149, "steps": 250, "lr": 2.299942189003901e-05, "grad_norm": 3.413327693939209, "loss_scale": 4.0, "retries": 0, "seconds": 668.5, "max_memory_gb": 17.08, "ce": 0.27467656442604493, "rank": 0.0}
|
| 150 |
+
{"step": 150, "steps": 250, "lr": 2.2703864929713144e-05, "grad_norm": 1.025615930557251, "loss_scale": 4.0, "retries": 0, "seconds": 673.2, "max_memory_gb": 17.08, "ce": 0.05739600681772572, "rank": 0.0}
|
| 151 |
+
{"step": 151, "steps": 250, "lr": 2.2409172445331257e-05, "grad_norm": 1.1344980001449585, "loss_scale": 4.0, "retries": 0, "seconds": 677.1, "max_memory_gb": 17.08, "ce": 0.015191434795269743, "rank": 0.0}
|
| 152 |
+
{"step": 152, "steps": 250, "lr": 2.211539755352996e-05, "grad_norm": 5.534689903259277, "loss_scale": 4.0, "retries": 0, "seconds": 680.9, "max_memory_gb": 17.08, "ce": 0.11884323588128609, "rank": 0.13786637783050537}
|
| 153 |
+
{"step": 153, "steps": 250, "lr": 2.1822593205555016e-05, "grad_norm": 0.41015729308128357, "loss_scale": 4.0, "retries": 0, "seconds": 684.9, "max_memory_gb": 17.08, "ce": 0.028785129314201185, "rank": 0.0}
|
| 154 |
+
{"step": 154, "steps": 250, "lr": 2.1530812177717236e-05, "grad_norm": 0.11788524687290192, "loss_scale": 4.0, "retries": 0, "seconds": 688.7, "max_memory_gb": 17.08, "ce": 0.004849969876886462, "rank": 0.0}
|
| 155 |
+
{"step": 155, "steps": 250, "lr": 2.1240107061879815e-05, "grad_norm": 0.8041627407073975, "loss_scale": 4.0, "retries": 0, "seconds": 692.6, "max_memory_gb": 17.08, "ce": 0.09353712194570107, "rank": 0.0}
|
| 156 |
+
{"step": 156, "steps": 250, "lr": 2.095053025597896e-05, "grad_norm": 0.6534699201583862, "loss_scale": 4.0, "retries": 0, "seconds": 696.5, "max_memory_gb": 17.08, "ce": 0.059144660976016894, "rank": 0.0}
|
| 157 |
+
{"step": 157, "steps": 250, "lr": 2.066213395457945e-05, "grad_norm": 0.3143753111362457, "loss_scale": 4.0, "retries": 0, "seconds": 700.4, "max_memory_gb": 17.08, "ce": 0.005169320437744318, "rank": 0.0}
|
| 158 |
+
{"step": 158, "steps": 250, "lr": 2.0374970139466877e-05, "grad_norm": 0.8269567489624023, "loss_scale": 4.0, "retries": 0, "seconds": 704.3, "max_memory_gb": 17.08, "ce": 0.08374992003700754, "rank": 0.0}
|
| 159 |
+
{"step": 159, "steps": 250, "lr": 2.0089090570278213e-05, "grad_norm": 0.22049228847026825, "loss_scale": 4.0, "retries": 0, "seconds": 708.1, "max_memory_gb": 17.08, "ce": 0.01014180741731252, "rank": 0.0}
|
| 160 |
+
{"step": 160, "steps": 250, "lr": 1.9804546775172457e-05, "grad_norm": 8.12972354888916, "loss_scale": 4.0, "retries": 0, "seconds": 712.1, "max_memory_gb": 17.08, "ce": 0.1917705119412858, "rank": 0.07946580648422241}
|
| 161 |
+
{"step": 161, "steps": 250, "lr": 1.9521390041542946e-05, "grad_norm": 3.255664348602295, "loss_scale": 4.0, "retries": 0, "seconds": 715.9, "max_memory_gb": 17.08, "ce": 0.1160099547023492, "rank": 0.0}
|
| 162 |
+
{"step": 162, "steps": 250, "lr": 1.923967140677313e-05, "grad_norm": 0.4950528144836426, "loss_scale": 4.0, "retries": 0, "seconds": 719.8, "max_memory_gb": 17.08, "ce": 0.009474198704992887, "rank": 0.0}
|
| 163 |
+
{"step": 163, "steps": 250, "lr": 1.8959441649037335e-05, "grad_norm": 0.4998510777950287, "loss_scale": 4.0, "retries": 0, "seconds": 723.4, "max_memory_gb": 17.08, "ce": 0.05252862794077373, "rank": 0.0}
|
| 164 |
+
{"step": 164, "steps": 250, "lr": 1.868075127814832e-05, "grad_norm": 1.2518972158432007, "loss_scale": 4.0, "retries": 0, "seconds": 727.1, "max_memory_gb": 17.08, "ce": 0.11524951871979283, "rank": 0.0}
|
| 165 |
+
{"step": 165, "steps": 250, "lr": 1.8403650526453143e-05, "grad_norm": 0.2526295781135559, "loss_scale": 4.0, "retries": 0, "seconds": 730.9, "max_memory_gb": 17.08, "ce": 0.009400092587839026, "rank": 0.0}
|
| 166 |
+
{"step": 166, "steps": 250, "lr": 1.812818933977912e-05, "grad_norm": 0.37430208921432495, "loss_scale": 4.0, "retries": 0, "seconds": 734.7, "max_memory_gb": 17.08, "ce": 0.01807590959560912, "rank": 0.0}
|
| 167 |
+
{"step": 167, "steps": 250, "lr": 1.785441736843128e-05, "grad_norm": 0.10188113898038864, "loss_scale": 4.0, "retries": 0, "seconds": 738.6, "max_memory_gb": 17.08, "ce": 0.0024518356187854806, "rank": 0.0}
|
| 168 |
+
{"step": 168, "steps": 250, "lr": 1.7582383958243314e-05, "grad_norm": 0.6407563090324402, "loss_scale": 4.0, "retries": 0, "seconds": 742.4, "max_memory_gb": 17.08, "ce": 0.06598083336939453, "rank": 0.0}
|
| 169 |
+
{"step": 169, "steps": 250, "lr": 1.7312138141683164e-05, "grad_norm": 1.9532816410064697, "loss_scale": 4.0, "retries": 0, "seconds": 746.2, "max_memory_gb": 17.08, "ce": 0.14403483684554885, "rank": 0.24935564398765564}
|
| 170 |
+
{"step": 170, "steps": 250, "lr": 1.704372862901521e-05, "grad_norm": 0.7385092377662659, "loss_scale": 4.0, "retries": 0, "seconds": 750.1, "max_memory_gb": 17.08, "ce": 0.06407798154486954, "rank": 0.0}
|
| 171 |
+
{"step": 171, "steps": 250, "lr": 1.6777203799520573e-05, "grad_norm": 1.1162549257278442, "loss_scale": 4.0, "retries": 0, "seconds": 753.9, "max_memory_gb": 17.08, "ce": 0.037442125977577234, "rank": 0.0}
|
| 172 |
+
{"step": 172, "steps": 250, "lr": 1.6512611692777008e-05, "grad_norm": 2.5368552207946777, "loss_scale": 4.0, "retries": 0, "seconds": 757.7, "max_memory_gb": 17.08, "ce": 0.1288725876429453, "rank": 0.0}
|
| 173 |
+
{"step": 173, "steps": 250, "lr": 1.6250000000000005e-05, "grad_norm": 1.340790033340454, "loss_scale": 4.0, "retries": 0, "seconds": 761.6, "max_memory_gb": 17.08, "ce": 0.12407387111761636, "rank": 0.0}
|
| 174 |
+
{"step": 174, "steps": 250, "lr": 1.598941605544678e-05, "grad_norm": 9.491345405578613, "loss_scale": 4.0, "retries": 0, "seconds": 765.4, "max_memory_gb": 17.08, "ce": 0.18543466549817822, "rank": 0.0}
|
| 175 |
+
{"step": 175, "steps": 250, "lr": 1.5730906827884506e-05, "grad_norm": 0.4562565088272095, "loss_scale": 4.0, "retries": 0, "seconds": 769.3, "max_memory_gb": 17.08, "ce": 0.023103531651486264, "rank": 0.0}
|
| 176 |
+
{"step": 176, "steps": 250, "lr": 1.5474518912124476e-05, "grad_norm": 0.6608002185821533, "loss_scale": 4.0, "retries": 0, "seconds": 773.0, "max_memory_gb": 17.08, "ce": 0.07518079336205119, "rank": 0.0}
|
| 177 |
+
{"step": 177, "steps": 250, "lr": 1.5220298520623646e-05, "grad_norm": 6.581915855407715, "loss_scale": 4.0, "retries": 0, "seconds": 777.0, "max_memory_gb": 17.08, "ce": 0.2419143907391117, "rank": 0.0}
|
| 178 |
+
{"step": 178, "steps": 250, "lr": 1.4968291475155121e-05, "grad_norm": 1.0373337268829346, "loss_scale": 4.0, "retries": 0, "seconds": 780.8, "max_memory_gb": 17.08, "ce": 0.06858392971480498, "rank": 0.0}
|
| 179 |
+
{"step": 179, "steps": 250, "lr": 1.4718543198548998e-05, "grad_norm": 0.8664576411247253, "loss_scale": 4.0, "retries": 0, "seconds": 784.6, "max_memory_gb": 17.08, "ce": 0.06389947885509173, "rank": 0.0}
|
| 180 |
+
{"step": 180, "steps": 250, "lr": 1.4471098706505227e-05, "grad_norm": 1.870497703552246, "loss_scale": 4.0, "retries": 0, "seconds": 788.5, "max_memory_gb": 17.08, "ce": 0.2034189255991805, "rank": 0.0}
|
| 181 |
+
{"step": 181, "steps": 250, "lr": 1.4226002599479743e-05, "grad_norm": 3.728292226791382, "loss_scale": 4.0, "retries": 0, "seconds": 792.3, "max_memory_gb": 17.08, "ce": 0.27761578129593545, "rank": 0.0}
|
| 182 |
+
{"step": 182, "steps": 250, "lr": 1.3983299054645471e-05, "grad_norm": 2.950726270675659, "loss_scale": 4.0, "retries": 0, "seconds": 796.1, "max_memory_gb": 17.08, "ce": 0.16081471202232933, "rank": 0.0}
|
| 183 |
+
{"step": 183, "steps": 250, "lr": 1.3743031817929724e-05, "grad_norm": 0.19506265223026276, "loss_scale": 4.0, "retries": 0, "seconds": 799.8, "max_memory_gb": 17.08, "ce": 0.0042029219557662145, "rank": 0.0}
|
| 184 |
+
{"step": 184, "steps": 250, "lr": 1.3505244196129164e-05, "grad_norm": 0.020438607782125473, "loss_scale": 4.0, "retries": 0, "seconds": 803.3, "max_memory_gb": 17.08, "ce": 0.0007334039451052377, "rank": 0.0}
|
| 185 |
+
{"step": 185, "steps": 250, "lr": 1.3269979049104014e-05, "grad_norm": 0.3019077479839325, "loss_scale": 4.0, "retries": 0, "seconds": 806.7, "max_memory_gb": 17.08, "ce": 0.00047747709231771296, "rank": 0.09780263900756836}
|
| 186 |
+
{"step": 186, "steps": 250, "lr": 1.3037278782052863e-05, "grad_norm": 0.5492751002311707, "loss_scale": 4.0, "retries": 0, "seconds": 810.2, "max_memory_gb": 17.08, "ce": 0.03522773271288315, "rank": 0.0}
|
| 187 |
+
{"step": 187, "steps": 250, "lr": 1.2807185337869343e-05, "grad_norm": 3.032564640045166, "loss_scale": 4.0, "retries": 0, "seconds": 813.8, "max_memory_gb": 17.08, "ce": 0.13108656970052834, "rank": 0.0}
|
| 188 |
+
{"step": 188, "steps": 250, "lr": 1.2579740189582115e-05, "grad_norm": 1.2983938455581665, "loss_scale": 4.0, "retries": 0, "seconds": 817.2, "max_memory_gb": 17.08, "ce": 0.10031538780640403, "rank": 0.0}
|
| 189 |
+
{"step": 189, "steps": 250, "lr": 1.2354984332879668e-05, "grad_norm": 1.5571157932281494, "loss_scale": 4.0, "retries": 0, "seconds": 820.6, "max_memory_gb": 17.08, "ce": 0.07642663174829067, "rank": 0.0}
|
| 190 |
+
{"step": 190, "steps": 250, "lr": 1.2132958278721064e-05, "grad_norm": 0.4975612461566925, "loss_scale": 4.0, "retries": 0, "seconds": 824.2, "max_memory_gb": 17.08, "ce": 0.02712161081853992, "rank": 0.12501569092273712}
|
| 191 |
+
{"step": 191, "steps": 250, "lr": 1.1913702046034016e-05, "grad_norm": 0.05463675409555435, "loss_scale": 4.0, "retries": 0, "seconds": 827.6, "max_memory_gb": 17.08, "ce": 0.0016837502730595588, "rank": 0.0}
|
| 192 |
+
{"step": 192, "steps": 250, "lr": 1.1697255154501756e-05, "grad_norm": 2.870175361633301, "loss_scale": 4.0, "retries": 0, "seconds": 831.0, "max_memory_gb": 17.08, "ce": 0.42456650708300003, "rank": 0.0}
|
| 193 |
+
{"step": 193, "steps": 250, "lr": 1.1483656617439842e-05, "grad_norm": 0.869422972202301, "loss_scale": 4.0, "retries": 0, "seconds": 834.4, "max_memory_gb": 17.08, "ce": 0.05028823073462263, "rank": 0.0}
|
| 194 |
+
{"step": 194, "steps": 250, "lr": 1.1272944934764177e-05, "grad_norm": 0.2829475700855255, "loss_scale": 4.0, "retries": 0, "seconds": 838.0, "max_memory_gb": 17.08, "ce": 0.010022616229207415, "rank": 0.12496370077133179}
|
| 195 |
+
{"step": 195, "steps": 250, "lr": 1.1065158086051684e-05, "grad_norm": 0.3686412572860718, "loss_scale": 4.0, "retries": 0, "seconds": 841.4, "max_memory_gb": 17.08, "ce": 0.005297246748796169, "rank": 0.0}
|
| 196 |
+
{"step": 196, "steps": 250, "lr": 1.086033352369467e-05, "grad_norm": 0.09547053277492523, "loss_scale": 4.0, "retries": 0, "seconds": 844.8, "max_memory_gb": 17.08, "ce": 0.0018050516582661658, "rank": 0.0}
|
| 197 |
+
{"step": 197, "steps": 250, "lr": 1.0658508166150224e-05, "grad_norm": 0.42285826802253723, "loss_scale": 4.0, "retries": 0, "seconds": 848.3, "max_memory_gb": 17.08, "ce": 0.013938153621438687, "rank": 0.0}
|
| 198 |
+
{"step": 198, "steps": 250, "lr": 1.0459718391285954e-05, "grad_norm": 0.6171932220458984, "loss_scale": 4.0, "retries": 0, "seconds": 851.8, "max_memory_gb": 17.08, "ce": 0.03350390935975156, "rank": 0.0}
|
| 199 |
+
{"step": 199, "steps": 250, "lr": 1.0264000029822999e-05, "grad_norm": 1.699407935142517, "loss_scale": 4.0, "retries": 0, "seconds": 855.2, "max_memory_gb": 17.08, "ce": 0.15192225622149635, "rank": 0.0}
|
| 200 |
+
{"step": 200, "steps": 250, "lr": 1.0071388358877775e-05, "grad_norm": 1.0192947387695312, "loss_scale": 4.0, "retries": 0, "seconds": 858.7, "max_memory_gb": 17.08, "ce": 0.12488331372787798, "rank": 0.0}
|
| 201 |
+
{"step": 201, "steps": 250, "lr": 9.881918095603501e-06, "grad_norm": 0.26503244042396545, "loss_scale": 4.0, "retries": 0, "seconds": 862.1, "max_memory_gb": 17.08, "ce": 0.008199449713629292, "rank": 0.0}
|
| 202 |
+
{"step": 202, "steps": 250, "lr": 9.695623390932612e-06, "grad_norm": 3.6362102031707764, "loss_scale": 4.0, "retries": 0, "seconds": 865.5, "max_memory_gb": 17.08, "ce": 0.17768515836178267, "rank": 0.0}
|
| 203 |
+
{"step": 203, "steps": 250, "lr": 9.51253782342123e-06, "grad_norm": 0.040635544806718826, "loss_scale": 4.0, "retries": 0, "seconds": 868.9, "max_memory_gb": 17.08, "ce": 0.0015485575304410304, "rank": 0.0}
|
| 204 |
+
{"step": 204, "steps": 250, "lr": 9.332694393196846e-06, "grad_norm": 0.8595575094223022, "loss_scale": 4.0, "retries": 0, "seconds": 872.4, "max_memory_gb": 17.08, "ce": 0.059699641085444455, "rank": 0.0}
|
| 205 |
+
{"step": 205, "steps": 250, "lr": 9.156125516010256e-06, "grad_norm": 0.13303303718566895, "loss_scale": 4.0, "retries": 0, "seconds": 875.8, "max_memory_gb": 17.08, "ce": 0.004263399398496404, "rank": 0.0}
|
| 206 |
+
{"step": 206, "steps": 250, "lr": 8.982863017392733e-06, "grad_norm": 0.7569140195846558, "loss_scale": 4.0, "retries": 0, "seconds": 879.2, "max_memory_gb": 17.08, "ce": 0.02504797518304258, "rank": 0.0}
|
| 207 |
+
{"step": 207, "steps": 250, "lr": 8.81293812691974e-06, "grad_norm": 0.49646323919296265, "loss_scale": 4.0, "retries": 0, "seconds": 882.8, "max_memory_gb": 17.08, "ce": 0.01611252756288195, "rank": 0.0}
|
| 208 |
+
{"step": 208, "steps": 250, "lr": 8.646381472581927e-06, "grad_norm": 0.640778660774231, "loss_scale": 4.0, "retries": 0, "seconds": 886.2, "max_memory_gb": 17.08, "ce": 0.028705803985758394, "rank": 0.0}
|
| 209 |
+
{"step": 209, "steps": 250, "lr": 8.483223075264622e-06, "grad_norm": 0.4318573474884033, "loss_scale": 4.0, "retries": 0, "seconds": 889.6, "max_memory_gb": 17.08, "ce": 0.013542607379804394, "rank": 0.0}
|
| 210 |
+
{"step": 210, "steps": 250, "lr": 8.323492343336747e-06, "grad_norm": 5.628416538238525, "loss_scale": 4.0, "retries": 0, "seconds": 893.1, "max_memory_gb": 17.08, "ce": 0.05100859912181477, "rank": 0.0}
|
| 211 |
+
{"step": 211, "steps": 250, "lr": 8.167218067350122e-06, "grad_norm": 0.4449547231197357, "loss_scale": 4.0, "retries": 0, "seconds": 896.6, "max_memory_gb": 17.08, "ce": 0.016200150101212785, "rank": 0.0}
|
| 212 |
+
{"step": 212, "steps": 250, "lr": 8.01442841485013e-06, "grad_norm": 2.1710238456726074, "loss_scale": 4.0, "retries": 0, "seconds": 900.0, "max_memory_gb": 17.08, "ce": 0.1909465601802367, "rank": 0.0}
|
| 213 |
+
{"step": 213, "steps": 250, "lr": 7.865150925298678e-06, "grad_norm": 2.5862553119659424, "loss_scale": 4.0, "retries": 0, "seconds": 903.6, "max_memory_gb": 17.08, "ce": 0.588958117324637, "rank": 0.0}
|
| 214 |
+
{"step": 214, "steps": 250, "lr": 7.719412505110375e-06, "grad_norm": 0.1935756951570511, "loss_scale": 4.0, "retries": 0, "seconds": 907.7, "max_memory_gb": 17.08, "ce": 0.006798575873290247, "rank": 0.0}
|
| 215 |
+
{"step": 215, "steps": 250, "lr": 7.57723942280278e-06, "grad_norm": 0.8158098459243774, "loss_scale": 4.0, "retries": 0, "seconds": 911.4, "max_memory_gb": 17.08, "ce": 0.017361248765155324, "rank": 0.0}
|
| 216 |
+
{"step": 216, "steps": 250, "lr": 7.438657304261687e-06, "grad_norm": 1.6870546340942383, "loss_scale": 4.0, "retries": 0, "seconds": 915.3, "max_memory_gb": 17.08, "ce": 0.022121690130006755, "rank": 0.0}
|
| 217 |
+
{"step": 217, "steps": 250, "lr": 7.30369112812219e-06, "grad_norm": 1.9691563844680786, "loss_scale": 4.0, "retries": 0, "seconds": 919.3, "max_memory_gb": 17.08, "ce": 0.07815290350390569, "rank": 0.12360627949237823}
|
| 218 |
+
{"step": 218, "steps": 250, "lr": 7.172365221266403e-06, "grad_norm": 8.84951114654541, "loss_scale": 4.0, "retries": 0, "seconds": 923.1, "max_memory_gb": 17.08, "ce": 0.4695682718943317, "rank": 0.0}
|
| 219 |
+
{"step": 219, "steps": 250, "lr": 7.0447032544387154e-06, "grad_norm": 0.2638499438762665, "loss_scale": 4.0, "retries": 0, "seconds": 926.9, "max_memory_gb": 17.08, "ce": 0.010855560364689154, "rank": 0.0}
|
| 220 |
+
{"step": 220, "steps": 250, "lr": 6.920728237979274e-06, "grad_norm": 2.6784279346466064, "loss_scale": 4.0, "retries": 0, "seconds": 930.9, "max_memory_gb": 17.08, "ce": 0.13839758338690444, "rank": 0.0}
|
| 221 |
+
{"step": 221, "steps": 250, "lr": 6.800462517676456e-06, "grad_norm": 3.9750099182128906, "loss_scale": 4.0, "retries": 0, "seconds": 934.7, "max_memory_gb": 17.08, "ce": 0.30654495779208446, "rank": 0.0}
|
| 222 |
+
{"step": 222, "steps": 250, "lr": 6.683927770739222e-06, "grad_norm": 0.7017527222633362, "loss_scale": 4.0, "retries": 0, "seconds": 938.5, "max_memory_gb": 17.08, "ce": 0.017471964692958863, "rank": 0.0}
|
| 223 |
+
{"step": 223, "steps": 250, "lr": 6.571145001889892e-06, "grad_norm": 4.833301544189453, "loss_scale": 4.0, "retries": 0, "seconds": 942.3, "max_memory_gb": 17.08, "ce": 0.1645362830513477, "rank": 0.0}
|
| 224 |
+
{"step": 224, "steps": 250, "lr": 6.46213453957817e-06, "grad_norm": 0.009059453383088112, "loss_scale": 4.0, "retries": 0, "seconds": 946.2, "max_memory_gb": 17.08, "ce": 0.00035265013673324574, "rank": 0.0}
|
| 225 |
+
{"step": 225, "steps": 250, "lr": 6.356916032317063e-06, "grad_norm": 1.6045725345611572, "loss_scale": 4.0, "retries": 0, "seconds": 950.1, "max_memory_gb": 17.08, "ce": 0.18105621709537445, "rank": 0.0}
|
| 226 |
+
{"step": 226, "steps": 250, "lr": 6.255508445141349e-06, "grad_norm": 1.1358994245529175, "loss_scale": 4.0, "retries": 0, "seconds": 953.8, "max_memory_gb": 17.08, "ce": 0.06828493359944332, "rank": 0.0}
|
| 227 |
+
{"step": 227, "steps": 250, "lr": 6.157930056189228e-06, "grad_norm": 0.08597422391176224, "loss_scale": 4.0, "retries": 0, "seconds": 957.8, "max_memory_gb": 17.08, "ce": 0.004897525438536832, "rank": 0.0}
|
| 228 |
+
{"step": 228, "steps": 250, "lr": 6.064198453407796e-06, "grad_norm": 1.4189664125442505, "loss_scale": 4.0, "retries": 0, "seconds": 961.6, "max_memory_gb": 17.08, "ce": 0.06643712644927291, "rank": 0.0}
|
| 229 |
+
{"step": 229, "steps": 250, "lr": 5.974330531382935e-06, "grad_norm": 2.696357250213623, "loss_scale": 4.0, "retries": 0, "seconds": 965.4, "max_memory_gb": 17.08, "ce": 0.4798386504744485, "rank": 0.0}
|
| 230 |
+
{"step": 230, "steps": 250, "lr": 5.888342488294125e-06, "grad_norm": 0.17160314321517944, "loss_scale": 4.0, "retries": 0, "seconds": 969.2, "max_memory_gb": 17.08, "ce": 0.010518901799514424, "rank": 0.0}
|
| 231 |
+
{"step": 231, "steps": 250, "lr": 5.806249822994849e-06, "grad_norm": 1.923841118812561, "loss_scale": 4.0, "retries": 0, "seconds": 973.1, "max_memory_gb": 17.08, "ce": 0.14281887980496322, "rank": 0.0}
|
| 232 |
+
{"step": 232, "steps": 250, "lr": 5.728067332218994e-06, "grad_norm": 0.2500399649143219, "loss_scale": 4.0, "retries": 0, "seconds": 976.9, "max_memory_gb": 17.08, "ce": 0.008259061640274012, "rank": 0.0}
|
| 233 |
+
{"step": 233, "steps": 250, "lr": 5.65380910791383e-06, "grad_norm": 0.4635057747364044, "loss_scale": 4.0, "retries": 0, "seconds": 980.7, "max_memory_gb": 17.08, "ce": 0.02376616532455955, "rank": 0.0}
|
| 234 |
+
{"step": 234, "steps": 250, "lr": 5.583488534700022e-06, "grad_norm": 2.2598793506622314, "loss_scale": 4.0, "retries": 0, "seconds": 984.6, "max_memory_gb": 17.08, "ce": 0.04141001332482119, "rank": 0.0}
|
| 235 |
+
{"step": 235, "steps": 250, "lr": 5.517118287459118e-06, "grad_norm": 9.102694511413574, "loss_scale": 4.0, "retries": 0, "seconds": 988.4, "max_memory_gb": 17.08, "ce": 0.47048180978072196, "rank": 0.0}
|
| 236 |
+
{"step": 236, "steps": 250, "lr": 5.454710329048978e-06, "grad_norm": 0.5594654083251953, "loss_scale": 4.0, "retries": 0, "seconds": 992.2, "max_memory_gb": 17.08, "ce": 0.05790760690206298, "rank": 0.0}
|
| 237 |
+
{"step": 237, "steps": 250, "lr": 5.3962759081475436e-06, "grad_norm": 1.3536300659179688, "loss_scale": 4.0, "retries": 0, "seconds": 996.1, "max_memory_gb": 17.08, "ce": 0.1715924365526007, "rank": 0.0}
|
| 238 |
+
{"step": 238, "steps": 250, "lr": 5.341825557225321e-06, "grad_norm": 4.235475063323975, "loss_scale": 4.0, "retries": 0, "seconds": 999.9, "max_memory_gb": 17.08, "ce": 0.3844917071442069, "rank": 0.0}
|
| 239 |
+
{"step": 239, "steps": 250, "lr": 5.291369090646961e-06, "grad_norm": 0.9675049185752869, "loss_scale": 4.0, "retries": 0, "seconds": 1003.8, "max_memory_gb": 17.08, "ce": 0.025666314252930533, "rank": 0.0}
|
| 240 |
+
{"step": 240, "steps": 250, "lr": 5.244915602902289e-06, "grad_norm": 4.847776412963867, "loss_scale": 4.0, "retries": 0, "seconds": 1007.6, "max_memory_gb": 17.08, "ce": 0.514346646850754, "rank": 0.0}
|
| 241 |
+
{"step": 241, "steps": 250, "lr": 5.20247346696706e-06, "grad_norm": 0.3249439597129822, "loss_scale": 4.0, "retries": 0, "seconds": 1011.6, "max_memory_gb": 17.08, "ce": 0.013160579197574407, "rank": 0.0}
|
| 242 |
+
{"step": 242, "steps": 250, "lr": 5.164050332793787e-06, "grad_norm": 3.1506423950195312, "loss_scale": 4.0, "retries": 0, "seconds": 1015.5, "max_memory_gb": 17.08, "ce": 0.2941715844021928, "rank": 0.12477260082960129}
|
| 243 |
+
{"step": 243, "steps": 250, "lr": 5.129653125932873e-06, "grad_norm": 2.7111213207244873, "loss_scale": 4.0, "retries": 0, "seconds": 1019.3, "max_memory_gb": 17.08, "ce": 0.17951173434812517, "rank": 0.0}
|
| 244 |
+
{"step": 244, "steps": 250, "lr": 5.099288046284329e-06, "grad_norm": 1.538849115371704, "loss_scale": 4.0, "retries": 0, "seconds": 1023.3, "max_memory_gb": 17.08, "ce": 0.05548055860617751, "rank": 0.0}
|
| 245 |
+
{"step": 245, "steps": 250, "lr": 5.072960566980277e-06, "grad_norm": 1.1655677556991577, "loss_scale": 4.0, "retries": 0, "seconds": 1027.2, "max_memory_gb": 17.08, "ce": 0.15053393656853586, "rank": 0.0}
|
| 246 |
+
{"step": 246, "steps": 250, "lr": 5.050675433398435e-06, "grad_norm": 0.6853697896003723, "loss_scale": 4.0, "retries": 0, "seconds": 1031.0, "max_memory_gb": 17.08, "ce": 0.04846272678059904, "rank": 0.0}
|
| 247 |
+
{"step": 247, "steps": 250, "lr": 5.032436662306811e-06, "grad_norm": 0.26122793555259705, "loss_scale": 4.0, "retries": 0, "seconds": 1035.0, "max_memory_gb": 17.08, "ce": 0.004803154475666815, "rank": 0.0}
|
| 248 |
+
{"step": 248, "steps": 250, "lr": 5.018247541139684e-06, "grad_norm": 0.1602485328912735, "loss_scale": 4.0, "retries": 0, "seconds": 1038.8, "max_memory_gb": 17.08, "ce": 0.0027472230276543996, "rank": 0.0}
|
| 249 |
+
{"step": 249, "steps": 250, "lr": 5.00811062740508e-06, "grad_norm": 0.357656866312027, "loss_scale": 4.0, "retries": 0, "seconds": 1042.7, "max_memory_gb": 17.08, "ce": 0.03116856122619538, "rank": 0.11820778995752335}
|
| 250 |
+
{"step": 250, "steps": 250, "lr": 5.002027748223778e-06, "grad_norm": 0.3717695474624634, "loss_scale": 4.0, "retries": 0, "seconds": 1046.5, "max_memory_gb": 17.08, "ce": 0.013440746972264606, "rank": 0.11253853142261505}
|
runs/fixed_internvl/removal/Shallow8_s73/adapter/README.md
ADDED
|
@@ -0,0 +1,207 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
base_model: /data/OPSD/assets/models/InternVL3_5-8B-HF
|
| 3 |
+
library_name: peft
|
| 4 |
+
pipeline_tag: text-generation
|
| 5 |
+
tags:
|
| 6 |
+
- base_model:adapter:/data/OPSD/assets/models/InternVL3_5-8B-HF
|
| 7 |
+
- lora
|
| 8 |
+
- transformers
|
| 9 |
+
---
|
| 10 |
+
|
| 11 |
+
# Model Card for Model ID
|
| 12 |
+
|
| 13 |
+
<!-- Provide a quick summary of what the model is/does. -->
|
| 14 |
+
|
| 15 |
+
|
| 16 |
+
|
| 17 |
+
## Model Details
|
| 18 |
+
|
| 19 |
+
### Model Description
|
| 20 |
+
|
| 21 |
+
<!-- Provide a longer summary of what this model is. -->
|
| 22 |
+
|
| 23 |
+
|
| 24 |
+
|
| 25 |
+
- **Developed by:** [More Information Needed]
|
| 26 |
+
- **Funded by [optional]:** [More Information Needed]
|
| 27 |
+
- **Shared by [optional]:** [More Information Needed]
|
| 28 |
+
- **Model type:** [More Information Needed]
|
| 29 |
+
- **Language(s) (NLP):** [More Information Needed]
|
| 30 |
+
- **License:** [More Information Needed]
|
| 31 |
+
- **Finetuned from model [optional]:** [More Information Needed]
|
| 32 |
+
|
| 33 |
+
### Model Sources [optional]
|
| 34 |
+
|
| 35 |
+
<!-- Provide the basic links for the model. -->
|
| 36 |
+
|
| 37 |
+
- **Repository:** [More Information Needed]
|
| 38 |
+
- **Paper [optional]:** [More Information Needed]
|
| 39 |
+
- **Demo [optional]:** [More Information Needed]
|
| 40 |
+
|
| 41 |
+
## Uses
|
| 42 |
+
|
| 43 |
+
<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
|
| 44 |
+
|
| 45 |
+
### Direct Use
|
| 46 |
+
|
| 47 |
+
<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
|
| 48 |
+
|
| 49 |
+
[More Information Needed]
|
| 50 |
+
|
| 51 |
+
### Downstream Use [optional]
|
| 52 |
+
|
| 53 |
+
<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
|
| 54 |
+
|
| 55 |
+
[More Information Needed]
|
| 56 |
+
|
| 57 |
+
### Out-of-Scope Use
|
| 58 |
+
|
| 59 |
+
<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
|
| 60 |
+
|
| 61 |
+
[More Information Needed]
|
| 62 |
+
|
| 63 |
+
## Bias, Risks, and Limitations
|
| 64 |
+
|
| 65 |
+
<!-- This section is meant to convey both technical and sociotechnical limitations. -->
|
| 66 |
+
|
| 67 |
+
[More Information Needed]
|
| 68 |
+
|
| 69 |
+
### Recommendations
|
| 70 |
+
|
| 71 |
+
<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
|
| 72 |
+
|
| 73 |
+
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
|
| 74 |
+
|
| 75 |
+
## How to Get Started with the Model
|
| 76 |
+
|
| 77 |
+
Use the code below to get started with the model.
|
| 78 |
+
|
| 79 |
+
[More Information Needed]
|
| 80 |
+
|
| 81 |
+
## Training Details
|
| 82 |
+
|
| 83 |
+
### Training Data
|
| 84 |
+
|
| 85 |
+
<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
|
| 86 |
+
|
| 87 |
+
[More Information Needed]
|
| 88 |
+
|
| 89 |
+
### Training Procedure
|
| 90 |
+
|
| 91 |
+
<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
|
| 92 |
+
|
| 93 |
+
#### Preprocessing [optional]
|
| 94 |
+
|
| 95 |
+
[More Information Needed]
|
| 96 |
+
|
| 97 |
+
|
| 98 |
+
#### Training Hyperparameters
|
| 99 |
+
|
| 100 |
+
- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
|
| 101 |
+
|
| 102 |
+
#### Speeds, Sizes, Times [optional]
|
| 103 |
+
|
| 104 |
+
<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
|
| 105 |
+
|
| 106 |
+
[More Information Needed]
|
| 107 |
+
|
| 108 |
+
## Evaluation
|
| 109 |
+
|
| 110 |
+
<!-- This section describes the evaluation protocols and provides the results. -->
|
| 111 |
+
|
| 112 |
+
### Testing Data, Factors & Metrics
|
| 113 |
+
|
| 114 |
+
#### Testing Data
|
| 115 |
+
|
| 116 |
+
<!-- This should link to a Dataset Card if possible. -->
|
| 117 |
+
|
| 118 |
+
[More Information Needed]
|
| 119 |
+
|
| 120 |
+
#### Factors
|
| 121 |
+
|
| 122 |
+
<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
|
| 123 |
+
|
| 124 |
+
[More Information Needed]
|
| 125 |
+
|
| 126 |
+
#### Metrics
|
| 127 |
+
|
| 128 |
+
<!-- These are the evaluation metrics being used, ideally with a description of why. -->
|
| 129 |
+
|
| 130 |
+
[More Information Needed]
|
| 131 |
+
|
| 132 |
+
### Results
|
| 133 |
+
|
| 134 |
+
[More Information Needed]
|
| 135 |
+
|
| 136 |
+
#### Summary
|
| 137 |
+
|
| 138 |
+
|
| 139 |
+
|
| 140 |
+
## Model Examination [optional]
|
| 141 |
+
|
| 142 |
+
<!-- Relevant interpretability work for the model goes here -->
|
| 143 |
+
|
| 144 |
+
[More Information Needed]
|
| 145 |
+
|
| 146 |
+
## Environmental Impact
|
| 147 |
+
|
| 148 |
+
<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
|
| 149 |
+
|
| 150 |
+
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
|
| 151 |
+
|
| 152 |
+
- **Hardware Type:** [More Information Needed]
|
| 153 |
+
- **Hours used:** [More Information Needed]
|
| 154 |
+
- **Cloud Provider:** [More Information Needed]
|
| 155 |
+
- **Compute Region:** [More Information Needed]
|
| 156 |
+
- **Carbon Emitted:** [More Information Needed]
|
| 157 |
+
|
| 158 |
+
## Technical Specifications [optional]
|
| 159 |
+
|
| 160 |
+
### Model Architecture and Objective
|
| 161 |
+
|
| 162 |
+
[More Information Needed]
|
| 163 |
+
|
| 164 |
+
### Compute Infrastructure
|
| 165 |
+
|
| 166 |
+
[More Information Needed]
|
| 167 |
+
|
| 168 |
+
#### Hardware
|
| 169 |
+
|
| 170 |
+
[More Information Needed]
|
| 171 |
+
|
| 172 |
+
#### Software
|
| 173 |
+
|
| 174 |
+
[More Information Needed]
|
| 175 |
+
|
| 176 |
+
## Citation [optional]
|
| 177 |
+
|
| 178 |
+
<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
|
| 179 |
+
|
| 180 |
+
**BibTeX:**
|
| 181 |
+
|
| 182 |
+
[More Information Needed]
|
| 183 |
+
|
| 184 |
+
**APA:**
|
| 185 |
+
|
| 186 |
+
[More Information Needed]
|
| 187 |
+
|
| 188 |
+
## Glossary [optional]
|
| 189 |
+
|
| 190 |
+
<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
|
| 191 |
+
|
| 192 |
+
[More Information Needed]
|
| 193 |
+
|
| 194 |
+
## More Information [optional]
|
| 195 |
+
|
| 196 |
+
[More Information Needed]
|
| 197 |
+
|
| 198 |
+
## Model Card Authors [optional]
|
| 199 |
+
|
| 200 |
+
[More Information Needed]
|
| 201 |
+
|
| 202 |
+
## Model Card Contact
|
| 203 |
+
|
| 204 |
+
[More Information Needed]
|
| 205 |
+
### Framework versions
|
| 206 |
+
|
| 207 |
+
- PEFT 0.20.0
|
runs/fixed_internvl/removal/Shallow8_s73/adapter/adapter_config.json
ADDED
|
@@ -0,0 +1,213 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"alora_invocation_tokens": null,
|
| 3 |
+
"alpha_pattern": {
|
| 4 |
+
"model.language_model.layers.0.mlp.down_proj": 144,
|
| 5 |
+
"model.language_model.layers.0.mlp.gate_proj": 144,
|
| 6 |
+
"model.language_model.layers.0.mlp.up_proj": 144,
|
| 7 |
+
"model.language_model.layers.0.self_attn.k_proj": 144,
|
| 8 |
+
"model.language_model.layers.0.self_attn.o_proj": 144,
|
| 9 |
+
"model.language_model.layers.0.self_attn.q_proj": 144,
|
| 10 |
+
"model.language_model.layers.0.self_attn.v_proj": 144,
|
| 11 |
+
"model.language_model.layers.1.mlp.down_proj": 144,
|
| 12 |
+
"model.language_model.layers.1.mlp.gate_proj": 144,
|
| 13 |
+
"model.language_model.layers.1.mlp.up_proj": 144,
|
| 14 |
+
"model.language_model.layers.1.self_attn.k_proj": 144,
|
| 15 |
+
"model.language_model.layers.1.self_attn.o_proj": 144,
|
| 16 |
+
"model.language_model.layers.1.self_attn.q_proj": 144,
|
| 17 |
+
"model.language_model.layers.1.self_attn.v_proj": 144,
|
| 18 |
+
"model.language_model.layers.2.mlp.down_proj": 144,
|
| 19 |
+
"model.language_model.layers.2.mlp.gate_proj": 144,
|
| 20 |
+
"model.language_model.layers.2.mlp.up_proj": 144,
|
| 21 |
+
"model.language_model.layers.2.self_attn.k_proj": 144,
|
| 22 |
+
"model.language_model.layers.2.self_attn.o_proj": 144,
|
| 23 |
+
"model.language_model.layers.2.self_attn.q_proj": 144,
|
| 24 |
+
"model.language_model.layers.2.self_attn.v_proj": 144,
|
| 25 |
+
"model.language_model.layers.3.mlp.down_proj": 144,
|
| 26 |
+
"model.language_model.layers.3.mlp.gate_proj": 144,
|
| 27 |
+
"model.language_model.layers.3.mlp.up_proj": 144,
|
| 28 |
+
"model.language_model.layers.3.self_attn.k_proj": 144,
|
| 29 |
+
"model.language_model.layers.3.self_attn.o_proj": 144,
|
| 30 |
+
"model.language_model.layers.3.self_attn.q_proj": 144,
|
| 31 |
+
"model.language_model.layers.3.self_attn.v_proj": 144,
|
| 32 |
+
"model.language_model.layers.4.mlp.down_proj": 144,
|
| 33 |
+
"model.language_model.layers.4.mlp.gate_proj": 144,
|
| 34 |
+
"model.language_model.layers.4.mlp.up_proj": 144,
|
| 35 |
+
"model.language_model.layers.4.self_attn.k_proj": 144,
|
| 36 |
+
"model.language_model.layers.4.self_attn.o_proj": 144,
|
| 37 |
+
"model.language_model.layers.4.self_attn.q_proj": 144,
|
| 38 |
+
"model.language_model.layers.4.self_attn.v_proj": 144,
|
| 39 |
+
"model.language_model.layers.5.mlp.down_proj": 144,
|
| 40 |
+
"model.language_model.layers.5.mlp.gate_proj": 144,
|
| 41 |
+
"model.language_model.layers.5.mlp.up_proj": 144,
|
| 42 |
+
"model.language_model.layers.5.self_attn.k_proj": 144,
|
| 43 |
+
"model.language_model.layers.5.self_attn.o_proj": 144,
|
| 44 |
+
"model.language_model.layers.5.self_attn.q_proj": 144,
|
| 45 |
+
"model.language_model.layers.5.self_attn.v_proj": 144,
|
| 46 |
+
"model.language_model.layers.6.mlp.down_proj": 144,
|
| 47 |
+
"model.language_model.layers.6.mlp.gate_proj": 144,
|
| 48 |
+
"model.language_model.layers.6.mlp.up_proj": 144,
|
| 49 |
+
"model.language_model.layers.6.self_attn.k_proj": 144,
|
| 50 |
+
"model.language_model.layers.6.self_attn.o_proj": 144,
|
| 51 |
+
"model.language_model.layers.6.self_attn.q_proj": 144,
|
| 52 |
+
"model.language_model.layers.6.self_attn.v_proj": 144,
|
| 53 |
+
"model.language_model.layers.7.mlp.down_proj": 144,
|
| 54 |
+
"model.language_model.layers.7.mlp.gate_proj": 144,
|
| 55 |
+
"model.language_model.layers.7.mlp.up_proj": 144,
|
| 56 |
+
"model.language_model.layers.7.self_attn.k_proj": 144,
|
| 57 |
+
"model.language_model.layers.7.self_attn.o_proj": 144,
|
| 58 |
+
"model.language_model.layers.7.self_attn.q_proj": 144,
|
| 59 |
+
"model.language_model.layers.7.self_attn.v_proj": 144
|
| 60 |
+
},
|
| 61 |
+
"arrow_config": null,
|
| 62 |
+
"auto_mapping": null,
|
| 63 |
+
"base_model_name_or_path": "/data/OPSD/assets/models/InternVL3_5-8B-HF",
|
| 64 |
+
"bias": "none",
|
| 65 |
+
"corda_config": null,
|
| 66 |
+
"ensure_weight_tying": false,
|
| 67 |
+
"eva_config": null,
|
| 68 |
+
"exclude_modules": null,
|
| 69 |
+
"fan_in_fan_out": false,
|
| 70 |
+
"inference_mode": true,
|
| 71 |
+
"init_lora_weights": true,
|
| 72 |
+
"layer_replication": null,
|
| 73 |
+
"layers_pattern": null,
|
| 74 |
+
"layers_to_transform": null,
|
| 75 |
+
"loftq_config": {},
|
| 76 |
+
"lora_alpha": 32,
|
| 77 |
+
"lora_bias": false,
|
| 78 |
+
"lora_dropout": 0.05,
|
| 79 |
+
"lora_ga_config": null,
|
| 80 |
+
"megatron_config": null,
|
| 81 |
+
"megatron_core": "megatron.core",
|
| 82 |
+
"modules_to_save": null,
|
| 83 |
+
"monteclora_config": null,
|
| 84 |
+
"peft_type": "LORA",
|
| 85 |
+
"peft_version": "0.20.0",
|
| 86 |
+
"qalora_group_size": 16,
|
| 87 |
+
"r": 16,
|
| 88 |
+
"rank_pattern": {
|
| 89 |
+
"model.language_model.layers.0.mlp.down_proj": 72,
|
| 90 |
+
"model.language_model.layers.0.mlp.gate_proj": 72,
|
| 91 |
+
"model.language_model.layers.0.mlp.up_proj": 72,
|
| 92 |
+
"model.language_model.layers.0.self_attn.k_proj": 72,
|
| 93 |
+
"model.language_model.layers.0.self_attn.o_proj": 72,
|
| 94 |
+
"model.language_model.layers.0.self_attn.q_proj": 72,
|
| 95 |
+
"model.language_model.layers.0.self_attn.v_proj": 72,
|
| 96 |
+
"model.language_model.layers.1.mlp.down_proj": 72,
|
| 97 |
+
"model.language_model.layers.1.mlp.gate_proj": 72,
|
| 98 |
+
"model.language_model.layers.1.mlp.up_proj": 72,
|
| 99 |
+
"model.language_model.layers.1.self_attn.k_proj": 72,
|
| 100 |
+
"model.language_model.layers.1.self_attn.o_proj": 72,
|
| 101 |
+
"model.language_model.layers.1.self_attn.q_proj": 72,
|
| 102 |
+
"model.language_model.layers.1.self_attn.v_proj": 72,
|
| 103 |
+
"model.language_model.layers.2.mlp.down_proj": 72,
|
| 104 |
+
"model.language_model.layers.2.mlp.gate_proj": 72,
|
| 105 |
+
"model.language_model.layers.2.mlp.up_proj": 72,
|
| 106 |
+
"model.language_model.layers.2.self_attn.k_proj": 72,
|
| 107 |
+
"model.language_model.layers.2.self_attn.o_proj": 72,
|
| 108 |
+
"model.language_model.layers.2.self_attn.q_proj": 72,
|
| 109 |
+
"model.language_model.layers.2.self_attn.v_proj": 72,
|
| 110 |
+
"model.language_model.layers.3.mlp.down_proj": 72,
|
| 111 |
+
"model.language_model.layers.3.mlp.gate_proj": 72,
|
| 112 |
+
"model.language_model.layers.3.mlp.up_proj": 72,
|
| 113 |
+
"model.language_model.layers.3.self_attn.k_proj": 72,
|
| 114 |
+
"model.language_model.layers.3.self_attn.o_proj": 72,
|
| 115 |
+
"model.language_model.layers.3.self_attn.q_proj": 72,
|
| 116 |
+
"model.language_model.layers.3.self_attn.v_proj": 72,
|
| 117 |
+
"model.language_model.layers.4.mlp.down_proj": 72,
|
| 118 |
+
"model.language_model.layers.4.mlp.gate_proj": 72,
|
| 119 |
+
"model.language_model.layers.4.mlp.up_proj": 72,
|
| 120 |
+
"model.language_model.layers.4.self_attn.k_proj": 72,
|
| 121 |
+
"model.language_model.layers.4.self_attn.o_proj": 72,
|
| 122 |
+
"model.language_model.layers.4.self_attn.q_proj": 72,
|
| 123 |
+
"model.language_model.layers.4.self_attn.v_proj": 72,
|
| 124 |
+
"model.language_model.layers.5.mlp.down_proj": 72,
|
| 125 |
+
"model.language_model.layers.5.mlp.gate_proj": 72,
|
| 126 |
+
"model.language_model.layers.5.mlp.up_proj": 72,
|
| 127 |
+
"model.language_model.layers.5.self_attn.k_proj": 72,
|
| 128 |
+
"model.language_model.layers.5.self_attn.o_proj": 72,
|
| 129 |
+
"model.language_model.layers.5.self_attn.q_proj": 72,
|
| 130 |
+
"model.language_model.layers.5.self_attn.v_proj": 72,
|
| 131 |
+
"model.language_model.layers.6.mlp.down_proj": 72,
|
| 132 |
+
"model.language_model.layers.6.mlp.gate_proj": 72,
|
| 133 |
+
"model.language_model.layers.6.mlp.up_proj": 72,
|
| 134 |
+
"model.language_model.layers.6.self_attn.k_proj": 72,
|
| 135 |
+
"model.language_model.layers.6.self_attn.o_proj": 72,
|
| 136 |
+
"model.language_model.layers.6.self_attn.q_proj": 72,
|
| 137 |
+
"model.language_model.layers.6.self_attn.v_proj": 72,
|
| 138 |
+
"model.language_model.layers.7.mlp.down_proj": 72,
|
| 139 |
+
"model.language_model.layers.7.mlp.gate_proj": 72,
|
| 140 |
+
"model.language_model.layers.7.mlp.up_proj": 72,
|
| 141 |
+
"model.language_model.layers.7.self_attn.k_proj": 72,
|
| 142 |
+
"model.language_model.layers.7.self_attn.o_proj": 72,
|
| 143 |
+
"model.language_model.layers.7.self_attn.q_proj": 72,
|
| 144 |
+
"model.language_model.layers.7.self_attn.v_proj": 72
|
| 145 |
+
},
|
| 146 |
+
"revision": null,
|
| 147 |
+
"target_modules": [
|
| 148 |
+
"model.language_model.layers.0.self_attn.q_proj",
|
| 149 |
+
"model.language_model.layers.6.mlp.up_proj",
|
| 150 |
+
"model.language_model.layers.5.mlp.down_proj",
|
| 151 |
+
"model.language_model.layers.4.self_attn.o_proj",
|
| 152 |
+
"model.language_model.layers.1.mlp.gate_proj",
|
| 153 |
+
"model.language_model.layers.4.mlp.gate_proj",
|
| 154 |
+
"model.language_model.layers.6.mlp.gate_proj",
|
| 155 |
+
"model.language_model.layers.1.self_attn.v_proj",
|
| 156 |
+
"model.language_model.layers.6.self_attn.v_proj",
|
| 157 |
+
"model.language_model.layers.1.self_attn.k_proj",
|
| 158 |
+
"model.language_model.layers.6.self_attn.o_proj",
|
| 159 |
+
"model.language_model.layers.6.mlp.down_proj",
|
| 160 |
+
"model.language_model.layers.2.self_attn.k_proj",
|
| 161 |
+
"model.language_model.layers.7.self_attn.k_proj",
|
| 162 |
+
"model.language_model.layers.0.mlp.down_proj",
|
| 163 |
+
"model.language_model.layers.5.self_attn.k_proj",
|
| 164 |
+
"model.language_model.layers.3.self_attn.v_proj",
|
| 165 |
+
"model.language_model.layers.7.mlp.up_proj",
|
| 166 |
+
"model.language_model.layers.3.self_attn.k_proj",
|
| 167 |
+
"model.language_model.layers.5.self_attn.o_proj",
|
| 168 |
+
"model.language_model.layers.7.mlp.down_proj",
|
| 169 |
+
"model.language_model.layers.2.self_attn.o_proj",
|
| 170 |
+
"model.language_model.layers.5.mlp.up_proj",
|
| 171 |
+
"model.language_model.layers.4.self_attn.q_proj",
|
| 172 |
+
"model.language_model.layers.2.self_attn.q_proj",
|
| 173 |
+
"model.language_model.layers.2.mlp.up_proj",
|
| 174 |
+
"model.language_model.layers.5.self_attn.q_proj",
|
| 175 |
+
"model.language_model.layers.6.self_attn.k_proj",
|
| 176 |
+
"model.language_model.layers.4.mlp.up_proj",
|
| 177 |
+
"model.language_model.layers.2.self_attn.v_proj",
|
| 178 |
+
"model.language_model.layers.3.self_attn.q_proj",
|
| 179 |
+
"model.language_model.layers.3.mlp.down_proj",
|
| 180 |
+
"model.language_model.layers.0.mlp.gate_proj",
|
| 181 |
+
"model.language_model.layers.4.self_attn.v_proj",
|
| 182 |
+
"model.language_model.layers.7.self_attn.q_proj",
|
| 183 |
+
"model.language_model.layers.3.mlp.up_proj",
|
| 184 |
+
"model.language_model.layers.7.mlp.gate_proj",
|
| 185 |
+
"model.language_model.layers.7.self_attn.o_proj",
|
| 186 |
+
"model.language_model.layers.7.self_attn.v_proj",
|
| 187 |
+
"model.language_model.layers.5.self_attn.v_proj",
|
| 188 |
+
"model.language_model.layers.6.self_attn.q_proj",
|
| 189 |
+
"model.language_model.layers.2.mlp.gate_proj",
|
| 190 |
+
"model.language_model.layers.3.self_attn.o_proj",
|
| 191 |
+
"model.language_model.layers.0.self_attn.o_proj",
|
| 192 |
+
"model.language_model.layers.1.mlp.down_proj",
|
| 193 |
+
"model.language_model.layers.3.mlp.gate_proj",
|
| 194 |
+
"model.language_model.layers.1.mlp.up_proj",
|
| 195 |
+
"model.language_model.layers.4.self_attn.k_proj",
|
| 196 |
+
"model.language_model.layers.0.self_attn.k_proj",
|
| 197 |
+
"model.language_model.layers.0.self_attn.v_proj",
|
| 198 |
+
"model.language_model.layers.1.self_attn.q_proj",
|
| 199 |
+
"model.language_model.layers.4.mlp.down_proj",
|
| 200 |
+
"model.language_model.layers.5.mlp.gate_proj",
|
| 201 |
+
"model.language_model.layers.1.self_attn.o_proj",
|
| 202 |
+
"model.language_model.layers.0.mlp.up_proj",
|
| 203 |
+
"model.language_model.layers.2.mlp.down_proj"
|
| 204 |
+
],
|
| 205 |
+
"target_parameters": null,
|
| 206 |
+
"task_type": "CAUSAL_LM",
|
| 207 |
+
"trainable_token_indices": null,
|
| 208 |
+
"use_bdlora": null,
|
| 209 |
+
"use_dora": false,
|
| 210 |
+
"use_qalora": false,
|
| 211 |
+
"use_rslora": false,
|
| 212 |
+
"velora_config": null
|
| 213 |
+
}
|
runs/fixed_internvl/removal/Shallow8_s73/config.json
ADDED
|
@@ -0,0 +1,43 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"behavior": "removal",
|
| 3 |
+
"seed": 73,
|
| 4 |
+
"model": "/data/OPSD/assets/models/InternVL3_5-8B-HF",
|
| 5 |
+
"backbone": "internvl",
|
| 6 |
+
"train": "/data/OPSD/assets/v3/removal.jsonl",
|
| 7 |
+
"items": 1000,
|
| 8 |
+
"steps": 250,
|
| 9 |
+
"epochs": 1,
|
| 10 |
+
"steps_per_epoch": 250,
|
| 11 |
+
"world_size": 1,
|
| 12 |
+
"lr": 5e-05,
|
| 13 |
+
"scheduler": "cosine",
|
| 14 |
+
"warmup_steps": 16,
|
| 15 |
+
"final_fraction": 0.1,
|
| 16 |
+
"clip": 1.0,
|
| 17 |
+
"initial_loss_scale": 4.0,
|
| 18 |
+
"arm": {
|
| 19 |
+
"arm": "Shallow8",
|
| 20 |
+
"layers": [
|
| 21 |
+
0,
|
| 22 |
+
1,
|
| 23 |
+
2,
|
| 24 |
+
3,
|
| 25 |
+
4,
|
| 26 |
+
5,
|
| 27 |
+
6,
|
| 28 |
+
7
|
| 29 |
+
],
|
| 30 |
+
"ranks": {
|
| 31 |
+
"0": 72,
|
| 32 |
+
"1": 72,
|
| 33 |
+
"2": 72,
|
| 34 |
+
"3": 72,
|
| 35 |
+
"4": 72,
|
| 36 |
+
"5": 72,
|
| 37 |
+
"6": 72,
|
| 38 |
+
"7": 72
|
| 39 |
+
},
|
| 40 |
+
"parameters": 43646976,
|
| 41 |
+
"budget": 43646976
|
| 42 |
+
}
|
| 43 |
+
}
|
runs/fixed_internvl/removal/Shallow8_s73/steps.jsonl
ADDED
|
@@ -0,0 +1,250 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{"step": 1, "steps": 250, "lr": 3.125e-06, "grad_norm": 12.212173461914062, "loss_scale": 4.0, "retries": 0, "seconds": 4.3, "max_memory_gb": 16.79, "ce": 0.8941777415893739, "rank": 0.0}
|
| 2 |
+
{"step": 2, "steps": 250, "lr": 6.25e-06, "grad_norm": 19.53752899169922, "loss_scale": 4.0, "retries": 0, "seconds": 7.8, "max_memory_gb": 17.07, "ce": 1.642250346019864, "rank": 0.003445565700531006}
|
| 3 |
+
{"step": 3, "steps": 250, "lr": 9.375000000000001e-06, "grad_norm": 10.52991771697998, "loss_scale": 4.0, "retries": 0, "seconds": 11.1, "max_memory_gb": 17.07, "ce": 0.6634746006893693, "rank": 0.08412738144397736}
|
| 4 |
+
{"step": 4, "steps": 250, "lr": 1.25e-05, "grad_norm": 8.62264633178711, "loss_scale": 4.0, "retries": 0, "seconds": 14.5, "max_memory_gb": 17.07, "ce": 0.6422890117100906, "rank": 0.0}
|
| 5 |
+
{"step": 5, "steps": 250, "lr": 1.5625e-05, "grad_norm": 3.8363964557647705, "loss_scale": 4.0, "retries": 0, "seconds": 17.9, "max_memory_gb": 17.08, "ce": 0.4966372136332211, "rank": 0.0}
|
| 6 |
+
{"step": 6, "steps": 250, "lr": 1.8750000000000002e-05, "grad_norm": 2.3741652965545654, "loss_scale": 4.0, "retries": 0, "seconds": 21.3, "max_memory_gb": 17.08, "ce": 0.49375065555796027, "rank": 0.0}
|
| 7 |
+
{"step": 7, "steps": 250, "lr": 2.1875e-05, "grad_norm": 0.29729408025741577, "loss_scale": 4.0, "retries": 0, "seconds": 24.6, "max_memory_gb": 17.08, "ce": 0.13777689109974744, "rank": 0.0}
|
| 8 |
+
{"step": 8, "steps": 250, "lr": 2.5e-05, "grad_norm": 1.612407922744751, "loss_scale": 4.0, "retries": 0, "seconds": 28.0, "max_memory_gb": 17.08, "ce": 0.647716429495631, "rank": 0.13283346593379974}
|
| 9 |
+
{"step": 9, "steps": 250, "lr": 2.8125000000000003e-05, "grad_norm": 1.9247379302978516, "loss_scale": 4.0, "retries": 0, "seconds": 31.4, "max_memory_gb": 17.08, "ce": 0.39326800405979156, "rank": 0.0}
|
| 10 |
+
{"step": 10, "steps": 250, "lr": 3.125e-05, "grad_norm": 1.8209309577941895, "loss_scale": 4.0, "retries": 0, "seconds": 34.7, "max_memory_gb": 17.08, "ce": 0.34533525178267155, "rank": 0.008915886282920837}
|
| 11 |
+
{"step": 11, "steps": 250, "lr": 3.4375e-05, "grad_norm": 1.0176275968551636, "loss_scale": 4.0, "retries": 0, "seconds": 38.1, "max_memory_gb": 17.08, "ce": 0.34197620226768777, "rank": 0.0}
|
| 12 |
+
{"step": 12, "steps": 250, "lr": 3.7500000000000003e-05, "grad_norm": 1.9738980531692505, "loss_scale": 4.0, "retries": 0, "seconds": 41.5, "max_memory_gb": 17.08, "ce": 0.20296036947183893, "rank": 0.0}
|
| 13 |
+
{"step": 13, "steps": 250, "lr": 4.0625000000000005e-05, "grad_norm": 1.51876699924469, "loss_scale": 4.0, "retries": 0, "seconds": 44.9, "max_memory_gb": 17.08, "ce": 0.3776860685647989, "rank": 0.0}
|
| 14 |
+
{"step": 14, "steps": 250, "lr": 4.375e-05, "grad_norm": 1.0923436880111694, "loss_scale": 4.0, "retries": 0, "seconds": 48.2, "max_memory_gb": 17.08, "ce": 0.2788307682294544, "rank": 0.0}
|
| 15 |
+
{"step": 15, "steps": 250, "lr": 4.6875e-05, "grad_norm": 5.872822284698486, "loss_scale": 4.0, "retries": 0, "seconds": 51.7, "max_memory_gb": 17.08, "ce": 0.2003311191801913, "rank": 0.0}
|
| 16 |
+
{"step": 16, "steps": 250, "lr": 5e-05, "grad_norm": 3.633631944656372, "loss_scale": 4.0, "retries": 0, "seconds": 55.0, "max_memory_gb": 17.08, "ce": 0.9424186146643478, "rank": 0.0}
|
| 17 |
+
{"step": 17, "steps": 250, "lr": 5e-05, "grad_norm": 2.260890245437622, "loss_scale": 4.0, "retries": 0, "seconds": 58.3, "max_memory_gb": 17.08, "ce": 0.27214608987560496, "rank": 0.0}
|
| 18 |
+
{"step": 18, "steps": 250, "lr": 4.999797225177623e-05, "grad_norm": 0.5739230513572693, "loss_scale": 4.0, "retries": 0, "seconds": 61.7, "max_memory_gb": 17.08, "ce": 0.20844487567956094, "rank": 0.12468159943819046}
|
| 19 |
+
{"step": 19, "steps": 250, "lr": 4.999188937259492e-05, "grad_norm": 4.116664886474609, "loss_scale": 4.0, "retries": 0, "seconds": 65.7, "max_memory_gb": 17.08, "ce": 0.3699001069799124, "rank": 0.0}
|
| 20 |
+
{"step": 20, "steps": 250, "lr": 4.998175245886032e-05, "grad_norm": 0.6310007572174072, "loss_scale": 4.0, "retries": 0, "seconds": 69.8, "max_memory_gb": 17.08, "ce": 0.3799026360211428, "rank": 0.0}
|
| 21 |
+
{"step": 21, "steps": 250, "lr": 4.996756333769319e-05, "grad_norm": 0.5607049465179443, "loss_scale": 4.0, "retries": 0, "seconds": 73.9, "max_memory_gb": 17.08, "ce": 0.2013262793479953, "rank": 0.0}
|
| 22 |
+
{"step": 22, "steps": 250, "lr": 4.994932456660157e-05, "grad_norm": 0.7286702990531921, "loss_scale": 4.0, "retries": 0, "seconds": 78.0, "max_memory_gb": 17.08, "ce": 0.3996586929424666, "rank": 0.0}
|
| 23 |
+
{"step": 23, "steps": 250, "lr": 4.992703943301973e-05, "grad_norm": 1.559095859527588, "loss_scale": 4.0, "retries": 0, "seconds": 82.1, "max_memory_gb": 17.08, "ce": 0.2454257004428655, "rank": 0.0}
|
| 24 |
+
{"step": 24, "steps": 250, "lr": 4.9900711953715675e-05, "grad_norm": 0.8275643587112427, "loss_scale": 4.0, "retries": 0, "seconds": 86.2, "max_memory_gb": 17.08, "ce": 0.3308222236810252, "rank": 0.0}
|
| 25 |
+
{"step": 25, "steps": 250, "lr": 4.987034687406713e-05, "grad_norm": 3.1600542068481445, "loss_scale": 4.0, "retries": 0, "seconds": 90.4, "max_memory_gb": 17.08, "ce": 0.17487401605467312, "rank": 0.03356245160102844}
|
| 26 |
+
{"step": 26, "steps": 250, "lr": 4.9835949667206215e-05, "grad_norm": 2.6371047496795654, "loss_scale": 4.0, "retries": 0, "seconds": 94.4, "max_memory_gb": 17.08, "ce": 0.35587091703200713, "rank": 0.0}
|
| 27 |
+
{"step": 27, "steps": 250, "lr": 4.9797526533032945e-05, "grad_norm": 5.959108352661133, "loss_scale": 4.0, "retries": 0, "seconds": 98.5, "max_memory_gb": 17.08, "ce": 0.38627583906054497, "rank": 0.031139783561229706}
|
| 28 |
+
{"step": 28, "steps": 250, "lr": 4.9755084397097716e-05, "grad_norm": 0.6927530765533447, "loss_scale": 4.0, "retries": 0, "seconds": 102.6, "max_memory_gb": 17.08, "ce": 0.21035034142551012, "rank": 0.0}
|
| 29 |
+
{"step": 29, "steps": 250, "lr": 4.970863090935304e-05, "grad_norm": 0.4838295876979828, "loss_scale": 4.0, "retries": 0, "seconds": 106.8, "max_memory_gb": 17.08, "ce": 0.14110952388728037, "rank": 0.0}
|
| 30 |
+
{"step": 30, "steps": 250, "lr": 4.965817444277468e-05, "grad_norm": 0.805014967918396, "loss_scale": 4.0, "retries": 0, "seconds": 110.8, "max_memory_gb": 17.08, "ce": 0.1757476640632376, "rank": 0.0}
|
| 31 |
+
{"step": 31, "steps": 250, "lr": 4.9603724091852456e-05, "grad_norm": 0.8756745457649231, "loss_scale": 4.0, "retries": 0, "seconds": 114.9, "max_memory_gb": 17.08, "ce": 0.18996822120971046, "rank": 0.0}
|
| 32 |
+
{"step": 32, "steps": 250, "lr": 4.9545289670951025e-05, "grad_norm": 2.2644898891448975, "loss_scale": 4.0, "retries": 0, "seconds": 119.0, "max_memory_gb": 17.08, "ce": 0.2992037629446713, "rank": 0.12528397142887115}
|
| 33 |
+
{"step": 33, "steps": 250, "lr": 4.948288171254089e-05, "grad_norm": 0.733788013458252, "loss_scale": 4.0, "retries": 0, "seconds": 123.1, "max_memory_gb": 17.08, "ce": 0.11121909788562334, "rank": 0.12515264749526978}
|
| 34 |
+
{"step": 34, "steps": 250, "lr": 4.941651146529998e-05, "grad_norm": 1.142496109008789, "loss_scale": 4.0, "retries": 0, "seconds": 127.1, "max_memory_gb": 17.08, "ce": 0.1731955801278673, "rank": 0.0}
|
| 35 |
+
{"step": 35, "steps": 250, "lr": 4.9346190892086174e-05, "grad_norm": 2.4748051166534424, "loss_scale": 4.0, "retries": 0, "seconds": 131.2, "max_memory_gb": 17.08, "ce": 0.1069610689592082, "rank": 0.0}
|
| 36 |
+
{"step": 36, "steps": 250, "lr": 4.927193266778101e-05, "grad_norm": 2.172100305557251, "loss_scale": 4.0, "retries": 0, "seconds": 135.3, "max_memory_gb": 17.08, "ce": 0.23898459743941203, "rank": 0.12144345045089722}
|
| 37 |
+
{"step": 37, "steps": 250, "lr": 4.919375017700515e-05, "grad_norm": 1.378049373626709, "loss_scale": 4.0, "retries": 0, "seconds": 139.5, "max_memory_gb": 17.08, "ce": 0.131652009265963, "rank": 0.08215253055095673}
|
| 38 |
+
{"step": 38, "steps": 250, "lr": 4.911165751170588e-05, "grad_norm": 0.8323381543159485, "loss_scale": 4.0, "retries": 0, "seconds": 143.9, "max_memory_gb": 17.08, "ce": 0.044880870753559066, "rank": 0.0}
|
| 39 |
+
{"step": 39, "steps": 250, "lr": 4.902566946861708e-05, "grad_norm": 9.43325138092041, "loss_scale": 4.0, "retries": 0, "seconds": 148.4, "max_memory_gb": 17.08, "ce": 0.2521259344018745, "rank": 0.0}
|
| 40 |
+
{"step": 40, "steps": 250, "lr": 4.89358015465922e-05, "grad_norm": 0.7776228785514832, "loss_scale": 4.0, "retries": 0, "seconds": 152.8, "max_memory_gb": 17.08, "ce": 0.0518645882656017, "rank": 0.0}
|
| 41 |
+
{"step": 41, "steps": 250, "lr": 4.884206994381078e-05, "grad_norm": 1.1955912113189697, "loss_scale": 4.0, "retries": 0, "seconds": 157.1, "max_memory_gb": 17.08, "ce": 0.024748651614572736, "rank": 0.09153006970882416}
|
| 42 |
+
{"step": 42, "steps": 250, "lr": 4.874449155485866e-05, "grad_norm": 7.234792232513428, "loss_scale": 4.0, "retries": 0, "seconds": 161.3, "max_memory_gb": 17.08, "ce": 0.30833539945422217, "rank": 0.0}
|
| 43 |
+
{"step": 43, "steps": 250, "lr": 4.864308396768294e-05, "grad_norm": 21.85984992980957, "loss_scale": 4.0, "retries": 0, "seconds": 165.4, "max_memory_gb": 17.08, "ce": 0.4614542764320504, "rank": 0.0}
|
| 44 |
+
{"step": 44, "steps": 250, "lr": 4.853786546042184e-05, "grad_norm": 0.9638859033584595, "loss_scale": 4.0, "retries": 0, "seconds": 169.5, "max_memory_gb": 17.08, "ce": 0.018139877504950164, "rank": 0.0}
|
| 45 |
+
{"step": 45, "steps": 250, "lr": 4.842885499811012e-05, "grad_norm": 1.6796702146530151, "loss_scale": 4.0, "retries": 0, "seconds": 173.5, "max_memory_gb": 17.08, "ce": 0.11873665680468548, "rank": 0.0}
|
| 46 |
+
{"step": 46, "steps": 250, "lr": 4.831607222926078e-05, "grad_norm": 2.5320792198181152, "loss_scale": 4.0, "retries": 0, "seconds": 177.7, "max_memory_gb": 17.08, "ce": 0.3480260459184592, "rank": 0.0}
|
| 47 |
+
{"step": 47, "steps": 250, "lr": 4.8199537482323545e-05, "grad_norm": 0.20245066285133362, "loss_scale": 4.0, "retries": 0, "seconds": 181.7, "max_memory_gb": 17.08, "ce": 0.006980679885884911, "rank": 0.0}
|
| 48 |
+
{"step": 48, "steps": 250, "lr": 4.807927176202073e-05, "grad_norm": 1.9306379556655884, "loss_scale": 4.0, "retries": 0, "seconds": 185.7, "max_memory_gb": 17.08, "ce": 0.06946594668643513, "rank": 0.0}
|
| 49 |
+
{"step": 49, "steps": 250, "lr": 4.795529674556129e-05, "grad_norm": 0.9725919365882874, "loss_scale": 4.0, "retries": 0, "seconds": 189.9, "max_memory_gb": 17.08, "ce": 0.017773063997083227, "rank": 0.1260581612586975}
|
| 50 |
+
{"step": 50, "steps": 250, "lr": 4.782763477873361e-05, "grad_norm": 0.3409208357334137, "loss_scale": 4.0, "retries": 0, "seconds": 193.9, "max_memory_gb": 17.08, "ce": 0.006044488880775134, "rank": 0.0}
|
| 51 |
+
{"step": 51, "steps": 250, "lr": 4.769630887187782e-05, "grad_norm": 1.3971489667892456, "loss_scale": 4.0, "retries": 0, "seconds": 198.0, "max_memory_gb": 17.08, "ce": 0.053697804212390565, "rank": 0.0}
|
| 52 |
+
{"step": 52, "steps": 250, "lr": 4.756134269573832e-05, "grad_norm": 8.085042953491211, "loss_scale": 4.0, "retries": 0, "seconds": 202.1, "max_memory_gb": 17.08, "ce": 0.3521267895048368, "rank": 0.0}
|
| 53 |
+
{"step": 53, "steps": 250, "lr": 4.7422760577197226e-05, "grad_norm": 4.7513957023620605, "loss_scale": 4.0, "retries": 0, "seconds": 206.1, "max_memory_gb": 17.08, "ce": 0.44984108326503147, "rank": 0.0}
|
| 54 |
+
{"step": 54, "steps": 250, "lr": 4.728058749488963e-05, "grad_norm": 2.0938127040863037, "loss_scale": 4.0, "retries": 0, "seconds": 210.1, "max_memory_gb": 17.08, "ce": 0.27919611361357966, "rank": 0.0}
|
| 55 |
+
{"step": 55, "steps": 250, "lr": 4.713484907470133e-05, "grad_norm": 1.707811713218689, "loss_scale": 4.0, "retries": 0, "seconds": 214.4, "max_memory_gb": 17.08, "ce": 0.02040667062465218, "rank": 0.0}
|
| 56 |
+
{"step": 56, "steps": 250, "lr": 4.6985571585149876e-05, "grad_norm": 0.8959540724754333, "loss_scale": 4.0, "retries": 0, "seconds": 218.5, "max_memory_gb": 17.08, "ce": 0.06463480223555962, "rank": 0.0}
|
| 57 |
+
{"step": 57, "steps": 250, "lr": 4.6832781932649884e-05, "grad_norm": 2.329482316970825, "loss_scale": 4.0, "retries": 0, "seconds": 222.6, "max_memory_gb": 17.08, "ce": 0.1651660698644264, "rank": 0.07626096904277802}
|
| 58 |
+
{"step": 58, "steps": 250, "lr": 4.667650765666325e-05, "grad_norm": 1.300006628036499, "loss_scale": 4.0, "retries": 0, "seconds": 226.7, "max_memory_gb": 17.08, "ce": 0.04263321844291568, "rank": 0.0}
|
| 59 |
+
{"step": 59, "steps": 250, "lr": 4.651677692473538e-05, "grad_norm": 0.43236175179481506, "loss_scale": 4.0, "retries": 0, "seconds": 231.0, "max_memory_gb": 17.08, "ce": 0.006645400558312531, "rank": 0.0}
|
| 60 |
+
{"step": 60, "steps": 250, "lr": 4.635361852741808e-05, "grad_norm": 0.9790313839912415, "loss_scale": 4.0, "retries": 0, "seconds": 235.0, "max_memory_gb": 17.08, "ce": 0.019702801058002706, "rank": 0.0}
|
| 61 |
+
{"step": 61, "steps": 250, "lr": 4.6187061873080264e-05, "grad_norm": 2.6670081615448, "loss_scale": 4.0, "retries": 0, "seconds": 238.7, "max_memory_gb": 17.08, "ce": 0.08072296295358683, "rank": 0.0}
|
| 62 |
+
{"step": 62, "steps": 250, "lr": 4.601713698260727e-05, "grad_norm": 4.543146133422852, "loss_scale": 4.0, "retries": 0, "seconds": 242.6, "max_memory_gb": 17.08, "ce": 0.39592873670062545, "rank": 0.0}
|
| 63 |
+
{"step": 63, "steps": 250, "lr": 4.5843874483989746e-05, "grad_norm": 0.13405413925647736, "loss_scale": 4.0, "retries": 0, "seconds": 246.3, "max_memory_gb": 17.08, "ce": 0.00802668552017849, "rank": 0.1250038594007492}
|
| 64 |
+
{"step": 64, "steps": 250, "lr": 4.566730560680316e-05, "grad_norm": 1.9112285375595093, "loss_scale": 4.0, "retries": 0, "seconds": 250.1, "max_memory_gb": 17.08, "ce": 0.21389864581942675, "rank": 0.0}
|
| 65 |
+
{"step": 65, "steps": 250, "lr": 4.548746217657878e-05, "grad_norm": 1.263197422027588, "loss_scale": 4.0, "retries": 0, "seconds": 253.8, "max_memory_gb": 17.08, "ce": 0.010103101852678265, "rank": 0.0}
|
| 66 |
+
{"step": 66, "steps": 250, "lr": 4.5304376609067396e-05, "grad_norm": 0.07391109317541122, "loss_scale": 4.0, "retries": 0, "seconds": 257.7, "max_memory_gb": 17.08, "ce": 0.004751508195113274, "rank": 0.0}
|
| 67 |
+
{"step": 67, "steps": 250, "lr": 4.5118081904396504e-05, "grad_norm": 5.516204833984375, "loss_scale": 4.0, "retries": 0, "seconds": 261.4, "max_memory_gb": 17.08, "ce": 0.3549019398433302, "rank": 0.0}
|
| 68 |
+
{"step": 68, "steps": 250, "lr": 4.4928611641122225e-05, "grad_norm": 0.8676789402961731, "loss_scale": 4.0, "retries": 0, "seconds": 265.1, "max_memory_gb": 17.08, "ce": 0.014506205345242051, "rank": 0.0}
|
| 69 |
+
{"step": 69, "steps": 250, "lr": 4.473599997017701e-05, "grad_norm": 2.2834105491638184, "loss_scale": 4.0, "retries": 0, "seconds": 268.9, "max_memory_gb": 17.08, "ce": 0.05124769757185277, "rank": 0.0}
|
| 70 |
+
{"step": 70, "steps": 250, "lr": 4.4540281608714053e-05, "grad_norm": 0.35388195514678955, "loss_scale": 4.0, "retries": 0, "seconds": 272.7, "max_memory_gb": 17.08, "ce": 0.015042846385767916, "rank": 0.0}
|
| 71 |
+
{"step": 71, "steps": 250, "lr": 4.434149183384977e-05, "grad_norm": 0.5449714064598083, "loss_scale": 4.0, "retries": 0, "seconds": 276.4, "max_memory_gb": 17.08, "ce": 0.019231321136828683, "rank": 0.0}
|
| 72 |
+
{"step": 72, "steps": 250, "lr": 4.413966647630534e-05, "grad_norm": 1.1807421445846558, "loss_scale": 4.0, "retries": 0, "seconds": 280.2, "max_memory_gb": 17.08, "ce": 0.01537487814644578, "rank": 0.0}
|
| 73 |
+
{"step": 73, "steps": 250, "lr": 4.3934841913948323e-05, "grad_norm": 1.5323989391326904, "loss_scale": 4.0, "retries": 0, "seconds": 284.0, "max_memory_gb": 17.08, "ce": 0.07475730224541621, "rank": 0.0}
|
| 74 |
+
{"step": 74, "steps": 250, "lr": 4.372705506523583e-05, "grad_norm": 0.7240369915962219, "loss_scale": 4.0, "retries": 0, "seconds": 287.7, "max_memory_gb": 17.08, "ce": 0.03212790608461091, "rank": 0.0}
|
| 75 |
+
{"step": 75, "steps": 250, "lr": 4.351634338256017e-05, "grad_norm": 5.636172771453857, "loss_scale": 4.0, "retries": 0, "seconds": 291.4, "max_memory_gb": 17.08, "ce": 0.07441795404190543, "rank": 0.0}
|
| 76 |
+
{"step": 76, "steps": 250, "lr": 4.330274484549825e-05, "grad_norm": 2.261362314224243, "loss_scale": 4.0, "retries": 0, "seconds": 295.2, "max_memory_gb": 17.08, "ce": 0.4941812502780749, "rank": 0.0}
|
| 77 |
+
{"step": 77, "steps": 250, "lr": 4.308629795396599e-05, "grad_norm": 0.0980444997549057, "loss_scale": 4.0, "retries": 0, "seconds": 299.0, "max_memory_gb": 17.08, "ce": 0.004034332344872382, "rank": 0.0}
|
| 78 |
+
{"step": 78, "steps": 250, "lr": 4.286704172127894e-05, "grad_norm": 3.564086675643921, "loss_scale": 4.0, "retries": 0, "seconds": 302.7, "max_memory_gb": 17.08, "ce": 0.24811177284732366, "rank": 0.0}
|
| 79 |
+
{"step": 79, "steps": 250, "lr": 4.264501566712033e-05, "grad_norm": 1.2897391319274902, "loss_scale": 4.0, "retries": 0, "seconds": 306.6, "max_memory_gb": 17.08, "ce": 0.042676671424715096, "rank": 0.0}
|
| 80 |
+
{"step": 80, "steps": 250, "lr": 4.2420259810417894e-05, "grad_norm": 2.8355538845062256, "loss_scale": 4.0, "retries": 0, "seconds": 310.3, "max_memory_gb": 17.08, "ce": 0.20074635207660663, "rank": 0.12499479949474335}
|
| 81 |
+
{"step": 81, "steps": 250, "lr": 4.219281466213066e-05, "grad_norm": 1.4929193258285522, "loss_scale": 4.0, "retries": 0, "seconds": 314.1, "max_memory_gb": 17.08, "ce": 0.0935371894729542, "rank": 0.12394589185714722}
|
| 82 |
+
{"step": 82, "steps": 250, "lr": 4.1962721217947135e-05, "grad_norm": 0.21032385528087616, "loss_scale": 4.0, "retries": 0, "seconds": 317.8, "max_memory_gb": 17.08, "ce": 0.007806892064763815, "rank": 0.0}
|
| 83 |
+
{"step": 83, "steps": 250, "lr": 4.173002095089599e-05, "grad_norm": 0.852388322353363, "loss_scale": 4.0, "retries": 0, "seconds": 321.6, "max_memory_gb": 17.08, "ce": 0.05925695889436611, "rank": 0.0}
|
| 84 |
+
{"step": 84, "steps": 250, "lr": 4.149475580387084e-05, "grad_norm": 1.1359655857086182, "loss_scale": 4.0, "retries": 0, "seconds": 325.3, "max_memory_gb": 17.08, "ce": 0.09212669633689075, "rank": 0.0}
|
| 85 |
+
{"step": 85, "steps": 250, "lr": 4.1256968182070275e-05, "grad_norm": 0.47754794359207153, "loss_scale": 4.0, "retries": 0, "seconds": 329.0, "max_memory_gb": 17.08, "ce": 0.010078802830548739, "rank": 0.0}
|
| 86 |
+
{"step": 86, "steps": 250, "lr": 4.101670094535453e-05, "grad_norm": 1.3229920864105225, "loss_scale": 4.0, "retries": 0, "seconds": 332.8, "max_memory_gb": 17.08, "ce": 0.13586906495402218, "rank": 0.0}
|
| 87 |
+
{"step": 87, "steps": 250, "lr": 4.077399740052027e-05, "grad_norm": 0.10300732403993607, "loss_scale": 4.0, "retries": 0, "seconds": 336.5, "max_memory_gb": 17.08, "ce": 0.004102866305402131, "rank": 0.0}
|
| 88 |
+
{"step": 88, "steps": 250, "lr": 4.0528901293494775e-05, "grad_norm": 0.712647557258606, "loss_scale": 4.0, "retries": 0, "seconds": 340.2, "max_memory_gb": 17.08, "ce": 0.021819988000970625, "rank": 0.0}
|
| 89 |
+
{"step": 89, "steps": 250, "lr": 4.0281456801451004e-05, "grad_norm": 2.604123830795288, "loss_scale": 4.0, "retries": 0, "seconds": 344.0, "max_memory_gb": 17.08, "ce": 0.11771337946538551, "rank": 0.0}
|
| 90 |
+
{"step": 90, "steps": 250, "lr": 4.0031708524844884e-05, "grad_norm": 0.571904182434082, "loss_scale": 4.0, "retries": 0, "seconds": 347.6, "max_memory_gb": 17.08, "ce": 0.03304807170479762, "rank": 0.0}
|
| 91 |
+
{"step": 91, "steps": 250, "lr": 3.977970147937635e-05, "grad_norm": 1.921425700187683, "loss_scale": 4.0, "retries": 0, "seconds": 351.3, "max_memory_gb": 17.08, "ce": 0.026212379271328246, "rank": 0.0}
|
| 92 |
+
{"step": 92, "steps": 250, "lr": 3.952548108787553e-05, "grad_norm": 4.321619987487793, "loss_scale": 4.0, "retries": 0, "seconds": 354.9, "max_memory_gb": 17.08, "ce": 0.17675770638743415, "rank": 0.0}
|
| 93 |
+
{"step": 93, "steps": 250, "lr": 3.9269093172115496e-05, "grad_norm": 9.496642112731934, "loss_scale": 4.0, "retries": 0, "seconds": 358.7, "max_memory_gb": 17.08, "ce": 0.8121576600751723, "rank": 0.0}
|
| 94 |
+
{"step": 94, "steps": 250, "lr": 3.901058394455323e-05, "grad_norm": 0.3861716389656067, "loss_scale": 4.0, "retries": 0, "seconds": 362.3, "max_memory_gb": 17.08, "ce": 0.0283896532887411, "rank": 0.0}
|
| 95 |
+
{"step": 95, "steps": 250, "lr": 3.875e-05, "grad_norm": 5.056302070617676, "loss_scale": 4.0, "retries": 0, "seconds": 366.0, "max_memory_gb": 17.08, "ce": 0.427974397493017, "rank": 0.0}
|
| 96 |
+
{"step": 96, "steps": 250, "lr": 3.8487388307223005e-05, "grad_norm": 1.314931869506836, "loss_scale": 4.0, "retries": 0, "seconds": 369.8, "max_memory_gb": 17.08, "ce": 0.08732501401755144, "rank": 0.0}
|
| 97 |
+
{"step": 97, "steps": 250, "lr": 3.822279620047943e-05, "grad_norm": 0.42710116505622864, "loss_scale": 4.0, "retries": 0, "seconds": 373.5, "max_memory_gb": 17.08, "ce": 0.01574293909197877, "rank": 0.0}
|
| 98 |
+
{"step": 98, "steps": 250, "lr": 3.795627137098479e-05, "grad_norm": 0.12185370177030563, "loss_scale": 4.0, "retries": 0, "seconds": 377.1, "max_memory_gb": 17.08, "ce": 0.0066662333156273235, "rank": 0.0}
|
| 99 |
+
{"step": 99, "steps": 250, "lr": 3.7687861858316844e-05, "grad_norm": 1.40840744972229, "loss_scale": 4.0, "retries": 0, "seconds": 380.9, "max_memory_gb": 17.08, "ce": 0.0546680578008818, "rank": 0.0}
|
| 100 |
+
{"step": 100, "steps": 250, "lr": 3.741761604175668e-05, "grad_norm": 2.1186773777008057, "loss_scale": 4.0, "retries": 0, "seconds": 384.6, "max_memory_gb": 17.08, "ce": 0.06579178703032085, "rank": 0.12498355656862259}
|
| 101 |
+
{"step": 101, "steps": 250, "lr": 3.714558263156872e-05, "grad_norm": 3.0268361568450928, "loss_scale": 4.0, "retries": 0, "seconds": 388.3, "max_memory_gb": 17.08, "ce": 0.18731665007749143, "rank": 0.0}
|
| 102 |
+
{"step": 102, "steps": 250, "lr": 3.6871810660220904e-05, "grad_norm": 0.20356398820877075, "loss_scale": 4.0, "retries": 0, "seconds": 392.0, "max_memory_gb": 17.08, "ce": 0.005296087366559732, "rank": 0.2311423420906067}
|
| 103 |
+
{"step": 103, "steps": 250, "lr": 3.659634947354686e-05, "grad_norm": 0.09683167934417725, "loss_scale": 4.0, "retries": 0, "seconds": 395.7, "max_memory_gb": 17.08, "ce": 0.0018098458117492555, "rank": 0.0}
|
| 104 |
+
{"step": 104, "steps": 250, "lr": 3.631924872185169e-05, "grad_norm": 2.2905876636505127, "loss_scale": 4.0, "retries": 0, "seconds": 399.4, "max_memory_gb": 17.08, "ce": 0.15950702544250817, "rank": 0.0}
|
| 105 |
+
{"step": 105, "steps": 250, "lr": 3.6040558350962674e-05, "grad_norm": 1.4475408792495728, "loss_scale": 4.0, "retries": 0, "seconds": 403.1, "max_memory_gb": 17.08, "ce": 0.12946462631225586, "rank": 0.0}
|
| 106 |
+
{"step": 106, "steps": 250, "lr": 3.576032859322687e-05, "grad_norm": 0.5000743865966797, "loss_scale": 4.0, "retries": 0, "seconds": 406.9, "max_memory_gb": 17.08, "ce": 0.028242915896043996, "rank": 0.0}
|
| 107 |
+
{"step": 107, "steps": 250, "lr": 3.5478609958457056e-05, "grad_norm": 0.533727765083313, "loss_scale": 4.0, "retries": 0, "seconds": 410.6, "max_memory_gb": 17.08, "ce": 0.02484775297307351, "rank": 0.0}
|
| 108 |
+
{"step": 108, "steps": 250, "lr": 3.519545322482755e-05, "grad_norm": 0.5098404288291931, "loss_scale": 4.0, "retries": 0, "seconds": 414.3, "max_memory_gb": 17.08, "ce": 0.021462390635861084, "rank": 0.0}
|
| 109 |
+
{"step": 109, "steps": 250, "lr": 3.491090942972179e-05, "grad_norm": 0.2556065618991852, "loss_scale": 4.0, "retries": 0, "seconds": 418.1, "max_memory_gb": 17.08, "ce": 0.00799956046648731, "rank": 0.0}
|
| 110 |
+
{"step": 110, "steps": 250, "lr": 3.4625029860533135e-05, "grad_norm": 0.05493856221437454, "loss_scale": 4.0, "retries": 0, "seconds": 421.7, "max_memory_gb": 17.08, "ce": 0.0017779083432287734, "rank": 0.0}
|
| 111 |
+
{"step": 111, "steps": 250, "lr": 3.433786604542056e-05, "grad_norm": 5.022639274597168, "loss_scale": 4.0, "retries": 0, "seconds": 425.4, "max_memory_gb": 17.08, "ce": 0.11292225826946378, "rank": 0.0}
|
| 112 |
+
{"step": 112, "steps": 250, "lr": 3.404946974402104e-05, "grad_norm": 0.1969456523656845, "loss_scale": 4.0, "retries": 0, "seconds": 429.1, "max_memory_gb": 17.08, "ce": 0.01026546741331913, "rank": 0.0}
|
| 113 |
+
{"step": 113, "steps": 250, "lr": 3.375989293812019e-05, "grad_norm": 4.696430683135986, "loss_scale": 4.0, "retries": 0, "seconds": 432.9, "max_memory_gb": 17.08, "ce": 0.46222678321555577, "rank": 0.0}
|
| 114 |
+
{"step": 114, "steps": 250, "lr": 3.3469187822282766e-05, "grad_norm": 2.069157600402832, "loss_scale": 4.0, "retries": 0, "seconds": 436.5, "max_memory_gb": 17.08, "ce": 0.24142971335868424, "rank": 0.0}
|
| 115 |
+
{"step": 115, "steps": 250, "lr": 3.317740679444499e-05, "grad_norm": 0.2494714856147766, "loss_scale": 4.0, "retries": 0, "seconds": 440.2, "max_memory_gb": 17.08, "ce": 0.009745366859533533, "rank": 0.0}
|
| 116 |
+
{"step": 116, "steps": 250, "lr": 3.2884602446470056e-05, "grad_norm": 4.460777282714844, "loss_scale": 4.0, "retries": 0, "seconds": 443.9, "max_memory_gb": 17.08, "ce": 0.4679188350937693, "rank": 0.0}
|
| 117 |
+
{"step": 117, "steps": 250, "lr": 3.259082755466876e-05, "grad_norm": 1.6322505474090576, "loss_scale": 4.0, "retries": 0, "seconds": 447.6, "max_memory_gb": 17.08, "ce": 0.036306243010585604, "rank": 0.0}
|
| 118 |
+
{"step": 118, "steps": 250, "lr": 3.229613507028686e-05, "grad_norm": 0.896525502204895, "loss_scale": 4.0, "retries": 0, "seconds": 451.3, "max_memory_gb": 17.08, "ce": 0.01470673736093886, "rank": 0.12582583725452423}
|
| 119 |
+
{"step": 119, "steps": 250, "lr": 3.2000578109961004e-05, "grad_norm": 0.0679524764418602, "loss_scale": 4.0, "retries": 0, "seconds": 454.9, "max_memory_gb": 17.08, "ce": 0.002887217652187246, "rank": 0.0}
|
| 120 |
+
{"step": 120, "steps": 250, "lr": 3.1704209946144616e-05, "grad_norm": 2.9987730979919434, "loss_scale": 4.0, "retries": 0, "seconds": 458.7, "max_memory_gb": 17.08, "ce": 0.19465058711557504, "rank": 0.0}
|
| 121 |
+
{"step": 121, "steps": 250, "lr": 3.140708399750594e-05, "grad_norm": 1.8543368577957153, "loss_scale": 4.0, "retries": 0, "seconds": 462.4, "max_memory_gb": 17.08, "ce": 0.14176315468193934, "rank": 0.0}
|
| 122 |
+
{"step": 122, "steps": 250, "lr": 3.1109253819299606e-05, "grad_norm": 1.5786765813827515, "loss_scale": 4.0, "retries": 0, "seconds": 466.1, "max_memory_gb": 17.08, "ce": 0.07060853841812786, "rank": 0.0}
|
| 123 |
+
{"step": 123, "steps": 250, "lr": 3.081077309371361e-05, "grad_norm": 5.733559608459473, "loss_scale": 4.0, "retries": 0, "seconds": 469.9, "max_memory_gb": 17.08, "ce": 0.42594436849958583, "rank": 0.0}
|
| 124 |
+
{"step": 124, "steps": 250, "lr": 3.051169562019339e-05, "grad_norm": 0.024161485955119133, "loss_scale": 4.0, "retries": 0, "seconds": 473.6, "max_memory_gb": 17.08, "ce": 0.0013083734069141428, "rank": 0.0}
|
| 125 |
+
{"step": 125, "steps": 250, "lr": 3.021207530574477e-05, "grad_norm": 3.6713125705718994, "loss_scale": 4.0, "retries": 0, "seconds": 477.3, "max_memory_gb": 17.08, "ce": 0.25581429837893666, "rank": 0.0}
|
| 126 |
+
{"step": 126, "steps": 250, "lr": 2.991196615521758e-05, "grad_norm": 0.34161630272865295, "loss_scale": 4.0, "retries": 0, "seconds": 481.0, "max_memory_gb": 17.08, "ce": 0.01962103038749774, "rank": 0.0}
|
| 127 |
+
{"step": 127, "steps": 250, "lr": 2.9611422261571536e-05, "grad_norm": 0.3747219741344452, "loss_scale": 4.0, "retries": 0, "seconds": 484.7, "max_memory_gb": 17.08, "ce": 0.006298865423104871, "rank": 0.0}
|
| 128 |
+
{"step": 128, "steps": 250, "lr": 2.931049779612634e-05, "grad_norm": 0.26130056381225586, "loss_scale": 4.0, "retries": 0, "seconds": 488.3, "max_memory_gb": 17.08, "ce": 0.011711178476161876, "rank": 0.0}
|
| 129 |
+
{"step": 129, "steps": 250, "lr": 2.9009246998797557e-05, "grad_norm": 0.8288151621818542, "loss_scale": 4.0, "retries": 0, "seconds": 491.9, "max_memory_gb": 17.08, "ce": 0.03131309808486549, "rank": 0.0}
|
| 130 |
+
{"step": 130, "steps": 250, "lr": 2.8707724168320238e-05, "grad_norm": 0.9031187891960144, "loss_scale": 4.0, "retries": 0, "seconds": 495.7, "max_memory_gb": 17.08, "ce": 0.05691689354171103, "rank": 0.0}
|
| 131 |
+
{"step": 131, "steps": 250, "lr": 2.840598365246184e-05, "grad_norm": 1.2660579681396484, "loss_scale": 4.0, "retries": 0, "seconds": 499.4, "max_memory_gb": 17.08, "ce": 0.10151081340848123, "rank": 0.0}
|
| 132 |
+
{"step": 132, "steps": 250, "lr": 2.8104079838226383e-05, "grad_norm": 0.22235819697380066, "loss_scale": 4.0, "retries": 0, "seconds": 503.1, "max_memory_gb": 17.08, "ce": 0.008838971911245608, "rank": 0.0}
|
| 133 |
+
{"step": 133, "steps": 250, "lr": 2.7802067142051446e-05, "grad_norm": 11.980902671813965, "loss_scale": 4.0, "retries": 0, "seconds": 506.8, "max_memory_gb": 17.08, "ce": 0.6236332577609573, "rank": 0.0}
|
| 134 |
+
{"step": 134, "steps": 250, "lr": 2.7500000000000004e-05, "grad_norm": 0.08028478175401688, "loss_scale": 4.0, "retries": 0, "seconds": 510.4, "max_memory_gb": 17.08, "ce": 0.0051557918268372305, "rank": 0.0}
|
| 135 |
+
{"step": 135, "steps": 250, "lr": 2.7197932857948566e-05, "grad_norm": 6.82779598236084, "loss_scale": 4.0, "retries": 0, "seconds": 514.0, "max_memory_gb": 17.08, "ce": 0.227736969968646, "rank": 0.0}
|
| 136 |
+
{"step": 136, "steps": 250, "lr": 2.689592016177363e-05, "grad_norm": 5.038846015930176, "loss_scale": 4.0, "retries": 0, "seconds": 517.8, "max_memory_gb": 17.08, "ce": 0.6129920683788441, "rank": 0.0}
|
| 137 |
+
{"step": 137, "steps": 250, "lr": 2.659401634753817e-05, "grad_norm": 6.013617515563965, "loss_scale": 4.0, "retries": 0, "seconds": 521.4, "max_memory_gb": 17.08, "ce": 0.12582398646753745, "rank": 0.0}
|
| 138 |
+
{"step": 138, "steps": 250, "lr": 2.6292275831679764e-05, "grad_norm": 0.07234561443328857, "loss_scale": 4.0, "retries": 0, "seconds": 525.0, "max_memory_gb": 17.08, "ce": 0.0020565098177485197, "rank": 0.09560047090053558}
|
| 139 |
+
{"step": 139, "steps": 250, "lr": 2.5990753001202445e-05, "grad_norm": 0.1520952433347702, "loss_scale": 4.0, "retries": 0, "seconds": 528.7, "max_memory_gb": 17.08, "ce": 0.005682671511749504, "rank": 0.0}
|
| 140 |
+
{"step": 140, "steps": 250, "lr": 2.5689502203873672e-05, "grad_norm": 1.1032254695892334, "loss_scale": 4.0, "retries": 0, "seconds": 532.4, "max_memory_gb": 17.08, "ce": 0.10591860937711317, "rank": 0.0}
|
| 141 |
+
{"step": 141, "steps": 250, "lr": 2.538857773842847e-05, "grad_norm": 0.07413294166326523, "loss_scale": 4.0, "retries": 0, "seconds": 536.0, "max_memory_gb": 17.08, "ce": 0.003516444513024908, "rank": 0.0}
|
| 142 |
+
{"step": 142, "steps": 250, "lr": 2.5088033844782426e-05, "grad_norm": 5.937716960906982, "loss_scale": 4.0, "retries": 0, "seconds": 539.7, "max_memory_gb": 17.08, "ce": 0.14513061480647593, "rank": 0.0}
|
| 143 |
+
{"step": 143, "steps": 250, "lr": 2.4787924694255237e-05, "grad_norm": 2.970893621444702, "loss_scale": 4.0, "retries": 0, "seconds": 543.4, "max_memory_gb": 17.08, "ce": 0.1729426504980438, "rank": 0.0}
|
| 144 |
+
{"step": 144, "steps": 250, "lr": 2.4488304379806627e-05, "grad_norm": 0.18398310244083405, "loss_scale": 4.0, "retries": 0, "seconds": 547.1, "max_memory_gb": 17.08, "ce": 0.008878000288973453, "rank": 0.0}
|
| 145 |
+
{"step": 145, "steps": 250, "lr": 2.4189226906286388e-05, "grad_norm": 0.7772497534751892, "loss_scale": 4.0, "retries": 0, "seconds": 550.7, "max_memory_gb": 17.08, "ce": 0.06362077825224333, "rank": 0.0}
|
| 146 |
+
{"step": 146, "steps": 250, "lr": 2.3890746180700395e-05, "grad_norm": 3.0496413707733154, "loss_scale": 4.0, "retries": 0, "seconds": 554.4, "max_memory_gb": 17.08, "ce": 0.15773221614563226, "rank": 0.0}
|
| 147 |
+
{"step": 147, "steps": 250, "lr": 2.359291600249407e-05, "grad_norm": 2.4556827545166016, "loss_scale": 4.0, "retries": 0, "seconds": 558.0, "max_memory_gb": 17.08, "ce": 0.06504790452254383, "rank": 0.0}
|
| 148 |
+
{"step": 148, "steps": 250, "lr": 2.3295790053855392e-05, "grad_norm": 0.6428364515304565, "loss_scale": 4.0, "retries": 0, "seconds": 561.7, "max_memory_gb": 17.08, "ce": 0.02775959180371501, "rank": 0.0}
|
| 149 |
+
{"step": 149, "steps": 250, "lr": 2.299942189003901e-05, "grad_norm": 4.951608180999756, "loss_scale": 4.0, "retries": 0, "seconds": 565.3, "max_memory_gb": 17.08, "ce": 0.6294398404752428, "rank": 0.11073410511016846}
|
| 150 |
+
{"step": 150, "steps": 250, "lr": 2.2703864929713144e-05, "grad_norm": 0.7127236127853394, "loss_scale": 4.0, "retries": 0, "seconds": 569.1, "max_memory_gb": 17.08, "ce": 0.06286520184585243, "rank": 0.0}
|
| 151 |
+
{"step": 151, "steps": 250, "lr": 2.2409172445331257e-05, "grad_norm": 1.0511727333068848, "loss_scale": 4.0, "retries": 0, "seconds": 572.7, "max_memory_gb": 17.08, "ce": 0.13398995196428132, "rank": 0.0}
|
| 152 |
+
{"step": 152, "steps": 250, "lr": 2.211539755352996e-05, "grad_norm": 2.4462976455688477, "loss_scale": 4.0, "retries": 0, "seconds": 576.3, "max_memory_gb": 17.08, "ce": 0.3291262801999437, "rank": 0.0}
|
| 153 |
+
{"step": 153, "steps": 250, "lr": 2.1822593205555016e-05, "grad_norm": 0.09329524636268616, "loss_scale": 4.0, "retries": 0, "seconds": 580.0, "max_memory_gb": 17.08, "ce": 0.001941391810305504, "rank": 0.0}
|
| 154 |
+
{"step": 154, "steps": 250, "lr": 2.1530812177717236e-05, "grad_norm": 2.379239559173584, "loss_scale": 4.0, "retries": 0, "seconds": 583.7, "max_memory_gb": 17.08, "ce": 0.07770185649587802, "rank": 0.0}
|
| 155 |
+
{"step": 155, "steps": 250, "lr": 2.1240107061879815e-05, "grad_norm": 0.42641377449035645, "loss_scale": 4.0, "retries": 0, "seconds": 587.3, "max_memory_gb": 17.08, "ce": 0.014270270741690183, "rank": 0.0}
|
| 156 |
+
{"step": 156, "steps": 250, "lr": 2.095053025597896e-05, "grad_norm": 0.6731432676315308, "loss_scale": 4.0, "retries": 0, "seconds": 591.0, "max_memory_gb": 17.08, "ce": 0.02730975139184011, "rank": 0.0}
|
| 157 |
+
{"step": 157, "steps": 250, "lr": 2.066213395457945e-05, "grad_norm": 2.234998941421509, "loss_scale": 4.0, "retries": 0, "seconds": 594.8, "max_memory_gb": 17.08, "ce": 0.33752349697533646, "rank": 0.0}
|
| 158 |
+
{"step": 158, "steps": 250, "lr": 2.0374970139466877e-05, "grad_norm": 0.2350037842988968, "loss_scale": 4.0, "retries": 0, "seconds": 598.4, "max_memory_gb": 17.08, "ce": 0.009402907667663385, "rank": 0.0}
|
| 159 |
+
{"step": 159, "steps": 250, "lr": 2.0089090570278213e-05, "grad_norm": 1.6774256229400635, "loss_scale": 4.0, "retries": 0, "seconds": 602.0, "max_memory_gb": 17.08, "ce": 0.019186244058801094, "rank": 0.1249534860253334}
|
| 160 |
+
{"step": 160, "steps": 250, "lr": 1.9804546775172457e-05, "grad_norm": 0.415021687746048, "loss_scale": 4.0, "retries": 0, "seconds": 605.8, "max_memory_gb": 17.08, "ce": 0.024501704379417788, "rank": 0.0}
|
| 161 |
+
{"step": 161, "steps": 250, "lr": 1.9521390041542946e-05, "grad_norm": 0.5947880148887634, "loss_scale": 4.0, "retries": 0, "seconds": 609.4, "max_memory_gb": 17.08, "ce": 0.05525858448982035, "rank": 0.0}
|
| 162 |
+
{"step": 162, "steps": 250, "lr": 1.923967140677313e-05, "grad_norm": 0.6860068440437317, "loss_scale": 4.0, "retries": 0, "seconds": 613.1, "max_memory_gb": 17.08, "ce": 0.02853027049195589, "rank": 0.0}
|
| 163 |
+
{"step": 163, "steps": 250, "lr": 1.8959441649037335e-05, "grad_norm": 1.194733738899231, "loss_scale": 4.0, "retries": 0, "seconds": 616.9, "max_memory_gb": 17.08, "ce": 0.05664567370899931, "rank": 0.0}
|
| 164 |
+
{"step": 164, "steps": 250, "lr": 1.868075127814832e-05, "grad_norm": 0.8436962962150574, "loss_scale": 4.0, "retries": 0, "seconds": 620.7, "max_memory_gb": 17.08, "ce": 0.03332410103939765, "rank": 0.0}
|
| 165 |
+
{"step": 165, "steps": 250, "lr": 1.8403650526453143e-05, "grad_norm": 0.03098321333527565, "loss_scale": 4.0, "retries": 0, "seconds": 624.6, "max_memory_gb": 17.08, "ce": 0.001197674835111684, "rank": 0.0}
|
| 166 |
+
{"step": 166, "steps": 250, "lr": 1.812818933977912e-05, "grad_norm": 0.0995892658829689, "loss_scale": 4.0, "retries": 0, "seconds": 628.2, "max_memory_gb": 17.08, "ce": 0.0027528906061888847, "rank": 0.0}
|
| 167 |
+
{"step": 167, "steps": 250, "lr": 1.785441736843128e-05, "grad_norm": 4.1044182777404785, "loss_scale": 4.0, "retries": 0, "seconds": 632.1, "max_memory_gb": 17.08, "ce": 0.3368585739917762, "rank": 0.06360951066017151}
|
| 168 |
+
{"step": 168, "steps": 250, "lr": 1.7582383958243314e-05, "grad_norm": 0.8566218018531799, "loss_scale": 4.0, "retries": 0, "seconds": 635.8, "max_memory_gb": 17.08, "ce": 0.08186243144245964, "rank": 0.0}
|
| 169 |
+
{"step": 169, "steps": 250, "lr": 1.7312138141683164e-05, "grad_norm": 0.2725003957748413, "loss_scale": 4.0, "retries": 0, "seconds": 639.4, "max_memory_gb": 17.08, "ce": 0.003961387638810265, "rank": 0.0}
|
| 170 |
+
{"step": 170, "steps": 250, "lr": 1.704372862901521e-05, "grad_norm": 1.8588006496429443, "loss_scale": 4.0, "retries": 0, "seconds": 643.2, "max_memory_gb": 17.08, "ce": 0.12696860752657813, "rank": 0.0}
|
| 171 |
+
{"step": 171, "steps": 250, "lr": 1.6777203799520573e-05, "grad_norm": 1.2281463146209717, "loss_scale": 4.0, "retries": 0, "seconds": 646.9, "max_memory_gb": 17.08, "ce": 0.03968458076678871, "rank": 0.0}
|
| 172 |
+
{"step": 172, "steps": 250, "lr": 1.6512611692777008e-05, "grad_norm": 3.1142210960388184, "loss_scale": 4.0, "retries": 0, "seconds": 650.5, "max_memory_gb": 17.08, "ce": 0.1886145129083161, "rank": 0.0}
|
| 173 |
+
{"step": 173, "steps": 250, "lr": 1.6250000000000005e-05, "grad_norm": 0.6775799989700317, "loss_scale": 4.0, "retries": 0, "seconds": 654.3, "max_memory_gb": 17.08, "ce": 0.03026275134232037, "rank": 0.0}
|
| 174 |
+
{"step": 174, "steps": 250, "lr": 1.598941605544678e-05, "grad_norm": 6.818516731262207, "loss_scale": 4.0, "retries": 0, "seconds": 658.0, "max_memory_gb": 17.08, "ce": 0.30563750329474715, "rank": 0.16025711596012115}
|
| 175 |
+
{"step": 175, "steps": 250, "lr": 1.5730906827884506e-05, "grad_norm": 2.071580648422241, "loss_scale": 4.0, "retries": 0, "seconds": 661.7, "max_memory_gb": 17.08, "ce": 0.14917561298170767, "rank": 0.0}
|
| 176 |
+
{"step": 176, "steps": 250, "lr": 1.5474518912124476e-05, "grad_norm": 4.275876522064209, "loss_scale": 4.0, "retries": 0, "seconds": 665.4, "max_memory_gb": 17.08, "ce": 0.35480842844526705, "rank": 0.0}
|
| 177 |
+
{"step": 177, "steps": 250, "lr": 1.5220298520623646e-05, "grad_norm": 0.9366891980171204, "loss_scale": 4.0, "retries": 0, "seconds": 669.2, "max_memory_gb": 17.08, "ce": 0.04416861530899041, "rank": 0.0}
|
| 178 |
+
{"step": 178, "steps": 250, "lr": 1.4968291475155121e-05, "grad_norm": 0.023805003613233566, "loss_scale": 4.0, "retries": 0, "seconds": 672.9, "max_memory_gb": 17.08, "ce": 0.0007756567116530277, "rank": 0.0}
|
| 179 |
+
{"step": 179, "steps": 250, "lr": 1.4718543198548998e-05, "grad_norm": 1.5361711978912354, "loss_scale": 4.0, "retries": 0, "seconds": 676.5, "max_memory_gb": 17.08, "ce": 0.05942614840432725, "rank": 0.0}
|
| 180 |
+
{"step": 180, "steps": 250, "lr": 1.4471098706505227e-05, "grad_norm": 0.3948069214820862, "loss_scale": 4.0, "retries": 0, "seconds": 680.3, "max_memory_gb": 17.08, "ce": 0.005554503444272996, "rank": 0.124751515686512}
|
| 181 |
+
{"step": 181, "steps": 250, "lr": 1.4226002599479743e-05, "grad_norm": 2.141144037246704, "loss_scale": 4.0, "retries": 0, "seconds": 684.0, "max_memory_gb": 17.08, "ce": 0.10796518040433511, "rank": 0.0}
|
| 182 |
+
{"step": 182, "steps": 250, "lr": 1.3983299054645471e-05, "grad_norm": 0.10720900446176529, "loss_scale": 4.0, "retries": 0, "seconds": 687.6, "max_memory_gb": 17.08, "ce": 0.002877863812045689, "rank": 0.0}
|
| 183 |
+
{"step": 183, "steps": 250, "lr": 1.3743031817929724e-05, "grad_norm": 4.667112827301025, "loss_scale": 4.0, "retries": 0, "seconds": 691.5, "max_memory_gb": 17.08, "ce": 0.18640738587964734, "rank": 0.0}
|
| 184 |
+
{"step": 184, "steps": 250, "lr": 1.3505244196129164e-05, "grad_norm": 3.8630082607269287, "loss_scale": 4.0, "retries": 0, "seconds": 695.2, "max_memory_gb": 17.08, "ce": 0.251210379992699, "rank": 0.0}
|
| 185 |
+
{"step": 185, "steps": 250, "lr": 1.3269979049104014e-05, "grad_norm": 3.932661294937134, "loss_scale": 4.0, "retries": 0, "seconds": 698.9, "max_memory_gb": 17.08, "ce": 0.08495986159016411, "rank": 0.0}
|
| 186 |
+
{"step": 186, "steps": 250, "lr": 1.3037278782052863e-05, "grad_norm": 3.4733686447143555, "loss_scale": 4.0, "retries": 0, "seconds": 702.5, "max_memory_gb": 17.08, "ce": 0.16934633539472088, "rank": 0.0}
|
| 187 |
+
{"step": 187, "steps": 250, "lr": 1.2807185337869343e-05, "grad_norm": 6.3446946144104, "loss_scale": 4.0, "retries": 0, "seconds": 706.3, "max_memory_gb": 17.08, "ce": 0.15194219522390995, "rank": 0.0}
|
| 188 |
+
{"step": 188, "steps": 250, "lr": 1.2579740189582115e-05, "grad_norm": 0.6286545991897583, "loss_scale": 4.0, "retries": 0, "seconds": 710.0, "max_memory_gb": 17.08, "ce": 0.01755314357262705, "rank": 0.1249895691871643}
|
| 189 |
+
{"step": 189, "steps": 250, "lr": 1.2354984332879668e-05, "grad_norm": 0.24331161379814148, "loss_scale": 4.0, "retries": 0, "seconds": 713.7, "max_memory_gb": 17.08, "ce": 0.00836049447326559, "rank": 0.0}
|
| 190 |
+
{"step": 190, "steps": 250, "lr": 1.2132958278721064e-05, "grad_norm": 4.884385585784912, "loss_scale": 4.0, "retries": 0, "seconds": 717.5, "max_memory_gb": 17.08, "ce": 0.05675247292083441, "rank": 0.0}
|
| 191 |
+
{"step": 191, "steps": 250, "lr": 1.1913702046034016e-05, "grad_norm": 0.05128437280654907, "loss_scale": 4.0, "retries": 0, "seconds": 721.1, "max_memory_gb": 17.08, "ce": 0.0018764025552968633, "rank": 0.12498379498720169}
|
| 192 |
+
{"step": 192, "steps": 250, "lr": 1.1697255154501756e-05, "grad_norm": 1.704095721244812, "loss_scale": 4.0, "retries": 0, "seconds": 724.8, "max_memory_gb": 17.08, "ce": 0.08250094148866083, "rank": 0.0}
|
| 193 |
+
{"step": 193, "steps": 250, "lr": 1.1483656617439842e-05, "grad_norm": 0.13309094309806824, "loss_scale": 4.0, "retries": 0, "seconds": 728.5, "max_memory_gb": 17.08, "ce": 0.0036364117663651996, "rank": 0.0}
|
| 194 |
+
{"step": 194, "steps": 250, "lr": 1.1272944934764177e-05, "grad_norm": 1.539103388786316, "loss_scale": 4.0, "retries": 0, "seconds": 732.3, "max_memory_gb": 17.08, "ce": 0.03663197294122256, "rank": 0.0}
|
| 195 |
+
{"step": 195, "steps": 250, "lr": 1.1065158086051684e-05, "grad_norm": 2.194751501083374, "loss_scale": 4.0, "retries": 0, "seconds": 736.0, "max_memory_gb": 17.08, "ce": 0.08286811708376263, "rank": 0.0}
|
| 196 |
+
{"step": 196, "steps": 250, "lr": 1.086033352369467e-05, "grad_norm": 0.03281080350279808, "loss_scale": 4.0, "retries": 0, "seconds": 739.6, "max_memory_gb": 17.08, "ce": 0.0008631238580392164, "rank": 0.0}
|
| 197 |
+
{"step": 197, "steps": 250, "lr": 1.0658508166150224e-05, "grad_norm": 0.802182137966156, "loss_scale": 4.0, "retries": 0, "seconds": 743.5, "max_memory_gb": 17.08, "ce": 0.0005935225622692997, "rank": 0.030061841011047363}
|
| 198 |
+
{"step": 198, "steps": 250, "lr": 1.0459718391285954e-05, "grad_norm": 2.271636724472046, "loss_scale": 4.0, "retries": 0, "seconds": 747.1, "max_memory_gb": 17.08, "ce": 0.10866363670561441, "rank": 0.0}
|
| 199 |
+
{"step": 199, "steps": 250, "lr": 1.0264000029822999e-05, "grad_norm": 0.03439005836844444, "loss_scale": 4.0, "retries": 0, "seconds": 750.8, "max_memory_gb": 17.08, "ce": 0.0004626791701980437, "rank": 0.0}
|
| 200 |
+
{"step": 200, "steps": 250, "lr": 1.0071388358877775e-05, "grad_norm": 1.944015383720398, "loss_scale": 4.0, "retries": 0, "seconds": 754.6, "max_memory_gb": 17.08, "ce": 0.20677675285469377, "rank": 0.0}
|
| 201 |
+
{"step": 201, "steps": 250, "lr": 9.881918095603501e-06, "grad_norm": 0.4903058409690857, "loss_scale": 4.0, "retries": 0, "seconds": 758.3, "max_memory_gb": 17.08, "ce": 0.012974693038813712, "rank": 0.0}
|
| 202 |
+
{"step": 202, "steps": 250, "lr": 9.695623390932612e-06, "grad_norm": 0.11196888983249664, "loss_scale": 4.0, "retries": 0, "seconds": 761.9, "max_memory_gb": 17.08, "ce": 0.004702936715830219, "rank": 0.0}
|
| 203 |
+
{"step": 203, "steps": 250, "lr": 9.51253782342123e-06, "grad_norm": 1.0838398933410645, "loss_scale": 4.0, "retries": 0, "seconds": 765.6, "max_memory_gb": 17.08, "ce": 0.027814785114287588, "rank": 0.0}
|
| 204 |
+
{"step": 204, "steps": 250, "lr": 9.332694393196846e-06, "grad_norm": 0.20892515778541565, "loss_scale": 4.0, "retries": 0, "seconds": 769.4, "max_memory_gb": 17.08, "ce": 0.0023665608662213344, "rank": 0.0}
|
| 205 |
+
{"step": 205, "steps": 250, "lr": 9.156125516010256e-06, "grad_norm": 3.0074217319488525, "loss_scale": 4.0, "retries": 0, "seconds": 773.0, "max_memory_gb": 17.08, "ce": 0.3977101996974852, "rank": 0.12499788403511047}
|
| 206 |
+
{"step": 206, "steps": 250, "lr": 8.982863017392733e-06, "grad_norm": 2.665703058242798, "loss_scale": 4.0, "retries": 0, "seconds": 776.7, "max_memory_gb": 17.08, "ce": 0.30770237862589056, "rank": 0.0}
|
| 207 |
+
{"step": 207, "steps": 250, "lr": 8.81293812691974e-06, "grad_norm": 0.2269548922777176, "loss_scale": 4.0, "retries": 0, "seconds": 780.4, "max_memory_gb": 17.08, "ce": 0.00457057062885724, "rank": 0.0}
|
| 208 |
+
{"step": 208, "steps": 250, "lr": 8.646381472581927e-06, "grad_norm": 0.14926329255104065, "loss_scale": 4.0, "retries": 0, "seconds": 784.1, "max_memory_gb": 17.08, "ce": 0.004395042313717568, "rank": 0.0}
|
| 209 |
+
{"step": 209, "steps": 250, "lr": 8.483223075264622e-06, "grad_norm": 0.26833397150039673, "loss_scale": 4.0, "retries": 0, "seconds": 787.7, "max_memory_gb": 17.08, "ce": 0.007363166837421886, "rank": 0.0}
|
| 210 |
+
{"step": 210, "steps": 250, "lr": 8.323492343336747e-06, "grad_norm": 1.5098434686660767, "loss_scale": 4.0, "retries": 0, "seconds": 791.5, "max_memory_gb": 17.08, "ce": 0.024087833253474855, "rank": 0.0}
|
| 211 |
+
{"step": 211, "steps": 250, "lr": 8.167218067350122e-06, "grad_norm": 3.5484745502471924, "loss_scale": 4.0, "retries": 0, "seconds": 795.2, "max_memory_gb": 17.08, "ce": 0.5063345010389639, "rank": 0.0}
|
| 212 |
+
{"step": 212, "steps": 250, "lr": 8.01442841485013e-06, "grad_norm": 1.3754783868789673, "loss_scale": 4.0, "retries": 0, "seconds": 798.9, "max_memory_gb": 17.08, "ce": 0.07831803403996673, "rank": 0.0}
|
| 213 |
+
{"step": 213, "steps": 250, "lr": 7.865150925298678e-06, "grad_norm": 2.2180917263031006, "loss_scale": 4.0, "retries": 0, "seconds": 802.6, "max_memory_gb": 17.08, "ce": 0.09080128697951295, "rank": 0.0}
|
| 214 |
+
{"step": 214, "steps": 250, "lr": 7.719412505110375e-06, "grad_norm": 1.516960859298706, "loss_scale": 4.0, "retries": 0, "seconds": 806.4, "max_memory_gb": 17.08, "ce": 0.06159509389544837, "rank": 0.0}
|
| 215 |
+
{"step": 215, "steps": 250, "lr": 7.57723942280278e-06, "grad_norm": 0.23074647784233093, "loss_scale": 4.0, "retries": 0, "seconds": 810.0, "max_memory_gb": 17.08, "ce": 0.007549004279098881, "rank": 0.0}
|
| 216 |
+
{"step": 216, "steps": 250, "lr": 7.438657304261687e-06, "grad_norm": 0.7323095798492432, "loss_scale": 4.0, "retries": 0, "seconds": 813.7, "max_memory_gb": 17.08, "ce": 0.027899305807295605, "rank": 0.0}
|
| 217 |
+
{"step": 217, "steps": 250, "lr": 7.30369112812219e-06, "grad_norm": 0.11863496899604797, "loss_scale": 4.0, "retries": 0, "seconds": 817.4, "max_memory_gb": 17.08, "ce": 0.003529963965775096, "rank": 0.0}
|
| 218 |
+
{"step": 218, "steps": 250, "lr": 7.172365221266403e-06, "grad_norm": 2.0874319076538086, "loss_scale": 4.0, "retries": 0, "seconds": 821.1, "max_memory_gb": 17.08, "ce": 0.19261202421898815, "rank": 0.0}
|
| 219 |
+
{"step": 219, "steps": 250, "lr": 7.0447032544387154e-06, "grad_norm": 0.6137192249298096, "loss_scale": 4.0, "retries": 0, "seconds": 824.7, "max_memory_gb": 17.08, "ce": 0.01006955043067137, "rank": 0.0}
|
| 220 |
+
{"step": 220, "steps": 250, "lr": 6.920728237979274e-06, "grad_norm": 2.550293207168579, "loss_scale": 4.0, "retries": 0, "seconds": 828.6, "max_memory_gb": 17.08, "ce": 0.11420049279513478, "rank": 0.0}
|
| 221 |
+
{"step": 221, "steps": 250, "lr": 6.800462517676456e-06, "grad_norm": 0.5316318273544312, "loss_scale": 4.0, "retries": 0, "seconds": 832.2, "max_memory_gb": 17.08, "ce": 0.01084415814955264, "rank": 0.0}
|
| 222 |
+
{"step": 222, "steps": 250, "lr": 6.683927770739222e-06, "grad_norm": 2.5910871028900146, "loss_scale": 4.0, "retries": 0, "seconds": 835.9, "max_memory_gb": 17.08, "ce": 0.07050702584592727, "rank": 0.0}
|
| 223 |
+
{"step": 223, "steps": 250, "lr": 6.571145001889892e-06, "grad_norm": 0.9818238019943237, "loss_scale": 4.0, "retries": 0, "seconds": 839.6, "max_memory_gb": 17.08, "ce": 0.08808083888993679, "rank": 0.0}
|
| 224 |
+
{"step": 224, "steps": 250, "lr": 6.46213453957817e-06, "grad_norm": 4.892595291137695, "loss_scale": 4.0, "retries": 0, "seconds": 843.4, "max_memory_gb": 17.08, "ce": 0.10109316066746032, "rank": 0.12498950958251953}
|
| 225 |
+
{"step": 225, "steps": 250, "lr": 6.356916032317063e-06, "grad_norm": 0.8747104406356812, "loss_scale": 4.0, "retries": 0, "seconds": 847.1, "max_memory_gb": 17.08, "ce": 0.027731139304137287, "rank": 0.0}
|
| 226 |
+
{"step": 226, "steps": 250, "lr": 6.255508445141349e-06, "grad_norm": 1.7030398845672607, "loss_scale": 4.0, "retries": 0, "seconds": 850.7, "max_memory_gb": 17.08, "ce": 0.04379406623070281, "rank": 0.0}
|
| 227 |
+
{"step": 227, "steps": 250, "lr": 6.157930056189228e-06, "grad_norm": 1.7874029874801636, "loss_scale": 4.0, "retries": 0, "seconds": 854.5, "max_memory_gb": 17.08, "ce": 0.09370130964680357, "rank": 0.0}
|
| 228 |
+
{"step": 228, "steps": 250, "lr": 6.064198453407796e-06, "grad_norm": 1.4528977870941162, "loss_scale": 4.0, "retries": 0, "seconds": 858.2, "max_memory_gb": 17.08, "ce": 0.10424364101436367, "rank": 0.0}
|
| 229 |
+
{"step": 229, "steps": 250, "lr": 5.974330531382935e-06, "grad_norm": 0.023001451045274734, "loss_scale": 4.0, "retries": 0, "seconds": 861.9, "max_memory_gb": 17.08, "ce": 0.0007045404795462673, "rank": 0.0}
|
| 230 |
+
{"step": 230, "steps": 250, "lr": 5.888342488294125e-06, "grad_norm": 0.7009673714637756, "loss_scale": 4.0, "retries": 0, "seconds": 865.6, "max_memory_gb": 17.08, "ce": 0.023470351662695066, "rank": 0.0}
|
| 231 |
+
{"step": 231, "steps": 250, "lr": 5.806249822994849e-06, "grad_norm": 1.9548633098602295, "loss_scale": 4.0, "retries": 0, "seconds": 869.3, "max_memory_gb": 17.08, "ce": 0.10960501269846645, "rank": 0.0}
|
| 232 |
+
{"step": 232, "steps": 250, "lr": 5.728067332218994e-06, "grad_norm": 0.004514210857450962, "loss_scale": 4.0, "retries": 0, "seconds": 873.0, "max_memory_gb": 17.08, "ce": 0.00023610164276988144, "rank": 0.12500101327896118}
|
| 233 |
+
{"step": 233, "steps": 250, "lr": 5.65380910791383e-06, "grad_norm": 2.8119900226593018, "loss_scale": 4.0, "retries": 0, "seconds": 876.7, "max_memory_gb": 17.08, "ce": 0.09194097344334295, "rank": 0.0}
|
| 234 |
+
{"step": 234, "steps": 250, "lr": 5.583488534700022e-06, "grad_norm": 2.4857780933380127, "loss_scale": 4.0, "retries": 0, "seconds": 880.5, "max_memory_gb": 17.08, "ce": 0.09108917941694017, "rank": 0.0}
|
| 235 |
+
{"step": 235, "steps": 250, "lr": 5.517118287459118e-06, "grad_norm": 8.722990036010742, "loss_scale": 4.0, "retries": 0, "seconds": 884.1, "max_memory_gb": 17.08, "ce": 1.1438881892519817, "rank": 0.0}
|
| 236 |
+
{"step": 236, "steps": 250, "lr": 5.454710329048978e-06, "grad_norm": 0.12056785821914673, "loss_scale": 4.0, "retries": 0, "seconds": 887.8, "max_memory_gb": 17.08, "ce": 0.002031116583225412, "rank": 0.0}
|
| 237 |
+
{"step": 237, "steps": 250, "lr": 5.3962759081475436e-06, "grad_norm": 0.15060406923294067, "loss_scale": 4.0, "retries": 0, "seconds": 891.2, "max_memory_gb": 17.08, "ce": 0.002234329845407501, "rank": 0.0}
|
| 238 |
+
{"step": 238, "steps": 250, "lr": 5.341825557225321e-06, "grad_norm": 0.34791192412376404, "loss_scale": 4.0, "retries": 0, "seconds": 894.6, "max_memory_gb": 17.08, "ce": 0.016124563513358225, "rank": 0.0}
|
| 239 |
+
{"step": 239, "steps": 250, "lr": 5.291369090646961e-06, "grad_norm": 1.468825101852417, "loss_scale": 4.0, "retries": 0, "seconds": 897.9, "max_memory_gb": 17.08, "ce": 0.11850225622788457, "rank": 0.0}
|
| 240 |
+
{"step": 240, "steps": 250, "lr": 5.244915602902289e-06, "grad_norm": 2.0664126873016357, "loss_scale": 4.0, "retries": 0, "seconds": 901.2, "max_memory_gb": 17.08, "ce": 0.04850879952618925, "rank": 0.0}
|
| 241 |
+
{"step": 241, "steps": 250, "lr": 5.20247346696706e-06, "grad_norm": 1.6951825618743896, "loss_scale": 4.0, "retries": 0, "seconds": 904.6, "max_memory_gb": 17.08, "ce": 0.08198678765597833, "rank": 0.12281257659196854}
|
| 242 |
+
{"step": 242, "steps": 250, "lr": 5.164050332793787e-06, "grad_norm": 0.4788489043712616, "loss_scale": 4.0, "retries": 0, "seconds": 907.9, "max_memory_gb": 17.08, "ce": 0.023631895668586367, "rank": 0.0}
|
| 243 |
+
{"step": 243, "steps": 250, "lr": 5.129653125932873e-06, "grad_norm": 1.0912790298461914, "loss_scale": 4.0, "retries": 0, "seconds": 911.3, "max_memory_gb": 17.08, "ce": 0.03709778643633399, "rank": 0.0}
|
| 244 |
+
{"step": 244, "steps": 250, "lr": 5.099288046284329e-06, "grad_norm": 1.4668965339660645, "loss_scale": 4.0, "retries": 0, "seconds": 914.6, "max_memory_gb": 17.08, "ce": 0.025230530007945617, "rank": 0.0}
|
| 245 |
+
{"step": 245, "steps": 250, "lr": 5.072960566980277e-06, "grad_norm": 0.3022083342075348, "loss_scale": 4.0, "retries": 0, "seconds": 918.0, "max_memory_gb": 17.08, "ce": 0.016516017755350276, "rank": 0.0}
|
| 246 |
+
{"step": 246, "steps": 250, "lr": 5.050675433398435e-06, "grad_norm": 1.964464545249939, "loss_scale": 4.0, "retries": 0, "seconds": 921.3, "max_memory_gb": 17.08, "ce": 0.11654917691521405, "rank": 0.0}
|
| 247 |
+
{"step": 247, "steps": 250, "lr": 5.032436662306811e-06, "grad_norm": 1.3952395915985107, "loss_scale": 4.0, "retries": 0, "seconds": 924.7, "max_memory_gb": 17.08, "ce": 0.021578514829343476, "rank": 0.0}
|
| 248 |
+
{"step": 248, "steps": 250, "lr": 5.018247541139684e-06, "grad_norm": 0.17109976708889008, "loss_scale": 4.0, "retries": 0, "seconds": 928.0, "max_memory_gb": 17.08, "ce": 0.0014298332982605189, "rank": 0.0}
|
| 249 |
+
{"step": 249, "steps": 250, "lr": 5.00811062740508e-06, "grad_norm": 0.3112108111381531, "loss_scale": 4.0, "retries": 0, "seconds": 931.3, "max_memory_gb": 17.08, "ce": 0.009574376453429068, "rank": 0.0}
|
| 250 |
+
{"step": 250, "steps": 250, "lr": 5.002027748223778e-06, "grad_norm": 13.596537590026855, "loss_scale": 4.0, "retries": 0, "seconds": 934.6, "max_memory_gb": 17.08, "ce": 0.31244463632924635, "rank": 0.0}
|
runs/fixed_internvl/removal/Top8_s42/adapter/README.md
ADDED
|
@@ -0,0 +1,207 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
base_model: /data/OPSD/assets/models/InternVL3_5-8B-HF
|
| 3 |
+
library_name: peft
|
| 4 |
+
pipeline_tag: text-generation
|
| 5 |
+
tags:
|
| 6 |
+
- base_model:adapter:/data/OPSD/assets/models/InternVL3_5-8B-HF
|
| 7 |
+
- lora
|
| 8 |
+
- transformers
|
| 9 |
+
---
|
| 10 |
+
|
| 11 |
+
# Model Card for Model ID
|
| 12 |
+
|
| 13 |
+
<!-- Provide a quick summary of what the model is/does. -->
|
| 14 |
+
|
| 15 |
+
|
| 16 |
+
|
| 17 |
+
## Model Details
|
| 18 |
+
|
| 19 |
+
### Model Description
|
| 20 |
+
|
| 21 |
+
<!-- Provide a longer summary of what this model is. -->
|
| 22 |
+
|
| 23 |
+
|
| 24 |
+
|
| 25 |
+
- **Developed by:** [More Information Needed]
|
| 26 |
+
- **Funded by [optional]:** [More Information Needed]
|
| 27 |
+
- **Shared by [optional]:** [More Information Needed]
|
| 28 |
+
- **Model type:** [More Information Needed]
|
| 29 |
+
- **Language(s) (NLP):** [More Information Needed]
|
| 30 |
+
- **License:** [More Information Needed]
|
| 31 |
+
- **Finetuned from model [optional]:** [More Information Needed]
|
| 32 |
+
|
| 33 |
+
### Model Sources [optional]
|
| 34 |
+
|
| 35 |
+
<!-- Provide the basic links for the model. -->
|
| 36 |
+
|
| 37 |
+
- **Repository:** [More Information Needed]
|
| 38 |
+
- **Paper [optional]:** [More Information Needed]
|
| 39 |
+
- **Demo [optional]:** [More Information Needed]
|
| 40 |
+
|
| 41 |
+
## Uses
|
| 42 |
+
|
| 43 |
+
<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
|
| 44 |
+
|
| 45 |
+
### Direct Use
|
| 46 |
+
|
| 47 |
+
<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
|
| 48 |
+
|
| 49 |
+
[More Information Needed]
|
| 50 |
+
|
| 51 |
+
### Downstream Use [optional]
|
| 52 |
+
|
| 53 |
+
<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
|
| 54 |
+
|
| 55 |
+
[More Information Needed]
|
| 56 |
+
|
| 57 |
+
### Out-of-Scope Use
|
| 58 |
+
|
| 59 |
+
<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
|
| 60 |
+
|
| 61 |
+
[More Information Needed]
|
| 62 |
+
|
| 63 |
+
## Bias, Risks, and Limitations
|
| 64 |
+
|
| 65 |
+
<!-- This section is meant to convey both technical and sociotechnical limitations. -->
|
| 66 |
+
|
| 67 |
+
[More Information Needed]
|
| 68 |
+
|
| 69 |
+
### Recommendations
|
| 70 |
+
|
| 71 |
+
<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
|
| 72 |
+
|
| 73 |
+
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
|
| 74 |
+
|
| 75 |
+
## How to Get Started with the Model
|
| 76 |
+
|
| 77 |
+
Use the code below to get started with the model.
|
| 78 |
+
|
| 79 |
+
[More Information Needed]
|
| 80 |
+
|
| 81 |
+
## Training Details
|
| 82 |
+
|
| 83 |
+
### Training Data
|
| 84 |
+
|
| 85 |
+
<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
|
| 86 |
+
|
| 87 |
+
[More Information Needed]
|
| 88 |
+
|
| 89 |
+
### Training Procedure
|
| 90 |
+
|
| 91 |
+
<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
|
| 92 |
+
|
| 93 |
+
#### Preprocessing [optional]
|
| 94 |
+
|
| 95 |
+
[More Information Needed]
|
| 96 |
+
|
| 97 |
+
|
| 98 |
+
#### Training Hyperparameters
|
| 99 |
+
|
| 100 |
+
- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
|
| 101 |
+
|
| 102 |
+
#### Speeds, Sizes, Times [optional]
|
| 103 |
+
|
| 104 |
+
<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
|
| 105 |
+
|
| 106 |
+
[More Information Needed]
|
| 107 |
+
|
| 108 |
+
## Evaluation
|
| 109 |
+
|
| 110 |
+
<!-- This section describes the evaluation protocols and provides the results. -->
|
| 111 |
+
|
| 112 |
+
### Testing Data, Factors & Metrics
|
| 113 |
+
|
| 114 |
+
#### Testing Data
|
| 115 |
+
|
| 116 |
+
<!-- This should link to a Dataset Card if possible. -->
|
| 117 |
+
|
| 118 |
+
[More Information Needed]
|
| 119 |
+
|
| 120 |
+
#### Factors
|
| 121 |
+
|
| 122 |
+
<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
|
| 123 |
+
|
| 124 |
+
[More Information Needed]
|
| 125 |
+
|
| 126 |
+
#### Metrics
|
| 127 |
+
|
| 128 |
+
<!-- These are the evaluation metrics being used, ideally with a description of why. -->
|
| 129 |
+
|
| 130 |
+
[More Information Needed]
|
| 131 |
+
|
| 132 |
+
### Results
|
| 133 |
+
|
| 134 |
+
[More Information Needed]
|
| 135 |
+
|
| 136 |
+
#### Summary
|
| 137 |
+
|
| 138 |
+
|
| 139 |
+
|
| 140 |
+
## Model Examination [optional]
|
| 141 |
+
|
| 142 |
+
<!-- Relevant interpretability work for the model goes here -->
|
| 143 |
+
|
| 144 |
+
[More Information Needed]
|
| 145 |
+
|
| 146 |
+
## Environmental Impact
|
| 147 |
+
|
| 148 |
+
<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
|
| 149 |
+
|
| 150 |
+
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
|
| 151 |
+
|
| 152 |
+
- **Hardware Type:** [More Information Needed]
|
| 153 |
+
- **Hours used:** [More Information Needed]
|
| 154 |
+
- **Cloud Provider:** [More Information Needed]
|
| 155 |
+
- **Compute Region:** [More Information Needed]
|
| 156 |
+
- **Carbon Emitted:** [More Information Needed]
|
| 157 |
+
|
| 158 |
+
## Technical Specifications [optional]
|
| 159 |
+
|
| 160 |
+
### Model Architecture and Objective
|
| 161 |
+
|
| 162 |
+
[More Information Needed]
|
| 163 |
+
|
| 164 |
+
### Compute Infrastructure
|
| 165 |
+
|
| 166 |
+
[More Information Needed]
|
| 167 |
+
|
| 168 |
+
#### Hardware
|
| 169 |
+
|
| 170 |
+
[More Information Needed]
|
| 171 |
+
|
| 172 |
+
#### Software
|
| 173 |
+
|
| 174 |
+
[More Information Needed]
|
| 175 |
+
|
| 176 |
+
## Citation [optional]
|
| 177 |
+
|
| 178 |
+
<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
|
| 179 |
+
|
| 180 |
+
**BibTeX:**
|
| 181 |
+
|
| 182 |
+
[More Information Needed]
|
| 183 |
+
|
| 184 |
+
**APA:**
|
| 185 |
+
|
| 186 |
+
[More Information Needed]
|
| 187 |
+
|
| 188 |
+
## Glossary [optional]
|
| 189 |
+
|
| 190 |
+
<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
|
| 191 |
+
|
| 192 |
+
[More Information Needed]
|
| 193 |
+
|
| 194 |
+
## More Information [optional]
|
| 195 |
+
|
| 196 |
+
[More Information Needed]
|
| 197 |
+
|
| 198 |
+
## Model Card Authors [optional]
|
| 199 |
+
|
| 200 |
+
[More Information Needed]
|
| 201 |
+
|
| 202 |
+
## Model Card Contact
|
| 203 |
+
|
| 204 |
+
[More Information Needed]
|
| 205 |
+
### Framework versions
|
| 206 |
+
|
| 207 |
+
- PEFT 0.20.0
|
runs/fixed_internvl/removal/Top8_s42/adapter/adapter_config.json
ADDED
|
@@ -0,0 +1,213 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"alora_invocation_tokens": null,
|
| 3 |
+
"alpha_pattern": {
|
| 4 |
+
"model.language_model.layers.0.mlp.down_proj": 144,
|
| 5 |
+
"model.language_model.layers.0.mlp.gate_proj": 144,
|
| 6 |
+
"model.language_model.layers.0.mlp.up_proj": 144,
|
| 7 |
+
"model.language_model.layers.0.self_attn.k_proj": 144,
|
| 8 |
+
"model.language_model.layers.0.self_attn.o_proj": 144,
|
| 9 |
+
"model.language_model.layers.0.self_attn.q_proj": 144,
|
| 10 |
+
"model.language_model.layers.0.self_attn.v_proj": 144,
|
| 11 |
+
"model.language_model.layers.1.mlp.down_proj": 144,
|
| 12 |
+
"model.language_model.layers.1.mlp.gate_proj": 144,
|
| 13 |
+
"model.language_model.layers.1.mlp.up_proj": 144,
|
| 14 |
+
"model.language_model.layers.1.self_attn.k_proj": 144,
|
| 15 |
+
"model.language_model.layers.1.self_attn.o_proj": 144,
|
| 16 |
+
"model.language_model.layers.1.self_attn.q_proj": 144,
|
| 17 |
+
"model.language_model.layers.1.self_attn.v_proj": 144,
|
| 18 |
+
"model.language_model.layers.2.mlp.down_proj": 144,
|
| 19 |
+
"model.language_model.layers.2.mlp.gate_proj": 144,
|
| 20 |
+
"model.language_model.layers.2.mlp.up_proj": 144,
|
| 21 |
+
"model.language_model.layers.2.self_attn.k_proj": 144,
|
| 22 |
+
"model.language_model.layers.2.self_attn.o_proj": 144,
|
| 23 |
+
"model.language_model.layers.2.self_attn.q_proj": 144,
|
| 24 |
+
"model.language_model.layers.2.self_attn.v_proj": 144,
|
| 25 |
+
"model.language_model.layers.3.mlp.down_proj": 144,
|
| 26 |
+
"model.language_model.layers.3.mlp.gate_proj": 144,
|
| 27 |
+
"model.language_model.layers.3.mlp.up_proj": 144,
|
| 28 |
+
"model.language_model.layers.3.self_attn.k_proj": 144,
|
| 29 |
+
"model.language_model.layers.3.self_attn.o_proj": 144,
|
| 30 |
+
"model.language_model.layers.3.self_attn.q_proj": 144,
|
| 31 |
+
"model.language_model.layers.3.self_attn.v_proj": 144,
|
| 32 |
+
"model.language_model.layers.4.mlp.down_proj": 144,
|
| 33 |
+
"model.language_model.layers.4.mlp.gate_proj": 144,
|
| 34 |
+
"model.language_model.layers.4.mlp.up_proj": 144,
|
| 35 |
+
"model.language_model.layers.4.self_attn.k_proj": 144,
|
| 36 |
+
"model.language_model.layers.4.self_attn.o_proj": 144,
|
| 37 |
+
"model.language_model.layers.4.self_attn.q_proj": 144,
|
| 38 |
+
"model.language_model.layers.4.self_attn.v_proj": 144,
|
| 39 |
+
"model.language_model.layers.5.mlp.down_proj": 144,
|
| 40 |
+
"model.language_model.layers.5.mlp.gate_proj": 144,
|
| 41 |
+
"model.language_model.layers.5.mlp.up_proj": 144,
|
| 42 |
+
"model.language_model.layers.5.self_attn.k_proj": 144,
|
| 43 |
+
"model.language_model.layers.5.self_attn.o_proj": 144,
|
| 44 |
+
"model.language_model.layers.5.self_attn.q_proj": 144,
|
| 45 |
+
"model.language_model.layers.5.self_attn.v_proj": 144,
|
| 46 |
+
"model.language_model.layers.6.mlp.down_proj": 144,
|
| 47 |
+
"model.language_model.layers.6.mlp.gate_proj": 144,
|
| 48 |
+
"model.language_model.layers.6.mlp.up_proj": 144,
|
| 49 |
+
"model.language_model.layers.6.self_attn.k_proj": 144,
|
| 50 |
+
"model.language_model.layers.6.self_attn.o_proj": 144,
|
| 51 |
+
"model.language_model.layers.6.self_attn.q_proj": 144,
|
| 52 |
+
"model.language_model.layers.6.self_attn.v_proj": 144,
|
| 53 |
+
"model.language_model.layers.7.mlp.down_proj": 144,
|
| 54 |
+
"model.language_model.layers.7.mlp.gate_proj": 144,
|
| 55 |
+
"model.language_model.layers.7.mlp.up_proj": 144,
|
| 56 |
+
"model.language_model.layers.7.self_attn.k_proj": 144,
|
| 57 |
+
"model.language_model.layers.7.self_attn.o_proj": 144,
|
| 58 |
+
"model.language_model.layers.7.self_attn.q_proj": 144,
|
| 59 |
+
"model.language_model.layers.7.self_attn.v_proj": 144
|
| 60 |
+
},
|
| 61 |
+
"arrow_config": null,
|
| 62 |
+
"auto_mapping": null,
|
| 63 |
+
"base_model_name_or_path": "/data/OPSD/assets/models/InternVL3_5-8B-HF",
|
| 64 |
+
"bias": "none",
|
| 65 |
+
"corda_config": null,
|
| 66 |
+
"ensure_weight_tying": false,
|
| 67 |
+
"eva_config": null,
|
| 68 |
+
"exclude_modules": null,
|
| 69 |
+
"fan_in_fan_out": false,
|
| 70 |
+
"inference_mode": true,
|
| 71 |
+
"init_lora_weights": true,
|
| 72 |
+
"layer_replication": null,
|
| 73 |
+
"layers_pattern": null,
|
| 74 |
+
"layers_to_transform": null,
|
| 75 |
+
"loftq_config": {},
|
| 76 |
+
"lora_alpha": 32,
|
| 77 |
+
"lora_bias": false,
|
| 78 |
+
"lora_dropout": 0.05,
|
| 79 |
+
"lora_ga_config": null,
|
| 80 |
+
"megatron_config": null,
|
| 81 |
+
"megatron_core": "megatron.core",
|
| 82 |
+
"modules_to_save": null,
|
| 83 |
+
"monteclora_config": null,
|
| 84 |
+
"peft_type": "LORA",
|
| 85 |
+
"peft_version": "0.20.0",
|
| 86 |
+
"qalora_group_size": 16,
|
| 87 |
+
"r": 16,
|
| 88 |
+
"rank_pattern": {
|
| 89 |
+
"model.language_model.layers.0.mlp.down_proj": 72,
|
| 90 |
+
"model.language_model.layers.0.mlp.gate_proj": 72,
|
| 91 |
+
"model.language_model.layers.0.mlp.up_proj": 72,
|
| 92 |
+
"model.language_model.layers.0.self_attn.k_proj": 72,
|
| 93 |
+
"model.language_model.layers.0.self_attn.o_proj": 72,
|
| 94 |
+
"model.language_model.layers.0.self_attn.q_proj": 72,
|
| 95 |
+
"model.language_model.layers.0.self_attn.v_proj": 72,
|
| 96 |
+
"model.language_model.layers.1.mlp.down_proj": 72,
|
| 97 |
+
"model.language_model.layers.1.mlp.gate_proj": 72,
|
| 98 |
+
"model.language_model.layers.1.mlp.up_proj": 72,
|
| 99 |
+
"model.language_model.layers.1.self_attn.k_proj": 72,
|
| 100 |
+
"model.language_model.layers.1.self_attn.o_proj": 72,
|
| 101 |
+
"model.language_model.layers.1.self_attn.q_proj": 72,
|
| 102 |
+
"model.language_model.layers.1.self_attn.v_proj": 72,
|
| 103 |
+
"model.language_model.layers.2.mlp.down_proj": 72,
|
| 104 |
+
"model.language_model.layers.2.mlp.gate_proj": 72,
|
| 105 |
+
"model.language_model.layers.2.mlp.up_proj": 72,
|
| 106 |
+
"model.language_model.layers.2.self_attn.k_proj": 72,
|
| 107 |
+
"model.language_model.layers.2.self_attn.o_proj": 72,
|
| 108 |
+
"model.language_model.layers.2.self_attn.q_proj": 72,
|
| 109 |
+
"model.language_model.layers.2.self_attn.v_proj": 72,
|
| 110 |
+
"model.language_model.layers.3.mlp.down_proj": 72,
|
| 111 |
+
"model.language_model.layers.3.mlp.gate_proj": 72,
|
| 112 |
+
"model.language_model.layers.3.mlp.up_proj": 72,
|
| 113 |
+
"model.language_model.layers.3.self_attn.k_proj": 72,
|
| 114 |
+
"model.language_model.layers.3.self_attn.o_proj": 72,
|
| 115 |
+
"model.language_model.layers.3.self_attn.q_proj": 72,
|
| 116 |
+
"model.language_model.layers.3.self_attn.v_proj": 72,
|
| 117 |
+
"model.language_model.layers.4.mlp.down_proj": 72,
|
| 118 |
+
"model.language_model.layers.4.mlp.gate_proj": 72,
|
| 119 |
+
"model.language_model.layers.4.mlp.up_proj": 72,
|
| 120 |
+
"model.language_model.layers.4.self_attn.k_proj": 72,
|
| 121 |
+
"model.language_model.layers.4.self_attn.o_proj": 72,
|
| 122 |
+
"model.language_model.layers.4.self_attn.q_proj": 72,
|
| 123 |
+
"model.language_model.layers.4.self_attn.v_proj": 72,
|
| 124 |
+
"model.language_model.layers.5.mlp.down_proj": 72,
|
| 125 |
+
"model.language_model.layers.5.mlp.gate_proj": 72,
|
| 126 |
+
"model.language_model.layers.5.mlp.up_proj": 72,
|
| 127 |
+
"model.language_model.layers.5.self_attn.k_proj": 72,
|
| 128 |
+
"model.language_model.layers.5.self_attn.o_proj": 72,
|
| 129 |
+
"model.language_model.layers.5.self_attn.q_proj": 72,
|
| 130 |
+
"model.language_model.layers.5.self_attn.v_proj": 72,
|
| 131 |
+
"model.language_model.layers.6.mlp.down_proj": 72,
|
| 132 |
+
"model.language_model.layers.6.mlp.gate_proj": 72,
|
| 133 |
+
"model.language_model.layers.6.mlp.up_proj": 72,
|
| 134 |
+
"model.language_model.layers.6.self_attn.k_proj": 72,
|
| 135 |
+
"model.language_model.layers.6.self_attn.o_proj": 72,
|
| 136 |
+
"model.language_model.layers.6.self_attn.q_proj": 72,
|
| 137 |
+
"model.language_model.layers.6.self_attn.v_proj": 72,
|
| 138 |
+
"model.language_model.layers.7.mlp.down_proj": 72,
|
| 139 |
+
"model.language_model.layers.7.mlp.gate_proj": 72,
|
| 140 |
+
"model.language_model.layers.7.mlp.up_proj": 72,
|
| 141 |
+
"model.language_model.layers.7.self_attn.k_proj": 72,
|
| 142 |
+
"model.language_model.layers.7.self_attn.o_proj": 72,
|
| 143 |
+
"model.language_model.layers.7.self_attn.q_proj": 72,
|
| 144 |
+
"model.language_model.layers.7.self_attn.v_proj": 72
|
| 145 |
+
},
|
| 146 |
+
"revision": null,
|
| 147 |
+
"target_modules": [
|
| 148 |
+
"model.language_model.layers.3.self_attn.k_proj",
|
| 149 |
+
"model.language_model.layers.1.self_attn.v_proj",
|
| 150 |
+
"model.language_model.layers.5.mlp.down_proj",
|
| 151 |
+
"model.language_model.layers.7.self_attn.o_proj",
|
| 152 |
+
"model.language_model.layers.5.self_attn.o_proj",
|
| 153 |
+
"model.language_model.layers.4.self_attn.o_proj",
|
| 154 |
+
"model.language_model.layers.3.self_attn.o_proj",
|
| 155 |
+
"model.language_model.layers.6.self_attn.v_proj",
|
| 156 |
+
"model.language_model.layers.6.mlp.down_proj",
|
| 157 |
+
"model.language_model.layers.3.self_attn.v_proj",
|
| 158 |
+
"model.language_model.layers.1.mlp.up_proj",
|
| 159 |
+
"model.language_model.layers.0.self_attn.k_proj",
|
| 160 |
+
"model.language_model.layers.7.mlp.up_proj",
|
| 161 |
+
"model.language_model.layers.1.self_attn.q_proj",
|
| 162 |
+
"model.language_model.layers.4.self_attn.q_proj",
|
| 163 |
+
"model.language_model.layers.7.mlp.down_proj",
|
| 164 |
+
"model.language_model.layers.6.mlp.up_proj",
|
| 165 |
+
"model.language_model.layers.2.mlp.gate_proj",
|
| 166 |
+
"model.language_model.layers.4.self_attn.v_proj",
|
| 167 |
+
"model.language_model.layers.1.mlp.gate_proj",
|
| 168 |
+
"model.language_model.layers.5.self_attn.v_proj",
|
| 169 |
+
"model.language_model.layers.5.mlp.gate_proj",
|
| 170 |
+
"model.language_model.layers.7.mlp.gate_proj",
|
| 171 |
+
"model.language_model.layers.2.self_attn.o_proj",
|
| 172 |
+
"model.language_model.layers.6.self_attn.o_proj",
|
| 173 |
+
"model.language_model.layers.3.mlp.gate_proj",
|
| 174 |
+
"model.language_model.layers.4.self_attn.k_proj",
|
| 175 |
+
"model.language_model.layers.2.self_attn.v_proj",
|
| 176 |
+
"model.language_model.layers.5.self_attn.k_proj",
|
| 177 |
+
"model.language_model.layers.6.self_attn.k_proj",
|
| 178 |
+
"model.language_model.layers.0.self_attn.q_proj",
|
| 179 |
+
"model.language_model.layers.1.mlp.down_proj",
|
| 180 |
+
"model.language_model.layers.2.mlp.up_proj",
|
| 181 |
+
"model.language_model.layers.3.mlp.up_proj",
|
| 182 |
+
"model.language_model.layers.1.self_attn.k_proj",
|
| 183 |
+
"model.language_model.layers.2.self_attn.k_proj",
|
| 184 |
+
"model.language_model.layers.2.mlp.down_proj",
|
| 185 |
+
"model.language_model.layers.6.mlp.gate_proj",
|
| 186 |
+
"model.language_model.layers.3.mlp.down_proj",
|
| 187 |
+
"model.language_model.layers.3.self_attn.q_proj",
|
| 188 |
+
"model.language_model.layers.0.mlp.up_proj",
|
| 189 |
+
"model.language_model.layers.4.mlp.down_proj",
|
| 190 |
+
"model.language_model.layers.7.self_attn.q_proj",
|
| 191 |
+
"model.language_model.layers.7.self_attn.k_proj",
|
| 192 |
+
"model.language_model.layers.2.self_attn.q_proj",
|
| 193 |
+
"model.language_model.layers.4.mlp.gate_proj",
|
| 194 |
+
"model.language_model.layers.0.mlp.down_proj",
|
| 195 |
+
"model.language_model.layers.1.self_attn.o_proj",
|
| 196 |
+
"model.language_model.layers.7.self_attn.v_proj",
|
| 197 |
+
"model.language_model.layers.0.mlp.gate_proj",
|
| 198 |
+
"model.language_model.layers.6.self_attn.q_proj",
|
| 199 |
+
"model.language_model.layers.0.self_attn.v_proj",
|
| 200 |
+
"model.language_model.layers.0.self_attn.o_proj",
|
| 201 |
+
"model.language_model.layers.5.self_attn.q_proj",
|
| 202 |
+
"model.language_model.layers.5.mlp.up_proj",
|
| 203 |
+
"model.language_model.layers.4.mlp.up_proj"
|
| 204 |
+
],
|
| 205 |
+
"target_parameters": null,
|
| 206 |
+
"task_type": "CAUSAL_LM",
|
| 207 |
+
"trainable_token_indices": null,
|
| 208 |
+
"use_bdlora": null,
|
| 209 |
+
"use_dora": false,
|
| 210 |
+
"use_qalora": false,
|
| 211 |
+
"use_rslora": false,
|
| 212 |
+
"velora_config": null
|
| 213 |
+
}
|
runs/fixed_internvl/removal/Top8_s42/config.json
ADDED
|
@@ -0,0 +1,43 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"behavior": "removal",
|
| 3 |
+
"seed": 42,
|
| 4 |
+
"model": "/data/OPSD/assets/models/InternVL3_5-8B-HF",
|
| 5 |
+
"backbone": "internvl",
|
| 6 |
+
"train": "/data/OPSD/assets/v3/removal.jsonl",
|
| 7 |
+
"items": 1000,
|
| 8 |
+
"steps": 250,
|
| 9 |
+
"epochs": 1,
|
| 10 |
+
"steps_per_epoch": 250,
|
| 11 |
+
"world_size": 1,
|
| 12 |
+
"lr": 5e-05,
|
| 13 |
+
"scheduler": "cosine",
|
| 14 |
+
"warmup_steps": 16,
|
| 15 |
+
"final_fraction": 0.1,
|
| 16 |
+
"clip": 1.0,
|
| 17 |
+
"initial_loss_scale": 4.0,
|
| 18 |
+
"arm": {
|
| 19 |
+
"arm": "Top8",
|
| 20 |
+
"layers": [
|
| 21 |
+
0,
|
| 22 |
+
1,
|
| 23 |
+
2,
|
| 24 |
+
3,
|
| 25 |
+
4,
|
| 26 |
+
5,
|
| 27 |
+
6,
|
| 28 |
+
7
|
| 29 |
+
],
|
| 30 |
+
"ranks": {
|
| 31 |
+
"0": 72,
|
| 32 |
+
"1": 72,
|
| 33 |
+
"2": 72,
|
| 34 |
+
"3": 72,
|
| 35 |
+
"4": 72,
|
| 36 |
+
"5": 72,
|
| 37 |
+
"6": 72,
|
| 38 |
+
"7": 72
|
| 39 |
+
},
|
| 40 |
+
"parameters": 43646976,
|
| 41 |
+
"budget": 43646976
|
| 42 |
+
}
|
| 43 |
+
}
|
runs/fixed_internvl/removal/Top8_s42/steps.jsonl
ADDED
|
@@ -0,0 +1,250 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{"step": 1, "steps": 250, "lr": 3.125e-06, "grad_norm": 10.691676139831543, "loss_scale": 4.0, "retries": 0, "seconds": 4.7, "max_memory_gb": 16.79, "ce": 1.0494785578921437, "rank": 0.15784265100955963}
|
| 2 |
+
{"step": 2, "steps": 250, "lr": 6.25e-06, "grad_norm": 7.804408073425293, "loss_scale": 4.0, "retries": 0, "seconds": 8.5, "max_memory_gb": 17.07, "ce": 0.8135887895907672, "rank": 0.0}
|
| 3 |
+
{"step": 3, "steps": 250, "lr": 9.375000000000001e-06, "grad_norm": 12.885881423950195, "loss_scale": 4.0, "retries": 0, "seconds": 12.2, "max_memory_gb": 17.08, "ce": 0.7824967906926759, "rank": 0.0}
|
| 4 |
+
{"step": 4, "steps": 250, "lr": 1.25e-05, "grad_norm": 10.217583656311035, "loss_scale": 4.0, "retries": 0, "seconds": 15.8, "max_memory_gb": 17.08, "ce": 0.7423212314388365, "rank": 0.0}
|
| 5 |
+
{"step": 5, "steps": 250, "lr": 1.5625e-05, "grad_norm": 3.229170083999634, "loss_scale": 4.0, "retries": 0, "seconds": 19.6, "max_memory_gb": 17.08, "ce": 0.574996144277975, "rank": 0.0}
|
| 6 |
+
{"step": 6, "steps": 250, "lr": 1.8750000000000002e-05, "grad_norm": 3.580338954925537, "loss_scale": 4.0, "retries": 0, "seconds": 23.3, "max_memory_gb": 17.08, "ce": 0.35614653649099637, "rank": 0.0}
|
| 7 |
+
{"step": 7, "steps": 250, "lr": 2.1875e-05, "grad_norm": 3.2063663005828857, "loss_scale": 4.0, "retries": 0, "seconds": 27.0, "max_memory_gb": 17.08, "ce": 0.5616362588480115, "rank": 0.0}
|
| 8 |
+
{"step": 8, "steps": 250, "lr": 2.5e-05, "grad_norm": 1.2037458419799805, "loss_scale": 4.0, "retries": 0, "seconds": 30.7, "max_memory_gb": 17.08, "ce": 0.23910664773393364, "rank": 0.0}
|
| 9 |
+
{"step": 9, "steps": 250, "lr": 2.8125000000000003e-05, "grad_norm": 3.2820215225219727, "loss_scale": 4.0, "retries": 0, "seconds": 34.4, "max_memory_gb": 17.08, "ce": 0.6261661924072541, "rank": 0.0}
|
| 10 |
+
{"step": 10, "steps": 250, "lr": 3.125e-05, "grad_norm": 3.9456229209899902, "loss_scale": 4.0, "retries": 0, "seconds": 38.1, "max_memory_gb": 17.08, "ce": 0.286986716411775, "rank": 0.20723816752433777}
|
| 11 |
+
{"step": 11, "steps": 250, "lr": 3.4375e-05, "grad_norm": 0.31589993834495544, "loss_scale": 4.0, "retries": 0, "seconds": 41.8, "max_memory_gb": 17.08, "ce": 0.15028927266985193, "rank": 0.0}
|
| 12 |
+
{"step": 12, "steps": 250, "lr": 3.7500000000000003e-05, "grad_norm": 4.409406661987305, "loss_scale": 4.0, "retries": 0, "seconds": 45.6, "max_memory_gb": 17.08, "ce": 0.6728485974563228, "rank": 0.0}
|
| 13 |
+
{"step": 13, "steps": 250, "lr": 4.0625000000000005e-05, "grad_norm": 0.9566377401351929, "loss_scale": 4.0, "retries": 0, "seconds": 49.3, "max_memory_gb": 17.08, "ce": 0.40213705971837044, "rank": 0.0}
|
| 14 |
+
{"step": 14, "steps": 250, "lr": 4.375e-05, "grad_norm": 0.5727354288101196, "loss_scale": 4.0, "retries": 0, "seconds": 53.0, "max_memory_gb": 17.08, "ce": 0.35213973227655515, "rank": 0.0}
|
| 15 |
+
{"step": 15, "steps": 250, "lr": 4.6875e-05, "grad_norm": 5.07331657409668, "loss_scale": 4.0, "retries": 0, "seconds": 56.8, "max_memory_gb": 17.08, "ce": 0.6299880559090525, "rank": 0.18837271630764008}
|
| 16 |
+
{"step": 16, "steps": 250, "lr": 5e-05, "grad_norm": 3.869905948638916, "loss_scale": 4.0, "retries": 0, "seconds": 60.5, "max_memory_gb": 17.08, "ce": 0.42875847020332003, "rank": 0.0}
|
| 17 |
+
{"step": 17, "steps": 250, "lr": 5e-05, "grad_norm": 0.638138473033905, "loss_scale": 4.0, "retries": 0, "seconds": 64.1, "max_memory_gb": 17.08, "ce": 0.3103224817896262, "rank": 0.11261819303035736}
|
| 18 |
+
{"step": 18, "steps": 250, "lr": 4.999797225177623e-05, "grad_norm": 0.8651050329208374, "loss_scale": 4.0, "retries": 0, "seconds": 67.8, "max_memory_gb": 17.08, "ce": 0.33708847872912884, "rank": 0.0}
|
| 19 |
+
{"step": 19, "steps": 250, "lr": 4.999188937259492e-05, "grad_norm": 0.30989158153533936, "loss_scale": 4.0, "retries": 0, "seconds": 71.6, "max_memory_gb": 17.08, "ce": 0.18835503373702522, "rank": 0.0}
|
| 20 |
+
{"step": 20, "steps": 250, "lr": 4.998175245886032e-05, "grad_norm": 0.718824565410614, "loss_scale": 4.0, "retries": 0, "seconds": 75.3, "max_memory_gb": 17.08, "ce": 0.30647914129076526, "rank": 0.0}
|
| 21 |
+
{"step": 21, "steps": 250, "lr": 4.996756333769319e-05, "grad_norm": 0.388731986284256, "loss_scale": 4.0, "retries": 0, "seconds": 79.0, "max_memory_gb": 17.08, "ce": 0.046613820595666766, "rank": 0.0}
|
| 22 |
+
{"step": 22, "steps": 250, "lr": 4.994932456660157e-05, "grad_norm": 0.7786906957626343, "loss_scale": 4.0, "retries": 0, "seconds": 82.8, "max_memory_gb": 17.08, "ce": 0.3465729793460923, "rank": 0.0}
|
| 23 |
+
{"step": 23, "steps": 250, "lr": 4.992703943301973e-05, "grad_norm": 1.217643141746521, "loss_scale": 4.0, "retries": 0, "seconds": 86.5, "max_memory_gb": 17.08, "ce": 0.2817509083033656, "rank": 0.0}
|
| 24 |
+
{"step": 24, "steps": 250, "lr": 4.9900711953715675e-05, "grad_norm": 1.2318627834320068, "loss_scale": 4.0, "retries": 0, "seconds": 90.2, "max_memory_gb": 17.08, "ce": 0.37649797480844427, "rank": 0.0}
|
| 25 |
+
{"step": 25, "steps": 250, "lr": 4.987034687406713e-05, "grad_norm": 1.0966107845306396, "loss_scale": 4.0, "retries": 0, "seconds": 93.9, "max_memory_gb": 17.08, "ce": 0.21539715163089568, "rank": 0.12481538951396942}
|
| 26 |
+
{"step": 26, "steps": 250, "lr": 4.9835949667206215e-05, "grad_norm": 2.6527316570281982, "loss_scale": 4.0, "retries": 0, "seconds": 97.7, "max_memory_gb": 17.08, "ce": 0.21283616205619182, "rank": 0.0}
|
| 27 |
+
{"step": 27, "steps": 250, "lr": 4.9797526533032945e-05, "grad_norm": 5.231834888458252, "loss_scale": 4.0, "retries": 0, "seconds": 101.3, "max_memory_gb": 17.08, "ce": 0.42989165599828993, "rank": 0.0}
|
| 28 |
+
{"step": 28, "steps": 250, "lr": 4.9755084397097716e-05, "grad_norm": 1.7847963571548462, "loss_scale": 4.0, "retries": 0, "seconds": 105.0, "max_memory_gb": 17.08, "ce": 0.37184783816337585, "rank": 0.0}
|
| 29 |
+
{"step": 29, "steps": 250, "lr": 4.970863090935304e-05, "grad_norm": 0.7355087399482727, "loss_scale": 4.0, "retries": 0, "seconds": 108.8, "max_memory_gb": 17.08, "ce": 0.13254440900345799, "rank": 0.0}
|
| 30 |
+
{"step": 30, "steps": 250, "lr": 4.965817444277468e-05, "grad_norm": 1.7726026773452759, "loss_scale": 4.0, "retries": 0, "seconds": 112.5, "max_memory_gb": 17.08, "ce": 0.19107751140836626, "rank": 0.12479858845472336}
|
| 31 |
+
{"step": 31, "steps": 250, "lr": 4.9603724091852456e-05, "grad_norm": 4.06675386428833, "loss_scale": 4.0, "retries": 0, "seconds": 116.2, "max_memory_gb": 17.08, "ce": 0.8482002863602247, "rank": 0.0}
|
| 32 |
+
{"step": 32, "steps": 250, "lr": 4.9545289670951025e-05, "grad_norm": 0.6915540099143982, "loss_scale": 4.0, "retries": 0, "seconds": 120.0, "max_memory_gb": 17.08, "ce": 0.03862485592617304, "rank": 0.0}
|
| 33 |
+
{"step": 33, "steps": 250, "lr": 4.948288171254089e-05, "grad_norm": 0.5810591578483582, "loss_scale": 4.0, "retries": 0, "seconds": 123.7, "max_memory_gb": 17.08, "ce": 0.02391524905345932, "rank": 0.0}
|
| 34 |
+
{"step": 34, "steps": 250, "lr": 4.941651146529998e-05, "grad_norm": 1.1775639057159424, "loss_scale": 4.0, "retries": 0, "seconds": 127.4, "max_memory_gb": 17.08, "ce": 0.054460631694382755, "rank": 0.0}
|
| 35 |
+
{"step": 35, "steps": 250, "lr": 4.9346190892086174e-05, "grad_norm": 1.175378441810608, "loss_scale": 4.0, "retries": 0, "seconds": 131.2, "max_memory_gb": 17.08, "ce": 0.07153238763987702, "rank": 0.0}
|
| 36 |
+
{"step": 36, "steps": 250, "lr": 4.927193266778101e-05, "grad_norm": 7.876972675323486, "loss_scale": 4.0, "retries": 0, "seconds": 134.9, "max_memory_gb": 17.08, "ce": 0.3460162879163704, "rank": 0.0}
|
| 37 |
+
{"step": 37, "steps": 250, "lr": 4.919375017700515e-05, "grad_norm": 9.562066078186035, "loss_scale": 4.0, "retries": 0, "seconds": 138.6, "max_memory_gb": 17.08, "ce": 0.2374020843803919, "rank": 0.0}
|
| 38 |
+
{"step": 38, "steps": 250, "lr": 4.911165751170588e-05, "grad_norm": 0.6004496216773987, "loss_scale": 4.0, "retries": 0, "seconds": 142.3, "max_memory_gb": 17.08, "ce": 0.04637745743821142, "rank": 0.0}
|
| 39 |
+
{"step": 39, "steps": 250, "lr": 4.902566946861708e-05, "grad_norm": 3.622157096862793, "loss_scale": 4.0, "retries": 0, "seconds": 146.1, "max_memory_gb": 17.08, "ce": 0.2122392266028328, "rank": 0.0}
|
| 40 |
+
{"step": 40, "steps": 250, "lr": 4.89358015465922e-05, "grad_norm": 1.5406360626220703, "loss_scale": 4.0, "retries": 0, "seconds": 149.8, "max_memory_gb": 17.08, "ce": 0.06602024081638547, "rank": 0.0}
|
| 41 |
+
{"step": 41, "steps": 250, "lr": 4.884206994381078e-05, "grad_norm": 0.26944366097450256, "loss_scale": 4.0, "retries": 0, "seconds": 153.4, "max_memory_gb": 17.08, "ce": 0.019001079235749785, "rank": 0.0}
|
| 42 |
+
{"step": 42, "steps": 250, "lr": 4.874449155485866e-05, "grad_norm": 2.006760597229004, "loss_scale": 4.0, "retries": 0, "seconds": 157.2, "max_memory_gb": 17.08, "ce": 0.2326121165801851, "rank": 0.0}
|
| 43 |
+
{"step": 43, "steps": 250, "lr": 4.864308396768294e-05, "grad_norm": 4.638998985290527, "loss_scale": 4.0, "retries": 0, "seconds": 160.9, "max_memory_gb": 17.08, "ce": 0.08547707009711303, "rank": 0.0}
|
| 44 |
+
{"step": 44, "steps": 250, "lr": 4.853786546042184e-05, "grad_norm": 4.9275946617126465, "loss_scale": 4.0, "retries": 0, "seconds": 164.6, "max_memory_gb": 17.08, "ce": 0.414029552528973, "rank": 0.0}
|
| 45 |
+
{"step": 45, "steps": 250, "lr": 4.842885499811012e-05, "grad_norm": 5.348782539367676, "loss_scale": 4.0, "retries": 0, "seconds": 168.3, "max_memory_gb": 17.08, "ce": 0.396804522426919, "rank": 0.0}
|
| 46 |
+
{"step": 46, "steps": 250, "lr": 4.831607222926078e-05, "grad_norm": 2.123690128326416, "loss_scale": 4.0, "retries": 0, "seconds": 172.1, "max_memory_gb": 17.08, "ce": 0.2723591717731324, "rank": 0.0}
|
| 47 |
+
{"step": 47, "steps": 250, "lr": 4.8199537482323545e-05, "grad_norm": 0.5065475106239319, "loss_scale": 4.0, "retries": 0, "seconds": 175.8, "max_memory_gb": 17.08, "ce": 0.016101611516205594, "rank": 0.0}
|
| 48 |
+
{"step": 48, "steps": 250, "lr": 4.807927176202073e-05, "grad_norm": 1.488065481185913, "loss_scale": 4.0, "retries": 0, "seconds": 179.4, "max_memory_gb": 17.08, "ce": 0.0991949191011372, "rank": 0.0}
|
| 49 |
+
{"step": 49, "steps": 250, "lr": 4.795529674556129e-05, "grad_norm": 2.2179267406463623, "loss_scale": 4.0, "retries": 0, "seconds": 183.3, "max_memory_gb": 17.08, "ce": 0.11582854932930786, "rank": 0.12500810623168945}
|
| 50 |
+
{"step": 50, "steps": 250, "lr": 4.782763477873361e-05, "grad_norm": 1.2149795293807983, "loss_scale": 4.0, "retries": 0, "seconds": 186.9, "max_memory_gb": 17.08, "ce": 0.053539310494670644, "rank": 0.0}
|
| 51 |
+
{"step": 51, "steps": 250, "lr": 4.769630887187782e-05, "grad_norm": 1.6411100625991821, "loss_scale": 4.0, "retries": 0, "seconds": 190.6, "max_memory_gb": 17.08, "ce": 0.044239137387194205, "rank": 0.0}
|
| 52 |
+
{"step": 52, "steps": 250, "lr": 4.756134269573832e-05, "grad_norm": 0.1883188933134079, "loss_scale": 4.0, "retries": 0, "seconds": 194.4, "max_memory_gb": 17.08, "ce": 0.009671279703070468, "rank": 0.12513020634651184}
|
| 53 |
+
{"step": 53, "steps": 250, "lr": 4.7422760577197226e-05, "grad_norm": 0.41489842534065247, "loss_scale": 4.0, "retries": 0, "seconds": 198.1, "max_memory_gb": 17.08, "ce": 0.018196581302618142, "rank": 0.0}
|
| 54 |
+
{"step": 54, "steps": 250, "lr": 4.728058749488963e-05, "grad_norm": 1.6352484226226807, "loss_scale": 4.0, "retries": 0, "seconds": 201.7, "max_memory_gb": 17.08, "ce": 0.11255701449624667, "rank": 0.0}
|
| 55 |
+
{"step": 55, "steps": 250, "lr": 4.713484907470133e-05, "grad_norm": 8.923645973205566, "loss_scale": 4.0, "retries": 0, "seconds": 205.3, "max_memory_gb": 17.08, "ce": 0.2698595281305529, "rank": 0.0}
|
| 56 |
+
{"step": 56, "steps": 250, "lr": 4.6985571585149876e-05, "grad_norm": 2.652595043182373, "loss_scale": 4.0, "retries": 0, "seconds": 209.1, "max_memory_gb": 17.08, "ce": 0.07582288721494024, "rank": 0.05480138957500458}
|
| 57 |
+
{"step": 57, "steps": 250, "lr": 4.6832781932649884e-05, "grad_norm": 0.6355048418045044, "loss_scale": 4.0, "retries": 0, "seconds": 212.8, "max_memory_gb": 17.08, "ce": 0.03530810284769359, "rank": 0.0}
|
| 58 |
+
{"step": 58, "steps": 250, "lr": 4.667650765666325e-05, "grad_norm": 4.825430393218994, "loss_scale": 4.0, "retries": 0, "seconds": 216.5, "max_memory_gb": 17.08, "ce": 0.15108069636880828, "rank": 0.0}
|
| 59 |
+
{"step": 59, "steps": 250, "lr": 4.651677692473538e-05, "grad_norm": 2.340358257293701, "loss_scale": 4.0, "retries": 0, "seconds": 220.3, "max_memory_gb": 17.08, "ce": 0.02829675387363295, "rank": 0.0}
|
| 60 |
+
{"step": 60, "steps": 250, "lr": 4.635361852741808e-05, "grad_norm": 2.8007547855377197, "loss_scale": 4.0, "retries": 0, "seconds": 224.0, "max_memory_gb": 17.08, "ce": 0.2107318790606456, "rank": 0.0}
|
| 61 |
+
{"step": 61, "steps": 250, "lr": 4.6187061873080264e-05, "grad_norm": 0.28430092334747314, "loss_scale": 4.0, "retries": 0, "seconds": 227.6, "max_memory_gb": 17.08, "ce": 0.019203025987508227, "rank": 0.0}
|
| 62 |
+
{"step": 62, "steps": 250, "lr": 4.601713698260727e-05, "grad_norm": 0.6902067065238953, "loss_scale": 4.0, "retries": 0, "seconds": 231.5, "max_memory_gb": 17.08, "ce": 0.03706418270030554, "rank": 0.0}
|
| 63 |
+
{"step": 63, "steps": 250, "lr": 4.5843874483989746e-05, "grad_norm": 3.443631172180176, "loss_scale": 4.0, "retries": 0, "seconds": 235.1, "max_memory_gb": 17.08, "ce": 0.2824572605222784, "rank": 0.0}
|
| 64 |
+
{"step": 64, "steps": 250, "lr": 4.566730560680316e-05, "grad_norm": 3.1055550575256348, "loss_scale": 4.0, "retries": 0, "seconds": 238.9, "max_memory_gb": 17.08, "ce": 0.11227165392483585, "rank": 0.0}
|
| 65 |
+
{"step": 65, "steps": 250, "lr": 4.548746217657878e-05, "grad_norm": 11.299427032470703, "loss_scale": 4.0, "retries": 0, "seconds": 242.5, "max_memory_gb": 17.08, "ce": 0.26455954981429386, "rank": 0.0}
|
| 66 |
+
{"step": 66, "steps": 250, "lr": 4.5304376609067396e-05, "grad_norm": 3.065624713897705, "loss_scale": 4.0, "retries": 0, "seconds": 246.3, "max_memory_gb": 17.08, "ce": 0.07921002913917619, "rank": 0.0}
|
| 67 |
+
{"step": 67, "steps": 250, "lr": 4.5118081904396504e-05, "grad_norm": 1.3136287927627563, "loss_scale": 4.0, "retries": 0, "seconds": 250.0, "max_memory_gb": 17.08, "ce": 0.0169538695786855, "rank": 0.0}
|
| 68 |
+
{"step": 68, "steps": 250, "lr": 4.4928611641122225e-05, "grad_norm": 0.5024136900901794, "loss_scale": 4.0, "retries": 0, "seconds": 253.6, "max_memory_gb": 17.08, "ce": 0.01837741296776585, "rank": 0.0}
|
| 69 |
+
{"step": 69, "steps": 250, "lr": 4.473599997017701e-05, "grad_norm": 1.9405924081802368, "loss_scale": 4.0, "retries": 0, "seconds": 257.4, "max_memory_gb": 17.08, "ce": 0.047705632088764105, "rank": 0.0}
|
| 70 |
+
{"step": 70, "steps": 250, "lr": 4.4540281608714053e-05, "grad_norm": 4.4603681564331055, "loss_scale": 4.0, "retries": 0, "seconds": 261.0, "max_memory_gb": 17.08, "ce": 0.26042851145848545, "rank": 0.0}
|
| 71 |
+
{"step": 71, "steps": 250, "lr": 4.434149183384977e-05, "grad_norm": 4.823938369750977, "loss_scale": 4.0, "retries": 0, "seconds": 264.7, "max_memory_gb": 17.08, "ce": 0.4160309007638716, "rank": 0.11986858397722244}
|
| 72 |
+
{"step": 72, "steps": 250, "lr": 4.413966647630534e-05, "grad_norm": 0.1264657825231552, "loss_scale": 4.0, "retries": 0, "seconds": 268.5, "max_memory_gb": 17.08, "ce": 0.005747176473136051, "rank": 0.0}
|
| 73 |
+
{"step": 73, "steps": 250, "lr": 4.3934841913948323e-05, "grad_norm": 6.024387836456299, "loss_scale": 4.0, "retries": 0, "seconds": 272.1, "max_memory_gb": 17.08, "ce": 0.501506503960627, "rank": 0.0}
|
| 74 |
+
{"step": 74, "steps": 250, "lr": 4.372705506523583e-05, "grad_norm": 1.3161141872406006, "loss_scale": 4.0, "retries": 0, "seconds": 275.8, "max_memory_gb": 17.08, "ce": 0.0566429466016416, "rank": 0.0}
|
| 75 |
+
{"step": 75, "steps": 250, "lr": 4.351634338256017e-05, "grad_norm": 2.1727540493011475, "loss_scale": 4.0, "retries": 0, "seconds": 279.4, "max_memory_gb": 17.08, "ce": 0.09132462871275493, "rank": 0.0}
|
| 76 |
+
{"step": 76, "steps": 250, "lr": 4.330274484549825e-05, "grad_norm": 0.5036621689796448, "loss_scale": 4.0, "retries": 0, "seconds": 283.3, "max_memory_gb": 17.08, "ce": 0.022140061540994793, "rank": 0.0}
|
| 77 |
+
{"step": 77, "steps": 250, "lr": 4.308629795396599e-05, "grad_norm": 0.4948430061340332, "loss_scale": 4.0, "retries": 0, "seconds": 286.9, "max_memory_gb": 17.08, "ce": 0.022560834506890615, "rank": 0.0}
|
| 78 |
+
{"step": 78, "steps": 250, "lr": 4.286704172127894e-05, "grad_norm": 0.6844249963760376, "loss_scale": 4.0, "retries": 0, "seconds": 290.6, "max_memory_gb": 17.08, "ce": 0.0385938712972802, "rank": 0.0}
|
| 79 |
+
{"step": 79, "steps": 250, "lr": 4.264501566712033e-05, "grad_norm": 2.371554374694824, "loss_scale": 4.0, "retries": 0, "seconds": 294.3, "max_memory_gb": 17.08, "ce": 0.140361051816285, "rank": 0.0}
|
| 80 |
+
{"step": 80, "steps": 250, "lr": 4.2420259810417894e-05, "grad_norm": 1.7707152366638184, "loss_scale": 4.0, "retries": 0, "seconds": 298.0, "max_memory_gb": 17.08, "ce": 0.23150964447407318, "rank": 0.0}
|
| 81 |
+
{"step": 81, "steps": 250, "lr": 4.219281466213066e-05, "grad_norm": 5.059403896331787, "loss_scale": 4.0, "retries": 0, "seconds": 301.7, "max_memory_gb": 17.08, "ce": 0.6019475751090795, "rank": 0.0}
|
| 82 |
+
{"step": 82, "steps": 250, "lr": 4.1962721217947135e-05, "grad_norm": 1.0930150747299194, "loss_scale": 4.0, "retries": 0, "seconds": 305.4, "max_memory_gb": 17.08, "ce": 0.0672371670971188, "rank": 0.0}
|
| 83 |
+
{"step": 83, "steps": 250, "lr": 4.173002095089599e-05, "grad_norm": 1.3892749547958374, "loss_scale": 4.0, "retries": 0, "seconds": 309.2, "max_memory_gb": 17.08, "ce": 0.05632814221780791, "rank": 0.0}
|
| 84 |
+
{"step": 84, "steps": 250, "lr": 4.149475580387084e-05, "grad_norm": 2.906236171722412, "loss_scale": 4.0, "retries": 0, "seconds": 312.8, "max_memory_gb": 17.08, "ce": 0.17927354178715404, "rank": 0.0}
|
| 85 |
+
{"step": 85, "steps": 250, "lr": 4.1256968182070275e-05, "grad_norm": 0.5484040379524231, "loss_scale": 4.0, "retries": 0, "seconds": 316.5, "max_memory_gb": 17.08, "ce": 0.033825453167821706, "rank": 0.0}
|
| 86 |
+
{"step": 86, "steps": 250, "lr": 4.101670094535453e-05, "grad_norm": 0.6244305372238159, "loss_scale": 4.0, "retries": 0, "seconds": 320.2, "max_memory_gb": 17.08, "ce": 0.01270697947620647, "rank": 0.12482592463493347}
|
| 87 |
+
{"step": 87, "steps": 250, "lr": 4.077399740052027e-05, "grad_norm": 0.7291508913040161, "loss_scale": 4.0, "retries": 0, "seconds": 323.9, "max_memory_gb": 17.08, "ce": 0.025358703082929424, "rank": 0.0}
|
| 88 |
+
{"step": 88, "steps": 250, "lr": 4.0528901293494775e-05, "grad_norm": 1.9190280437469482, "loss_scale": 4.0, "retries": 0, "seconds": 327.5, "max_memory_gb": 17.08, "ce": 0.05959681924599636, "rank": 0.0}
|
| 89 |
+
{"step": 89, "steps": 250, "lr": 4.0281456801451004e-05, "grad_norm": 2.2947843074798584, "loss_scale": 4.0, "retries": 0, "seconds": 331.3, "max_memory_gb": 17.08, "ce": 0.03479913807849755, "rank": 0.0}
|
| 90 |
+
{"step": 90, "steps": 250, "lr": 4.0031708524844884e-05, "grad_norm": 0.2777138650417328, "loss_scale": 4.0, "retries": 0, "seconds": 335.0, "max_memory_gb": 17.08, "ce": 0.0006916315695093544, "rank": 0.10475580394268036}
|
| 91 |
+
{"step": 91, "steps": 250, "lr": 3.977970147937635e-05, "grad_norm": 0.47676122188568115, "loss_scale": 4.0, "retries": 0, "seconds": 338.7, "max_memory_gb": 17.08, "ce": 0.01536820729234023, "rank": 0.0}
|
| 92 |
+
{"step": 92, "steps": 250, "lr": 3.952548108787553e-05, "grad_norm": 0.3571583330631256, "loss_scale": 4.0, "retries": 0, "seconds": 342.4, "max_memory_gb": 17.08, "ce": 0.015083745498486678, "rank": 0.0}
|
| 93 |
+
{"step": 93, "steps": 250, "lr": 3.9269093172115496e-05, "grad_norm": 4.255301475524902, "loss_scale": 4.0, "retries": 0, "seconds": 346.2, "max_memory_gb": 17.08, "ce": 0.21922313540198957, "rank": 0.0}
|
| 94 |
+
{"step": 94, "steps": 250, "lr": 3.901058394455323e-05, "grad_norm": 4.409915447235107, "loss_scale": 4.0, "retries": 0, "seconds": 349.8, "max_memory_gb": 17.08, "ce": 0.24210555104582454, "rank": 0.0}
|
| 95 |
+
{"step": 95, "steps": 250, "lr": 3.875e-05, "grad_norm": 0.35147160291671753, "loss_scale": 4.0, "retries": 0, "seconds": 353.4, "max_memory_gb": 17.08, "ce": 0.014968147735999082, "rank": 0.0}
|
| 96 |
+
{"step": 96, "steps": 250, "lr": 3.8487388307223005e-05, "grad_norm": 0.9339438080787659, "loss_scale": 4.0, "retries": 0, "seconds": 357.3, "max_memory_gb": 17.08, "ce": 0.0675809818039852, "rank": 0.0}
|
| 97 |
+
{"step": 97, "steps": 250, "lr": 3.822279620047943e-05, "grad_norm": 1.6388819217681885, "loss_scale": 4.0, "retries": 0, "seconds": 360.9, "max_memory_gb": 17.08, "ce": 0.06639542199491189, "rank": 0.0}
|
| 98 |
+
{"step": 98, "steps": 250, "lr": 3.795627137098479e-05, "grad_norm": 0.1303391456604004, "loss_scale": 4.0, "retries": 0, "seconds": 364.6, "max_memory_gb": 17.08, "ce": 0.007446669624741276, "rank": 0.0}
|
| 99 |
+
{"step": 99, "steps": 250, "lr": 3.7687861858316844e-05, "grad_norm": 1.028083086013794, "loss_scale": 4.0, "retries": 0, "seconds": 368.4, "max_memory_gb": 17.08, "ce": 0.06883146855943778, "rank": 0.0}
|
| 100 |
+
{"step": 100, "steps": 250, "lr": 3.741761604175668e-05, "grad_norm": 0.08231966942548752, "loss_scale": 4.0, "retries": 0, "seconds": 372.1, "max_memory_gb": 17.08, "ce": 0.004325362248437159, "rank": 0.0}
|
| 101 |
+
{"step": 101, "steps": 250, "lr": 3.714558263156872e-05, "grad_norm": 2.442047595977783, "loss_scale": 4.0, "retries": 0, "seconds": 375.8, "max_memory_gb": 17.08, "ce": 0.19035508987741423, "rank": 0.0}
|
| 102 |
+
{"step": 102, "steps": 250, "lr": 3.6871810660220904e-05, "grad_norm": 0.49296972155570984, "loss_scale": 4.0, "retries": 0, "seconds": 379.5, "max_memory_gb": 17.08, "ce": 0.01632754669705605, "rank": 0.0}
|
| 103 |
+
{"step": 103, "steps": 250, "lr": 3.659634947354686e-05, "grad_norm": 2.837468147277832, "loss_scale": 4.0, "retries": 0, "seconds": 383.3, "max_memory_gb": 17.08, "ce": 0.2316885398067825, "rank": 0.0}
|
| 104 |
+
{"step": 104, "steps": 250, "lr": 3.631924872185169e-05, "grad_norm": 1.8574976921081543, "loss_scale": 4.0, "retries": 0, "seconds": 387.1, "max_memory_gb": 17.08, "ce": 0.1288706031550646, "rank": 0.0}
|
| 105 |
+
{"step": 105, "steps": 250, "lr": 3.6040558350962674e-05, "grad_norm": 6.677602767944336, "loss_scale": 4.0, "retries": 0, "seconds": 390.5, "max_memory_gb": 17.08, "ce": 0.3712064020606931, "rank": 0.0}
|
| 106 |
+
{"step": 106, "steps": 250, "lr": 3.576032859322687e-05, "grad_norm": 5.953583717346191, "loss_scale": 4.0, "retries": 0, "seconds": 393.9, "max_memory_gb": 17.08, "ce": 0.27858136402164746, "rank": 0.0}
|
| 107 |
+
{"step": 107, "steps": 250, "lr": 3.5478609958457056e-05, "grad_norm": 2.4366652965545654, "loss_scale": 4.0, "retries": 0, "seconds": 397.3, "max_memory_gb": 17.08, "ce": 0.22769702399273228, "rank": 0.0}
|
| 108 |
+
{"step": 108, "steps": 250, "lr": 3.519545322482755e-05, "grad_norm": 3.3124680519104004, "loss_scale": 4.0, "retries": 0, "seconds": 400.7, "max_memory_gb": 17.08, "ce": 0.25613929823339276, "rank": 0.0}
|
| 109 |
+
{"step": 109, "steps": 250, "lr": 3.491090942972179e-05, "grad_norm": 0.923081636428833, "loss_scale": 4.0, "retries": 0, "seconds": 404.1, "max_memory_gb": 17.08, "ce": 0.11402735775300243, "rank": 0.0}
|
| 110 |
+
{"step": 110, "steps": 250, "lr": 3.4625029860533135e-05, "grad_norm": 0.8095406889915466, "loss_scale": 4.0, "retries": 0, "seconds": 407.4, "max_memory_gb": 17.08, "ce": 0.01993023938666738, "rank": 0.0}
|
| 111 |
+
{"step": 111, "steps": 250, "lr": 3.433786604542056e-05, "grad_norm": 0.07486672699451447, "loss_scale": 4.0, "retries": 0, "seconds": 410.8, "max_memory_gb": 17.08, "ce": 0.0027182392648228415, "rank": 0.12500053644180298}
|
| 112 |
+
{"step": 112, "steps": 250, "lr": 3.404946974402104e-05, "grad_norm": 2.1774868965148926, "loss_scale": 4.0, "retries": 0, "seconds": 414.1, "max_memory_gb": 17.08, "ce": 0.09889988615441325, "rank": 0.0}
|
| 113 |
+
{"step": 113, "steps": 250, "lr": 3.375989293812019e-05, "grad_norm": 2.552816152572632, "loss_scale": 4.0, "retries": 0, "seconds": 417.6, "max_memory_gb": 17.08, "ce": 0.4590872088319884, "rank": 0.0}
|
| 114 |
+
{"step": 114, "steps": 250, "lr": 3.3469187822282766e-05, "grad_norm": 0.11627291142940521, "loss_scale": 4.0, "retries": 0, "seconds": 420.9, "max_memory_gb": 17.08, "ce": 0.007029657037492143, "rank": 0.0}
|
| 115 |
+
{"step": 115, "steps": 250, "lr": 3.317740679444499e-05, "grad_norm": 5.372796535491943, "loss_scale": 4.0, "retries": 0, "seconds": 424.2, "max_memory_gb": 17.08, "ce": 0.2223298092794721, "rank": 0.0}
|
| 116 |
+
{"step": 116, "steps": 250, "lr": 3.2884602446470056e-05, "grad_norm": 1.1435208320617676, "loss_scale": 4.0, "retries": 0, "seconds": 427.7, "max_memory_gb": 17.08, "ce": 0.03753424562273722, "rank": 0.0}
|
| 117 |
+
{"step": 117, "steps": 250, "lr": 3.259082755466876e-05, "grad_norm": 2.9922549724578857, "loss_scale": 4.0, "retries": 0, "seconds": 431.0, "max_memory_gb": 17.08, "ce": 0.30565739416124416, "rank": 0.0}
|
| 118 |
+
{"step": 118, "steps": 250, "lr": 3.229613507028686e-05, "grad_norm": 2.7325432300567627, "loss_scale": 4.0, "retries": 0, "seconds": 434.4, "max_memory_gb": 17.08, "ce": 0.2884217557093507, "rank": 0.1251503825187683}
|
| 119 |
+
{"step": 119, "steps": 250, "lr": 3.2000578109961004e-05, "grad_norm": 0.5185896754264832, "loss_scale": 4.0, "retries": 0, "seconds": 437.7, "max_memory_gb": 17.08, "ce": 0.028540259641090415, "rank": 0.0}
|
| 120 |
+
{"step": 120, "steps": 250, "lr": 3.1704209946144616e-05, "grad_norm": 0.843997597694397, "loss_scale": 4.0, "retries": 0, "seconds": 441.1, "max_memory_gb": 17.08, "ce": 0.022682330691168318, "rank": 0.0}
|
| 121 |
+
{"step": 121, "steps": 250, "lr": 3.140708399750594e-05, "grad_norm": 1.2205028533935547, "loss_scale": 4.0, "retries": 0, "seconds": 444.5, "max_memory_gb": 17.08, "ce": 0.049910030946648476, "rank": 0.0}
|
| 122 |
+
{"step": 122, "steps": 250, "lr": 3.1109253819299606e-05, "grad_norm": 0.30344337224960327, "loss_scale": 4.0, "retries": 0, "seconds": 448.2, "max_memory_gb": 17.08, "ce": 0.019071242687459744, "rank": 0.0}
|
| 123 |
+
{"step": 123, "steps": 250, "lr": 3.081077309371361e-05, "grad_norm": 0.3827250301837921, "loss_scale": 4.0, "retries": 0, "seconds": 452.1, "max_memory_gb": 17.08, "ce": 0.011806804422121786, "rank": 0.0}
|
| 124 |
+
{"step": 124, "steps": 250, "lr": 3.051169562019339e-05, "grad_norm": 0.5752105712890625, "loss_scale": 4.0, "retries": 0, "seconds": 455.8, "max_memory_gb": 17.08, "ce": 0.11377881802400225, "rank": 0.12688499689102173}
|
| 125 |
+
{"step": 125, "steps": 250, "lr": 3.021207530574477e-05, "grad_norm": 2.7292113304138184, "loss_scale": 4.0, "retries": 0, "seconds": 459.4, "max_memory_gb": 17.08, "ce": 0.027279478639684385, "rank": 0.0}
|
| 126 |
+
{"step": 126, "steps": 250, "lr": 2.991196615521758e-05, "grad_norm": 2.502009391784668, "loss_scale": 4.0, "retries": 0, "seconds": 463.2, "max_memory_gb": 17.08, "ce": 0.46293273950686853, "rank": 0.0}
|
| 127 |
+
{"step": 127, "steps": 250, "lr": 2.9611422261571536e-05, "grad_norm": 1.358400821685791, "loss_scale": 4.0, "retries": 0, "seconds": 467.1, "max_memory_gb": 17.08, "ce": 0.08879294545477023, "rank": 0.0}
|
| 128 |
+
{"step": 128, "steps": 250, "lr": 2.931049779612634e-05, "grad_norm": 0.6845846176147461, "loss_scale": 4.0, "retries": 0, "seconds": 470.9, "max_memory_gb": 17.08, "ce": 0.025588656486434047, "rank": 0.005725786089897156}
|
| 129 |
+
{"step": 129, "steps": 250, "lr": 2.9009246998797557e-05, "grad_norm": 5.410309314727783, "loss_scale": 4.0, "retries": 0, "seconds": 474.7, "max_memory_gb": 17.08, "ce": 0.24996990312138223, "rank": 0.0}
|
| 130 |
+
{"step": 130, "steps": 250, "lr": 2.8707724168320238e-05, "grad_norm": 0.6144523620605469, "loss_scale": 4.0, "retries": 0, "seconds": 478.5, "max_memory_gb": 17.08, "ce": 0.029322030875846394, "rank": 0.0}
|
| 131 |
+
{"step": 131, "steps": 250, "lr": 2.840598365246184e-05, "grad_norm": 1.0195485353469849, "loss_scale": 4.0, "retries": 0, "seconds": 482.3, "max_memory_gb": 17.08, "ce": 0.14057443487581622, "rank": 0.0}
|
| 132 |
+
{"step": 132, "steps": 250, "lr": 2.8104079838226383e-05, "grad_norm": 0.054054949432611465, "loss_scale": 4.0, "retries": 0, "seconds": 486.0, "max_memory_gb": 17.08, "ce": 0.0013362859367589408, "rank": 0.0}
|
| 133 |
+
{"step": 133, "steps": 250, "lr": 2.7802067142051446e-05, "grad_norm": 0.14806413650512695, "loss_scale": 4.0, "retries": 0, "seconds": 489.9, "max_memory_gb": 17.08, "ce": 0.00633954012664617, "rank": 0.0}
|
| 134 |
+
{"step": 134, "steps": 250, "lr": 2.7500000000000004e-05, "grad_norm": 1.2403080463409424, "loss_scale": 4.0, "retries": 0, "seconds": 493.6, "max_memory_gb": 17.08, "ce": 0.014573276397641166, "rank": 0.0}
|
| 135 |
+
{"step": 135, "steps": 250, "lr": 2.7197932857948566e-05, "grad_norm": 2.508633852005005, "loss_scale": 4.0, "retries": 0, "seconds": 497.4, "max_memory_gb": 17.08, "ce": 0.41415587738993054, "rank": 0.0}
|
| 136 |
+
{"step": 136, "steps": 250, "lr": 2.689592016177363e-05, "grad_norm": 0.2264193743467331, "loss_scale": 4.0, "retries": 0, "seconds": 501.3, "max_memory_gb": 17.08, "ce": 0.006598836811463116, "rank": 0.0}
|
| 137 |
+
{"step": 137, "steps": 250, "lr": 2.659401634753817e-05, "grad_norm": 4.172929763793945, "loss_scale": 4.0, "retries": 0, "seconds": 505.1, "max_memory_gb": 17.08, "ce": 0.27033484507046524, "rank": 0.0}
|
| 138 |
+
{"step": 138, "steps": 250, "lr": 2.6292275831679764e-05, "grad_norm": 0.2873633801937103, "loss_scale": 4.0, "retries": 0, "seconds": 508.9, "max_memory_gb": 17.08, "ce": 0.012857941674155882, "rank": 0.0}
|
| 139 |
+
{"step": 139, "steps": 250, "lr": 2.5990753001202445e-05, "grad_norm": 2.488574743270874, "loss_scale": 4.0, "retries": 0, "seconds": 512.6, "max_memory_gb": 17.08, "ce": 0.2535317399615451, "rank": 0.0}
|
| 140 |
+
{"step": 140, "steps": 250, "lr": 2.5689502203873672e-05, "grad_norm": 0.1408565491437912, "loss_scale": 4.0, "retries": 0, "seconds": 516.1, "max_memory_gb": 17.08, "ce": 0.004808668760233559, "rank": 0.0}
|
| 141 |
+
{"step": 141, "steps": 250, "lr": 2.538857773842847e-05, "grad_norm": 1.7373452186584473, "loss_scale": 4.0, "retries": 0, "seconds": 519.6, "max_memory_gb": 17.08, "ce": 0.06701559333487239, "rank": 0.0}
|
| 142 |
+
{"step": 142, "steps": 250, "lr": 2.5088033844782426e-05, "grad_norm": 0.36946403980255127, "loss_scale": 4.0, "retries": 0, "seconds": 523.0, "max_memory_gb": 17.08, "ce": 0.021190516717979335, "rank": 0.0}
|
| 143 |
+
{"step": 143, "steps": 250, "lr": 2.4787924694255237e-05, "grad_norm": 1.3236507177352905, "loss_scale": 4.0, "retries": 0, "seconds": 526.5, "max_memory_gb": 17.08, "ce": 0.12140855767756875, "rank": 0.0}
|
| 144 |
+
{"step": 144, "steps": 250, "lr": 2.4488304379806627e-05, "grad_norm": 0.649997889995575, "loss_scale": 4.0, "retries": 0, "seconds": 529.9, "max_memory_gb": 17.08, "ce": 0.05001902867661556, "rank": 0.1252065896987915}
|
| 145 |
+
{"step": 145, "steps": 250, "lr": 2.4189226906286388e-05, "grad_norm": 0.18336111307144165, "loss_scale": 4.0, "retries": 0, "seconds": 533.2, "max_memory_gb": 17.08, "ce": 0.007689519110044785, "rank": 0.0}
|
| 146 |
+
{"step": 146, "steps": 250, "lr": 2.3890746180700395e-05, "grad_norm": 0.3644232749938965, "loss_scale": 4.0, "retries": 0, "seconds": 537.0, "max_memory_gb": 17.08, "ce": 0.004792083520442247, "rank": 0.0}
|
| 147 |
+
{"step": 147, "steps": 250, "lr": 2.359291600249407e-05, "grad_norm": 1.613411545753479, "loss_scale": 4.0, "retries": 0, "seconds": 540.8, "max_memory_gb": 17.08, "ce": 0.06977198562890408, "rank": 0.0}
|
| 148 |
+
{"step": 148, "steps": 250, "lr": 2.3295790053855392e-05, "grad_norm": 10.262188911437988, "loss_scale": 4.0, "retries": 0, "seconds": 544.6, "max_memory_gb": 17.08, "ce": 0.30471571309772116, "rank": 0.0}
|
| 149 |
+
{"step": 149, "steps": 250, "lr": 2.299942189003901e-05, "grad_norm": 3.5242879390716553, "loss_scale": 4.0, "retries": 0, "seconds": 548.3, "max_memory_gb": 17.08, "ce": 0.2510504493220651, "rank": 0.0}
|
| 150 |
+
{"step": 150, "steps": 250, "lr": 2.2703864929713144e-05, "grad_norm": 0.9484810829162598, "loss_scale": 4.0, "retries": 0, "seconds": 552.2, "max_memory_gb": 17.08, "ce": 0.0466997891230676, "rank": 0.0}
|
| 151 |
+
{"step": 151, "steps": 250, "lr": 2.2409172445331257e-05, "grad_norm": 0.31570279598236084, "loss_scale": 4.0, "retries": 0, "seconds": 556.0, "max_memory_gb": 17.08, "ce": 0.007162807729400811, "rank": 0.0}
|
| 152 |
+
{"step": 152, "steps": 250, "lr": 2.211539755352996e-05, "grad_norm": 6.103851318359375, "loss_scale": 4.0, "retries": 0, "seconds": 559.7, "max_memory_gb": 17.08, "ce": 0.17480744202293863, "rank": 0.12088407576084137}
|
| 153 |
+
{"step": 153, "steps": 250, "lr": 2.1822593205555016e-05, "grad_norm": 1.632975459098816, "loss_scale": 4.0, "retries": 0, "seconds": 563.3, "max_memory_gb": 17.08, "ce": 0.03818129879891785, "rank": 0.0}
|
| 154 |
+
{"step": 154, "steps": 250, "lr": 2.1530812177717236e-05, "grad_norm": 0.12479119747877121, "loss_scale": 4.0, "retries": 0, "seconds": 566.7, "max_memory_gb": 17.08, "ce": 0.004157760954967671, "rank": 0.0}
|
| 155 |
+
{"step": 155, "steps": 250, "lr": 2.1240107061879815e-05, "grad_norm": 0.8841564655303955, "loss_scale": 4.0, "retries": 0, "seconds": 570.1, "max_memory_gb": 17.08, "ce": 0.08812126201337378, "rank": 0.0}
|
| 156 |
+
{"step": 156, "steps": 250, "lr": 2.095053025597896e-05, "grad_norm": 0.8801204562187195, "loss_scale": 4.0, "retries": 0, "seconds": 573.5, "max_memory_gb": 17.08, "ce": 0.06500660124584101, "rank": 0.0}
|
| 157 |
+
{"step": 157, "steps": 250, "lr": 2.066213395457945e-05, "grad_norm": 0.3231325149536133, "loss_scale": 4.0, "retries": 0, "seconds": 577.0, "max_memory_gb": 17.08, "ce": 0.003637488371168729, "rank": 0.0}
|
| 158 |
+
{"step": 158, "steps": 250, "lr": 2.0374970139466877e-05, "grad_norm": 1.060255527496338, "loss_scale": 4.0, "retries": 0, "seconds": 580.4, "max_memory_gb": 17.08, "ce": 0.0902011103607947, "rank": 0.0}
|
| 159 |
+
{"step": 159, "steps": 250, "lr": 2.0089090570278213e-05, "grad_norm": 0.21448887884616852, "loss_scale": 4.0, "retries": 0, "seconds": 583.8, "max_memory_gb": 17.08, "ce": 0.009733003871588153, "rank": 0.0}
|
| 160 |
+
{"step": 160, "steps": 250, "lr": 1.9804546775172457e-05, "grad_norm": 12.542317390441895, "loss_scale": 4.0, "retries": 0, "seconds": 587.3, "max_memory_gb": 17.08, "ce": 0.2310448988027929, "rank": 0.15337270498275757}
|
| 161 |
+
{"step": 161, "steps": 250, "lr": 1.9521390041542946e-05, "grad_norm": 2.948892116546631, "loss_scale": 4.0, "retries": 0, "seconds": 590.6, "max_memory_gb": 17.08, "ce": 0.10614285029078019, "rank": 0.0}
|
| 162 |
+
{"step": 162, "steps": 250, "lr": 1.923967140677313e-05, "grad_norm": 1.3973559141159058, "loss_scale": 4.0, "retries": 0, "seconds": 594.0, "max_memory_gb": 17.08, "ce": 0.0201564408826016, "rank": 0.0}
|
| 163 |
+
{"step": 163, "steps": 250, "lr": 1.8959441649037335e-05, "grad_norm": 0.4208316206932068, "loss_scale": 4.0, "retries": 0, "seconds": 597.4, "max_memory_gb": 17.08, "ce": 0.050112908424125635, "rank": 0.0}
|
| 164 |
+
{"step": 164, "steps": 250, "lr": 1.868075127814832e-05, "grad_norm": 1.1145272254943848, "loss_scale": 4.0, "retries": 0, "seconds": 600.8, "max_memory_gb": 17.08, "ce": 0.10478596222674241, "rank": 0.0}
|
| 165 |
+
{"step": 165, "steps": 250, "lr": 1.8403650526453143e-05, "grad_norm": 0.22797051072120667, "loss_scale": 4.0, "retries": 0, "seconds": 604.1, "max_memory_gb": 17.08, "ce": 0.008360287987670745, "rank": 0.0}
|
| 166 |
+
{"step": 166, "steps": 250, "lr": 1.812818933977912e-05, "grad_norm": 0.4558038115501404, "loss_scale": 4.0, "retries": 0, "seconds": 607.5, "max_memory_gb": 17.08, "ce": 0.022554795368705527, "rank": 0.0}
|
| 167 |
+
{"step": 167, "steps": 250, "lr": 1.785441736843128e-05, "grad_norm": 0.06506755203008652, "loss_scale": 4.0, "retries": 0, "seconds": 610.9, "max_memory_gb": 17.08, "ce": 0.0019667260609139703, "rank": 0.0}
|
| 168 |
+
{"step": 168, "steps": 250, "lr": 1.7582383958243314e-05, "grad_norm": 0.7714893817901611, "loss_scale": 4.0, "retries": 0, "seconds": 614.3, "max_memory_gb": 17.08, "ce": 0.0767023688131303, "rank": 0.0}
|
| 169 |
+
{"step": 169, "steps": 250, "lr": 1.7312138141683164e-05, "grad_norm": 2.406773328781128, "loss_scale": 4.0, "retries": 0, "seconds": 617.6, "max_memory_gb": 17.08, "ce": 0.10203207028462202, "rank": 0.24972619861364365}
|
| 170 |
+
{"step": 170, "steps": 250, "lr": 1.704372862901521e-05, "grad_norm": 0.7838379740715027, "loss_scale": 4.0, "retries": 0, "seconds": 621.1, "max_memory_gb": 17.08, "ce": 0.05663866359009262, "rank": 0.0}
|
| 171 |
+
{"step": 171, "steps": 250, "lr": 1.6777203799520573e-05, "grad_norm": 0.69991534948349, "loss_scale": 4.0, "retries": 0, "seconds": 624.4, "max_memory_gb": 17.08, "ce": 0.03518043402891635, "rank": 0.0}
|
| 172 |
+
{"step": 172, "steps": 250, "lr": 1.6512611692777008e-05, "grad_norm": 2.691805839538574, "loss_scale": 4.0, "retries": 0, "seconds": 627.8, "max_memory_gb": 17.08, "ce": 0.13451774742497946, "rank": 0.0}
|
| 173 |
+
{"step": 173, "steps": 250, "lr": 1.6250000000000005e-05, "grad_norm": 1.3104779720306396, "loss_scale": 4.0, "retries": 0, "seconds": 631.3, "max_memory_gb": 17.08, "ce": 0.12241476798953954, "rank": 0.0}
|
| 174 |
+
{"step": 174, "steps": 250, "lr": 1.598941605544678e-05, "grad_norm": 10.038827896118164, "loss_scale": 4.0, "retries": 0, "seconds": 634.6, "max_memory_gb": 17.08, "ce": 0.20936825750504795, "rank": 0.0}
|
| 175 |
+
{"step": 175, "steps": 250, "lr": 1.5730906827884506e-05, "grad_norm": 0.5475443005561829, "loss_scale": 4.0, "retries": 0, "seconds": 638.0, "max_memory_gb": 17.08, "ce": 0.026533031932558515, "rank": 0.0}
|
| 176 |
+
{"step": 176, "steps": 250, "lr": 1.5474518912124476e-05, "grad_norm": 0.8795338869094849, "loss_scale": 4.0, "retries": 0, "seconds": 641.3, "max_memory_gb": 17.08, "ce": 0.07376623892560019, "rank": 0.0}
|
| 177 |
+
{"step": 177, "steps": 250, "lr": 1.5220298520623646e-05, "grad_norm": 4.907354354858398, "loss_scale": 4.0, "retries": 0, "seconds": 644.8, "max_memory_gb": 17.08, "ce": 0.17269844293923597, "rank": 0.018148988485336304}
|
| 178 |
+
{"step": 178, "steps": 250, "lr": 1.4968291475155121e-05, "grad_norm": 1.116195559501648, "loss_scale": 4.0, "retries": 0, "seconds": 648.1, "max_memory_gb": 17.08, "ce": 0.06384910623864926, "rank": 0.0}
|
| 179 |
+
{"step": 179, "steps": 250, "lr": 1.4718543198548998e-05, "grad_norm": 0.7337468862533569, "loss_scale": 4.0, "retries": 0, "seconds": 651.4, "max_memory_gb": 17.08, "ce": 0.056439761567389723, "rank": 0.0}
|
| 180 |
+
{"step": 180, "steps": 250, "lr": 1.4471098706505227e-05, "grad_norm": 1.6447030305862427, "loss_scale": 4.0, "retries": 0, "seconds": 654.9, "max_memory_gb": 17.08, "ce": 0.2013918791717515, "rank": 0.0}
|
| 181 |
+
{"step": 181, "steps": 250, "lr": 1.4226002599479743e-05, "grad_norm": 4.876752853393555, "loss_scale": 4.0, "retries": 0, "seconds": 658.3, "max_memory_gb": 17.08, "ce": 0.2767501828911918, "rank": 0.0}
|
| 182 |
+
{"step": 182, "steps": 250, "lr": 1.3983299054645471e-05, "grad_norm": 4.952079772949219, "loss_scale": 4.0, "retries": 0, "seconds": 661.6, "max_memory_gb": 17.08, "ce": 0.15986255385087134, "rank": 0.0}
|
| 183 |
+
{"step": 183, "steps": 250, "lr": 1.3743031817929724e-05, "grad_norm": 0.16274717450141907, "loss_scale": 4.0, "retries": 0, "seconds": 665.1, "max_memory_gb": 17.08, "ce": 0.003256687860812235, "rank": 0.0}
|
| 184 |
+
{"step": 184, "steps": 250, "lr": 1.3505244196129164e-05, "grad_norm": 0.01628054305911064, "loss_scale": 4.0, "retries": 0, "seconds": 668.5, "max_memory_gb": 17.08, "ce": 0.0004887616591986443, "rank": 0.0}
|
| 185 |
+
{"step": 185, "steps": 250, "lr": 1.3269979049104014e-05, "grad_norm": 0.693131148815155, "loss_scale": 4.0, "retries": 0, "seconds": 671.8, "max_memory_gb": 17.08, "ce": 0.0005449626687550335, "rank": 0.07005800306797028}
|
| 186 |
+
{"step": 186, "steps": 250, "lr": 1.3037278782052863e-05, "grad_norm": 0.5983429551124573, "loss_scale": 4.0, "retries": 0, "seconds": 675.2, "max_memory_gb": 17.08, "ce": 0.026087008374815923, "rank": 0.0}
|
| 187 |
+
{"step": 187, "steps": 250, "lr": 1.2807185337869343e-05, "grad_norm": 5.718741416931152, "loss_scale": 4.0, "retries": 0, "seconds": 678.7, "max_memory_gb": 17.08, "ce": 0.15373649974117143, "rank": 0.0}
|
| 188 |
+
{"step": 188, "steps": 250, "lr": 1.2579740189582115e-05, "grad_norm": 1.106239914894104, "loss_scale": 4.0, "retries": 0, "seconds": 682.0, "max_memory_gb": 17.08, "ce": 0.09863125428273634, "rank": 0.0}
|
| 189 |
+
{"step": 189, "steps": 250, "lr": 1.2354984332879668e-05, "grad_norm": 1.4475860595703125, "loss_scale": 4.0, "retries": 0, "seconds": 685.4, "max_memory_gb": 17.08, "ce": 0.05777315977047692, "rank": 0.0}
|
| 190 |
+
{"step": 190, "steps": 250, "lr": 1.2132958278721064e-05, "grad_norm": 0.4179704189300537, "loss_scale": 4.0, "retries": 0, "seconds": 688.9, "max_memory_gb": 17.08, "ce": 0.012697505757387262, "rank": 0.1251913607120514}
|
| 191 |
+
{"step": 191, "steps": 250, "lr": 1.1913702046034016e-05, "grad_norm": 0.15236879885196686, "loss_scale": 4.0, "retries": 0, "seconds": 692.2, "max_memory_gb": 17.08, "ce": 0.0024526345200683863, "rank": 0.0}
|
| 192 |
+
{"step": 192, "steps": 250, "lr": 1.1697255154501756e-05, "grad_norm": 2.993042469024658, "loss_scale": 4.0, "retries": 0, "seconds": 695.5, "max_memory_gb": 17.08, "ce": 0.4127285974655024, "rank": 0.0}
|
| 193 |
+
{"step": 193, "steps": 250, "lr": 1.1483656617439842e-05, "grad_norm": 0.8944069147109985, "loss_scale": 4.0, "retries": 0, "seconds": 698.9, "max_memory_gb": 17.08, "ce": 0.047658917595072126, "rank": 0.0}
|
| 194 |
+
{"step": 194, "steps": 250, "lr": 1.1272944934764177e-05, "grad_norm": 0.4083530604839325, "loss_scale": 4.0, "retries": 0, "seconds": 702.4, "max_memory_gb": 17.08, "ce": 0.012167166163749243, "rank": 0.12497194856405258}
|
| 195 |
+
{"step": 195, "steps": 250, "lr": 1.1065158086051684e-05, "grad_norm": 1.1337858438491821, "loss_scale": 4.0, "retries": 0, "seconds": 705.8, "max_memory_gb": 17.08, "ce": 0.013764591772087442, "rank": 0.0}
|
| 196 |
+
{"step": 196, "steps": 250, "lr": 1.086033352369467e-05, "grad_norm": 0.025276098400354385, "loss_scale": 4.0, "retries": 0, "seconds": 709.1, "max_memory_gb": 17.08, "ce": 0.0007128690340323374, "rank": 0.0}
|
| 197 |
+
{"step": 197, "steps": 250, "lr": 1.0658508166150224e-05, "grad_norm": 0.46014076471328735, "loss_scale": 4.0, "retries": 0, "seconds": 712.6, "max_memory_gb": 17.08, "ce": 0.015668416113840067, "rank": 0.0}
|
| 198 |
+
{"step": 198, "steps": 250, "lr": 1.0459718391285954e-05, "grad_norm": 0.7143874168395996, "loss_scale": 4.0, "retries": 0, "seconds": 716.0, "max_memory_gb": 17.08, "ce": 0.030972265814853017, "rank": 0.0}
|
| 199 |
+
{"step": 199, "steps": 250, "lr": 1.0264000029822999e-05, "grad_norm": 1.619291067123413, "loss_scale": 4.0, "retries": 0, "seconds": 719.3, "max_memory_gb": 17.08, "ce": 0.14967216935292527, "rank": 0.0}
|
| 200 |
+
{"step": 200, "steps": 250, "lr": 1.0071388358877775e-05, "grad_norm": 0.9720853567123413, "loss_scale": 4.0, "retries": 0, "seconds": 722.8, "max_memory_gb": 17.08, "ce": 0.12566443376454117, "rank": 0.0}
|
| 201 |
+
{"step": 201, "steps": 250, "lr": 9.881918095603501e-06, "grad_norm": 0.20511457324028015, "loss_scale": 4.0, "retries": 0, "seconds": 726.2, "max_memory_gb": 17.08, "ce": 0.007077227845002199, "rank": 0.0}
|
| 202 |
+
{"step": 202, "steps": 250, "lr": 9.695623390932612e-06, "grad_norm": 1.242024540901184, "loss_scale": 4.0, "retries": 0, "seconds": 729.6, "max_memory_gb": 17.08, "ce": 0.1455054657499204, "rank": 0.0}
|
| 203 |
+
{"step": 203, "steps": 250, "lr": 9.51253782342123e-06, "grad_norm": 0.02686779946088791, "loss_scale": 4.0, "retries": 0, "seconds": 732.9, "max_memory_gb": 17.08, "ce": 0.0011050359289583866, "rank": 0.0}
|
| 204 |
+
{"step": 204, "steps": 250, "lr": 9.332694393196846e-06, "grad_norm": 0.8342055678367615, "loss_scale": 4.0, "retries": 0, "seconds": 736.4, "max_memory_gb": 17.08, "ce": 0.06345439583674306, "rank": 0.0}
|
| 205 |
+
{"step": 205, "steps": 250, "lr": 9.156125516010256e-06, "grad_norm": 0.08273044973611832, "loss_scale": 4.0, "retries": 0, "seconds": 739.7, "max_memory_gb": 17.08, "ce": 0.0025006004607348586, "rank": 0.0}
|
| 206 |
+
{"step": 206, "steps": 250, "lr": 8.982863017392733e-06, "grad_norm": 0.5884702205657959, "loss_scale": 4.0, "retries": 0, "seconds": 743.1, "max_memory_gb": 17.08, "ce": 0.024216581434302498, "rank": 0.0}
|
| 207 |
+
{"step": 207, "steps": 250, "lr": 8.81293812691974e-06, "grad_norm": 0.593450129032135, "loss_scale": 4.0, "retries": 0, "seconds": 746.6, "max_memory_gb": 17.08, "ce": 0.02060092550823356, "rank": 0.0}
|
| 208 |
+
{"step": 208, "steps": 250, "lr": 8.646381472581927e-06, "grad_norm": 0.8610337376594543, "loss_scale": 4.0, "retries": 0, "seconds": 750.0, "max_memory_gb": 17.08, "ce": 0.04000380436627893, "rank": 0.0}
|
| 209 |
+
{"step": 209, "steps": 250, "lr": 8.483223075264622e-06, "grad_norm": 0.55583655834198, "loss_scale": 4.0, "retries": 0, "seconds": 753.3, "max_memory_gb": 17.08, "ce": 0.015219948077174195, "rank": 0.0}
|
| 210 |
+
{"step": 210, "steps": 250, "lr": 8.323492343336747e-06, "grad_norm": 0.9619877338409424, "loss_scale": 4.0, "retries": 0, "seconds": 756.8, "max_memory_gb": 17.08, "ce": 0.041084015999331314, "rank": 0.0}
|
| 211 |
+
{"step": 211, "steps": 250, "lr": 8.167218067350122e-06, "grad_norm": 0.5133141875267029, "loss_scale": 4.0, "retries": 0, "seconds": 760.2, "max_memory_gb": 17.08, "ce": 0.01904812776047038, "rank": 0.0}
|
| 212 |
+
{"step": 212, "steps": 250, "lr": 8.01442841485013e-06, "grad_norm": 2.1792540550231934, "loss_scale": 4.0, "retries": 0, "seconds": 763.5, "max_memory_gb": 17.08, "ce": 0.20954018235624972, "rank": 0.0}
|
| 213 |
+
{"step": 213, "steps": 250, "lr": 7.865150925298678e-06, "grad_norm": 2.7072744369506836, "loss_scale": 4.0, "retries": 0, "seconds": 766.9, "max_memory_gb": 17.08, "ce": 0.5927372452806594, "rank": 0.0}
|
| 214 |
+
{"step": 214, "steps": 250, "lr": 7.719412505110375e-06, "grad_norm": 0.17309357225894928, "loss_scale": 4.0, "retries": 0, "seconds": 770.4, "max_memory_gb": 17.08, "ce": 0.005844520608661696, "rank": 0.0}
|
| 215 |
+
{"step": 215, "steps": 250, "lr": 7.57723942280278e-06, "grad_norm": 0.7680523991584778, "loss_scale": 4.0, "retries": 0, "seconds": 773.7, "max_memory_gb": 17.08, "ce": 0.015691469998273533, "rank": 0.0}
|
| 216 |
+
{"step": 216, "steps": 250, "lr": 7.438657304261687e-06, "grad_norm": 1.2900086641311646, "loss_scale": 4.0, "retries": 0, "seconds": 777.1, "max_memory_gb": 17.08, "ce": 0.02138908813958551, "rank": 0.0}
|
| 217 |
+
{"step": 217, "steps": 250, "lr": 7.30369112812219e-06, "grad_norm": 2.0110223293304443, "loss_scale": 4.0, "retries": 0, "seconds": 780.6, "max_memory_gb": 17.08, "ce": 0.07032519643053092, "rank": 0.12475273013114929}
|
| 218 |
+
{"step": 218, "steps": 250, "lr": 7.172365221266403e-06, "grad_norm": 6.471837997436523, "loss_scale": 4.0, "retries": 0, "seconds": 783.9, "max_memory_gb": 17.08, "ce": 0.3861814311321723, "rank": 0.0}
|
| 219 |
+
{"step": 219, "steps": 250, "lr": 7.0447032544387154e-06, "grad_norm": 0.19589726626873016, "loss_scale": 4.0, "retries": 0, "seconds": 787.3, "max_memory_gb": 17.08, "ce": 0.008034634012801689, "rank": 0.0}
|
| 220 |
+
{"step": 220, "steps": 250, "lr": 6.920728237979274e-06, "grad_norm": 2.3276848793029785, "loss_scale": 4.0, "retries": 0, "seconds": 790.8, "max_memory_gb": 17.08, "ce": 0.11760847874984393, "rank": 0.0}
|
| 221 |
+
{"step": 221, "steps": 250, "lr": 6.800462517676456e-06, "grad_norm": 4.254265785217285, "loss_scale": 4.0, "retries": 0, "seconds": 794.2, "max_memory_gb": 17.08, "ce": 0.3940272539330181, "rank": 0.0}
|
| 222 |
+
{"step": 222, "steps": 250, "lr": 6.683927770739222e-06, "grad_norm": 0.4847339987754822, "loss_scale": 4.0, "retries": 0, "seconds": 797.6, "max_memory_gb": 17.08, "ce": 0.013591807201919437, "rank": 0.0}
|
| 223 |
+
{"step": 223, "steps": 250, "lr": 6.571145001889892e-06, "grad_norm": 5.56392240524292, "loss_scale": 4.0, "retries": 0, "seconds": 801.0, "max_memory_gb": 17.08, "ce": 0.12411647256340075, "rank": 0.0}
|
| 224 |
+
{"step": 224, "steps": 250, "lr": 6.46213453957817e-06, "grad_norm": 0.01722959615290165, "loss_scale": 4.0, "retries": 0, "seconds": 804.5, "max_memory_gb": 17.08, "ce": 0.00044337462622934254, "rank": 0.0}
|
| 225 |
+
{"step": 225, "steps": 250, "lr": 6.356916032317063e-06, "grad_norm": 1.5112788677215576, "loss_scale": 4.0, "retries": 0, "seconds": 807.9, "max_memory_gb": 17.08, "ce": 0.15694166944103927, "rank": 0.0}
|
| 226 |
+
{"step": 226, "steps": 250, "lr": 6.255508445141349e-06, "grad_norm": 1.0605710744857788, "loss_scale": 4.0, "retries": 0, "seconds": 811.3, "max_memory_gb": 17.08, "ce": 0.060005622699463856, "rank": 0.0}
|
| 227 |
+
{"step": 227, "steps": 250, "lr": 6.157930056189228e-06, "grad_norm": 0.10075467079877853, "loss_scale": 4.0, "retries": 0, "seconds": 814.8, "max_memory_gb": 17.08, "ce": 0.0044211891527083935, "rank": 0.0}
|
| 228 |
+
{"step": 228, "steps": 250, "lr": 6.064198453407796e-06, "grad_norm": 1.1266297101974487, "loss_scale": 4.0, "retries": 0, "seconds": 818.2, "max_memory_gb": 17.08, "ce": 0.061759709802572615, "rank": 0.0}
|
| 229 |
+
{"step": 229, "steps": 250, "lr": 5.974330531382935e-06, "grad_norm": 3.4966084957122803, "loss_scale": 4.0, "retries": 0, "seconds": 821.6, "max_memory_gb": 17.08, "ce": 0.5309390828151663, "rank": 0.0}
|
| 230 |
+
{"step": 230, "steps": 250, "lr": 5.888342488294125e-06, "grad_norm": 0.19268304109573364, "loss_scale": 4.0, "retries": 0, "seconds": 825.0, "max_memory_gb": 17.08, "ce": 0.010529255771416501, "rank": 0.0}
|
| 231 |
+
{"step": 231, "steps": 250, "lr": 5.806249822994849e-06, "grad_norm": 6.078901290893555, "loss_scale": 4.0, "retries": 0, "seconds": 828.5, "max_memory_gb": 17.08, "ce": 0.19319520480235042, "rank": 0.0}
|
| 232 |
+
{"step": 232, "steps": 250, "lr": 5.728067332218994e-06, "grad_norm": 0.8665956258773804, "loss_scale": 4.0, "retries": 0, "seconds": 831.9, "max_memory_gb": 17.08, "ce": 0.0066945358285011025, "rank": 0.069107286632061}
|
| 233 |
+
{"step": 233, "steps": 250, "lr": 5.65380910791383e-06, "grad_norm": 0.33760741353034973, "loss_scale": 4.0, "retries": 0, "seconds": 835.3, "max_memory_gb": 17.08, "ce": 0.019206892913189222, "rank": 0.0}
|
| 234 |
+
{"step": 234, "steps": 250, "lr": 5.583488534700022e-06, "grad_norm": 2.8919403553009033, "loss_scale": 4.0, "retries": 0, "seconds": 838.8, "max_memory_gb": 17.08, "ce": 0.04680053432366549, "rank": 0.0}
|
| 235 |
+
{"step": 235, "steps": 250, "lr": 5.517118287459118e-06, "grad_norm": 10.424700736999512, "loss_scale": 4.0, "retries": 0, "seconds": 842.2, "max_memory_gb": 17.08, "ce": 0.4837765186402976, "rank": 0.0}
|
| 236 |
+
{"step": 236, "steps": 250, "lr": 5.454710329048978e-06, "grad_norm": 0.6367963552474976, "loss_scale": 4.0, "retries": 0, "seconds": 845.6, "max_memory_gb": 17.08, "ce": 0.056418620490148896, "rank": 0.0}
|
| 237 |
+
{"step": 237, "steps": 250, "lr": 5.3962759081475436e-06, "grad_norm": 1.2725841999053955, "loss_scale": 4.0, "retries": 0, "seconds": 849.1, "max_memory_gb": 17.08, "ce": 0.17401076815667693, "rank": 0.0}
|
| 238 |
+
{"step": 238, "steps": 250, "lr": 5.341825557225321e-06, "grad_norm": 4.711215019226074, "loss_scale": 4.0, "retries": 0, "seconds": 852.5, "max_memory_gb": 17.08, "ce": 0.4584649802475269, "rank": 0.0}
|
| 239 |
+
{"step": 239, "steps": 250, "lr": 5.291369090646961e-06, "grad_norm": 1.3098196983337402, "loss_scale": 4.0, "retries": 0, "seconds": 855.9, "max_memory_gb": 17.08, "ce": 0.028632254256564238, "rank": 0.0}
|
| 240 |
+
{"step": 240, "steps": 250, "lr": 5.244915602902289e-06, "grad_norm": 4.919606685638428, "loss_scale": 4.0, "retries": 0, "seconds": 859.3, "max_memory_gb": 17.08, "ce": 0.5874380725381343, "rank": 0.0}
|
| 241 |
+
{"step": 241, "steps": 250, "lr": 5.20247346696706e-06, "grad_norm": 0.20648464560508728, "loss_scale": 4.0, "retries": 0, "seconds": 862.8, "max_memory_gb": 17.08, "ce": 0.009030167224409524, "rank": 0.0}
|
| 242 |
+
{"step": 242, "steps": 250, "lr": 5.164050332793787e-06, "grad_norm": 2.8929598331451416, "loss_scale": 4.0, "retries": 0, "seconds": 866.3, "max_memory_gb": 17.08, "ce": 0.30351204899943696, "rank": 0.12482074648141861}
|
| 243 |
+
{"step": 243, "steps": 250, "lr": 5.129653125932873e-06, "grad_norm": 2.6201038360595703, "loss_scale": 4.0, "retries": 0, "seconds": 869.7, "max_memory_gb": 17.08, "ce": 0.15982616899145796, "rank": 0.0}
|
| 244 |
+
{"step": 244, "steps": 250, "lr": 5.099288046284329e-06, "grad_norm": 1.9801653623580933, "loss_scale": 4.0, "retries": 0, "seconds": 873.1, "max_memory_gb": 17.08, "ce": 0.07444357139229396, "rank": 0.0}
|
| 245 |
+
{"step": 245, "steps": 250, "lr": 5.072960566980277e-06, "grad_norm": 1.137176513671875, "loss_scale": 4.0, "retries": 0, "seconds": 876.5, "max_memory_gb": 17.08, "ce": 0.13982082279858332, "rank": 0.0}
|
| 246 |
+
{"step": 246, "steps": 250, "lr": 5.050675433398435e-06, "grad_norm": 0.7369658350944519, "loss_scale": 4.0, "retries": 0, "seconds": 879.9, "max_memory_gb": 17.08, "ce": 0.04374349415047618, "rank": 0.0}
|
| 247 |
+
{"step": 247, "steps": 250, "lr": 5.032436662306811e-06, "grad_norm": 0.2583094537258148, "loss_scale": 4.0, "retries": 0, "seconds": 883.4, "max_memory_gb": 17.08, "ce": 0.005126382652633765, "rank": 0.0}
|
| 248 |
+
{"step": 248, "steps": 250, "lr": 5.018247541139684e-06, "grad_norm": 0.23901598155498505, "loss_scale": 4.0, "retries": 0, "seconds": 886.8, "max_memory_gb": 17.08, "ce": 0.004068636947067716, "rank": 0.0}
|
| 249 |
+
{"step": 249, "steps": 250, "lr": 5.00811062740508e-06, "grad_norm": 0.32987096905708313, "loss_scale": 4.0, "retries": 0, "seconds": 890.2, "max_memory_gb": 17.08, "ce": 0.028490178695847135, "rank": 0.1198502853512764}
|
| 250 |
+
{"step": 250, "steps": 250, "lr": 5.002027748223778e-06, "grad_norm": 0.4531836211681366, "loss_scale": 4.0, "retries": 0, "seconds": 893.6, "max_memory_gb": 17.08, "ce": 0.019808433913340195, "rank": 0.1113767996430397}
|
runs/fixed_internvl/removal/Top8_s73/adapter/README.md
ADDED
|
@@ -0,0 +1,207 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
base_model: /data/OPSD/assets/models/InternVL3_5-8B-HF
|
| 3 |
+
library_name: peft
|
| 4 |
+
pipeline_tag: text-generation
|
| 5 |
+
tags:
|
| 6 |
+
- base_model:adapter:/data/OPSD/assets/models/InternVL3_5-8B-HF
|
| 7 |
+
- lora
|
| 8 |
+
- transformers
|
| 9 |
+
---
|
| 10 |
+
|
| 11 |
+
# Model Card for Model ID
|
| 12 |
+
|
| 13 |
+
<!-- Provide a quick summary of what the model is/does. -->
|
| 14 |
+
|
| 15 |
+
|
| 16 |
+
|
| 17 |
+
## Model Details
|
| 18 |
+
|
| 19 |
+
### Model Description
|
| 20 |
+
|
| 21 |
+
<!-- Provide a longer summary of what this model is. -->
|
| 22 |
+
|
| 23 |
+
|
| 24 |
+
|
| 25 |
+
- **Developed by:** [More Information Needed]
|
| 26 |
+
- **Funded by [optional]:** [More Information Needed]
|
| 27 |
+
- **Shared by [optional]:** [More Information Needed]
|
| 28 |
+
- **Model type:** [More Information Needed]
|
| 29 |
+
- **Language(s) (NLP):** [More Information Needed]
|
| 30 |
+
- **License:** [More Information Needed]
|
| 31 |
+
- **Finetuned from model [optional]:** [More Information Needed]
|
| 32 |
+
|
| 33 |
+
### Model Sources [optional]
|
| 34 |
+
|
| 35 |
+
<!-- Provide the basic links for the model. -->
|
| 36 |
+
|
| 37 |
+
- **Repository:** [More Information Needed]
|
| 38 |
+
- **Paper [optional]:** [More Information Needed]
|
| 39 |
+
- **Demo [optional]:** [More Information Needed]
|
| 40 |
+
|
| 41 |
+
## Uses
|
| 42 |
+
|
| 43 |
+
<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
|
| 44 |
+
|
| 45 |
+
### Direct Use
|
| 46 |
+
|
| 47 |
+
<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
|
| 48 |
+
|
| 49 |
+
[More Information Needed]
|
| 50 |
+
|
| 51 |
+
### Downstream Use [optional]
|
| 52 |
+
|
| 53 |
+
<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
|
| 54 |
+
|
| 55 |
+
[More Information Needed]
|
| 56 |
+
|
| 57 |
+
### Out-of-Scope Use
|
| 58 |
+
|
| 59 |
+
<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
|
| 60 |
+
|
| 61 |
+
[More Information Needed]
|
| 62 |
+
|
| 63 |
+
## Bias, Risks, and Limitations
|
| 64 |
+
|
| 65 |
+
<!-- This section is meant to convey both technical and sociotechnical limitations. -->
|
| 66 |
+
|
| 67 |
+
[More Information Needed]
|
| 68 |
+
|
| 69 |
+
### Recommendations
|
| 70 |
+
|
| 71 |
+
<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
|
| 72 |
+
|
| 73 |
+
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
|
| 74 |
+
|
| 75 |
+
## How to Get Started with the Model
|
| 76 |
+
|
| 77 |
+
Use the code below to get started with the model.
|
| 78 |
+
|
| 79 |
+
[More Information Needed]
|
| 80 |
+
|
| 81 |
+
## Training Details
|
| 82 |
+
|
| 83 |
+
### Training Data
|
| 84 |
+
|
| 85 |
+
<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
|
| 86 |
+
|
| 87 |
+
[More Information Needed]
|
| 88 |
+
|
| 89 |
+
### Training Procedure
|
| 90 |
+
|
| 91 |
+
<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
|
| 92 |
+
|
| 93 |
+
#### Preprocessing [optional]
|
| 94 |
+
|
| 95 |
+
[More Information Needed]
|
| 96 |
+
|
| 97 |
+
|
| 98 |
+
#### Training Hyperparameters
|
| 99 |
+
|
| 100 |
+
- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
|
| 101 |
+
|
| 102 |
+
#### Speeds, Sizes, Times [optional]
|
| 103 |
+
|
| 104 |
+
<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
|
| 105 |
+
|
| 106 |
+
[More Information Needed]
|
| 107 |
+
|
| 108 |
+
## Evaluation
|
| 109 |
+
|
| 110 |
+
<!-- This section describes the evaluation protocols and provides the results. -->
|
| 111 |
+
|
| 112 |
+
### Testing Data, Factors & Metrics
|
| 113 |
+
|
| 114 |
+
#### Testing Data
|
| 115 |
+
|
| 116 |
+
<!-- This should link to a Dataset Card if possible. -->
|
| 117 |
+
|
| 118 |
+
[More Information Needed]
|
| 119 |
+
|
| 120 |
+
#### Factors
|
| 121 |
+
|
| 122 |
+
<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
|
| 123 |
+
|
| 124 |
+
[More Information Needed]
|
| 125 |
+
|
| 126 |
+
#### Metrics
|
| 127 |
+
|
| 128 |
+
<!-- These are the evaluation metrics being used, ideally with a description of why. -->
|
| 129 |
+
|
| 130 |
+
[More Information Needed]
|
| 131 |
+
|
| 132 |
+
### Results
|
| 133 |
+
|
| 134 |
+
[More Information Needed]
|
| 135 |
+
|
| 136 |
+
#### Summary
|
| 137 |
+
|
| 138 |
+
|
| 139 |
+
|
| 140 |
+
## Model Examination [optional]
|
| 141 |
+
|
| 142 |
+
<!-- Relevant interpretability work for the model goes here -->
|
| 143 |
+
|
| 144 |
+
[More Information Needed]
|
| 145 |
+
|
| 146 |
+
## Environmental Impact
|
| 147 |
+
|
| 148 |
+
<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
|
| 149 |
+
|
| 150 |
+
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
|
| 151 |
+
|
| 152 |
+
- **Hardware Type:** [More Information Needed]
|
| 153 |
+
- **Hours used:** [More Information Needed]
|
| 154 |
+
- **Cloud Provider:** [More Information Needed]
|
| 155 |
+
- **Compute Region:** [More Information Needed]
|
| 156 |
+
- **Carbon Emitted:** [More Information Needed]
|
| 157 |
+
|
| 158 |
+
## Technical Specifications [optional]
|
| 159 |
+
|
| 160 |
+
### Model Architecture and Objective
|
| 161 |
+
|
| 162 |
+
[More Information Needed]
|
| 163 |
+
|
| 164 |
+
### Compute Infrastructure
|
| 165 |
+
|
| 166 |
+
[More Information Needed]
|
| 167 |
+
|
| 168 |
+
#### Hardware
|
| 169 |
+
|
| 170 |
+
[More Information Needed]
|
| 171 |
+
|
| 172 |
+
#### Software
|
| 173 |
+
|
| 174 |
+
[More Information Needed]
|
| 175 |
+
|
| 176 |
+
## Citation [optional]
|
| 177 |
+
|
| 178 |
+
<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
|
| 179 |
+
|
| 180 |
+
**BibTeX:**
|
| 181 |
+
|
| 182 |
+
[More Information Needed]
|
| 183 |
+
|
| 184 |
+
**APA:**
|
| 185 |
+
|
| 186 |
+
[More Information Needed]
|
| 187 |
+
|
| 188 |
+
## Glossary [optional]
|
| 189 |
+
|
| 190 |
+
<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
|
| 191 |
+
|
| 192 |
+
[More Information Needed]
|
| 193 |
+
|
| 194 |
+
## More Information [optional]
|
| 195 |
+
|
| 196 |
+
[More Information Needed]
|
| 197 |
+
|
| 198 |
+
## Model Card Authors [optional]
|
| 199 |
+
|
| 200 |
+
[More Information Needed]
|
| 201 |
+
|
| 202 |
+
## Model Card Contact
|
| 203 |
+
|
| 204 |
+
[More Information Needed]
|
| 205 |
+
### Framework versions
|
| 206 |
+
|
| 207 |
+
- PEFT 0.20.0
|
runs/fixed_internvl/removal/Top8_s73/adapter/adapter_config.json
ADDED
|
@@ -0,0 +1,213 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"alora_invocation_tokens": null,
|
| 3 |
+
"alpha_pattern": {
|
| 4 |
+
"model.language_model.layers.0.mlp.down_proj": 144,
|
| 5 |
+
"model.language_model.layers.0.mlp.gate_proj": 144,
|
| 6 |
+
"model.language_model.layers.0.mlp.up_proj": 144,
|
| 7 |
+
"model.language_model.layers.0.self_attn.k_proj": 144,
|
| 8 |
+
"model.language_model.layers.0.self_attn.o_proj": 144,
|
| 9 |
+
"model.language_model.layers.0.self_attn.q_proj": 144,
|
| 10 |
+
"model.language_model.layers.0.self_attn.v_proj": 144,
|
| 11 |
+
"model.language_model.layers.1.mlp.down_proj": 144,
|
| 12 |
+
"model.language_model.layers.1.mlp.gate_proj": 144,
|
| 13 |
+
"model.language_model.layers.1.mlp.up_proj": 144,
|
| 14 |
+
"model.language_model.layers.1.self_attn.k_proj": 144,
|
| 15 |
+
"model.language_model.layers.1.self_attn.o_proj": 144,
|
| 16 |
+
"model.language_model.layers.1.self_attn.q_proj": 144,
|
| 17 |
+
"model.language_model.layers.1.self_attn.v_proj": 144,
|
| 18 |
+
"model.language_model.layers.2.mlp.down_proj": 144,
|
| 19 |
+
"model.language_model.layers.2.mlp.gate_proj": 144,
|
| 20 |
+
"model.language_model.layers.2.mlp.up_proj": 144,
|
| 21 |
+
"model.language_model.layers.2.self_attn.k_proj": 144,
|
| 22 |
+
"model.language_model.layers.2.self_attn.o_proj": 144,
|
| 23 |
+
"model.language_model.layers.2.self_attn.q_proj": 144,
|
| 24 |
+
"model.language_model.layers.2.self_attn.v_proj": 144,
|
| 25 |
+
"model.language_model.layers.3.mlp.down_proj": 144,
|
| 26 |
+
"model.language_model.layers.3.mlp.gate_proj": 144,
|
| 27 |
+
"model.language_model.layers.3.mlp.up_proj": 144,
|
| 28 |
+
"model.language_model.layers.3.self_attn.k_proj": 144,
|
| 29 |
+
"model.language_model.layers.3.self_attn.o_proj": 144,
|
| 30 |
+
"model.language_model.layers.3.self_attn.q_proj": 144,
|
| 31 |
+
"model.language_model.layers.3.self_attn.v_proj": 144,
|
| 32 |
+
"model.language_model.layers.4.mlp.down_proj": 144,
|
| 33 |
+
"model.language_model.layers.4.mlp.gate_proj": 144,
|
| 34 |
+
"model.language_model.layers.4.mlp.up_proj": 144,
|
| 35 |
+
"model.language_model.layers.4.self_attn.k_proj": 144,
|
| 36 |
+
"model.language_model.layers.4.self_attn.o_proj": 144,
|
| 37 |
+
"model.language_model.layers.4.self_attn.q_proj": 144,
|
| 38 |
+
"model.language_model.layers.4.self_attn.v_proj": 144,
|
| 39 |
+
"model.language_model.layers.5.mlp.down_proj": 144,
|
| 40 |
+
"model.language_model.layers.5.mlp.gate_proj": 144,
|
| 41 |
+
"model.language_model.layers.5.mlp.up_proj": 144,
|
| 42 |
+
"model.language_model.layers.5.self_attn.k_proj": 144,
|
| 43 |
+
"model.language_model.layers.5.self_attn.o_proj": 144,
|
| 44 |
+
"model.language_model.layers.5.self_attn.q_proj": 144,
|
| 45 |
+
"model.language_model.layers.5.self_attn.v_proj": 144,
|
| 46 |
+
"model.language_model.layers.6.mlp.down_proj": 144,
|
| 47 |
+
"model.language_model.layers.6.mlp.gate_proj": 144,
|
| 48 |
+
"model.language_model.layers.6.mlp.up_proj": 144,
|
| 49 |
+
"model.language_model.layers.6.self_attn.k_proj": 144,
|
| 50 |
+
"model.language_model.layers.6.self_attn.o_proj": 144,
|
| 51 |
+
"model.language_model.layers.6.self_attn.q_proj": 144,
|
| 52 |
+
"model.language_model.layers.6.self_attn.v_proj": 144,
|
| 53 |
+
"model.language_model.layers.7.mlp.down_proj": 144,
|
| 54 |
+
"model.language_model.layers.7.mlp.gate_proj": 144,
|
| 55 |
+
"model.language_model.layers.7.mlp.up_proj": 144,
|
| 56 |
+
"model.language_model.layers.7.self_attn.k_proj": 144,
|
| 57 |
+
"model.language_model.layers.7.self_attn.o_proj": 144,
|
| 58 |
+
"model.language_model.layers.7.self_attn.q_proj": 144,
|
| 59 |
+
"model.language_model.layers.7.self_attn.v_proj": 144
|
| 60 |
+
},
|
| 61 |
+
"arrow_config": null,
|
| 62 |
+
"auto_mapping": null,
|
| 63 |
+
"base_model_name_or_path": "/data/OPSD/assets/models/InternVL3_5-8B-HF",
|
| 64 |
+
"bias": "none",
|
| 65 |
+
"corda_config": null,
|
| 66 |
+
"ensure_weight_tying": false,
|
| 67 |
+
"eva_config": null,
|
| 68 |
+
"exclude_modules": null,
|
| 69 |
+
"fan_in_fan_out": false,
|
| 70 |
+
"inference_mode": true,
|
| 71 |
+
"init_lora_weights": true,
|
| 72 |
+
"layer_replication": null,
|
| 73 |
+
"layers_pattern": null,
|
| 74 |
+
"layers_to_transform": null,
|
| 75 |
+
"loftq_config": {},
|
| 76 |
+
"lora_alpha": 32,
|
| 77 |
+
"lora_bias": false,
|
| 78 |
+
"lora_dropout": 0.05,
|
| 79 |
+
"lora_ga_config": null,
|
| 80 |
+
"megatron_config": null,
|
| 81 |
+
"megatron_core": "megatron.core",
|
| 82 |
+
"modules_to_save": null,
|
| 83 |
+
"monteclora_config": null,
|
| 84 |
+
"peft_type": "LORA",
|
| 85 |
+
"peft_version": "0.20.0",
|
| 86 |
+
"qalora_group_size": 16,
|
| 87 |
+
"r": 16,
|
| 88 |
+
"rank_pattern": {
|
| 89 |
+
"model.language_model.layers.0.mlp.down_proj": 72,
|
| 90 |
+
"model.language_model.layers.0.mlp.gate_proj": 72,
|
| 91 |
+
"model.language_model.layers.0.mlp.up_proj": 72,
|
| 92 |
+
"model.language_model.layers.0.self_attn.k_proj": 72,
|
| 93 |
+
"model.language_model.layers.0.self_attn.o_proj": 72,
|
| 94 |
+
"model.language_model.layers.0.self_attn.q_proj": 72,
|
| 95 |
+
"model.language_model.layers.0.self_attn.v_proj": 72,
|
| 96 |
+
"model.language_model.layers.1.mlp.down_proj": 72,
|
| 97 |
+
"model.language_model.layers.1.mlp.gate_proj": 72,
|
| 98 |
+
"model.language_model.layers.1.mlp.up_proj": 72,
|
| 99 |
+
"model.language_model.layers.1.self_attn.k_proj": 72,
|
| 100 |
+
"model.language_model.layers.1.self_attn.o_proj": 72,
|
| 101 |
+
"model.language_model.layers.1.self_attn.q_proj": 72,
|
| 102 |
+
"model.language_model.layers.1.self_attn.v_proj": 72,
|
| 103 |
+
"model.language_model.layers.2.mlp.down_proj": 72,
|
| 104 |
+
"model.language_model.layers.2.mlp.gate_proj": 72,
|
| 105 |
+
"model.language_model.layers.2.mlp.up_proj": 72,
|
| 106 |
+
"model.language_model.layers.2.self_attn.k_proj": 72,
|
| 107 |
+
"model.language_model.layers.2.self_attn.o_proj": 72,
|
| 108 |
+
"model.language_model.layers.2.self_attn.q_proj": 72,
|
| 109 |
+
"model.language_model.layers.2.self_attn.v_proj": 72,
|
| 110 |
+
"model.language_model.layers.3.mlp.down_proj": 72,
|
| 111 |
+
"model.language_model.layers.3.mlp.gate_proj": 72,
|
| 112 |
+
"model.language_model.layers.3.mlp.up_proj": 72,
|
| 113 |
+
"model.language_model.layers.3.self_attn.k_proj": 72,
|
| 114 |
+
"model.language_model.layers.3.self_attn.o_proj": 72,
|
| 115 |
+
"model.language_model.layers.3.self_attn.q_proj": 72,
|
| 116 |
+
"model.language_model.layers.3.self_attn.v_proj": 72,
|
| 117 |
+
"model.language_model.layers.4.mlp.down_proj": 72,
|
| 118 |
+
"model.language_model.layers.4.mlp.gate_proj": 72,
|
| 119 |
+
"model.language_model.layers.4.mlp.up_proj": 72,
|
| 120 |
+
"model.language_model.layers.4.self_attn.k_proj": 72,
|
| 121 |
+
"model.language_model.layers.4.self_attn.o_proj": 72,
|
| 122 |
+
"model.language_model.layers.4.self_attn.q_proj": 72,
|
| 123 |
+
"model.language_model.layers.4.self_attn.v_proj": 72,
|
| 124 |
+
"model.language_model.layers.5.mlp.down_proj": 72,
|
| 125 |
+
"model.language_model.layers.5.mlp.gate_proj": 72,
|
| 126 |
+
"model.language_model.layers.5.mlp.up_proj": 72,
|
| 127 |
+
"model.language_model.layers.5.self_attn.k_proj": 72,
|
| 128 |
+
"model.language_model.layers.5.self_attn.o_proj": 72,
|
| 129 |
+
"model.language_model.layers.5.self_attn.q_proj": 72,
|
| 130 |
+
"model.language_model.layers.5.self_attn.v_proj": 72,
|
| 131 |
+
"model.language_model.layers.6.mlp.down_proj": 72,
|
| 132 |
+
"model.language_model.layers.6.mlp.gate_proj": 72,
|
| 133 |
+
"model.language_model.layers.6.mlp.up_proj": 72,
|
| 134 |
+
"model.language_model.layers.6.self_attn.k_proj": 72,
|
| 135 |
+
"model.language_model.layers.6.self_attn.o_proj": 72,
|
| 136 |
+
"model.language_model.layers.6.self_attn.q_proj": 72,
|
| 137 |
+
"model.language_model.layers.6.self_attn.v_proj": 72,
|
| 138 |
+
"model.language_model.layers.7.mlp.down_proj": 72,
|
| 139 |
+
"model.language_model.layers.7.mlp.gate_proj": 72,
|
| 140 |
+
"model.language_model.layers.7.mlp.up_proj": 72,
|
| 141 |
+
"model.language_model.layers.7.self_attn.k_proj": 72,
|
| 142 |
+
"model.language_model.layers.7.self_attn.o_proj": 72,
|
| 143 |
+
"model.language_model.layers.7.self_attn.q_proj": 72,
|
| 144 |
+
"model.language_model.layers.7.self_attn.v_proj": 72
|
| 145 |
+
},
|
| 146 |
+
"revision": null,
|
| 147 |
+
"target_modules": [
|
| 148 |
+
"model.language_model.layers.3.self_attn.v_proj",
|
| 149 |
+
"model.language_model.layers.7.mlp.gate_proj",
|
| 150 |
+
"model.language_model.layers.7.mlp.up_proj",
|
| 151 |
+
"model.language_model.layers.4.mlp.up_proj",
|
| 152 |
+
"model.language_model.layers.1.self_attn.o_proj",
|
| 153 |
+
"model.language_model.layers.2.mlp.gate_proj",
|
| 154 |
+
"model.language_model.layers.6.mlp.down_proj",
|
| 155 |
+
"model.language_model.layers.0.self_attn.o_proj",
|
| 156 |
+
"model.language_model.layers.6.self_attn.o_proj",
|
| 157 |
+
"model.language_model.layers.0.mlp.up_proj",
|
| 158 |
+
"model.language_model.layers.3.mlp.down_proj",
|
| 159 |
+
"model.language_model.layers.5.self_attn.k_proj",
|
| 160 |
+
"model.language_model.layers.2.self_attn.q_proj",
|
| 161 |
+
"model.language_model.layers.0.self_attn.q_proj",
|
| 162 |
+
"model.language_model.layers.6.mlp.up_proj",
|
| 163 |
+
"model.language_model.layers.4.mlp.gate_proj",
|
| 164 |
+
"model.language_model.layers.0.mlp.down_proj",
|
| 165 |
+
"model.language_model.layers.4.self_attn.o_proj",
|
| 166 |
+
"model.language_model.layers.1.mlp.down_proj",
|
| 167 |
+
"model.language_model.layers.2.mlp.down_proj",
|
| 168 |
+
"model.language_model.layers.4.self_attn.q_proj",
|
| 169 |
+
"model.language_model.layers.7.self_attn.q_proj",
|
| 170 |
+
"model.language_model.layers.4.self_attn.v_proj",
|
| 171 |
+
"model.language_model.layers.7.self_attn.v_proj",
|
| 172 |
+
"model.language_model.layers.1.self_attn.v_proj",
|
| 173 |
+
"model.language_model.layers.3.self_attn.q_proj",
|
| 174 |
+
"model.language_model.layers.1.self_attn.k_proj",
|
| 175 |
+
"model.language_model.layers.3.self_attn.k_proj",
|
| 176 |
+
"model.language_model.layers.5.mlp.gate_proj",
|
| 177 |
+
"model.language_model.layers.5.self_attn.q_proj",
|
| 178 |
+
"model.language_model.layers.6.self_attn.v_proj",
|
| 179 |
+
"model.language_model.layers.0.mlp.gate_proj",
|
| 180 |
+
"model.language_model.layers.2.mlp.up_proj",
|
| 181 |
+
"model.language_model.layers.7.self_attn.k_proj",
|
| 182 |
+
"model.language_model.layers.1.mlp.up_proj",
|
| 183 |
+
"model.language_model.layers.6.self_attn.q_proj",
|
| 184 |
+
"model.language_model.layers.5.mlp.down_proj",
|
| 185 |
+
"model.language_model.layers.3.mlp.up_proj",
|
| 186 |
+
"model.language_model.layers.4.mlp.down_proj",
|
| 187 |
+
"model.language_model.layers.3.mlp.gate_proj",
|
| 188 |
+
"model.language_model.layers.6.self_attn.k_proj",
|
| 189 |
+
"model.language_model.layers.3.self_attn.o_proj",
|
| 190 |
+
"model.language_model.layers.0.self_attn.k_proj",
|
| 191 |
+
"model.language_model.layers.1.mlp.gate_proj",
|
| 192 |
+
"model.language_model.layers.2.self_attn.v_proj",
|
| 193 |
+
"model.language_model.layers.5.self_attn.o_proj",
|
| 194 |
+
"model.language_model.layers.2.self_attn.k_proj",
|
| 195 |
+
"model.language_model.layers.7.self_attn.o_proj",
|
| 196 |
+
"model.language_model.layers.0.self_attn.v_proj",
|
| 197 |
+
"model.language_model.layers.5.self_attn.v_proj",
|
| 198 |
+
"model.language_model.layers.1.self_attn.q_proj",
|
| 199 |
+
"model.language_model.layers.6.mlp.gate_proj",
|
| 200 |
+
"model.language_model.layers.4.self_attn.k_proj",
|
| 201 |
+
"model.language_model.layers.7.mlp.down_proj",
|
| 202 |
+
"model.language_model.layers.5.mlp.up_proj",
|
| 203 |
+
"model.language_model.layers.2.self_attn.o_proj"
|
| 204 |
+
],
|
| 205 |
+
"target_parameters": null,
|
| 206 |
+
"task_type": "CAUSAL_LM",
|
| 207 |
+
"trainable_token_indices": null,
|
| 208 |
+
"use_bdlora": null,
|
| 209 |
+
"use_dora": false,
|
| 210 |
+
"use_qalora": false,
|
| 211 |
+
"use_rslora": false,
|
| 212 |
+
"velora_config": null
|
| 213 |
+
}
|
runs/fixed_internvl/removal/Top8_s73/config.json
ADDED
|
@@ -0,0 +1,43 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"behavior": "removal",
|
| 3 |
+
"seed": 73,
|
| 4 |
+
"model": "/data/OPSD/assets/models/InternVL3_5-8B-HF",
|
| 5 |
+
"backbone": "internvl",
|
| 6 |
+
"train": "/data/OPSD/assets/v3/removal.jsonl",
|
| 7 |
+
"items": 1000,
|
| 8 |
+
"steps": 250,
|
| 9 |
+
"epochs": 1,
|
| 10 |
+
"steps_per_epoch": 250,
|
| 11 |
+
"world_size": 1,
|
| 12 |
+
"lr": 5e-05,
|
| 13 |
+
"scheduler": "cosine",
|
| 14 |
+
"warmup_steps": 16,
|
| 15 |
+
"final_fraction": 0.1,
|
| 16 |
+
"clip": 1.0,
|
| 17 |
+
"initial_loss_scale": 4.0,
|
| 18 |
+
"arm": {
|
| 19 |
+
"arm": "Top8",
|
| 20 |
+
"layers": [
|
| 21 |
+
0,
|
| 22 |
+
1,
|
| 23 |
+
2,
|
| 24 |
+
3,
|
| 25 |
+
4,
|
| 26 |
+
5,
|
| 27 |
+
6,
|
| 28 |
+
7
|
| 29 |
+
],
|
| 30 |
+
"ranks": {
|
| 31 |
+
"0": 72,
|
| 32 |
+
"1": 72,
|
| 33 |
+
"2": 72,
|
| 34 |
+
"3": 72,
|
| 35 |
+
"4": 72,
|
| 36 |
+
"5": 72,
|
| 37 |
+
"6": 72,
|
| 38 |
+
"7": 72
|
| 39 |
+
},
|
| 40 |
+
"parameters": 43646976,
|
| 41 |
+
"budget": 43646976
|
| 42 |
+
}
|
| 43 |
+
}
|
runs/fixed_internvl/removal/Top8_s73/steps.jsonl
ADDED
|
@@ -0,0 +1,250 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{"step": 1, "steps": 250, "lr": 3.125e-06, "grad_norm": 12.212206840515137, "loss_scale": 4.0, "retries": 0, "seconds": 4.7, "max_memory_gb": 16.79, "ce": 0.8941777415893739, "rank": 0.0}
|
| 2 |
+
{"step": 2, "steps": 250, "lr": 6.25e-06, "grad_norm": 19.551116943359375, "loss_scale": 4.0, "retries": 0, "seconds": 8.8, "max_memory_gb": 17.07, "ce": 1.6409455789253116, "rank": 0.005284667015075684}
|
| 3 |
+
{"step": 3, "steps": 250, "lr": 9.375000000000001e-06, "grad_norm": 10.51677131652832, "loss_scale": 4.0, "retries": 0, "seconds": 12.5, "max_memory_gb": 17.07, "ce": 0.6642871385265607, "rank": 0.08140014111995697}
|
| 4 |
+
{"step": 4, "steps": 250, "lr": 1.25e-05, "grad_norm": 8.627418518066406, "loss_scale": 4.0, "retries": 0, "seconds": 16.6, "max_memory_gb": 17.07, "ce": 0.6422704591386719, "rank": 0.0}
|
| 5 |
+
{"step": 5, "steps": 250, "lr": 1.5625e-05, "grad_norm": 3.855302333831787, "loss_scale": 4.0, "retries": 0, "seconds": 20.7, "max_memory_gb": 17.08, "ce": 0.4976856684515951, "rank": 0.0}
|
| 6 |
+
{"step": 6, "steps": 250, "lr": 1.8750000000000002e-05, "grad_norm": 2.373321056365967, "loss_scale": 4.0, "retries": 0, "seconds": 25.1, "max_memory_gb": 17.08, "ce": 0.4937693849205971, "rank": 0.0}
|
| 7 |
+
{"step": 7, "steps": 250, "lr": 2.1875e-05, "grad_norm": 0.2952851951122284, "loss_scale": 4.0, "retries": 0, "seconds": 29.5, "max_memory_gb": 17.08, "ce": 0.13770377020045999, "rank": 0.0}
|
| 8 |
+
{"step": 8, "steps": 250, "lr": 2.5e-05, "grad_norm": 1.6122791767120361, "loss_scale": 4.0, "retries": 0, "seconds": 34.2, "max_memory_gb": 17.08, "ce": 0.6476408479793463, "rank": 0.13141077756881714}
|
| 9 |
+
{"step": 9, "steps": 250, "lr": 2.8125000000000003e-05, "grad_norm": 1.9131189584732056, "loss_scale": 4.0, "retries": 0, "seconds": 38.8, "max_memory_gb": 17.08, "ce": 0.3936593473190442, "rank": 0.0}
|
| 10 |
+
{"step": 10, "steps": 250, "lr": 3.125e-05, "grad_norm": 1.8017429113388062, "loss_scale": 4.0, "retries": 0, "seconds": 43.6, "max_memory_gb": 17.08, "ce": 0.34593087524262955, "rank": 0.008345022797584534}
|
| 11 |
+
{"step": 11, "steps": 250, "lr": 3.4375e-05, "grad_norm": 1.0154439210891724, "loss_scale": 4.0, "retries": 0, "seconds": 48.3, "max_memory_gb": 17.08, "ce": 0.3421913835336454, "rank": 0.0}
|
| 12 |
+
{"step": 12, "steps": 250, "lr": 3.7500000000000003e-05, "grad_norm": 1.96696138381958, "loss_scale": 4.0, "retries": 0, "seconds": 53.1, "max_memory_gb": 17.08, "ce": 0.2023702124997726, "rank": 0.0}
|
| 13 |
+
{"step": 13, "steps": 250, "lr": 4.0625000000000005e-05, "grad_norm": 1.4922231435775757, "loss_scale": 4.0, "retries": 0, "seconds": 57.8, "max_memory_gb": 17.08, "ce": 0.37764417195649, "rank": 0.0}
|
| 14 |
+
{"step": 14, "steps": 250, "lr": 4.375e-05, "grad_norm": 1.0965903997421265, "loss_scale": 4.0, "retries": 0, "seconds": 62.6, "max_memory_gb": 17.08, "ce": 0.2786800739431783, "rank": 0.0}
|
| 15 |
+
{"step": 15, "steps": 250, "lr": 4.6875e-05, "grad_norm": 5.962850570678711, "loss_scale": 4.0, "retries": 0, "seconds": 67.4, "max_memory_gb": 17.08, "ce": 0.20104270317824557, "rank": 0.0}
|
| 16 |
+
{"step": 16, "steps": 250, "lr": 5e-05, "grad_norm": 3.572892904281616, "loss_scale": 4.0, "retries": 0, "seconds": 72.2, "max_memory_gb": 17.08, "ce": 0.9399923705786932, "rank": 0.0}
|
| 17 |
+
{"step": 17, "steps": 250, "lr": 5e-05, "grad_norm": 2.2056846618652344, "loss_scale": 4.0, "retries": 0, "seconds": 76.9, "max_memory_gb": 17.08, "ce": 0.2723891969071701, "rank": 0.0}
|
| 18 |
+
{"step": 18, "steps": 250, "lr": 4.999797225177623e-05, "grad_norm": 0.5796476602554321, "loss_scale": 4.0, "retries": 0, "seconds": 81.7, "max_memory_gb": 17.08, "ce": 0.2082996159006143, "rank": 0.12467323243618011}
|
| 19 |
+
{"step": 19, "steps": 250, "lr": 4.999188937259492e-05, "grad_norm": 4.37310791015625, "loss_scale": 4.0, "retries": 0, "seconds": 86.3, "max_memory_gb": 17.08, "ce": 0.37416790357383434, "rank": 0.0}
|
| 20 |
+
{"step": 20, "steps": 250, "lr": 4.998175245886032e-05, "grad_norm": 0.6298394203186035, "loss_scale": 4.0, "retries": 0, "seconds": 91.0, "max_memory_gb": 17.08, "ce": 0.3795890662295278, "rank": 0.0}
|
| 21 |
+
{"step": 21, "steps": 250, "lr": 4.996756333769319e-05, "grad_norm": 0.5515211224555969, "loss_scale": 4.0, "retries": 0, "seconds": 95.7, "max_memory_gb": 17.08, "ce": 0.201230193633819, "rank": 0.0}
|
| 22 |
+
{"step": 22, "steps": 250, "lr": 4.994932456660157e-05, "grad_norm": 0.7271602749824524, "loss_scale": 4.0, "retries": 0, "seconds": 100.5, "max_memory_gb": 17.08, "ce": 0.3991761893266812, "rank": 0.0}
|
| 23 |
+
{"step": 23, "steps": 250, "lr": 4.992703943301973e-05, "grad_norm": 2.281179666519165, "loss_scale": 4.0, "retries": 0, "seconds": 105.2, "max_memory_gb": 17.08, "ce": 0.24534398911055177, "rank": 0.0}
|
| 24 |
+
{"step": 24, "steps": 250, "lr": 4.9900711953715675e-05, "grad_norm": 0.7987722158432007, "loss_scale": 4.0, "retries": 0, "seconds": 110.0, "max_memory_gb": 17.08, "ce": 0.3316447283141315, "rank": 0.0}
|
| 25 |
+
{"step": 25, "steps": 250, "lr": 4.987034687406713e-05, "grad_norm": 3.1532576084136963, "loss_scale": 4.0, "retries": 0, "seconds": 115.1, "max_memory_gb": 17.08, "ce": 0.17665703129023314, "rank": 0.03388618677854538}
|
| 26 |
+
{"step": 26, "steps": 250, "lr": 4.9835949667206215e-05, "grad_norm": 2.434912919998169, "loss_scale": 4.0, "retries": 0, "seconds": 119.9, "max_memory_gb": 17.08, "ce": 0.3535329451842699, "rank": 0.0}
|
| 27 |
+
{"step": 27, "steps": 250, "lr": 4.9797526533032945e-05, "grad_norm": 6.409736156463623, "loss_scale": 4.0, "retries": 0, "seconds": 124.6, "max_memory_gb": 17.08, "ce": 0.39567677676677704, "rank": 0.033513933420181274}
|
| 28 |
+
{"step": 28, "steps": 250, "lr": 4.9755084397097716e-05, "grad_norm": 0.6967154741287231, "loss_scale": 4.0, "retries": 0, "seconds": 129.3, "max_memory_gb": 17.08, "ce": 0.21186468793894164, "rank": 0.0}
|
| 29 |
+
{"step": 29, "steps": 250, "lr": 4.970863090935304e-05, "grad_norm": 0.49029871821403503, "loss_scale": 4.0, "retries": 0, "seconds": 134.1, "max_memory_gb": 17.08, "ce": 0.14289218457997777, "rank": 0.0}
|
| 30 |
+
{"step": 30, "steps": 250, "lr": 4.965817444277468e-05, "grad_norm": 0.8196736574172974, "loss_scale": 4.0, "retries": 0, "seconds": 138.8, "max_memory_gb": 17.08, "ce": 0.17782264729612507, "rank": 0.0}
|
| 31 |
+
{"step": 31, "steps": 250, "lr": 4.9603724091852456e-05, "grad_norm": 0.8709579706192017, "loss_scale": 4.0, "retries": 0, "seconds": 143.5, "max_memory_gb": 17.08, "ce": 0.19191656589100603, "rank": 0.0}
|
| 32 |
+
{"step": 32, "steps": 250, "lr": 4.9545289670951025e-05, "grad_norm": 2.240795135498047, "loss_scale": 4.0, "retries": 0, "seconds": 148.3, "max_memory_gb": 17.08, "ce": 0.30077275732764974, "rank": 0.1253044754266739}
|
| 33 |
+
{"step": 33, "steps": 250, "lr": 4.948288171254089e-05, "grad_norm": 0.774005651473999, "loss_scale": 4.0, "retries": 0, "seconds": 152.9, "max_memory_gb": 17.08, "ce": 0.11302853628876619, "rank": 0.12516485154628754}
|
| 34 |
+
{"step": 34, "steps": 250, "lr": 4.941651146529998e-05, "grad_norm": 1.0728771686553955, "loss_scale": 4.0, "retries": 0, "seconds": 157.5, "max_memory_gb": 17.08, "ce": 0.17027833203792397, "rank": 0.0}
|
| 35 |
+
{"step": 35, "steps": 250, "lr": 4.9346190892086174e-05, "grad_norm": 2.1157002449035645, "loss_scale": 4.0, "retries": 0, "seconds": 162.3, "max_memory_gb": 17.08, "ce": 0.10860981678342796, "rank": 0.0}
|
| 36 |
+
{"step": 36, "steps": 250, "lr": 4.927193266778101e-05, "grad_norm": 2.1934597492218018, "loss_scale": 4.0, "retries": 0, "seconds": 167.0, "max_memory_gb": 17.08, "ce": 0.24866484542144462, "rank": 0.12118687480688095}
|
| 37 |
+
{"step": 37, "steps": 250, "lr": 4.919375017700515e-05, "grad_norm": 1.3259915113449097, "loss_scale": 4.0, "retries": 0, "seconds": 171.7, "max_memory_gb": 17.08, "ce": 0.13517935152776772, "rank": 0.08644936233758926}
|
| 38 |
+
{"step": 38, "steps": 250, "lr": 4.911165751170588e-05, "grad_norm": 0.8479568958282471, "loss_scale": 4.0, "retries": 0, "seconds": 176.3, "max_memory_gb": 17.08, "ce": 0.04736131393769938, "rank": 0.0}
|
| 39 |
+
{"step": 39, "steps": 250, "lr": 4.902566946861708e-05, "grad_norm": 3.027576446533203, "loss_scale": 4.0, "retries": 0, "seconds": 181.0, "max_memory_gb": 17.08, "ce": 0.2418913576586874, "rank": 0.0}
|
| 40 |
+
{"step": 40, "steps": 250, "lr": 4.89358015465922e-05, "grad_norm": 0.7845349311828613, "loss_scale": 4.0, "retries": 0, "seconds": 185.7, "max_memory_gb": 17.08, "ce": 0.05329810753089248, "rank": 0.0}
|
| 41 |
+
{"step": 41, "steps": 250, "lr": 4.884206994381078e-05, "grad_norm": 0.9732480645179749, "loss_scale": 4.0, "retries": 0, "seconds": 190.3, "max_memory_gb": 17.08, "ce": 0.027044994933930866, "rank": 0.10201050341129303}
|
| 42 |
+
{"step": 42, "steps": 250, "lr": 4.874449155485866e-05, "grad_norm": 6.987360954284668, "loss_scale": 4.0, "retries": 0, "seconds": 195.1, "max_memory_gb": 17.08, "ce": 0.35651231265774186, "rank": 0.0}
|
| 43 |
+
{"step": 43, "steps": 250, "lr": 4.864308396768294e-05, "grad_norm": 19.400314331054688, "loss_scale": 4.0, "retries": 0, "seconds": 199.8, "max_memory_gb": 17.08, "ce": 0.42005902642267756, "rank": 0.0}
|
| 44 |
+
{"step": 44, "steps": 250, "lr": 4.853786546042184e-05, "grad_norm": 0.7323870658874512, "loss_scale": 4.0, "retries": 0, "seconds": 204.6, "max_memory_gb": 17.08, "ce": 0.01817928276705061, "rank": 0.0}
|
| 45 |
+
{"step": 45, "steps": 250, "lr": 4.842885499811012e-05, "grad_norm": 1.5360339879989624, "loss_scale": 4.0, "retries": 0, "seconds": 209.4, "max_memory_gb": 17.08, "ce": 0.11590165841334965, "rank": 0.0}
|
| 46 |
+
{"step": 46, "steps": 250, "lr": 4.831607222926078e-05, "grad_norm": 2.741771697998047, "loss_scale": 4.0, "retries": 0, "seconds": 214.0, "max_memory_gb": 17.08, "ce": 0.33487329436343316, "rank": 0.0}
|
| 47 |
+
{"step": 47, "steps": 250, "lr": 4.8199537482323545e-05, "grad_norm": 0.21839465200901031, "loss_scale": 4.0, "retries": 0, "seconds": 218.5, "max_memory_gb": 17.08, "ce": 0.00880005183034882, "rank": 0.0}
|
| 48 |
+
{"step": 48, "steps": 250, "lr": 4.807927176202073e-05, "grad_norm": 1.9628221988677979, "loss_scale": 4.0, "retries": 0, "seconds": 223.1, "max_memory_gb": 17.08, "ce": 0.07585377436276275, "rank": 0.0}
|
| 49 |
+
{"step": 49, "steps": 250, "lr": 4.795529674556129e-05, "grad_norm": 0.5416969656944275, "loss_scale": 4.0, "retries": 0, "seconds": 228.0, "max_memory_gb": 17.08, "ce": 0.016026174442231422, "rank": 0.1264568716287613}
|
| 50 |
+
{"step": 50, "steps": 250, "lr": 4.782763477873361e-05, "grad_norm": 0.30954957008361816, "loss_scale": 4.0, "retries": 0, "seconds": 232.6, "max_memory_gb": 17.08, "ce": 0.006967632282680825, "rank": 0.0}
|
| 51 |
+
{"step": 51, "steps": 250, "lr": 4.769630887187782e-05, "grad_norm": 1.4616656303405762, "loss_scale": 4.0, "retries": 0, "seconds": 237.0, "max_memory_gb": 17.08, "ce": 0.07133848603871229, "rank": 0.0}
|
| 52 |
+
{"step": 52, "steps": 250, "lr": 4.756134269573832e-05, "grad_norm": 5.37498664855957, "loss_scale": 4.0, "retries": 0, "seconds": 241.8, "max_memory_gb": 17.08, "ce": 0.30336370708755567, "rank": 0.0}
|
| 53 |
+
{"step": 53, "steps": 250, "lr": 4.7422760577197226e-05, "grad_norm": 4.702791690826416, "loss_scale": 4.0, "retries": 0, "seconds": 246.5, "max_memory_gb": 17.08, "ce": 0.4808191733404108, "rank": 0.0}
|
| 54 |
+
{"step": 54, "steps": 250, "lr": 4.728058749488963e-05, "grad_norm": 1.950957179069519, "loss_scale": 4.0, "retries": 0, "seconds": 251.1, "max_memory_gb": 17.08, "ce": 0.25638746086042374, "rank": 0.0}
|
| 55 |
+
{"step": 55, "steps": 250, "lr": 4.713484907470133e-05, "grad_norm": 1.0607430934906006, "loss_scale": 4.0, "retries": 0, "seconds": 256.0, "max_memory_gb": 17.08, "ce": 0.01746274014294613, "rank": 0.0}
|
| 56 |
+
{"step": 56, "steps": 250, "lr": 4.6985571585149876e-05, "grad_norm": 1.184228539466858, "loss_scale": 4.0, "retries": 0, "seconds": 260.8, "max_memory_gb": 17.08, "ce": 0.07180308139140834, "rank": 0.0}
|
| 57 |
+
{"step": 57, "steps": 250, "lr": 4.6832781932649884e-05, "grad_norm": 2.4631755352020264, "loss_scale": 4.0, "retries": 0, "seconds": 265.5, "max_memory_gb": 17.08, "ce": 0.17298678088900488, "rank": 0.06867493689060211}
|
| 58 |
+
{"step": 58, "steps": 250, "lr": 4.667650765666325e-05, "grad_norm": 1.237842082977295, "loss_scale": 4.0, "retries": 0, "seconds": 270.1, "max_memory_gb": 17.08, "ce": 0.04050578845408381, "rank": 0.0}
|
| 59 |
+
{"step": 59, "steps": 250, "lr": 4.651677692473538e-05, "grad_norm": 1.109027624130249, "loss_scale": 4.0, "retries": 0, "seconds": 275.0, "max_memory_gb": 17.08, "ce": 0.012290761262590877, "rank": 0.0}
|
| 60 |
+
{"step": 60, "steps": 250, "lr": 4.635361852741808e-05, "grad_norm": 1.4177950620651245, "loss_scale": 4.0, "retries": 0, "seconds": 279.7, "max_memory_gb": 17.08, "ce": 0.020924392914423606, "rank": 0.0}
|
| 61 |
+
{"step": 61, "steps": 250, "lr": 4.6187061873080264e-05, "grad_norm": 3.3049333095550537, "loss_scale": 4.0, "retries": 0, "seconds": 284.3, "max_memory_gb": 17.08, "ce": 0.09414947214099811, "rank": 0.0}
|
| 62 |
+
{"step": 62, "steps": 250, "lr": 4.601713698260727e-05, "grad_norm": 4.777611255645752, "loss_scale": 4.0, "retries": 0, "seconds": 289.1, "max_memory_gb": 17.08, "ce": 0.3924443584396613, "rank": 0.0}
|
| 63 |
+
{"step": 63, "steps": 250, "lr": 4.5843874483989746e-05, "grad_norm": 0.14109013974666595, "loss_scale": 4.0, "retries": 0, "seconds": 293.7, "max_memory_gb": 17.08, "ce": 0.009122358107333639, "rank": 0.12500816583633423}
|
| 64 |
+
{"step": 64, "steps": 250, "lr": 4.566730560680316e-05, "grad_norm": 2.7130541801452637, "loss_scale": 4.0, "retries": 0, "seconds": 298.4, "max_memory_gb": 17.08, "ce": 0.2317647449162905, "rank": 0.023255303502082825}
|
| 65 |
+
{"step": 65, "steps": 250, "lr": 4.548746217657878e-05, "grad_norm": 0.2514759302139282, "loss_scale": 4.0, "retries": 0, "seconds": 303.3, "max_memory_gb": 17.08, "ce": 0.005555404927690688, "rank": 0.0}
|
| 66 |
+
{"step": 66, "steps": 250, "lr": 4.5304376609067396e-05, "grad_norm": 0.15214847028255463, "loss_scale": 4.0, "retries": 0, "seconds": 308.1, "max_memory_gb": 17.08, "ce": 0.0073639116944832494, "rank": 0.0}
|
| 67 |
+
{"step": 67, "steps": 250, "lr": 4.5118081904396504e-05, "grad_norm": 4.358682155609131, "loss_scale": 4.0, "retries": 0, "seconds": 312.7, "max_memory_gb": 17.08, "ce": 0.3685140227298689, "rank": 0.0}
|
| 68 |
+
{"step": 68, "steps": 250, "lr": 4.4928611641122225e-05, "grad_norm": 0.5207884907722473, "loss_scale": 4.0, "retries": 0, "seconds": 317.4, "max_memory_gb": 17.08, "ce": 0.015618573921528878, "rank": 0.0}
|
| 69 |
+
{"step": 69, "steps": 250, "lr": 4.473599997017701e-05, "grad_norm": 0.6232402920722961, "loss_scale": 4.0, "retries": 0, "seconds": 322.3, "max_memory_gb": 17.08, "ce": 0.05692675948193937, "rank": 0.0}
|
| 70 |
+
{"step": 70, "steps": 250, "lr": 4.4540281608714053e-05, "grad_norm": 6.404812812805176, "loss_scale": 4.0, "retries": 0, "seconds": 326.8, "max_memory_gb": 17.08, "ce": 0.05496409052648232, "rank": 0.0}
|
| 71 |
+
{"step": 71, "steps": 250, "lr": 4.434149183384977e-05, "grad_norm": 0.46975138783454895, "loss_scale": 4.0, "retries": 0, "seconds": 331.5, "max_memory_gb": 17.08, "ce": 0.017292771862628342, "rank": 0.0}
|
| 72 |
+
{"step": 72, "steps": 250, "lr": 4.413966647630534e-05, "grad_norm": 0.4352131485939026, "loss_scale": 4.0, "retries": 0, "seconds": 336.2, "max_memory_gb": 17.08, "ce": 0.008102779592235265, "rank": 0.0}
|
| 73 |
+
{"step": 73, "steps": 250, "lr": 4.3934841913948323e-05, "grad_norm": 1.1280144453048706, "loss_scale": 4.0, "retries": 0, "seconds": 340.7, "max_memory_gb": 17.08, "ce": 0.07289412886677837, "rank": 0.0}
|
| 74 |
+
{"step": 74, "steps": 250, "lr": 4.372705506523583e-05, "grad_norm": 0.5265095233917236, "loss_scale": 4.0, "retries": 0, "seconds": 345.0, "max_memory_gb": 17.08, "ce": 0.02520757023467013, "rank": 0.0}
|
| 75 |
+
{"step": 75, "steps": 250, "lr": 4.351634338256017e-05, "grad_norm": 1.7468034029006958, "loss_scale": 4.0, "retries": 0, "seconds": 349.6, "max_memory_gb": 17.08, "ce": 0.049460263765354284, "rank": 0.0}
|
| 76 |
+
{"step": 76, "steps": 250, "lr": 4.330274484549825e-05, "grad_norm": 2.027005195617676, "loss_scale": 4.0, "retries": 0, "seconds": 354.3, "max_memory_gb": 17.08, "ce": 0.46957844028526097, "rank": 0.0}
|
| 77 |
+
{"step": 77, "steps": 250, "lr": 4.308629795396599e-05, "grad_norm": 0.0992317795753479, "loss_scale": 4.0, "retries": 0, "seconds": 358.8, "max_memory_gb": 17.08, "ce": 0.0039532803743256295, "rank": 0.0}
|
| 78 |
+
{"step": 78, "steps": 250, "lr": 4.286704172127894e-05, "grad_norm": 1.5385271310806274, "loss_scale": 4.0, "retries": 0, "seconds": 363.7, "max_memory_gb": 17.08, "ce": 0.2620131675982975, "rank": 0.0}
|
| 79 |
+
{"step": 79, "steps": 250, "lr": 4.264501566712033e-05, "grad_norm": 0.8796354532241821, "loss_scale": 4.0, "retries": 0, "seconds": 368.4, "max_memory_gb": 17.08, "ce": 0.03256119203342678, "rank": 0.0}
|
| 80 |
+
{"step": 80, "steps": 250, "lr": 4.2420259810417894e-05, "grad_norm": 3.0472500324249268, "loss_scale": 4.0, "retries": 0, "seconds": 373.1, "max_memory_gb": 17.08, "ce": 0.21283848066524058, "rank": 0.12499503791332245}
|
| 81 |
+
{"step": 81, "steps": 250, "lr": 4.219281466213066e-05, "grad_norm": 1.2782243490219116, "loss_scale": 4.0, "retries": 0, "seconds": 377.8, "max_memory_gb": 17.08, "ce": 0.09096623907407775, "rank": 0.1228083148598671}
|
| 82 |
+
{"step": 82, "steps": 250, "lr": 4.1962721217947135e-05, "grad_norm": 0.2888147234916687, "loss_scale": 4.0, "retries": 0, "seconds": 382.4, "max_memory_gb": 17.08, "ce": 0.012013811190627166, "rank": 0.0}
|
| 83 |
+
{"step": 83, "steps": 250, "lr": 4.173002095089599e-05, "grad_norm": 1.0692424774169922, "loss_scale": 4.0, "retries": 0, "seconds": 386.9, "max_memory_gb": 17.08, "ce": 0.0644334034222993, "rank": 0.0}
|
| 84 |
+
{"step": 84, "steps": 250, "lr": 4.149475580387084e-05, "grad_norm": 1.3331784009933472, "loss_scale": 4.0, "retries": 0, "seconds": 391.8, "max_memory_gb": 17.08, "ce": 0.09229576059487954, "rank": 0.0}
|
| 85 |
+
{"step": 85, "steps": 250, "lr": 4.1256968182070275e-05, "grad_norm": 0.6823840737342834, "loss_scale": 4.0, "retries": 0, "seconds": 396.4, "max_memory_gb": 17.08, "ce": 0.014524101435995362, "rank": 0.0}
|
| 86 |
+
{"step": 86, "steps": 250, "lr": 4.101670094535453e-05, "grad_norm": 1.2043994665145874, "loss_scale": 4.0, "retries": 0, "seconds": 401.2, "max_memory_gb": 17.08, "ce": 0.12422405806864845, "rank": 0.0}
|
| 87 |
+
{"step": 87, "steps": 250, "lr": 4.077399740052027e-05, "grad_norm": 0.11679844558238983, "loss_scale": 4.0, "retries": 0, "seconds": 405.8, "max_memory_gb": 17.08, "ce": 0.005177040100079466, "rank": 0.0}
|
| 88 |
+
{"step": 88, "steps": 250, "lr": 4.0528901293494775e-05, "grad_norm": 0.663480281829834, "loss_scale": 4.0, "retries": 0, "seconds": 410.5, "max_memory_gb": 17.08, "ce": 0.02025400212733075, "rank": 0.0}
|
| 89 |
+
{"step": 89, "steps": 250, "lr": 4.0281456801451004e-05, "grad_norm": 2.702471971511841, "loss_scale": 4.0, "retries": 0, "seconds": 415.2, "max_memory_gb": 17.08, "ce": 0.12053665080202336, "rank": 0.0}
|
| 90 |
+
{"step": 90, "steps": 250, "lr": 4.0031708524844884e-05, "grad_norm": 0.65545254945755, "loss_scale": 4.0, "retries": 0, "seconds": 419.8, "max_memory_gb": 17.08, "ce": 0.031328407381806755, "rank": 0.0}
|
| 91 |
+
{"step": 91, "steps": 250, "lr": 3.977970147937635e-05, "grad_norm": 1.4477813243865967, "loss_scale": 4.0, "retries": 0, "seconds": 424.4, "max_memory_gb": 17.08, "ce": 0.015845761056880292, "rank": 0.0}
|
| 92 |
+
{"step": 92, "steps": 250, "lr": 3.952548108787553e-05, "grad_norm": 20.506454467773438, "loss_scale": 4.0, "retries": 0, "seconds": 429.2, "max_memory_gb": 17.08, "ce": 0.43802356343621796, "rank": 0.0}
|
| 93 |
+
{"step": 93, "steps": 250, "lr": 3.9269093172115496e-05, "grad_norm": 10.074060440063477, "loss_scale": 4.0, "retries": 0, "seconds": 433.5, "max_memory_gb": 17.08, "ce": 0.6923764244729682, "rank": 0.0}
|
| 94 |
+
{"step": 94, "steps": 250, "lr": 3.901058394455323e-05, "grad_norm": 0.4188583493232727, "loss_scale": 4.0, "retries": 0, "seconds": 438.0, "max_memory_gb": 17.08, "ce": 0.031501984814326534, "rank": 0.0}
|
| 95 |
+
{"step": 95, "steps": 250, "lr": 3.875e-05, "grad_norm": 4.268287181854248, "loss_scale": 4.0, "retries": 0, "seconds": 442.6, "max_memory_gb": 17.08, "ce": 0.4624792763352161, "rank": 0.0}
|
| 96 |
+
{"step": 96, "steps": 250, "lr": 3.8487388307223005e-05, "grad_norm": 0.872646689414978, "loss_scale": 4.0, "retries": 0, "seconds": 447.4, "max_memory_gb": 17.08, "ce": 0.0732316137818998, "rank": 0.0}
|
| 97 |
+
{"step": 97, "steps": 250, "lr": 3.822279620047943e-05, "grad_norm": 0.2936825156211853, "loss_scale": 4.0, "retries": 0, "seconds": 452.1, "max_memory_gb": 17.08, "ce": 0.015408379209475243, "rank": 0.0}
|
| 98 |
+
{"step": 98, "steps": 250, "lr": 3.795627137098479e-05, "grad_norm": 0.14519180357456207, "loss_scale": 4.0, "retries": 0, "seconds": 456.7, "max_memory_gb": 17.08, "ce": 0.008257250277893036, "rank": 0.0}
|
| 99 |
+
{"step": 99, "steps": 250, "lr": 3.7687861858316844e-05, "grad_norm": 1.3866770267486572, "loss_scale": 4.0, "retries": 0, "seconds": 461.3, "max_memory_gb": 17.08, "ce": 0.05427673847589176, "rank": 0.0}
|
| 100 |
+
{"step": 100, "steps": 250, "lr": 3.741761604175668e-05, "grad_norm": 0.7893432378768921, "loss_scale": 4.0, "retries": 0, "seconds": 465.9, "max_memory_gb": 17.08, "ce": 0.06649536487111618, "rank": 0.1250016689300537}
|
| 101 |
+
{"step": 101, "steps": 250, "lr": 3.714558263156872e-05, "grad_norm": 2.6571602821350098, "loss_scale": 4.0, "retries": 0, "seconds": 470.6, "max_memory_gb": 17.08, "ce": 0.1515396659876842, "rank": 0.0}
|
| 102 |
+
{"step": 102, "steps": 250, "lr": 3.6871810660220904e-05, "grad_norm": 0.4798254668712616, "loss_scale": 4.0, "retries": 0, "seconds": 475.2, "max_memory_gb": 17.08, "ce": 0.0126075437315194, "rank": 0.232321634888649}
|
| 103 |
+
{"step": 103, "steps": 250, "lr": 3.659634947354686e-05, "grad_norm": 0.06223558261990547, "loss_scale": 4.0, "retries": 0, "seconds": 480.0, "max_memory_gb": 17.08, "ce": 0.002384346500207357, "rank": 0.0}
|
| 104 |
+
{"step": 104, "steps": 250, "lr": 3.631924872185169e-05, "grad_norm": 1.7122623920440674, "loss_scale": 4.0, "retries": 0, "seconds": 484.7, "max_memory_gb": 17.08, "ce": 0.1570407678555057, "rank": 0.0}
|
| 105 |
+
{"step": 105, "steps": 250, "lr": 3.6040558350962674e-05, "grad_norm": 1.4310860633850098, "loss_scale": 4.0, "retries": 0, "seconds": 489.1, "max_memory_gb": 17.08, "ce": 0.1302035754051758, "rank": 0.0}
|
| 106 |
+
{"step": 106, "steps": 250, "lr": 3.576032859322687e-05, "grad_norm": 0.6000993251800537, "loss_scale": 4.0, "retries": 0, "seconds": 493.7, "max_memory_gb": 17.08, "ce": 0.03740129758898547, "rank": 0.0}
|
| 107 |
+
{"step": 107, "steps": 250, "lr": 3.5478609958457056e-05, "grad_norm": 0.391814261674881, "loss_scale": 4.0, "retries": 0, "seconds": 498.4, "max_memory_gb": 17.08, "ce": 0.023658054517909477, "rank": 0.0}
|
| 108 |
+
{"step": 108, "steps": 250, "lr": 3.519545322482755e-05, "grad_norm": 0.6591939330101013, "loss_scale": 4.0, "retries": 0, "seconds": 503.0, "max_memory_gb": 17.08, "ce": 0.024789348277408862, "rank": 0.0}
|
| 109 |
+
{"step": 109, "steps": 250, "lr": 3.491090942972179e-05, "grad_norm": 0.3626777231693268, "loss_scale": 4.0, "retries": 0, "seconds": 507.7, "max_memory_gb": 17.08, "ce": 0.013649861599333235, "rank": 0.0}
|
| 110 |
+
{"step": 110, "steps": 250, "lr": 3.4625029860533135e-05, "grad_norm": 0.0781179890036583, "loss_scale": 4.0, "retries": 0, "seconds": 512.3, "max_memory_gb": 17.08, "ce": 0.0022503566933664843, "rank": 0.0}
|
| 111 |
+
{"step": 111, "steps": 250, "lr": 3.433786604542056e-05, "grad_norm": 2.8439271450042725, "loss_scale": 4.0, "retries": 0, "seconds": 516.8, "max_memory_gb": 17.08, "ce": 0.12326857734569785, "rank": 0.0}
|
| 112 |
+
{"step": 112, "steps": 250, "lr": 3.404946974402104e-05, "grad_norm": 0.2356872707605362, "loss_scale": 4.0, "retries": 0, "seconds": 520.9, "max_memory_gb": 17.08, "ce": 0.0138495939263521, "rank": 0.0}
|
| 113 |
+
{"step": 113, "steps": 250, "lr": 3.375989293812019e-05, "grad_norm": 4.307086944580078, "loss_scale": 4.0, "retries": 0, "seconds": 525.5, "max_memory_gb": 17.08, "ce": 0.5494062178704553, "rank": 0.0}
|
| 114 |
+
{"step": 114, "steps": 250, "lr": 3.3469187822282766e-05, "grad_norm": 1.9508861303329468, "loss_scale": 4.0, "retries": 0, "seconds": 530.0, "max_memory_gb": 17.08, "ce": 0.23929387206226238, "rank": 0.0}
|
| 115 |
+
{"step": 115, "steps": 250, "lr": 3.317740679444499e-05, "grad_norm": 0.42906004190444946, "loss_scale": 4.0, "retries": 0, "seconds": 534.1, "max_memory_gb": 17.08, "ce": 0.015368481562290981, "rank": 0.0}
|
| 116 |
+
{"step": 116, "steps": 250, "lr": 3.2884602446470056e-05, "grad_norm": 4.429605960845947, "loss_scale": 4.0, "retries": 0, "seconds": 538.7, "max_memory_gb": 17.08, "ce": 0.4904902381749707, "rank": 0.0}
|
| 117 |
+
{"step": 117, "steps": 250, "lr": 3.259082755466876e-05, "grad_norm": 2.4481709003448486, "loss_scale": 4.0, "retries": 0, "seconds": 543.2, "max_memory_gb": 17.08, "ce": 0.0446817003094111, "rank": 0.0}
|
| 118 |
+
{"step": 118, "steps": 250, "lr": 3.229613507028686e-05, "grad_norm": 1.0249744653701782, "loss_scale": 4.0, "retries": 0, "seconds": 547.9, "max_memory_gb": 17.08, "ce": 0.018014334907093144, "rank": 0.12619228661060333}
|
| 119 |
+
{"step": 119, "steps": 250, "lr": 3.2000578109961004e-05, "grad_norm": 0.12947461009025574, "loss_scale": 4.0, "retries": 0, "seconds": 552.6, "max_memory_gb": 17.08, "ce": 0.0038280310182017274, "rank": 0.0}
|
| 120 |
+
{"step": 120, "steps": 250, "lr": 3.1704209946144616e-05, "grad_norm": 5.213824272155762, "loss_scale": 4.0, "retries": 0, "seconds": 557.3, "max_memory_gb": 17.08, "ce": 0.2348533358399436, "rank": 0.0}
|
| 121 |
+
{"step": 121, "steps": 250, "lr": 3.140708399750594e-05, "grad_norm": 1.679429054260254, "loss_scale": 4.0, "retries": 0, "seconds": 561.6, "max_memory_gb": 17.08, "ce": 0.14745506506187667, "rank": 0.0}
|
| 122 |
+
{"step": 122, "steps": 250, "lr": 3.1109253819299606e-05, "grad_norm": 2.0046000480651855, "loss_scale": 4.0, "retries": 0, "seconds": 566.2, "max_memory_gb": 17.08, "ce": 0.08289928426165716, "rank": 0.0}
|
| 123 |
+
{"step": 123, "steps": 250, "lr": 3.081077309371361e-05, "grad_norm": 6.3535051345825195, "loss_scale": 4.0, "retries": 0, "seconds": 570.9, "max_memory_gb": 17.08, "ce": 0.48481881464431353, "rank": 0.0}
|
| 124 |
+
{"step": 124, "steps": 250, "lr": 3.051169562019339e-05, "grad_norm": 0.030215244740247726, "loss_scale": 4.0, "retries": 0, "seconds": 575.6, "max_memory_gb": 17.08, "ce": 0.0015912055343960674, "rank": 0.0}
|
| 125 |
+
{"step": 125, "steps": 250, "lr": 3.021207530574477e-05, "grad_norm": 2.976579189300537, "loss_scale": 4.0, "retries": 0, "seconds": 580.3, "max_memory_gb": 17.08, "ce": 0.3043670994356944, "rank": 0.0}
|
| 126 |
+
{"step": 126, "steps": 250, "lr": 2.991196615521758e-05, "grad_norm": 0.3399859368801117, "loss_scale": 4.0, "retries": 0, "seconds": 585.0, "max_memory_gb": 17.08, "ce": 0.018755924602373852, "rank": 0.0}
|
| 127 |
+
{"step": 127, "steps": 250, "lr": 2.9611422261571536e-05, "grad_norm": 0.3885674774646759, "loss_scale": 4.0, "retries": 0, "seconds": 589.6, "max_memory_gb": 17.08, "ce": 0.006429052488556408, "rank": 0.0}
|
| 128 |
+
{"step": 128, "steps": 250, "lr": 2.931049779612634e-05, "grad_norm": 0.7321231365203857, "loss_scale": 4.0, "retries": 0, "seconds": 594.2, "max_memory_gb": 17.08, "ce": 0.017349736818687234, "rank": 0.0}
|
| 129 |
+
{"step": 129, "steps": 250, "lr": 2.9009246998797557e-05, "grad_norm": 0.7767072916030884, "loss_scale": 4.0, "retries": 0, "seconds": 599.0, "max_memory_gb": 17.08, "ce": 0.030202532559997053, "rank": 0.0}
|
| 130 |
+
{"step": 130, "steps": 250, "lr": 2.8707724168320238e-05, "grad_norm": 1.0347352027893066, "loss_scale": 4.0, "retries": 0, "seconds": 603.6, "max_memory_gb": 17.08, "ce": 0.051737159586991766, "rank": 0.0}
|
| 131 |
+
{"step": 131, "steps": 250, "lr": 2.840598365246184e-05, "grad_norm": 1.3923269510269165, "loss_scale": 4.0, "retries": 0, "seconds": 608.2, "max_memory_gb": 17.08, "ce": 0.11104684807469312, "rank": 0.0}
|
| 132 |
+
{"step": 132, "steps": 250, "lr": 2.8104079838226383e-05, "grad_norm": 0.2852421998977661, "loss_scale": 4.0, "retries": 0, "seconds": 612.9, "max_memory_gb": 17.08, "ce": 0.010193469619480311, "rank": 0.0}
|
| 133 |
+
{"step": 133, "steps": 250, "lr": 2.7802067142051446e-05, "grad_norm": 8.754173278808594, "loss_scale": 4.0, "retries": 0, "seconds": 617.6, "max_memory_gb": 17.08, "ce": 0.5173003164418333, "rank": 0.0}
|
| 134 |
+
{"step": 134, "steps": 250, "lr": 2.7500000000000004e-05, "grad_norm": 0.07861241698265076, "loss_scale": 4.0, "retries": 0, "seconds": 622.1, "max_memory_gb": 17.08, "ce": 0.004785235953022493, "rank": 0.0}
|
| 135 |
+
{"step": 135, "steps": 250, "lr": 2.7197932857948566e-05, "grad_norm": 6.263345241546631, "loss_scale": 4.0, "retries": 0, "seconds": 626.7, "max_memory_gb": 17.08, "ce": 0.18211496756384804, "rank": 0.0}
|
| 136 |
+
{"step": 136, "steps": 250, "lr": 2.689592016177363e-05, "grad_norm": 4.972492218017578, "loss_scale": 4.0, "retries": 0, "seconds": 631.5, "max_memory_gb": 17.08, "ce": 0.5854532822932015, "rank": 0.0}
|
| 137 |
+
{"step": 137, "steps": 250, "lr": 2.659401634753817e-05, "grad_norm": 4.044590473175049, "loss_scale": 4.0, "retries": 0, "seconds": 636.1, "max_memory_gb": 17.08, "ce": 0.07680390353982602, "rank": 0.0}
|
| 138 |
+
{"step": 138, "steps": 250, "lr": 2.6292275831679764e-05, "grad_norm": 0.07773441821336746, "loss_scale": 4.0, "retries": 0, "seconds": 640.7, "max_memory_gb": 17.08, "ce": 0.001928278909645087, "rank": 0.09545674920082092}
|
| 139 |
+
{"step": 139, "steps": 250, "lr": 2.5990753001202445e-05, "grad_norm": 0.14490360021591187, "loss_scale": 4.0, "retries": 0, "seconds": 645.4, "max_memory_gb": 17.08, "ce": 0.0057511332383910485, "rank": 0.0}
|
| 140 |
+
{"step": 140, "steps": 250, "lr": 2.5689502203873672e-05, "grad_norm": 0.9352902173995972, "loss_scale": 4.0, "retries": 0, "seconds": 650.2, "max_memory_gb": 17.08, "ce": 0.10031261117501344, "rank": 0.0}
|
| 141 |
+
{"step": 141, "steps": 250, "lr": 2.538857773842847e-05, "grad_norm": 0.05997638404369354, "loss_scale": 4.0, "retries": 0, "seconds": 654.8, "max_memory_gb": 17.08, "ce": 0.0033005804830281704, "rank": 0.0}
|
| 142 |
+
{"step": 142, "steps": 250, "lr": 2.5088033844782426e-05, "grad_norm": 1.8210817575454712, "loss_scale": 4.0, "retries": 0, "seconds": 659.5, "max_memory_gb": 17.08, "ce": 0.1672518637687972, "rank": 0.0}
|
| 143 |
+
{"step": 143, "steps": 250, "lr": 2.4787924694255237e-05, "grad_norm": 2.6694462299346924, "loss_scale": 4.0, "retries": 0, "seconds": 664.4, "max_memory_gb": 17.08, "ce": 0.165599003470561, "rank": 0.0}
|
| 144 |
+
{"step": 144, "steps": 250, "lr": 2.4488304379806627e-05, "grad_norm": 0.2433902621269226, "loss_scale": 4.0, "retries": 0, "seconds": 669.0, "max_memory_gb": 17.08, "ce": 0.012500268767098532, "rank": 0.0}
|
| 145 |
+
{"step": 145, "steps": 250, "lr": 2.4189226906286388e-05, "grad_norm": 0.684335470199585, "loss_scale": 4.0, "retries": 0, "seconds": 673.7, "max_memory_gb": 17.08, "ce": 0.06298316582751795, "rank": 0.0}
|
| 146 |
+
{"step": 146, "steps": 250, "lr": 2.3890746180700395e-05, "grad_norm": 3.5988707542419434, "loss_scale": 4.0, "retries": 0, "seconds": 678.5, "max_memory_gb": 17.08, "ce": 0.1661194906964738, "rank": 0.0}
|
| 147 |
+
{"step": 147, "steps": 250, "lr": 2.359291600249407e-05, "grad_norm": 2.265808343887329, "loss_scale": 4.0, "retries": 0, "seconds": 683.2, "max_memory_gb": 17.08, "ce": 0.06287744876408397, "rank": 0.0}
|
| 148 |
+
{"step": 148, "steps": 250, "lr": 2.3295790053855392e-05, "grad_norm": 0.7449353337287903, "loss_scale": 4.0, "retries": 0, "seconds": 687.8, "max_memory_gb": 17.08, "ce": 0.02919139564210127, "rank": 0.0}
|
| 149 |
+
{"step": 149, "steps": 250, "lr": 2.299942189003901e-05, "grad_norm": 5.384766101837158, "loss_scale": 4.0, "retries": 0, "seconds": 692.6, "max_memory_gb": 17.08, "ce": 0.6580459380184038, "rank": 0.14335358142852783}
|
| 150 |
+
{"step": 150, "steps": 250, "lr": 2.2703864929713144e-05, "grad_norm": 0.6783493161201477, "loss_scale": 4.0, "retries": 0, "seconds": 697.3, "max_memory_gb": 17.08, "ce": 0.06366291317499417, "rank": 0.0}
|
| 151 |
+
{"step": 151, "steps": 250, "lr": 2.2409172445331257e-05, "grad_norm": 1.2306337356567383, "loss_scale": 4.0, "retries": 0, "seconds": 701.9, "max_memory_gb": 17.08, "ce": 0.15251365605524825, "rank": 0.0}
|
| 152 |
+
{"step": 152, "steps": 250, "lr": 2.211539755352996e-05, "grad_norm": 2.542447090148926, "loss_scale": 4.0, "retries": 0, "seconds": 706.6, "max_memory_gb": 17.08, "ce": 0.3320249888993203, "rank": 0.0}
|
| 153 |
+
{"step": 153, "steps": 250, "lr": 2.1822593205555016e-05, "grad_norm": 0.2488982379436493, "loss_scale": 4.0, "retries": 0, "seconds": 711.4, "max_memory_gb": 17.08, "ce": 0.005362922899166733, "rank": 0.0}
|
| 154 |
+
{"step": 154, "steps": 250, "lr": 2.1530812177717236e-05, "grad_norm": 1.9267512559890747, "loss_scale": 4.0, "retries": 0, "seconds": 715.9, "max_memory_gb": 17.08, "ce": 0.06979422974359295, "rank": 0.0}
|
| 155 |
+
{"step": 155, "steps": 250, "lr": 2.1240107061879815e-05, "grad_norm": 0.3573933243751526, "loss_scale": 4.0, "retries": 0, "seconds": 720.8, "max_memory_gb": 17.08, "ce": 0.014216844445854804, "rank": 0.0}
|
| 156 |
+
{"step": 156, "steps": 250, "lr": 2.095053025597896e-05, "grad_norm": 0.8037008047103882, "loss_scale": 4.0, "retries": 0, "seconds": 725.6, "max_memory_gb": 17.08, "ce": 0.04381386890418071, "rank": 0.0}
|
| 157 |
+
{"step": 157, "steps": 250, "lr": 2.066213395457945e-05, "grad_norm": 1.8418055772781372, "loss_scale": 4.0, "retries": 0, "seconds": 730.4, "max_memory_gb": 17.08, "ce": 0.3274974525404559, "rank": 0.0}
|
| 158 |
+
{"step": 158, "steps": 250, "lr": 2.0374970139466877e-05, "grad_norm": 0.1882912963628769, "loss_scale": 4.0, "retries": 0, "seconds": 735.0, "max_memory_gb": 17.08, "ce": 0.0097693689431253, "rank": 0.0}
|
| 159 |
+
{"step": 159, "steps": 250, "lr": 2.0089090570278213e-05, "grad_norm": 0.22279579937458038, "loss_scale": 4.0, "retries": 0, "seconds": 739.7, "max_memory_gb": 17.08, "ce": 0.003278932983107552, "rank": 0.12492374330759048}
|
| 160 |
+
{"step": 160, "steps": 250, "lr": 1.9804546775172457e-05, "grad_norm": 0.2844611406326294, "loss_scale": 4.0, "retries": 0, "seconds": 744.4, "max_memory_gb": 17.08, "ce": 0.021428225020372338, "rank": 0.0}
|
| 161 |
+
{"step": 161, "steps": 250, "lr": 1.9521390041542946e-05, "grad_norm": 0.5136367678642273, "loss_scale": 4.0, "retries": 0, "seconds": 749.0, "max_memory_gb": 17.08, "ce": 0.06117370566971658, "rank": 0.0}
|
| 162 |
+
{"step": 162, "steps": 250, "lr": 1.923967140677313e-05, "grad_norm": 0.6631088256835938, "loss_scale": 4.0, "retries": 0, "seconds": 753.8, "max_memory_gb": 17.08, "ce": 0.029577137886917626, "rank": 0.0}
|
| 163 |
+
{"step": 163, "steps": 250, "lr": 1.8959441649037335e-05, "grad_norm": 1.2172081470489502, "loss_scale": 4.0, "retries": 0, "seconds": 758.3, "max_memory_gb": 17.08, "ce": 0.07428083310196598, "rank": 0.0}
|
| 164 |
+
{"step": 164, "steps": 250, "lr": 1.868075127814832e-05, "grad_norm": 0.6041750907897949, "loss_scale": 4.0, "retries": 0, "seconds": 763.0, "max_memory_gb": 17.08, "ce": 0.03216958864777553, "rank": 0.0}
|
| 165 |
+
{"step": 165, "steps": 250, "lr": 1.8403650526453143e-05, "grad_norm": 0.08842422068119049, "loss_scale": 4.0, "retries": 0, "seconds": 767.8, "max_memory_gb": 17.08, "ce": 0.002692909406960098, "rank": 0.0}
|
| 166 |
+
{"step": 166, "steps": 250, "lr": 1.812818933977912e-05, "grad_norm": 0.15784144401550293, "loss_scale": 4.0, "retries": 0, "seconds": 772.8, "max_memory_gb": 17.08, "ce": 0.004337714022312866, "rank": 0.0}
|
| 167 |
+
{"step": 167, "steps": 250, "lr": 1.785441736843128e-05, "grad_norm": 2.8269245624542236, "loss_scale": 4.0, "retries": 0, "seconds": 777.5, "max_memory_gb": 17.08, "ce": 0.31434813024861796, "rank": 0.028603702783584595}
|
| 168 |
+
{"step": 168, "steps": 250, "lr": 1.7582383958243314e-05, "grad_norm": 0.8097910284996033, "loss_scale": 4.0, "retries": 0, "seconds": 782.1, "max_memory_gb": 17.08, "ce": 0.0886411692144975, "rank": 0.0}
|
| 169 |
+
{"step": 169, "steps": 250, "lr": 1.7312138141683164e-05, "grad_norm": 0.17564383149147034, "loss_scale": 4.0, "retries": 0, "seconds": 786.7, "max_memory_gb": 17.08, "ce": 0.0038350571867340477, "rank": 0.0}
|
| 170 |
+
{"step": 170, "steps": 250, "lr": 1.704372862901521e-05, "grad_norm": 2.1763315200805664, "loss_scale": 4.0, "retries": 0, "seconds": 791.5, "max_memory_gb": 17.08, "ce": 0.11642362395832606, "rank": 0.0}
|
| 171 |
+
{"step": 171, "steps": 250, "lr": 1.6777203799520573e-05, "grad_norm": 1.314692497253418, "loss_scale": 4.0, "retries": 0, "seconds": 796.1, "max_memory_gb": 17.08, "ce": 0.05391111531662318, "rank": 0.0}
|
| 172 |
+
{"step": 172, "steps": 250, "lr": 1.6512611692777008e-05, "grad_norm": 2.830461263656616, "loss_scale": 4.0, "retries": 0, "seconds": 800.7, "max_memory_gb": 17.08, "ce": 0.19767361638560033, "rank": 0.0}
|
| 173 |
+
{"step": 173, "steps": 250, "lr": 1.6250000000000005e-05, "grad_norm": 0.8530616164207458, "loss_scale": 4.0, "retries": 0, "seconds": 805.4, "max_memory_gb": 17.08, "ce": 0.04097604548996969, "rank": 0.0}
|
| 174 |
+
{"step": 174, "steps": 250, "lr": 1.598941605544678e-05, "grad_norm": 6.15323543548584, "loss_scale": 4.0, "retries": 0, "seconds": 810.0, "max_memory_gb": 17.08, "ce": 0.30566799781308873, "rank": 0.1780332773923874}
|
| 175 |
+
{"step": 175, "steps": 250, "lr": 1.5730906827884506e-05, "grad_norm": 1.8121833801269531, "loss_scale": 4.0, "retries": 0, "seconds": 814.6, "max_memory_gb": 17.08, "ce": 0.1711098377459166, "rank": 0.0}
|
| 176 |
+
{"step": 176, "steps": 250, "lr": 1.5474518912124476e-05, "grad_norm": 4.560214042663574, "loss_scale": 4.0, "retries": 0, "seconds": 819.3, "max_memory_gb": 17.08, "ce": 0.3399432661854007, "rank": 0.0}
|
| 177 |
+
{"step": 177, "steps": 250, "lr": 1.5220298520623646e-05, "grad_norm": 1.1573890447616577, "loss_scale": 4.0, "retries": 0, "seconds": 823.9, "max_memory_gb": 17.08, "ce": 0.05091378909537525, "rank": 0.0}
|
| 178 |
+
{"step": 178, "steps": 250, "lr": 1.4968291475155121e-05, "grad_norm": 0.06602068990468979, "loss_scale": 4.0, "retries": 0, "seconds": 828.6, "max_memory_gb": 17.08, "ce": 0.001517828357691542, "rank": 0.0}
|
| 179 |
+
{"step": 179, "steps": 250, "lr": 1.4718543198548998e-05, "grad_norm": 1.2227765321731567, "loss_scale": 4.0, "retries": 0, "seconds": 833.1, "max_memory_gb": 17.08, "ce": 0.07031621853025172, "rank": 0.0}
|
| 180 |
+
{"step": 180, "steps": 250, "lr": 1.4471098706505227e-05, "grad_norm": 2.306497573852539, "loss_scale": 4.0, "retries": 0, "seconds": 838.0, "max_memory_gb": 17.08, "ce": 0.026572514891995525, "rank": 0.12504546344280243}
|
| 181 |
+
{"step": 181, "steps": 250, "lr": 1.4226002599479743e-05, "grad_norm": 1.8943291902542114, "loss_scale": 4.0, "retries": 0, "seconds": 842.6, "max_memory_gb": 17.08, "ce": 0.11150740309903995, "rank": 0.0}
|
| 182 |
+
{"step": 182, "steps": 250, "lr": 1.3983299054645471e-05, "grad_norm": 0.18868118524551392, "loss_scale": 4.0, "retries": 0, "seconds": 847.3, "max_memory_gb": 17.08, "ce": 0.005270452227705391, "rank": 0.0}
|
| 183 |
+
{"step": 183, "steps": 250, "lr": 1.3743031817929724e-05, "grad_norm": 3.8634555339813232, "loss_scale": 4.0, "retries": 0, "seconds": 852.0, "max_memory_gb": 17.08, "ce": 0.17796826349785988, "rank": 0.0}
|
| 184 |
+
{"step": 184, "steps": 250, "lr": 1.3505244196129164e-05, "grad_norm": 2.4938273429870605, "loss_scale": 4.0, "retries": 0, "seconds": 856.7, "max_memory_gb": 17.08, "ce": 0.20640138417002163, "rank": 0.0}
|
| 185 |
+
{"step": 185, "steps": 250, "lr": 1.3269979049104014e-05, "grad_norm": 4.539885997772217, "loss_scale": 4.0, "retries": 0, "seconds": 861.2, "max_memory_gb": 17.08, "ce": 0.1286649363937613, "rank": 0.0}
|
| 186 |
+
{"step": 186, "steps": 250, "lr": 1.3037278782052863e-05, "grad_norm": 2.994295120239258, "loss_scale": 4.0, "retries": 0, "seconds": 865.9, "max_memory_gb": 17.08, "ce": 0.1503748214290681, "rank": 0.0}
|
| 187 |
+
{"step": 187, "steps": 250, "lr": 1.2807185337869343e-05, "grad_norm": 2.8625776767730713, "loss_scale": 4.0, "retries": 0, "seconds": 870.6, "max_memory_gb": 17.08, "ce": 0.12503465841518846, "rank": 0.0}
|
| 188 |
+
{"step": 188, "steps": 250, "lr": 1.2579740189582115e-05, "grad_norm": 0.3194305896759033, "loss_scale": 4.0, "retries": 0, "seconds": 875.1, "max_memory_gb": 17.08, "ce": 0.01781422319606918, "rank": 0.12498937547206879}
|
| 189 |
+
{"step": 189, "steps": 250, "lr": 1.2354984332879668e-05, "grad_norm": 0.2016531229019165, "loss_scale": 4.0, "retries": 0, "seconds": 879.7, "max_memory_gb": 17.08, "ce": 0.00907518605754376, "rank": 0.0}
|
| 190 |
+
{"step": 190, "steps": 250, "lr": 1.2132958278721064e-05, "grad_norm": 2.909085988998413, "loss_scale": 4.0, "retries": 0, "seconds": 884.4, "max_memory_gb": 17.08, "ce": 0.040545474354985345, "rank": 0.0}
|
| 191 |
+
{"step": 191, "steps": 250, "lr": 1.1913702046034016e-05, "grad_norm": 0.0839397981762886, "loss_scale": 4.0, "retries": 0, "seconds": 889.1, "max_memory_gb": 17.08, "ce": 0.00320932300428467, "rank": 0.12492804229259491}
|
| 192 |
+
{"step": 192, "steps": 250, "lr": 1.1697255154501756e-05, "grad_norm": 1.6405528783798218, "loss_scale": 4.0, "retries": 0, "seconds": 893.7, "max_memory_gb": 17.08, "ce": 0.08454395689477678, "rank": 0.0}
|
| 193 |
+
{"step": 193, "steps": 250, "lr": 1.1483656617439842e-05, "grad_norm": 0.22890885174274445, "loss_scale": 4.0, "retries": 0, "seconds": 898.5, "max_memory_gb": 17.08, "ce": 0.006754092224127817, "rank": 0.0}
|
| 194 |
+
{"step": 194, "steps": 250, "lr": 1.1272944934764177e-05, "grad_norm": 1.231444001197815, "loss_scale": 4.0, "retries": 0, "seconds": 903.1, "max_memory_gb": 17.08, "ce": 0.029754064782082423, "rank": 0.0}
|
| 195 |
+
{"step": 195, "steps": 250, "lr": 1.1065158086051684e-05, "grad_norm": 2.340461254119873, "loss_scale": 4.0, "retries": 0, "seconds": 907.8, "max_memory_gb": 17.08, "ce": 0.10978584138410952, "rank": 0.0}
|
| 196 |
+
{"step": 196, "steps": 250, "lr": 1.086033352369467e-05, "grad_norm": 0.08239416033029556, "loss_scale": 4.0, "retries": 0, "seconds": 912.6, "max_memory_gb": 17.08, "ce": 0.0017505658548770953, "rank": 0.0}
|
| 197 |
+
{"step": 197, "steps": 250, "lr": 1.0658508166150224e-05, "grad_norm": 0.7343981266021729, "loss_scale": 4.0, "retries": 0, "seconds": 917.5, "max_memory_gb": 17.08, "ce": 0.0006169856129645268, "rank": 0.023961372673511505}
|
| 198 |
+
{"step": 198, "steps": 250, "lr": 1.0459718391285954e-05, "grad_norm": 1.0100876092910767, "loss_scale": 4.0, "retries": 0, "seconds": 922.1, "max_memory_gb": 17.08, "ce": 0.09962311936124024, "rank": 0.0}
|
| 199 |
+
{"step": 199, "steps": 250, "lr": 1.0264000029822999e-05, "grad_norm": 0.05292070657014847, "loss_scale": 4.0, "retries": 0, "seconds": 926.9, "max_memory_gb": 17.08, "ce": 0.000713274874669878, "rank": 0.0}
|
| 200 |
+
{"step": 200, "steps": 250, "lr": 1.0071388358877775e-05, "grad_norm": 1.646235704421997, "loss_scale": 4.0, "retries": 0, "seconds": 931.8, "max_memory_gb": 17.08, "ce": 0.21201330076291924, "rank": 0.0}
|
| 201 |
+
{"step": 201, "steps": 250, "lr": 9.881918095603501e-06, "grad_norm": 0.6166127324104309, "loss_scale": 4.0, "retries": 0, "seconds": 936.4, "max_memory_gb": 17.08, "ce": 0.024713195758522488, "rank": 0.0}
|
| 202 |
+
{"step": 202, "steps": 250, "lr": 9.695623390932612e-06, "grad_norm": 0.15646617114543915, "loss_scale": 4.0, "retries": 0, "seconds": 941.0, "max_memory_gb": 17.08, "ce": 0.007766901538388993, "rank": 0.0}
|
| 203 |
+
{"step": 203, "steps": 250, "lr": 9.51253782342123e-06, "grad_norm": 1.0194422006607056, "loss_scale": 4.0, "retries": 0, "seconds": 945.7, "max_memory_gb": 17.08, "ce": 0.03540571120743152, "rank": 0.0}
|
| 204 |
+
{"step": 204, "steps": 250, "lr": 9.332694393196846e-06, "grad_norm": 0.19385236501693726, "loss_scale": 4.0, "retries": 0, "seconds": 950.3, "max_memory_gb": 17.08, "ce": 0.002480868924067181, "rank": 0.0}
|
| 205 |
+
{"step": 205, "steps": 250, "lr": 9.156125516010256e-06, "grad_norm": 2.3957505226135254, "loss_scale": 4.0, "retries": 0, "seconds": 955.0, "max_memory_gb": 17.08, "ce": 0.3216032830334825, "rank": 0.12499384582042694}
|
| 206 |
+
{"step": 206, "steps": 250, "lr": 8.982863017392733e-06, "grad_norm": 1.9785224199295044, "loss_scale": 4.0, "retries": 0, "seconds": 959.6, "max_memory_gb": 17.08, "ce": 0.27564998936441043, "rank": 0.0}
|
| 207 |
+
{"step": 207, "steps": 250, "lr": 8.81293812691974e-06, "grad_norm": 0.21897223591804504, "loss_scale": 4.0, "retries": 0, "seconds": 964.4, "max_memory_gb": 17.08, "ce": 0.00530465917103129, "rank": 0.0}
|
| 208 |
+
{"step": 208, "steps": 250, "lr": 8.646381472581927e-06, "grad_norm": 0.12500806152820587, "loss_scale": 4.0, "retries": 0, "seconds": 969.0, "max_memory_gb": 17.08, "ce": 0.004351432991825277, "rank": 0.0}
|
| 209 |
+
{"step": 209, "steps": 250, "lr": 8.483223075264622e-06, "grad_norm": 0.17574144899845123, "loss_scale": 4.0, "retries": 0, "seconds": 973.5, "max_memory_gb": 17.08, "ce": 0.007114770517546276, "rank": 0.0}
|
| 210 |
+
{"step": 210, "steps": 250, "lr": 8.323492343336747e-06, "grad_norm": 1.8164128065109253, "loss_scale": 4.0, "retries": 0, "seconds": 978.1, "max_memory_gb": 17.08, "ce": 0.03127076910197957, "rank": 0.0}
|
| 211 |
+
{"step": 211, "steps": 250, "lr": 8.167218067350122e-06, "grad_norm": 3.1209816932678223, "loss_scale": 4.0, "retries": 0, "seconds": 982.8, "max_memory_gb": 17.08, "ce": 0.5261967649430517, "rank": 0.0}
|
| 212 |
+
{"step": 212, "steps": 250, "lr": 8.01442841485013e-06, "grad_norm": 0.9453786015510559, "loss_scale": 4.0, "retries": 0, "seconds": 987.4, "max_memory_gb": 17.08, "ce": 0.08740738165624862, "rank": 0.0}
|
| 213 |
+
{"step": 213, "steps": 250, "lr": 7.865150925298678e-06, "grad_norm": 2.0980539321899414, "loss_scale": 4.0, "retries": 0, "seconds": 992.1, "max_memory_gb": 17.08, "ce": 0.0807879764183781, "rank": 0.0}
|
| 214 |
+
{"step": 214, "steps": 250, "lr": 7.719412505110375e-06, "grad_norm": 1.6094887256622314, "loss_scale": 4.0, "retries": 0, "seconds": 996.9, "max_memory_gb": 17.08, "ce": 0.07887853413694756, "rank": 0.0}
|
| 215 |
+
{"step": 215, "steps": 250, "lr": 7.57723942280278e-06, "grad_norm": 0.23680131137371063, "loss_scale": 4.0, "retries": 0, "seconds": 1001.5, "max_memory_gb": 17.08, "ce": 0.009017485134791059, "rank": 0.0}
|
| 216 |
+
{"step": 216, "steps": 250, "lr": 7.438657304261687e-06, "grad_norm": 0.6698941588401794, "loss_scale": 4.0, "retries": 0, "seconds": 1006.1, "max_memory_gb": 17.08, "ce": 0.029875767665316744, "rank": 0.0}
|
| 217 |
+
{"step": 217, "steps": 250, "lr": 7.30369112812219e-06, "grad_norm": 0.15819765627384186, "loss_scale": 4.0, "retries": 0, "seconds": 1010.8, "max_memory_gb": 17.08, "ce": 0.004686170331297035, "rank": 0.0}
|
| 218 |
+
{"step": 218, "steps": 250, "lr": 7.172365221266403e-06, "grad_norm": 1.7480106353759766, "loss_scale": 4.0, "retries": 0, "seconds": 1015.5, "max_memory_gb": 17.08, "ce": 0.1768641093071892, "rank": 0.0}
|
| 219 |
+
{"step": 219, "steps": 250, "lr": 7.0447032544387154e-06, "grad_norm": 0.39525312185287476, "loss_scale": 4.0, "retries": 0, "seconds": 1020.1, "max_memory_gb": 17.08, "ce": 0.007328813281219482, "rank": 0.0}
|
| 220 |
+
{"step": 220, "steps": 250, "lr": 6.920728237979274e-06, "grad_norm": 1.6326851844787598, "loss_scale": 4.0, "retries": 0, "seconds": 1024.9, "max_memory_gb": 17.08, "ce": 0.12199705382045067, "rank": 0.0}
|
| 221 |
+
{"step": 221, "steps": 250, "lr": 6.800462517676456e-06, "grad_norm": 0.5818838477134705, "loss_scale": 4.0, "retries": 0, "seconds": 1029.5, "max_memory_gb": 17.08, "ce": 0.01295090495182194, "rank": 0.0}
|
| 222 |
+
{"step": 222, "steps": 250, "lr": 6.683927770739222e-06, "grad_norm": 7.796609878540039, "loss_scale": 4.0, "retries": 0, "seconds": 1034.2, "max_memory_gb": 17.08, "ce": 0.05876229355544638, "rank": 0.0}
|
| 223 |
+
{"step": 223, "steps": 250, "lr": 6.571145001889892e-06, "grad_norm": 2.5239436626434326, "loss_scale": 4.0, "retries": 0, "seconds": 1039.0, "max_memory_gb": 17.08, "ce": 0.12284652823541364, "rank": 0.0}
|
| 224 |
+
{"step": 224, "steps": 250, "lr": 6.46213453957817e-06, "grad_norm": 4.927499294281006, "loss_scale": 4.0, "retries": 0, "seconds": 1043.7, "max_memory_gb": 17.08, "ce": 0.10661064946384613, "rank": 0.12499240040779114}
|
| 225 |
+
{"step": 225, "steps": 250, "lr": 6.356916032317063e-06, "grad_norm": 0.8357697129249573, "loss_scale": 4.0, "retries": 0, "seconds": 1048.4, "max_memory_gb": 17.08, "ce": 0.03218394126156454, "rank": 0.0}
|
| 226 |
+
{"step": 226, "steps": 250, "lr": 6.255508445141349e-06, "grad_norm": 1.488944411277771, "loss_scale": 4.0, "retries": 0, "seconds": 1053.1, "max_memory_gb": 17.08, "ce": 0.03694644200982111, "rank": 0.0}
|
| 227 |
+
{"step": 227, "steps": 250, "lr": 6.157930056189228e-06, "grad_norm": 1.2886749505996704, "loss_scale": 4.0, "retries": 0, "seconds": 1057.8, "max_memory_gb": 17.08, "ce": 0.10429970646328002, "rank": 0.0}
|
| 228 |
+
{"step": 228, "steps": 250, "lr": 6.064198453407796e-06, "grad_norm": 1.2581472396850586, "loss_scale": 4.0, "retries": 0, "seconds": 1062.0, "max_memory_gb": 17.08, "ce": 0.1083549209283774, "rank": 0.0}
|
| 229 |
+
{"step": 229, "steps": 250, "lr": 5.974330531382935e-06, "grad_norm": 0.0305628702044487, "loss_scale": 4.0, "retries": 0, "seconds": 1066.1, "max_memory_gb": 17.08, "ce": 0.0009765599115780788, "rank": 0.0}
|
| 230 |
+
{"step": 230, "steps": 250, "lr": 5.888342488294125e-06, "grad_norm": 0.6138394474983215, "loss_scale": 4.0, "retries": 0, "seconds": 1071.0, "max_memory_gb": 17.08, "ce": 0.024571615758588905, "rank": 0.0}
|
| 231 |
+
{"step": 231, "steps": 250, "lr": 5.806249822994849e-06, "grad_norm": 2.14902400970459, "loss_scale": 4.0, "retries": 0, "seconds": 1075.6, "max_memory_gb": 17.08, "ce": 0.147008130089489, "rank": 0.0}
|
| 232 |
+
{"step": 232, "steps": 250, "lr": 5.728067332218994e-06, "grad_norm": 0.006471020635217428, "loss_scale": 4.0, "retries": 0, "seconds": 1080.3, "max_memory_gb": 17.08, "ce": 0.00028143622381548994, "rank": 0.12499894201755524}
|
| 233 |
+
{"step": 233, "steps": 250, "lr": 5.65380910791383e-06, "grad_norm": 2.6316165924072266, "loss_scale": 4.0, "retries": 0, "seconds": 1085.0, "max_memory_gb": 17.08, "ce": 0.09596868140079096, "rank": 0.0}
|
| 234 |
+
{"step": 234, "steps": 250, "lr": 5.583488534700022e-06, "grad_norm": 1.9554591178894043, "loss_scale": 4.0, "retries": 0, "seconds": 1089.7, "max_memory_gb": 17.08, "ce": 0.0745889813983922, "rank": 0.0}
|
| 235 |
+
{"step": 235, "steps": 250, "lr": 5.517118287459118e-06, "grad_norm": 5.388273239135742, "loss_scale": 4.0, "retries": 0, "seconds": 1094.1, "max_memory_gb": 17.08, "ce": 1.0265846242521661, "rank": 0.0}
|
| 236 |
+
{"step": 236, "steps": 250, "lr": 5.454710329048978e-06, "grad_norm": 0.1169591024518013, "loss_scale": 4.0, "retries": 0, "seconds": 1098.8, "max_memory_gb": 17.08, "ce": 0.0033803580072913064, "rank": 0.0}
|
| 237 |
+
{"step": 237, "steps": 250, "lr": 5.3962759081475436e-06, "grad_norm": 0.185594379901886, "loss_scale": 4.0, "retries": 0, "seconds": 1103.6, "max_memory_gb": 17.08, "ce": 0.0025583222737282085, "rank": 0.0}
|
| 238 |
+
{"step": 238, "steps": 250, "lr": 5.341825557225321e-06, "grad_norm": 0.3357573449611664, "loss_scale": 4.0, "retries": 0, "seconds": 1107.9, "max_memory_gb": 17.08, "ce": 0.022942228279362098, "rank": 0.0}
|
| 239 |
+
{"step": 239, "steps": 250, "lr": 5.291369090646961e-06, "grad_norm": 1.28952956199646, "loss_scale": 4.0, "retries": 0, "seconds": 1112.0, "max_memory_gb": 17.08, "ce": 0.11286815443600062, "rank": 0.0}
|
| 240 |
+
{"step": 240, "steps": 250, "lr": 5.244915602902289e-06, "grad_norm": 1.7841829061508179, "loss_scale": 4.0, "retries": 0, "seconds": 1116.4, "max_memory_gb": 17.08, "ce": 0.04522084357040512, "rank": 0.0}
|
| 241 |
+
{"step": 241, "steps": 250, "lr": 5.20247346696706e-06, "grad_norm": 1.014228105545044, "loss_scale": 4.0, "retries": 0, "seconds": 1120.7, "max_memory_gb": 17.08, "ce": 0.09255980322927826, "rank": 0.12219125032424927}
|
| 242 |
+
{"step": 242, "steps": 250, "lr": 5.164050332793787e-06, "grad_norm": 0.6693577766418457, "loss_scale": 4.0, "retries": 0, "seconds": 1125.4, "max_memory_gb": 17.08, "ce": 0.03613468877313153, "rank": 0.0}
|
| 243 |
+
{"step": 243, "steps": 250, "lr": 5.129653125932873e-06, "grad_norm": 0.87331622838974, "loss_scale": 4.0, "retries": 0, "seconds": 1130.1, "max_memory_gb": 17.08, "ce": 0.034716816436684894, "rank": 0.0}
|
| 244 |
+
{"step": 244, "steps": 250, "lr": 5.099288046284329e-06, "grad_norm": 0.937362015247345, "loss_scale": 4.0, "retries": 0, "seconds": 1135.0, "max_memory_gb": 17.08, "ce": 0.021504777005361575, "rank": 0.0}
|
| 245 |
+
{"step": 245, "steps": 250, "lr": 5.072960566980277e-06, "grad_norm": 0.36813923716545105, "loss_scale": 4.0, "retries": 0, "seconds": 1138.9, "max_memory_gb": 17.08, "ce": 0.019557929710572353, "rank": 0.0}
|
| 246 |
+
{"step": 246, "steps": 250, "lr": 5.050675433398435e-06, "grad_norm": 1.1346279382705688, "loss_scale": 4.0, "retries": 0, "seconds": 1143.6, "max_memory_gb": 17.08, "ce": 0.11202445301978514, "rank": 0.0}
|
| 247 |
+
{"step": 247, "steps": 250, "lr": 5.032436662306811e-06, "grad_norm": 1.2855141162872314, "loss_scale": 4.0, "retries": 0, "seconds": 1148.3, "max_memory_gb": 17.08, "ce": 0.048527460658988275, "rank": 0.0}
|
| 248 |
+
{"step": 248, "steps": 250, "lr": 5.018247541139684e-06, "grad_norm": 0.04190698266029358, "loss_scale": 4.0, "retries": 0, "seconds": 1152.9, "max_memory_gb": 17.08, "ce": 0.0006081754210924828, "rank": 0.0}
|
| 249 |
+
{"step": 249, "steps": 250, "lr": 5.00811062740508e-06, "grad_norm": 0.46482300758361816, "loss_scale": 4.0, "retries": 0, "seconds": 1157.6, "max_memory_gb": 17.08, "ce": 0.011926783699493626, "rank": 0.0}
|
| 250 |
+
{"step": 250, "steps": 250, "lr": 5.002027748223778e-06, "grad_norm": 4.936796188354492, "loss_scale": 4.0, "retries": 0, "seconds": 1162.1, "max_memory_gb": 17.08, "ce": 0.11591351917877546, "rank": 0.0}
|
runs/fixed_internvl/removal/TopDelta8_s42/adapter/README.md
ADDED
|
@@ -0,0 +1,207 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
base_model: /data/OPSD/assets/models/InternVL3_5-8B-HF
|
| 3 |
+
library_name: peft
|
| 4 |
+
pipeline_tag: text-generation
|
| 5 |
+
tags:
|
| 6 |
+
- base_model:adapter:/data/OPSD/assets/models/InternVL3_5-8B-HF
|
| 7 |
+
- lora
|
| 8 |
+
- transformers
|
| 9 |
+
---
|
| 10 |
+
|
| 11 |
+
# Model Card for Model ID
|
| 12 |
+
|
| 13 |
+
<!-- Provide a quick summary of what the model is/does. -->
|
| 14 |
+
|
| 15 |
+
|
| 16 |
+
|
| 17 |
+
## Model Details
|
| 18 |
+
|
| 19 |
+
### Model Description
|
| 20 |
+
|
| 21 |
+
<!-- Provide a longer summary of what this model is. -->
|
| 22 |
+
|
| 23 |
+
|
| 24 |
+
|
| 25 |
+
- **Developed by:** [More Information Needed]
|
| 26 |
+
- **Funded by [optional]:** [More Information Needed]
|
| 27 |
+
- **Shared by [optional]:** [More Information Needed]
|
| 28 |
+
- **Model type:** [More Information Needed]
|
| 29 |
+
- **Language(s) (NLP):** [More Information Needed]
|
| 30 |
+
- **License:** [More Information Needed]
|
| 31 |
+
- **Finetuned from model [optional]:** [More Information Needed]
|
| 32 |
+
|
| 33 |
+
### Model Sources [optional]
|
| 34 |
+
|
| 35 |
+
<!-- Provide the basic links for the model. -->
|
| 36 |
+
|
| 37 |
+
- **Repository:** [More Information Needed]
|
| 38 |
+
- **Paper [optional]:** [More Information Needed]
|
| 39 |
+
- **Demo [optional]:** [More Information Needed]
|
| 40 |
+
|
| 41 |
+
## Uses
|
| 42 |
+
|
| 43 |
+
<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
|
| 44 |
+
|
| 45 |
+
### Direct Use
|
| 46 |
+
|
| 47 |
+
<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
|
| 48 |
+
|
| 49 |
+
[More Information Needed]
|
| 50 |
+
|
| 51 |
+
### Downstream Use [optional]
|
| 52 |
+
|
| 53 |
+
<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
|
| 54 |
+
|
| 55 |
+
[More Information Needed]
|
| 56 |
+
|
| 57 |
+
### Out-of-Scope Use
|
| 58 |
+
|
| 59 |
+
<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
|
| 60 |
+
|
| 61 |
+
[More Information Needed]
|
| 62 |
+
|
| 63 |
+
## Bias, Risks, and Limitations
|
| 64 |
+
|
| 65 |
+
<!-- This section is meant to convey both technical and sociotechnical limitations. -->
|
| 66 |
+
|
| 67 |
+
[More Information Needed]
|
| 68 |
+
|
| 69 |
+
### Recommendations
|
| 70 |
+
|
| 71 |
+
<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
|
| 72 |
+
|
| 73 |
+
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
|
| 74 |
+
|
| 75 |
+
## How to Get Started with the Model
|
| 76 |
+
|
| 77 |
+
Use the code below to get started with the model.
|
| 78 |
+
|
| 79 |
+
[More Information Needed]
|
| 80 |
+
|
| 81 |
+
## Training Details
|
| 82 |
+
|
| 83 |
+
### Training Data
|
| 84 |
+
|
| 85 |
+
<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
|
| 86 |
+
|
| 87 |
+
[More Information Needed]
|
| 88 |
+
|
| 89 |
+
### Training Procedure
|
| 90 |
+
|
| 91 |
+
<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
|
| 92 |
+
|
| 93 |
+
#### Preprocessing [optional]
|
| 94 |
+
|
| 95 |
+
[More Information Needed]
|
| 96 |
+
|
| 97 |
+
|
| 98 |
+
#### Training Hyperparameters
|
| 99 |
+
|
| 100 |
+
- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
|
| 101 |
+
|
| 102 |
+
#### Speeds, Sizes, Times [optional]
|
| 103 |
+
|
| 104 |
+
<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
|
| 105 |
+
|
| 106 |
+
[More Information Needed]
|
| 107 |
+
|
| 108 |
+
## Evaluation
|
| 109 |
+
|
| 110 |
+
<!-- This section describes the evaluation protocols and provides the results. -->
|
| 111 |
+
|
| 112 |
+
### Testing Data, Factors & Metrics
|
| 113 |
+
|
| 114 |
+
#### Testing Data
|
| 115 |
+
|
| 116 |
+
<!-- This should link to a Dataset Card if possible. -->
|
| 117 |
+
|
| 118 |
+
[More Information Needed]
|
| 119 |
+
|
| 120 |
+
#### Factors
|
| 121 |
+
|
| 122 |
+
<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
|
| 123 |
+
|
| 124 |
+
[More Information Needed]
|
| 125 |
+
|
| 126 |
+
#### Metrics
|
| 127 |
+
|
| 128 |
+
<!-- These are the evaluation metrics being used, ideally with a description of why. -->
|
| 129 |
+
|
| 130 |
+
[More Information Needed]
|
| 131 |
+
|
| 132 |
+
### Results
|
| 133 |
+
|
| 134 |
+
[More Information Needed]
|
| 135 |
+
|
| 136 |
+
#### Summary
|
| 137 |
+
|
| 138 |
+
|
| 139 |
+
|
| 140 |
+
## Model Examination [optional]
|
| 141 |
+
|
| 142 |
+
<!-- Relevant interpretability work for the model goes here -->
|
| 143 |
+
|
| 144 |
+
[More Information Needed]
|
| 145 |
+
|
| 146 |
+
## Environmental Impact
|
| 147 |
+
|
| 148 |
+
<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
|
| 149 |
+
|
| 150 |
+
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
|
| 151 |
+
|
| 152 |
+
- **Hardware Type:** [More Information Needed]
|
| 153 |
+
- **Hours used:** [More Information Needed]
|
| 154 |
+
- **Cloud Provider:** [More Information Needed]
|
| 155 |
+
- **Compute Region:** [More Information Needed]
|
| 156 |
+
- **Carbon Emitted:** [More Information Needed]
|
| 157 |
+
|
| 158 |
+
## Technical Specifications [optional]
|
| 159 |
+
|
| 160 |
+
### Model Architecture and Objective
|
| 161 |
+
|
| 162 |
+
[More Information Needed]
|
| 163 |
+
|
| 164 |
+
### Compute Infrastructure
|
| 165 |
+
|
| 166 |
+
[More Information Needed]
|
| 167 |
+
|
| 168 |
+
#### Hardware
|
| 169 |
+
|
| 170 |
+
[More Information Needed]
|
| 171 |
+
|
| 172 |
+
#### Software
|
| 173 |
+
|
| 174 |
+
[More Information Needed]
|
| 175 |
+
|
| 176 |
+
## Citation [optional]
|
| 177 |
+
|
| 178 |
+
<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
|
| 179 |
+
|
| 180 |
+
**BibTeX:**
|
| 181 |
+
|
| 182 |
+
[More Information Needed]
|
| 183 |
+
|
| 184 |
+
**APA:**
|
| 185 |
+
|
| 186 |
+
[More Information Needed]
|
| 187 |
+
|
| 188 |
+
## Glossary [optional]
|
| 189 |
+
|
| 190 |
+
<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
|
| 191 |
+
|
| 192 |
+
[More Information Needed]
|
| 193 |
+
|
| 194 |
+
## More Information [optional]
|
| 195 |
+
|
| 196 |
+
[More Information Needed]
|
| 197 |
+
|
| 198 |
+
## Model Card Authors [optional]
|
| 199 |
+
|
| 200 |
+
[More Information Needed]
|
| 201 |
+
|
| 202 |
+
## Model Card Contact
|
| 203 |
+
|
| 204 |
+
[More Information Needed]
|
| 205 |
+
### Framework versions
|
| 206 |
+
|
| 207 |
+
- PEFT 0.20.0
|
runs/fixed_internvl/removal/TopDelta8_s42/adapter/adapter_config.json
ADDED
|
@@ -0,0 +1,213 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"alora_invocation_tokens": null,
|
| 3 |
+
"alpha_pattern": {
|
| 4 |
+
"model.language_model.layers.13.mlp.down_proj": 144,
|
| 5 |
+
"model.language_model.layers.13.mlp.gate_proj": 144,
|
| 6 |
+
"model.language_model.layers.13.mlp.up_proj": 144,
|
| 7 |
+
"model.language_model.layers.13.self_attn.k_proj": 144,
|
| 8 |
+
"model.language_model.layers.13.self_attn.o_proj": 144,
|
| 9 |
+
"model.language_model.layers.13.self_attn.q_proj": 144,
|
| 10 |
+
"model.language_model.layers.13.self_attn.v_proj": 144,
|
| 11 |
+
"model.language_model.layers.15.mlp.down_proj": 144,
|
| 12 |
+
"model.language_model.layers.15.mlp.gate_proj": 144,
|
| 13 |
+
"model.language_model.layers.15.mlp.up_proj": 144,
|
| 14 |
+
"model.language_model.layers.15.self_attn.k_proj": 144,
|
| 15 |
+
"model.language_model.layers.15.self_attn.o_proj": 144,
|
| 16 |
+
"model.language_model.layers.15.self_attn.q_proj": 144,
|
| 17 |
+
"model.language_model.layers.15.self_attn.v_proj": 144,
|
| 18 |
+
"model.language_model.layers.17.mlp.down_proj": 144,
|
| 19 |
+
"model.language_model.layers.17.mlp.gate_proj": 144,
|
| 20 |
+
"model.language_model.layers.17.mlp.up_proj": 144,
|
| 21 |
+
"model.language_model.layers.17.self_attn.k_proj": 144,
|
| 22 |
+
"model.language_model.layers.17.self_attn.o_proj": 144,
|
| 23 |
+
"model.language_model.layers.17.self_attn.q_proj": 144,
|
| 24 |
+
"model.language_model.layers.17.self_attn.v_proj": 144,
|
| 25 |
+
"model.language_model.layers.18.mlp.down_proj": 144,
|
| 26 |
+
"model.language_model.layers.18.mlp.gate_proj": 144,
|
| 27 |
+
"model.language_model.layers.18.mlp.up_proj": 144,
|
| 28 |
+
"model.language_model.layers.18.self_attn.k_proj": 144,
|
| 29 |
+
"model.language_model.layers.18.self_attn.o_proj": 144,
|
| 30 |
+
"model.language_model.layers.18.self_attn.q_proj": 144,
|
| 31 |
+
"model.language_model.layers.18.self_attn.v_proj": 144,
|
| 32 |
+
"model.language_model.layers.19.mlp.down_proj": 144,
|
| 33 |
+
"model.language_model.layers.19.mlp.gate_proj": 144,
|
| 34 |
+
"model.language_model.layers.19.mlp.up_proj": 144,
|
| 35 |
+
"model.language_model.layers.19.self_attn.k_proj": 144,
|
| 36 |
+
"model.language_model.layers.19.self_attn.o_proj": 144,
|
| 37 |
+
"model.language_model.layers.19.self_attn.q_proj": 144,
|
| 38 |
+
"model.language_model.layers.19.self_attn.v_proj": 144,
|
| 39 |
+
"model.language_model.layers.21.mlp.down_proj": 144,
|
| 40 |
+
"model.language_model.layers.21.mlp.gate_proj": 144,
|
| 41 |
+
"model.language_model.layers.21.mlp.up_proj": 144,
|
| 42 |
+
"model.language_model.layers.21.self_attn.k_proj": 144,
|
| 43 |
+
"model.language_model.layers.21.self_attn.o_proj": 144,
|
| 44 |
+
"model.language_model.layers.21.self_attn.q_proj": 144,
|
| 45 |
+
"model.language_model.layers.21.self_attn.v_proj": 144,
|
| 46 |
+
"model.language_model.layers.23.mlp.down_proj": 144,
|
| 47 |
+
"model.language_model.layers.23.mlp.gate_proj": 144,
|
| 48 |
+
"model.language_model.layers.23.mlp.up_proj": 144,
|
| 49 |
+
"model.language_model.layers.23.self_attn.k_proj": 144,
|
| 50 |
+
"model.language_model.layers.23.self_attn.o_proj": 144,
|
| 51 |
+
"model.language_model.layers.23.self_attn.q_proj": 144,
|
| 52 |
+
"model.language_model.layers.23.self_attn.v_proj": 144,
|
| 53 |
+
"model.language_model.layers.24.mlp.down_proj": 144,
|
| 54 |
+
"model.language_model.layers.24.mlp.gate_proj": 144,
|
| 55 |
+
"model.language_model.layers.24.mlp.up_proj": 144,
|
| 56 |
+
"model.language_model.layers.24.self_attn.k_proj": 144,
|
| 57 |
+
"model.language_model.layers.24.self_attn.o_proj": 144,
|
| 58 |
+
"model.language_model.layers.24.self_attn.q_proj": 144,
|
| 59 |
+
"model.language_model.layers.24.self_attn.v_proj": 144
|
| 60 |
+
},
|
| 61 |
+
"arrow_config": null,
|
| 62 |
+
"auto_mapping": null,
|
| 63 |
+
"base_model_name_or_path": "/data/OPSD/assets/models/InternVL3_5-8B-HF",
|
| 64 |
+
"bias": "none",
|
| 65 |
+
"corda_config": null,
|
| 66 |
+
"ensure_weight_tying": false,
|
| 67 |
+
"eva_config": null,
|
| 68 |
+
"exclude_modules": null,
|
| 69 |
+
"fan_in_fan_out": false,
|
| 70 |
+
"inference_mode": true,
|
| 71 |
+
"init_lora_weights": true,
|
| 72 |
+
"layer_replication": null,
|
| 73 |
+
"layers_pattern": null,
|
| 74 |
+
"layers_to_transform": null,
|
| 75 |
+
"loftq_config": {},
|
| 76 |
+
"lora_alpha": 32,
|
| 77 |
+
"lora_bias": false,
|
| 78 |
+
"lora_dropout": 0.05,
|
| 79 |
+
"lora_ga_config": null,
|
| 80 |
+
"megatron_config": null,
|
| 81 |
+
"megatron_core": "megatron.core",
|
| 82 |
+
"modules_to_save": null,
|
| 83 |
+
"monteclora_config": null,
|
| 84 |
+
"peft_type": "LORA",
|
| 85 |
+
"peft_version": "0.20.0",
|
| 86 |
+
"qalora_group_size": 16,
|
| 87 |
+
"r": 16,
|
| 88 |
+
"rank_pattern": {
|
| 89 |
+
"model.language_model.layers.13.mlp.down_proj": 72,
|
| 90 |
+
"model.language_model.layers.13.mlp.gate_proj": 72,
|
| 91 |
+
"model.language_model.layers.13.mlp.up_proj": 72,
|
| 92 |
+
"model.language_model.layers.13.self_attn.k_proj": 72,
|
| 93 |
+
"model.language_model.layers.13.self_attn.o_proj": 72,
|
| 94 |
+
"model.language_model.layers.13.self_attn.q_proj": 72,
|
| 95 |
+
"model.language_model.layers.13.self_attn.v_proj": 72,
|
| 96 |
+
"model.language_model.layers.15.mlp.down_proj": 72,
|
| 97 |
+
"model.language_model.layers.15.mlp.gate_proj": 72,
|
| 98 |
+
"model.language_model.layers.15.mlp.up_proj": 72,
|
| 99 |
+
"model.language_model.layers.15.self_attn.k_proj": 72,
|
| 100 |
+
"model.language_model.layers.15.self_attn.o_proj": 72,
|
| 101 |
+
"model.language_model.layers.15.self_attn.q_proj": 72,
|
| 102 |
+
"model.language_model.layers.15.self_attn.v_proj": 72,
|
| 103 |
+
"model.language_model.layers.17.mlp.down_proj": 72,
|
| 104 |
+
"model.language_model.layers.17.mlp.gate_proj": 72,
|
| 105 |
+
"model.language_model.layers.17.mlp.up_proj": 72,
|
| 106 |
+
"model.language_model.layers.17.self_attn.k_proj": 72,
|
| 107 |
+
"model.language_model.layers.17.self_attn.o_proj": 72,
|
| 108 |
+
"model.language_model.layers.17.self_attn.q_proj": 72,
|
| 109 |
+
"model.language_model.layers.17.self_attn.v_proj": 72,
|
| 110 |
+
"model.language_model.layers.18.mlp.down_proj": 72,
|
| 111 |
+
"model.language_model.layers.18.mlp.gate_proj": 72,
|
| 112 |
+
"model.language_model.layers.18.mlp.up_proj": 72,
|
| 113 |
+
"model.language_model.layers.18.self_attn.k_proj": 72,
|
| 114 |
+
"model.language_model.layers.18.self_attn.o_proj": 72,
|
| 115 |
+
"model.language_model.layers.18.self_attn.q_proj": 72,
|
| 116 |
+
"model.language_model.layers.18.self_attn.v_proj": 72,
|
| 117 |
+
"model.language_model.layers.19.mlp.down_proj": 72,
|
| 118 |
+
"model.language_model.layers.19.mlp.gate_proj": 72,
|
| 119 |
+
"model.language_model.layers.19.mlp.up_proj": 72,
|
| 120 |
+
"model.language_model.layers.19.self_attn.k_proj": 72,
|
| 121 |
+
"model.language_model.layers.19.self_attn.o_proj": 72,
|
| 122 |
+
"model.language_model.layers.19.self_attn.q_proj": 72,
|
| 123 |
+
"model.language_model.layers.19.self_attn.v_proj": 72,
|
| 124 |
+
"model.language_model.layers.21.mlp.down_proj": 72,
|
| 125 |
+
"model.language_model.layers.21.mlp.gate_proj": 72,
|
| 126 |
+
"model.language_model.layers.21.mlp.up_proj": 72,
|
| 127 |
+
"model.language_model.layers.21.self_attn.k_proj": 72,
|
| 128 |
+
"model.language_model.layers.21.self_attn.o_proj": 72,
|
| 129 |
+
"model.language_model.layers.21.self_attn.q_proj": 72,
|
| 130 |
+
"model.language_model.layers.21.self_attn.v_proj": 72,
|
| 131 |
+
"model.language_model.layers.23.mlp.down_proj": 72,
|
| 132 |
+
"model.language_model.layers.23.mlp.gate_proj": 72,
|
| 133 |
+
"model.language_model.layers.23.mlp.up_proj": 72,
|
| 134 |
+
"model.language_model.layers.23.self_attn.k_proj": 72,
|
| 135 |
+
"model.language_model.layers.23.self_attn.o_proj": 72,
|
| 136 |
+
"model.language_model.layers.23.self_attn.q_proj": 72,
|
| 137 |
+
"model.language_model.layers.23.self_attn.v_proj": 72,
|
| 138 |
+
"model.language_model.layers.24.mlp.down_proj": 72,
|
| 139 |
+
"model.language_model.layers.24.mlp.gate_proj": 72,
|
| 140 |
+
"model.language_model.layers.24.mlp.up_proj": 72,
|
| 141 |
+
"model.language_model.layers.24.self_attn.k_proj": 72,
|
| 142 |
+
"model.language_model.layers.24.self_attn.o_proj": 72,
|
| 143 |
+
"model.language_model.layers.24.self_attn.q_proj": 72,
|
| 144 |
+
"model.language_model.layers.24.self_attn.v_proj": 72
|
| 145 |
+
},
|
| 146 |
+
"revision": null,
|
| 147 |
+
"target_modules": [
|
| 148 |
+
"model.language_model.layers.17.mlp.up_proj",
|
| 149 |
+
"model.language_model.layers.24.mlp.down_proj",
|
| 150 |
+
"model.language_model.layers.19.mlp.down_proj",
|
| 151 |
+
"model.language_model.layers.19.mlp.up_proj",
|
| 152 |
+
"model.language_model.layers.23.self_attn.o_proj",
|
| 153 |
+
"model.language_model.layers.17.mlp.gate_proj",
|
| 154 |
+
"model.language_model.layers.18.self_attn.k_proj",
|
| 155 |
+
"model.language_model.layers.24.mlp.gate_proj",
|
| 156 |
+
"model.language_model.layers.13.self_attn.q_proj",
|
| 157 |
+
"model.language_model.layers.19.self_attn.q_proj",
|
| 158 |
+
"model.language_model.layers.19.self_attn.k_proj",
|
| 159 |
+
"model.language_model.layers.21.self_attn.v_proj",
|
| 160 |
+
"model.language_model.layers.24.self_attn.k_proj",
|
| 161 |
+
"model.language_model.layers.23.mlp.down_proj",
|
| 162 |
+
"model.language_model.layers.15.self_attn.k_proj",
|
| 163 |
+
"model.language_model.layers.21.mlp.down_proj",
|
| 164 |
+
"model.language_model.layers.13.mlp.down_proj",
|
| 165 |
+
"model.language_model.layers.15.mlp.gate_proj",
|
| 166 |
+
"model.language_model.layers.21.self_attn.o_proj",
|
| 167 |
+
"model.language_model.layers.23.self_attn.k_proj",
|
| 168 |
+
"model.language_model.layers.24.self_attn.q_proj",
|
| 169 |
+
"model.language_model.layers.21.self_attn.q_proj",
|
| 170 |
+
"model.language_model.layers.17.mlp.down_proj",
|
| 171 |
+
"model.language_model.layers.23.self_attn.v_proj",
|
| 172 |
+
"model.language_model.layers.15.mlp.up_proj",
|
| 173 |
+
"model.language_model.layers.13.self_attn.k_proj",
|
| 174 |
+
"model.language_model.layers.13.mlp.gate_proj",
|
| 175 |
+
"model.language_model.layers.17.self_attn.k_proj",
|
| 176 |
+
"model.language_model.layers.13.mlp.up_proj",
|
| 177 |
+
"model.language_model.layers.15.self_attn.q_proj",
|
| 178 |
+
"model.language_model.layers.17.self_attn.q_proj",
|
| 179 |
+
"model.language_model.layers.15.mlp.down_proj",
|
| 180 |
+
"model.language_model.layers.18.self_attn.v_proj",
|
| 181 |
+
"model.language_model.layers.21.mlp.up_proj",
|
| 182 |
+
"model.language_model.layers.21.mlp.gate_proj",
|
| 183 |
+
"model.language_model.layers.23.mlp.gate_proj",
|
| 184 |
+
"model.language_model.layers.21.self_attn.k_proj",
|
| 185 |
+
"model.language_model.layers.24.self_attn.o_proj",
|
| 186 |
+
"model.language_model.layers.15.self_attn.o_proj",
|
| 187 |
+
"model.language_model.layers.18.self_attn.q_proj",
|
| 188 |
+
"model.language_model.layers.18.self_attn.o_proj",
|
| 189 |
+
"model.language_model.layers.17.self_attn.v_proj",
|
| 190 |
+
"model.language_model.layers.17.self_attn.o_proj",
|
| 191 |
+
"model.language_model.layers.19.self_attn.v_proj",
|
| 192 |
+
"model.language_model.layers.13.self_attn.v_proj",
|
| 193 |
+
"model.language_model.layers.23.mlp.up_proj",
|
| 194 |
+
"model.language_model.layers.24.self_attn.v_proj",
|
| 195 |
+
"model.language_model.layers.18.mlp.up_proj",
|
| 196 |
+
"model.language_model.layers.15.self_attn.v_proj",
|
| 197 |
+
"model.language_model.layers.19.mlp.gate_proj",
|
| 198 |
+
"model.language_model.layers.18.mlp.gate_proj",
|
| 199 |
+
"model.language_model.layers.13.self_attn.o_proj",
|
| 200 |
+
"model.language_model.layers.23.self_attn.q_proj",
|
| 201 |
+
"model.language_model.layers.24.mlp.up_proj",
|
| 202 |
+
"model.language_model.layers.19.self_attn.o_proj",
|
| 203 |
+
"model.language_model.layers.18.mlp.down_proj"
|
| 204 |
+
],
|
| 205 |
+
"target_parameters": null,
|
| 206 |
+
"task_type": "CAUSAL_LM",
|
| 207 |
+
"trainable_token_indices": null,
|
| 208 |
+
"use_bdlora": null,
|
| 209 |
+
"use_dora": false,
|
| 210 |
+
"use_qalora": false,
|
| 211 |
+
"use_rslora": false,
|
| 212 |
+
"velora_config": null
|
| 213 |
+
}
|
runs/fixed_internvl/removal/TopDelta8_s42/config.json
ADDED
|
@@ -0,0 +1,43 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"behavior": "removal",
|
| 3 |
+
"seed": 42,
|
| 4 |
+
"model": "/data/OPSD/assets/models/InternVL3_5-8B-HF",
|
| 5 |
+
"backbone": "internvl",
|
| 6 |
+
"train": "/data/OPSD/assets/v3/removal.jsonl",
|
| 7 |
+
"items": 1000,
|
| 8 |
+
"steps": 250,
|
| 9 |
+
"epochs": 1,
|
| 10 |
+
"steps_per_epoch": 250,
|
| 11 |
+
"world_size": 1,
|
| 12 |
+
"lr": 5e-05,
|
| 13 |
+
"scheduler": "cosine",
|
| 14 |
+
"warmup_steps": 16,
|
| 15 |
+
"final_fraction": 0.1,
|
| 16 |
+
"clip": 1.0,
|
| 17 |
+
"initial_loss_scale": 4.0,
|
| 18 |
+
"arm": {
|
| 19 |
+
"arm": "TopDelta8",
|
| 20 |
+
"layers": [
|
| 21 |
+
13,
|
| 22 |
+
15,
|
| 23 |
+
17,
|
| 24 |
+
18,
|
| 25 |
+
19,
|
| 26 |
+
21,
|
| 27 |
+
23,
|
| 28 |
+
24
|
| 29 |
+
],
|
| 30 |
+
"ranks": {
|
| 31 |
+
"13": 72,
|
| 32 |
+
"15": 72,
|
| 33 |
+
"17": 72,
|
| 34 |
+
"18": 72,
|
| 35 |
+
"19": 72,
|
| 36 |
+
"21": 72,
|
| 37 |
+
"23": 72,
|
| 38 |
+
"24": 72
|
| 39 |
+
},
|
| 40 |
+
"parameters": 43646976,
|
| 41 |
+
"budget": 43646976
|
| 42 |
+
}
|
| 43 |
+
}
|
runs/fixed_internvl/removal/TopDelta8_s42/steps.jsonl
ADDED
|
@@ -0,0 +1,250 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{"step": 1, "steps": 250, "lr": 3.125e-06, "grad_norm": 12.42832088470459, "loss_scale": 4.0, "retries": 0, "seconds": 5.0, "max_memory_gb": 16.79, "ce": 1.0494785578921437, "rank": 0.15784265100955963}
|
| 2 |
+
{"step": 2, "steps": 250, "lr": 6.25e-06, "grad_norm": 7.893779277801514, "loss_scale": 4.0, "retries": 0, "seconds": 8.9, "max_memory_gb": 17.11, "ce": 0.815269090356594, "rank": 0.0}
|
| 3 |
+
{"step": 3, "steps": 250, "lr": 9.375000000000001e-06, "grad_norm": 17.14238739013672, "loss_scale": 4.0, "retries": 0, "seconds": 12.3, "max_memory_gb": 17.12, "ce": 0.7558296176721342, "rank": 0.0}
|
| 4 |
+
{"step": 4, "steps": 250, "lr": 1.25e-05, "grad_norm": 9.357633590698242, "loss_scale": 4.0, "retries": 0, "seconds": 15.7, "max_memory_gb": 17.12, "ce": 0.6447016536330921, "rank": 0.0}
|
| 5 |
+
{"step": 5, "steps": 250, "lr": 1.5625e-05, "grad_norm": 2.3532867431640625, "loss_scale": 4.0, "retries": 0, "seconds": 19.1, "max_memory_gb": 17.12, "ce": 0.5305187370686326, "rank": 0.0}
|
| 6 |
+
{"step": 6, "steps": 250, "lr": 1.8750000000000002e-05, "grad_norm": 2.769531488418579, "loss_scale": 4.0, "retries": 0, "seconds": 22.5, "max_memory_gb": 17.12, "ce": 0.30357705877395347, "rank": 0.0}
|
| 7 |
+
{"step": 7, "steps": 250, "lr": 2.1875e-05, "grad_norm": 2.707801103591919, "loss_scale": 4.0, "retries": 0, "seconds": 25.9, "max_memory_gb": 17.12, "ce": 0.49893750064074993, "rank": 0.0}
|
| 8 |
+
{"step": 8, "steps": 250, "lr": 2.5e-05, "grad_norm": 1.9520862102508545, "loss_scale": 4.0, "retries": 0, "seconds": 29.2, "max_memory_gb": 17.12, "ce": 0.24317460617191955, "rank": 0.0}
|
| 9 |
+
{"step": 9, "steps": 250, "lr": 2.8125000000000003e-05, "grad_norm": 3.435497999191284, "loss_scale": 4.0, "retries": 0, "seconds": 32.7, "max_memory_gb": 17.12, "ce": 0.5815261328825727, "rank": 0.0}
|
| 10 |
+
{"step": 10, "steps": 250, "lr": 3.125e-05, "grad_norm": 1.7784301042556763, "loss_scale": 4.0, "retries": 0, "seconds": 36.1, "max_memory_gb": 17.12, "ce": 0.23574840047513135, "rank": 0.20394697785377502}
|
| 11 |
+
{"step": 11, "steps": 250, "lr": 3.4375e-05, "grad_norm": 0.3676748275756836, "loss_scale": 4.0, "retries": 0, "seconds": 39.4, "max_memory_gb": 17.12, "ce": 0.14021930189073828, "rank": 0.0}
|
| 12 |
+
{"step": 12, "steps": 250, "lr": 3.7500000000000003e-05, "grad_norm": 5.321396350860596, "loss_scale": 4.0, "retries": 0, "seconds": 42.9, "max_memory_gb": 17.12, "ce": 0.5705950050905813, "rank": 0.0}
|
| 13 |
+
{"step": 13, "steps": 250, "lr": 4.0625000000000005e-05, "grad_norm": 1.09409761428833, "loss_scale": 4.0, "retries": 0, "seconds": 46.3, "max_memory_gb": 17.12, "ce": 0.36259925039485097, "rank": 0.0}
|
| 14 |
+
{"step": 14, "steps": 250, "lr": 4.375e-05, "grad_norm": 0.5644330978393555, "loss_scale": 4.0, "retries": 0, "seconds": 49.7, "max_memory_gb": 17.12, "ce": 0.31000175973167643, "rank": 0.0}
|
| 15 |
+
{"step": 15, "steps": 250, "lr": 4.6875e-05, "grad_norm": 6.082078456878662, "loss_scale": 4.0, "retries": 0, "seconds": 53.2, "max_memory_gb": 17.12, "ce": 0.4719627918675542, "rank": 0.1901818811893463}
|
| 16 |
+
{"step": 16, "steps": 250, "lr": 5e-05, "grad_norm": 1.2117881774902344, "loss_scale": 4.0, "retries": 0, "seconds": 56.6, "max_memory_gb": 17.12, "ce": 0.3131949807448109, "rank": 0.0}
|
| 17 |
+
{"step": 17, "steps": 250, "lr": 5e-05, "grad_norm": 0.8701798319816589, "loss_scale": 4.0, "retries": 0, "seconds": 60.0, "max_memory_gb": 17.13, "ce": 0.20702843059552833, "rank": 0.10046374052762985}
|
| 18 |
+
{"step": 18, "steps": 250, "lr": 4.999797225177623e-05, "grad_norm": 2.2141635417938232, "loss_scale": 4.0, "retries": 0, "seconds": 63.5, "max_memory_gb": 17.13, "ce": 0.24802493350580335, "rank": 0.0}
|
| 19 |
+
{"step": 19, "steps": 250, "lr": 4.999188937259492e-05, "grad_norm": 0.7007525563240051, "loss_scale": 4.0, "retries": 0, "seconds": 66.8, "max_memory_gb": 17.13, "ce": 0.08095678312747623, "rank": 0.0}
|
| 20 |
+
{"step": 20, "steps": 250, "lr": 4.998175245886032e-05, "grad_norm": 1.1538885831832886, "loss_scale": 4.0, "retries": 0, "seconds": 70.3, "max_memory_gb": 17.13, "ce": 0.0978921094792895, "rank": 0.0}
|
| 21 |
+
{"step": 21, "steps": 250, "lr": 4.996756333769319e-05, "grad_norm": 0.17451444268226624, "loss_scale": 4.0, "retries": 0, "seconds": 73.6, "max_memory_gb": 17.13, "ce": 0.015422077296534553, "rank": 0.0}
|
| 22 |
+
{"step": 22, "steps": 250, "lr": 4.994932456660157e-05, "grad_norm": 2.304096221923828, "loss_scale": 4.0, "retries": 0, "seconds": 77.1, "max_memory_gb": 17.13, "ce": 0.15000250631487688, "rank": 0.0}
|
| 23 |
+
{"step": 23, "steps": 250, "lr": 4.992703943301973e-05, "grad_norm": 1.3055553436279297, "loss_scale": 4.0, "retries": 0, "seconds": 80.5, "max_memory_gb": 17.13, "ce": 0.09805250814133615, "rank": 0.0}
|
| 24 |
+
{"step": 24, "steps": 250, "lr": 4.9900711953715675e-05, "grad_norm": 2.053431510925293, "loss_scale": 4.0, "retries": 0, "seconds": 83.9, "max_memory_gb": 17.13, "ce": 0.20166278073156718, "rank": 0.0}
|
| 25 |
+
{"step": 25, "steps": 250, "lr": 4.987034687406713e-05, "grad_norm": 0.6271782517433167, "loss_scale": 4.0, "retries": 0, "seconds": 87.4, "max_memory_gb": 17.13, "ce": 0.02984295622627542, "rank": 0.0}
|
| 26 |
+
{"step": 26, "steps": 250, "lr": 4.9835949667206215e-05, "grad_norm": 2.19781231880188, "loss_scale": 4.0, "retries": 0, "seconds": 90.8, "max_memory_gb": 17.13, "ce": 0.04678909329402359, "rank": 0.0}
|
| 27 |
+
{"step": 27, "steps": 250, "lr": 4.9797526533032945e-05, "grad_norm": 17.472396850585938, "loss_scale": 4.0, "retries": 0, "seconds": 94.1, "max_memory_gb": 17.13, "ce": 0.6455953952854543, "rank": 0.0}
|
| 28 |
+
{"step": 28, "steps": 250, "lr": 4.9755084397097716e-05, "grad_norm": 2.015190839767456, "loss_scale": 4.0, "retries": 0, "seconds": 97.6, "max_memory_gb": 17.13, "ce": 0.17449136844516033, "rank": 0.0}
|
| 29 |
+
{"step": 29, "steps": 250, "lr": 4.970863090935304e-05, "grad_norm": 4.435781002044678, "loss_scale": 4.0, "retries": 0, "seconds": 101.0, "max_memory_gb": 17.13, "ce": 0.12248160442322842, "rank": 0.0}
|
| 30 |
+
{"step": 30, "steps": 250, "lr": 4.965817444277468e-05, "grad_norm": 1.9441335201263428, "loss_scale": 4.0, "retries": 0, "seconds": 104.4, "max_memory_gb": 17.13, "ce": 0.08801555921127147, "rank": 0.0}
|
| 31 |
+
{"step": 31, "steps": 250, "lr": 4.9603724091852456e-05, "grad_norm": 8.135625839233398, "loss_scale": 4.0, "retries": 0, "seconds": 107.8, "max_memory_gb": 17.13, "ce": 0.6851668731633254, "rank": 0.0}
|
| 32 |
+
{"step": 32, "steps": 250, "lr": 4.9545289670951025e-05, "grad_norm": 0.9354689717292786, "loss_scale": 4.0, "retries": 0, "seconds": 111.3, "max_memory_gb": 17.13, "ce": 0.026194167519861367, "rank": 0.0}
|
| 33 |
+
{"step": 33, "steps": 250, "lr": 4.948288171254089e-05, "grad_norm": 0.22322027385234833, "loss_scale": 4.0, "retries": 0, "seconds": 114.7, "max_memory_gb": 17.13, "ce": 0.004097490904769074, "rank": 0.0}
|
| 34 |
+
{"step": 34, "steps": 250, "lr": 4.941651146529998e-05, "grad_norm": 0.11790601909160614, "loss_scale": 4.0, "retries": 0, "seconds": 118.1, "max_memory_gb": 17.13, "ce": 0.0025820244468377496, "rank": 0.0}
|
| 35 |
+
{"step": 35, "steps": 250, "lr": 4.9346190892086174e-05, "grad_norm": 2.1285462379455566, "loss_scale": 4.0, "retries": 0, "seconds": 121.6, "max_memory_gb": 17.13, "ce": 0.08700434590809891, "rank": 0.0}
|
| 36 |
+
{"step": 36, "steps": 250, "lr": 4.927193266778101e-05, "grad_norm": 3.2615315914154053, "loss_scale": 4.0, "retries": 0, "seconds": 125.0, "max_memory_gb": 17.13, "ce": 0.2717903643574573, "rank": 0.01379886269569397}
|
| 37 |
+
{"step": 37, "steps": 250, "lr": 4.919375017700515e-05, "grad_norm": 2.0373804569244385, "loss_scale": 4.0, "retries": 0, "seconds": 128.4, "max_memory_gb": 17.13, "ce": 0.17754528383738943, "rank": 0.0}
|
| 38 |
+
{"step": 38, "steps": 250, "lr": 4.911165751170588e-05, "grad_norm": 0.13012389838695526, "loss_scale": 4.0, "retries": 0, "seconds": 131.8, "max_memory_gb": 17.13, "ce": 0.003628663940617116, "rank": 0.0}
|
| 39 |
+
{"step": 39, "steps": 250, "lr": 4.902566946861708e-05, "grad_norm": 4.969915390014648, "loss_scale": 4.0, "retries": 0, "seconds": 135.3, "max_memory_gb": 17.13, "ce": 0.2887288817928493, "rank": 0.0}
|
| 40 |
+
{"step": 40, "steps": 250, "lr": 4.89358015465922e-05, "grad_norm": 0.35662251710891724, "loss_scale": 4.0, "retries": 0, "seconds": 138.7, "max_memory_gb": 17.13, "ce": 0.00667690249701991, "rank": 0.0}
|
| 41 |
+
{"step": 41, "steps": 250, "lr": 4.884206994381078e-05, "grad_norm": 0.5957790613174438, "loss_scale": 4.0, "retries": 0, "seconds": 142.1, "max_memory_gb": 17.13, "ce": 0.013222039946640507, "rank": 0.0}
|
| 42 |
+
{"step": 42, "steps": 250, "lr": 4.874449155485866e-05, "grad_norm": 2.086395740509033, "loss_scale": 4.0, "retries": 0, "seconds": 145.6, "max_memory_gb": 17.13, "ce": 0.14174616977379628, "rank": 0.0}
|
| 43 |
+
{"step": 43, "steps": 250, "lr": 4.864308396768294e-05, "grad_norm": 2.4269070625305176, "loss_scale": 4.0, "retries": 0, "seconds": 149.0, "max_memory_gb": 17.13, "ce": 0.05981520008845109, "rank": 0.0}
|
| 44 |
+
{"step": 44, "steps": 250, "lr": 4.853786546042184e-05, "grad_norm": 4.892577648162842, "loss_scale": 4.0, "retries": 0, "seconds": 152.3, "max_memory_gb": 17.13, "ce": 0.3011240893347349, "rank": 0.0}
|
| 45 |
+
{"step": 45, "steps": 250, "lr": 4.842885499811012e-05, "grad_norm": 5.334975719451904, "loss_scale": 4.0, "retries": 0, "seconds": 155.7, "max_memory_gb": 17.13, "ce": 0.41594380589776847, "rank": 0.0}
|
| 46 |
+
{"step": 46, "steps": 250, "lr": 4.831607222926078e-05, "grad_norm": 2.43658447265625, "loss_scale": 4.0, "retries": 0, "seconds": 159.2, "max_memory_gb": 17.13, "ce": 0.24088987579307286, "rank": 0.0}
|
| 47 |
+
{"step": 47, "steps": 250, "lr": 4.8199537482323545e-05, "grad_norm": 0.07971987128257751, "loss_scale": 4.0, "retries": 0, "seconds": 162.6, "max_memory_gb": 17.13, "ce": 0.0017306971240031999, "rank": 0.0}
|
| 48 |
+
{"step": 48, "steps": 250, "lr": 4.807927176202073e-05, "grad_norm": 1.0021895170211792, "loss_scale": 4.0, "retries": 0, "seconds": 166.0, "max_memory_gb": 17.13, "ce": 0.0858339745682315, "rank": 0.0}
|
| 49 |
+
{"step": 49, "steps": 250, "lr": 4.795529674556129e-05, "grad_norm": 6.678441524505615, "loss_scale": 4.0, "retries": 0, "seconds": 169.8, "max_memory_gb": 17.13, "ce": 0.10849449616580387, "rank": 0.1249023973941803}
|
| 50 |
+
{"step": 50, "steps": 250, "lr": 4.782763477873361e-05, "grad_norm": 0.45790863037109375, "loss_scale": 4.0, "retries": 0, "seconds": 173.5, "max_memory_gb": 17.13, "ce": 0.017678195798907836, "rank": 0.0}
|
| 51 |
+
{"step": 51, "steps": 250, "lr": 4.769630887187782e-05, "grad_norm": 1.0719575881958008, "loss_scale": 4.0, "retries": 0, "seconds": 177.5, "max_memory_gb": 17.13, "ce": 0.04240243987442227, "rank": 0.0}
|
| 52 |
+
{"step": 52, "steps": 250, "lr": 4.756134269573832e-05, "grad_norm": 0.3750506341457367, "loss_scale": 4.0, "retries": 0, "seconds": 181.6, "max_memory_gb": 17.13, "ce": 0.005242392949298846, "rank": 0.1251302808523178}
|
| 53 |
+
{"step": 53, "steps": 250, "lr": 4.7422760577197226e-05, "grad_norm": 0.5587911605834961, "loss_scale": 4.0, "retries": 0, "seconds": 185.4, "max_memory_gb": 17.13, "ce": 0.021889666981451228, "rank": 0.0}
|
| 54 |
+
{"step": 54, "steps": 250, "lr": 4.728058749488963e-05, "grad_norm": 2.753998279571533, "loss_scale": 4.0, "retries": 0, "seconds": 189.2, "max_memory_gb": 17.13, "ce": 0.1381977120213378, "rank": 0.0}
|
| 55 |
+
{"step": 55, "steps": 250, "lr": 4.713484907470133e-05, "grad_norm": 11.444178581237793, "loss_scale": 4.0, "retries": 0, "seconds": 193.1, "max_memory_gb": 17.13, "ce": 0.5413148886181318, "rank": 0.0}
|
| 56 |
+
{"step": 56, "steps": 250, "lr": 4.6985571585149876e-05, "grad_norm": 0.12162213027477264, "loss_scale": 4.0, "retries": 0, "seconds": 196.8, "max_memory_gb": 17.13, "ce": 0.004838552466480905, "rank": 0.0}
|
| 57 |
+
{"step": 57, "steps": 250, "lr": 4.6832781932649884e-05, "grad_norm": 0.49590954184532166, "loss_scale": 4.0, "retries": 0, "seconds": 200.5, "max_memory_gb": 17.13, "ce": 0.02431708265328325, "rank": 0.0}
|
| 58 |
+
{"step": 58, "steps": 250, "lr": 4.667650765666325e-05, "grad_norm": 1.8659871816635132, "loss_scale": 4.0, "retries": 0, "seconds": 204.3, "max_memory_gb": 17.13, "ce": 0.12914627263762668, "rank": 0.0}
|
| 59 |
+
{"step": 59, "steps": 250, "lr": 4.651677692473538e-05, "grad_norm": 0.3618692457675934, "loss_scale": 4.0, "retries": 0, "seconds": 208.1, "max_memory_gb": 17.13, "ce": 0.01108870050512678, "rank": 0.0}
|
| 60 |
+
{"step": 60, "steps": 250, "lr": 4.635361852741808e-05, "grad_norm": 1.5759788751602173, "loss_scale": 4.0, "retries": 0, "seconds": 211.8, "max_memory_gb": 17.13, "ce": 0.16503017647093543, "rank": 0.0}
|
| 61 |
+
{"step": 61, "steps": 250, "lr": 4.6187061873080264e-05, "grad_norm": 0.4214089512825012, "loss_scale": 4.0, "retries": 0, "seconds": 215.5, "max_memory_gb": 17.13, "ce": 0.02369837955490084, "rank": 0.0}
|
| 62 |
+
{"step": 62, "steps": 250, "lr": 4.601713698260727e-05, "grad_norm": 0.6317011117935181, "loss_scale": 4.0, "retries": 0, "seconds": 219.0, "max_memory_gb": 17.13, "ce": 0.0459246090454144, "rank": 0.0}
|
| 63 |
+
{"step": 63, "steps": 250, "lr": 4.5843874483989746e-05, "grad_norm": 4.472061634063721, "loss_scale": 4.0, "retries": 0, "seconds": 222.4, "max_memory_gb": 17.13, "ce": 0.3158838201084109, "rank": 0.0}
|
| 64 |
+
{"step": 64, "steps": 250, "lr": 4.566730560680316e-05, "grad_norm": 0.7950683236122131, "loss_scale": 4.0, "retries": 0, "seconds": 225.7, "max_memory_gb": 17.13, "ce": 0.03859797454242653, "rank": 0.0}
|
| 65 |
+
{"step": 65, "steps": 250, "lr": 4.548746217657878e-05, "grad_norm": 6.0343732833862305, "loss_scale": 4.0, "retries": 0, "seconds": 229.1, "max_memory_gb": 17.13, "ce": 0.1589387839912888, "rank": 0.0}
|
| 66 |
+
{"step": 66, "steps": 250, "lr": 4.5304376609067396e-05, "grad_norm": 6.884658336639404, "loss_scale": 4.0, "retries": 0, "seconds": 232.6, "max_memory_gb": 17.13, "ce": 0.09059440453287948, "rank": 0.0}
|
| 67 |
+
{"step": 67, "steps": 250, "lr": 4.5118081904396504e-05, "grad_norm": 0.18305833637714386, "loss_scale": 4.0, "retries": 0, "seconds": 235.9, "max_memory_gb": 17.13, "ce": 0.006446292875494919, "rank": 0.0}
|
| 68 |
+
{"step": 68, "steps": 250, "lr": 4.4928611641122225e-05, "grad_norm": 0.17405565083026886, "loss_scale": 4.0, "retries": 0, "seconds": 239.3, "max_memory_gb": 17.13, "ce": 0.007660197455152229, "rank": 0.0}
|
| 69 |
+
{"step": 69, "steps": 250, "lr": 4.473599997017701e-05, "grad_norm": 0.973021388053894, "loss_scale": 4.0, "retries": 0, "seconds": 242.8, "max_memory_gb": 17.13, "ce": 0.044183232589602994, "rank": 0.0}
|
| 70 |
+
{"step": 70, "steps": 250, "lr": 4.4540281608714053e-05, "grad_norm": 2.676849603652954, "loss_scale": 4.0, "retries": 0, "seconds": 246.2, "max_memory_gb": 17.13, "ce": 0.28010512931223275, "rank": 0.0}
|
| 71 |
+
{"step": 71, "steps": 250, "lr": 4.434149183384977e-05, "grad_norm": 6.301036834716797, "loss_scale": 4.0, "retries": 0, "seconds": 249.5, "max_memory_gb": 17.13, "ce": 0.604440534094465, "rank": 0.12352905422449112}
|
| 72 |
+
{"step": 72, "steps": 250, "lr": 4.413966647630534e-05, "grad_norm": 0.012148779816925526, "loss_scale": 4.0, "retries": 0, "seconds": 253.0, "max_memory_gb": 17.13, "ce": 0.0005916050956926711, "rank": 0.0}
|
| 73 |
+
{"step": 73, "steps": 250, "lr": 4.3934841913948323e-05, "grad_norm": 5.2380595207214355, "loss_scale": 4.0, "retries": 0, "seconds": 256.4, "max_memory_gb": 17.13, "ce": 0.5423867252570744, "rank": 0.0}
|
| 74 |
+
{"step": 74, "steps": 250, "lr": 4.372705506523583e-05, "grad_norm": 0.9129985570907593, "loss_scale": 4.0, "retries": 0, "seconds": 259.8, "max_memory_gb": 17.13, "ce": 0.03459433692478342, "rank": 0.0}
|
| 75 |
+
{"step": 75, "steps": 250, "lr": 4.351634338256017e-05, "grad_norm": 0.6639518141746521, "loss_scale": 4.0, "retries": 0, "seconds": 263.2, "max_memory_gb": 17.13, "ce": 0.027859406315656088, "rank": 0.0}
|
| 76 |
+
{"step": 76, "steps": 250, "lr": 4.330274484549825e-05, "grad_norm": 3.2979767322540283, "loss_scale": 4.0, "retries": 0, "seconds": 266.7, "max_memory_gb": 17.13, "ce": 0.20498515755298286, "rank": 0.0}
|
| 77 |
+
{"step": 77, "steps": 250, "lr": 4.308629795396599e-05, "grad_norm": 0.8690163493156433, "loss_scale": 4.0, "retries": 0, "seconds": 270.1, "max_memory_gb": 17.13, "ce": 0.05243843599214415, "rank": 0.0}
|
| 78 |
+
{"step": 78, "steps": 250, "lr": 4.286704172127894e-05, "grad_norm": 0.28045177459716797, "loss_scale": 4.0, "retries": 0, "seconds": 273.5, "max_memory_gb": 17.13, "ce": 0.0211413851211546, "rank": 0.0}
|
| 79 |
+
{"step": 79, "steps": 250, "lr": 4.264501566712033e-05, "grad_norm": 0.8254481554031372, "loss_scale": 4.0, "retries": 0, "seconds": 276.9, "max_memory_gb": 17.13, "ce": 0.08529584692706749, "rank": 0.0}
|
| 80 |
+
{"step": 80, "steps": 250, "lr": 4.2420259810417894e-05, "grad_norm": 1.2849535942077637, "loss_scale": 4.0, "retries": 0, "seconds": 280.3, "max_memory_gb": 17.13, "ce": 0.17823275223264545, "rank": 0.0}
|
| 81 |
+
{"step": 81, "steps": 250, "lr": 4.219281466213066e-05, "grad_norm": 4.385602951049805, "loss_scale": 4.0, "retries": 0, "seconds": 283.6, "max_memory_gb": 17.13, "ce": 0.6228682438307942, "rank": 0.0}
|
| 82 |
+
{"step": 82, "steps": 250, "lr": 4.1962721217947135e-05, "grad_norm": 0.5329232811927795, "loss_scale": 4.0, "retries": 0, "seconds": 287.0, "max_memory_gb": 17.13, "ce": 0.0298928739848634, "rank": 0.0}
|
| 83 |
+
{"step": 83, "steps": 250, "lr": 4.173002095089599e-05, "grad_norm": 1.026944875717163, "loss_scale": 4.0, "retries": 0, "seconds": 290.5, "max_memory_gb": 17.13, "ce": 0.05518224841443953, "rank": 0.0}
|
| 84 |
+
{"step": 84, "steps": 250, "lr": 4.149475580387084e-05, "grad_norm": 1.600122332572937, "loss_scale": 4.0, "retries": 0, "seconds": 293.8, "max_memory_gb": 17.13, "ce": 0.14757376720831417, "rank": 0.0}
|
| 85 |
+
{"step": 85, "steps": 250, "lr": 4.1256968182070275e-05, "grad_norm": 0.25757449865341187, "loss_scale": 4.0, "retries": 0, "seconds": 297.2, "max_memory_gb": 17.13, "ce": 0.029039281199743527, "rank": 0.0}
|
| 86 |
+
{"step": 86, "steps": 250, "lr": 4.101670094535453e-05, "grad_norm": 2.419757843017578, "loss_scale": 4.0, "retries": 0, "seconds": 300.6, "max_memory_gb": 17.13, "ce": 0.08403840536811913, "rank": 0.12490066140890121}
|
| 87 |
+
{"step": 87, "steps": 250, "lr": 4.077399740052027e-05, "grad_norm": 0.8359214067459106, "loss_scale": 4.0, "retries": 0, "seconds": 304.0, "max_memory_gb": 17.13, "ce": 0.04388667556048631, "rank": 0.0}
|
| 88 |
+
{"step": 88, "steps": 250, "lr": 4.0528901293494775e-05, "grad_norm": 2.6556665897369385, "loss_scale": 4.0, "retries": 0, "seconds": 307.3, "max_memory_gb": 17.13, "ce": 0.10496237903521433, "rank": 0.0}
|
| 89 |
+
{"step": 89, "steps": 250, "lr": 4.0281456801451004e-05, "grad_norm": 0.06854204833507538, "loss_scale": 4.0, "retries": 0, "seconds": 310.9, "max_memory_gb": 17.13, "ce": 0.0034660847468330758, "rank": 0.0}
|
| 90 |
+
{"step": 90, "steps": 250, "lr": 4.0031708524844884e-05, "grad_norm": 0.005850014742463827, "loss_scale": 4.0, "retries": 0, "seconds": 314.3, "max_memory_gb": 17.13, "ce": 0.0003715939961921322, "rank": 0.0}
|
| 91 |
+
{"step": 91, "steps": 250, "lr": 3.977970147937635e-05, "grad_norm": 0.764488160610199, "loss_scale": 4.0, "retries": 0, "seconds": 317.6, "max_memory_gb": 17.13, "ce": 0.02436660695093451, "rank": 0.0}
|
| 92 |
+
{"step": 92, "steps": 250, "lr": 3.952548108787553e-05, "grad_norm": 0.5490195155143738, "loss_scale": 4.0, "retries": 0, "seconds": 321.1, "max_memory_gb": 17.13, "ce": 0.03169873207480123, "rank": 0.0}
|
| 93 |
+
{"step": 93, "steps": 250, "lr": 3.9269093172115496e-05, "grad_norm": 2.4138309955596924, "loss_scale": 4.0, "retries": 0, "seconds": 324.5, "max_memory_gb": 17.13, "ce": 0.15014778150180064, "rank": 0.0}
|
| 94 |
+
{"step": 94, "steps": 250, "lr": 3.901058394455323e-05, "grad_norm": 1.2870945930480957, "loss_scale": 4.0, "retries": 0, "seconds": 327.8, "max_memory_gb": 17.13, "ce": 0.22192623737282702, "rank": 0.0}
|
| 95 |
+
{"step": 95, "steps": 250, "lr": 3.875e-05, "grad_norm": 0.9632744789123535, "loss_scale": 4.0, "retries": 0, "seconds": 331.2, "max_memory_gb": 17.13, "ce": 0.03512687124475633, "rank": 0.0}
|
| 96 |
+
{"step": 96, "steps": 250, "lr": 3.8487388307223005e-05, "grad_norm": 0.6990203261375427, "loss_scale": 4.0, "retries": 0, "seconds": 334.6, "max_memory_gb": 17.13, "ce": 0.08826095812855783, "rank": 0.0}
|
| 97 |
+
{"step": 97, "steps": 250, "lr": 3.822279620047943e-05, "grad_norm": 0.4523006081581116, "loss_scale": 4.0, "retries": 0, "seconds": 338.0, "max_memory_gb": 17.13, "ce": 0.021540496556781363, "rank": 0.0}
|
| 98 |
+
{"step": 98, "steps": 250, "lr": 3.795627137098479e-05, "grad_norm": 0.350757896900177, "loss_scale": 4.0, "retries": 0, "seconds": 341.3, "max_memory_gb": 17.13, "ce": 0.015017170450846606, "rank": 0.0}
|
| 99 |
+
{"step": 99, "steps": 250, "lr": 3.7687861858316844e-05, "grad_norm": 4.1406707763671875, "loss_scale": 4.0, "retries": 0, "seconds": 344.8, "max_memory_gb": 17.13, "ce": 0.17264881277287714, "rank": 0.0}
|
| 100 |
+
{"step": 100, "steps": 250, "lr": 3.741761604175668e-05, "grad_norm": 0.04988433048129082, "loss_scale": 4.0, "retries": 0, "seconds": 348.2, "max_memory_gb": 17.13, "ce": 0.002996161580995249, "rank": 0.0}
|
| 101 |
+
{"step": 101, "steps": 250, "lr": 3.714558263156872e-05, "grad_norm": 2.2555856704711914, "loss_scale": 4.0, "retries": 0, "seconds": 351.6, "max_memory_gb": 17.13, "ce": 0.19666969340282492, "rank": 0.0}
|
| 102 |
+
{"step": 102, "steps": 250, "lr": 3.6871810660220904e-05, "grad_norm": 0.03894335404038429, "loss_scale": 4.0, "retries": 0, "seconds": 354.9, "max_memory_gb": 17.13, "ce": 0.0014833125315902862, "rank": 0.0}
|
| 103 |
+
{"step": 103, "steps": 250, "lr": 3.659634947354686e-05, "grad_norm": 3.201284408569336, "loss_scale": 4.0, "retries": 0, "seconds": 358.4, "max_memory_gb": 17.13, "ce": 0.36527846173885337, "rank": 0.0}
|
| 104 |
+
{"step": 104, "steps": 250, "lr": 3.631924872185169e-05, "grad_norm": 1.0764318704605103, "loss_scale": 4.0, "retries": 0, "seconds": 361.7, "max_memory_gb": 17.13, "ce": 0.08551471398072863, "rank": 0.0}
|
| 105 |
+
{"step": 105, "steps": 250, "lr": 3.6040558350962674e-05, "grad_norm": 4.006342887878418, "loss_scale": 4.0, "retries": 0, "seconds": 365.1, "max_memory_gb": 17.13, "ce": 0.15242940481320488, "rank": 0.0}
|
| 106 |
+
{"step": 106, "steps": 250, "lr": 3.576032859322687e-05, "grad_norm": 4.0807271003723145, "loss_scale": 4.0, "retries": 0, "seconds": 368.5, "max_memory_gb": 17.13, "ce": 0.27055742164293406, "rank": 0.0}
|
| 107 |
+
{"step": 107, "steps": 250, "lr": 3.5478609958457056e-05, "grad_norm": 1.3225754499435425, "loss_scale": 4.0, "retries": 0, "seconds": 371.9, "max_memory_gb": 17.13, "ce": 0.15132985958450718, "rank": 0.0}
|
| 108 |
+
{"step": 108, "steps": 250, "lr": 3.519545322482755e-05, "grad_norm": 2.1839492321014404, "loss_scale": 4.0, "retries": 0, "seconds": 375.3, "max_memory_gb": 17.13, "ce": 0.31314843149561966, "rank": 0.0}
|
| 109 |
+
{"step": 109, "steps": 250, "lr": 3.491090942972179e-05, "grad_norm": 0.6839438080787659, "loss_scale": 4.0, "retries": 0, "seconds": 378.8, "max_memory_gb": 17.13, "ce": 0.11963848173854785, "rank": 0.0}
|
| 110 |
+
{"step": 110, "steps": 250, "lr": 3.4625029860533135e-05, "grad_norm": 0.6763941049575806, "loss_scale": 4.0, "retries": 0, "seconds": 382.2, "max_memory_gb": 17.13, "ce": 0.019853277308357065, "rank": 0.0}
|
| 111 |
+
{"step": 111, "steps": 250, "lr": 3.433786604542056e-05, "grad_norm": 0.06504754722118378, "loss_scale": 4.0, "retries": 0, "seconds": 385.6, "max_memory_gb": 17.13, "ce": 0.0015722952955457004, "rank": 0.12500518560409546}
|
| 112 |
+
{"step": 112, "steps": 250, "lr": 3.404946974402104e-05, "grad_norm": 1.3451037406921387, "loss_scale": 4.0, "retries": 0, "seconds": 388.9, "max_memory_gb": 17.13, "ce": 0.0693795992201558, "rank": 0.0}
|
| 113 |
+
{"step": 113, "steps": 250, "lr": 3.375989293812019e-05, "grad_norm": 1.9885562658309937, "loss_scale": 4.0, "retries": 0, "seconds": 392.4, "max_memory_gb": 17.13, "ce": 0.4567960647673317, "rank": 0.0}
|
| 114 |
+
{"step": 114, "steps": 250, "lr": 3.3469187822282766e-05, "grad_norm": 0.09982169419527054, "loss_scale": 4.0, "retries": 0, "seconds": 395.8, "max_memory_gb": 17.13, "ce": 0.005389627791032581, "rank": 0.0}
|
| 115 |
+
{"step": 115, "steps": 250, "lr": 3.317740679444499e-05, "grad_norm": 6.308236122131348, "loss_scale": 4.0, "retries": 0, "seconds": 399.1, "max_memory_gb": 17.13, "ce": 0.34164882279492303, "rank": 0.0}
|
| 116 |
+
{"step": 116, "steps": 250, "lr": 3.2884602446470056e-05, "grad_norm": 0.8576182126998901, "loss_scale": 4.0, "retries": 0, "seconds": 402.7, "max_memory_gb": 17.13, "ce": 0.02290016194638156, "rank": 0.0}
|
| 117 |
+
{"step": 117, "steps": 250, "lr": 3.259082755466876e-05, "grad_norm": 2.9946019649505615, "loss_scale": 4.0, "retries": 0, "seconds": 406.1, "max_memory_gb": 17.13, "ce": 0.30053280533218185, "rank": 0.0}
|
| 118 |
+
{"step": 118, "steps": 250, "lr": 3.229613507028686e-05, "grad_norm": 2.32064151763916, "loss_scale": 4.0, "retries": 0, "seconds": 409.4, "max_memory_gb": 17.13, "ce": 0.33186128412012295, "rank": 0.12501557171344757}
|
| 119 |
+
{"step": 119, "steps": 250, "lr": 3.2000578109961004e-05, "grad_norm": 0.44863438606262207, "loss_scale": 4.0, "retries": 0, "seconds": 412.8, "max_memory_gb": 17.13, "ce": 0.02184442267173381, "rank": 0.0}
|
| 120 |
+
{"step": 120, "steps": 250, "lr": 3.1704209946144616e-05, "grad_norm": 2.0364561080932617, "loss_scale": 4.0, "retries": 0, "seconds": 416.3, "max_memory_gb": 17.13, "ce": 0.08488624681103829, "rank": 0.0}
|
| 121 |
+
{"step": 121, "steps": 250, "lr": 3.140708399750594e-05, "grad_norm": 0.9861705899238586, "loss_scale": 4.0, "retries": 0, "seconds": 419.7, "max_memory_gb": 17.13, "ce": 0.07651326829545724, "rank": 0.0}
|
| 122 |
+
{"step": 122, "steps": 250, "lr": 3.1109253819299606e-05, "grad_norm": 0.42127755284309387, "loss_scale": 4.0, "retries": 0, "seconds": 423.1, "max_memory_gb": 17.13, "ce": 0.02297824456491071, "rank": 0.0}
|
| 123 |
+
{"step": 123, "steps": 250, "lr": 3.081077309371361e-05, "grad_norm": 0.02709081955254078, "loss_scale": 4.0, "retries": 0, "seconds": 426.6, "max_memory_gb": 17.13, "ce": 0.001399579833559983, "rank": 0.0}
|
| 124 |
+
{"step": 124, "steps": 250, "lr": 3.051169562019339e-05, "grad_norm": 1.5418267250061035, "loss_scale": 4.0, "retries": 0, "seconds": 430.0, "max_memory_gb": 17.13, "ce": 0.14726970301944675, "rank": 0.2096032202243805}
|
| 125 |
+
{"step": 125, "steps": 250, "lr": 3.021207530574477e-05, "grad_norm": 0.4660196900367737, "loss_scale": 4.0, "retries": 0, "seconds": 433.4, "max_memory_gb": 17.13, "ce": 0.01104540449432534, "rank": 0.0}
|
| 126 |
+
{"step": 126, "steps": 250, "lr": 2.991196615521758e-05, "grad_norm": 1.528245449066162, "loss_scale": 4.0, "retries": 0, "seconds": 436.8, "max_memory_gb": 17.13, "ce": 0.364530456268767, "rank": 0.0}
|
| 127 |
+
{"step": 127, "steps": 250, "lr": 2.9611422261571536e-05, "grad_norm": 0.44840505719184875, "loss_scale": 4.0, "retries": 0, "seconds": 440.2, "max_memory_gb": 17.13, "ce": 0.0504530771422651, "rank": 0.0}
|
| 128 |
+
{"step": 128, "steps": 250, "lr": 2.931049779612634e-05, "grad_norm": 0.5218819975852966, "loss_scale": 4.0, "retries": 0, "seconds": 443.6, "max_memory_gb": 17.13, "ce": 0.04248843509685685, "rank": 0.0}
|
| 129 |
+
{"step": 129, "steps": 250, "lr": 2.9009246998797557e-05, "grad_norm": 2.762774705886841, "loss_scale": 4.0, "retries": 0, "seconds": 447.1, "max_memory_gb": 17.13, "ce": 0.17386546636862477, "rank": 0.0}
|
| 130 |
+
{"step": 130, "steps": 250, "lr": 2.8707724168320238e-05, "grad_norm": 0.523213267326355, "loss_scale": 4.0, "retries": 0, "seconds": 450.4, "max_memory_gb": 17.13, "ce": 0.03855325815766264, "rank": 0.0}
|
| 131 |
+
{"step": 131, "steps": 250, "lr": 2.840598365246184e-05, "grad_norm": 0.4362519681453705, "loss_scale": 4.0, "retries": 0, "seconds": 453.8, "max_memory_gb": 17.13, "ce": 0.122408005959187, "rank": 0.0}
|
| 132 |
+
{"step": 132, "steps": 250, "lr": 2.8104079838226383e-05, "grad_norm": 0.02242162451148033, "loss_scale": 4.0, "retries": 0, "seconds": 457.2, "max_memory_gb": 17.13, "ce": 0.001322622579664312, "rank": 0.0}
|
| 133 |
+
{"step": 133, "steps": 250, "lr": 2.7802067142051446e-05, "grad_norm": 0.18168160319328308, "loss_scale": 4.0, "retries": 0, "seconds": 460.7, "max_memory_gb": 17.13, "ce": 0.010860398640943458, "rank": 0.0}
|
| 134 |
+
{"step": 134, "steps": 250, "lr": 2.7500000000000004e-05, "grad_norm": 2.2536792755126953, "loss_scale": 4.0, "retries": 0, "seconds": 464.0, "max_memory_gb": 17.13, "ce": 0.0802104474259977, "rank": 0.0}
|
| 135 |
+
{"step": 135, "steps": 250, "lr": 2.7197932857948566e-05, "grad_norm": 1.560841679573059, "loss_scale": 4.0, "retries": 0, "seconds": 467.4, "max_memory_gb": 17.13, "ce": 0.36444205370116833, "rank": 0.0}
|
| 136 |
+
{"step": 136, "steps": 250, "lr": 2.689592016177363e-05, "grad_norm": 0.23289331793785095, "loss_scale": 4.0, "retries": 0, "seconds": 470.9, "max_memory_gb": 17.13, "ce": 0.008204637682865723, "rank": 0.0}
|
| 137 |
+
{"step": 137, "steps": 250, "lr": 2.659401634753817e-05, "grad_norm": 2.9624314308166504, "loss_scale": 4.0, "retries": 0, "seconds": 474.3, "max_memory_gb": 17.13, "ce": 0.381892890358813, "rank": 0.0}
|
| 138 |
+
{"step": 138, "steps": 250, "lr": 2.6292275831679764e-05, "grad_norm": 0.576200544834137, "loss_scale": 4.0, "retries": 0, "seconds": 477.7, "max_memory_gb": 17.14, "ce": 0.023317317150940653, "rank": 0.0}
|
| 139 |
+
{"step": 139, "steps": 250, "lr": 2.5990753001202445e-05, "grad_norm": 2.067539691925049, "loss_scale": 4.0, "retries": 0, "seconds": 481.1, "max_memory_gb": 17.14, "ce": 0.19350442725226458, "rank": 0.0}
|
| 140 |
+
{"step": 140, "steps": 250, "lr": 2.5689502203873672e-05, "grad_norm": 0.03329019993543625, "loss_scale": 4.0, "retries": 0, "seconds": 484.5, "max_memory_gb": 17.14, "ce": 0.0021524221820072853, "rank": 0.0}
|
| 141 |
+
{"step": 141, "steps": 250, "lr": 2.538857773842847e-05, "grad_norm": 0.9048910737037659, "loss_scale": 4.0, "retries": 0, "seconds": 488.0, "max_memory_gb": 17.14, "ce": 0.066048501052137, "rank": 0.0}
|
| 142 |
+
{"step": 142, "steps": 250, "lr": 2.5088033844782426e-05, "grad_norm": 0.24466003477573395, "loss_scale": 4.0, "retries": 0, "seconds": 491.4, "max_memory_gb": 17.14, "ce": 0.02129354491535196, "rank": 0.0}
|
| 143 |
+
{"step": 143, "steps": 250, "lr": 2.4787924694255237e-05, "grad_norm": 1.3820006847381592, "loss_scale": 4.0, "retries": 0, "seconds": 494.9, "max_memory_gb": 17.14, "ce": 0.1775090469436691, "rank": 0.0}
|
| 144 |
+
{"step": 144, "steps": 250, "lr": 2.4488304379806627e-05, "grad_norm": 0.5617222189903259, "loss_scale": 4.0, "retries": 0, "seconds": 498.3, "max_memory_gb": 17.14, "ce": 0.05954441474023042, "rank": 0.12549690902233124}
|
| 145 |
+
{"step": 145, "steps": 250, "lr": 2.4189226906286388e-05, "grad_norm": 0.08209459483623505, "loss_scale": 4.0, "retries": 0, "seconds": 501.7, "max_memory_gb": 17.14, "ce": 0.006440077954266599, "rank": 0.0}
|
| 146 |
+
{"step": 146, "steps": 250, "lr": 2.3890746180700395e-05, "grad_norm": 0.2856885492801666, "loss_scale": 4.0, "retries": 0, "seconds": 505.1, "max_memory_gb": 17.14, "ce": 0.020349903267742775, "rank": 0.0}
|
| 147 |
+
{"step": 147, "steps": 250, "lr": 2.359291600249407e-05, "grad_norm": 0.7994946241378784, "loss_scale": 4.0, "retries": 0, "seconds": 508.5, "max_memory_gb": 17.14, "ce": 0.053358849641881534, "rank": 0.0}
|
| 148 |
+
{"step": 148, "steps": 250, "lr": 2.3295790053855392e-05, "grad_norm": 1.8122507333755493, "loss_scale": 4.0, "retries": 0, "seconds": 511.9, "max_memory_gb": 17.14, "ce": 0.26439919333733997, "rank": 0.0}
|
| 149 |
+
{"step": 149, "steps": 250, "lr": 2.299942189003901e-05, "grad_norm": 2.1741385459899902, "loss_scale": 4.0, "retries": 0, "seconds": 515.3, "max_memory_gb": 17.14, "ce": 0.2603334892546627, "rank": 0.0}
|
| 150 |
+
{"step": 150, "steps": 250, "lr": 2.2703864929713144e-05, "grad_norm": 0.5491346120834351, "loss_scale": 4.0, "retries": 0, "seconds": 518.8, "max_memory_gb": 17.14, "ce": 0.030814123878798227, "rank": 0.0}
|
| 151 |
+
{"step": 151, "steps": 250, "lr": 2.2409172445331257e-05, "grad_norm": 0.03970259055495262, "loss_scale": 4.0, "retries": 0, "seconds": 522.2, "max_memory_gb": 17.14, "ce": 0.0021948448147668387, "rank": 0.0}
|
| 152 |
+
{"step": 152, "steps": 250, "lr": 2.211539755352996e-05, "grad_norm": 3.3334200382232666, "loss_scale": 4.0, "retries": 0, "seconds": 525.6, "max_memory_gb": 17.14, "ce": 0.1091606829313605, "rank": 0.039842694997787476}
|
| 153 |
+
{"step": 153, "steps": 250, "lr": 2.1822593205555016e-05, "grad_norm": 0.15752683579921722, "loss_scale": 4.0, "retries": 0, "seconds": 529.2, "max_memory_gb": 17.14, "ce": 0.010429024145651056, "rank": 0.0}
|
| 154 |
+
{"step": 154, "steps": 250, "lr": 2.1530812177717236e-05, "grad_norm": 0.08656812459230423, "loss_scale": 4.0, "retries": 0, "seconds": 532.6, "max_memory_gb": 17.14, "ce": 0.005649004996143958, "rank": 0.0}
|
| 155 |
+
{"step": 155, "steps": 250, "lr": 2.1240107061879815e-05, "grad_norm": 0.3334428369998932, "loss_scale": 4.0, "retries": 0, "seconds": 536.2, "max_memory_gb": 17.14, "ce": 0.10826637165610009, "rank": 0.0}
|
| 156 |
+
{"step": 156, "steps": 250, "lr": 2.095053025597896e-05, "grad_norm": 1.1018704175949097, "loss_scale": 4.0, "retries": 0, "seconds": 540.4, "max_memory_gb": 17.14, "ce": 0.09371667082950808, "rank": 0.0}
|
| 157 |
+
{"step": 157, "steps": 250, "lr": 2.066213395457945e-05, "grad_norm": 0.30173611640930176, "loss_scale": 4.0, "retries": 0, "seconds": 544.7, "max_memory_gb": 17.14, "ce": 0.007416370053761057, "rank": 0.0}
|
| 158 |
+
{"step": 158, "steps": 250, "lr": 2.0374970139466877e-05, "grad_norm": 0.5321121215820312, "loss_scale": 4.0, "retries": 0, "seconds": 548.8, "max_memory_gb": 17.14, "ce": 0.059480784014340315, "rank": 0.0}
|
| 159 |
+
{"step": 159, "steps": 250, "lr": 2.0089090570278213e-05, "grad_norm": 0.09884887933731079, "loss_scale": 4.0, "retries": 0, "seconds": 553.0, "max_memory_gb": 17.14, "ce": 0.0059841806196345715, "rank": 0.0}
|
| 160 |
+
{"step": 160, "steps": 250, "lr": 1.9804546775172457e-05, "grad_norm": 5.352631568908691, "loss_scale": 4.0, "retries": 0, "seconds": 557.2, "max_memory_gb": 17.14, "ce": 0.22695723096330767, "rank": 0.12639382481575012}
|
| 161 |
+
{"step": 161, "steps": 250, "lr": 1.9521390041542946e-05, "grad_norm": 3.3346269130706787, "loss_scale": 4.0, "retries": 0, "seconds": 561.4, "max_memory_gb": 17.14, "ce": 0.1588896341763757, "rank": 0.0}
|
| 162 |
+
{"step": 162, "steps": 250, "lr": 1.923967140677313e-05, "grad_norm": 1.5724692344665527, "loss_scale": 4.0, "retries": 0, "seconds": 565.5, "max_memory_gb": 17.14, "ce": 0.02863693688686908, "rank": 0.0}
|
| 163 |
+
{"step": 163, "steps": 250, "lr": 1.8959441649037335e-05, "grad_norm": 0.3759657144546509, "loss_scale": 4.0, "retries": 0, "seconds": 569.8, "max_memory_gb": 17.14, "ce": 0.05839727609554757, "rank": 0.0}
|
| 164 |
+
{"step": 164, "steps": 250, "lr": 1.868075127814832e-05, "grad_norm": 1.2346726655960083, "loss_scale": 4.0, "retries": 0, "seconds": 573.9, "max_memory_gb": 17.14, "ce": 0.15479048986662747, "rank": 0.0}
|
| 165 |
+
{"step": 165, "steps": 250, "lr": 1.8403650526453143e-05, "grad_norm": 0.0940764844417572, "loss_scale": 4.0, "retries": 0, "seconds": 578.1, "max_memory_gb": 17.14, "ce": 0.005688035518687684, "rank": 0.0}
|
| 166 |
+
{"step": 166, "steps": 250, "lr": 1.812818933977912e-05, "grad_norm": 0.19554086029529572, "loss_scale": 4.0, "retries": 0, "seconds": 582.3, "max_memory_gb": 17.14, "ce": 0.012376193082673126, "rank": 0.0}
|
| 167 |
+
{"step": 167, "steps": 250, "lr": 1.785441736843128e-05, "grad_norm": 0.026239344850182533, "loss_scale": 4.0, "retries": 0, "seconds": 586.4, "max_memory_gb": 17.14, "ce": 0.0015301015059776546, "rank": 0.0}
|
| 168 |
+
{"step": 168, "steps": 250, "lr": 1.7582383958243314e-05, "grad_norm": 0.4452998638153076, "loss_scale": 4.0, "retries": 0, "seconds": 590.6, "max_memory_gb": 17.14, "ce": 0.03451446320832474, "rank": 0.0}
|
| 169 |
+
{"step": 169, "steps": 250, "lr": 1.7312138141683164e-05, "grad_norm": 1.6722594499588013, "loss_scale": 4.0, "retries": 0, "seconds": 594.7, "max_memory_gb": 17.14, "ce": 0.05925960121430762, "rank": 0.24949494004249573}
|
| 170 |
+
{"step": 170, "steps": 250, "lr": 1.704372862901521e-05, "grad_norm": 0.5370288491249084, "loss_scale": 4.0, "retries": 0, "seconds": 599.3, "max_memory_gb": 17.14, "ce": 0.05816608199438633, "rank": 0.0}
|
| 171 |
+
{"step": 171, "steps": 250, "lr": 1.6777203799520573e-05, "grad_norm": 1.6773368120193481, "loss_scale": 4.0, "retries": 0, "seconds": 603.8, "max_memory_gb": 17.14, "ce": 0.08552741391304153, "rank": 0.0}
|
| 172 |
+
{"step": 172, "steps": 250, "lr": 1.6512611692777008e-05, "grad_norm": 1.854698896408081, "loss_scale": 4.0, "retries": 0, "seconds": 608.3, "max_memory_gb": 17.14, "ce": 0.15172306782005762, "rank": 0.0}
|
| 173 |
+
{"step": 173, "steps": 250, "lr": 1.6250000000000005e-05, "grad_norm": 0.6195264458656311, "loss_scale": 4.0, "retries": 0, "seconds": 612.9, "max_memory_gb": 17.14, "ce": 0.1050201181878947, "rank": 0.0}
|
| 174 |
+
{"step": 174, "steps": 250, "lr": 1.598941605544678e-05, "grad_norm": 1.4054714441299438, "loss_scale": 4.0, "retries": 0, "seconds": 617.4, "max_memory_gb": 17.14, "ce": 0.02160106458268274, "rank": 0.0}
|
| 175 |
+
{"step": 175, "steps": 250, "lr": 1.5730906827884506e-05, "grad_norm": 0.14891472458839417, "loss_scale": 4.0, "retries": 0, "seconds": 621.9, "max_memory_gb": 17.14, "ce": 0.008957210044172825, "rank": 0.0}
|
| 176 |
+
{"step": 176, "steps": 250, "lr": 1.5474518912124476e-05, "grad_norm": 0.5474185347557068, "loss_scale": 4.0, "retries": 0, "seconds": 626.4, "max_memory_gb": 17.14, "ce": 0.064846827488509, "rank": 0.0}
|
| 177 |
+
{"step": 177, "steps": 250, "lr": 1.5220298520623646e-05, "grad_norm": 2.5265135765075684, "loss_scale": 4.0, "retries": 0, "seconds": 631.0, "max_memory_gb": 17.14, "ce": 0.13196381176749128, "rank": 0.035071372985839844}
|
| 178 |
+
{"step": 178, "steps": 250, "lr": 1.4968291475155121e-05, "grad_norm": 0.6893824338912964, "loss_scale": 4.0, "retries": 0, "seconds": 635.5, "max_memory_gb": 17.14, "ce": 0.034700041475161925, "rank": 0.0}
|
| 179 |
+
{"step": 179, "steps": 250, "lr": 1.4718543198548998e-05, "grad_norm": 0.6343981623649597, "loss_scale": 4.0, "retries": 0, "seconds": 640.0, "max_memory_gb": 17.14, "ce": 0.07186406396431266, "rank": 0.0}
|
| 180 |
+
{"step": 180, "steps": 250, "lr": 1.4471098706505227e-05, "grad_norm": 1.001916766166687, "loss_scale": 4.0, "retries": 0, "seconds": 644.6, "max_memory_gb": 17.14, "ce": 0.1961812633817317, "rank": 0.0}
|
| 181 |
+
{"step": 181, "steps": 250, "lr": 1.4226002599479743e-05, "grad_norm": 2.568105459213257, "loss_scale": 4.0, "retries": 0, "seconds": 649.1, "max_memory_gb": 17.14, "ce": 0.3493020614014313, "rank": 0.0}
|
| 182 |
+
{"step": 182, "steps": 250, "lr": 1.3983299054645471e-05, "grad_norm": 1.9339854717254639, "loss_scale": 4.0, "retries": 0, "seconds": 653.6, "max_memory_gb": 17.14, "ce": 0.1732045810422278, "rank": 0.0}
|
| 183 |
+
{"step": 183, "steps": 250, "lr": 1.3743031817929724e-05, "grad_norm": 0.035271432250738144, "loss_scale": 4.0, "retries": 0, "seconds": 658.4, "max_memory_gb": 17.14, "ce": 0.0016799729919512174, "rank": 0.0}
|
| 184 |
+
{"step": 184, "steps": 250, "lr": 1.3505244196129164e-05, "grad_norm": 0.004687133245170116, "loss_scale": 4.0, "retries": 0, "seconds": 662.1, "max_memory_gb": 17.14, "ce": 0.0003102393077369925, "rank": 0.0}
|
| 185 |
+
{"step": 185, "steps": 250, "lr": 1.3269979049104014e-05, "grad_norm": 0.47911521792411804, "loss_scale": 4.0, "retries": 0, "seconds": 665.7, "max_memory_gb": 17.14, "ce": 0.00041187386159435846, "rank": 0.06935910880565643}
|
| 186 |
+
{"step": 186, "steps": 250, "lr": 1.3037278782052863e-05, "grad_norm": 0.32647350430488586, "loss_scale": 4.0, "retries": 0, "seconds": 669.4, "max_memory_gb": 17.14, "ce": 0.014495183541839651, "rank": 0.0}
|
| 187 |
+
{"step": 187, "steps": 250, "lr": 1.2807185337869343e-05, "grad_norm": 2.0271034240722656, "loss_scale": 4.0, "retries": 0, "seconds": 673.0, "max_memory_gb": 17.14, "ce": 0.19314818704197023, "rank": 0.0}
|
| 188 |
+
{"step": 188, "steps": 250, "lr": 1.2579740189582115e-05, "grad_norm": 1.7330647706985474, "loss_scale": 4.0, "retries": 0, "seconds": 676.4, "max_memory_gb": 17.14, "ce": 0.10653801270836993, "rank": 0.0}
|
| 189 |
+
{"step": 189, "steps": 250, "lr": 1.2354984332879668e-05, "grad_norm": 1.0646849870681763, "loss_scale": 4.0, "retries": 0, "seconds": 679.9, "max_memory_gb": 17.14, "ce": 0.08954543405309323, "rank": 0.0}
|
| 190 |
+
{"step": 190, "steps": 250, "lr": 1.2132958278721064e-05, "grad_norm": 0.34127429127693176, "loss_scale": 4.0, "retries": 0, "seconds": 683.5, "max_memory_gb": 17.14, "ce": 0.02513190456375014, "rank": 0.12499065697193146}
|
| 191 |
+
{"step": 191, "steps": 250, "lr": 1.1913702046034016e-05, "grad_norm": 0.019389165565371513, "loss_scale": 4.0, "retries": 0, "seconds": 686.9, "max_memory_gb": 17.14, "ce": 0.0010006851595107946, "rank": 0.0}
|
| 192 |
+
{"step": 192, "steps": 250, "lr": 1.1697255154501756e-05, "grad_norm": 1.7798352241516113, "loss_scale": 4.0, "retries": 0, "seconds": 690.3, "max_memory_gb": 17.14, "ce": 0.34835940829816536, "rank": 0.0}
|
| 193 |
+
{"step": 193, "steps": 250, "lr": 1.1483656617439842e-05, "grad_norm": 0.2238897979259491, "loss_scale": 4.0, "retries": 0, "seconds": 693.8, "max_memory_gb": 17.14, "ce": 0.012636628553082119, "rank": 0.0}
|
| 194 |
+
{"step": 194, "steps": 250, "lr": 1.1272944934764177e-05, "grad_norm": 0.20044560730457306, "loss_scale": 4.0, "retries": 0, "seconds": 697.4, "max_memory_gb": 17.14, "ce": 0.0077204410679314606, "rank": 0.12498463690280914}
|
| 195 |
+
{"step": 195, "steps": 250, "lr": 1.1065158086051684e-05, "grad_norm": 0.03615788370370865, "loss_scale": 4.0, "retries": 0, "seconds": 700.8, "max_memory_gb": 17.14, "ce": 0.0011418854523981281, "rank": 0.0}
|
| 196 |
+
{"step": 196, "steps": 250, "lr": 1.086033352369467e-05, "grad_norm": 0.006737919524312019, "loss_scale": 4.0, "retries": 0, "seconds": 704.2, "max_memory_gb": 17.14, "ce": 0.0003713932333084813, "rank": 0.0}
|
| 197 |
+
{"step": 197, "steps": 250, "lr": 1.0658508166150224e-05, "grad_norm": 0.3779560327529907, "loss_scale": 4.0, "retries": 0, "seconds": 707.8, "max_memory_gb": 17.14, "ce": 0.016256074718512536, "rank": 0.0}
|
| 198 |
+
{"step": 198, "steps": 250, "lr": 1.0459718391285954e-05, "grad_norm": 0.45736801624298096, "loss_scale": 4.0, "retries": 0, "seconds": 711.2, "max_memory_gb": 17.14, "ce": 0.04213773832998413, "rank": 0.0}
|
| 199 |
+
{"step": 199, "steps": 250, "lr": 1.0264000029822999e-05, "grad_norm": 1.226369023323059, "loss_scale": 4.0, "retries": 0, "seconds": 714.7, "max_memory_gb": 17.14, "ce": 0.1534483642574287, "rank": 0.0}
|
| 200 |
+
{"step": 200, "steps": 250, "lr": 1.0071388358877775e-05, "grad_norm": 0.6446638107299805, "loss_scale": 4.0, "retries": 0, "seconds": 718.2, "max_memory_gb": 17.14, "ce": 0.08786725170102727, "rank": 0.0}
|
| 201 |
+
{"step": 201, "steps": 250, "lr": 9.881918095603501e-06, "grad_norm": 1.2733914852142334, "loss_scale": 4.0, "retries": 0, "seconds": 721.6, "max_memory_gb": 17.14, "ce": 0.016867487947820337, "rank": 0.0}
|
| 202 |
+
{"step": 202, "steps": 250, "lr": 9.695623390932612e-06, "grad_norm": 1.0757428407669067, "loss_scale": 4.0, "retries": 0, "seconds": 725.1, "max_memory_gb": 17.14, "ce": 0.31129653432299165, "rank": 0.0}
|
| 203 |
+
{"step": 203, "steps": 250, "lr": 9.51253782342123e-06, "grad_norm": 0.05602607503533363, "loss_scale": 4.0, "retries": 0, "seconds": 728.6, "max_memory_gb": 17.14, "ce": 0.0028461952674661006, "rank": 0.0}
|
| 204 |
+
{"step": 204, "steps": 250, "lr": 9.332694393196846e-06, "grad_norm": 0.6362344026565552, "loss_scale": 4.0, "retries": 0, "seconds": 732.1, "max_memory_gb": 17.14, "ce": 0.06058122745889705, "rank": 0.0}
|
| 205 |
+
{"step": 205, "steps": 250, "lr": 9.156125516010256e-06, "grad_norm": 0.06494426727294922, "loss_scale": 4.0, "retries": 0, "seconds": 735.5, "max_memory_gb": 17.14, "ce": 0.003932191898456949, "rank": 0.0}
|
| 206 |
+
{"step": 206, "steps": 250, "lr": 8.982863017392733e-06, "grad_norm": 0.9483063220977783, "loss_scale": 4.0, "retries": 0, "seconds": 738.9, "max_memory_gb": 17.14, "ce": 0.05294718444065438, "rank": 0.0}
|
| 207 |
+
{"step": 207, "steps": 250, "lr": 8.81293812691974e-06, "grad_norm": 0.07825028896331787, "loss_scale": 4.0, "retries": 0, "seconds": 742.5, "max_memory_gb": 17.14, "ce": 0.003243081081109267, "rank": 0.0}
|
| 208 |
+
{"step": 208, "steps": 250, "lr": 8.646381472581927e-06, "grad_norm": 0.3452921509742737, "loss_scale": 4.0, "retries": 0, "seconds": 745.9, "max_memory_gb": 17.14, "ce": 0.024772810797912825, "rank": 0.0}
|
| 209 |
+
{"step": 209, "steps": 250, "lr": 8.483223075264622e-06, "grad_norm": 0.31802067160606384, "loss_scale": 4.0, "retries": 0, "seconds": 749.4, "max_memory_gb": 17.14, "ce": 0.014115621126620681, "rank": 0.0}
|
| 210 |
+
{"step": 210, "steps": 250, "lr": 8.323492343336747e-06, "grad_norm": 0.6111275553703308, "loss_scale": 4.0, "retries": 0, "seconds": 752.9, "max_memory_gb": 17.14, "ce": 0.04367305395726362, "rank": 0.0}
|
| 211 |
+
{"step": 211, "steps": 250, "lr": 8.167218067350122e-06, "grad_norm": 0.22443446516990662, "loss_scale": 4.0, "retries": 0, "seconds": 756.3, "max_memory_gb": 17.14, "ce": 0.009909881395628872, "rank": 0.0}
|
| 212 |
+
{"step": 212, "steps": 250, "lr": 8.01442841485013e-06, "grad_norm": 1.2395015954971313, "loss_scale": 4.0, "retries": 0, "seconds": 759.7, "max_memory_gb": 17.14, "ce": 0.2053315034481784, "rank": 0.0}
|
| 213 |
+
{"step": 213, "steps": 250, "lr": 7.865150925298678e-06, "grad_norm": 1.42086923122406, "loss_scale": 4.0, "retries": 0, "seconds": 763.2, "max_memory_gb": 17.14, "ce": 0.4823860714714101, "rank": 0.0}
|
| 214 |
+
{"step": 214, "steps": 250, "lr": 7.719412505110375e-06, "grad_norm": 0.1696472316980362, "loss_scale": 4.0, "retries": 0, "seconds": 766.7, "max_memory_gb": 17.14, "ce": 0.012185707424123393, "rank": 0.0}
|
| 215 |
+
{"step": 215, "steps": 250, "lr": 7.57723942280278e-06, "grad_norm": 0.7911519408226013, "loss_scale": 4.0, "retries": 0, "seconds": 770.1, "max_memory_gb": 17.14, "ce": 0.025583542095773737, "rank": 0.0}
|
| 216 |
+
{"step": 216, "steps": 250, "lr": 7.438657304261687e-06, "grad_norm": 0.84166419506073, "loss_scale": 4.0, "retries": 0, "seconds": 773.6, "max_memory_gb": 17.14, "ce": 0.033215551048215275, "rank": 0.0}
|
| 217 |
+
{"step": 217, "steps": 250, "lr": 7.30369112812219e-06, "grad_norm": 0.7230973839759827, "loss_scale": 4.0, "retries": 0, "seconds": 777.2, "max_memory_gb": 17.14, "ce": 0.059773079306978616, "rank": 0.12485560029745102}
|
| 218 |
+
{"step": 218, "steps": 250, "lr": 7.172365221266403e-06, "grad_norm": 1.9923118352890015, "loss_scale": 4.0, "retries": 0, "seconds": 780.6, "max_memory_gb": 17.14, "ce": 0.28116448033006236, "rank": 0.0}
|
| 219 |
+
{"step": 219, "steps": 250, "lr": 7.0447032544387154e-06, "grad_norm": 0.2732953131198883, "loss_scale": 4.0, "retries": 0, "seconds": 784.1, "max_memory_gb": 17.14, "ce": 0.01438130276164884, "rank": 0.0}
|
| 220 |
+
{"step": 220, "steps": 250, "lr": 6.920728237979274e-06, "grad_norm": 1.5783413648605347, "loss_scale": 4.0, "retries": 0, "seconds": 787.6, "max_memory_gb": 17.14, "ce": 0.0994636413643093, "rank": 0.0}
|
| 221 |
+
{"step": 221, "steps": 250, "lr": 6.800462517676456e-06, "grad_norm": 2.4063854217529297, "loss_scale": 4.0, "retries": 0, "seconds": 791.0, "max_memory_gb": 17.14, "ce": 0.24455577551907481, "rank": 0.0}
|
| 222 |
+
{"step": 222, "steps": 250, "lr": 6.683927770739222e-06, "grad_norm": 0.9690293669700623, "loss_scale": 4.0, "retries": 0, "seconds": 794.4, "max_memory_gb": 17.14, "ce": 0.040255319328480255, "rank": 0.0}
|
| 223 |
+
{"step": 223, "steps": 250, "lr": 6.571145001889892e-06, "grad_norm": 1.617644190788269, "loss_scale": 4.0, "retries": 0, "seconds": 797.9, "max_memory_gb": 17.14, "ce": 0.054363910478969046, "rank": 0.0}
|
| 224 |
+
{"step": 224, "steps": 250, "lr": 6.46213453957817e-06, "grad_norm": 0.005548081360757351, "loss_scale": 4.0, "retries": 0, "seconds": 801.4, "max_memory_gb": 17.14, "ce": 0.0003139857452083561, "rank": 0.0}
|
| 225 |
+
{"step": 225, "steps": 250, "lr": 6.356916032317063e-06, "grad_norm": 1.3050445318222046, "loss_scale": 4.0, "retries": 0, "seconds": 804.8, "max_memory_gb": 17.14, "ce": 0.23629874694142927, "rank": 0.0}
|
| 226 |
+
{"step": 226, "steps": 250, "lr": 6.255508445141349e-06, "grad_norm": 1.7348296642303467, "loss_scale": 4.0, "retries": 0, "seconds": 808.3, "max_memory_gb": 17.14, "ce": 0.14510540823630436, "rank": 0.0}
|
| 227 |
+
{"step": 227, "steps": 250, "lr": 6.157930056189228e-06, "grad_norm": 0.08612186461687088, "loss_scale": 4.0, "retries": 0, "seconds": 811.8, "max_memory_gb": 17.14, "ce": 0.006967899324990867, "rank": 0.0}
|
| 228 |
+
{"step": 228, "steps": 250, "lr": 6.064198453407796e-06, "grad_norm": 0.8604782223701477, "loss_scale": 4.0, "retries": 0, "seconds": 815.2, "max_memory_gb": 17.14, "ce": 0.07508576819327573, "rank": 0.0}
|
| 229 |
+
{"step": 229, "steps": 250, "lr": 5.974330531382935e-06, "grad_norm": 1.4531700611114502, "loss_scale": 4.0, "retries": 0, "seconds": 818.7, "max_memory_gb": 17.14, "ce": 0.34888743595729466, "rank": 0.0}
|
| 230 |
+
{"step": 230, "steps": 250, "lr": 5.888342488294125e-06, "grad_norm": 0.16113419830799103, "loss_scale": 4.0, "retries": 0, "seconds": 822.1, "max_memory_gb": 17.14, "ce": 0.012814236683880154, "rank": 0.0}
|
| 231 |
+
{"step": 231, "steps": 250, "lr": 5.806249822994849e-06, "grad_norm": 1.7102042436599731, "loss_scale": 4.0, "retries": 0, "seconds": 825.6, "max_memory_gb": 17.14, "ce": 0.11928221145262796, "rank": 0.0}
|
| 232 |
+
{"step": 232, "steps": 250, "lr": 5.728067332218994e-06, "grad_norm": 0.4743361175060272, "loss_scale": 4.0, "retries": 0, "seconds": 829.1, "max_memory_gb": 17.14, "ce": 0.00440877591699973, "rank": 0.051097020506858826}
|
| 233 |
+
{"step": 233, "steps": 250, "lr": 5.65380910791383e-06, "grad_norm": 0.3504549264907837, "loss_scale": 4.0, "retries": 0, "seconds": 832.5, "max_memory_gb": 17.14, "ce": 0.03142366794418194, "rank": 0.0}
|
| 234 |
+
{"step": 234, "steps": 250, "lr": 5.583488534700022e-06, "grad_norm": 1.1814296245574951, "loss_scale": 4.0, "retries": 0, "seconds": 836.0, "max_memory_gb": 17.14, "ce": 0.035055400581313734, "rank": 0.0}
|
| 235 |
+
{"step": 235, "steps": 250, "lr": 5.517118287459118e-06, "grad_norm": 5.795176982879639, "loss_scale": 4.0, "retries": 0, "seconds": 839.4, "max_memory_gb": 17.14, "ce": 0.5054483225462718, "rank": 0.07471555471420288}
|
| 236 |
+
{"step": 236, "steps": 250, "lr": 5.454710329048978e-06, "grad_norm": 0.5556782484054565, "loss_scale": 4.0, "retries": 0, "seconds": 842.9, "max_memory_gb": 17.14, "ce": 0.06607511277979938, "rank": 0.0}
|
| 237 |
+
{"step": 237, "steps": 250, "lr": 5.3962759081475436e-06, "grad_norm": 0.9163450002670288, "loss_scale": 4.0, "retries": 0, "seconds": 846.4, "max_memory_gb": 17.14, "ce": 0.1486251118421933, "rank": 0.0}
|
| 238 |
+
{"step": 238, "steps": 250, "lr": 5.341825557225321e-06, "grad_norm": 4.058011054992676, "loss_scale": 4.0, "retries": 0, "seconds": 849.8, "max_memory_gb": 17.14, "ce": 0.27584238086183177, "rank": 0.0}
|
| 239 |
+
{"step": 239, "steps": 250, "lr": 5.291369090646961e-06, "grad_norm": 0.4857001304626465, "loss_scale": 4.0, "retries": 0, "seconds": 853.3, "max_memory_gb": 17.14, "ce": 0.020087991661966953, "rank": 0.0}
|
| 240 |
+
{"step": 240, "steps": 250, "lr": 5.244915602902289e-06, "grad_norm": 2.656404972076416, "loss_scale": 4.0, "retries": 0, "seconds": 856.9, "max_memory_gb": 17.14, "ce": 0.3960058884613318, "rank": 0.0}
|
| 241 |
+
{"step": 241, "steps": 250, "lr": 5.20247346696706e-06, "grad_norm": 0.2600562572479248, "loss_scale": 4.0, "retries": 0, "seconds": 860.3, "max_memory_gb": 17.14, "ce": 0.019085730385086208, "rank": 0.0}
|
| 242 |
+
{"step": 242, "steps": 250, "lr": 5.164050332793787e-06, "grad_norm": 1.4288945198059082, "loss_scale": 4.0, "retries": 0, "seconds": 863.8, "max_memory_gb": 17.14, "ce": 0.19778137945604612, "rank": 0.12479734420776367}
|
| 243 |
+
{"step": 243, "steps": 250, "lr": 5.129653125932873e-06, "grad_norm": 1.791490912437439, "loss_scale": 4.0, "retries": 0, "seconds": 867.2, "max_memory_gb": 17.14, "ce": 0.1506677176712401, "rank": 0.0}
|
| 244 |
+
{"step": 244, "steps": 250, "lr": 5.099288046284329e-06, "grad_norm": 0.8138139247894287, "loss_scale": 4.0, "retries": 0, "seconds": 870.7, "max_memory_gb": 17.14, "ce": 0.05356975647737272, "rank": 0.0}
|
| 245 |
+
{"step": 245, "steps": 250, "lr": 5.072960566980277e-06, "grad_norm": 0.7735481858253479, "loss_scale": 4.0, "retries": 0, "seconds": 874.1, "max_memory_gb": 17.14, "ce": 0.1455503655007533, "rank": 0.0}
|
| 246 |
+
{"step": 246, "steps": 250, "lr": 5.050675433398435e-06, "grad_norm": 0.67010098695755, "loss_scale": 4.0, "retries": 0, "seconds": 877.6, "max_memory_gb": 17.14, "ce": 0.06416940519375203, "rank": 0.0}
|
| 247 |
+
{"step": 247, "steps": 250, "lr": 5.032436662306811e-06, "grad_norm": 0.0472814105451107, "loss_scale": 4.0, "retries": 0, "seconds": 881.2, "max_memory_gb": 17.14, "ce": 0.002714589244533272, "rank": 0.0}
|
| 248 |
+
{"step": 248, "steps": 250, "lr": 5.018247541139684e-06, "grad_norm": 0.025020737200975418, "loss_scale": 4.0, "retries": 0, "seconds": 884.6, "max_memory_gb": 17.14, "ce": 0.0010724577368819155, "rank": 0.0}
|
| 249 |
+
{"step": 249, "steps": 250, "lr": 5.00811062740508e-06, "grad_norm": 0.4078994691371918, "loss_scale": 4.0, "retries": 0, "seconds": 888.1, "max_memory_gb": 17.14, "ce": 0.03904333116724956, "rank": 0.11016958951950073}
|
| 250 |
+
{"step": 250, "steps": 250, "lr": 5.002027748223778e-06, "grad_norm": 0.11733339726924896, "loss_scale": 4.0, "retries": 0, "seconds": 891.5, "max_memory_gb": 17.14, "ce": 0.004158614630341617, "rank": 0.113809734582901}
|
runs/fixed_internvl/removal/TopDelta8_s73/adapter/README.md
ADDED
|
@@ -0,0 +1,207 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
base_model: /data/OPSD/assets/models/InternVL3_5-8B-HF
|
| 3 |
+
library_name: peft
|
| 4 |
+
pipeline_tag: text-generation
|
| 5 |
+
tags:
|
| 6 |
+
- base_model:adapter:/data/OPSD/assets/models/InternVL3_5-8B-HF
|
| 7 |
+
- lora
|
| 8 |
+
- transformers
|
| 9 |
+
---
|
| 10 |
+
|
| 11 |
+
# Model Card for Model ID
|
| 12 |
+
|
| 13 |
+
<!-- Provide a quick summary of what the model is/does. -->
|
| 14 |
+
|
| 15 |
+
|
| 16 |
+
|
| 17 |
+
## Model Details
|
| 18 |
+
|
| 19 |
+
### Model Description
|
| 20 |
+
|
| 21 |
+
<!-- Provide a longer summary of what this model is. -->
|
| 22 |
+
|
| 23 |
+
|
| 24 |
+
|
| 25 |
+
- **Developed by:** [More Information Needed]
|
| 26 |
+
- **Funded by [optional]:** [More Information Needed]
|
| 27 |
+
- **Shared by [optional]:** [More Information Needed]
|
| 28 |
+
- **Model type:** [More Information Needed]
|
| 29 |
+
- **Language(s) (NLP):** [More Information Needed]
|
| 30 |
+
- **License:** [More Information Needed]
|
| 31 |
+
- **Finetuned from model [optional]:** [More Information Needed]
|
| 32 |
+
|
| 33 |
+
### Model Sources [optional]
|
| 34 |
+
|
| 35 |
+
<!-- Provide the basic links for the model. -->
|
| 36 |
+
|
| 37 |
+
- **Repository:** [More Information Needed]
|
| 38 |
+
- **Paper [optional]:** [More Information Needed]
|
| 39 |
+
- **Demo [optional]:** [More Information Needed]
|
| 40 |
+
|
| 41 |
+
## Uses
|
| 42 |
+
|
| 43 |
+
<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
|
| 44 |
+
|
| 45 |
+
### Direct Use
|
| 46 |
+
|
| 47 |
+
<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
|
| 48 |
+
|
| 49 |
+
[More Information Needed]
|
| 50 |
+
|
| 51 |
+
### Downstream Use [optional]
|
| 52 |
+
|
| 53 |
+
<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
|
| 54 |
+
|
| 55 |
+
[More Information Needed]
|
| 56 |
+
|
| 57 |
+
### Out-of-Scope Use
|
| 58 |
+
|
| 59 |
+
<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
|
| 60 |
+
|
| 61 |
+
[More Information Needed]
|
| 62 |
+
|
| 63 |
+
## Bias, Risks, and Limitations
|
| 64 |
+
|
| 65 |
+
<!-- This section is meant to convey both technical and sociotechnical limitations. -->
|
| 66 |
+
|
| 67 |
+
[More Information Needed]
|
| 68 |
+
|
| 69 |
+
### Recommendations
|
| 70 |
+
|
| 71 |
+
<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
|
| 72 |
+
|
| 73 |
+
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
|
| 74 |
+
|
| 75 |
+
## How to Get Started with the Model
|
| 76 |
+
|
| 77 |
+
Use the code below to get started with the model.
|
| 78 |
+
|
| 79 |
+
[More Information Needed]
|
| 80 |
+
|
| 81 |
+
## Training Details
|
| 82 |
+
|
| 83 |
+
### Training Data
|
| 84 |
+
|
| 85 |
+
<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
|
| 86 |
+
|
| 87 |
+
[More Information Needed]
|
| 88 |
+
|
| 89 |
+
### Training Procedure
|
| 90 |
+
|
| 91 |
+
<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
|
| 92 |
+
|
| 93 |
+
#### Preprocessing [optional]
|
| 94 |
+
|
| 95 |
+
[More Information Needed]
|
| 96 |
+
|
| 97 |
+
|
| 98 |
+
#### Training Hyperparameters
|
| 99 |
+
|
| 100 |
+
- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
|
| 101 |
+
|
| 102 |
+
#### Speeds, Sizes, Times [optional]
|
| 103 |
+
|
| 104 |
+
<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
|
| 105 |
+
|
| 106 |
+
[More Information Needed]
|
| 107 |
+
|
| 108 |
+
## Evaluation
|
| 109 |
+
|
| 110 |
+
<!-- This section describes the evaluation protocols and provides the results. -->
|
| 111 |
+
|
| 112 |
+
### Testing Data, Factors & Metrics
|
| 113 |
+
|
| 114 |
+
#### Testing Data
|
| 115 |
+
|
| 116 |
+
<!-- This should link to a Dataset Card if possible. -->
|
| 117 |
+
|
| 118 |
+
[More Information Needed]
|
| 119 |
+
|
| 120 |
+
#### Factors
|
| 121 |
+
|
| 122 |
+
<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
|
| 123 |
+
|
| 124 |
+
[More Information Needed]
|
| 125 |
+
|
| 126 |
+
#### Metrics
|
| 127 |
+
|
| 128 |
+
<!-- These are the evaluation metrics being used, ideally with a description of why. -->
|
| 129 |
+
|
| 130 |
+
[More Information Needed]
|
| 131 |
+
|
| 132 |
+
### Results
|
| 133 |
+
|
| 134 |
+
[More Information Needed]
|
| 135 |
+
|
| 136 |
+
#### Summary
|
| 137 |
+
|
| 138 |
+
|
| 139 |
+
|
| 140 |
+
## Model Examination [optional]
|
| 141 |
+
|
| 142 |
+
<!-- Relevant interpretability work for the model goes here -->
|
| 143 |
+
|
| 144 |
+
[More Information Needed]
|
| 145 |
+
|
| 146 |
+
## Environmental Impact
|
| 147 |
+
|
| 148 |
+
<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
|
| 149 |
+
|
| 150 |
+
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
|
| 151 |
+
|
| 152 |
+
- **Hardware Type:** [More Information Needed]
|
| 153 |
+
- **Hours used:** [More Information Needed]
|
| 154 |
+
- **Cloud Provider:** [More Information Needed]
|
| 155 |
+
- **Compute Region:** [More Information Needed]
|
| 156 |
+
- **Carbon Emitted:** [More Information Needed]
|
| 157 |
+
|
| 158 |
+
## Technical Specifications [optional]
|
| 159 |
+
|
| 160 |
+
### Model Architecture and Objective
|
| 161 |
+
|
| 162 |
+
[More Information Needed]
|
| 163 |
+
|
| 164 |
+
### Compute Infrastructure
|
| 165 |
+
|
| 166 |
+
[More Information Needed]
|
| 167 |
+
|
| 168 |
+
#### Hardware
|
| 169 |
+
|
| 170 |
+
[More Information Needed]
|
| 171 |
+
|
| 172 |
+
#### Software
|
| 173 |
+
|
| 174 |
+
[More Information Needed]
|
| 175 |
+
|
| 176 |
+
## Citation [optional]
|
| 177 |
+
|
| 178 |
+
<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
|
| 179 |
+
|
| 180 |
+
**BibTeX:**
|
| 181 |
+
|
| 182 |
+
[More Information Needed]
|
| 183 |
+
|
| 184 |
+
**APA:**
|
| 185 |
+
|
| 186 |
+
[More Information Needed]
|
| 187 |
+
|
| 188 |
+
## Glossary [optional]
|
| 189 |
+
|
| 190 |
+
<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
|
| 191 |
+
|
| 192 |
+
[More Information Needed]
|
| 193 |
+
|
| 194 |
+
## More Information [optional]
|
| 195 |
+
|
| 196 |
+
[More Information Needed]
|
| 197 |
+
|
| 198 |
+
## Model Card Authors [optional]
|
| 199 |
+
|
| 200 |
+
[More Information Needed]
|
| 201 |
+
|
| 202 |
+
## Model Card Contact
|
| 203 |
+
|
| 204 |
+
[More Information Needed]
|
| 205 |
+
### Framework versions
|
| 206 |
+
|
| 207 |
+
- PEFT 0.20.0
|
runs/fixed_internvl/removal/TopDelta8_s73/adapter/adapter_config.json
ADDED
|
@@ -0,0 +1,213 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"alora_invocation_tokens": null,
|
| 3 |
+
"alpha_pattern": {
|
| 4 |
+
"model.language_model.layers.13.mlp.down_proj": 144,
|
| 5 |
+
"model.language_model.layers.13.mlp.gate_proj": 144,
|
| 6 |
+
"model.language_model.layers.13.mlp.up_proj": 144,
|
| 7 |
+
"model.language_model.layers.13.self_attn.k_proj": 144,
|
| 8 |
+
"model.language_model.layers.13.self_attn.o_proj": 144,
|
| 9 |
+
"model.language_model.layers.13.self_attn.q_proj": 144,
|
| 10 |
+
"model.language_model.layers.13.self_attn.v_proj": 144,
|
| 11 |
+
"model.language_model.layers.15.mlp.down_proj": 144,
|
| 12 |
+
"model.language_model.layers.15.mlp.gate_proj": 144,
|
| 13 |
+
"model.language_model.layers.15.mlp.up_proj": 144,
|
| 14 |
+
"model.language_model.layers.15.self_attn.k_proj": 144,
|
| 15 |
+
"model.language_model.layers.15.self_attn.o_proj": 144,
|
| 16 |
+
"model.language_model.layers.15.self_attn.q_proj": 144,
|
| 17 |
+
"model.language_model.layers.15.self_attn.v_proj": 144,
|
| 18 |
+
"model.language_model.layers.17.mlp.down_proj": 144,
|
| 19 |
+
"model.language_model.layers.17.mlp.gate_proj": 144,
|
| 20 |
+
"model.language_model.layers.17.mlp.up_proj": 144,
|
| 21 |
+
"model.language_model.layers.17.self_attn.k_proj": 144,
|
| 22 |
+
"model.language_model.layers.17.self_attn.o_proj": 144,
|
| 23 |
+
"model.language_model.layers.17.self_attn.q_proj": 144,
|
| 24 |
+
"model.language_model.layers.17.self_attn.v_proj": 144,
|
| 25 |
+
"model.language_model.layers.18.mlp.down_proj": 144,
|
| 26 |
+
"model.language_model.layers.18.mlp.gate_proj": 144,
|
| 27 |
+
"model.language_model.layers.18.mlp.up_proj": 144,
|
| 28 |
+
"model.language_model.layers.18.self_attn.k_proj": 144,
|
| 29 |
+
"model.language_model.layers.18.self_attn.o_proj": 144,
|
| 30 |
+
"model.language_model.layers.18.self_attn.q_proj": 144,
|
| 31 |
+
"model.language_model.layers.18.self_attn.v_proj": 144,
|
| 32 |
+
"model.language_model.layers.19.mlp.down_proj": 144,
|
| 33 |
+
"model.language_model.layers.19.mlp.gate_proj": 144,
|
| 34 |
+
"model.language_model.layers.19.mlp.up_proj": 144,
|
| 35 |
+
"model.language_model.layers.19.self_attn.k_proj": 144,
|
| 36 |
+
"model.language_model.layers.19.self_attn.o_proj": 144,
|
| 37 |
+
"model.language_model.layers.19.self_attn.q_proj": 144,
|
| 38 |
+
"model.language_model.layers.19.self_attn.v_proj": 144,
|
| 39 |
+
"model.language_model.layers.21.mlp.down_proj": 144,
|
| 40 |
+
"model.language_model.layers.21.mlp.gate_proj": 144,
|
| 41 |
+
"model.language_model.layers.21.mlp.up_proj": 144,
|
| 42 |
+
"model.language_model.layers.21.self_attn.k_proj": 144,
|
| 43 |
+
"model.language_model.layers.21.self_attn.o_proj": 144,
|
| 44 |
+
"model.language_model.layers.21.self_attn.q_proj": 144,
|
| 45 |
+
"model.language_model.layers.21.self_attn.v_proj": 144,
|
| 46 |
+
"model.language_model.layers.23.mlp.down_proj": 144,
|
| 47 |
+
"model.language_model.layers.23.mlp.gate_proj": 144,
|
| 48 |
+
"model.language_model.layers.23.mlp.up_proj": 144,
|
| 49 |
+
"model.language_model.layers.23.self_attn.k_proj": 144,
|
| 50 |
+
"model.language_model.layers.23.self_attn.o_proj": 144,
|
| 51 |
+
"model.language_model.layers.23.self_attn.q_proj": 144,
|
| 52 |
+
"model.language_model.layers.23.self_attn.v_proj": 144,
|
| 53 |
+
"model.language_model.layers.24.mlp.down_proj": 144,
|
| 54 |
+
"model.language_model.layers.24.mlp.gate_proj": 144,
|
| 55 |
+
"model.language_model.layers.24.mlp.up_proj": 144,
|
| 56 |
+
"model.language_model.layers.24.self_attn.k_proj": 144,
|
| 57 |
+
"model.language_model.layers.24.self_attn.o_proj": 144,
|
| 58 |
+
"model.language_model.layers.24.self_attn.q_proj": 144,
|
| 59 |
+
"model.language_model.layers.24.self_attn.v_proj": 144
|
| 60 |
+
},
|
| 61 |
+
"arrow_config": null,
|
| 62 |
+
"auto_mapping": null,
|
| 63 |
+
"base_model_name_or_path": "/data/OPSD/assets/models/InternVL3_5-8B-HF",
|
| 64 |
+
"bias": "none",
|
| 65 |
+
"corda_config": null,
|
| 66 |
+
"ensure_weight_tying": false,
|
| 67 |
+
"eva_config": null,
|
| 68 |
+
"exclude_modules": null,
|
| 69 |
+
"fan_in_fan_out": false,
|
| 70 |
+
"inference_mode": true,
|
| 71 |
+
"init_lora_weights": true,
|
| 72 |
+
"layer_replication": null,
|
| 73 |
+
"layers_pattern": null,
|
| 74 |
+
"layers_to_transform": null,
|
| 75 |
+
"loftq_config": {},
|
| 76 |
+
"lora_alpha": 32,
|
| 77 |
+
"lora_bias": false,
|
| 78 |
+
"lora_dropout": 0.05,
|
| 79 |
+
"lora_ga_config": null,
|
| 80 |
+
"megatron_config": null,
|
| 81 |
+
"megatron_core": "megatron.core",
|
| 82 |
+
"modules_to_save": null,
|
| 83 |
+
"monteclora_config": null,
|
| 84 |
+
"peft_type": "LORA",
|
| 85 |
+
"peft_version": "0.20.0",
|
| 86 |
+
"qalora_group_size": 16,
|
| 87 |
+
"r": 16,
|
| 88 |
+
"rank_pattern": {
|
| 89 |
+
"model.language_model.layers.13.mlp.down_proj": 72,
|
| 90 |
+
"model.language_model.layers.13.mlp.gate_proj": 72,
|
| 91 |
+
"model.language_model.layers.13.mlp.up_proj": 72,
|
| 92 |
+
"model.language_model.layers.13.self_attn.k_proj": 72,
|
| 93 |
+
"model.language_model.layers.13.self_attn.o_proj": 72,
|
| 94 |
+
"model.language_model.layers.13.self_attn.q_proj": 72,
|
| 95 |
+
"model.language_model.layers.13.self_attn.v_proj": 72,
|
| 96 |
+
"model.language_model.layers.15.mlp.down_proj": 72,
|
| 97 |
+
"model.language_model.layers.15.mlp.gate_proj": 72,
|
| 98 |
+
"model.language_model.layers.15.mlp.up_proj": 72,
|
| 99 |
+
"model.language_model.layers.15.self_attn.k_proj": 72,
|
| 100 |
+
"model.language_model.layers.15.self_attn.o_proj": 72,
|
| 101 |
+
"model.language_model.layers.15.self_attn.q_proj": 72,
|
| 102 |
+
"model.language_model.layers.15.self_attn.v_proj": 72,
|
| 103 |
+
"model.language_model.layers.17.mlp.down_proj": 72,
|
| 104 |
+
"model.language_model.layers.17.mlp.gate_proj": 72,
|
| 105 |
+
"model.language_model.layers.17.mlp.up_proj": 72,
|
| 106 |
+
"model.language_model.layers.17.self_attn.k_proj": 72,
|
| 107 |
+
"model.language_model.layers.17.self_attn.o_proj": 72,
|
| 108 |
+
"model.language_model.layers.17.self_attn.q_proj": 72,
|
| 109 |
+
"model.language_model.layers.17.self_attn.v_proj": 72,
|
| 110 |
+
"model.language_model.layers.18.mlp.down_proj": 72,
|
| 111 |
+
"model.language_model.layers.18.mlp.gate_proj": 72,
|
| 112 |
+
"model.language_model.layers.18.mlp.up_proj": 72,
|
| 113 |
+
"model.language_model.layers.18.self_attn.k_proj": 72,
|
| 114 |
+
"model.language_model.layers.18.self_attn.o_proj": 72,
|
| 115 |
+
"model.language_model.layers.18.self_attn.q_proj": 72,
|
| 116 |
+
"model.language_model.layers.18.self_attn.v_proj": 72,
|
| 117 |
+
"model.language_model.layers.19.mlp.down_proj": 72,
|
| 118 |
+
"model.language_model.layers.19.mlp.gate_proj": 72,
|
| 119 |
+
"model.language_model.layers.19.mlp.up_proj": 72,
|
| 120 |
+
"model.language_model.layers.19.self_attn.k_proj": 72,
|
| 121 |
+
"model.language_model.layers.19.self_attn.o_proj": 72,
|
| 122 |
+
"model.language_model.layers.19.self_attn.q_proj": 72,
|
| 123 |
+
"model.language_model.layers.19.self_attn.v_proj": 72,
|
| 124 |
+
"model.language_model.layers.21.mlp.down_proj": 72,
|
| 125 |
+
"model.language_model.layers.21.mlp.gate_proj": 72,
|
| 126 |
+
"model.language_model.layers.21.mlp.up_proj": 72,
|
| 127 |
+
"model.language_model.layers.21.self_attn.k_proj": 72,
|
| 128 |
+
"model.language_model.layers.21.self_attn.o_proj": 72,
|
| 129 |
+
"model.language_model.layers.21.self_attn.q_proj": 72,
|
| 130 |
+
"model.language_model.layers.21.self_attn.v_proj": 72,
|
| 131 |
+
"model.language_model.layers.23.mlp.down_proj": 72,
|
| 132 |
+
"model.language_model.layers.23.mlp.gate_proj": 72,
|
| 133 |
+
"model.language_model.layers.23.mlp.up_proj": 72,
|
| 134 |
+
"model.language_model.layers.23.self_attn.k_proj": 72,
|
| 135 |
+
"model.language_model.layers.23.self_attn.o_proj": 72,
|
| 136 |
+
"model.language_model.layers.23.self_attn.q_proj": 72,
|
| 137 |
+
"model.language_model.layers.23.self_attn.v_proj": 72,
|
| 138 |
+
"model.language_model.layers.24.mlp.down_proj": 72,
|
| 139 |
+
"model.language_model.layers.24.mlp.gate_proj": 72,
|
| 140 |
+
"model.language_model.layers.24.mlp.up_proj": 72,
|
| 141 |
+
"model.language_model.layers.24.self_attn.k_proj": 72,
|
| 142 |
+
"model.language_model.layers.24.self_attn.o_proj": 72,
|
| 143 |
+
"model.language_model.layers.24.self_attn.q_proj": 72,
|
| 144 |
+
"model.language_model.layers.24.self_attn.v_proj": 72
|
| 145 |
+
},
|
| 146 |
+
"revision": null,
|
| 147 |
+
"target_modules": [
|
| 148 |
+
"model.language_model.layers.19.self_attn.q_proj",
|
| 149 |
+
"model.language_model.layers.19.self_attn.v_proj",
|
| 150 |
+
"model.language_model.layers.24.self_attn.k_proj",
|
| 151 |
+
"model.language_model.layers.24.self_attn.o_proj",
|
| 152 |
+
"model.language_model.layers.23.self_attn.o_proj",
|
| 153 |
+
"model.language_model.layers.15.self_attn.q_proj",
|
| 154 |
+
"model.language_model.layers.17.self_attn.q_proj",
|
| 155 |
+
"model.language_model.layers.21.self_attn.o_proj",
|
| 156 |
+
"model.language_model.layers.23.self_attn.q_proj",
|
| 157 |
+
"model.language_model.layers.15.self_attn.v_proj",
|
| 158 |
+
"model.language_model.layers.13.mlp.gate_proj",
|
| 159 |
+
"model.language_model.layers.24.self_attn.q_proj",
|
| 160 |
+
"model.language_model.layers.15.self_attn.o_proj",
|
| 161 |
+
"model.language_model.layers.23.mlp.gate_proj",
|
| 162 |
+
"model.language_model.layers.18.mlp.gate_proj",
|
| 163 |
+
"model.language_model.layers.15.mlp.gate_proj",
|
| 164 |
+
"model.language_model.layers.19.mlp.gate_proj",
|
| 165 |
+
"model.language_model.layers.24.self_attn.v_proj",
|
| 166 |
+
"model.language_model.layers.21.self_attn.q_proj",
|
| 167 |
+
"model.language_model.layers.13.self_attn.o_proj",
|
| 168 |
+
"model.language_model.layers.17.mlp.up_proj",
|
| 169 |
+
"model.language_model.layers.13.mlp.up_proj",
|
| 170 |
+
"model.language_model.layers.17.mlp.down_proj",
|
| 171 |
+
"model.language_model.layers.17.self_attn.o_proj",
|
| 172 |
+
"model.language_model.layers.23.mlp.up_proj",
|
| 173 |
+
"model.language_model.layers.17.mlp.gate_proj",
|
| 174 |
+
"model.language_model.layers.17.self_attn.k_proj",
|
| 175 |
+
"model.language_model.layers.24.mlp.down_proj",
|
| 176 |
+
"model.language_model.layers.18.mlp.down_proj",
|
| 177 |
+
"model.language_model.layers.13.mlp.down_proj",
|
| 178 |
+
"model.language_model.layers.23.self_attn.k_proj",
|
| 179 |
+
"model.language_model.layers.19.mlp.down_proj",
|
| 180 |
+
"model.language_model.layers.23.mlp.down_proj",
|
| 181 |
+
"model.language_model.layers.24.mlp.gate_proj",
|
| 182 |
+
"model.language_model.layers.21.mlp.gate_proj",
|
| 183 |
+
"model.language_model.layers.23.self_attn.v_proj",
|
| 184 |
+
"model.language_model.layers.18.self_attn.k_proj",
|
| 185 |
+
"model.language_model.layers.17.self_attn.v_proj",
|
| 186 |
+
"model.language_model.layers.19.mlp.up_proj",
|
| 187 |
+
"model.language_model.layers.18.mlp.up_proj",
|
| 188 |
+
"model.language_model.layers.18.self_attn.v_proj",
|
| 189 |
+
"model.language_model.layers.21.self_attn.v_proj",
|
| 190 |
+
"model.language_model.layers.15.mlp.up_proj",
|
| 191 |
+
"model.language_model.layers.13.self_attn.v_proj",
|
| 192 |
+
"model.language_model.layers.18.self_attn.q_proj",
|
| 193 |
+
"model.language_model.layers.24.mlp.up_proj",
|
| 194 |
+
"model.language_model.layers.19.self_attn.o_proj",
|
| 195 |
+
"model.language_model.layers.21.mlp.down_proj",
|
| 196 |
+
"model.language_model.layers.19.self_attn.k_proj",
|
| 197 |
+
"model.language_model.layers.13.self_attn.q_proj",
|
| 198 |
+
"model.language_model.layers.18.self_attn.o_proj",
|
| 199 |
+
"model.language_model.layers.21.mlp.up_proj",
|
| 200 |
+
"model.language_model.layers.13.self_attn.k_proj",
|
| 201 |
+
"model.language_model.layers.21.self_attn.k_proj",
|
| 202 |
+
"model.language_model.layers.15.self_attn.k_proj",
|
| 203 |
+
"model.language_model.layers.15.mlp.down_proj"
|
| 204 |
+
],
|
| 205 |
+
"target_parameters": null,
|
| 206 |
+
"task_type": "CAUSAL_LM",
|
| 207 |
+
"trainable_token_indices": null,
|
| 208 |
+
"use_bdlora": null,
|
| 209 |
+
"use_dora": false,
|
| 210 |
+
"use_qalora": false,
|
| 211 |
+
"use_rslora": false,
|
| 212 |
+
"velora_config": null
|
| 213 |
+
}
|
runs/joint_qwen/joint/seed13/adapter/README.md
ADDED
|
@@ -0,0 +1,207 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
base_model: /data/OPSD/assets/models/Qwen3.5-9B
|
| 3 |
+
library_name: peft
|
| 4 |
+
pipeline_tag: text-generation
|
| 5 |
+
tags:
|
| 6 |
+
- base_model:adapter:/data/OPSD/assets/models/Qwen3.5-9B
|
| 7 |
+
- lora
|
| 8 |
+
- transformers
|
| 9 |
+
---
|
| 10 |
+
|
| 11 |
+
# Model Card for Model ID
|
| 12 |
+
|
| 13 |
+
<!-- Provide a quick summary of what the model is/does. -->
|
| 14 |
+
|
| 15 |
+
|
| 16 |
+
|
| 17 |
+
## Model Details
|
| 18 |
+
|
| 19 |
+
### Model Description
|
| 20 |
+
|
| 21 |
+
<!-- Provide a longer summary of what this model is. -->
|
| 22 |
+
|
| 23 |
+
|
| 24 |
+
|
| 25 |
+
- **Developed by:** [More Information Needed]
|
| 26 |
+
- **Funded by [optional]:** [More Information Needed]
|
| 27 |
+
- **Shared by [optional]:** [More Information Needed]
|
| 28 |
+
- **Model type:** [More Information Needed]
|
| 29 |
+
- **Language(s) (NLP):** [More Information Needed]
|
| 30 |
+
- **License:** [More Information Needed]
|
| 31 |
+
- **Finetuned from model [optional]:** [More Information Needed]
|
| 32 |
+
|
| 33 |
+
### Model Sources [optional]
|
| 34 |
+
|
| 35 |
+
<!-- Provide the basic links for the model. -->
|
| 36 |
+
|
| 37 |
+
- **Repository:** [More Information Needed]
|
| 38 |
+
- **Paper [optional]:** [More Information Needed]
|
| 39 |
+
- **Demo [optional]:** [More Information Needed]
|
| 40 |
+
|
| 41 |
+
## Uses
|
| 42 |
+
|
| 43 |
+
<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
|
| 44 |
+
|
| 45 |
+
### Direct Use
|
| 46 |
+
|
| 47 |
+
<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
|
| 48 |
+
|
| 49 |
+
[More Information Needed]
|
| 50 |
+
|
| 51 |
+
### Downstream Use [optional]
|
| 52 |
+
|
| 53 |
+
<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
|
| 54 |
+
|
| 55 |
+
[More Information Needed]
|
| 56 |
+
|
| 57 |
+
### Out-of-Scope Use
|
| 58 |
+
|
| 59 |
+
<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
|
| 60 |
+
|
| 61 |
+
[More Information Needed]
|
| 62 |
+
|
| 63 |
+
## Bias, Risks, and Limitations
|
| 64 |
+
|
| 65 |
+
<!-- This section is meant to convey both technical and sociotechnical limitations. -->
|
| 66 |
+
|
| 67 |
+
[More Information Needed]
|
| 68 |
+
|
| 69 |
+
### Recommendations
|
| 70 |
+
|
| 71 |
+
<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
|
| 72 |
+
|
| 73 |
+
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
|
| 74 |
+
|
| 75 |
+
## How to Get Started with the Model
|
| 76 |
+
|
| 77 |
+
Use the code below to get started with the model.
|
| 78 |
+
|
| 79 |
+
[More Information Needed]
|
| 80 |
+
|
| 81 |
+
## Training Details
|
| 82 |
+
|
| 83 |
+
### Training Data
|
| 84 |
+
|
| 85 |
+
<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
|
| 86 |
+
|
| 87 |
+
[More Information Needed]
|
| 88 |
+
|
| 89 |
+
### Training Procedure
|
| 90 |
+
|
| 91 |
+
<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
|
| 92 |
+
|
| 93 |
+
#### Preprocessing [optional]
|
| 94 |
+
|
| 95 |
+
[More Information Needed]
|
| 96 |
+
|
| 97 |
+
|
| 98 |
+
#### Training Hyperparameters
|
| 99 |
+
|
| 100 |
+
- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
|
| 101 |
+
|
| 102 |
+
#### Speeds, Sizes, Times [optional]
|
| 103 |
+
|
| 104 |
+
<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
|
| 105 |
+
|
| 106 |
+
[More Information Needed]
|
| 107 |
+
|
| 108 |
+
## Evaluation
|
| 109 |
+
|
| 110 |
+
<!-- This section describes the evaluation protocols and provides the results. -->
|
| 111 |
+
|
| 112 |
+
### Testing Data, Factors & Metrics
|
| 113 |
+
|
| 114 |
+
#### Testing Data
|
| 115 |
+
|
| 116 |
+
<!-- This should link to a Dataset Card if possible. -->
|
| 117 |
+
|
| 118 |
+
[More Information Needed]
|
| 119 |
+
|
| 120 |
+
#### Factors
|
| 121 |
+
|
| 122 |
+
<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
|
| 123 |
+
|
| 124 |
+
[More Information Needed]
|
| 125 |
+
|
| 126 |
+
#### Metrics
|
| 127 |
+
|
| 128 |
+
<!-- These are the evaluation metrics being used, ideally with a description of why. -->
|
| 129 |
+
|
| 130 |
+
[More Information Needed]
|
| 131 |
+
|
| 132 |
+
### Results
|
| 133 |
+
|
| 134 |
+
[More Information Needed]
|
| 135 |
+
|
| 136 |
+
#### Summary
|
| 137 |
+
|
| 138 |
+
|
| 139 |
+
|
| 140 |
+
## Model Examination [optional]
|
| 141 |
+
|
| 142 |
+
<!-- Relevant interpretability work for the model goes here -->
|
| 143 |
+
|
| 144 |
+
[More Information Needed]
|
| 145 |
+
|
| 146 |
+
## Environmental Impact
|
| 147 |
+
|
| 148 |
+
<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
|
| 149 |
+
|
| 150 |
+
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
|
| 151 |
+
|
| 152 |
+
- **Hardware Type:** [More Information Needed]
|
| 153 |
+
- **Hours used:** [More Information Needed]
|
| 154 |
+
- **Cloud Provider:** [More Information Needed]
|
| 155 |
+
- **Compute Region:** [More Information Needed]
|
| 156 |
+
- **Carbon Emitted:** [More Information Needed]
|
| 157 |
+
|
| 158 |
+
## Technical Specifications [optional]
|
| 159 |
+
|
| 160 |
+
### Model Architecture and Objective
|
| 161 |
+
|
| 162 |
+
[More Information Needed]
|
| 163 |
+
|
| 164 |
+
### Compute Infrastructure
|
| 165 |
+
|
| 166 |
+
[More Information Needed]
|
| 167 |
+
|
| 168 |
+
#### Hardware
|
| 169 |
+
|
| 170 |
+
[More Information Needed]
|
| 171 |
+
|
| 172 |
+
#### Software
|
| 173 |
+
|
| 174 |
+
[More Information Needed]
|
| 175 |
+
|
| 176 |
+
## Citation [optional]
|
| 177 |
+
|
| 178 |
+
<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
|
| 179 |
+
|
| 180 |
+
**BibTeX:**
|
| 181 |
+
|
| 182 |
+
[More Information Needed]
|
| 183 |
+
|
| 184 |
+
**APA:**
|
| 185 |
+
|
| 186 |
+
[More Information Needed]
|
| 187 |
+
|
| 188 |
+
## Glossary [optional]
|
| 189 |
+
|
| 190 |
+
<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
|
| 191 |
+
|
| 192 |
+
[More Information Needed]
|
| 193 |
+
|
| 194 |
+
## More Information [optional]
|
| 195 |
+
|
| 196 |
+
[More Information Needed]
|
| 197 |
+
|
| 198 |
+
## Model Card Authors [optional]
|
| 199 |
+
|
| 200 |
+
[More Information Needed]
|
| 201 |
+
|
| 202 |
+
## Model Card Contact
|
| 203 |
+
|
| 204 |
+
[More Information Needed]
|
| 205 |
+
### Framework versions
|
| 206 |
+
|
| 207 |
+
- PEFT 0.20.0
|
runs/joint_qwen/joint/seed13/adapter/adapter_config.json
ADDED
|
@@ -0,0 +1,55 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"alora_invocation_tokens": null,
|
| 3 |
+
"alpha_pattern": {},
|
| 4 |
+
"arrow_config": null,
|
| 5 |
+
"auto_mapping": null,
|
| 6 |
+
"base_model_name_or_path": "/data/OPSD/assets/models/Qwen3.5-9B",
|
| 7 |
+
"bias": "none",
|
| 8 |
+
"corda_config": null,
|
| 9 |
+
"ensure_weight_tying": false,
|
| 10 |
+
"eva_config": null,
|
| 11 |
+
"exclude_modules": null,
|
| 12 |
+
"fan_in_fan_out": false,
|
| 13 |
+
"inference_mode": true,
|
| 14 |
+
"init_lora_weights": true,
|
| 15 |
+
"layer_replication": null,
|
| 16 |
+
"layers_pattern": null,
|
| 17 |
+
"layers_to_transform": null,
|
| 18 |
+
"loftq_config": {},
|
| 19 |
+
"lora_alpha": 32,
|
| 20 |
+
"lora_bias": false,
|
| 21 |
+
"lora_dropout": 0.05,
|
| 22 |
+
"lora_ga_config": null,
|
| 23 |
+
"megatron_config": null,
|
| 24 |
+
"megatron_core": "megatron.core",
|
| 25 |
+
"modules_to_save": null,
|
| 26 |
+
"monteclora_config": null,
|
| 27 |
+
"peft_type": "LORA",
|
| 28 |
+
"peft_version": "0.20.0",
|
| 29 |
+
"qalora_group_size": 16,
|
| 30 |
+
"r": 16,
|
| 31 |
+
"rank_pattern": {},
|
| 32 |
+
"revision": null,
|
| 33 |
+
"target_modules": [
|
| 34 |
+
"k_proj",
|
| 35 |
+
"down_proj",
|
| 36 |
+
"q_proj",
|
| 37 |
+
"in_proj_qkv",
|
| 38 |
+
"out_proj",
|
| 39 |
+
"in_proj_a",
|
| 40 |
+
"o_proj",
|
| 41 |
+
"in_proj_b",
|
| 42 |
+
"v_proj",
|
| 43 |
+
"in_proj_z",
|
| 44 |
+
"up_proj",
|
| 45 |
+
"gate_proj"
|
| 46 |
+
],
|
| 47 |
+
"target_parameters": null,
|
| 48 |
+
"task_type": "CAUSAL_LM",
|
| 49 |
+
"trainable_token_indices": null,
|
| 50 |
+
"use_bdlora": null,
|
| 51 |
+
"use_dora": false,
|
| 52 |
+
"use_qalora": false,
|
| 53 |
+
"use_rslora": false,
|
| 54 |
+
"velora_config": null
|
| 55 |
+
}
|
runs/joint_qwen/joint/seed13/config.json
ADDED
|
@@ -0,0 +1,32 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"seed": 13,
|
| 3 |
+
"model": "/data/OPSD/assets/models/Qwen3.5-9B",
|
| 4 |
+
"backbone": "qwen3_5",
|
| 5 |
+
"train": "/data/OPSD/work/joint/joint.jsonl",
|
| 6 |
+
"records": 1000,
|
| 7 |
+
"steps": 500,
|
| 8 |
+
"steps_run": 500,
|
| 9 |
+
"category_weights": {
|
| 10 |
+
"original": 1.0,
|
| 11 |
+
"removal": 1.0,
|
| 12 |
+
"replacement": 1.0,
|
| 13 |
+
"context": 0.5,
|
| 14 |
+
"no_image": 0.02
|
| 15 |
+
},
|
| 16 |
+
"task_weight": 0.5,
|
| 17 |
+
"phases": [
|
| 18 |
+
{
|
| 19 |
+
"color": "b",
|
| 20 |
+
"count": "b"
|
| 21 |
+
}
|
| 22 |
+
],
|
| 23 |
+
"lora_rank": 16,
|
| 24 |
+
"lr": {
|
| 25 |
+
"peak": 2e-05,
|
| 26 |
+
"warmup": 16,
|
| 27 |
+
"final": 2e-06,
|
| 28 |
+
"decay": "cosine"
|
| 29 |
+
},
|
| 30 |
+
"clip": 1.0,
|
| 31 |
+
"initial_loss_scale": 4.0
|
| 32 |
+
}
|
runs/joint_qwen/joint/seed13/steps.jsonl
ADDED
|
@@ -0,0 +1,500 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{"step": 1, "steps": 500, "lr": 1.25e-06, "loss": 3.527496380562132, "grad_norm": 36.36846923828125, "loss_scale": 4.0, "retries": 0, "seconds": 16.8, "max_memory_gb": 18.64}
|
| 2 |
+
{"step": 2, "steps": 500, "lr": 2.5e-06, "loss": 3.9675237678668727, "grad_norm": 38.71554183959961, "loss_scale": 4.0, "retries": 0, "seconds": 29.1, "max_memory_gb": 18.97}
|
| 3 |
+
{"step": 3, "steps": 500, "lr": 3.7500000000000005e-06, "loss": 4.20239464735443, "grad_norm": 42.41682815551758, "loss_scale": 4.0, "retries": 0, "seconds": 41.4, "max_memory_gb": 18.97}
|
| 4 |
+
{"step": 4, "steps": 500, "lr": 5e-06, "loss": 3.2454022927717734, "grad_norm": 45.11293029785156, "loss_scale": 4.0, "retries": 0, "seconds": 53.7, "max_memory_gb": 18.97}
|
| 5 |
+
{"step": 5, "steps": 500, "lr": 6.25e-06, "loss": 3.360758603296497, "grad_norm": 37.66035842895508, "loss_scale": 4.0, "retries": 0, "seconds": 66.7, "max_memory_gb": 18.97}
|
| 6 |
+
{"step": 6, "steps": 500, "lr": 7.500000000000001e-06, "loss": 3.0306594094092194, "grad_norm": 38.799564361572266, "loss_scale": 4.0, "retries": 0, "seconds": 79.3, "max_memory_gb": 18.97}
|
| 7 |
+
{"step": 7, "steps": 500, "lr": 8.750000000000001e-06, "loss": 2.6599591232158923, "grad_norm": 41.20008087158203, "loss_scale": 4.0, "retries": 0, "seconds": 91.7, "max_memory_gb": 18.97}
|
| 8 |
+
{"step": 8, "steps": 500, "lr": 1e-05, "loss": 1.7166817548092117, "grad_norm": 28.55248260498047, "loss_scale": 4.0, "retries": 0, "seconds": 104.4, "max_memory_gb": 19.06}
|
| 9 |
+
{"step": 9, "steps": 500, "lr": 1.125e-05, "loss": 1.587481755890291, "grad_norm": 21.768869400024414, "loss_scale": 4.0, "retries": 0, "seconds": 116.8, "max_memory_gb": 19.06}
|
| 10 |
+
{"step": 10, "steps": 500, "lr": 1.25e-05, "loss": 1.3697371864479715, "grad_norm": 19.374540328979492, "loss_scale": 4.0, "retries": 0, "seconds": 129.2, "max_memory_gb": 19.06}
|
| 11 |
+
{"step": 11, "steps": 500, "lr": 1.375e-05, "loss": 1.1219726772978902, "grad_norm": 22.103334426879883, "loss_scale": 4.0, "retries": 0, "seconds": 141.6, "max_memory_gb": 19.06}
|
| 12 |
+
{"step": 12, "steps": 500, "lr": 1.5000000000000002e-05, "loss": 1.0237178582359443, "grad_norm": 10.601616859436035, "loss_scale": 4.0, "retries": 0, "seconds": 154.1, "max_memory_gb": 19.06}
|
| 13 |
+
{"step": 13, "steps": 500, "lr": 1.6250000000000002e-05, "loss": 0.5068736434490844, "grad_norm": 4.290500640869141, "loss_scale": 4.0, "retries": 0, "seconds": 166.6, "max_memory_gb": 19.06}
|
| 14 |
+
{"step": 14, "steps": 500, "lr": 1.7500000000000002e-05, "loss": 0.3520327210108834, "grad_norm": 2.7792956829071045, "loss_scale": 4.0, "retries": 0, "seconds": 179.0, "max_memory_gb": 19.06}
|
| 15 |
+
{"step": 15, "steps": 500, "lr": 1.8750000000000002e-05, "loss": 0.30530595621729106, "grad_norm": 2.0023105144500732, "loss_scale": 4.0, "retries": 0, "seconds": 191.6, "max_memory_gb": 19.06}
|
| 16 |
+
{"step": 16, "steps": 500, "lr": 2e-05, "loss": 0.3049729304356416, "grad_norm": 1.734248399734497, "loss_scale": 4.0, "retries": 0, "seconds": 204.0, "max_memory_gb": 19.06}
|
| 17 |
+
{"step": 17, "steps": 500, "lr": 1.999981040799864e-05, "loss": 0.3039912037519653, "grad_norm": 2.5008506774902344, "loss_scale": 4.0, "retries": 0, "seconds": 216.5, "max_memory_gb": 19.06}
|
| 18 |
+
{"step": 18, "steps": 500, "lr": 1.9999241639982356e-05, "loss": 0.37826178107273084, "grad_norm": 2.0857372283935547, "loss_scale": 4.0, "retries": 0, "seconds": 229.2, "max_memory_gb": 19.06}
|
| 19 |
+
{"step": 19, "steps": 500, "lr": 1.9998293719914233e-05, "loss": 0.254635303664889, "grad_norm": 1.5878432989120483, "loss_scale": 4.0, "retries": 0, "seconds": 241.7, "max_memory_gb": 19.06}
|
| 20 |
+
{"step": 20, "steps": 500, "lr": 1.9996966687731615e-05, "loss": 0.3244179988118545, "grad_norm": 2.742936611175537, "loss_scale": 4.0, "retries": 0, "seconds": 254.8, "max_memory_gb": 19.16}
|
| 21 |
+
{"step": 21, "steps": 500, "lr": 1.9995260599344435e-05, "loss": 0.2166251786931587, "grad_norm": 3.462557792663574, "loss_scale": 4.0, "retries": 0, "seconds": 267.2, "max_memory_gb": 19.16}
|
| 22 |
+
{"step": 22, "steps": 500, "lr": 1.9993175526632854e-05, "loss": 0.18078293845544316, "grad_norm": 2.3079752922058105, "loss_scale": 4.0, "retries": 0, "seconds": 279.7, "max_memory_gb": 19.16}
|
| 23 |
+
{"step": 23, "steps": 500, "lr": 1.9990711557444226e-05, "loss": 0.3050905592125756, "grad_norm": 3.2516329288482666, "loss_scale": 4.0, "retries": 0, "seconds": 292.1, "max_memory_gb": 19.16}
|
| 24 |
+
{"step": 24, "steps": 500, "lr": 1.998786879558941e-05, "loss": 0.11885023652840492, "grad_norm": 1.5160839557647705, "loss_scale": 4.0, "retries": 0, "seconds": 304.6, "max_memory_gb": 19.16}
|
| 25 |
+
{"step": 25, "steps": 500, "lr": 1.998464736083838e-05, "loss": 0.11399109604165214, "grad_norm": 1.5161336660385132, "loss_scale": 4.0, "retries": 0, "seconds": 317.2, "max_memory_gb": 19.16}
|
| 26 |
+
{"step": 26, "steps": 500, "lr": 1.99810473889152e-05, "loss": 0.08386831930362282, "grad_norm": 1.217116117477417, "loss_scale": 4.0, "retries": 0, "seconds": 329.6, "max_memory_gb": 19.16}
|
| 27 |
+
{"step": 27, "steps": 500, "lr": 1.9977069031492283e-05, "loss": 0.1305161403037015, "grad_norm": 1.899092674255371, "loss_scale": 4.0, "retries": 0, "seconds": 342.0, "max_memory_gb": 19.16}
|
| 28 |
+
{"step": 28, "steps": 500, "lr": 1.9972712456184023e-05, "loss": 0.04408031433790298, "grad_norm": 1.0132421255111694, "loss_scale": 4.0, "retries": 0, "seconds": 354.5, "max_memory_gb": 19.16}
|
| 29 |
+
{"step": 29, "steps": 500, "lr": 1.996797784653971e-05, "loss": 0.03602224707422217, "grad_norm": 1.210845947265625, "loss_scale": 4.0, "retries": 0, "seconds": 366.9, "max_memory_gb": 19.16}
|
| 30 |
+
{"step": 30, "steps": 500, "lr": 1.9962865402035815e-05, "loss": 0.09783134505927263, "grad_norm": 5.868536949157715, "loss_scale": 4.0, "retries": 0, "seconds": 379.4, "max_memory_gb": 19.16}
|
| 31 |
+
{"step": 31, "steps": 500, "lr": 1.9957375338067585e-05, "loss": 0.014043308745551094, "grad_norm": 0.316273033618927, "loss_scale": 4.0, "retries": 0, "seconds": 392.0, "max_memory_gb": 19.16}
|
| 32 |
+
{"step": 32, "steps": 500, "lr": 1.995150788593995e-05, "loss": 0.029058716546800988, "grad_norm": 7.240785121917725, "loss_scale": 4.0, "retries": 0, "seconds": 404.8, "max_memory_gb": 19.16}
|
| 33 |
+
{"step": 33, "steps": 500, "lr": 1.9945263292857805e-05, "loss": 0.12925056826679793, "grad_norm": 8.265257835388184, "loss_scale": 4.0, "retries": 0, "seconds": 417.0, "max_memory_gb": 19.16}
|
| 34 |
+
{"step": 34, "steps": 500, "lr": 1.9938641821915566e-05, "loss": 0.06246603375207502, "grad_norm": 3.9257771968841553, "loss_scale": 4.0, "retries": 0, "seconds": 429.4, "max_memory_gb": 19.16}
|
| 35 |
+
{"step": 35, "steps": 500, "lr": 1.993164375208611e-05, "loss": 0.01612100780766923, "grad_norm": 0.669647753238678, "loss_scale": 4.0, "retries": 0, "seconds": 442.0, "max_memory_gb": 19.16}
|
| 36 |
+
{"step": 36, "steps": 500, "lr": 1.9924269378209e-05, "loss": 0.09480572261724197, "grad_norm": 4.396464824676514, "loss_scale": 4.0, "retries": 0, "seconds": 454.4, "max_memory_gb": 19.16}
|
| 37 |
+
{"step": 37, "steps": 500, "lr": 1.9916519010978086e-05, "loss": 0.25387514254147064, "grad_norm": 10.213497161865234, "loss_scale": 4.0, "retries": 0, "seconds": 466.9, "max_memory_gb": 19.16}
|
| 38 |
+
{"step": 38, "steps": 500, "lr": 1.9908392976928396e-05, "loss": 0.010510053807427705, "grad_norm": 0.4610890746116638, "loss_scale": 4.0, "retries": 0, "seconds": 479.6, "max_memory_gb": 19.16}
|
| 39 |
+
{"step": 39, "steps": 500, "lr": 1.9899891618422388e-05, "loss": 0.008427637090293996, "grad_norm": 0.3024575710296631, "loss_scale": 4.0, "retries": 0, "seconds": 492.1, "max_memory_gb": 19.16}
|
| 40 |
+
{"step": 40, "steps": 500, "lr": 1.989101529363553e-05, "loss": 0.06668798846815414, "grad_norm": 5.036306858062744, "loss_scale": 4.0, "retries": 0, "seconds": 504.5, "max_memory_gb": 19.16}
|
| 41 |
+
{"step": 41, "steps": 500, "lr": 1.9881764376541183e-05, "loss": 0.006549929072662053, "grad_norm": 0.3919302523136139, "loss_scale": 4.0, "retries": 0, "seconds": 516.9, "max_memory_gb": 19.16}
|
| 42 |
+
{"step": 42, "steps": 500, "lr": 1.9872139256894883e-05, "loss": 0.005223767450739946, "grad_norm": 0.33194223046302795, "loss_scale": 4.0, "retries": 0, "seconds": 529.3, "max_memory_gb": 19.16}
|
| 43 |
+
{"step": 43, "steps": 500, "lr": 1.98621403402179e-05, "loss": 0.00774179907569884, "grad_norm": 0.5452665090560913, "loss_scale": 4.0, "retries": 0, "seconds": 541.8, "max_memory_gb": 19.16}
|
| 44 |
+
{"step": 44, "steps": 500, "lr": 1.9851768047780144e-05, "loss": 0.0032809961298260696, "grad_norm": 0.12935495376586914, "loss_scale": 4.0, "retries": 0, "seconds": 554.6, "max_memory_gb": 19.16}
|
| 45 |
+
{"step": 45, "steps": 500, "lr": 1.9841022816582437e-05, "loss": 0.021723836252569774, "grad_norm": 1.7656503915786743, "loss_scale": 4.0, "retries": 0, "seconds": 567.0, "max_memory_gb": 19.16}
|
| 46 |
+
{"step": 46, "steps": 500, "lr": 1.9829905099338088e-05, "loss": 0.006290678451452178, "grad_norm": 0.7977901697158813, "loss_scale": 4.0, "retries": 0, "seconds": 579.4, "max_memory_gb": 19.16}
|
| 47 |
+
{"step": 47, "steps": 500, "lr": 1.9818415364453816e-05, "loss": 0.002248685290810086, "grad_norm": 0.10991512984037399, "loss_scale": 4.0, "retries": 0, "seconds": 591.7, "max_memory_gb": 19.16}
|
| 48 |
+
{"step": 48, "steps": 500, "lr": 1.9806554096010038e-05, "loss": 0.05313860339340046, "grad_norm": 3.98486590385437, "loss_scale": 4.0, "retries": 0, "seconds": 604.3, "max_memory_gb": 19.16}
|
| 49 |
+
{"step": 49, "steps": 500, "lr": 1.9794321793740438e-05, "loss": 0.007800749923386651, "grad_norm": 1.1651071310043335, "loss_scale": 4.0, "retries": 0, "seconds": 616.7, "max_memory_gb": 19.16}
|
| 50 |
+
{"step": 50, "steps": 500, "lr": 1.9781718973010948e-05, "loss": 0.14943012156394597, "grad_norm": 9.225044250488281, "loss_scale": 4.0, "retries": 0, "seconds": 629.0, "max_memory_gb": 19.16}
|
| 51 |
+
{"step": 51, "steps": 500, "lr": 1.9768746164798014e-05, "loss": 0.001463294484092356, "grad_norm": 0.18439994752407074, "loss_scale": 4.0, "retries": 0, "seconds": 641.3, "max_memory_gb": 19.16}
|
| 52 |
+
{"step": 52, "steps": 500, "lr": 1.9755403915666227e-05, "loss": 0.0025658339515954053, "grad_norm": 0.5571502447128296, "loss_scale": 4.0, "retries": 0, "seconds": 653.7, "max_memory_gb": 19.16}
|
| 53 |
+
{"step": 53, "steps": 500, "lr": 1.9741692787745306e-05, "loss": 0.00010200240485082966, "grad_norm": 0.0050964802503585815, "loss_scale": 4.0, "retries": 0, "seconds": 666.0, "max_memory_gb": 19.16}
|
| 54 |
+
{"step": 54, "steps": 500, "lr": 1.9727613358706398e-05, "loss": 0.00015914406946005886, "grad_norm": 0.011179578490555286, "loss_scale": 4.0, "retries": 0, "seconds": 678.0, "max_memory_gb": 19.16}
|
| 55 |
+
{"step": 55, "steps": 500, "lr": 1.9713166221737755e-05, "loss": 0.10317599579028718, "grad_norm": 6.370945930480957, "loss_scale": 4.0, "retries": 0, "seconds": 689.9, "max_memory_gb": 19.16}
|
| 56 |
+
{"step": 56, "steps": 500, "lr": 1.9698351985519734e-05, "loss": 0.2427050227022805, "grad_norm": 21.063703536987305, "loss_scale": 4.0, "retries": 0, "seconds": 702.2, "max_memory_gb": 19.16}
|
| 57 |
+
{"step": 57, "steps": 500, "lr": 1.968317127419916e-05, "loss": 0.001818060276855766, "grad_norm": 0.68830806016922, "loss_scale": 4.0, "retries": 0, "seconds": 714.4, "max_memory_gb": 19.16}
|
| 58 |
+
{"step": 58, "steps": 500, "lr": 1.9667624727363018e-05, "loss": 0.0006452485265488018, "grad_norm": 0.0599467009305954, "loss_scale": 4.0, "retries": 0, "seconds": 726.8, "max_memory_gb": 19.16}
|
| 59 |
+
{"step": 59, "steps": 500, "lr": 1.9651713000011512e-05, "loss": 0.0010906124549631036, "grad_norm": 0.1290503591299057, "loss_scale": 4.0, "retries": 0, "seconds": 739.4, "max_memory_gb": 19.16}
|
| 60 |
+
{"step": 60, "steps": 500, "lr": 1.9635436762530477e-05, "loss": 0.035581621336102764, "grad_norm": 2.0382087230682373, "loss_scale": 4.0, "retries": 0, "seconds": 751.8, "max_memory_gb": 19.16}
|
| 61 |
+
{"step": 61, "steps": 500, "lr": 1.9618796700663118e-05, "loss": 0.001634993273814122, "grad_norm": 0.2773577868938446, "loss_scale": 4.0, "retries": 0, "seconds": 764.1, "max_memory_gb": 19.16}
|
| 62 |
+
{"step": 62, "steps": 500, "lr": 1.960179351548113e-05, "loss": 0.042601123529408376, "grad_norm": 6.1665849685668945, "loss_scale": 4.0, "retries": 0, "seconds": 776.3, "max_memory_gb": 19.16}
|
| 63 |
+
{"step": 63, "steps": 500, "lr": 1.958442792335516e-05, "loss": 6.2427532941036e-05, "grad_norm": 0.004212655127048492, "loss_scale": 4.0, "retries": 0, "seconds": 788.6, "max_memory_gb": 19.16}
|
| 64 |
+
{"step": 64, "steps": 500, "lr": 1.956670065592463e-05, "loss": 0.000818483779734339, "grad_norm": 0.10766290128231049, "loss_scale": 4.0, "retries": 0, "seconds": 801.2, "max_memory_gb": 19.16}
|
| 65 |
+
{"step": 65, "steps": 500, "lr": 1.9548612460066884e-05, "loss": 0.00017786543100907116, "grad_norm": 0.013373848050832748, "loss_scale": 4.0, "retries": 0, "seconds": 813.5, "max_memory_gb": 19.16}
|
| 66 |
+
{"step": 66, "steps": 500, "lr": 1.9530164097865766e-05, "loss": 0.2314999919610366, "grad_norm": 16.01551055908203, "loss_scale": 4.0, "retries": 0, "seconds": 826.1, "max_memory_gb": 19.16}
|
| 67 |
+
{"step": 67, "steps": 500, "lr": 1.9511356346579475e-05, "loss": 0.08416503852416764, "grad_norm": 19.387128829956055, "loss_scale": 4.0, "retries": 0, "seconds": 838.4, "max_memory_gb": 19.16}
|
| 68 |
+
{"step": 68, "steps": 500, "lr": 1.949218999860784e-05, "loss": 0.00022383024854313115, "grad_norm": 0.05139235779643059, "loss_scale": 4.0, "retries": 0, "seconds": 850.9, "max_memory_gb": 19.16}
|
| 69 |
+
{"step": 69, "steps": 500, "lr": 1.9472665861458913e-05, "loss": 0.0005914067148550958, "grad_norm": 0.2342938631772995, "loss_scale": 4.0, "retries": 0, "seconds": 863.3, "max_memory_gb": 19.16}
|
| 70 |
+
{"step": 70, "steps": 500, "lr": 1.9452784757714977e-05, "loss": 0.00010400024759023422, "grad_norm": 0.0076057808473706245, "loss_scale": 4.0, "retries": 0, "seconds": 875.7, "max_memory_gb": 19.16}
|
| 71 |
+
{"step": 71, "steps": 500, "lr": 1.9432547524997857e-05, "loss": 0.00029045234892002077, "grad_norm": 0.024913150817155838, "loss_scale": 4.0, "retries": 0, "seconds": 888.0, "max_memory_gb": 19.16}
|
| 72 |
+
{"step": 72, "steps": 500, "lr": 1.941195501593366e-05, "loss": 0.000858411874473704, "grad_norm": 0.20690563321113586, "loss_scale": 4.0, "retries": 0, "seconds": 900.4, "max_memory_gb": 19.16}
|
| 73 |
+
{"step": 73, "steps": 500, "lr": 1.939100809811683e-05, "loss": 0.004837612197117122, "grad_norm": 0.39705777168273926, "loss_scale": 4.0, "retries": 0, "seconds": 912.9, "max_memory_gb": 19.16}
|
| 74 |
+
{"step": 74, "steps": 500, "lr": 1.93697076540736e-05, "loss": 6.957085640573786e-05, "grad_norm": 0.005460440646857023, "loss_scale": 4.0, "retries": 0, "seconds": 925.5, "max_memory_gb": 19.16}
|
| 75 |
+
{"step": 75, "steps": 500, "lr": 1.934805458122483e-05, "loss": 0.264593126234806, "grad_norm": 11.75994873046875, "loss_scale": 4.0, "retries": 0, "seconds": 938.0, "max_memory_gb": 19.16}
|
| 76 |
+
{"step": 76, "steps": 500, "lr": 1.9326049791848156e-05, "loss": 0.02168334845718967, "grad_norm": 2.744661331176758, "loss_scale": 4.0, "retries": 0, "seconds": 950.7, "max_memory_gb": 19.16}
|
| 77 |
+
{"step": 77, "steps": 500, "lr": 1.9303694213039598e-05, "loss": 0.00014832096617955028, "grad_norm": 0.012613624334335327, "loss_scale": 4.0, "retries": 0, "seconds": 962.9, "max_memory_gb": 19.16}
|
| 78 |
+
{"step": 78, "steps": 500, "lr": 1.9280988786674468e-05, "loss": 0.015629164540004646, "grad_norm": 0.9370248317718506, "loss_scale": 4.0, "retries": 0, "seconds": 975.1, "max_memory_gb": 19.16}
|
| 79 |
+
{"step": 79, "steps": 500, "lr": 1.925793446936771e-05, "loss": 0.003826566603964905, "grad_norm": 0.5274428129196167, "loss_scale": 4.0, "retries": 0, "seconds": 987.4, "max_memory_gb": 19.16}
|
| 80 |
+
{"step": 80, "steps": 500, "lr": 1.923453223243358e-05, "loss": 0.00012217683902445, "grad_norm": 0.007819321937859058, "loss_scale": 4.0, "retries": 0, "seconds": 999.7, "max_memory_gb": 19.16}
|
| 81 |
+
{"step": 81, "steps": 500, "lr": 1.9210783061844728e-05, "loss": 0.36907780623816805, "grad_norm": 9.345929145812988, "loss_scale": 4.0, "retries": 0, "seconds": 1012.5, "max_memory_gb": 19.16}
|
| 82 |
+
{"step": 82, "steps": 500, "lr": 1.9186687958190668e-05, "loss": 0.0014765047125176913, "grad_norm": 0.17978636920452118, "loss_scale": 4.0, "retries": 0, "seconds": 1024.3, "max_memory_gb": 19.16}
|
| 83 |
+
{"step": 83, "steps": 500, "lr": 1.91622479366356e-05, "loss": 0.0002061702824331329, "grad_norm": 0.042785558849573135, "loss_scale": 4.0, "retries": 0, "seconds": 1036.8, "max_memory_gb": 19.16}
|
| 84 |
+
{"step": 84, "steps": 500, "lr": 1.9137464026875653e-05, "loss": 0.025883445182894326, "grad_norm": 1.1832311153411865, "loss_scale": 4.0, "retries": 0, "seconds": 1049.1, "max_memory_gb": 19.16}
|
| 85 |
+
{"step": 85, "steps": 500, "lr": 1.9112337273095516e-05, "loss": 0.01780707685907792, "grad_norm": 3.5983967781066895, "loss_scale": 4.0, "retries": 0, "seconds": 1061.6, "max_memory_gb": 19.16}
|
| 86 |
+
{"step": 86, "steps": 500, "lr": 1.908686873392441e-05, "loss": 8.498727872797606e-05, "grad_norm": 0.005195921286940575, "loss_scale": 4.0, "retries": 0, "seconds": 1074.4, "max_memory_gb": 19.16}
|
| 87 |
+
{"step": 87, "steps": 500, "lr": 1.9061059482391522e-05, "loss": 0.8909262607159855, "grad_norm": 13.68019962310791, "loss_scale": 4.0, "retries": 0, "seconds": 1086.8, "max_memory_gb": 19.16}
|
| 88 |
+
{"step": 88, "steps": 500, "lr": 1.9034910605880777e-05, "loss": 9.329573772683244e-05, "grad_norm": 0.008153255097568035, "loss_scale": 4.0, "retries": 0, "seconds": 1099.2, "max_memory_gb": 19.16}
|
| 89 |
+
{"step": 89, "steps": 500, "lr": 1.9008423206085016e-05, "loss": 5.777008242123896e-05, "grad_norm": 0.004449270199984312, "loss_scale": 4.0, "retries": 0, "seconds": 1111.6, "max_memory_gb": 19.16}
|
| 90 |
+
{"step": 90, "steps": 500, "lr": 1.898159839895961e-05, "loss": 0.009160765496977903, "grad_norm": 0.5215350389480591, "loss_scale": 4.0, "retries": 0, "seconds": 1123.9, "max_memory_gb": 19.16}
|
| 91 |
+
{"step": 91, "steps": 500, "lr": 1.895443731467542e-05, "loss": 4.0848670886232824e-05, "grad_norm": 0.002664630301296711, "loss_scale": 4.0, "retries": 0, "seconds": 1136.5, "max_memory_gb": 19.16}
|
| 92 |
+
{"step": 92, "steps": 500, "lr": 1.892694109757118e-05, "loss": 0.03332172954479519, "grad_norm": 2.976327896118164, "loss_scale": 4.0, "retries": 0, "seconds": 1148.8, "max_memory_gb": 19.16}
|
| 93 |
+
{"step": 93, "steps": 500, "lr": 1.8899110906105304e-05, "loss": 0.001635130369077264, "grad_norm": 0.16478246450424194, "loss_scale": 4.0, "retries": 0, "seconds": 1161.3, "max_memory_gb": 19.16}
|
| 94 |
+
{"step": 94, "steps": 500, "lr": 1.887094791280705e-05, "loss": 7.042513804704372e-05, "grad_norm": 0.00476272264495492, "loss_scale": 4.0, "retries": 0, "seconds": 1173.7, "max_memory_gb": 19.16}
|
| 95 |
+
{"step": 95, "steps": 500, "lr": 1.8842453304227153e-05, "loss": 0.006642128021347237, "grad_norm": 0.5051144361495972, "loss_scale": 4.0, "retries": 0, "seconds": 1186.1, "max_memory_gb": 19.16}
|
| 96 |
+
{"step": 96, "steps": 500, "lr": 1.8813628280887796e-05, "loss": 0.2825600585493581, "grad_norm": 10.184247970581055, "loss_scale": 4.0, "retries": 0, "seconds": 1198.4, "max_memory_gb": 19.16}
|
| 97 |
+
{"step": 97, "steps": 500, "lr": 1.8784474057232066e-05, "loss": 0.019805969205219372, "grad_norm": 2.299154758453369, "loss_scale": 4.0, "retries": 0, "seconds": 1210.8, "max_memory_gb": 19.16}
|
| 98 |
+
{"step": 98, "steps": 500, "lr": 1.8754991861572767e-05, "loss": 0.0006945716083641495, "grad_norm": 0.06618080288171768, "loss_scale": 4.0, "retries": 0, "seconds": 1223.3, "max_memory_gb": 19.16}
|
| 99 |
+
{"step": 99, "steps": 500, "lr": 1.872518293604067e-05, "loss": 0.09239934947897734, "grad_norm": 4.045955181121826, "loss_scale": 4.0, "retries": 0, "seconds": 1235.4, "max_memory_gb": 19.16}
|
| 100 |
+
{"step": 100, "steps": 500, "lr": 1.8695048536532184e-05, "loss": 0.9613017473651397, "grad_norm": 24.49225425720215, "loss_scale": 4.0, "retries": 0, "seconds": 1247.8, "max_memory_gb": 19.16}
|
| 101 |
+
{"step": 101, "steps": 500, "lr": 1.8664589932656442e-05, "loss": 0.5301998903712901, "grad_norm": 6.455207824707031, "loss_scale": 4.0, "retries": 0, "seconds": 1260.1, "max_memory_gb": 19.16}
|
| 102 |
+
{"step": 102, "steps": 500, "lr": 1.8633808407681826e-05, "loss": 0.31073298579163255, "grad_norm": 6.747021675109863, "loss_scale": 4.0, "retries": 0, "seconds": 1272.6, "max_memory_gb": 19.16}
|
| 103 |
+
{"step": 103, "steps": 500, "lr": 1.8602705258481866e-05, "loss": 0.016869631363859904, "grad_norm": 0.9995926022529602, "loss_scale": 4.0, "retries": 0, "seconds": 1285.1, "max_memory_gb": 19.16}
|
| 104 |
+
{"step": 104, "steps": 500, "lr": 1.8571281795480632e-05, "loss": 0.002425909457604741, "grad_norm": 0.1614343374967575, "loss_scale": 4.0, "retries": 0, "seconds": 1297.5, "max_memory_gb": 19.16}
|
| 105 |
+
{"step": 105, "steps": 500, "lr": 1.8539539342597513e-05, "loss": 0.0027812170290298414, "grad_norm": 0.278399258852005, "loss_scale": 4.0, "retries": 0, "seconds": 1309.8, "max_memory_gb": 19.16}
|
| 106 |
+
{"step": 106, "steps": 500, "lr": 1.8507479237191432e-05, "loss": 0.00153073258485909, "grad_norm": 0.11044581979513168, "loss_scale": 4.0, "retries": 0, "seconds": 1322.2, "max_memory_gb": 19.16}
|
| 107 |
+
{"step": 107, "steps": 500, "lr": 1.847510283000451e-05, "loss": 0.0005271214524317326, "grad_norm": 0.11155296117067337, "loss_scale": 4.0, "retries": 0, "seconds": 1334.6, "max_memory_gb": 19.16}
|
| 108 |
+
{"step": 108, "steps": 500, "lr": 1.844241148510516e-05, "loss": 0.10709217935162875, "grad_norm": 6.845663070678711, "loss_scale": 4.0, "retries": 0, "seconds": 1346.7, "max_memory_gb": 19.16}
|
| 109 |
+
{"step": 109, "steps": 500, "lr": 1.84094065798306e-05, "loss": 0.02412935699431852, "grad_norm": 2.2956361770629883, "loss_scale": 4.0, "retries": 0, "seconds": 1358.8, "max_memory_gb": 19.16}
|
| 110 |
+
{"step": 110, "steps": 500, "lr": 1.8376089504728846e-05, "loss": 0.009672037263271704, "grad_norm": 0.9747887253761292, "loss_scale": 4.0, "retries": 0, "seconds": 1371.2, "max_memory_gb": 19.16}
|
| 111 |
+
{"step": 111, "steps": 500, "lr": 1.8342461663500107e-05, "loss": 0.5968284696147377, "grad_norm": 21.040361404418945, "loss_scale": 4.0, "retries": 0, "seconds": 1383.7, "max_memory_gb": 19.16}
|
| 112 |
+
{"step": 112, "steps": 500, "lr": 1.8308524472937653e-05, "loss": 0.11660383938251136, "grad_norm": 4.223559379577637, "loss_scale": 4.0, "retries": 0, "seconds": 1396.0, "max_memory_gb": 19.16}
|
| 113 |
+
{"step": 113, "steps": 500, "lr": 1.827427936286812e-05, "loss": 0.0005408374325862104, "grad_norm": 0.032297175377607346, "loss_scale": 4.0, "retries": 0, "seconds": 1408.5, "max_memory_gb": 19.16}
|
| 114 |
+
{"step": 114, "steps": 500, "lr": 1.8239727776091283e-05, "loss": 0.034862322352919056, "grad_norm": 1.977813959121704, "loss_scale": 4.0, "retries": 0, "seconds": 1420.9, "max_memory_gb": 19.16}
|
| 115 |
+
{"step": 115, "steps": 500, "lr": 1.8204871168319246e-05, "loss": 0.09453362477925384, "grad_norm": 5.641546249389648, "loss_scale": 4.0, "retries": 0, "seconds": 1433.7, "max_memory_gb": 19.16}
|
| 116 |
+
{"step": 116, "steps": 500, "lr": 1.8169711008115124e-05, "loss": 0.00025854522868616993, "grad_norm": 0.02427092380821705, "loss_scale": 4.0, "retries": 0, "seconds": 1446.1, "max_memory_gb": 19.16}
|
| 117 |
+
{"step": 117, "steps": 500, "lr": 1.813424877683118e-05, "loss": 0.4036576999980174, "grad_norm": 13.129405975341797, "loss_scale": 4.0, "retries": 0, "seconds": 1458.6, "max_memory_gb": 19.16}
|
| 118 |
+
{"step": 118, "steps": 500, "lr": 1.8098485968546385e-05, "loss": 0.00020578980269644637, "grad_norm": 0.012828411534428596, "loss_scale": 4.0, "retries": 0, "seconds": 1470.4, "max_memory_gb": 19.16}
|
| 119 |
+
{"step": 119, "steps": 500, "lr": 1.806242409000349e-05, "loss": 0.00010604608645404701, "grad_norm": 0.007631337270140648, "loss_scale": 4.0, "retries": 0, "seconds": 1483.1, "max_memory_gb": 19.16}
|
| 120 |
+
{"step": 120, "steps": 500, "lr": 1.8026064660545554e-05, "loss": 0.003580261752663461, "grad_norm": 0.3603745996952057, "loss_scale": 4.0, "retries": 0, "seconds": 1495.3, "max_memory_gb": 19.16}
|
| 121 |
+
{"step": 121, "steps": 500, "lr": 1.7989409212051905e-05, "loss": 0.0003285496007598774, "grad_norm": 0.022533323615789413, "loss_scale": 4.0, "retries": 0, "seconds": 1507.5, "max_memory_gb": 19.16}
|
| 122 |
+
{"step": 122, "steps": 500, "lr": 1.7952459288873623e-05, "loss": 0.00565324026889158, "grad_norm": 0.9033071994781494, "loss_scale": 4.0, "retries": 0, "seconds": 1520.1, "max_memory_gb": 19.16}
|
| 123 |
+
{"step": 123, "steps": 500, "lr": 1.7915216447768455e-05, "loss": 0.002893028445460547, "grad_norm": 0.18865299224853516, "loss_scale": 4.0, "retries": 0, "seconds": 1532.0, "max_memory_gb": 19.16}
|
| 124 |
+
{"step": 124, "steps": 500, "lr": 1.7877682257835244e-05, "loss": 0.030379845713588472, "grad_norm": 1.7791965007781982, "loss_scale": 4.0, "retries": 0, "seconds": 1544.4, "max_memory_gb": 19.16}
|
| 125 |
+
{"step": 125, "steps": 500, "lr": 1.783985830044781e-05, "loss": 0.00448508849103746, "grad_norm": 0.3061215877532959, "loss_scale": 4.0, "retries": 0, "seconds": 1556.5, "max_memory_gb": 19.16}
|
| 126 |
+
{"step": 126, "steps": 500, "lr": 1.7801746169188328e-05, "loss": 0.037081623776895285, "grad_norm": 1.228693962097168, "loss_scale": 4.0, "retries": 0, "seconds": 1568.8, "max_memory_gb": 19.16}
|
| 127 |
+
{"step": 127, "steps": 500, "lr": 1.7763347469780178e-05, "loss": 0.0018295933200508073, "grad_norm": 0.35339510440826416, "loss_scale": 4.0, "retries": 0, "seconds": 1581.2, "max_memory_gb": 19.16}
|
| 128 |
+
{"step": 128, "steps": 500, "lr": 1.772466382002032e-05, "loss": 0.00010205012950398836, "grad_norm": 0.006306814029812813, "loss_scale": 4.0, "retries": 0, "seconds": 1593.9, "max_memory_gb": 19.16}
|
| 129 |
+
{"step": 129, "steps": 500, "lr": 1.7685696849711094e-05, "loss": 0.1449901456498921, "grad_norm": 3.4136414527893066, "loss_scale": 4.0, "retries": 0, "seconds": 1606.3, "max_memory_gb": 19.16}
|
| 130 |
+
{"step": 130, "steps": 500, "lr": 1.7646448200591597e-05, "loss": 6.3141003496722e-05, "grad_norm": 0.002934962511062622, "loss_scale": 4.0, "retries": 0, "seconds": 1618.7, "max_memory_gb": 19.16}
|
| 131 |
+
{"step": 131, "steps": 500, "lr": 1.760691952626847e-05, "loss": 0.2654458563826979, "grad_norm": 5.056732654571533, "loss_scale": 4.0, "retries": 0, "seconds": 1631.6, "max_memory_gb": 19.16}
|
| 132 |
+
{"step": 132, "steps": 500, "lr": 1.756711249214627e-05, "loss": 8.58520309944569e-05, "grad_norm": 0.004926701541990042, "loss_scale": 4.0, "retries": 0, "seconds": 1643.9, "max_memory_gb": 19.16}
|
| 133 |
+
{"step": 133, "steps": 500, "lr": 1.752702877535728e-05, "loss": 0.00016887189806621708, "grad_norm": 0.008062723092734814, "loss_scale": 4.0, "retries": 0, "seconds": 1656.5, "max_memory_gb": 19.16}
|
| 134 |
+
{"step": 134, "steps": 500, "lr": 1.7486670064690848e-05, "loss": 0.0004409958185782805, "grad_norm": 0.034226976335048676, "loss_scale": 4.0, "retries": 0, "seconds": 1668.9, "max_memory_gb": 19.16}
|
| 135 |
+
{"step": 135, "steps": 500, "lr": 1.7446038060522255e-05, "loss": 0.0032793980371695597, "grad_norm": 0.2115081548690796, "loss_scale": 4.0, "retries": 0, "seconds": 1681.2, "max_memory_gb": 19.16}
|
| 136 |
+
{"step": 136, "steps": 500, "lr": 1.7405134474741042e-05, "loss": 0.0010278168622696949, "grad_norm": 0.060169827193021774, "loss_scale": 4.0, "retries": 0, "seconds": 1693.6, "max_memory_gb": 19.16}
|
| 137 |
+
{"step": 137, "steps": 500, "lr": 1.736396103067893e-05, "loss": 0.38725867100364236, "grad_norm": 5.044195175170898, "loss_scale": 4.0, "retries": 0, "seconds": 1705.9, "max_memory_gb": 19.16}
|
| 138 |
+
{"step": 138, "steps": 500, "lr": 1.732251946303717e-05, "loss": 0.003622783078310735, "grad_norm": 0.21687503159046173, "loss_scale": 4.0, "retries": 0, "seconds": 1718.3, "max_memory_gb": 19.16}
|
| 139 |
+
{"step": 139, "steps": 500, "lr": 1.728081151781349e-05, "loss": 0.0027148553078497825, "grad_norm": 0.16228632628917694, "loss_scale": 4.0, "retries": 0, "seconds": 1730.7, "max_memory_gb": 19.16}
|
| 140 |
+
{"step": 140, "steps": 500, "lr": 1.7238838952228502e-05, "loss": 0.03382253962660219, "grad_norm": 1.7382317781448364, "loss_scale": 4.0, "retries": 0, "seconds": 1743.1, "max_memory_gb": 19.16}
|
| 141 |
+
{"step": 141, "steps": 500, "lr": 1.719660353465171e-05, "loss": 0.09434818950043161, "grad_norm": 4.4147491455078125, "loss_scale": 4.0, "retries": 0, "seconds": 1755.4, "max_memory_gb": 19.16}
|
| 142 |
+
{"step": 142, "steps": 500, "lr": 1.7154107044526965e-05, "loss": 0.0031596460708336535, "grad_norm": 0.13152563571929932, "loss_scale": 4.0, "retries": 0, "seconds": 1767.6, "max_memory_gb": 19.16}
|
| 143 |
+
{"step": 143, "steps": 500, "lr": 1.7111351272297514e-05, "loss": 0.0006553165860171104, "grad_norm": 0.04895675927400589, "loss_scale": 4.0, "retries": 0, "seconds": 1780.0, "max_memory_gb": 19.16}
|
| 144 |
+
{"step": 144, "steps": 500, "lr": 1.7068338019330564e-05, "loss": 0.0005991080363607828, "grad_norm": 0.02994554117321968, "loss_scale": 4.0, "retries": 0, "seconds": 1792.3, "max_memory_gb": 19.16}
|
| 145 |
+
{"step": 145, "steps": 500, "lr": 1.702506909784138e-05, "loss": 0.00046328419753391165, "grad_norm": 0.03885841369628906, "loss_scale": 4.0, "retries": 0, "seconds": 1804.7, "max_memory_gb": 19.16}
|
| 146 |
+
{"step": 146, "steps": 500, "lr": 1.6981546330816953e-05, "loss": 0.00014676804725736886, "grad_norm": 0.006926890462636948, "loss_scale": 4.0, "retries": 0, "seconds": 1816.9, "max_memory_gb": 19.16}
|
| 147 |
+
{"step": 147, "steps": 500, "lr": 1.6937771551939167e-05, "loss": 0.00011207978316592332, "grad_norm": 0.008427276276051998, "loss_scale": 4.0, "retries": 0, "seconds": 1829.3, "max_memory_gb": 19.16}
|
| 148 |
+
{"step": 148, "steps": 500, "lr": 1.6893746605507567e-05, "loss": 0.010011743693295019, "grad_norm": 0.8380138874053955, "loss_scale": 4.0, "retries": 0, "seconds": 1841.3, "max_memory_gb": 19.16}
|
| 149 |
+
{"step": 149, "steps": 500, "lr": 1.684947334636164e-05, "loss": 0.020853557000482296, "grad_norm": 1.2396947145462036, "loss_scale": 4.0, "retries": 0, "seconds": 1853.6, "max_memory_gb": 19.16}
|
| 150 |
+
{"step": 150, "steps": 500, "lr": 1.6804953639802675e-05, "loss": 0.008028252031271615, "grad_norm": 0.3473833501338959, "loss_scale": 4.0, "retries": 0, "seconds": 1866.2, "max_memory_gb": 19.16}
|
| 151 |
+
{"step": 151, "steps": 500, "lr": 1.6760189361515182e-05, "loss": 0.0001852274729918174, "grad_norm": 0.0099209388718009, "loss_scale": 4.0, "retries": 0, "seconds": 1878.4, "max_memory_gb": 19.16}
|
| 152 |
+
{"step": 152, "steps": 500, "lr": 1.6715182397487845e-05, "loss": 0.011535413744746073, "grad_norm": 1.0876377820968628, "loss_scale": 4.0, "retries": 0, "seconds": 1890.7, "max_memory_gb": 19.16}
|
| 153 |
+
{"step": 153, "steps": 500, "lr": 1.6669934643934088e-05, "loss": 0.0003316688484202049, "grad_norm": 0.017406318336725235, "loss_scale": 4.0, "retries": 0, "seconds": 1902.9, "max_memory_gb": 19.16}
|
| 154 |
+
{"step": 154, "steps": 500, "lr": 1.6624448007212168e-05, "loss": 0.00031234870131155196, "grad_norm": 0.028755255043506622, "loss_scale": 4.0, "retries": 0, "seconds": 1915.2, "max_memory_gb": 19.16}
|
| 155 |
+
{"step": 155, "steps": 500, "lr": 1.6578724403744854e-05, "loss": 0.0007623326413915329, "grad_norm": 0.04403872415423393, "loss_scale": 4.0, "retries": 0, "seconds": 1927.6, "max_memory_gb": 19.16}
|
| 156 |
+
{"step": 156, "steps": 500, "lr": 1.653276575993871e-05, "loss": 9.941580941456736e-05, "grad_norm": 0.00931652169674635, "loss_scale": 4.0, "retries": 0, "seconds": 1940.2, "max_memory_gb": 19.16}
|
| 157 |
+
{"step": 157, "steps": 500, "lr": 1.6486574012102896e-05, "loss": 0.01160117127907641, "grad_norm": 0.5918827056884766, "loss_scale": 4.0, "retries": 0, "seconds": 1952.5, "max_memory_gb": 19.16}
|
| 158 |
+
{"step": 158, "steps": 500, "lr": 1.6440151106367617e-05, "loss": 0.020187233765358057, "grad_norm": 0.9596201777458191, "loss_scale": 4.0, "retries": 0, "seconds": 1964.7, "max_memory_gb": 19.16}
|
| 159 |
+
{"step": 159, "steps": 500, "lr": 1.6393498998602123e-05, "loss": 4.5442503704862254e-05, "grad_norm": 0.0023098436649888754, "loss_scale": 4.0, "retries": 0, "seconds": 1977.2, "max_memory_gb": 19.16}
|
| 160 |
+
{"step": 160, "steps": 500, "lr": 1.6346619654332297e-05, "loss": 0.2203855096552652, "grad_norm": 7.331798076629639, "loss_scale": 4.0, "retries": 0, "seconds": 1989.9, "max_memory_gb": 19.16}
|
| 161 |
+
{"step": 161, "steps": 500, "lr": 1.6299515048657848e-05, "loss": 0.11910498404543417, "grad_norm": 4.45792818069458, "loss_scale": 4.0, "retries": 0, "seconds": 2002.4, "max_memory_gb": 19.16}
|
| 162 |
+
{"step": 162, "steps": 500, "lr": 1.6252187166169105e-05, "loss": 0.16172481799011962, "grad_norm": 12.371193885803223, "loss_scale": 4.0, "retries": 0, "seconds": 2014.8, "max_memory_gb": 19.16}
|
| 163 |
+
{"step": 163, "steps": 500, "lr": 1.6204638000863392e-05, "loss": 0.0006843822121538133, "grad_norm": 0.0783221423625946, "loss_scale": 4.0, "retries": 0, "seconds": 2027.5, "max_memory_gb": 19.16}
|
| 164 |
+
{"step": 164, "steps": 500, "lr": 1.615686955606102e-05, "loss": 0.34722021342588183, "grad_norm": 8.446325302124023, "loss_scale": 4.0, "retries": 0, "seconds": 2040.2, "max_memory_gb": 19.16}
|
| 165 |
+
{"step": 165, "steps": 500, "lr": 1.610888384432089e-05, "loss": 7.466893096105568e-05, "grad_norm": 0.0039074900560081005, "loss_scale": 4.0, "retries": 0, "seconds": 2052.6, "max_memory_gb": 19.16}
|
| 166 |
+
{"step": 166, "steps": 500, "lr": 1.60606828873557e-05, "loss": 3.680591767621619e-05, "grad_norm": 0.0016927530523389578, "loss_scale": 4.0, "retries": 0, "seconds": 2064.9, "max_memory_gb": 19.16}
|
| 167 |
+
{"step": 167, "steps": 500, "lr": 1.601226871594676e-05, "loss": 0.027971824307184336, "grad_norm": 2.634012460708618, "loss_scale": 4.0, "retries": 0, "seconds": 2077.3, "max_memory_gb": 19.16}
|
| 168 |
+
{"step": 168, "steps": 500, "lr": 1.5963643369858436e-05, "loss": 0.014742922310615955, "grad_norm": 1.404932975769043, "loss_scale": 4.0, "retries": 0, "seconds": 2089.4, "max_memory_gb": 19.16}
|
| 169 |
+
{"step": 169, "steps": 500, "lr": 1.591480889775221e-05, "loss": 0.0015076524268774385, "grad_norm": 0.1411173790693283, "loss_scale": 4.0, "retries": 0, "seconds": 2101.9, "max_memory_gb": 19.16}
|
| 170 |
+
{"step": 170, "steps": 500, "lr": 1.586576735710038e-05, "loss": 0.08433588366670172, "grad_norm": 2.141730308532715, "loss_scale": 4.0, "retries": 0, "seconds": 2114.3, "max_memory_gb": 19.16}
|
| 171 |
+
{"step": 171, "steps": 500, "lr": 1.5816520814099346e-05, "loss": 0.02788913819377276, "grad_norm": 1.5396580696105957, "loss_scale": 4.0, "retries": 0, "seconds": 2126.7, "max_memory_gb": 19.16}
|
| 172 |
+
{"step": 172, "steps": 500, "lr": 1.576707134358259e-05, "loss": 0.002288503615292567, "grad_norm": 0.16180674731731415, "loss_scale": 4.0, "retries": 0, "seconds": 2139.3, "max_memory_gb": 19.16}
|
| 173 |
+
{"step": 173, "steps": 500, "lr": 1.5717421028933236e-05, "loss": 0.005553824258673113, "grad_norm": 0.6700640320777893, "loss_scale": 4.0, "retries": 0, "seconds": 2151.8, "max_memory_gb": 19.16}
|
| 174 |
+
{"step": 174, "steps": 500, "lr": 1.5667571961996297e-05, "loss": 0.0001674820004082407, "grad_norm": 0.010141344740986824, "loss_scale": 4.0, "retries": 0, "seconds": 2164.3, "max_memory_gb": 19.16}
|
| 175 |
+
{"step": 175, "steps": 500, "lr": 1.5617526242990515e-05, "loss": 0.022046124768988887, "grad_norm": 2.095414638519287, "loss_scale": 4.0, "retries": 0, "seconds": 2176.6, "max_memory_gb": 19.16}
|
| 176 |
+
{"step": 176, "steps": 500, "lr": 1.5567285980419903e-05, "loss": 0.0012415240785120898, "grad_norm": 0.23540422320365906, "loss_scale": 4.0, "retries": 0, "seconds": 2188.9, "max_memory_gb": 19.16}
|
| 177 |
+
{"step": 177, "steps": 500, "lr": 1.551685329098488e-05, "loss": 0.009813195121800143, "grad_norm": 0.8447192907333374, "loss_scale": 4.0, "retries": 0, "seconds": 2201.4, "max_memory_gb": 19.16}
|
| 178 |
+
{"step": 178, "steps": 500, "lr": 1.5466230299493127e-05, "loss": 9.688577695176163e-05, "grad_norm": 0.005770058836787939, "loss_scale": 4.0, "retries": 0, "seconds": 2213.9, "max_memory_gb": 19.16}
|
| 179 |
+
{"step": 179, "steps": 500, "lr": 1.541541913877003e-05, "loss": 0.5028633819813861, "grad_norm": 3.9313440322875977, "loss_scale": 4.0, "retries": 0, "seconds": 2226.3, "max_memory_gb": 19.16}
|
| 180 |
+
{"step": 180, "steps": 500, "lr": 1.5364421949568853e-05, "loss": 7.914865174220931e-05, "grad_norm": 0.005011214874684811, "loss_scale": 4.0, "retries": 0, "seconds": 2239.2, "max_memory_gb": 19.16}
|
| 181 |
+
{"step": 181, "steps": 500, "lr": 1.5313240880480514e-05, "loss": 7.108874539728629e-05, "grad_norm": 0.00424942746758461, "loss_scale": 4.0, "retries": 0, "seconds": 2251.8, "max_memory_gb": 19.16}
|
| 182 |
+
{"step": 182, "steps": 500, "lr": 1.5261878087843087e-05, "loss": 0.00035952866537446556, "grad_norm": 0.0336538627743721, "loss_scale": 4.0, "retries": 0, "seconds": 2264.4, "max_memory_gb": 19.16}
|
| 183 |
+
{"step": 183, "steps": 500, "lr": 1.521033573565094e-05, "loss": 6.057594369224262e-05, "grad_norm": 0.0032440500799566507, "loss_scale": 4.0, "retries": 0, "seconds": 2277.0, "max_memory_gb": 19.16}
|
| 184 |
+
{"step": 184, "steps": 500, "lr": 1.5158615995463566e-05, "loss": 0.002261648598513168, "grad_norm": 0.10970473289489746, "loss_scale": 4.0, "retries": 0, "seconds": 2289.6, "max_memory_gb": 19.16}
|
| 185 |
+
{"step": 185, "steps": 500, "lr": 1.510672104631408e-05, "loss": 0.19164508563871802, "grad_norm": 5.218684196472168, "loss_scale": 4.0, "retries": 0, "seconds": 2302.0, "max_memory_gb": 19.16}
|
| 186 |
+
{"step": 186, "steps": 500, "lr": 1.5054653074617441e-05, "loss": 5.8915232960482015e-05, "grad_norm": 0.0029307317454367876, "loss_scale": 4.0, "retries": 0, "seconds": 2314.6, "max_memory_gb": 19.16}
|
| 187 |
+
{"step": 187, "steps": 500, "lr": 1.50024142740783e-05, "loss": 4.30465102244749e-05, "grad_norm": 0.0023089488968253136, "loss_scale": 4.0, "retries": 0, "seconds": 2327.1, "max_memory_gb": 19.16}
|
| 188 |
+
{"step": 188, "steps": 500, "lr": 1.4950006845598603e-05, "loss": 0.006155346766185879, "grad_norm": 0.43195298314094543, "loss_scale": 4.0, "retries": 0, "seconds": 2339.7, "max_memory_gb": 19.16}
|
| 189 |
+
{"step": 189, "steps": 500, "lr": 1.4897432997184845e-05, "loss": 0.024292862886609742, "grad_norm": 1.3977261781692505, "loss_scale": 4.0, "retries": 0, "seconds": 2352.2, "max_memory_gb": 19.16}
|
| 190 |
+
{"step": 190, "steps": 500, "lr": 1.4844694943855065e-05, "loss": 0.15402277686870938, "grad_norm": 3.0805933475494385, "loss_scale": 4.0, "retries": 0, "seconds": 2364.9, "max_memory_gb": 19.16}
|
| 191 |
+
{"step": 191, "steps": 500, "lr": 1.4791794907545496e-05, "loss": 8.848726543610817e-05, "grad_norm": 0.005159593652933836, "loss_scale": 4.0, "retries": 0, "seconds": 2377.6, "max_memory_gb": 19.16}
|
| 192 |
+
{"step": 192, "steps": 500, "lr": 1.4738735117016979e-05, "loss": 0.00014620160663980644, "grad_norm": 0.00951379258185625, "loss_scale": 4.0, "retries": 0, "seconds": 2390.2, "max_memory_gb": 19.16}
|
| 193 |
+
{"step": 193, "steps": 500, "lr": 1.4685517807761037e-05, "loss": 0.001036932089624026, "grad_norm": 0.08190803974866867, "loss_scale": 4.0, "retries": 0, "seconds": 2402.7, "max_memory_gb": 19.16}
|
| 194 |
+
{"step": 194, "steps": 500, "lr": 1.463214522190571e-05, "loss": 0.05048089216662985, "grad_norm": 2.9275388717651367, "loss_scale": 4.0, "retries": 0, "seconds": 2415.3, "max_memory_gb": 19.16}
|
| 195 |
+
{"step": 195, "steps": 500, "lr": 1.4578619608121079e-05, "loss": 0.0010351963529057105, "grad_norm": 0.08706967532634735, "loss_scale": 4.0, "retries": 0, "seconds": 2427.9, "max_memory_gb": 19.16}
|
| 196 |
+
{"step": 196, "steps": 500, "lr": 1.4524943221524533e-05, "loss": 0.00011250932960200028, "grad_norm": 0.00879885908216238, "loss_scale": 4.0, "retries": 0, "seconds": 2440.6, "max_memory_gb": 19.16}
|
| 197 |
+
{"step": 197, "steps": 500, "lr": 1.4471118323585751e-05, "loss": 0.02885881903976766, "grad_norm": 0.941164493560791, "loss_scale": 4.0, "retries": 0, "seconds": 2452.8, "max_memory_gb": 19.16}
|
| 198 |
+
{"step": 198, "steps": 500, "lr": 1.4417147182031426e-05, "loss": 6.643769865180813e-05, "grad_norm": 0.003115325700491667, "loss_scale": 4.0, "retries": 0, "seconds": 2465.4, "max_memory_gb": 19.16}
|
| 199 |
+
{"step": 199, "steps": 500, "lr": 1.4363032070749723e-05, "loss": 0.34668904465296047, "grad_norm": 6.6335835456848145, "loss_scale": 4.0, "retries": 0, "seconds": 2478.1, "max_memory_gb": 19.16}
|
| 200 |
+
{"step": 200, "steps": 500, "lr": 1.430877526969448e-05, "loss": 0.001438156289954019, "grad_norm": 0.09056846052408218, "loss_scale": 4.0, "retries": 0, "seconds": 2490.9, "max_memory_gb": 19.16}
|
| 201 |
+
{"step": 201, "steps": 500, "lr": 1.4254379064789138e-05, "loss": 0.11732112462617682, "grad_norm": 2.298553466796875, "loss_scale": 4.0, "retries": 0, "seconds": 2503.6, "max_memory_gb": 19.16}
|
| 202 |
+
{"step": 202, "steps": 500, "lr": 1.4199845747830443e-05, "loss": 0.0006784847992817463, "grad_norm": 0.03536621481180191, "loss_scale": 4.0, "retries": 0, "seconds": 2516.4, "max_memory_gb": 19.16}
|
| 203 |
+
{"step": 203, "steps": 500, "lr": 1.4145177616391886e-05, "loss": 0.3501881857048931, "grad_norm": 3.698071002960205, "loss_scale": 4.0, "retries": 0, "seconds": 2529.1, "max_memory_gb": 19.16}
|
| 204 |
+
{"step": 204, "steps": 500, "lr": 1.4090376973726898e-05, "loss": 0.00047036942676082947, "grad_norm": 0.0446900799870491, "loss_scale": 4.0, "retries": 0, "seconds": 2541.8, "max_memory_gb": 19.16}
|
| 205 |
+
{"step": 205, "steps": 500, "lr": 1.4035446128671815e-05, "loss": 7.322075489665539e-05, "grad_norm": 0.00332086649723351, "loss_scale": 4.0, "retries": 0, "seconds": 2554.5, "max_memory_gb": 19.16}
|
| 206 |
+
{"step": 206, "steps": 500, "lr": 1.3980387395548608e-05, "loss": 0.046388983523854684, "grad_norm": 0.9716008305549622, "loss_scale": 4.0, "retries": 0, "seconds": 2567.2, "max_memory_gb": 19.16}
|
| 207 |
+
{"step": 207, "steps": 500, "lr": 1.3925203094067364e-05, "loss": 0.000165066669503424, "grad_norm": 0.009224816225469112, "loss_scale": 4.0, "retries": 0, "seconds": 2579.8, "max_memory_gb": 19.16}
|
| 208 |
+
{"step": 208, "steps": 500, "lr": 1.3869895549228569e-05, "loss": 0.004211011811006219, "grad_norm": 0.16322050988674164, "loss_scale": 4.0, "retries": 0, "seconds": 2592.2, "max_memory_gb": 19.16}
|
| 209 |
+
{"step": 209, "steps": 500, "lr": 1.381446709122513e-05, "loss": 0.00016239714236896611, "grad_norm": 0.0072106909938156605, "loss_scale": 4.0, "retries": 0, "seconds": 2605.1, "max_memory_gb": 19.16}
|
| 210 |
+
{"step": 210, "steps": 500, "lr": 1.3758920055344226e-05, "loss": 0.00022790751653577575, "grad_norm": 0.010727690532803535, "loss_scale": 4.0, "retries": 0, "seconds": 2617.8, "max_memory_gb": 19.16}
|
| 211 |
+
{"step": 211, "steps": 500, "lr": 1.3703256781868898e-05, "loss": 0.0026031231768270536, "grad_norm": 0.14385348558425903, "loss_scale": 4.0, "retries": 0, "seconds": 2630.7, "max_memory_gb": 19.16}
|
| 212 |
+
{"step": 212, "steps": 500, "lr": 1.3647479615979466e-05, "loss": 0.00023171314654229553, "grad_norm": 0.010166074149310589, "loss_scale": 4.0, "retries": 0, "seconds": 2643.3, "max_memory_gb": 19.16}
|
| 213 |
+
{"step": 213, "steps": 500, "lr": 1.359159090765471e-05, "loss": 0.029291027856102537, "grad_norm": 2.201709270477295, "loss_scale": 4.0, "retries": 0, "seconds": 2656.0, "max_memory_gb": 19.16}
|
| 214 |
+
{"step": 214, "steps": 500, "lr": 1.3535593011572868e-05, "loss": 0.02052727485790291, "grad_norm": 0.7015287280082703, "loss_scale": 4.0, "retries": 0, "seconds": 2668.6, "max_memory_gb": 19.16}
|
| 215 |
+
{"step": 215, "steps": 500, "lr": 1.347948828701242e-05, "loss": 0.0004899786282418394, "grad_norm": 0.01959659717977047, "loss_scale": 4.0, "retries": 0, "seconds": 2681.3, "max_memory_gb": 19.16}
|
| 216 |
+
{"step": 216, "steps": 500, "lr": 1.3423279097752711e-05, "loss": 0.05550714595322213, "grad_norm": 3.2152459621429443, "loss_scale": 4.0, "retries": 0, "seconds": 2694.0, "max_memory_gb": 19.16}
|
| 217 |
+
{"step": 217, "steps": 500, "lr": 1.3366967811974332e-05, "loss": 0.03657756888684494, "grad_norm": 1.5576246976852417, "loss_scale": 4.0, "retries": 0, "seconds": 2706.7, "max_memory_gb": 19.16}
|
| 218 |
+
{"step": 218, "steps": 500, "lr": 1.3310556802159368e-05, "loss": 0.02055065904427135, "grad_norm": 0.6445297002792358, "loss_scale": 4.0, "retries": 0, "seconds": 2719.3, "max_memory_gb": 19.16}
|
| 219 |
+
{"step": 219, "steps": 500, "lr": 1.325404844499143e-05, "loss": 0.0447717343725096, "grad_norm": 1.709696650505066, "loss_scale": 4.0, "retries": 0, "seconds": 2732.0, "max_memory_gb": 19.16}
|
| 220 |
+
{"step": 220, "steps": 500, "lr": 1.3197445121255522e-05, "loss": 0.033547627105339095, "grad_norm": 1.6931309700012207, "loss_scale": 4.0, "retries": 0, "seconds": 2744.5, "max_memory_gb": 19.16}
|
| 221 |
+
{"step": 221, "steps": 500, "lr": 1.3140749215737745e-05, "loss": 0.00025473948705274034, "grad_norm": 0.017342878505587578, "loss_scale": 4.0, "retries": 0, "seconds": 2757.0, "max_memory_gb": 19.16}
|
| 222 |
+
{"step": 222, "steps": 500, "lr": 1.3083963117124801e-05, "loss": 0.00029252137811785025, "grad_norm": 0.013399563729763031, "loss_scale": 4.0, "retries": 0, "seconds": 2769.8, "max_memory_gb": 19.16}
|
| 223 |
+
{"step": 223, "steps": 500, "lr": 1.3027089217903386e-05, "loss": 0.25353283537068905, "grad_norm": 6.03549861907959, "loss_scale": 4.0, "retries": 0, "seconds": 2782.6, "max_memory_gb": 19.16}
|
| 224 |
+
{"step": 224, "steps": 500, "lr": 1.2970129914259354e-05, "loss": 0.0001598478530695502, "grad_norm": 0.006505039986222982, "loss_scale": 4.0, "retries": 0, "seconds": 2795.2, "max_memory_gb": 19.16}
|
| 225 |
+
{"step": 225, "steps": 500, "lr": 1.2913087605976789e-05, "loss": 0.004694043451484669, "grad_norm": 0.18157336115837097, "loss_scale": 4.0, "retries": 0, "seconds": 2807.6, "max_memory_gb": 19.16}
|
| 226 |
+
{"step": 226, "steps": 500, "lr": 1.2855964696336894e-05, "loss": 0.0005673351979980494, "grad_norm": 0.024761563166975975, "loss_scale": 4.0, "retries": 0, "seconds": 2820.3, "max_memory_gb": 19.16}
|
| 227 |
+
{"step": 227, "steps": 500, "lr": 1.2798763592016722e-05, "loss": 0.00042461964554521824, "grad_norm": 0.027162084355950356, "loss_scale": 4.0, "retries": 0, "seconds": 2832.9, "max_memory_gb": 19.16}
|
| 228 |
+
{"step": 228, "steps": 500, "lr": 1.2741486702987796e-05, "loss": 0.058711596809850816, "grad_norm": 5.458958148956299, "loss_scale": 4.0, "retries": 0, "seconds": 2845.6, "max_memory_gb": 19.16}
|
| 229 |
+
{"step": 229, "steps": 500, "lr": 1.268413644241457e-05, "loss": 0.03865429424669484, "grad_norm": 0.9248268008232117, "loss_scale": 4.0, "retries": 0, "seconds": 2858.3, "max_memory_gb": 19.16}
|
| 230 |
+
{"step": 230, "steps": 500, "lr": 1.2626715226552741e-05, "loss": 0.012824594967267703, "grad_norm": 1.1282761096954346, "loss_scale": 4.0, "retries": 0, "seconds": 2871.0, "max_memory_gb": 19.16}
|
| 231 |
+
{"step": 231, "steps": 500, "lr": 1.2569225474647482e-05, "loss": 0.3157587434711838, "grad_norm": 3.075057029724121, "loss_scale": 4.0, "retries": 0, "seconds": 2883.9, "max_memory_gb": 19.16}
|
| 232 |
+
{"step": 232, "steps": 500, "lr": 1.2511669608831483e-05, "loss": 0.01491643750186582, "grad_norm": 0.46381866931915283, "loss_scale": 4.0, "retries": 0, "seconds": 2896.6, "max_memory_gb": 19.16}
|
| 233 |
+
{"step": 233, "steps": 500, "lr": 1.2454050054022921e-05, "loss": 5.864539407308588e-05, "grad_norm": 0.0025655580684542656, "loss_scale": 4.0, "retries": 0, "seconds": 2909.1, "max_memory_gb": 19.16}
|
| 234 |
+
{"step": 234, "steps": 500, "lr": 1.2396369237823284e-05, "loss": 0.028419602465315958, "grad_norm": 0.8653475642204285, "loss_scale": 4.0, "retries": 0, "seconds": 2921.7, "max_memory_gb": 19.16}
|
| 235 |
+
{"step": 235, "steps": 500, "lr": 1.2338629590415109e-05, "loss": 0.000862655249430976, "grad_norm": 0.06171179935336113, "loss_scale": 4.0, "retries": 0, "seconds": 2934.0, "max_memory_gb": 19.16}
|
| 236 |
+
{"step": 236, "steps": 500, "lr": 1.2280833544459568e-05, "loss": 0.000937649901123388, "grad_norm": 0.04266377165913582, "loss_scale": 4.0, "retries": 0, "seconds": 2946.8, "max_memory_gb": 19.16}
|
| 237 |
+
{"step": 237, "steps": 500, "lr": 1.2222983534994002e-05, "loss": 0.00011416282520506436, "grad_norm": 0.004715098533779383, "loss_scale": 4.0, "retries": 0, "seconds": 2959.9, "max_memory_gb": 19.16}
|
| 238 |
+
{"step": 238, "steps": 500, "lr": 1.2165081999329318e-05, "loss": 0.025979249770743586, "grad_norm": 1.2666950225830078, "loss_scale": 4.0, "retries": 0, "seconds": 2972.4, "max_memory_gb": 19.16}
|
| 239 |
+
{"step": 239, "steps": 500, "lr": 1.2107131376947301e-05, "loss": 0.1187278630568058, "grad_norm": 2.8619863986968994, "loss_scale": 4.0, "retries": 0, "seconds": 2984.8, "max_memory_gb": 19.16}
|
| 240 |
+
{"step": 240, "steps": 500, "lr": 1.2049134109397833e-05, "loss": 0.00034586149221447414, "grad_norm": 0.01637110859155655, "loss_scale": 4.0, "retries": 0, "seconds": 2997.2, "max_memory_gb": 19.16}
|
| 241 |
+
{"step": 241, "steps": 500, "lr": 1.1991092640196032e-05, "loss": 9.383689301113341e-05, "grad_norm": 0.003920629620552063, "loss_scale": 4.0, "retries": 0, "seconds": 3009.5, "max_memory_gb": 19.16}
|
| 242 |
+
{"step": 242, "steps": 500, "lr": 1.1933009414719298e-05, "loss": 0.007836731777171157, "grad_norm": 0.49274179339408875, "loss_scale": 4.0, "retries": 0, "seconds": 3022.2, "max_memory_gb": 19.16}
|
| 243 |
+
{"step": 243, "steps": 500, "lr": 1.1874886880104294e-05, "loss": 0.016554599652202094, "grad_norm": 0.8581364750862122, "loss_scale": 4.0, "retries": 0, "seconds": 3034.8, "max_memory_gb": 19.16}
|
| 244 |
+
{"step": 244, "steps": 500, "lr": 1.181672748514383e-05, "loss": 0.0022657186259597115, "grad_norm": 0.10771134495735168, "loss_scale": 4.0, "retries": 0, "seconds": 3047.3, "max_memory_gb": 19.16}
|
| 245 |
+
{"step": 245, "steps": 500, "lr": 1.1758533680183702e-05, "loss": 0.0002927902651001866, "grad_norm": 0.014048715122044086, "loss_scale": 4.0, "retries": 0, "seconds": 3059.8, "max_memory_gb": 19.16}
|
| 246 |
+
{"step": 246, "steps": 500, "lr": 1.1700307917019457e-05, "loss": 8.09572271639974e-05, "grad_norm": 0.003668318036943674, "loss_scale": 4.0, "retries": 0, "seconds": 3072.5, "max_memory_gb": 19.16}
|
| 247 |
+
{"step": 247, "steps": 500, "lr": 1.164205264879309e-05, "loss": 0.0002675337023664394, "grad_norm": 0.01727626845240593, "loss_scale": 4.0, "retries": 0, "seconds": 3085.0, "max_memory_gb": 19.16}
|
| 248 |
+
{"step": 248, "steps": 500, "lr": 1.1583770329889687e-05, "loss": 0.0011818033443866873, "grad_norm": 0.05142395570874214, "loss_scale": 4.0, "retries": 0, "seconds": 3097.7, "max_memory_gb": 19.16}
|
| 249 |
+
{"step": 249, "steps": 500, "lr": 1.152546341583402e-05, "loss": 6.594339188578405e-05, "grad_norm": 0.002842144574970007, "loss_scale": 4.0, "retries": 0, "seconds": 3110.6, "max_memory_gb": 19.16}
|
| 250 |
+
{"step": 250, "steps": 500, "lr": 1.1467134363187098e-05, "loss": 0.0007181831221549312, "grad_norm": 0.03853136673569679, "loss_scale": 4.0, "retries": 0, "seconds": 3123.2, "max_memory_gb": 19.16}
|
| 251 |
+
{"step": 251, "steps": 500, "lr": 1.1408785629442657e-05, "loss": 0.04058616819350453, "grad_norm": 3.216813802719116, "loss_scale": 4.0, "retries": 0, "seconds": 3135.9, "max_memory_gb": 19.16}
|
| 252 |
+
{"step": 252, "steps": 500, "lr": 1.1350419672923639e-05, "loss": 0.0614485579270081, "grad_norm": 1.3319921493530273, "loss_scale": 4.0, "retries": 0, "seconds": 3148.5, "max_memory_gb": 19.16}
|
| 253 |
+
{"step": 253, "steps": 500, "lr": 1.1292038952678597e-05, "loss": 0.09865294260044072, "grad_norm": 3.8391520977020264, "loss_scale": 4.0, "retries": 0, "seconds": 3161.4, "max_memory_gb": 19.16}
|
| 254 |
+
{"step": 254, "steps": 500, "lr": 1.1233645928378104e-05, "loss": 0.08120035919204259, "grad_norm": 1.9952702522277832, "loss_scale": 4.0, "retries": 0, "seconds": 3174.1, "max_memory_gb": 19.16}
|
| 255 |
+
{"step": 255, "steps": 500, "lr": 1.117524306021113e-05, "loss": 0.0002595632933600262, "grad_norm": 0.011313853785395622, "loss_scale": 4.0, "retries": 0, "seconds": 3187.3, "max_memory_gb": 19.16}
|
| 256 |
+
{"step": 256, "steps": 500, "lr": 1.1116832808781377e-05, "loss": 0.00356366070470606, "grad_norm": 0.15976504981517792, "loss_scale": 4.0, "retries": 0, "seconds": 3200.0, "max_memory_gb": 19.16}
|
| 257 |
+
{"step": 257, "steps": 500, "lr": 1.1058417635003616e-05, "loss": 0.00015376904576152646, "grad_norm": 0.007936706766486168, "loss_scale": 4.0, "retries": 0, "seconds": 3212.7, "max_memory_gb": 19.16}
|
| 258 |
+
{"step": 258, "steps": 500, "lr": 1.1e-05, "loss": 0.00029591979166422214, "grad_norm": 0.012469107285141945, "loss_scale": 4.0, "retries": 0, "seconds": 3225.5, "max_memory_gb": 19.16}
|
| 259 |
+
{"step": 259, "steps": 500, "lr": 1.0941582364996383e-05, "loss": 0.0002554496676545816, "grad_norm": 0.01579202711582184, "loss_scale": 4.0, "retries": 0, "seconds": 3238.2, "max_memory_gb": 19.16}
|
| 260 |
+
{"step": 260, "steps": 500, "lr": 1.0883167191218624e-05, "loss": 0.09858174873212472, "grad_norm": 3.579481601715088, "loss_scale": 4.0, "retries": 0, "seconds": 3250.9, "max_memory_gb": 19.16}
|
| 261 |
+
{"step": 261, "steps": 500, "lr": 1.0824756939788867e-05, "loss": 0.09651535989889289, "grad_norm": 8.700040817260742, "loss_scale": 4.0, "retries": 0, "seconds": 3263.7, "max_memory_gb": 19.16}
|
| 262 |
+
{"step": 262, "steps": 500, "lr": 1.0766354071621896e-05, "loss": 0.08458508845990828, "grad_norm": 3.741769790649414, "loss_scale": 4.0, "retries": 0, "seconds": 3276.4, "max_memory_gb": 19.16}
|
| 263 |
+
{"step": 263, "steps": 500, "lr": 1.0707961047321403e-05, "loss": 0.4085564150200712, "grad_norm": 10.37220287322998, "loss_scale": 4.0, "retries": 0, "seconds": 3289.1, "max_memory_gb": 19.16}
|
| 264 |
+
{"step": 264, "steps": 500, "lr": 1.064958032707636e-05, "loss": 0.0002593387715137397, "grad_norm": 0.015644267201423645, "loss_scale": 4.0, "retries": 0, "seconds": 3301.8, "max_memory_gb": 19.16}
|
| 265 |
+
{"step": 265, "steps": 500, "lr": 1.0591214370557342e-05, "loss": 0.0002989678778042368, "grad_norm": 0.022623298689723015, "loss_scale": 4.0, "retries": 0, "seconds": 3314.5, "max_memory_gb": 19.16}
|
| 266 |
+
{"step": 266, "steps": 500, "lr": 1.0532865636812903e-05, "loss": 0.06013415878445792, "grad_norm": 3.8483386039733887, "loss_scale": 4.0, "retries": 0, "seconds": 3327.3, "max_memory_gb": 19.16}
|
| 267 |
+
{"step": 267, "steps": 500, "lr": 1.047453658416598e-05, "loss": 0.0004548665316302835, "grad_norm": 0.021580226719379425, "loss_scale": 4.0, "retries": 0, "seconds": 3340.2, "max_memory_gb": 19.16}
|
| 268 |
+
{"step": 268, "steps": 500, "lr": 1.0416229670110312e-05, "loss": 0.006189461111406755, "grad_norm": 0.2519790828227997, "loss_scale": 4.0, "retries": 0, "seconds": 3353.0, "max_memory_gb": 19.16}
|
| 269 |
+
{"step": 269, "steps": 500, "lr": 1.0357947351206909e-05, "loss": 6.444251227900499e-05, "grad_norm": 0.0029353899881243706, "loss_scale": 4.0, "retries": 0, "seconds": 3365.6, "max_memory_gb": 19.16}
|
| 270 |
+
{"step": 270, "steps": 500, "lr": 1.029969208298054e-05, "loss": 0.04268782768393853, "grad_norm": 1.4573663473129272, "loss_scale": 4.0, "retries": 0, "seconds": 3378.4, "max_memory_gb": 19.16}
|
| 271 |
+
{"step": 271, "steps": 500, "lr": 1.02414663198163e-05, "loss": 0.06471246656862335, "grad_norm": 1.6653279066085815, "loss_scale": 4.0, "retries": 0, "seconds": 3391.3, "max_memory_gb": 19.16}
|
| 272 |
+
{"step": 272, "steps": 500, "lr": 1.0183272514856171e-05, "loss": 0.7474573085617585, "grad_norm": 10.127269744873047, "loss_scale": 4.0, "retries": 0, "seconds": 3404.1, "max_memory_gb": 19.16}
|
| 273 |
+
{"step": 273, "steps": 500, "lr": 1.0125113119895709e-05, "loss": 0.00905305051996534, "grad_norm": 0.4514332115650177, "loss_scale": 4.0, "retries": 0, "seconds": 3416.9, "max_memory_gb": 19.16}
|
| 274 |
+
{"step": 274, "steps": 500, "lr": 1.0066990585280703e-05, "loss": 0.000694584893008968, "grad_norm": 0.07227959483861923, "loss_scale": 4.0, "retries": 0, "seconds": 3430.0, "max_memory_gb": 19.16}
|
| 275 |
+
{"step": 275, "steps": 500, "lr": 1.0008907359803967e-05, "loss": 0.00031839912093739104, "grad_norm": 0.03208701312541962, "loss_scale": 4.0, "retries": 0, "seconds": 3442.9, "max_memory_gb": 19.16}
|
| 276 |
+
{"step": 276, "steps": 500, "lr": 9.950865890602168e-06, "loss": 0.0003587770693539742, "grad_norm": 0.033815816044807434, "loss_scale": 4.0, "retries": 0, "seconds": 3455.6, "max_memory_gb": 19.16}
|
| 277 |
+
{"step": 277, "steps": 500, "lr": 9.892868623052698e-06, "loss": 7.459662434298893e-05, "grad_norm": 0.004020271357148886, "loss_scale": 4.0, "retries": 0, "seconds": 3468.3, "max_memory_gb": 19.16}
|
| 278 |
+
{"step": 278, "steps": 500, "lr": 9.834918000670681e-06, "loss": 0.002503659816439226, "grad_norm": 0.13144147396087646, "loss_scale": 4.0, "retries": 0, "seconds": 3480.7, "max_memory_gb": 19.16}
|
| 279 |
+
{"step": 279, "steps": 500, "lr": 9.777016465006e-06, "loss": 0.028310906559010043, "grad_norm": 2.2546818256378174, "loss_scale": 4.0, "retries": 0, "seconds": 3493.7, "max_memory_gb": 19.16}
|
| 280 |
+
{"step": 280, "steps": 500, "lr": 9.719166455540433e-06, "loss": 0.009365632430951173, "grad_norm": 0.9710189700126648, "loss_scale": 4.0, "retries": 0, "seconds": 3506.5, "max_memory_gb": 19.16}
|
| 281 |
+
{"step": 281, "steps": 500, "lr": 9.661370409584892e-06, "loss": 0.008038086522260941, "grad_norm": 0.31680700182914734, "loss_scale": 4.0, "retries": 0, "seconds": 3519.4, "max_memory_gb": 19.16}
|
| 282 |
+
{"step": 282, "steps": 500, "lr": 9.603630762176718e-06, "loss": 0.00015113790222833566, "grad_norm": 0.006654209457337856, "loss_scale": 4.0, "retries": 0, "seconds": 3532.1, "max_memory_gb": 19.16}
|
| 283 |
+
{"step": 283, "steps": 500, "lr": 9.54594994597708e-06, "loss": 0.0005154512708817164, "grad_norm": 0.02237640507519245, "loss_scale": 4.0, "retries": 0, "seconds": 3545.2, "max_memory_gb": 19.16}
|
| 284 |
+
{"step": 284, "steps": 500, "lr": 9.488330391168515e-06, "loss": 7.81978518850999e-05, "grad_norm": 0.003831462934613228, "loss_scale": 4.0, "retries": 0, "seconds": 3557.9, "max_memory_gb": 19.16}
|
| 285 |
+
{"step": 285, "steps": 500, "lr": 9.430774525352518e-06, "loss": 0.03651162918294466, "grad_norm": 0.9842357635498047, "loss_scale": 4.0, "retries": 0, "seconds": 3570.6, "max_memory_gb": 19.16}
|
| 286 |
+
{"step": 286, "steps": 500, "lr": 9.373284773447258e-06, "loss": 0.0027801366948923796, "grad_norm": 0.2791273593902588, "loss_scale": 4.0, "retries": 0, "seconds": 3583.3, "max_memory_gb": 19.16}
|
| 287 |
+
{"step": 287, "steps": 500, "lr": 9.315863557585435e-06, "loss": 0.11893548925230459, "grad_norm": 3.4204511642456055, "loss_scale": 4.0, "retries": 0, "seconds": 3596.0, "max_memory_gb": 19.16}
|
| 288 |
+
{"step": 288, "steps": 500, "lr": 9.258513297012203e-06, "loss": 0.00014310040201466603, "grad_norm": 0.006683045532554388, "loss_scale": 4.0, "retries": 0, "seconds": 3608.8, "max_memory_gb": 19.16}
|
| 289 |
+
{"step": 289, "steps": 500, "lr": 9.201236407983281e-06, "loss": 0.00011060868871481945, "grad_norm": 0.005618404597043991, "loss_scale": 4.0, "retries": 0, "seconds": 3621.5, "max_memory_gb": 19.16}
|
| 290 |
+
{"step": 290, "steps": 500, "lr": 9.144035303663109e-06, "loss": 0.0013703449265097913, "grad_norm": 0.17739883065223694, "loss_scale": 4.0, "retries": 0, "seconds": 3634.0, "max_memory_gb": 19.16}
|
| 291 |
+
{"step": 291, "steps": 500, "lr": 9.08691239402321e-06, "loss": 0.0018989541223700976, "grad_norm": 0.09935017675161362, "loss_scale": 4.0, "retries": 0, "seconds": 3646.6, "max_memory_gb": 19.16}
|
| 292 |
+
{"step": 292, "steps": 500, "lr": 9.029870085740647e-06, "loss": 0.00037346188017037014, "grad_norm": 0.022561566904187202, "loss_scale": 4.0, "retries": 0, "seconds": 3659.3, "max_memory_gb": 19.16}
|
| 293 |
+
{"step": 293, "steps": 500, "lr": 8.972910782096616e-06, "loss": 0.0003156783307050856, "grad_norm": 0.018630539998412132, "loss_scale": 4.0, "retries": 0, "seconds": 3672.0, "max_memory_gb": 19.16}
|
| 294 |
+
{"step": 294, "steps": 500, "lr": 8.916036882875198e-06, "loss": 9.536930888936529e-05, "grad_norm": 0.0041272290982306, "loss_scale": 4.0, "retries": 0, "seconds": 3684.7, "max_memory_gb": 19.16}
|
| 295 |
+
{"step": 295, "steps": 500, "lr": 8.859250784262256e-06, "loss": 0.0002544803853109815, "grad_norm": 0.03742282837629318, "loss_scale": 4.0, "retries": 0, "seconds": 3697.4, "max_memory_gb": 19.16}
|
| 296 |
+
{"step": 296, "steps": 500, "lr": 8.802554878744479e-06, "loss": 0.06475260814679794, "grad_norm": 1.8178471326828003, "loss_scale": 4.0, "retries": 0, "seconds": 3710.2, "max_memory_gb": 19.16}
|
| 297 |
+
{"step": 297, "steps": 500, "lr": 8.74595155500857e-06, "loss": 0.0008795026142717111, "grad_norm": 0.03997319936752319, "loss_scale": 4.0, "retries": 0, "seconds": 3722.7, "max_memory_gb": 19.16}
|
| 298 |
+
{"step": 298, "steps": 500, "lr": 8.689443197840635e-06, "loss": 0.005202939040941602, "grad_norm": 0.48795536160469055, "loss_scale": 4.0, "retries": 0, "seconds": 3735.5, "max_memory_gb": 19.16}
|
| 299 |
+
{"step": 299, "steps": 500, "lr": 8.633032188025669e-06, "loss": 0.21940235151843332, "grad_norm": 6.358510971069336, "loss_scale": 4.0, "retries": 0, "seconds": 3748.1, "max_memory_gb": 19.16}
|
| 300 |
+
{"step": 300, "steps": 500, "lr": 8.576720902247288e-06, "loss": 0.013997900218356564, "grad_norm": 1.8637049198150635, "loss_scale": 4.0, "retries": 0, "seconds": 3760.8, "max_memory_gb": 19.16}
|
| 301 |
+
{"step": 301, "steps": 500, "lr": 8.52051171298758e-06, "loss": 0.0006282404770197238, "grad_norm": 0.03783520311117172, "loss_scale": 4.0, "retries": 0, "seconds": 3773.9, "max_memory_gb": 19.16}
|
| 302 |
+
{"step": 302, "steps": 500, "lr": 8.464406988427132e-06, "loss": 0.4196122582504555, "grad_norm": 9.123861312866211, "loss_scale": 4.0, "retries": 0, "seconds": 3786.9, "max_memory_gb": 19.16}
|
| 303 |
+
{"step": 303, "steps": 500, "lr": 8.40840909234529e-06, "loss": 0.08872662729976873, "grad_norm": 9.441596984863281, "loss_scale": 4.0, "retries": 0, "seconds": 3799.3, "max_memory_gb": 19.16}
|
| 304 |
+
{"step": 304, "steps": 500, "lr": 8.352520384020533e-06, "loss": 0.00019217796466105858, "grad_norm": 0.011438257060945034, "loss_scale": 4.0, "retries": 0, "seconds": 3812.2, "max_memory_gb": 19.16}
|
| 305 |
+
{"step": 305, "steps": 500, "lr": 8.296743218131103e-06, "loss": 0.001034016498542976, "grad_norm": 0.07246891409158707, "loss_scale": 4.0, "retries": 0, "seconds": 3824.9, "max_memory_gb": 19.16}
|
| 306 |
+
{"step": 306, "steps": 500, "lr": 8.241079944655777e-06, "loss": 0.034287991351029425, "grad_norm": 3.064605474472046, "loss_scale": 4.0, "retries": 0, "seconds": 3837.5, "max_memory_gb": 19.16}
|
| 307 |
+
{"step": 307, "steps": 500, "lr": 8.185532908774872e-06, "loss": 9.667552169968408e-05, "grad_norm": 0.004469488747417927, "loss_scale": 4.0, "retries": 0, "seconds": 3850.1, "max_memory_gb": 19.16}
|
| 308 |
+
{"step": 308, "steps": 500, "lr": 8.130104450771434e-06, "loss": 0.00012283865858493797, "grad_norm": 0.006669987458735704, "loss_scale": 4.0, "retries": 0, "seconds": 3862.7, "max_memory_gb": 19.16}
|
| 309 |
+
{"step": 309, "steps": 500, "lr": 8.074796905932634e-06, "loss": 0.004870387248636441, "grad_norm": 0.48947393894195557, "loss_scale": 4.0, "retries": 0, "seconds": 3875.1, "max_memory_gb": 19.16}
|
| 310 |
+
{"step": 310, "steps": 500, "lr": 8.019612604451395e-06, "loss": 0.03297083151832423, "grad_norm": 2.0119714736938477, "loss_scale": 4.0, "retries": 0, "seconds": 3887.8, "max_memory_gb": 19.16}
|
| 311 |
+
{"step": 311, "steps": 500, "lr": 7.964553871328185e-06, "loss": 0.00013319362421591245, "grad_norm": 0.006565264891833067, "loss_scale": 4.0, "retries": 0, "seconds": 3900.5, "max_memory_gb": 19.16}
|
| 312 |
+
{"step": 312, "steps": 500, "lr": 7.909623026273106e-06, "loss": 0.000131890720516801, "grad_norm": 0.008992503397166729, "loss_scale": 4.0, "retries": 0, "seconds": 3913.6, "max_memory_gb": 19.16}
|
| 313 |
+
{"step": 313, "steps": 500, "lr": 7.854822383608116e-06, "loss": 0.0008161541998545504, "grad_norm": 0.0399765782058239, "loss_scale": 4.0, "retries": 0, "seconds": 3926.3, "max_memory_gb": 19.16}
|
| 314 |
+
{"step": 314, "steps": 500, "lr": 7.800154252169558e-06, "loss": 0.00016787199529940983, "grad_norm": 0.008016292937099934, "loss_scale": 4.0, "retries": 0, "seconds": 3939.1, "max_memory_gb": 19.16}
|
| 315 |
+
{"step": 315, "steps": 500, "lr": 7.745620935210861e-06, "loss": 0.09369810687112269, "grad_norm": 1.6402008533477783, "loss_scale": 4.0, "retries": 0, "seconds": 3951.8, "max_memory_gb": 19.16}
|
| 316 |
+
{"step": 316, "steps": 500, "lr": 7.691224730305518e-06, "loss": 0.0008249900561943172, "grad_norm": 0.03937331214547157, "loss_scale": 4.0, "retries": 0, "seconds": 3964.6, "max_memory_gb": 19.16}
|
| 317 |
+
{"step": 317, "steps": 500, "lr": 7.636967929250276e-06, "loss": 0.0012237590223766017, "grad_norm": 0.10574006289243698, "loss_scale": 4.0, "retries": 0, "seconds": 3977.3, "max_memory_gb": 19.16}
|
| 318 |
+
{"step": 318, "steps": 500, "lr": 7.5828528179685755e-06, "loss": 6.420577706090957e-05, "grad_norm": 0.00245215417817235, "loss_scale": 4.0, "retries": 0, "seconds": 3989.9, "max_memory_gb": 19.16}
|
| 319 |
+
{"step": 319, "steps": 500, "lr": 7.528881676414252e-06, "loss": 0.0001119817018827069, "grad_norm": 0.004629025235772133, "loss_scale": 4.0, "retries": 0, "seconds": 4002.4, "max_memory_gb": 19.16}
|
| 320 |
+
{"step": 320, "steps": 500, "lr": 7.475056778475468e-06, "loss": 0.003048769750571203, "grad_norm": 0.1510339081287384, "loss_scale": 4.0, "retries": 0, "seconds": 4015.1, "max_memory_gb": 19.16}
|
| 321 |
+
{"step": 321, "steps": 500, "lr": 7.421380391878924e-06, "loss": 8.955216452994301e-05, "grad_norm": 0.0038828523829579353, "loss_scale": 4.0, "retries": 0, "seconds": 4027.5, "max_memory_gb": 19.16}
|
| 322 |
+
{"step": 322, "steps": 500, "lr": 7.367854778094291e-06, "loss": 0.0001093289491171328, "grad_norm": 0.005529636982828379, "loss_scale": 4.0, "retries": 0, "seconds": 4040.0, "max_memory_gb": 19.16}
|
| 323 |
+
{"step": 323, "steps": 500, "lr": 7.314482192238962e-06, "loss": 0.011770115059789532, "grad_norm": 0.46995222568511963, "loss_scale": 4.0, "retries": 0, "seconds": 4052.8, "max_memory_gb": 19.16}
|
| 324 |
+
{"step": 324, "steps": 500, "lr": 7.261264882983024e-06, "loss": 0.05851374153574666, "grad_norm": 2.893994092941284, "loss_scale": 4.0, "retries": 0, "seconds": 4065.7, "max_memory_gb": 19.16}
|
| 325 |
+
{"step": 325, "steps": 500, "lr": 7.208205092454502e-06, "loss": 5.8918300907025284e-05, "grad_norm": 0.0023364839144051075, "loss_scale": 4.0, "retries": 0, "seconds": 4078.4, "max_memory_gb": 19.16}
|
| 326 |
+
{"step": 326, "steps": 500, "lr": 7.155305056144935e-06, "loss": 0.010864442379613592, "grad_norm": 0.7235226035118103, "loss_scale": 4.0, "retries": 0, "seconds": 4090.9, "max_memory_gb": 19.16}
|
| 327 |
+
{"step": 327, "steps": 500, "lr": 7.102567002815155e-06, "loss": 0.00027329130865506494, "grad_norm": 0.02112644910812378, "loss_scale": 4.0, "retries": 0, "seconds": 4103.7, "max_memory_gb": 19.16}
|
| 328 |
+
{"step": 328, "steps": 500, "lr": 7.0499931544013985e-06, "loss": 0.00018858752612645316, "grad_norm": 0.011456282809376717, "loss_scale": 4.0, "retries": 0, "seconds": 4116.7, "max_memory_gb": 19.16}
|
| 329 |
+
{"step": 329, "steps": 500, "lr": 6.997585725921701e-06, "loss": 0.0003631881675325779, "grad_norm": 0.024614449590444565, "loss_scale": 4.0, "retries": 0, "seconds": 4129.2, "max_memory_gb": 19.16}
|
| 330 |
+
{"step": 330, "steps": 500, "lr": 6.94534692538256e-06, "loss": 8.381451677699012e-05, "grad_norm": 0.004574912134557962, "loss_scale": 4.0, "retries": 0, "seconds": 4141.9, "max_memory_gb": 19.16}
|
| 331 |
+
{"step": 331, "steps": 500, "lr": 6.893278953685919e-06, "loss": 0.0017649901515658878, "grad_norm": 0.15501999855041504, "loss_scale": 4.0, "retries": 0, "seconds": 4154.7, "max_memory_gb": 19.16}
|
| 332 |
+
{"step": 332, "steps": 500, "lr": 6.841384004536434e-06, "loss": 0.13202012386628587, "grad_norm": 3.160874128341675, "loss_scale": 4.0, "retries": 0, "seconds": 4166.8, "max_memory_gb": 19.16}
|
| 333 |
+
{"step": 333, "steps": 500, "lr": 6.789664264349063e-06, "loss": 0.004716446416706917, "grad_norm": 0.25230878591537476, "loss_scale": 4.0, "retries": 0, "seconds": 4179.6, "max_memory_gb": 19.16}
|
| 334 |
+
{"step": 334, "steps": 500, "lr": 6.738121912156913e-06, "loss": 7.150248946820947e-05, "grad_norm": 0.002905531320720911, "loss_scale": 4.0, "retries": 0, "seconds": 4192.4, "max_memory_gb": 19.16}
|
| 335 |
+
{"step": 335, "steps": 500, "lr": 6.686759119519493e-06, "loss": 5.819342856677229e-05, "grad_norm": 0.0027944219764322042, "loss_scale": 4.0, "retries": 0, "seconds": 4205.1, "max_memory_gb": 19.16}
|
| 336 |
+
{"step": 336, "steps": 500, "lr": 6.63557805043115e-06, "loss": 0.028059621592105222, "grad_norm": 1.098417043685913, "loss_scale": 4.0, "retries": 0, "seconds": 4218.2, "max_memory_gb": 19.16}
|
| 337 |
+
{"step": 337, "steps": 500, "lr": 6.5845808612299715e-06, "loss": 0.21799340045891896, "grad_norm": 7.606483459472656, "loss_scale": 4.0, "retries": 0, "seconds": 4230.7, "max_memory_gb": 19.16}
|
| 338 |
+
{"step": 338, "steps": 500, "lr": 6.5337697005068745e-06, "loss": 0.14785583421749748, "grad_norm": 2.5556507110595703, "loss_scale": 4.0, "retries": 0, "seconds": 4243.2, "max_memory_gb": 19.16}
|
| 339 |
+
{"step": 339, "steps": 500, "lr": 6.48314670901512e-06, "loss": 0.0005691442363965651, "grad_norm": 0.05923148989677429, "loss_scale": 4.0, "retries": 0, "seconds": 4255.9, "max_memory_gb": 19.16}
|
| 340 |
+
{"step": 340, "steps": 500, "lr": 6.4327140195801e-06, "loss": 0.011392109606886563, "grad_norm": 0.5835305452346802, "loss_scale": 4.0, "retries": 0, "seconds": 4268.6, "max_memory_gb": 19.16}
|
| 341 |
+
{"step": 341, "steps": 500, "lr": 6.382473757009482e-06, "loss": 0.00035603218921923235, "grad_norm": 0.01698751375079155, "loss_scale": 4.0, "retries": 0, "seconds": 4281.4, "max_memory_gb": 19.16}
|
| 342 |
+
{"step": 342, "steps": 500, "lr": 6.332428038003706e-06, "loss": 0.0012848029006717484, "grad_norm": 0.07906191051006317, "loss_scale": 4.0, "retries": 0, "seconds": 4294.2, "max_memory_gb": 19.16}
|
| 343 |
+
{"step": 343, "steps": 500, "lr": 6.282578971066761e-06, "loss": 0.004013634132919917, "grad_norm": 0.2410963773727417, "loss_scale": 4.0, "retries": 0, "seconds": 4307.0, "max_memory_gb": 19.16}
|
| 344 |
+
{"step": 344, "steps": 500, "lr": 6.232928656417414e-06, "loss": 0.002776410741649795, "grad_norm": 0.14270934462547302, "loss_scale": 4.0, "retries": 0, "seconds": 4319.7, "max_memory_gb": 19.16}
|
| 345 |
+
{"step": 345, "steps": 500, "lr": 6.183479185900656e-06, "loss": 0.002672963116248701, "grad_norm": 0.23976433277130127, "loss_scale": 4.0, "retries": 0, "seconds": 4332.4, "max_memory_gb": 19.16}
|
| 346 |
+
{"step": 346, "steps": 500, "lr": 6.134232642899625e-06, "loss": 0.0014360607042207944, "grad_norm": 0.12666292488574982, "loss_scale": 4.0, "retries": 0, "seconds": 4345.1, "max_memory_gb": 19.16}
|
| 347 |
+
{"step": 347, "steps": 500, "lr": 6.085191102247789e-06, "loss": 8.879994769276065e-05, "grad_norm": 0.004492596723139286, "loss_scale": 4.0, "retries": 0, "seconds": 4357.9, "max_memory_gb": 19.16}
|
| 348 |
+
{"step": 348, "steps": 500, "lr": 6.036356630141565e-06, "loss": 0.12288155420444273, "grad_norm": 2.85644268989563, "loss_scale": 4.0, "retries": 0, "seconds": 4370.5, "max_memory_gb": 19.16}
|
| 349 |
+
{"step": 349, "steps": 500, "lr": 5.987731284053239e-06, "loss": 0.0006113966226270018, "grad_norm": 0.03927795961499214, "loss_scale": 4.0, "retries": 0, "seconds": 4382.8, "max_memory_gb": 19.16}
|
| 350 |
+
{"step": 350, "steps": 500, "lr": 5.9393171126443e-06, "loss": 0.00011458318748096147, "grad_norm": 0.006228385027498007, "loss_scale": 4.0, "retries": 0, "seconds": 4395.2, "max_memory_gb": 19.16}
|
| 351 |
+
{"step": 351, "steps": 500, "lr": 5.89111615567911e-06, "loss": 0.11620152172452239, "grad_norm": 2.9015414714813232, "loss_scale": 4.0, "retries": 0, "seconds": 4407.4, "max_memory_gb": 19.16}
|
| 352 |
+
{"step": 352, "steps": 500, "lr": 5.843130443938982e-06, "loss": 0.09655973829074521, "grad_norm": 2.2192187309265137, "loss_scale": 4.0, "retries": 0, "seconds": 4420.0, "max_memory_gb": 19.16}
|
| 353 |
+
{"step": 353, "steps": 500, "lr": 5.795361999136612e-06, "loss": 0.00016229002569254837, "grad_norm": 0.007930255495011806, "loss_scale": 4.0, "retries": 0, "seconds": 4432.4, "max_memory_gb": 19.16}
|
| 354 |
+
{"step": 354, "steps": 500, "lr": 5.747812833830894e-06, "loss": 0.00014965270940013974, "grad_norm": 0.007448078133165836, "loss_scale": 4.0, "retries": 0, "seconds": 4444.6, "max_memory_gb": 19.16}
|
| 355 |
+
{"step": 355, "steps": 500, "lr": 5.700484951342151e-06, "loss": 0.00024456841571210674, "grad_norm": 0.011896634474396706, "loss_scale": 4.0, "retries": 0, "seconds": 4457.0, "max_memory_gb": 19.16}
|
| 356 |
+
{"step": 356, "steps": 500, "lr": 5.6533803456677035e-06, "loss": 0.0005631037809673052, "grad_norm": 0.035616789013147354, "loss_scale": 4.0, "retries": 0, "seconds": 4469.4, "max_memory_gb": 19.16}
|
| 357 |
+
{"step": 357, "steps": 500, "lr": 5.606501001397878e-06, "loss": 0.002608983065344588, "grad_norm": 0.1368495225906372, "loss_scale": 4.0, "retries": 0, "seconds": 4481.8, "max_memory_gb": 19.16}
|
| 358 |
+
{"step": 358, "steps": 500, "lr": 5.559848893632384e-06, "loss": 0.00010118025920488353, "grad_norm": 0.006038842257112265, "loss_scale": 4.0, "retries": 0, "seconds": 4494.4, "max_memory_gb": 19.16}
|
| 359 |
+
{"step": 359, "steps": 500, "lr": 5.513425987897107e-06, "loss": 0.08606793315411211, "grad_norm": 3.3046228885650635, "loss_scale": 4.0, "retries": 0, "seconds": 4506.9, "max_memory_gb": 19.16}
|
| 360 |
+
{"step": 360, "steps": 500, "lr": 5.467234240061293e-06, "loss": 0.2026087069930534, "grad_norm": 4.5673065185546875, "loss_scale": 4.0, "retries": 0, "seconds": 4519.3, "max_memory_gb": 19.16}
|
| 361 |
+
{"step": 361, "steps": 500, "lr": 5.421275596255145e-06, "loss": 0.01754147422293913, "grad_norm": 1.521700143814087, "loss_scale": 4.0, "retries": 0, "seconds": 4531.7, "max_memory_gb": 19.16}
|
| 362 |
+
{"step": 362, "steps": 500, "lr": 5.375551992787835e-06, "loss": 9.427278256556417e-05, "grad_norm": 0.0044240280985832214, "loss_scale": 4.0, "retries": 0, "seconds": 4544.1, "max_memory_gb": 19.16}
|
| 363 |
+
{"step": 363, "steps": 500, "lr": 5.330065356065913e-06, "loss": 0.04223246409127445, "grad_norm": 9.272345542907715, "loss_scale": 4.0, "retries": 0, "seconds": 4556.8, "max_memory_gb": 19.16}
|
| 364 |
+
{"step": 364, "steps": 500, "lr": 5.284817602512153e-06, "loss": 0.16234561238587214, "grad_norm": 8.56887149810791, "loss_scale": 4.0, "retries": 0, "seconds": 4568.9, "max_memory_gb": 19.16}
|
| 365 |
+
{"step": 365, "steps": 500, "lr": 5.239810638484819e-06, "loss": 0.041300742823068504, "grad_norm": 1.5548663139343262, "loss_scale": 4.0, "retries": 0, "seconds": 4581.7, "max_memory_gb": 19.16}
|
| 366 |
+
{"step": 366, "steps": 500, "lr": 5.195046360197325e-06, "loss": 0.007044838380621018, "grad_norm": 0.7214199900627136, "loss_scale": 4.0, "retries": 0, "seconds": 4594.0, "max_memory_gb": 19.16}
|
| 367 |
+
{"step": 367, "steps": 500, "lr": 5.150526653638363e-06, "loss": 0.0027215419454736625, "grad_norm": 0.33012932538986206, "loss_scale": 4.0, "retries": 0, "seconds": 4606.4, "max_memory_gb": 19.16}
|
| 368 |
+
{"step": 368, "steps": 500, "lr": 5.106253394492435e-06, "loss": 0.000685751156812454, "grad_norm": 0.12590618431568146, "loss_scale": 4.0, "retries": 0, "seconds": 4618.8, "max_memory_gb": 19.16}
|
| 369 |
+
{"step": 369, "steps": 500, "lr": 5.062228448060833e-06, "loss": 0.0008181156184202171, "grad_norm": 0.0820736214518547, "loss_scale": 4.0, "retries": 0, "seconds": 4630.9, "max_memory_gb": 19.16}
|
| 370 |
+
{"step": 370, "steps": 500, "lr": 5.018453669183048e-06, "loss": 0.0002579442201419046, "grad_norm": 0.011928917840123177, "loss_scale": 4.0, "retries": 0, "seconds": 4643.4, "max_memory_gb": 19.16}
|
| 371 |
+
{"step": 371, "steps": 500, "lr": 4.974930902158616e-06, "loss": 0.007698546689534231, "grad_norm": 0.5882173776626587, "loss_scale": 4.0, "retries": 0, "seconds": 4656.0, "max_memory_gb": 19.17}
|
| 372 |
+
{"step": 372, "steps": 500, "lr": 4.931661980669439e-06, "loss": 6.950735904741701e-05, "grad_norm": 0.0030386352445930243, "loss_scale": 4.0, "retries": 0, "seconds": 4668.4, "max_memory_gb": 19.17}
|
| 373 |
+
{"step": 373, "steps": 500, "lr": 4.8886487277024854e-06, "loss": 0.00013123559876310705, "grad_norm": 0.005470382981002331, "loss_scale": 4.0, "retries": 0, "seconds": 4681.1, "max_memory_gb": 19.17}
|
| 374 |
+
{"step": 374, "steps": 500, "lr": 4.845892955473037e-06, "loss": 4.660737243449892e-05, "grad_norm": 0.0020694397389888763, "loss_scale": 4.0, "retries": 0, "seconds": 4693.6, "max_memory_gb": 19.17}
|
| 375 |
+
{"step": 375, "steps": 500, "lr": 4.80339646534829e-06, "loss": 0.003368427140974911, "grad_norm": 0.22972644865512848, "loss_scale": 4.0, "retries": 0, "seconds": 4705.7, "max_memory_gb": 19.17}
|
| 376 |
+
{"step": 376, "steps": 500, "lr": 4.761161047771502e-06, "loss": 9.751956419925345e-05, "grad_norm": 0.0036321876104921103, "loss_scale": 4.0, "retries": 0, "seconds": 4718.1, "max_memory_gb": 19.17}
|
| 377 |
+
{"step": 377, "steps": 500, "lr": 4.719188482186515e-06, "loss": 0.00013741701409551092, "grad_norm": 0.006432952359318733, "loss_scale": 4.0, "retries": 0, "seconds": 4730.5, "max_memory_gb": 19.17}
|
| 378 |
+
{"step": 378, "steps": 500, "lr": 4.677480536962832e-06, "loss": 7.304365690494954e-05, "grad_norm": 0.002679346362128854, "loss_scale": 4.0, "retries": 0, "seconds": 4743.1, "max_memory_gb": 19.17}
|
| 379 |
+
{"step": 379, "steps": 500, "lr": 4.6360389693210725e-06, "loss": 0.0017004151805301926, "grad_norm": 0.17208895087242126, "loss_scale": 4.0, "retries": 0, "seconds": 4755.6, "max_memory_gb": 19.17}
|
| 380 |
+
{"step": 380, "steps": 500, "lr": 4.594865525258959e-06, "loss": 4.8761318920882836e-05, "grad_norm": 0.0017838540952652693, "loss_scale": 4.0, "retries": 0, "seconds": 4768.3, "max_memory_gb": 19.17}
|
| 381 |
+
{"step": 381, "steps": 500, "lr": 4.5539619394777495e-06, "loss": 0.004902074275895014, "grad_norm": 0.2693362236022949, "loss_scale": 4.0, "retries": 0, "seconds": 4780.3, "max_memory_gb": 19.17}
|
| 382 |
+
{"step": 382, "steps": 500, "lr": 4.5133299353091496e-06, "loss": 0.013506809034104414, "grad_norm": 0.487946093082428, "loss_scale": 4.0, "retries": 0, "seconds": 4792.6, "max_memory_gb": 19.17}
|
| 383 |
+
{"step": 383, "steps": 500, "lr": 4.472971224642723e-06, "loss": 8.059437942620207e-05, "grad_norm": 0.0032466331031173468, "loss_scale": 4.0, "retries": 0, "seconds": 4805.0, "max_memory_gb": 19.17}
|
| 384 |
+
{"step": 384, "steps": 500, "lr": 4.43288750785373e-06, "loss": 0.00016903182322959188, "grad_norm": 0.008805685676634312, "loss_scale": 4.0, "retries": 0, "seconds": 4817.3, "max_memory_gb": 19.17}
|
| 385 |
+
{"step": 385, "steps": 500, "lr": 4.393080473731534e-06, "loss": 0.00013962814620330476, "grad_norm": 0.0069900136440992355, "loss_scale": 4.0, "retries": 0, "seconds": 4829.7, "max_memory_gb": 19.17}
|
| 386 |
+
{"step": 386, "steps": 500, "lr": 4.353551799408406e-06, "loss": 0.0005582225616649339, "grad_norm": 0.032074667513370514, "loss_scale": 4.0, "retries": 0, "seconds": 4842.0, "max_memory_gb": 19.17}
|
| 387 |
+
{"step": 387, "steps": 500, "lr": 4.314303150288909e-06, "loss": 0.020576564575032374, "grad_norm": 1.1091793775558472, "loss_scale": 4.0, "retries": 0, "seconds": 4854.4, "max_memory_gb": 19.17}
|
| 388 |
+
{"step": 388, "steps": 500, "lr": 4.275336179979683e-06, "loss": 7.534270438017384e-05, "grad_norm": 0.002957005985081196, "loss_scale": 4.0, "retries": 0, "seconds": 4867.0, "max_memory_gb": 19.17}
|
| 389 |
+
{"step": 389, "steps": 500, "lr": 4.236652530219822e-06, "loss": 0.0014532411960756647, "grad_norm": 0.0796135887503624, "loss_scale": 4.0, "retries": 0, "seconds": 4879.2, "max_memory_gb": 19.17}
|
| 390 |
+
{"step": 390, "steps": 500, "lr": 4.198253830811677e-06, "loss": 0.3414049692192173, "grad_norm": 4.5512495040893555, "loss_scale": 4.0, "retries": 0, "seconds": 4891.6, "max_memory_gb": 19.17}
|
| 391 |
+
{"step": 391, "steps": 500, "lr": 4.160141699552193e-06, "loss": 9.109386886310941e-05, "grad_norm": 0.0037259887903928757, "loss_scale": 4.0, "retries": 0, "seconds": 4904.1, "max_memory_gb": 19.17}
|
| 392 |
+
{"step": 392, "steps": 500, "lr": 4.122317742164759e-06, "loss": 0.015198234810862294, "grad_norm": 3.588336706161499, "loss_scale": 4.0, "retries": 0, "seconds": 4916.6, "max_memory_gb": 19.17}
|
| 393 |
+
{"step": 393, "steps": 500, "lr": 4.084783552231548e-06, "loss": 5.468839172935573e-05, "grad_norm": 0.0020639803260564804, "loss_scale": 4.0, "retries": 0, "seconds": 4929.1, "max_memory_gb": 19.17}
|
| 394 |
+
{"step": 394, "steps": 500, "lr": 4.047540711126381e-06, "loss": 0.0002165014780592339, "grad_norm": 0.011745176278054714, "loss_scale": 4.0, "retries": 0, "seconds": 4941.6, "max_memory_gb": 19.17}
|
| 395 |
+
{"step": 395, "steps": 500, "lr": 4.010590787948095e-06, "loss": 0.0016683485679219639, "grad_norm": 0.1221529096364975, "loss_scale": 4.0, "retries": 0, "seconds": 4954.1, "max_memory_gb": 19.17}
|
| 396 |
+
{"step": 396, "steps": 500, "lr": 3.973935339454447e-06, "loss": 9.810217942045215e-05, "grad_norm": 0.0067647965624928474, "loss_scale": 4.0, "retries": 0, "seconds": 4966.3, "max_memory_gb": 19.17}
|
| 397 |
+
{"step": 397, "steps": 500, "lr": 3.9375759099965105e-06, "loss": 0.025611380368766066, "grad_norm": 1.1795425415039062, "loss_scale": 4.0, "retries": 0, "seconds": 4978.7, "max_memory_gb": 19.17}
|
| 398 |
+
{"step": 398, "steps": 500, "lr": 3.901514031453619e-06, "loss": 0.015249734322635919, "grad_norm": 0.8049196004867554, "loss_scale": 4.0, "retries": 0, "seconds": 4990.8, "max_memory_gb": 19.17}
|
| 399 |
+
{"step": 399, "steps": 500, "lr": 3.865751223168823e-06, "loss": 0.5643505163543329, "grad_norm": 10.063736915588379, "loss_scale": 4.0, "retries": 0, "seconds": 5004.0, "max_memory_gb": 19.17}
|
| 400 |
+
{"step": 400, "steps": 500, "lr": 3.830288991884876e-06, "loss": 0.07401846350168483, "grad_norm": 1.6214014291763306, "loss_scale": 4.0, "retries": 0, "seconds": 5016.9, "max_memory_gb": 19.17}
|
| 401 |
+
{"step": 401, "steps": 500, "lr": 3.7951288316807574e-06, "loss": 0.0001607947578721988, "grad_norm": 0.008132078684866428, "loss_scale": 4.0, "retries": 0, "seconds": 5029.4, "max_memory_gb": 19.17}
|
| 402 |
+
{"step": 402, "steps": 500, "lr": 3.76027222390872e-06, "loss": 0.05441373705438159, "grad_norm": 1.0869462490081787, "loss_scale": 4.0, "retries": 0, "seconds": 5042.3, "max_memory_gb": 19.17}
|
| 403 |
+
{"step": 403, "steps": 500, "lr": 3.725720637131883e-06, "loss": 0.0006226317213056312, "grad_norm": 0.04529207572340965, "loss_scale": 4.0, "retries": 0, "seconds": 5054.8, "max_memory_gb": 19.17}
|
| 404 |
+
{"step": 404, "steps": 500, "lr": 3.6914755270623518e-06, "loss": 5.0328270124179084e-05, "grad_norm": 0.0023851613514125347, "loss_scale": 4.0, "retries": 0, "seconds": 5067.4, "max_memory_gb": 19.17}
|
| 405 |
+
{"step": 405, "steps": 500, "lr": 3.6575383364998937e-06, "loss": 0.0049671972065229155, "grad_norm": 0.25760549306869507, "loss_scale": 4.0, "retries": 0, "seconds": 5080.0, "max_memory_gb": 19.17}
|
| 406 |
+
{"step": 406, "steps": 500, "lr": 3.623910495271154e-06, "loss": 0.14227915049489392, "grad_norm": 7.8227410316467285, "loss_scale": 4.0, "retries": 0, "seconds": 5093.0, "max_memory_gb": 19.17}
|
| 407 |
+
{"step": 407, "steps": 500, "lr": 3.5905934201693992e-06, "loss": 0.00037204830077929404, "grad_norm": 0.026498345658183098, "loss_scale": 4.0, "retries": 0, "seconds": 5105.5, "max_memory_gb": 19.17}
|
| 408 |
+
{"step": 408, "steps": 500, "lr": 3.557588514894842e-06, "loss": 0.18842465494893448, "grad_norm": 2.402170419692993, "loss_scale": 4.0, "retries": 0, "seconds": 5118.4, "max_memory_gb": 19.17}
|
| 409 |
+
{"step": 409, "steps": 500, "lr": 3.524897169995492e-06, "loss": 0.0003788799798381992, "grad_norm": 0.02580391615629196, "loss_scale": 4.0, "retries": 0, "seconds": 5130.9, "max_memory_gb": 19.17}
|
| 410 |
+
{"step": 410, "steps": 500, "lr": 3.492520762808572e-06, "loss": 7.699997020567522e-05, "grad_norm": 0.0031088769901543856, "loss_scale": 4.0, "retries": 0, "seconds": 5143.4, "max_memory_gb": 19.17}
|
| 411 |
+
{"step": 411, "steps": 500, "lr": 3.4604606574024906e-06, "loss": 0.05702265893322853, "grad_norm": 2.998905658721924, "loss_scale": 4.0, "retries": 0, "seconds": 5156.0, "max_memory_gb": 19.17}
|
| 412 |
+
{"step": 412, "steps": 500, "lr": 3.428718204519368e-06, "loss": 0.23101292154126704, "grad_norm": 2.9563896656036377, "loss_scale": 4.0, "retries": 0, "seconds": 5168.3, "max_memory_gb": 19.17}
|
| 413 |
+
{"step": 413, "steps": 500, "lr": 3.3972947415181376e-06, "loss": 0.038176922683512464, "grad_norm": 1.738150954246521, "loss_scale": 4.0, "retries": 0, "seconds": 5180.9, "max_memory_gb": 19.17}
|
| 414 |
+
{"step": 414, "steps": 500, "lr": 3.3661915923181757e-06, "loss": 0.00016606778476877727, "grad_norm": 0.008000647649168968, "loss_scale": 4.0, "retries": 0, "seconds": 5193.4, "max_memory_gb": 19.17}
|
| 415 |
+
{"step": 415, "steps": 500, "lr": 3.335410067343559e-06, "loss": 0.22989241255103568, "grad_norm": 4.882193088531494, "loss_scale": 4.0, "retries": 0, "seconds": 5205.9, "max_memory_gb": 19.17}
|
| 416 |
+
{"step": 416, "steps": 500, "lr": 3.3049514634678196e-06, "loss": 0.00047759283138372086, "grad_norm": 0.026422539725899696, "loss_scale": 4.0, "retries": 0, "seconds": 5218.7, "max_memory_gb": 19.17}
|
| 417 |
+
{"step": 417, "steps": 500, "lr": 3.2748170639593357e-06, "loss": 0.00594751729574806, "grad_norm": 0.31235986948013306, "loss_scale": 4.0, "retries": 0, "seconds": 5231.4, "max_memory_gb": 19.17}
|
| 418 |
+
{"step": 418, "steps": 500, "lr": 3.245008138427235e-06, "loss": 0.0236214732357005, "grad_norm": 1.327041506767273, "loss_scale": 4.0, "retries": 0, "seconds": 5244.1, "max_memory_gb": 19.17}
|
| 419 |
+
{"step": 419, "steps": 500, "lr": 3.215525942767934e-06, "loss": 0.00012002029960843753, "grad_norm": 0.005063994787633419, "loss_scale": 4.0, "retries": 0, "seconds": 5256.8, "max_memory_gb": 19.17}
|
| 420 |
+
{"step": 420, "steps": 500, "lr": 3.1863717191122057e-06, "loss": 0.08676443340226052, "grad_norm": 4.598430633544922, "loss_scale": 4.0, "retries": 0, "seconds": 5269.7, "max_memory_gb": 19.17}
|
| 421 |
+
{"step": 421, "steps": 500, "lr": 3.1575466957728514e-06, "loss": 0.0006367704681906617, "grad_norm": 0.02561657875776291, "loss_scale": 4.0, "retries": 0, "seconds": 5282.5, "max_memory_gb": 19.17}
|
| 422 |
+
{"step": 422, "steps": 500, "lr": 3.129052087192952e-06, "loss": 0.009546319182008618, "grad_norm": 0.3673267066478729, "loss_scale": 4.0, "retries": 0, "seconds": 5295.4, "max_memory_gb": 19.17}
|
| 423 |
+
{"step": 423, "steps": 500, "lr": 3.1008890938946993e-06, "loss": 9.423885782408352e-05, "grad_norm": 0.003495278535410762, "loss_scale": 4.0, "retries": 0, "seconds": 5308.2, "max_memory_gb": 19.17}
|
| 424 |
+
{"step": 424, "steps": 500, "lr": 3.0730589024288223e-06, "loss": 0.0007253844813238754, "grad_norm": 0.03323390334844589, "loss_scale": 4.0, "retries": 0, "seconds": 5320.9, "max_memory_gb": 19.17}
|
| 425 |
+
{"step": 425, "steps": 500, "lr": 3.0455626853245828e-06, "loss": 0.03546485733705719, "grad_norm": 1.2958229780197144, "loss_scale": 4.0, "retries": 0, "seconds": 5333.2, "max_memory_gb": 19.17}
|
| 426 |
+
{"step": 426, "steps": 500, "lr": 3.0184016010403933e-06, "loss": 0.0052895360262875165, "grad_norm": 0.21841749548912048, "loss_scale": 4.0, "retries": 0, "seconds": 5346.2, "max_memory_gb": 19.17}
|
| 427 |
+
{"step": 427, "steps": 500, "lr": 2.9915767939149864e-06, "loss": 0.0004699575089261998, "grad_norm": 0.042620476335287094, "loss_scale": 4.0, "retries": 0, "seconds": 5358.9, "max_memory_gb": 19.17}
|
| 428 |
+
{"step": 428, "steps": 500, "lr": 2.9650893941192267e-06, "loss": 0.014590223902881791, "grad_norm": 0.5136966109275818, "loss_scale": 4.0, "retries": 0, "seconds": 5371.5, "max_memory_gb": 19.17}
|
| 429 |
+
{"step": 429, "steps": 500, "lr": 2.9389405176084782e-06, "loss": 0.00041419699476819005, "grad_norm": 0.020021652802824974, "loss_scale": 4.0, "retries": 0, "seconds": 5383.9, "max_memory_gb": 19.17}
|
| 430 |
+
{"step": 430, "steps": 500, "lr": 2.9131312660755903e-06, "loss": 7.490737903284837e-05, "grad_norm": 0.0029473621398210526, "loss_scale": 4.0, "retries": 0, "seconds": 5396.6, "max_memory_gb": 19.17}
|
| 431 |
+
{"step": 431, "steps": 500, "lr": 2.887662726904487e-06, "loss": 0.08645219627368825, "grad_norm": 10.466649055480957, "loss_scale": 4.0, "retries": 0, "seconds": 5409.2, "max_memory_gb": 19.17}
|
| 432 |
+
{"step": 432, "steps": 500, "lr": 2.8625359731243483e-06, "loss": 0.002541551977432475, "grad_norm": 0.13436557352542877, "loss_scale": 4.0, "retries": 0, "seconds": 5421.9, "max_memory_gb": 19.17}
|
| 433 |
+
{"step": 433, "steps": 500, "lr": 2.837752063364405e-06, "loss": 0.005240920496284864, "grad_norm": 0.6005041599273682, "loss_scale": 4.0, "retries": 0, "seconds": 5434.8, "max_memory_gb": 19.17}
|
| 434 |
+
{"step": 434, "steps": 500, "lr": 2.813312041809334e-06, "loss": 0.006179804642597858, "grad_norm": 0.6594776511192322, "loss_scale": 4.0, "retries": 0, "seconds": 5447.4, "max_memory_gb": 19.17}
|
| 435 |
+
{"step": 435, "steps": 500, "lr": 2.7892169381552727e-06, "loss": 0.006623588911842468, "grad_norm": 1.3039469718933105, "loss_scale": 4.0, "retries": 0, "seconds": 5460.1, "max_memory_gb": 19.17}
|
| 436 |
+
{"step": 436, "steps": 500, "lr": 2.7654677675664216e-06, "loss": 0.004018722616716664, "grad_norm": 0.42941582202911377, "loss_scale": 4.0, "retries": 0, "seconds": 5472.7, "max_memory_gb": 19.17}
|
| 437 |
+
{"step": 437, "steps": 500, "lr": 2.742065530632291e-06, "loss": 0.004122591758418408, "grad_norm": 0.16385237872600555, "loss_scale": 4.0, "retries": 0, "seconds": 5485.2, "max_memory_gb": 19.17}
|
| 438 |
+
{"step": 438, "steps": 500, "lr": 2.719011213325533e-06, "loss": 0.00010357547073040639, "grad_norm": 0.004518157336860895, "loss_scale": 4.0, "retries": 0, "seconds": 5497.7, "max_memory_gb": 19.17}
|
| 439 |
+
{"step": 439, "steps": 500, "lr": 2.6963057869604053e-06, "loss": 0.0008561668588061946, "grad_norm": 0.03910350427031517, "loss_scale": 4.0, "retries": 0, "seconds": 5510.9, "max_memory_gb": 19.17}
|
| 440 |
+
{"step": 440, "steps": 500, "lr": 2.6739502081518463e-06, "loss": 0.0004370461171096064, "grad_norm": 0.02169892191886902, "loss_scale": 4.0, "retries": 0, "seconds": 5523.5, "max_memory_gb": 19.17}
|
| 441 |
+
{"step": 441, "steps": 500, "lr": 2.651945418775174e-06, "loss": 0.012907799079336906, "grad_norm": 0.49696433544158936, "loss_scale": 4.0, "retries": 0, "seconds": 5536.1, "max_memory_gb": 19.17}
|
| 442 |
+
{"step": 442, "steps": 500, "lr": 2.6302923459264005e-06, "loss": 0.006864972074688261, "grad_norm": 2.2987430095672607, "loss_scale": 4.0, "retries": 0, "seconds": 5549.0, "max_memory_gb": 19.17}
|
| 443 |
+
{"step": 443, "steps": 500, "lr": 2.608991901883174e-06, "loss": 0.003040473964822675, "grad_norm": 0.1554512232542038, "loss_scale": 4.0, "retries": 0, "seconds": 5561.7, "max_memory_gb": 19.17}
|
| 444 |
+
{"step": 444, "steps": 500, "lr": 2.588044984066341e-06, "loss": 0.0028326659950618127, "grad_norm": 0.4618130028247833, "loss_scale": 4.0, "retries": 0, "seconds": 5574.4, "max_memory_gb": 19.17}
|
| 445 |
+
{"step": 445, "steps": 500, "lr": 2.5674524750021456e-06, "loss": 0.00011566560764012222, "grad_norm": 0.00491959135979414, "loss_scale": 4.0, "retries": 0, "seconds": 5587.0, "max_memory_gb": 19.17}
|
| 446 |
+
{"step": 446, "steps": 500, "lr": 2.5472152422850257e-06, "loss": 0.1399058269080859, "grad_norm": 9.115571022033691, "loss_scale": 4.0, "retries": 0, "seconds": 5599.7, "max_memory_gb": 19.17}
|
| 447 |
+
{"step": 447, "steps": 500, "lr": 2.5273341385410883e-06, "loss": 0.007430031594700703, "grad_norm": 0.568047046661377, "loss_scale": 4.0, "retries": 0, "seconds": 5612.5, "max_memory_gb": 19.17}
|
| 448 |
+
{"step": 448, "steps": 500, "lr": 2.507810001392163e-06, "loss": 0.005099083882317824, "grad_norm": 0.371329665184021, "loss_scale": 4.0, "retries": 0, "seconds": 5625.2, "max_memory_gb": 19.17}
|
| 449 |
+
{"step": 449, "steps": 500, "lr": 2.488643653420525e-06, "loss": 0.0002591992754384426, "grad_norm": 0.017424412071704865, "loss_scale": 4.0, "retries": 0, "seconds": 5638.2, "max_memory_gb": 19.17}
|
| 450 |
+
{"step": 450, "steps": 500, "lr": 2.4698359021342357e-06, "loss": 0.06126168597763188, "grad_norm": 1.5254740715026855, "loss_scale": 4.0, "retries": 0, "seconds": 5650.8, "max_memory_gb": 19.17}
|
| 451 |
+
{"step": 451, "steps": 500, "lr": 2.451387539933117e-06, "loss": 0.005044863809305608, "grad_norm": 0.5271198749542236, "loss_scale": 4.0, "retries": 0, "seconds": 5663.4, "max_memory_gb": 19.17}
|
| 452 |
+
{"step": 452, "steps": 500, "lr": 2.4332993440753743e-06, "loss": 0.0026974533296905434, "grad_norm": 0.10406762361526489, "loss_scale": 4.0, "retries": 0, "seconds": 5676.1, "max_memory_gb": 19.17}
|
| 453 |
+
{"step": 453, "steps": 500, "lr": 2.415572076644838e-06, "loss": 0.0017644238560334006, "grad_norm": 0.08415082842111588, "loss_scale": 4.0, "retries": 0, "seconds": 5688.6, "max_memory_gb": 19.17}
|
| 454 |
+
{"step": 454, "steps": 500, "lr": 2.3982064845188708e-06, "loss": 0.0016159898333295462, "grad_norm": 0.0764935314655304, "loss_scale": 4.0, "retries": 0, "seconds": 5701.4, "max_memory_gb": 19.17}
|
| 455 |
+
{"step": 455, "steps": 500, "lr": 2.3812032993368828e-06, "loss": 0.0017672924658478969, "grad_norm": 0.3587868809700012, "loss_scale": 4.0, "retries": 0, "seconds": 5714.1, "max_memory_gb": 19.17}
|
| 456 |
+
{"step": 456, "steps": 500, "lr": 2.3645632374695245e-06, "loss": 0.039855882070541665, "grad_norm": 0.7570313215255737, "loss_scale": 4.0, "retries": 0, "seconds": 5726.4, "max_memory_gb": 19.17}
|
| 457 |
+
{"step": 457, "steps": 500, "lr": 2.3482869999884874e-06, "loss": 0.026938180150582722, "grad_norm": 1.7329963445663452, "loss_scale": 4.0, "retries": 0, "seconds": 5738.9, "max_memory_gb": 19.17}
|
| 458 |
+
{"step": 458, "steps": 500, "lr": 2.332375272636983e-06, "loss": 8.926376896764313e-05, "grad_norm": 0.003464092267677188, "loss_scale": 4.0, "retries": 0, "seconds": 5751.7, "max_memory_gb": 19.17}
|
| 459 |
+
{"step": 459, "steps": 500, "lr": 2.316828725800841e-06, "loss": 0.00044345563212676677, "grad_norm": 0.022977206856012344, "loss_scale": 4.0, "retries": 0, "seconds": 5764.4, "max_memory_gb": 19.17}
|
| 460 |
+
{"step": 460, "steps": 500, "lr": 2.301648014480267e-06, "loss": 0.0031944831309984193, "grad_norm": 0.14539945125579834, "loss_scale": 4.0, "retries": 0, "seconds": 5777.1, "max_memory_gb": 19.17}
|
| 461 |
+
{"step": 461, "steps": 500, "lr": 2.2868337782622484e-06, "loss": 0.00020146242617225988, "grad_norm": 0.009565298445522785, "loss_scale": 4.0, "retries": 0, "seconds": 5789.8, "max_memory_gb": 19.17}
|
| 462 |
+
{"step": 462, "steps": 500, "lr": 2.272386641293606e-06, "loss": 0.00015948710619992116, "grad_norm": 0.018332447856664658, "loss_scale": 4.0, "retries": 0, "seconds": 5802.6, "max_memory_gb": 19.17}
|
| 463 |
+
{"step": 463, "steps": 500, "lr": 2.258307212254698e-06, "loss": 0.00013633410356884596, "grad_norm": 0.007413444109261036, "loss_scale": 4.0, "retries": 0, "seconds": 5815.4, "max_memory_gb": 19.17}
|
| 464 |
+
{"step": 464, "steps": 500, "lr": 2.2445960843337744e-06, "loss": 0.02368515792304555, "grad_norm": 1.580428123474121, "loss_scale": 4.0, "retries": 0, "seconds": 5827.9, "max_memory_gb": 19.17}
|
| 465 |
+
{"step": 465, "steps": 500, "lr": 2.231253835201989e-06, "loss": 5.852070892780666e-05, "grad_norm": 0.002281497698277235, "loss_scale": 4.0, "retries": 0, "seconds": 5840.9, "max_memory_gb": 19.17}
|
| 466 |
+
{"step": 466, "steps": 500, "lr": 2.218281026989053e-06, "loss": 0.04193388030524209, "grad_norm": 1.1379765272140503, "loss_scale": 4.0, "retries": 0, "seconds": 5853.9, "max_memory_gb": 19.17}
|
| 467 |
+
{"step": 467, "steps": 500, "lr": 2.2056782062595655e-06, "loss": 0.00021743518749417498, "grad_norm": 0.01815137267112732, "loss_scale": 4.0, "retries": 0, "seconds": 5866.6, "max_memory_gb": 19.17}
|
| 468 |
+
{"step": 468, "steps": 500, "lr": 2.193445903989966e-06, "loss": 0.10460946017904216, "grad_norm": 11.445799827575684, "loss_scale": 4.0, "retries": 0, "seconds": 5879.0, "max_memory_gb": 19.17}
|
| 469 |
+
{"step": 469, "steps": 500, "lr": 2.1815846355461853e-06, "loss": 5.686989565199177e-05, "grad_norm": 0.0024192258715629578, "loss_scale": 4.0, "retries": 0, "seconds": 5891.9, "max_memory_gb": 19.17}
|
| 470 |
+
{"step": 470, "steps": 500, "lr": 2.1700949006619163e-06, "loss": 0.00035915833715514304, "grad_norm": 0.029642650857567787, "loss_scale": 4.0, "retries": 0, "seconds": 5904.6, "max_memory_gb": 19.17}
|
| 471 |
+
{"step": 471, "steps": 500, "lr": 2.158977183417566e-06, "loss": 0.014904191591238139, "grad_norm": 1.3378701210021973, "loss_scale": 4.0, "retries": 0, "seconds": 5917.2, "max_memory_gb": 19.17}
|
| 472 |
+
{"step": 472, "steps": 500, "lr": 2.1482319522198587e-06, "loss": 8.133144226783109e-05, "grad_norm": 0.0041050962172448635, "loss_scale": 4.0, "retries": 0, "seconds": 5930.1, "max_memory_gb": 19.17}
|
| 473 |
+
{"step": 473, "steps": 500, "lr": 2.137859659782103e-06, "loss": 0.008106274884684437, "grad_norm": 1.0439542531967163, "loss_scale": 4.0, "retries": 0, "seconds": 5942.8, "max_memory_gb": 19.17}
|
| 474 |
+
{"step": 474, "steps": 500, "lr": 2.1278607431051185e-06, "loss": 0.006596562142290582, "grad_norm": 0.9469068050384521, "loss_scale": 4.0, "retries": 0, "seconds": 5955.5, "max_memory_gb": 19.17}
|
| 475 |
+
{"step": 475, "steps": 500, "lr": 2.1182356234588196e-06, "loss": 0.00023811668954852973, "grad_norm": 0.015487522818148136, "loss_scale": 4.0, "retries": 0, "seconds": 5968.2, "max_memory_gb": 19.17}
|
| 476 |
+
{"step": 476, "steps": 500, "lr": 2.1089847063644735e-06, "loss": 0.0009833040513688668, "grad_norm": 0.04603530466556549, "loss_scale": 4.0, "retries": 0, "seconds": 5980.6, "max_memory_gb": 19.17}
|
| 477 |
+
{"step": 477, "steps": 500, "lr": 2.100108381577611e-06, "loss": 0.11427207060895936, "grad_norm": 4.364441394805908, "loss_scale": 4.0, "retries": 0, "seconds": 5993.3, "max_memory_gb": 19.17}
|
| 478 |
+
{"step": 478, "steps": 500, "lr": 2.091607023071606e-06, "loss": 0.021885449016451723, "grad_norm": 0.914095401763916, "loss_scale": 4.0, "retries": 0, "seconds": 6006.0, "max_memory_gb": 19.17}
|
| 479 |
+
{"step": 479, "steps": 500, "lr": 2.0834809890219166e-06, "loss": 0.037303381522419125, "grad_norm": 2.5529510974884033, "loss_scale": 4.0, "retries": 0, "seconds": 6019.1, "max_memory_gb": 19.17}
|
| 480 |
+
{"step": 480, "steps": 500, "lr": 2.0757306217910033e-06, "loss": 7.552680481963266e-05, "grad_norm": 0.003466052934527397, "loss_scale": 4.0, "retries": 0, "seconds": 6031.8, "max_memory_gb": 19.17}
|
| 481 |
+
{"step": 481, "steps": 500, "lr": 2.068356247913894e-06, "loss": 0.07399387924642105, "grad_norm": 1.7366958856582642, "loss_scale": 4.0, "retries": 0, "seconds": 6044.5, "max_memory_gb": 19.17}
|
| 482 |
+
{"step": 482, "steps": 500, "lr": 2.0613581780844348e-06, "loss": 0.00014913400431995882, "grad_norm": 0.0113829979673028, "loss_scale": 4.0, "retries": 0, "seconds": 6057.3, "max_memory_gb": 19.17}
|
| 483 |
+
{"step": 483, "steps": 500, "lr": 2.0547367071421955e-06, "loss": 0.000631452350497532, "grad_norm": 0.055082112550735474, "loss_scale": 4.0, "retries": 0, "seconds": 6070.1, "max_memory_gb": 19.17}
|
| 484 |
+
{"step": 484, "steps": 500, "lr": 2.048492114060048e-06, "loss": 0.0004171084475604982, "grad_norm": 0.029150288552045822, "loss_scale": 4.0, "retries": 0, "seconds": 6082.8, "max_memory_gb": 19.17}
|
| 485 |
+
{"step": 485, "steps": 500, "lr": 2.0426246619324153e-06, "loss": 0.0015843572456436632, "grad_norm": 0.20342902839183807, "loss_scale": 4.0, "retries": 0, "seconds": 6095.6, "max_memory_gb": 19.17}
|
| 486 |
+
{"step": 486, "steps": 500, "lr": 2.0371345979641848e-06, "loss": 0.00358810525566417, "grad_norm": 0.369268536567688, "loss_scale": 4.0, "retries": 0, "seconds": 6108.3, "max_memory_gb": 19.17}
|
| 487 |
+
{"step": 487, "steps": 500, "lr": 2.032022153460292e-06, "loss": 4.6738112742290966e-05, "grad_norm": 0.001670673256739974, "loss_scale": 4.0, "retries": 0, "seconds": 6121.0, "max_memory_gb": 19.17}
|
| 488 |
+
{"step": 488, "steps": 500, "lr": 2.0272875438159797e-06, "loss": 0.1289277503540208, "grad_norm": 3.8611316680908203, "loss_scale": 4.0, "retries": 0, "seconds": 6133.7, "max_memory_gb": 19.17}
|
| 489 |
+
{"step": 489, "steps": 500, "lr": 2.022930968507718e-06, "loss": 0.36808285752238135, "grad_norm": 6.472497463226318, "loss_scale": 4.0, "retries": 0, "seconds": 6146.4, "max_memory_gb": 19.17}
|
| 490 |
+
{"step": 490, "steps": 500, "lr": 2.0189526110848026e-06, "loss": 0.19088877949694838, "grad_norm": 3.804316997528076, "loss_scale": 4.0, "retries": 0, "seconds": 6159.2, "max_memory_gb": 19.17}
|
| 491 |
+
{"step": 491, "steps": 500, "lr": 2.0153526391616207e-06, "loss": 0.00016673830447705936, "grad_norm": 0.007550959009677172, "loss_scale": 4.0, "retries": 0, "seconds": 6171.9, "max_memory_gb": 19.17}
|
| 492 |
+
{"step": 492, "steps": 500, "lr": 2.0121312044105927e-06, "loss": 0.03995604700111761, "grad_norm": 3.3923494815826416, "loss_scale": 4.0, "retries": 0, "seconds": 6184.4, "max_memory_gb": 19.17}
|
| 493 |
+
{"step": 493, "steps": 500, "lr": 2.009288442555774e-06, "loss": 0.0015057290727017144, "grad_norm": 0.05684838071465492, "loss_scale": 4.0, "retries": 0, "seconds": 6197.1, "max_memory_gb": 19.17}
|
| 494 |
+
{"step": 494, "steps": 500, "lr": 2.0068244733671474e-06, "loss": 0.006558311886931538, "grad_norm": 1.5262912511825562, "loss_scale": 4.0, "retries": 0, "seconds": 6209.8, "max_memory_gb": 19.17}
|
| 495 |
+
{"step": 495, "steps": 500, "lr": 2.004739400655566e-06, "loss": 0.0002398104689508397, "grad_norm": 0.013527629896998405, "loss_scale": 4.0, "retries": 0, "seconds": 6222.5, "max_memory_gb": 19.17}
|
| 496 |
+
{"step": 496, "steps": 500, "lr": 2.003033312268388e-06, "loss": 0.003191179822388015, "grad_norm": 0.16540350019931793, "loss_scale": 4.0, "retries": 0, "seconds": 6235.1, "max_memory_gb": 19.17}
|
| 497 |
+
{"step": 497, "steps": 500, "lr": 2.0017062800857716e-06, "loss": 0.06674635631793664, "grad_norm": 5.095034599304199, "loss_scale": 4.0, "retries": 0, "seconds": 6247.5, "max_memory_gb": 19.17}
|
| 498 |
+
{"step": 498, "steps": 500, "lr": 2.000758360017648e-06, "loss": 0.06695255352620548, "grad_norm": 3.0819790363311768, "loss_scale": 4.0, "retries": 0, "seconds": 6260.3, "max_memory_gb": 19.17}
|
| 499 |
+
{"step": 499, "steps": 500, "lr": 2.000189592001363e-06, "loss": 0.00015542290923452708, "grad_norm": 0.006749611347913742, "loss_scale": 4.0, "retries": 0, "seconds": 6273.0, "max_memory_gb": 19.17}
|
| 500 |
+
{"step": 500, "steps": 500, "lr": 2e-06, "loss": 0.001497570351433679, "grad_norm": 0.11838455498218536, "loss_scale": 4.0, "retries": 0, "seconds": 6285.8, "max_memory_gb": 19.17}
|
runs/joint_qwen/joint/seed42/adapter/README.md
ADDED
|
@@ -0,0 +1,207 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
base_model: /data/OPSD/assets/models/Qwen3.5-9B
|
| 3 |
+
library_name: peft
|
| 4 |
+
pipeline_tag: text-generation
|
| 5 |
+
tags:
|
| 6 |
+
- base_model:adapter:/data/OPSD/assets/models/Qwen3.5-9B
|
| 7 |
+
- lora
|
| 8 |
+
- transformers
|
| 9 |
+
---
|
| 10 |
+
|
| 11 |
+
# Model Card for Model ID
|
| 12 |
+
|
| 13 |
+
<!-- Provide a quick summary of what the model is/does. -->
|
| 14 |
+
|
| 15 |
+
|
| 16 |
+
|
| 17 |
+
## Model Details
|
| 18 |
+
|
| 19 |
+
### Model Description
|
| 20 |
+
|
| 21 |
+
<!-- Provide a longer summary of what this model is. -->
|
| 22 |
+
|
| 23 |
+
|
| 24 |
+
|
| 25 |
+
- **Developed by:** [More Information Needed]
|
| 26 |
+
- **Funded by [optional]:** [More Information Needed]
|
| 27 |
+
- **Shared by [optional]:** [More Information Needed]
|
| 28 |
+
- **Model type:** [More Information Needed]
|
| 29 |
+
- **Language(s) (NLP):** [More Information Needed]
|
| 30 |
+
- **License:** [More Information Needed]
|
| 31 |
+
- **Finetuned from model [optional]:** [More Information Needed]
|
| 32 |
+
|
| 33 |
+
### Model Sources [optional]
|
| 34 |
+
|
| 35 |
+
<!-- Provide the basic links for the model. -->
|
| 36 |
+
|
| 37 |
+
- **Repository:** [More Information Needed]
|
| 38 |
+
- **Paper [optional]:** [More Information Needed]
|
| 39 |
+
- **Demo [optional]:** [More Information Needed]
|
| 40 |
+
|
| 41 |
+
## Uses
|
| 42 |
+
|
| 43 |
+
<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
|
| 44 |
+
|
| 45 |
+
### Direct Use
|
| 46 |
+
|
| 47 |
+
<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
|
| 48 |
+
|
| 49 |
+
[More Information Needed]
|
| 50 |
+
|
| 51 |
+
### Downstream Use [optional]
|
| 52 |
+
|
| 53 |
+
<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
|
| 54 |
+
|
| 55 |
+
[More Information Needed]
|
| 56 |
+
|
| 57 |
+
### Out-of-Scope Use
|
| 58 |
+
|
| 59 |
+
<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
|
| 60 |
+
|
| 61 |
+
[More Information Needed]
|
| 62 |
+
|
| 63 |
+
## Bias, Risks, and Limitations
|
| 64 |
+
|
| 65 |
+
<!-- This section is meant to convey both technical and sociotechnical limitations. -->
|
| 66 |
+
|
| 67 |
+
[More Information Needed]
|
| 68 |
+
|
| 69 |
+
### Recommendations
|
| 70 |
+
|
| 71 |
+
<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
|
| 72 |
+
|
| 73 |
+
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
|
| 74 |
+
|
| 75 |
+
## How to Get Started with the Model
|
| 76 |
+
|
| 77 |
+
Use the code below to get started with the model.
|
| 78 |
+
|
| 79 |
+
[More Information Needed]
|
| 80 |
+
|
| 81 |
+
## Training Details
|
| 82 |
+
|
| 83 |
+
### Training Data
|
| 84 |
+
|
| 85 |
+
<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
|
| 86 |
+
|
| 87 |
+
[More Information Needed]
|
| 88 |
+
|
| 89 |
+
### Training Procedure
|
| 90 |
+
|
| 91 |
+
<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
|
| 92 |
+
|
| 93 |
+
#### Preprocessing [optional]
|
| 94 |
+
|
| 95 |
+
[More Information Needed]
|
| 96 |
+
|
| 97 |
+
|
| 98 |
+
#### Training Hyperparameters
|
| 99 |
+
|
| 100 |
+
- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
|
| 101 |
+
|
| 102 |
+
#### Speeds, Sizes, Times [optional]
|
| 103 |
+
|
| 104 |
+
<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
|
| 105 |
+
|
| 106 |
+
[More Information Needed]
|
| 107 |
+
|
| 108 |
+
## Evaluation
|
| 109 |
+
|
| 110 |
+
<!-- This section describes the evaluation protocols and provides the results. -->
|
| 111 |
+
|
| 112 |
+
### Testing Data, Factors & Metrics
|
| 113 |
+
|
| 114 |
+
#### Testing Data
|
| 115 |
+
|
| 116 |
+
<!-- This should link to a Dataset Card if possible. -->
|
| 117 |
+
|
| 118 |
+
[More Information Needed]
|
| 119 |
+
|
| 120 |
+
#### Factors
|
| 121 |
+
|
| 122 |
+
<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
|
| 123 |
+
|
| 124 |
+
[More Information Needed]
|
| 125 |
+
|
| 126 |
+
#### Metrics
|
| 127 |
+
|
| 128 |
+
<!-- These are the evaluation metrics being used, ideally with a description of why. -->
|
| 129 |
+
|
| 130 |
+
[More Information Needed]
|
| 131 |
+
|
| 132 |
+
### Results
|
| 133 |
+
|
| 134 |
+
[More Information Needed]
|
| 135 |
+
|
| 136 |
+
#### Summary
|
| 137 |
+
|
| 138 |
+
|
| 139 |
+
|
| 140 |
+
## Model Examination [optional]
|
| 141 |
+
|
| 142 |
+
<!-- Relevant interpretability work for the model goes here -->
|
| 143 |
+
|
| 144 |
+
[More Information Needed]
|
| 145 |
+
|
| 146 |
+
## Environmental Impact
|
| 147 |
+
|
| 148 |
+
<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
|
| 149 |
+
|
| 150 |
+
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
|
| 151 |
+
|
| 152 |
+
- **Hardware Type:** [More Information Needed]
|
| 153 |
+
- **Hours used:** [More Information Needed]
|
| 154 |
+
- **Cloud Provider:** [More Information Needed]
|
| 155 |
+
- **Compute Region:** [More Information Needed]
|
| 156 |
+
- **Carbon Emitted:** [More Information Needed]
|
| 157 |
+
|
| 158 |
+
## Technical Specifications [optional]
|
| 159 |
+
|
| 160 |
+
### Model Architecture and Objective
|
| 161 |
+
|
| 162 |
+
[More Information Needed]
|
| 163 |
+
|
| 164 |
+
### Compute Infrastructure
|
| 165 |
+
|
| 166 |
+
[More Information Needed]
|
| 167 |
+
|
| 168 |
+
#### Hardware
|
| 169 |
+
|
| 170 |
+
[More Information Needed]
|
| 171 |
+
|
| 172 |
+
#### Software
|
| 173 |
+
|
| 174 |
+
[More Information Needed]
|
| 175 |
+
|
| 176 |
+
## Citation [optional]
|
| 177 |
+
|
| 178 |
+
<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
|
| 179 |
+
|
| 180 |
+
**BibTeX:**
|
| 181 |
+
|
| 182 |
+
[More Information Needed]
|
| 183 |
+
|
| 184 |
+
**APA:**
|
| 185 |
+
|
| 186 |
+
[More Information Needed]
|
| 187 |
+
|
| 188 |
+
## Glossary [optional]
|
| 189 |
+
|
| 190 |
+
<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
|
| 191 |
+
|
| 192 |
+
[More Information Needed]
|
| 193 |
+
|
| 194 |
+
## More Information [optional]
|
| 195 |
+
|
| 196 |
+
[More Information Needed]
|
| 197 |
+
|
| 198 |
+
## Model Card Authors [optional]
|
| 199 |
+
|
| 200 |
+
[More Information Needed]
|
| 201 |
+
|
| 202 |
+
## Model Card Contact
|
| 203 |
+
|
| 204 |
+
[More Information Needed]
|
| 205 |
+
### Framework versions
|
| 206 |
+
|
| 207 |
+
- PEFT 0.20.0
|
runs/joint_qwen/joint/seed42/adapter/adapter_config.json
ADDED
|
@@ -0,0 +1,55 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"alora_invocation_tokens": null,
|
| 3 |
+
"alpha_pattern": {},
|
| 4 |
+
"arrow_config": null,
|
| 5 |
+
"auto_mapping": null,
|
| 6 |
+
"base_model_name_or_path": "/data/OPSD/assets/models/Qwen3.5-9B",
|
| 7 |
+
"bias": "none",
|
| 8 |
+
"corda_config": null,
|
| 9 |
+
"ensure_weight_tying": false,
|
| 10 |
+
"eva_config": null,
|
| 11 |
+
"exclude_modules": null,
|
| 12 |
+
"fan_in_fan_out": false,
|
| 13 |
+
"inference_mode": true,
|
| 14 |
+
"init_lora_weights": true,
|
| 15 |
+
"layer_replication": null,
|
| 16 |
+
"layers_pattern": null,
|
| 17 |
+
"layers_to_transform": null,
|
| 18 |
+
"loftq_config": {},
|
| 19 |
+
"lora_alpha": 32,
|
| 20 |
+
"lora_bias": false,
|
| 21 |
+
"lora_dropout": 0.05,
|
| 22 |
+
"lora_ga_config": null,
|
| 23 |
+
"megatron_config": null,
|
| 24 |
+
"megatron_core": "megatron.core",
|
| 25 |
+
"modules_to_save": null,
|
| 26 |
+
"monteclora_config": null,
|
| 27 |
+
"peft_type": "LORA",
|
| 28 |
+
"peft_version": "0.20.0",
|
| 29 |
+
"qalora_group_size": 16,
|
| 30 |
+
"r": 16,
|
| 31 |
+
"rank_pattern": {},
|
| 32 |
+
"revision": null,
|
| 33 |
+
"target_modules": [
|
| 34 |
+
"in_proj_b",
|
| 35 |
+
"in_proj_a",
|
| 36 |
+
"in_proj_z",
|
| 37 |
+
"q_proj",
|
| 38 |
+
"up_proj",
|
| 39 |
+
"gate_proj",
|
| 40 |
+
"down_proj",
|
| 41 |
+
"out_proj",
|
| 42 |
+
"o_proj",
|
| 43 |
+
"k_proj",
|
| 44 |
+
"in_proj_qkv",
|
| 45 |
+
"v_proj"
|
| 46 |
+
],
|
| 47 |
+
"target_parameters": null,
|
| 48 |
+
"task_type": "CAUSAL_LM",
|
| 49 |
+
"trainable_token_indices": null,
|
| 50 |
+
"use_bdlora": null,
|
| 51 |
+
"use_dora": false,
|
| 52 |
+
"use_qalora": false,
|
| 53 |
+
"use_rslora": false,
|
| 54 |
+
"velora_config": null
|
| 55 |
+
}
|
runs/joint_qwen/joint/seed42/adapter/adapter_model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:74b60ce5f62b6b62d7ace14fe72bbc2a13eb42a8105a14027c636f8fbb479afe
|
| 3 |
+
size 173188512
|
runs/joint_qwen/joint/seed42/config.json
ADDED
|
@@ -0,0 +1,32 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"seed": 42,
|
| 3 |
+
"model": "/data/OPSD/assets/models/Qwen3.5-9B",
|
| 4 |
+
"backbone": "qwen3_5",
|
| 5 |
+
"train": "/data/OPSD/work/joint/joint.jsonl",
|
| 6 |
+
"records": 1000,
|
| 7 |
+
"steps": 500,
|
| 8 |
+
"steps_run": 500,
|
| 9 |
+
"category_weights": {
|
| 10 |
+
"original": 1.0,
|
| 11 |
+
"removal": 1.0,
|
| 12 |
+
"replacement": 1.0,
|
| 13 |
+
"context": 0.5,
|
| 14 |
+
"no_image": 0.02
|
| 15 |
+
},
|
| 16 |
+
"task_weight": 0.5,
|
| 17 |
+
"phases": [
|
| 18 |
+
{
|
| 19 |
+
"color": "b",
|
| 20 |
+
"count": "b"
|
| 21 |
+
}
|
| 22 |
+
],
|
| 23 |
+
"lora_rank": 16,
|
| 24 |
+
"lr": {
|
| 25 |
+
"peak": 2e-05,
|
| 26 |
+
"warmup": 16,
|
| 27 |
+
"final": 2e-06,
|
| 28 |
+
"decay": "cosine"
|
| 29 |
+
},
|
| 30 |
+
"clip": 1.0,
|
| 31 |
+
"initial_loss_scale": 4.0
|
| 32 |
+
}
|
runs/joint_qwen/joint/seed42/steps.jsonl
ADDED
|
@@ -0,0 +1,500 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{"step": 1, "steps": 500, "lr": 1.25e-06, "loss": 3.5424784503199844, "grad_norm": 44.1450309753418, "loss_scale": 4.0, "retries": 0, "seconds": 15.7, "max_memory_gb": 18.54}
|
| 2 |
+
{"step": 2, "steps": 500, "lr": 2.5e-06, "loss": 3.2844074165279213, "grad_norm": 46.27538299560547, "loss_scale": 4.0, "retries": 0, "seconds": 28.2, "max_memory_gb": 18.96}
|
| 3 |
+
{"step": 3, "steps": 500, "lr": 3.7500000000000005e-06, "loss": 3.75298415937207, "grad_norm": 43.78131866455078, "loss_scale": 4.0, "retries": 0, "seconds": 40.8, "max_memory_gb": 19.15}
|
| 4 |
+
{"step": 4, "steps": 500, "lr": 5e-06, "loss": 3.21255869824778, "grad_norm": 39.87434768676758, "loss_scale": 4.0, "retries": 0, "seconds": 53.2, "max_memory_gb": 19.15}
|
| 5 |
+
{"step": 5, "steps": 500, "lr": 6.25e-06, "loss": 3.0070376910946583, "grad_norm": 36.36997604370117, "loss_scale": 4.0, "retries": 0, "seconds": 65.7, "max_memory_gb": 19.15}
|
| 6 |
+
{"step": 6, "steps": 500, "lr": 7.500000000000001e-06, "loss": 3.0431658330966123, "grad_norm": 35.60318374633789, "loss_scale": 4.0, "retries": 0, "seconds": 78.0, "max_memory_gb": 19.15}
|
| 7 |
+
{"step": 7, "steps": 500, "lr": 8.750000000000001e-06, "loss": 2.170868859372356, "grad_norm": 33.0391960144043, "loss_scale": 4.0, "retries": 0, "seconds": 89.9, "max_memory_gb": 19.15}
|
| 8 |
+
{"step": 8, "steps": 500, "lr": 1e-05, "loss": 1.9102673572911457, "grad_norm": 31.046567916870117, "loss_scale": 4.0, "retries": 0, "seconds": 102.0, "max_memory_gb": 19.15}
|
| 9 |
+
{"step": 9, "steps": 500, "lr": 1.125e-05, "loss": 1.4043698787519878, "grad_norm": 23.784317016601562, "loss_scale": 4.0, "retries": 0, "seconds": 114.1, "max_memory_gb": 19.15}
|
| 10 |
+
{"step": 10, "steps": 500, "lr": 1.25e-05, "loss": 0.972348777577281, "grad_norm": 20.349651336669922, "loss_scale": 4.0, "retries": 0, "seconds": 126.3, "max_memory_gb": 19.15}
|
| 11 |
+
{"step": 11, "steps": 500, "lr": 1.375e-05, "loss": 0.6641758115623485, "grad_norm": 11.124773979187012, "loss_scale": 4.0, "retries": 0, "seconds": 138.2, "max_memory_gb": 19.15}
|
| 12 |
+
{"step": 12, "steps": 500, "lr": 1.5000000000000002e-05, "loss": 0.8397949208420786, "grad_norm": 10.232976913452148, "loss_scale": 4.0, "retries": 0, "seconds": 150.5, "max_memory_gb": 19.15}
|
| 13 |
+
{"step": 13, "steps": 500, "lr": 1.6250000000000002e-05, "loss": 0.5139915079238232, "grad_norm": 2.480334520339966, "loss_scale": 4.0, "retries": 0, "seconds": 162.6, "max_memory_gb": 19.15}
|
| 14 |
+
{"step": 14, "steps": 500, "lr": 1.7500000000000002e-05, "loss": 0.40553204679269006, "grad_norm": 4.827327251434326, "loss_scale": 4.0, "retries": 0, "seconds": 174.5, "max_memory_gb": 19.15}
|
| 15 |
+
{"step": 15, "steps": 500, "lr": 1.8750000000000002e-05, "loss": 0.338838051718829, "grad_norm": 2.3172693252563477, "loss_scale": 4.0, "retries": 0, "seconds": 186.6, "max_memory_gb": 19.15}
|
| 16 |
+
{"step": 16, "steps": 500, "lr": 2e-05, "loss": 0.33816927417964576, "grad_norm": 1.9915984869003296, "loss_scale": 4.0, "retries": 0, "seconds": 198.6, "max_memory_gb": 19.15}
|
| 17 |
+
{"step": 17, "steps": 500, "lr": 1.999981040799864e-05, "loss": 0.28106047108832916, "grad_norm": 2.134721040725708, "loss_scale": 4.0, "retries": 0, "seconds": 211.0, "max_memory_gb": 19.15}
|
| 18 |
+
{"step": 18, "steps": 500, "lr": 1.9999241639982356e-05, "loss": 0.2597257009448103, "grad_norm": 3.71720552444458, "loss_scale": 4.0, "retries": 0, "seconds": 223.0, "max_memory_gb": 19.15}
|
| 19 |
+
{"step": 19, "steps": 500, "lr": 1.9998293719914233e-05, "loss": 0.2455842882786369, "grad_norm": 2.403961420059204, "loss_scale": 4.0, "retries": 0, "seconds": 235.0, "max_memory_gb": 19.15}
|
| 20 |
+
{"step": 20, "steps": 500, "lr": 1.9996966687731615e-05, "loss": 0.39929720408822916, "grad_norm": 2.844583034515381, "loss_scale": 4.0, "retries": 0, "seconds": 247.2, "max_memory_gb": 19.15}
|
| 21 |
+
{"step": 21, "steps": 500, "lr": 1.9995260599344435e-05, "loss": 0.22573536335469482, "grad_norm": 2.1913936138153076, "loss_scale": 4.0, "retries": 0, "seconds": 259.3, "max_memory_gb": 19.15}
|
| 22 |
+
{"step": 22, "steps": 500, "lr": 1.9993175526632854e-05, "loss": 0.2771270607693375, "grad_norm": 3.2266242504119873, "loss_scale": 4.0, "retries": 0, "seconds": 271.7, "max_memory_gb": 19.15}
|
| 23 |
+
{"step": 23, "steps": 500, "lr": 1.9990711557444226e-05, "loss": 0.2211439059478421, "grad_norm": 2.5715017318725586, "loss_scale": 4.0, "retries": 0, "seconds": 283.7, "max_memory_gb": 19.15}
|
| 24 |
+
{"step": 24, "steps": 500, "lr": 1.998786879558941e-05, "loss": 0.18272283316499935, "grad_norm": 2.70710825920105, "loss_scale": 4.0, "retries": 0, "seconds": 296.0, "max_memory_gb": 19.15}
|
| 25 |
+
{"step": 25, "steps": 500, "lr": 1.998464736083838e-05, "loss": 0.14934923233919437, "grad_norm": 1.4737316370010376, "loss_scale": 4.0, "retries": 0, "seconds": 308.2, "max_memory_gb": 19.15}
|
| 26 |
+
{"step": 26, "steps": 500, "lr": 1.99810473889152e-05, "loss": 0.11718861334851352, "grad_norm": 1.4353859424591064, "loss_scale": 4.0, "retries": 0, "seconds": 320.2, "max_memory_gb": 19.15}
|
| 27 |
+
{"step": 27, "steps": 500, "lr": 1.9977069031492283e-05, "loss": 0.11679849883181635, "grad_norm": 1.2683556079864502, "loss_scale": 4.0, "retries": 0, "seconds": 332.3, "max_memory_gb": 19.15}
|
| 28 |
+
{"step": 28, "steps": 500, "lr": 1.9972712456184023e-05, "loss": 0.09366604574262799, "grad_norm": 1.0947880744934082, "loss_scale": 4.0, "retries": 0, "seconds": 344.4, "max_memory_gb": 19.15}
|
| 29 |
+
{"step": 29, "steps": 500, "lr": 1.996797784653971e-05, "loss": 0.13322851346972378, "grad_norm": 1.6633352041244507, "loss_scale": 4.0, "retries": 0, "seconds": 356.5, "max_memory_gb": 19.15}
|
| 30 |
+
{"step": 30, "steps": 500, "lr": 1.9962865402035815e-05, "loss": 0.13566849101434028, "grad_norm": 2.608520269393921, "loss_scale": 4.0, "retries": 0, "seconds": 368.9, "max_memory_gb": 19.15}
|
| 31 |
+
{"step": 31, "steps": 500, "lr": 1.9957375338067585e-05, "loss": 0.04528306560446668, "grad_norm": 1.0237846374511719, "loss_scale": 4.0, "retries": 0, "seconds": 381.2, "max_memory_gb": 19.15}
|
| 32 |
+
{"step": 32, "steps": 500, "lr": 1.995150788593995e-05, "loss": 0.11120738022972632, "grad_norm": 1.6662797927856445, "loss_scale": 4.0, "retries": 0, "seconds": 393.1, "max_memory_gb": 19.15}
|
| 33 |
+
{"step": 33, "steps": 500, "lr": 1.9945263292857805e-05, "loss": 0.0731447501192716, "grad_norm": 1.760659098625183, "loss_scale": 4.0, "retries": 0, "seconds": 406.3, "max_memory_gb": 19.15}
|
| 34 |
+
{"step": 34, "steps": 500, "lr": 1.9938641821915566e-05, "loss": 0.02122764958601858, "grad_norm": 0.7910311222076416, "loss_scale": 4.0, "retries": 0, "seconds": 418.9, "max_memory_gb": 19.15}
|
| 35 |
+
{"step": 35, "steps": 500, "lr": 1.993164375208611e-05, "loss": 0.03690910005885102, "grad_norm": 4.7176833152771, "loss_scale": 4.0, "retries": 0, "seconds": 431.3, "max_memory_gb": 19.15}
|
| 36 |
+
{"step": 36, "steps": 500, "lr": 1.9924269378209e-05, "loss": 0.027042759895597228, "grad_norm": 1.7017813920974731, "loss_scale": 4.0, "retries": 0, "seconds": 443.5, "max_memory_gb": 19.15}
|
| 37 |
+
{"step": 37, "steps": 500, "lr": 1.9916519010978086e-05, "loss": 0.03900783969749517, "grad_norm": 2.4600629806518555, "loss_scale": 4.0, "retries": 0, "seconds": 455.8, "max_memory_gb": 19.15}
|
| 38 |
+
{"step": 38, "steps": 500, "lr": 1.9908392976928396e-05, "loss": 0.2667258848191936, "grad_norm": 41.16464614868164, "loss_scale": 4.0, "retries": 0, "seconds": 467.9, "max_memory_gb": 19.15}
|
| 39 |
+
{"step": 39, "steps": 500, "lr": 1.9899891618422388e-05, "loss": 0.03548645282837077, "grad_norm": 3.611496925354004, "loss_scale": 4.0, "retries": 0, "seconds": 480.2, "max_memory_gb": 19.15}
|
| 40 |
+
{"step": 40, "steps": 500, "lr": 1.989101529363553e-05, "loss": 0.01264478759067474, "grad_norm": 2.5507216453552246, "loss_scale": 4.0, "retries": 0, "seconds": 492.8, "max_memory_gb": 19.15}
|
| 41 |
+
{"step": 41, "steps": 500, "lr": 1.9881764376541183e-05, "loss": 0.0041300837279849975, "grad_norm": 0.14490960538387299, "loss_scale": 4.0, "retries": 0, "seconds": 505.2, "max_memory_gb": 19.15}
|
| 42 |
+
{"step": 42, "steps": 500, "lr": 1.9872139256894883e-05, "loss": 0.00257794507741561, "grad_norm": 0.10654066503047943, "loss_scale": 4.0, "retries": 0, "seconds": 517.5, "max_memory_gb": 19.15}
|
| 43 |
+
{"step": 43, "steps": 500, "lr": 1.98621403402179e-05, "loss": 0.023182672503811445, "grad_norm": 4.390697479248047, "loss_scale": 4.0, "retries": 0, "seconds": 529.6, "max_memory_gb": 19.15}
|
| 44 |
+
{"step": 44, "steps": 500, "lr": 1.9851768047780144e-05, "loss": 0.0024312677379431206, "grad_norm": 0.11428307741880417, "loss_scale": 4.0, "retries": 0, "seconds": 541.9, "max_memory_gb": 19.15}
|
| 45 |
+
{"step": 45, "steps": 500, "lr": 1.9841022816582437e-05, "loss": 0.0017917825064056492, "grad_norm": 0.11139579117298126, "loss_scale": 4.0, "retries": 0, "seconds": 554.2, "max_memory_gb": 19.15}
|
| 46 |
+
{"step": 46, "steps": 500, "lr": 1.9829905099338088e-05, "loss": 0.0008772853887173618, "grad_norm": 0.08614717423915863, "loss_scale": 4.0, "retries": 0, "seconds": 566.6, "max_memory_gb": 19.15}
|
| 47 |
+
{"step": 47, "steps": 500, "lr": 1.9818415364453816e-05, "loss": 0.12341323979669416, "grad_norm": 8.015828132629395, "loss_scale": 4.0, "retries": 0, "seconds": 579.0, "max_memory_gb": 19.15}
|
| 48 |
+
{"step": 48, "steps": 500, "lr": 1.9806554096010038e-05, "loss": 0.0037149581608605076, "grad_norm": 0.2969008982181549, "loss_scale": 4.0, "retries": 0, "seconds": 591.2, "max_memory_gb": 19.15}
|
| 49 |
+
{"step": 49, "steps": 500, "lr": 1.9794321793740438e-05, "loss": 0.0012103976585897478, "grad_norm": 0.07255707681179047, "loss_scale": 4.0, "retries": 0, "seconds": 603.5, "max_memory_gb": 19.15}
|
| 50 |
+
{"step": 50, "steps": 500, "lr": 1.9781718973010948e-05, "loss": 0.0035877267246029673, "grad_norm": 0.3359256386756897, "loss_scale": 4.0, "retries": 0, "seconds": 615.6, "max_memory_gb": 19.15}
|
| 51 |
+
{"step": 51, "steps": 500, "lr": 1.9768746164798014e-05, "loss": 0.39239632395334567, "grad_norm": 18.890806198120117, "loss_scale": 4.0, "retries": 0, "seconds": 627.7, "max_memory_gb": 19.15}
|
| 52 |
+
{"step": 52, "steps": 500, "lr": 1.9755403915666227e-05, "loss": 0.0099657849031574, "grad_norm": 0.9667932391166687, "loss_scale": 4.0, "retries": 0, "seconds": 639.8, "max_memory_gb": 19.15}
|
| 53 |
+
{"step": 53, "steps": 500, "lr": 1.9741692787745306e-05, "loss": 0.0002121979902964003, "grad_norm": 0.015398156829178333, "loss_scale": 4.0, "retries": 0, "seconds": 652.1, "max_memory_gb": 19.15}
|
| 54 |
+
{"step": 54, "steps": 500, "lr": 1.9727613358706398e-05, "loss": 0.00033780143269462377, "grad_norm": 0.02620106004178524, "loss_scale": 4.0, "retries": 0, "seconds": 664.2, "max_memory_gb": 19.15}
|
| 55 |
+
{"step": 55, "steps": 500, "lr": 1.9713166221737755e-05, "loss": 0.03948395060391878, "grad_norm": 4.060560703277588, "loss_scale": 4.0, "retries": 0, "seconds": 676.4, "max_memory_gb": 19.15}
|
| 56 |
+
{"step": 56, "steps": 500, "lr": 1.9698351985519734e-05, "loss": 0.003186218424425004, "grad_norm": 0.6223495602607727, "loss_scale": 4.0, "retries": 0, "seconds": 688.5, "max_memory_gb": 19.15}
|
| 57 |
+
{"step": 57, "steps": 500, "lr": 1.968317127419916e-05, "loss": 0.03138715738773421, "grad_norm": 4.068129062652588, "loss_scale": 4.0, "retries": 0, "seconds": 700.6, "max_memory_gb": 19.15}
|
| 58 |
+
{"step": 58, "steps": 500, "lr": 1.9667624727363018e-05, "loss": 0.291754565846649, "grad_norm": 10.068485260009766, "loss_scale": 4.0, "retries": 0, "seconds": 712.8, "max_memory_gb": 19.15}
|
| 59 |
+
{"step": 59, "steps": 500, "lr": 1.9651713000011512e-05, "loss": 0.058465363980133134, "grad_norm": 10.497566223144531, "loss_scale": 4.0, "retries": 0, "seconds": 724.9, "max_memory_gb": 19.15}
|
| 60 |
+
{"step": 60, "steps": 500, "lr": 1.9635436762530477e-05, "loss": 0.0007109713140397131, "grad_norm": 0.15128254890441895, "loss_scale": 4.0, "retries": 0, "seconds": 737.5, "max_memory_gb": 19.15}
|
| 61 |
+
{"step": 61, "steps": 500, "lr": 1.9618796700663118e-05, "loss": 8.139470084245204e-05, "grad_norm": 0.006160569842904806, "loss_scale": 4.0, "retries": 0, "seconds": 749.6, "max_memory_gb": 19.15}
|
| 62 |
+
{"step": 62, "steps": 500, "lr": 1.960179351548113e-05, "loss": 0.19520760186074687, "grad_norm": 5.167180061340332, "loss_scale": 4.0, "retries": 0, "seconds": 762.2, "max_memory_gb": 19.15}
|
| 63 |
+
{"step": 63, "steps": 500, "lr": 1.958442792335516e-05, "loss": 0.08137614936353674, "grad_norm": 6.129891872406006, "loss_scale": 4.0, "retries": 0, "seconds": 774.3, "max_memory_gb": 19.15}
|
| 64 |
+
{"step": 64, "steps": 500, "lr": 1.956670065592463e-05, "loss": 0.24216385284001068, "grad_norm": 8.082837104797363, "loss_scale": 4.0, "retries": 0, "seconds": 786.5, "max_memory_gb": 19.15}
|
| 65 |
+
{"step": 65, "steps": 500, "lr": 1.9548612460066884e-05, "loss": 7.993246721195067e-05, "grad_norm": 0.005197289865463972, "loss_scale": 4.0, "retries": 0, "seconds": 798.7, "max_memory_gb": 19.15}
|
| 66 |
+
{"step": 66, "steps": 500, "lr": 1.9530164097865766e-05, "loss": 0.4055078671837125, "grad_norm": 13.773167610168457, "loss_scale": 4.0, "retries": 0, "seconds": 810.8, "max_memory_gb": 19.15}
|
| 67 |
+
{"step": 67, "steps": 500, "lr": 1.9511356346579475e-05, "loss": 0.00014241741939584205, "grad_norm": 0.009651939384639263, "loss_scale": 4.0, "retries": 0, "seconds": 822.9, "max_memory_gb": 19.15}
|
| 68 |
+
{"step": 68, "steps": 500, "lr": 1.949218999860784e-05, "loss": 0.004792265428684253, "grad_norm": 0.4409538805484772, "loss_scale": 4.0, "retries": 0, "seconds": 835.1, "max_memory_gb": 19.15}
|
| 69 |
+
{"step": 69, "steps": 500, "lr": 1.9472665861458913e-05, "loss": 9.916651586520815e-05, "grad_norm": 0.007952747866511345, "loss_scale": 4.0, "retries": 0, "seconds": 847.3, "max_memory_gb": 19.15}
|
| 70 |
+
{"step": 70, "steps": 500, "lr": 1.9452784757714977e-05, "loss": 0.17454317913330383, "grad_norm": 6.737672805786133, "loss_scale": 4.0, "retries": 0, "seconds": 859.6, "max_memory_gb": 19.15}
|
| 71 |
+
{"step": 71, "steps": 500, "lr": 1.9432547524997857e-05, "loss": 0.6189727714094472, "grad_norm": 19.424428939819336, "loss_scale": 4.0, "retries": 0, "seconds": 871.8, "max_memory_gb": 19.15}
|
| 72 |
+
{"step": 72, "steps": 500, "lr": 1.941195501593366e-05, "loss": 0.000600713553343336, "grad_norm": 0.04552701115608215, "loss_scale": 4.0, "retries": 0, "seconds": 883.9, "max_memory_gb": 19.15}
|
| 73 |
+
{"step": 73, "steps": 500, "lr": 1.939100809811683e-05, "loss": 0.00025886800257653294, "grad_norm": 0.015756286680698395, "loss_scale": 4.0, "retries": 0, "seconds": 896.1, "max_memory_gb": 19.15}
|
| 74 |
+
{"step": 74, "steps": 500, "lr": 1.93697076540736e-05, "loss": 0.00019774009119415, "grad_norm": 0.015146699734032154, "loss_scale": 4.0, "retries": 0, "seconds": 908.2, "max_memory_gb": 19.15}
|
| 75 |
+
{"step": 75, "steps": 500, "lr": 1.934805458122483e-05, "loss": 0.023337010537716196, "grad_norm": 1.8694852590560913, "loss_scale": 4.0, "retries": 0, "seconds": 920.6, "max_memory_gb": 19.15}
|
| 76 |
+
{"step": 76, "steps": 500, "lr": 1.9326049791848156e-05, "loss": 0.0022357485501784554, "grad_norm": 0.17122164368629456, "loss_scale": 4.0, "retries": 0, "seconds": 932.5, "max_memory_gb": 19.15}
|
| 77 |
+
{"step": 77, "steps": 500, "lr": 1.9303694213039598e-05, "loss": 0.0048435673021355215, "grad_norm": 0.31461378931999207, "loss_scale": 4.0, "retries": 0, "seconds": 945.2, "max_memory_gb": 19.15}
|
| 78 |
+
{"step": 78, "steps": 500, "lr": 1.9280988786674468e-05, "loss": 0.0032287546139080155, "grad_norm": 0.19250088930130005, "loss_scale": 4.0, "retries": 0, "seconds": 957.7, "max_memory_gb": 19.15}
|
| 79 |
+
{"step": 79, "steps": 500, "lr": 1.925793446936771e-05, "loss": 0.0018950342008048951, "grad_norm": 0.1569601595401764, "loss_scale": 4.0, "retries": 0, "seconds": 970.1, "max_memory_gb": 19.15}
|
| 80 |
+
{"step": 80, "steps": 500, "lr": 1.923453223243358e-05, "loss": 0.01815287815125802, "grad_norm": 0.9999164342880249, "loss_scale": 4.0, "retries": 0, "seconds": 982.5, "max_memory_gb": 19.15}
|
| 81 |
+
{"step": 81, "steps": 500, "lr": 1.9210783061844728e-05, "loss": 0.005360881909754509, "grad_norm": 0.36619827151298523, "loss_scale": 4.0, "retries": 0, "seconds": 994.8, "max_memory_gb": 19.15}
|
| 82 |
+
{"step": 82, "steps": 500, "lr": 1.9186687958190668e-05, "loss": 0.00031010594925233323, "grad_norm": 0.018617156893014908, "loss_scale": 4.0, "retries": 0, "seconds": 1007.4, "max_memory_gb": 19.15}
|
| 83 |
+
{"step": 83, "steps": 500, "lr": 1.91622479366356e-05, "loss": 0.01739215203435213, "grad_norm": 1.2290589809417725, "loss_scale": 4.0, "retries": 0, "seconds": 1019.7, "max_memory_gb": 19.15}
|
| 84 |
+
{"step": 84, "steps": 500, "lr": 1.9137464026875653e-05, "loss": 0.568504329533508, "grad_norm": 9.029523849487305, "loss_scale": 4.0, "retries": 0, "seconds": 1032.2, "max_memory_gb": 19.15}
|
| 85 |
+
{"step": 85, "steps": 500, "lr": 1.9112337273095516e-05, "loss": 0.05998325666421971, "grad_norm": 1.7050551176071167, "loss_scale": 4.0, "retries": 0, "seconds": 1044.5, "max_memory_gb": 19.15}
|
| 86 |
+
{"step": 86, "steps": 500, "lr": 1.908686873392441e-05, "loss": 0.0012599772668099006, "grad_norm": 0.1624671369791031, "loss_scale": 4.0, "retries": 0, "seconds": 1056.8, "max_memory_gb": 19.15}
|
| 87 |
+
{"step": 87, "steps": 500, "lr": 1.9061059482391522e-05, "loss": 0.005980782504097574, "grad_norm": 0.41807374358177185, "loss_scale": 4.0, "retries": 0, "seconds": 1069.5, "max_memory_gb": 19.15}
|
| 88 |
+
{"step": 88, "steps": 500, "lr": 1.9034910605880777e-05, "loss": 0.12907459841529673, "grad_norm": 3.4815118312835693, "loss_scale": 4.0, "retries": 0, "seconds": 1081.8, "max_memory_gb": 19.15}
|
| 89 |
+
{"step": 89, "steps": 500, "lr": 1.9008423206085016e-05, "loss": 0.0020948729372982852, "grad_norm": 0.1576426476240158, "loss_scale": 4.0, "retries": 0, "seconds": 1094.0, "max_memory_gb": 19.15}
|
| 90 |
+
{"step": 90, "steps": 500, "lr": 1.898159839895961e-05, "loss": 0.15160266926762814, "grad_norm": 2.181119680404663, "loss_scale": 4.0, "retries": 0, "seconds": 1106.3, "max_memory_gb": 19.15}
|
| 91 |
+
{"step": 91, "steps": 500, "lr": 1.895443731467542e-05, "loss": 0.013028526800636347, "grad_norm": 0.8199450969696045, "loss_scale": 4.0, "retries": 0, "seconds": 1118.6, "max_memory_gb": 19.15}
|
| 92 |
+
{"step": 92, "steps": 500, "lr": 1.892694109757118e-05, "loss": 0.0007698747472617966, "grad_norm": 0.07834496349096298, "loss_scale": 4.0, "retries": 0, "seconds": 1130.8, "max_memory_gb": 19.15}
|
| 93 |
+
{"step": 93, "steps": 500, "lr": 1.8899110906105304e-05, "loss": 0.026545492272899457, "grad_norm": 1.3623350858688354, "loss_scale": 4.0, "retries": 0, "seconds": 1142.7, "max_memory_gb": 19.15}
|
| 94 |
+
{"step": 94, "steps": 500, "lr": 1.887094791280705e-05, "loss": 0.00040929059411031295, "grad_norm": 0.03690618276596069, "loss_scale": 4.0, "retries": 0, "seconds": 1154.9, "max_memory_gb": 19.15}
|
| 95 |
+
{"step": 95, "steps": 500, "lr": 1.8842453304227153e-05, "loss": 0.05135217668732201, "grad_norm": 3.398747205734253, "loss_scale": 4.0, "retries": 0, "seconds": 1166.9, "max_memory_gb": 19.15}
|
| 96 |
+
{"step": 96, "steps": 500, "lr": 1.8813628280887796e-05, "loss": 0.24575380437169192, "grad_norm": 10.139664649963379, "loss_scale": 4.0, "retries": 0, "seconds": 1179.1, "max_memory_gb": 19.15}
|
| 97 |
+
{"step": 97, "steps": 500, "lr": 1.8784474057232066e-05, "loss": 0.00048030915331764856, "grad_norm": 0.037811700254678726, "loss_scale": 4.0, "retries": 0, "seconds": 1191.3, "max_memory_gb": 19.15}
|
| 98 |
+
{"step": 98, "steps": 500, "lr": 1.8754991861572767e-05, "loss": 0.024113786753497898, "grad_norm": 0.67043137550354, "loss_scale": 4.0, "retries": 0, "seconds": 1203.4, "max_memory_gb": 19.15}
|
| 99 |
+
{"step": 99, "steps": 500, "lr": 1.872518293604067e-05, "loss": 0.12679577492582958, "grad_norm": 4.062295913696289, "loss_scale": 4.0, "retries": 0, "seconds": 1215.7, "max_memory_gb": 19.15}
|
| 100 |
+
{"step": 100, "steps": 500, "lr": 1.8695048536532184e-05, "loss": 0.04712441893426248, "grad_norm": 1.4646085500717163, "loss_scale": 4.0, "retries": 0, "seconds": 1228.0, "max_memory_gb": 19.15}
|
| 101 |
+
{"step": 101, "steps": 500, "lr": 1.8664589932656442e-05, "loss": 0.0004388405043920846, "grad_norm": 0.03298215940594673, "loss_scale": 4.0, "retries": 0, "seconds": 1240.3, "max_memory_gb": 19.15}
|
| 102 |
+
{"step": 102, "steps": 500, "lr": 1.8633808407681826e-05, "loss": 0.011700460613734654, "grad_norm": 0.4692147374153137, "loss_scale": 4.0, "retries": 0, "seconds": 1252.8, "max_memory_gb": 19.15}
|
| 103 |
+
{"step": 103, "steps": 500, "lr": 1.8602705258481866e-05, "loss": 0.001047878475880928, "grad_norm": 0.10747146606445312, "loss_scale": 4.0, "retries": 0, "seconds": 1265.0, "max_memory_gb": 19.15}
|
| 104 |
+
{"step": 104, "steps": 500, "lr": 1.8571281795480632e-05, "loss": 0.0006453483203007308, "grad_norm": 0.06362205743789673, "loss_scale": 4.0, "retries": 0, "seconds": 1276.9, "max_memory_gb": 19.15}
|
| 105 |
+
{"step": 105, "steps": 500, "lr": 1.8539539342597513e-05, "loss": 0.0023554576251130625, "grad_norm": 0.22359827160835266, "loss_scale": 4.0, "retries": 0, "seconds": 1289.6, "max_memory_gb": 19.15}
|
| 106 |
+
{"step": 106, "steps": 500, "lr": 1.8507479237191432e-05, "loss": 0.2176632716813691, "grad_norm": 3.8978731632232666, "loss_scale": 4.0, "retries": 0, "seconds": 1301.8, "max_memory_gb": 19.15}
|
| 107 |
+
{"step": 107, "steps": 500, "lr": 1.847510283000451e-05, "loss": 0.0002023669342592753, "grad_norm": 0.02436703070998192, "loss_scale": 4.0, "retries": 0, "seconds": 1314.0, "max_memory_gb": 19.15}
|
| 108 |
+
{"step": 108, "steps": 500, "lr": 1.844241148510516e-05, "loss": 0.0012252290614262827, "grad_norm": 0.14546692371368408, "loss_scale": 4.0, "retries": 0, "seconds": 1325.9, "max_memory_gb": 19.15}
|
| 109 |
+
{"step": 109, "steps": 500, "lr": 1.84094065798306e-05, "loss": 0.004398470111482311, "grad_norm": 0.23845739662647247, "loss_scale": 4.0, "retries": 0, "seconds": 1338.1, "max_memory_gb": 19.15}
|
| 110 |
+
{"step": 110, "steps": 500, "lr": 1.8376089504728846e-05, "loss": 0.0030837522397384677, "grad_norm": 0.21768857538700104, "loss_scale": 4.0, "retries": 0, "seconds": 1350.4, "max_memory_gb": 19.15}
|
| 111 |
+
{"step": 111, "steps": 500, "lr": 1.8342461663500107e-05, "loss": 0.15897256335518833, "grad_norm": 2.375944137573242, "loss_scale": 4.0, "retries": 0, "seconds": 1362.8, "max_memory_gb": 19.16}
|
| 112 |
+
{"step": 112, "steps": 500, "lr": 1.8308524472937653e-05, "loss": 0.00025685029416680743, "grad_norm": 0.01903511770069599, "loss_scale": 4.0, "retries": 0, "seconds": 1374.9, "max_memory_gb": 19.16}
|
| 113 |
+
{"step": 113, "steps": 500, "lr": 1.827427936286812e-05, "loss": 0.04270052506446766, "grad_norm": 3.4696524143218994, "loss_scale": 4.0, "retries": 0, "seconds": 1387.0, "max_memory_gb": 19.16}
|
| 114 |
+
{"step": 114, "steps": 500, "lr": 1.8239727776091283e-05, "loss": 0.008603726132357473, "grad_norm": 0.709299623966217, "loss_scale": 4.0, "retries": 0, "seconds": 1399.3, "max_memory_gb": 19.16}
|
| 115 |
+
{"step": 115, "steps": 500, "lr": 1.8204871168319246e-05, "loss": 0.009176258694589542, "grad_norm": 0.9885153770446777, "loss_scale": 4.0, "retries": 0, "seconds": 1411.8, "max_memory_gb": 19.16}
|
| 116 |
+
{"step": 116, "steps": 500, "lr": 1.8169711008115124e-05, "loss": 0.00029379495440066836, "grad_norm": 0.019322814419865608, "loss_scale": 4.0, "retries": 0, "seconds": 1424.1, "max_memory_gb": 19.16}
|
| 117 |
+
{"step": 117, "steps": 500, "lr": 1.813424877683118e-05, "loss": 0.0001436335293709073, "grad_norm": 0.009531721472740173, "loss_scale": 4.0, "retries": 0, "seconds": 1436.3, "max_memory_gb": 19.16}
|
| 118 |
+
{"step": 118, "steps": 500, "lr": 1.8098485968546385e-05, "loss": 0.035943101052721846, "grad_norm": 1.3460972309112549, "loss_scale": 4.0, "retries": 0, "seconds": 1448.5, "max_memory_gb": 19.16}
|
| 119 |
+
{"step": 119, "steps": 500, "lr": 1.806242409000349e-05, "loss": 0.12212121595712233, "grad_norm": 11.129670143127441, "loss_scale": 4.0, "retries": 0, "seconds": 1460.5, "max_memory_gb": 19.16}
|
| 120 |
+
{"step": 120, "steps": 500, "lr": 1.8026064660545554e-05, "loss": 0.0031787671468919147, "grad_norm": 0.18417076766490936, "loss_scale": 4.0, "retries": 0, "seconds": 1472.8, "max_memory_gb": 19.16}
|
| 121 |
+
{"step": 121, "steps": 500, "lr": 1.7989409212051905e-05, "loss": 0.0033801603945200695, "grad_norm": 0.2552855312824249, "loss_scale": 4.0, "retries": 0, "seconds": 1485.3, "max_memory_gb": 19.16}
|
| 122 |
+
{"step": 122, "steps": 500, "lr": 1.7952459288873623e-05, "loss": 0.0014592895185707925, "grad_norm": 0.14452433586120605, "loss_scale": 4.0, "retries": 0, "seconds": 1497.2, "max_memory_gb": 19.16}
|
| 123 |
+
{"step": 123, "steps": 500, "lr": 1.7915216447768455e-05, "loss": 0.0307019552560839, "grad_norm": 1.9227021932601929, "loss_scale": 4.0, "retries": 0, "seconds": 1509.4, "max_memory_gb": 19.16}
|
| 124 |
+
{"step": 124, "steps": 500, "lr": 1.7877682257835244e-05, "loss": 0.14984235613963454, "grad_norm": 3.3638546466827393, "loss_scale": 4.0, "retries": 0, "seconds": 1521.5, "max_memory_gb": 19.16}
|
| 125 |
+
{"step": 125, "steps": 500, "lr": 1.783985830044781e-05, "loss": 7.727861296302764e-05, "grad_norm": 0.0038262433372437954, "loss_scale": 4.0, "retries": 0, "seconds": 1533.6, "max_memory_gb": 19.16}
|
| 126 |
+
{"step": 126, "steps": 500, "lr": 1.7801746169188328e-05, "loss": 0.03413860023614358, "grad_norm": 1.6957168579101562, "loss_scale": 4.0, "retries": 0, "seconds": 1545.7, "max_memory_gb": 19.16}
|
| 127 |
+
{"step": 127, "steps": 500, "lr": 1.7763347469780178e-05, "loss": 0.006461863051393895, "grad_norm": 1.2780706882476807, "loss_scale": 4.0, "retries": 0, "seconds": 1558.0, "max_memory_gb": 19.16}
|
| 128 |
+
{"step": 128, "steps": 500, "lr": 1.772466382002032e-05, "loss": 7.218002450186263e-05, "grad_norm": 0.003709694603458047, "loss_scale": 4.0, "retries": 0, "seconds": 1570.0, "max_memory_gb": 19.16}
|
| 129 |
+
{"step": 129, "steps": 500, "lr": 1.7685696849711094e-05, "loss": 0.021501554891681388, "grad_norm": 0.9458094239234924, "loss_scale": 4.0, "retries": 0, "seconds": 1582.2, "max_memory_gb": 19.16}
|
| 130 |
+
{"step": 130, "steps": 500, "lr": 1.7646448200591597e-05, "loss": 6.363267247102364e-05, "grad_norm": 0.0034351039212197065, "loss_scale": 4.0, "retries": 0, "seconds": 1594.3, "max_memory_gb": 19.16}
|
| 131 |
+
{"step": 131, "steps": 500, "lr": 1.760691952626847e-05, "loss": 0.10531851162818641, "grad_norm": 3.8493263721466064, "loss_scale": 4.0, "retries": 0, "seconds": 1606.3, "max_memory_gb": 19.16}
|
| 132 |
+
{"step": 132, "steps": 500, "lr": 1.756711249214627e-05, "loss": 9.307523558406544e-05, "grad_norm": 0.005931384861469269, "loss_scale": 4.0, "retries": 0, "seconds": 1618.4, "max_memory_gb": 19.16}
|
| 133 |
+
{"step": 133, "steps": 500, "lr": 1.752702877535728e-05, "loss": 0.003464073805699544, "grad_norm": 0.6207319498062134, "loss_scale": 4.0, "retries": 0, "seconds": 1630.9, "max_memory_gb": 19.16}
|
| 134 |
+
{"step": 134, "steps": 500, "lr": 1.7486670064690848e-05, "loss": 4.618611070257699e-05, "grad_norm": 0.0027663230430334806, "loss_scale": 4.0, "retries": 0, "seconds": 1643.4, "max_memory_gb": 19.16}
|
| 135 |
+
{"step": 135, "steps": 500, "lr": 1.7446038060522255e-05, "loss": 1.1055575319815762, "grad_norm": 16.942276000976562, "loss_scale": 4.0, "retries": 0, "seconds": 1655.7, "max_memory_gb": 19.16}
|
| 136 |
+
{"step": 136, "steps": 500, "lr": 1.7405134474741042e-05, "loss": 0.16580259042649464, "grad_norm": 3.053361177444458, "loss_scale": 4.0, "retries": 0, "seconds": 1668.1, "max_memory_gb": 19.16}
|
| 137 |
+
{"step": 137, "steps": 500, "lr": 1.736396103067893e-05, "loss": 0.4869254395367865, "grad_norm": 25.54742431640625, "loss_scale": 4.0, "retries": 0, "seconds": 1680.6, "max_memory_gb": 19.16}
|
| 138 |
+
{"step": 138, "steps": 500, "lr": 1.732251946303717e-05, "loss": 0.0003339904366209174, "grad_norm": 0.027053050696849823, "loss_scale": 4.0, "retries": 0, "seconds": 1693.5, "max_memory_gb": 19.16}
|
| 139 |
+
{"step": 139, "steps": 500, "lr": 1.728081151781349e-05, "loss": 0.06277204340964009, "grad_norm": 5.115485191345215, "loss_scale": 4.0, "retries": 0, "seconds": 1705.8, "max_memory_gb": 19.16}
|
| 140 |
+
{"step": 140, "steps": 500, "lr": 1.7238838952228502e-05, "loss": 0.08090928548220043, "grad_norm": 5.43334436416626, "loss_scale": 4.0, "retries": 0, "seconds": 1718.4, "max_memory_gb": 19.16}
|
| 141 |
+
{"step": 141, "steps": 500, "lr": 1.719660353465171e-05, "loss": 0.08573882401825277, "grad_norm": 2.3262252807617188, "loss_scale": 4.0, "retries": 0, "seconds": 1730.8, "max_memory_gb": 19.16}
|
| 142 |
+
{"step": 142, "steps": 500, "lr": 1.7154107044526965e-05, "loss": 0.001122470706522191, "grad_norm": 0.07266005128622055, "loss_scale": 4.0, "retries": 0, "seconds": 1743.4, "max_memory_gb": 19.16}
|
| 143 |
+
{"step": 143, "steps": 500, "lr": 1.7111351272297514e-05, "loss": 0.2145914307548367, "grad_norm": 4.835561275482178, "loss_scale": 4.0, "retries": 0, "seconds": 1756.1, "max_memory_gb": 19.16}
|
| 144 |
+
{"step": 144, "steps": 500, "lr": 1.7068338019330564e-05, "loss": 0.2156895709089988, "grad_norm": 9.400127410888672, "loss_scale": 4.0, "retries": 0, "seconds": 1768.8, "max_memory_gb": 19.16}
|
| 145 |
+
{"step": 145, "steps": 500, "lr": 1.702506909784138e-05, "loss": 0.18886864664262829, "grad_norm": 2.5802948474884033, "loss_scale": 4.0, "retries": 0, "seconds": 1781.2, "max_memory_gb": 19.16}
|
| 146 |
+
{"step": 146, "steps": 500, "lr": 1.6981546330816953e-05, "loss": 0.003904934542984765, "grad_norm": 0.1647668033838272, "loss_scale": 4.0, "retries": 0, "seconds": 1793.9, "max_memory_gb": 19.16}
|
| 147 |
+
{"step": 147, "steps": 500, "lr": 1.6937771551939167e-05, "loss": 0.0010557135521325604, "grad_norm": 0.10598733276128769, "loss_scale": 4.0, "retries": 0, "seconds": 1806.3, "max_memory_gb": 19.16}
|
| 148 |
+
{"step": 148, "steps": 500, "lr": 1.6893746605507567e-05, "loss": 0.07103374042585321, "grad_norm": 2.732401132583618, "loss_scale": 4.0, "retries": 0, "seconds": 1818.9, "max_memory_gb": 19.16}
|
| 149 |
+
{"step": 149, "steps": 500, "lr": 1.684947334636164e-05, "loss": 0.07157801387672241, "grad_norm": 1.5930049419403076, "loss_scale": 4.0, "retries": 0, "seconds": 1830.9, "max_memory_gb": 19.16}
|
| 150 |
+
{"step": 150, "steps": 500, "lr": 1.6804953639802675e-05, "loss": 0.04745717433382363, "grad_norm": 2.590047597885132, "loss_scale": 4.0, "retries": 0, "seconds": 1843.3, "max_memory_gb": 19.16}
|
| 151 |
+
{"step": 151, "steps": 500, "lr": 1.6760189361515182e-05, "loss": 0.09193389591473533, "grad_norm": 1.448494553565979, "loss_scale": 4.0, "retries": 0, "seconds": 1855.7, "max_memory_gb": 19.16}
|
| 152 |
+
{"step": 152, "steps": 500, "lr": 1.6715182397487845e-05, "loss": 0.007728854833596251, "grad_norm": 0.29711437225341797, "loss_scale": 4.0, "retries": 0, "seconds": 1868.4, "max_memory_gb": 19.16}
|
| 153 |
+
{"step": 153, "steps": 500, "lr": 1.6669934643934088e-05, "loss": 0.0020760230593168676, "grad_norm": 0.10983864963054657, "loss_scale": 4.0, "retries": 0, "seconds": 1880.7, "max_memory_gb": 19.16}
|
| 154 |
+
{"step": 154, "steps": 500, "lr": 1.6624448007212168e-05, "loss": 0.2183012018437942, "grad_norm": 1.5041892528533936, "loss_scale": 4.0, "retries": 0, "seconds": 1893.6, "max_memory_gb": 19.16}
|
| 155 |
+
{"step": 155, "steps": 500, "lr": 1.6578724403744854e-05, "loss": 0.019019265422354114, "grad_norm": 1.282515525817871, "loss_scale": 4.0, "retries": 0, "seconds": 1906.0, "max_memory_gb": 19.16}
|
| 156 |
+
{"step": 156, "steps": 500, "lr": 1.653276575993871e-05, "loss": 0.01271809776835653, "grad_norm": 0.4315701723098755, "loss_scale": 4.0, "retries": 0, "seconds": 1918.1, "max_memory_gb": 19.16}
|
| 157 |
+
{"step": 157, "steps": 500, "lr": 1.6486574012102896e-05, "loss": 0.0012156207031611386, "grad_norm": 0.05279054492712021, "loss_scale": 4.0, "retries": 0, "seconds": 1930.5, "max_memory_gb": 19.16}
|
| 158 |
+
{"step": 158, "steps": 500, "lr": 1.6440151106367617e-05, "loss": 0.09664091243485018, "grad_norm": 1.5425541400909424, "loss_scale": 4.0, "retries": 0, "seconds": 1943.4, "max_memory_gb": 19.16}
|
| 159 |
+
{"step": 159, "steps": 500, "lr": 1.6393498998602123e-05, "loss": 0.0010624341211124295, "grad_norm": 0.05115602910518646, "loss_scale": 4.0, "retries": 0, "seconds": 1955.9, "max_memory_gb": 19.16}
|
| 160 |
+
{"step": 160, "steps": 500, "lr": 1.6346619654332297e-05, "loss": 0.012462521838890761, "grad_norm": 0.7704785466194153, "loss_scale": 4.0, "retries": 0, "seconds": 1968.0, "max_memory_gb": 19.16}
|
| 161 |
+
{"step": 161, "steps": 500, "lr": 1.6299515048657848e-05, "loss": 0.0035125557187711824, "grad_norm": 0.12659640610218048, "loss_scale": 4.0, "retries": 0, "seconds": 1980.6, "max_memory_gb": 19.16}
|
| 162 |
+
{"step": 162, "steps": 500, "lr": 1.6252187166169105e-05, "loss": 0.0029850037183430027, "grad_norm": 0.14208915829658508, "loss_scale": 4.0, "retries": 0, "seconds": 1993.0, "max_memory_gb": 19.16}
|
| 163 |
+
{"step": 163, "steps": 500, "lr": 1.6204638000863392e-05, "loss": 0.02993431137110747, "grad_norm": 1.2987407445907593, "loss_scale": 4.0, "retries": 0, "seconds": 2005.5, "max_memory_gb": 19.16}
|
| 164 |
+
{"step": 164, "steps": 500, "lr": 1.615686955606102e-05, "loss": 0.8672917608422391, "grad_norm": 8.747241973876953, "loss_scale": 4.0, "retries": 0, "seconds": 2018.0, "max_memory_gb": 19.16}
|
| 165 |
+
{"step": 165, "steps": 500, "lr": 1.610888384432089e-05, "loss": 0.0056945488110862015, "grad_norm": 0.3283745050430298, "loss_scale": 4.0, "retries": 0, "seconds": 2030.6, "max_memory_gb": 19.16}
|
| 166 |
+
{"step": 166, "steps": 500, "lr": 1.60606828873557e-05, "loss": 0.008905917942684026, "grad_norm": 0.3934962749481201, "loss_scale": 4.0, "retries": 0, "seconds": 2043.0, "max_memory_gb": 19.16}
|
| 167 |
+
{"step": 167, "steps": 500, "lr": 1.601226871594676e-05, "loss": 0.004816797113149236, "grad_norm": 0.23903778195381165, "loss_scale": 4.0, "retries": 0, "seconds": 2055.6, "max_memory_gb": 19.16}
|
| 168 |
+
{"step": 168, "steps": 500, "lr": 1.5963643369858436e-05, "loss": 0.004746439742699626, "grad_norm": 0.22784611582756042, "loss_scale": 4.0, "retries": 0, "seconds": 2068.1, "max_memory_gb": 19.16}
|
| 169 |
+
{"step": 169, "steps": 500, "lr": 1.591480889775221e-05, "loss": 0.09177786627324995, "grad_norm": 4.163424968719482, "loss_scale": 4.0, "retries": 0, "seconds": 2080.5, "max_memory_gb": 19.16}
|
| 170 |
+
{"step": 170, "steps": 500, "lr": 1.586576735710038e-05, "loss": 0.02102677370765543, "grad_norm": 1.1479623317718506, "loss_scale": 4.0, "retries": 0, "seconds": 2092.9, "max_memory_gb": 19.16}
|
| 171 |
+
{"step": 171, "steps": 500, "lr": 1.5816520814099346e-05, "loss": 0.007564465534166879, "grad_norm": 0.49911606311798096, "loss_scale": 4.0, "retries": 0, "seconds": 2105.0, "max_memory_gb": 19.16}
|
| 172 |
+
{"step": 172, "steps": 500, "lr": 1.576707134358259e-05, "loss": 0.0050479866810994945, "grad_norm": 0.40527963638305664, "loss_scale": 4.0, "retries": 0, "seconds": 2117.7, "max_memory_gb": 19.16}
|
| 173 |
+
{"step": 173, "steps": 500, "lr": 1.5717421028933236e-05, "loss": 0.025123150997470395, "grad_norm": 0.5865683555603027, "loss_scale": 4.0, "retries": 0, "seconds": 2130.1, "max_memory_gb": 19.16}
|
| 174 |
+
{"step": 174, "steps": 500, "lr": 1.5667571961996297e-05, "loss": 0.000826120195514787, "grad_norm": 0.040049921721220016, "loss_scale": 4.0, "retries": 0, "seconds": 2142.5, "max_memory_gb": 19.16}
|
| 175 |
+
{"step": 175, "steps": 500, "lr": 1.5617526242990515e-05, "loss": 0.12730251569287662, "grad_norm": 7.42474365234375, "loss_scale": 4.0, "retries": 0, "seconds": 2154.9, "max_memory_gb": 19.16}
|
| 176 |
+
{"step": 176, "steps": 500, "lr": 1.5567285980419903e-05, "loss": 0.0019243880430043316, "grad_norm": 0.08438156545162201, "loss_scale": 4.0, "retries": 0, "seconds": 2167.3, "max_memory_gb": 19.16}
|
| 177 |
+
{"step": 177, "steps": 500, "lr": 1.551685329098488e-05, "loss": 0.0768252191385479, "grad_norm": 2.8380818367004395, "loss_scale": 4.0, "retries": 0, "seconds": 2179.8, "max_memory_gb": 19.16}
|
| 178 |
+
{"step": 178, "steps": 500, "lr": 1.5466230299493127e-05, "loss": 0.001465620916199193, "grad_norm": 0.0737033411860466, "loss_scale": 4.0, "retries": 0, "seconds": 2192.2, "max_memory_gb": 19.16}
|
| 179 |
+
{"step": 179, "steps": 500, "lr": 1.541541913877003e-05, "loss": 0.00033366337176878665, "grad_norm": 0.013760973699390888, "loss_scale": 4.0, "retries": 0, "seconds": 2204.9, "max_memory_gb": 19.16}
|
| 180 |
+
{"step": 180, "steps": 500, "lr": 1.5364421949568853e-05, "loss": 0.1027082502964318, "grad_norm": 4.012368679046631, "loss_scale": 4.0, "retries": 0, "seconds": 2216.7, "max_memory_gb": 19.16}
|
| 181 |
+
{"step": 181, "steps": 500, "lr": 1.5313240880480514e-05, "loss": 0.0006977235485485587, "grad_norm": 0.02979789301753044, "loss_scale": 4.0, "retries": 0, "seconds": 2229.0, "max_memory_gb": 19.16}
|
| 182 |
+
{"step": 182, "steps": 500, "lr": 1.5261878087843087e-05, "loss": 0.0012712359630313586, "grad_norm": 0.09362711757421494, "loss_scale": 4.0, "retries": 0, "seconds": 2241.3, "max_memory_gb": 19.16}
|
| 183 |
+
{"step": 183, "steps": 500, "lr": 1.521033573565094e-05, "loss": 0.15918814591800998, "grad_norm": 2.052555799484253, "loss_scale": 4.0, "retries": 0, "seconds": 2253.3, "max_memory_gb": 19.16}
|
| 184 |
+
{"step": 184, "steps": 500, "lr": 1.5158615995463566e-05, "loss": 0.03155418150267699, "grad_norm": 1.232677936553955, "loss_scale": 4.0, "retries": 0, "seconds": 2265.9, "max_memory_gb": 19.16}
|
| 185 |
+
{"step": 185, "steps": 500, "lr": 1.510672104631408e-05, "loss": 0.05321361888346391, "grad_norm": 3.053924083709717, "loss_scale": 4.0, "retries": 0, "seconds": 2278.1, "max_memory_gb": 19.16}
|
| 186 |
+
{"step": 186, "steps": 500, "lr": 1.5054653074617441e-05, "loss": 0.0021795961824244096, "grad_norm": 0.09065897017717361, "loss_scale": 4.0, "retries": 0, "seconds": 2290.7, "max_memory_gb": 19.16}
|
| 187 |
+
{"step": 187, "steps": 500, "lr": 1.50024142740783e-05, "loss": 0.00016652788336945045, "grad_norm": 0.00784487184137106, "loss_scale": 4.0, "retries": 0, "seconds": 2303.3, "max_memory_gb": 19.16}
|
| 188 |
+
{"step": 188, "steps": 500, "lr": 1.4950006845598603e-05, "loss": 0.00034123846278982296, "grad_norm": 0.013497265987098217, "loss_scale": 4.0, "retries": 0, "seconds": 2315.6, "max_memory_gb": 19.16}
|
| 189 |
+
{"step": 189, "steps": 500, "lr": 1.4897432997184845e-05, "loss": 0.007682363318555087, "grad_norm": 0.21836324036121368, "loss_scale": 4.0, "retries": 0, "seconds": 2328.0, "max_memory_gb": 19.16}
|
| 190 |
+
{"step": 190, "steps": 500, "lr": 1.4844694943855065e-05, "loss": 0.05053348026962113, "grad_norm": 0.9695649743080139, "loss_scale": 4.0, "retries": 0, "seconds": 2340.4, "max_memory_gb": 19.16}
|
| 191 |
+
{"step": 191, "steps": 500, "lr": 1.4791794907545496e-05, "loss": 0.04705967564661313, "grad_norm": 1.1787434816360474, "loss_scale": 4.0, "retries": 0, "seconds": 2353.0, "max_memory_gb": 19.16}
|
| 192 |
+
{"step": 192, "steps": 500, "lr": 1.4738735117016979e-05, "loss": 0.0006734800873883532, "grad_norm": 0.027692867442965508, "loss_scale": 4.0, "retries": 0, "seconds": 2365.4, "max_memory_gb": 19.16}
|
| 193 |
+
{"step": 193, "steps": 500, "lr": 1.4685517807761037e-05, "loss": 0.0005830615741408772, "grad_norm": 0.027608128264546394, "loss_scale": 4.0, "retries": 0, "seconds": 2377.9, "max_memory_gb": 19.16}
|
| 194 |
+
{"step": 194, "steps": 500, "lr": 1.463214522190571e-05, "loss": 0.0006698804472555574, "grad_norm": 0.028783800080418587, "loss_scale": 4.0, "retries": 0, "seconds": 2390.2, "max_memory_gb": 19.16}
|
| 195 |
+
{"step": 195, "steps": 500, "lr": 1.4578619608121079e-05, "loss": 0.09945328983032845, "grad_norm": 5.6141791343688965, "loss_scale": 4.0, "retries": 0, "seconds": 2402.3, "max_memory_gb": 19.16}
|
| 196 |
+
{"step": 196, "steps": 500, "lr": 1.4524943221524533e-05, "loss": 0.0015905474604716637, "grad_norm": 0.08615847676992416, "loss_scale": 4.0, "retries": 0, "seconds": 2414.7, "max_memory_gb": 19.16}
|
| 197 |
+
{"step": 197, "steps": 500, "lr": 1.4471118323585751e-05, "loss": 0.0006546937940418668, "grad_norm": 0.03332037851214409, "loss_scale": 4.0, "retries": 0, "seconds": 2427.0, "max_memory_gb": 19.16}
|
| 198 |
+
{"step": 198, "steps": 500, "lr": 1.4417147182031426e-05, "loss": 0.0002496996613672309, "grad_norm": 0.010348095558583736, "loss_scale": 4.0, "retries": 0, "seconds": 2439.5, "max_memory_gb": 19.16}
|
| 199 |
+
{"step": 199, "steps": 500, "lr": 1.4363032070749723e-05, "loss": 0.00494319951648818, "grad_norm": 0.4863109588623047, "loss_scale": 4.0, "retries": 0, "seconds": 2451.9, "max_memory_gb": 19.16}
|
| 200 |
+
{"step": 200, "steps": 500, "lr": 1.430877526969448e-05, "loss": 0.01627023684324575, "grad_norm": 0.6437428593635559, "loss_scale": 4.0, "retries": 0, "seconds": 2464.3, "max_memory_gb": 19.16}
|
| 201 |
+
{"step": 201, "steps": 500, "lr": 1.4254379064789138e-05, "loss": 0.09737386346463694, "grad_norm": 10.53452205657959, "loss_scale": 4.0, "retries": 0, "seconds": 2476.6, "max_memory_gb": 19.16}
|
| 202 |
+
{"step": 202, "steps": 500, "lr": 1.4199845747830443e-05, "loss": 0.002452526119384402, "grad_norm": 0.10759394615888596, "loss_scale": 4.0, "retries": 0, "seconds": 2489.1, "max_memory_gb": 19.16}
|
| 203 |
+
{"step": 203, "steps": 500, "lr": 1.4145177616391886e-05, "loss": 0.012253471602259034, "grad_norm": 0.8281735181808472, "loss_scale": 4.0, "retries": 0, "seconds": 2501.6, "max_memory_gb": 19.16}
|
| 204 |
+
{"step": 204, "steps": 500, "lr": 1.4090376973726898e-05, "loss": 0.10096940726378474, "grad_norm": 3.1076464653015137, "loss_scale": 4.0, "retries": 0, "seconds": 2513.6, "max_memory_gb": 19.16}
|
| 205 |
+
{"step": 205, "steps": 500, "lr": 1.4035446128671815e-05, "loss": 0.0037670660651863076, "grad_norm": 0.1745852530002594, "loss_scale": 4.0, "retries": 0, "seconds": 2526.0, "max_memory_gb": 19.16}
|
| 206 |
+
{"step": 206, "steps": 500, "lr": 1.3980387395548608e-05, "loss": 0.22838731313333677, "grad_norm": 5.3228654861450195, "loss_scale": 4.0, "retries": 0, "seconds": 2538.4, "max_memory_gb": 19.16}
|
| 207 |
+
{"step": 207, "steps": 500, "lr": 1.3925203094067364e-05, "loss": 0.0011759459184190748, "grad_norm": 0.05201467499136925, "loss_scale": 4.0, "retries": 0, "seconds": 2550.7, "max_memory_gb": 19.16}
|
| 208 |
+
{"step": 208, "steps": 500, "lr": 1.3869895549228569e-05, "loss": 0.0010630632383959503, "grad_norm": 0.052825331687927246, "loss_scale": 4.0, "retries": 0, "seconds": 2563.5, "max_memory_gb": 19.16}
|
| 209 |
+
{"step": 209, "steps": 500, "lr": 1.381446709122513e-05, "loss": 0.009529107835987814, "grad_norm": 0.4106357991695404, "loss_scale": 4.0, "retries": 0, "seconds": 2575.4, "max_memory_gb": 19.16}
|
| 210 |
+
{"step": 210, "steps": 500, "lr": 1.3758920055344226e-05, "loss": 0.0015423188827257534, "grad_norm": 0.08073461800813675, "loss_scale": 4.0, "retries": 0, "seconds": 2587.8, "max_memory_gb": 19.16}
|
| 211 |
+
{"step": 211, "steps": 500, "lr": 1.3703256781868898e-05, "loss": 0.037787099422579545, "grad_norm": 1.8046501874923706, "loss_scale": 4.0, "retries": 0, "seconds": 2600.2, "max_memory_gb": 19.16}
|
| 212 |
+
{"step": 212, "steps": 500, "lr": 1.3647479615979466e-05, "loss": 0.0007861870402207802, "grad_norm": 0.08294835686683655, "loss_scale": 4.0, "retries": 0, "seconds": 2612.1, "max_memory_gb": 19.16}
|
| 213 |
+
{"step": 213, "steps": 500, "lr": 1.359159090765471e-05, "loss": 0.0002143288961943571, "grad_norm": 0.011460425332188606, "loss_scale": 4.0, "retries": 0, "seconds": 2624.6, "max_memory_gb": 19.16}
|
| 214 |
+
{"step": 214, "steps": 500, "lr": 1.3535593011572868e-05, "loss": 0.028570605466701305, "grad_norm": 0.9035517573356628, "loss_scale": 4.0, "retries": 0, "seconds": 2637.0, "max_memory_gb": 19.16}
|
| 215 |
+
{"step": 215, "steps": 500, "lr": 1.347948828701242e-05, "loss": 0.0007715645385143365, "grad_norm": 0.0560917854309082, "loss_scale": 4.0, "retries": 0, "seconds": 2649.4, "max_memory_gb": 19.16}
|
| 216 |
+
{"step": 216, "steps": 500, "lr": 1.3423279097752711e-05, "loss": 0.0020980906251576344, "grad_norm": 0.09529955685138702, "loss_scale": 4.0, "retries": 0, "seconds": 2662.0, "max_memory_gb": 19.16}
|
| 217 |
+
{"step": 217, "steps": 500, "lr": 1.3366967811974332e-05, "loss": 0.00016662594724824552, "grad_norm": 0.008285680785775185, "loss_scale": 4.0, "retries": 0, "seconds": 2674.7, "max_memory_gb": 19.16}
|
| 218 |
+
{"step": 218, "steps": 500, "lr": 1.3310556802159368e-05, "loss": 0.07139841394810922, "grad_norm": 6.16837215423584, "loss_scale": 4.0, "retries": 0, "seconds": 2686.8, "max_memory_gb": 19.16}
|
| 219 |
+
{"step": 219, "steps": 500, "lr": 1.325404844499143e-05, "loss": 0.00038963314458842745, "grad_norm": 0.020282665267586708, "loss_scale": 4.0, "retries": 0, "seconds": 2699.1, "max_memory_gb": 19.16}
|
| 220 |
+
{"step": 220, "steps": 500, "lr": 1.3197445121255522e-05, "loss": 0.0025868715479501507, "grad_norm": 0.12204836308956146, "loss_scale": 4.0, "retries": 0, "seconds": 2711.7, "max_memory_gb": 19.16}
|
| 221 |
+
{"step": 221, "steps": 500, "lr": 1.3140749215737745e-05, "loss": 0.0006031243951735054, "grad_norm": 0.04284139722585678, "loss_scale": 4.0, "retries": 0, "seconds": 2724.0, "max_memory_gb": 19.16}
|
| 222 |
+
{"step": 222, "steps": 500, "lr": 1.3083963117124801e-05, "loss": 4.183172241828908e-05, "grad_norm": 0.0021033743396401405, "loss_scale": 4.0, "retries": 0, "seconds": 2736.5, "max_memory_gb": 19.16}
|
| 223 |
+
{"step": 223, "steps": 500, "lr": 1.3027089217903386e-05, "loss": 0.0001662324356278656, "grad_norm": 0.008475326001644135, "loss_scale": 4.0, "retries": 0, "seconds": 2748.9, "max_memory_gb": 19.16}
|
| 224 |
+
{"step": 224, "steps": 500, "lr": 1.2970129914259354e-05, "loss": 0.0013235023730389498, "grad_norm": 0.07281097769737244, "loss_scale": 4.0, "retries": 0, "seconds": 2761.3, "max_memory_gb": 19.16}
|
| 225 |
+
{"step": 225, "steps": 500, "lr": 1.2913087605976789e-05, "loss": 0.19816166462764137, "grad_norm": 7.162214756011963, "loss_scale": 4.0, "retries": 0, "seconds": 2773.7, "max_memory_gb": 19.16}
|
| 226 |
+
{"step": 226, "steps": 500, "lr": 1.2855964696336894e-05, "loss": 0.3899827592189553, "grad_norm": 16.463016510009766, "loss_scale": 4.0, "retries": 0, "seconds": 2786.1, "max_memory_gb": 19.16}
|
| 227 |
+
{"step": 227, "steps": 500, "lr": 1.2798763592016722e-05, "loss": 0.049561124897040426, "grad_norm": 1.7683732509613037, "loss_scale": 4.0, "retries": 0, "seconds": 2798.6, "max_memory_gb": 19.16}
|
| 228 |
+
{"step": 228, "steps": 500, "lr": 1.2741486702987796e-05, "loss": 0.05405490634107548, "grad_norm": 1.009128451347351, "loss_scale": 4.0, "retries": 0, "seconds": 2811.0, "max_memory_gb": 19.16}
|
| 229 |
+
{"step": 229, "steps": 500, "lr": 1.268413644241457e-05, "loss": 0.314375282809709, "grad_norm": 5.788825035095215, "loss_scale": 4.0, "retries": 0, "seconds": 2823.4, "max_memory_gb": 19.16}
|
| 230 |
+
{"step": 230, "steps": 500, "lr": 1.2626715226552741e-05, "loss": 6.441245480895671e-05, "grad_norm": 0.0031673694029450417, "loss_scale": 4.0, "retries": 0, "seconds": 2835.9, "max_memory_gb": 19.16}
|
| 231 |
+
{"step": 231, "steps": 500, "lr": 1.2569225474647482e-05, "loss": 0.0010476858546310682, "grad_norm": 0.07493429630994797, "loss_scale": 4.0, "retries": 0, "seconds": 2848.3, "max_memory_gb": 19.16}
|
| 232 |
+
{"step": 232, "steps": 500, "lr": 1.2511669608831483e-05, "loss": 0.0015850410415083618, "grad_norm": 0.09738493710756302, "loss_scale": 4.0, "retries": 0, "seconds": 2860.7, "max_memory_gb": 19.16}
|
| 233 |
+
{"step": 233, "steps": 500, "lr": 1.2454050054022921e-05, "loss": 0.0134211464631455, "grad_norm": 0.8045576214790344, "loss_scale": 4.0, "retries": 0, "seconds": 2873.1, "max_memory_gb": 19.16}
|
| 234 |
+
{"step": 234, "steps": 500, "lr": 1.2396369237823284e-05, "loss": 0.0005463696926555756, "grad_norm": 0.08607921749353409, "loss_scale": 4.0, "retries": 0, "seconds": 2885.5, "max_memory_gb": 19.16}
|
| 235 |
+
{"step": 235, "steps": 500, "lr": 1.2338629590415109e-05, "loss": 0.29727278402753593, "grad_norm": 6.91740608215332, "loss_scale": 4.0, "retries": 0, "seconds": 2898.1, "max_memory_gb": 19.16}
|
| 236 |
+
{"step": 236, "steps": 500, "lr": 1.2280833544459568e-05, "loss": 0.0010873680653101687, "grad_norm": 0.22707706689834595, "loss_scale": 4.0, "retries": 0, "seconds": 2910.9, "max_memory_gb": 19.16}
|
| 237 |
+
{"step": 237, "steps": 500, "lr": 1.2222983534994002e-05, "loss": 0.00018259477003177633, "grad_norm": 0.01027350127696991, "loss_scale": 4.0, "retries": 0, "seconds": 2923.3, "max_memory_gb": 19.16}
|
| 238 |
+
{"step": 238, "steps": 500, "lr": 1.2165081999329318e-05, "loss": 0.008127633674871765, "grad_norm": 0.4385242462158203, "loss_scale": 4.0, "retries": 0, "seconds": 2935.4, "max_memory_gb": 19.16}
|
| 239 |
+
{"step": 239, "steps": 500, "lr": 1.2107131376947301e-05, "loss": 0.0007054053221741635, "grad_norm": 0.039298590272665024, "loss_scale": 4.0, "retries": 0, "seconds": 2947.8, "max_memory_gb": 19.16}
|
| 240 |
+
{"step": 240, "steps": 500, "lr": 1.2049134109397833e-05, "loss": 0.0026477332947133, "grad_norm": 0.23670357465744019, "loss_scale": 4.0, "retries": 0, "seconds": 2960.3, "max_memory_gb": 19.16}
|
| 241 |
+
{"step": 241, "steps": 500, "lr": 1.1991092640196032e-05, "loss": 0.055233520805693756, "grad_norm": 1.9894062280654907, "loss_scale": 4.0, "retries": 0, "seconds": 2972.7, "max_memory_gb": 19.16}
|
| 242 |
+
{"step": 242, "steps": 500, "lr": 1.1933009414719298e-05, "loss": 8.67776917927317e-05, "grad_norm": 0.005007793661206961, "loss_scale": 4.0, "retries": 0, "seconds": 2985.5, "max_memory_gb": 19.16}
|
| 243 |
+
{"step": 243, "steps": 500, "lr": 1.1874886880104294e-05, "loss": 0.002155656579006404, "grad_norm": 0.1380593180656433, "loss_scale": 4.0, "retries": 0, "seconds": 2997.9, "max_memory_gb": 19.16}
|
| 244 |
+
{"step": 244, "steps": 500, "lr": 1.181672748514383e-05, "loss": 0.0014801846075584988, "grad_norm": 0.24274323880672455, "loss_scale": 4.0, "retries": 0, "seconds": 3010.5, "max_memory_gb": 19.16}
|
| 245 |
+
{"step": 245, "steps": 500, "lr": 1.1758533680183702e-05, "loss": 0.00014087356839938408, "grad_norm": 0.007506936322897673, "loss_scale": 4.0, "retries": 0, "seconds": 3022.6, "max_memory_gb": 19.16}
|
| 246 |
+
{"step": 246, "steps": 500, "lr": 1.1700307917019457e-05, "loss": 0.00132321189078042, "grad_norm": 0.3148077428340912, "loss_scale": 4.0, "retries": 0, "seconds": 3035.0, "max_memory_gb": 19.16}
|
| 247 |
+
{"step": 247, "steps": 500, "lr": 1.164205264879309e-05, "loss": 0.045899262505155414, "grad_norm": 2.5595200061798096, "loss_scale": 4.0, "retries": 0, "seconds": 3047.5, "max_memory_gb": 19.16}
|
| 248 |
+
{"step": 248, "steps": 500, "lr": 1.1583770329889687e-05, "loss": 0.00015491239157066678, "grad_norm": 0.011142429895699024, "loss_scale": 4.0, "retries": 0, "seconds": 3059.8, "max_memory_gb": 19.16}
|
| 249 |
+
{"step": 249, "steps": 500, "lr": 1.152546341583402e-05, "loss": 0.0001711938790446187, "grad_norm": 0.007633868604898453, "loss_scale": 4.0, "retries": 0, "seconds": 3072.3, "max_memory_gb": 19.16}
|
| 250 |
+
{"step": 250, "steps": 500, "lr": 1.1467134363187098e-05, "loss": 6.124041238499765e-05, "grad_norm": 0.002986983861774206, "loss_scale": 4.0, "retries": 0, "seconds": 3084.7, "max_memory_gb": 19.16}
|
| 251 |
+
{"step": 251, "steps": 500, "lr": 1.1408785629442657e-05, "loss": 0.050894346275590224, "grad_norm": 1.2870993614196777, "loss_scale": 4.0, "retries": 0, "seconds": 3097.0, "max_memory_gb": 19.16}
|
| 252 |
+
{"step": 252, "steps": 500, "lr": 1.1350419672923639e-05, "loss": 3.4318375188746586e-05, "grad_norm": 0.0013954306486994028, "loss_scale": 4.0, "retries": 0, "seconds": 3109.5, "max_memory_gb": 19.16}
|
| 253 |
+
{"step": 253, "steps": 500, "lr": 1.1292038952678597e-05, "loss": 0.0009545441095475193, "grad_norm": 0.0778486430644989, "loss_scale": 4.0, "retries": 0, "seconds": 3121.9, "max_memory_gb": 19.16}
|
| 254 |
+
{"step": 254, "steps": 500, "lr": 1.1233645928378104e-05, "loss": 0.006165579534655858, "grad_norm": 0.7639111876487732, "loss_scale": 4.0, "retries": 0, "seconds": 3134.4, "max_memory_gb": 19.16}
|
| 255 |
+
{"step": 255, "steps": 500, "lr": 1.117524306021113e-05, "loss": 7.871321749322844e-05, "grad_norm": 0.0033469039481133223, "loss_scale": 4.0, "retries": 0, "seconds": 3146.8, "max_memory_gb": 19.16}
|
| 256 |
+
{"step": 256, "steps": 500, "lr": 1.1116832808781377e-05, "loss": 9.81140717078494e-05, "grad_norm": 0.00592323811724782, "loss_scale": 4.0, "retries": 0, "seconds": 3159.4, "max_memory_gb": 19.16}
|
| 257 |
+
{"step": 257, "steps": 500, "lr": 1.1058417635003616e-05, "loss": 9.629694753308307e-05, "grad_norm": 0.005199533421546221, "loss_scale": 4.0, "retries": 0, "seconds": 3171.7, "max_memory_gb": 19.16}
|
| 258 |
+
{"step": 258, "steps": 500, "lr": 1.1e-05, "loss": 0.017958682478406214, "grad_norm": 1.8180508613586426, "loss_scale": 4.0, "retries": 0, "seconds": 3184.1, "max_memory_gb": 19.16}
|
| 259 |
+
{"step": 259, "steps": 500, "lr": 1.0941582364996383e-05, "loss": 9.626446574111986e-05, "grad_norm": 0.0052261957898736, "loss_scale": 4.0, "retries": 0, "seconds": 3196.5, "max_memory_gb": 19.16}
|
| 260 |
+
{"step": 260, "steps": 500, "lr": 1.0883167191218624e-05, "loss": 0.003128613956011283, "grad_norm": 0.13721096515655518, "loss_scale": 4.0, "retries": 0, "seconds": 3209.0, "max_memory_gb": 19.16}
|
| 261 |
+
{"step": 261, "steps": 500, "lr": 1.0824756939788867e-05, "loss": 0.03684665275864839, "grad_norm": 1.1485472917556763, "loss_scale": 4.0, "retries": 0, "seconds": 3221.5, "max_memory_gb": 19.16}
|
| 262 |
+
{"step": 262, "steps": 500, "lr": 1.0766354071621896e-05, "loss": 0.00040840810195484005, "grad_norm": 0.038897980004549026, "loss_scale": 4.0, "retries": 0, "seconds": 3233.9, "max_memory_gb": 19.16}
|
| 263 |
+
{"step": 263, "steps": 500, "lr": 1.0707961047321403e-05, "loss": 0.11452487443394882, "grad_norm": 2.359825849533081, "loss_scale": 4.0, "retries": 0, "seconds": 3246.3, "max_memory_gb": 19.16}
|
| 264 |
+
{"step": 264, "steps": 500, "lr": 1.064958032707636e-05, "loss": 0.0016523307022920423, "grad_norm": 0.10568113625049591, "loss_scale": 4.0, "retries": 0, "seconds": 3258.8, "max_memory_gb": 19.16}
|
| 265 |
+
{"step": 265, "steps": 500, "lr": 1.0591214370557342e-05, "loss": 0.35374049228104043, "grad_norm": 31.04873275756836, "loss_scale": 4.0, "retries": 0, "seconds": 3271.2, "max_memory_gb": 19.16}
|
| 266 |
+
{"step": 266, "steps": 500, "lr": 1.0532865636812903e-05, "loss": 0.0012207363906368714, "grad_norm": 0.1871935874223709, "loss_scale": 4.0, "retries": 0, "seconds": 3283.4, "max_memory_gb": 19.16}
|
| 267 |
+
{"step": 267, "steps": 500, "lr": 1.047453658416598e-05, "loss": 5.66415970692059e-05, "grad_norm": 0.002660239115357399, "loss_scale": 4.0, "retries": 0, "seconds": 3295.7, "max_memory_gb": 19.16}
|
| 268 |
+
{"step": 268, "steps": 500, "lr": 1.0416229670110312e-05, "loss": 2.4284363024081177e-05, "grad_norm": 0.001056751236319542, "loss_scale": 4.0, "retries": 0, "seconds": 3308.5, "max_memory_gb": 19.16}
|
| 269 |
+
{"step": 269, "steps": 500, "lr": 1.0357947351206909e-05, "loss": 0.014961489455133276, "grad_norm": 0.6566606163978577, "loss_scale": 4.0, "retries": 0, "seconds": 3321.2, "max_memory_gb": 19.16}
|
| 270 |
+
{"step": 270, "steps": 500, "lr": 1.029969208298054e-05, "loss": 0.6137308955154572, "grad_norm": 4.628920555114746, "loss_scale": 4.0, "retries": 0, "seconds": 3333.6, "max_memory_gb": 19.16}
|
| 271 |
+
{"step": 271, "steps": 500, "lr": 1.02414663198163e-05, "loss": 0.0004535825647966618, "grad_norm": 0.03673001751303673, "loss_scale": 4.0, "retries": 0, "seconds": 3346.2, "max_memory_gb": 19.16}
|
| 272 |
+
{"step": 272, "steps": 500, "lr": 1.0183272514856171e-05, "loss": 0.021197692690507287, "grad_norm": 1.1359689235687256, "loss_scale": 4.0, "retries": 0, "seconds": 3358.9, "max_memory_gb": 19.16}
|
| 273 |
+
{"step": 273, "steps": 500, "lr": 1.0125113119895709e-05, "loss": 0.2022900088197065, "grad_norm": 7.642131805419922, "loss_scale": 4.0, "retries": 0, "seconds": 3371.2, "max_memory_gb": 19.16}
|
| 274 |
+
{"step": 274, "steps": 500, "lr": 1.0066990585280703e-05, "loss": 0.01190144347924228, "grad_norm": 0.8647705316543579, "loss_scale": 4.0, "retries": 0, "seconds": 3383.6, "max_memory_gb": 19.16}
|
| 275 |
+
{"step": 275, "steps": 500, "lr": 1.0008907359803967e-05, "loss": 0.0019498337415600729, "grad_norm": 0.15990954637527466, "loss_scale": 4.0, "retries": 0, "seconds": 3395.6, "max_memory_gb": 19.16}
|
| 276 |
+
{"step": 276, "steps": 500, "lr": 9.950865890602168e-06, "loss": 0.09167139936804537, "grad_norm": 2.18131685256958, "loss_scale": 4.0, "retries": 0, "seconds": 3408.4, "max_memory_gb": 19.16}
|
| 277 |
+
{"step": 277, "steps": 500, "lr": 9.892868623052698e-06, "loss": 0.1060468539639924, "grad_norm": 5.428877353668213, "loss_scale": 4.0, "retries": 0, "seconds": 3421.1, "max_memory_gb": 19.16}
|
| 278 |
+
{"step": 278, "steps": 500, "lr": 9.834918000670681e-06, "loss": 0.00959937639199702, "grad_norm": 0.9202473759651184, "loss_scale": 4.0, "retries": 0, "seconds": 3433.5, "max_memory_gb": 19.16}
|
| 279 |
+
{"step": 279, "steps": 500, "lr": 9.777016465006e-06, "loss": 6.16968933878132e-05, "grad_norm": 0.0021409662440419197, "loss_scale": 4.0, "retries": 0, "seconds": 3446.0, "max_memory_gb": 19.16}
|
| 280 |
+
{"step": 280, "steps": 500, "lr": 9.719166455540433e-06, "loss": 3.971619993850792e-05, "grad_norm": 0.0018143197521567345, "loss_scale": 4.0, "retries": 0, "seconds": 3458.1, "max_memory_gb": 19.16}
|
| 281 |
+
{"step": 281, "steps": 500, "lr": 9.661370409584892e-06, "loss": 0.253127278139069, "grad_norm": 7.480123519897461, "loss_scale": 4.0, "retries": 0, "seconds": 3470.6, "max_memory_gb": 19.16}
|
| 282 |
+
{"step": 282, "steps": 500, "lr": 9.603630762176718e-06, "loss": 0.09221740051525146, "grad_norm": 1.7745463848114014, "loss_scale": 4.0, "retries": 0, "seconds": 3483.1, "max_memory_gb": 19.16}
|
| 283 |
+
{"step": 283, "steps": 500, "lr": 9.54594994597708e-06, "loss": 0.0027712046069568305, "grad_norm": 0.33692672848701477, "loss_scale": 4.0, "retries": 0, "seconds": 3495.5, "max_memory_gb": 19.16}
|
| 284 |
+
{"step": 284, "steps": 500, "lr": 9.488330391168515e-06, "loss": 0.011751417770017012, "grad_norm": 0.4246314764022827, "loss_scale": 4.0, "retries": 0, "seconds": 3507.9, "max_memory_gb": 19.16}
|
| 285 |
+
{"step": 285, "steps": 500, "lr": 9.430774525352518e-06, "loss": 0.0010231712006546188, "grad_norm": 0.04163600504398346, "loss_scale": 4.0, "retries": 0, "seconds": 3520.3, "max_memory_gb": 19.16}
|
| 286 |
+
{"step": 286, "steps": 500, "lr": 9.373284773447258e-06, "loss": 0.0012936372945311157, "grad_norm": 0.10030714422464371, "loss_scale": 4.0, "retries": 0, "seconds": 3532.7, "max_memory_gb": 19.16}
|
| 287 |
+
{"step": 287, "steps": 500, "lr": 9.315863557585435e-06, "loss": 0.0037019556265030155, "grad_norm": 0.45315614342689514, "loss_scale": 4.0, "retries": 0, "seconds": 3545.2, "max_memory_gb": 19.16}
|
| 288 |
+
{"step": 288, "steps": 500, "lr": 9.258513297012203e-06, "loss": 6.452378279933865e-05, "grad_norm": 0.0027862628921866417, "loss_scale": 4.0, "retries": 0, "seconds": 3557.7, "max_memory_gb": 19.16}
|
| 289 |
+
{"step": 289, "steps": 500, "lr": 9.201236407983281e-06, "loss": 0.0020400153187978276, "grad_norm": 0.19505952298641205, "loss_scale": 4.0, "retries": 0, "seconds": 3570.1, "max_memory_gb": 19.16}
|
| 290 |
+
{"step": 290, "steps": 500, "lr": 9.144035303663109e-06, "loss": 0.00033906190406505425, "grad_norm": 0.018431657925248146, "loss_scale": 4.0, "retries": 0, "seconds": 3582.8, "max_memory_gb": 19.16}
|
| 291 |
+
{"step": 291, "steps": 500, "lr": 9.08691239402321e-06, "loss": 0.00042151565030611567, "grad_norm": 0.01999763958156109, "loss_scale": 4.0, "retries": 0, "seconds": 3595.3, "max_memory_gb": 19.16}
|
| 292 |
+
{"step": 292, "steps": 500, "lr": 9.029870085740647e-06, "loss": 0.0001542668213849135, "grad_norm": 0.007536076474934816, "loss_scale": 4.0, "retries": 0, "seconds": 3607.7, "max_memory_gb": 19.16}
|
| 293 |
+
{"step": 293, "steps": 500, "lr": 8.972910782096616e-06, "loss": 0.024975296594088932, "grad_norm": 2.3101422786712646, "loss_scale": 4.0, "retries": 0, "seconds": 3619.8, "max_memory_gb": 19.16}
|
| 294 |
+
{"step": 294, "steps": 500, "lr": 8.916036882875198e-06, "loss": 6.996309834180052e-05, "grad_norm": 0.0030438860412687063, "loss_scale": 4.0, "retries": 0, "seconds": 3632.1, "max_memory_gb": 19.16}
|
| 295 |
+
{"step": 295, "steps": 500, "lr": 8.859250784262256e-06, "loss": 0.008924091279554472, "grad_norm": 0.5903571248054504, "loss_scale": 4.0, "retries": 0, "seconds": 3644.5, "max_memory_gb": 19.16}
|
| 296 |
+
{"step": 296, "steps": 500, "lr": 8.802554878744479e-06, "loss": 0.007176157042058602, "grad_norm": 0.3520984351634979, "loss_scale": 4.0, "retries": 0, "seconds": 3656.9, "max_memory_gb": 19.16}
|
| 297 |
+
{"step": 297, "steps": 500, "lr": 8.74595155500857e-06, "loss": 0.0001392547996268625, "grad_norm": 0.00581852113828063, "loss_scale": 4.0, "retries": 0, "seconds": 3669.3, "max_memory_gb": 19.16}
|
| 298 |
+
{"step": 298, "steps": 500, "lr": 8.689443197840635e-06, "loss": 5.992152289929271e-05, "grad_norm": 0.002596857724711299, "loss_scale": 4.0, "retries": 0, "seconds": 3681.7, "max_memory_gb": 19.16}
|
| 299 |
+
{"step": 299, "steps": 500, "lr": 8.633032188025669e-06, "loss": 0.0005910017238399881, "grad_norm": 0.024177083745598793, "loss_scale": 4.0, "retries": 0, "seconds": 3694.1, "max_memory_gb": 19.16}
|
| 300 |
+
{"step": 300, "steps": 500, "lr": 8.576720902247288e-06, "loss": 0.005527148139679835, "grad_norm": 0.24666081368923187, "loss_scale": 4.0, "retries": 0, "seconds": 3706.7, "max_memory_gb": 19.16}
|
| 301 |
+
{"step": 301, "steps": 500, "lr": 8.52051171298758e-06, "loss": 0.002740451324585406, "grad_norm": 0.13802854716777802, "loss_scale": 4.0, "retries": 0, "seconds": 3719.5, "max_memory_gb": 19.16}
|
| 302 |
+
{"step": 302, "steps": 500, "lr": 8.464406988427132e-06, "loss": 0.32676554715088846, "grad_norm": 5.717441558837891, "loss_scale": 4.0, "retries": 0, "seconds": 3731.9, "max_memory_gb": 19.16}
|
| 303 |
+
{"step": 303, "steps": 500, "lr": 8.40840909234529e-06, "loss": 0.0001355250949744273, "grad_norm": 0.006526165176182985, "loss_scale": 4.0, "retries": 0, "seconds": 3744.3, "max_memory_gb": 19.16}
|
| 304 |
+
{"step": 304, "steps": 500, "lr": 8.352520384020533e-06, "loss": 9.74757130775701e-05, "grad_norm": 0.004684180952608585, "loss_scale": 4.0, "retries": 0, "seconds": 3756.4, "max_memory_gb": 19.16}
|
| 305 |
+
{"step": 305, "steps": 500, "lr": 8.296743218131103e-06, "loss": 9.174283564499837e-05, "grad_norm": 0.0049806274473667145, "loss_scale": 4.0, "retries": 0, "seconds": 3768.7, "max_memory_gb": 19.16}
|
| 306 |
+
{"step": 306, "steps": 500, "lr": 8.241079944655777e-06, "loss": 0.020373602175205008, "grad_norm": 2.408829927444458, "loss_scale": 4.0, "retries": 0, "seconds": 3781.1, "max_memory_gb": 19.16}
|
| 307 |
+
{"step": 307, "steps": 500, "lr": 8.185532908774872e-06, "loss": 0.0128924469546932, "grad_norm": 1.2610738277435303, "loss_scale": 4.0, "retries": 0, "seconds": 3793.5, "max_memory_gb": 19.16}
|
| 308 |
+
{"step": 308, "steps": 500, "lr": 8.130104450771434e-06, "loss": 0.00019162874310232135, "grad_norm": 0.009931759908795357, "loss_scale": 4.0, "retries": 0, "seconds": 3805.9, "max_memory_gb": 19.16}
|
| 309 |
+
{"step": 309, "steps": 500, "lr": 8.074796905932634e-06, "loss": 0.06480715823719407, "grad_norm": 4.36243200302124, "loss_scale": 4.0, "retries": 0, "seconds": 3818.2, "max_memory_gb": 19.16}
|
| 310 |
+
{"step": 310, "steps": 500, "lr": 8.019612604451395e-06, "loss": 0.00013242721296726563, "grad_norm": 0.006824493873864412, "loss_scale": 4.0, "retries": 0, "seconds": 3830.6, "max_memory_gb": 19.16}
|
| 311 |
+
{"step": 311, "steps": 500, "lr": 7.964553871328185e-06, "loss": 0.001984990448902899, "grad_norm": 0.1334272027015686, "loss_scale": 4.0, "retries": 0, "seconds": 3843.3, "max_memory_gb": 19.16}
|
| 312 |
+
{"step": 312, "steps": 500, "lr": 7.909623026273106e-06, "loss": 8.508294113146968e-05, "grad_norm": 0.0035627298057079315, "loss_scale": 4.0, "retries": 0, "seconds": 3855.7, "max_memory_gb": 19.16}
|
| 313 |
+
{"step": 313, "steps": 500, "lr": 7.854822383608116e-06, "loss": 0.03385149000662192, "grad_norm": 1.054113745689392, "loss_scale": 4.0, "retries": 0, "seconds": 3867.3, "max_memory_gb": 19.16}
|
| 314 |
+
{"step": 314, "steps": 500, "lr": 7.800154252169558e-06, "loss": 0.0012627081414807021, "grad_norm": 0.090950608253479, "loss_scale": 4.0, "retries": 0, "seconds": 3879.7, "max_memory_gb": 19.16}
|
| 315 |
+
{"step": 315, "steps": 500, "lr": 7.745620935210861e-06, "loss": 5.5605831549469684e-05, "grad_norm": 0.0023149694316089153, "loss_scale": 4.0, "retries": 0, "seconds": 3891.9, "max_memory_gb": 19.16}
|
| 316 |
+
{"step": 316, "steps": 500, "lr": 7.691224730305518e-06, "loss": 0.01030859976173624, "grad_norm": 0.8018465638160706, "loss_scale": 4.0, "retries": 0, "seconds": 3904.4, "max_memory_gb": 19.16}
|
| 317 |
+
{"step": 317, "steps": 500, "lr": 7.636967929250276e-06, "loss": 0.3670268424977456, "grad_norm": 5.631945610046387, "loss_scale": 4.0, "retries": 0, "seconds": 3916.7, "max_memory_gb": 19.16}
|
| 318 |
+
{"step": 318, "steps": 500, "lr": 7.5828528179685755e-06, "loss": 0.007283190099707844, "grad_norm": 0.25145816802978516, "loss_scale": 4.0, "retries": 0, "seconds": 3929.2, "max_memory_gb": 19.16}
|
| 319 |
+
{"step": 319, "steps": 500, "lr": 7.528881676414252e-06, "loss": 0.0024579357383978304, "grad_norm": 0.11473747342824936, "loss_scale": 4.0, "retries": 0, "seconds": 3941.6, "max_memory_gb": 19.16}
|
| 320 |
+
{"step": 320, "steps": 500, "lr": 7.475056778475468e-06, "loss": 7.950111937434073e-05, "grad_norm": 0.004522442817687988, "loss_scale": 4.0, "retries": 0, "seconds": 3954.3, "max_memory_gb": 19.16}
|
| 321 |
+
{"step": 321, "steps": 500, "lr": 7.421380391878924e-06, "loss": 0.15133627754255613, "grad_norm": 7.397266864776611, "loss_scale": 4.0, "retries": 0, "seconds": 3966.5, "max_memory_gb": 19.16}
|
| 322 |
+
{"step": 322, "steps": 500, "lr": 7.367854778094291e-06, "loss": 0.0002702407180508585, "grad_norm": 0.011908890679478645, "loss_scale": 4.0, "retries": 0, "seconds": 3979.0, "max_memory_gb": 19.16}
|
| 323 |
+
{"step": 323, "steps": 500, "lr": 7.314482192238962e-06, "loss": 6.053735706146313e-05, "grad_norm": 0.0024007554166018963, "loss_scale": 4.0, "retries": 0, "seconds": 3991.4, "max_memory_gb": 19.16}
|
| 324 |
+
{"step": 324, "steps": 500, "lr": 7.261264882983024e-06, "loss": 0.049398936805136955, "grad_norm": 1.9227278232574463, "loss_scale": 4.0, "retries": 0, "seconds": 4003.8, "max_memory_gb": 19.16}
|
| 325 |
+
{"step": 325, "steps": 500, "lr": 7.208205092454502e-06, "loss": 0.0003553615116851978, "grad_norm": 0.024007312953472137, "loss_scale": 4.0, "retries": 0, "seconds": 4015.9, "max_memory_gb": 19.16}
|
| 326 |
+
{"step": 326, "steps": 500, "lr": 7.155305056144935e-06, "loss": 0.0015102590672985243, "grad_norm": 0.11390697211027145, "loss_scale": 4.0, "retries": 0, "seconds": 4028.3, "max_memory_gb": 19.16}
|
| 327 |
+
{"step": 327, "steps": 500, "lr": 7.102567002815155e-06, "loss": 0.00024985514098509816, "grad_norm": 0.016956575214862823, "loss_scale": 4.0, "retries": 0, "seconds": 4040.8, "max_memory_gb": 19.16}
|
| 328 |
+
{"step": 328, "steps": 500, "lr": 7.0499931544013985e-06, "loss": 0.06287071209721326, "grad_norm": 4.6633076667785645, "loss_scale": 4.0, "retries": 0, "seconds": 4052.9, "max_memory_gb": 19.16}
|
| 329 |
+
{"step": 329, "steps": 500, "lr": 6.997585725921701e-06, "loss": 4.823112956583745e-05, "grad_norm": 0.002085720654577017, "loss_scale": 4.0, "retries": 0, "seconds": 4065.2, "max_memory_gb": 19.16}
|
| 330 |
+
{"step": 330, "steps": 500, "lr": 6.94534692538256e-06, "loss": 0.0004900272629749288, "grad_norm": 0.027110235765576363, "loss_scale": 4.0, "retries": 0, "seconds": 4077.5, "max_memory_gb": 19.16}
|
| 331 |
+
{"step": 331, "steps": 500, "lr": 6.893278953685919e-06, "loss": 0.04399238627518995, "grad_norm": 1.590151309967041, "loss_scale": 4.0, "retries": 0, "seconds": 4090.2, "max_memory_gb": 19.16}
|
| 332 |
+
{"step": 332, "steps": 500, "lr": 6.841384004536434e-06, "loss": 0.0002791421991093754, "grad_norm": 0.012655661441385746, "loss_scale": 4.0, "retries": 0, "seconds": 4102.6, "max_memory_gb": 19.16}
|
| 333 |
+
{"step": 333, "steps": 500, "lr": 6.789664264349063e-06, "loss": 0.0005826356493721299, "grad_norm": 0.026561785489320755, "loss_scale": 4.0, "retries": 0, "seconds": 4115.2, "max_memory_gb": 19.16}
|
| 334 |
+
{"step": 334, "steps": 500, "lr": 6.738121912156913e-06, "loss": 0.00035363886123398265, "grad_norm": 0.014303409494459629, "loss_scale": 4.0, "retries": 0, "seconds": 4127.6, "max_memory_gb": 19.16}
|
| 335 |
+
{"step": 335, "steps": 500, "lr": 6.686759119519493e-06, "loss": 0.00024399566996955647, "grad_norm": 0.01659475453197956, "loss_scale": 4.0, "retries": 0, "seconds": 4140.3, "max_memory_gb": 19.16}
|
| 336 |
+
{"step": 336, "steps": 500, "lr": 6.63557805043115e-06, "loss": 0.03288290367212001, "grad_norm": 1.523998737335205, "loss_scale": 4.0, "retries": 0, "seconds": 4152.7, "max_memory_gb": 19.16}
|
| 337 |
+
{"step": 337, "steps": 500, "lr": 6.5845808612299715e-06, "loss": 0.0017080101061540374, "grad_norm": 0.07228218764066696, "loss_scale": 4.0, "retries": 0, "seconds": 4165.1, "max_memory_gb": 19.16}
|
| 338 |
+
{"step": 338, "steps": 500, "lr": 6.5337697005068745e-06, "loss": 9.111203790852912e-05, "grad_norm": 0.0034891183022409678, "loss_scale": 4.0, "retries": 0, "seconds": 4177.1, "max_memory_gb": 19.16}
|
| 339 |
+
{"step": 339, "steps": 500, "lr": 6.48314670901512e-06, "loss": 0.00018465828728024422, "grad_norm": 0.009213286451995373, "loss_scale": 4.0, "retries": 0, "seconds": 4189.4, "max_memory_gb": 19.16}
|
| 340 |
+
{"step": 340, "steps": 500, "lr": 6.4327140195801e-06, "loss": 0.036452536843380685, "grad_norm": 2.7637081146240234, "loss_scale": 4.0, "retries": 0, "seconds": 4201.7, "max_memory_gb": 19.16}
|
| 341 |
+
{"step": 341, "steps": 500, "lr": 6.382473757009482e-06, "loss": 0.025243216742489173, "grad_norm": 2.4171626567840576, "loss_scale": 4.0, "retries": 0, "seconds": 4214.1, "max_memory_gb": 19.16}
|
| 342 |
+
{"step": 342, "steps": 500, "lr": 6.332428038003706e-06, "loss": 3.412314620086157e-05, "grad_norm": 0.0017921172548085451, "loss_scale": 4.0, "retries": 0, "seconds": 4226.3, "max_memory_gb": 19.16}
|
| 343 |
+
{"step": 343, "steps": 500, "lr": 6.282578971066761e-06, "loss": 3.5302959310131015e-05, "grad_norm": 0.0019491759594529867, "loss_scale": 4.0, "retries": 0, "seconds": 4238.7, "max_memory_gb": 19.16}
|
| 344 |
+
{"step": 344, "steps": 500, "lr": 6.232928656417414e-06, "loss": 0.00014341606581974085, "grad_norm": 0.012384884990751743, "loss_scale": 4.0, "retries": 0, "seconds": 4250.8, "max_memory_gb": 19.16}
|
| 345 |
+
{"step": 345, "steps": 500, "lr": 6.183479185900656e-06, "loss": 0.0019485126751736072, "grad_norm": 0.23261606693267822, "loss_scale": 4.0, "retries": 0, "seconds": 4263.2, "max_memory_gb": 19.16}
|
| 346 |
+
{"step": 346, "steps": 500, "lr": 6.134232642899625e-06, "loss": 0.010661344257940833, "grad_norm": 0.6862505078315735, "loss_scale": 4.0, "retries": 0, "seconds": 4275.6, "max_memory_gb": 19.16}
|
| 347 |
+
{"step": 347, "steps": 500, "lr": 6.085191102247789e-06, "loss": 0.00011166512862620888, "grad_norm": 0.008754328824579716, "loss_scale": 4.0, "retries": 0, "seconds": 4287.8, "max_memory_gb": 19.16}
|
| 348 |
+
{"step": 348, "steps": 500, "lr": 6.036356630141565e-06, "loss": 0.0006203108049579564, "grad_norm": 0.06149661913514137, "loss_scale": 4.0, "retries": 0, "seconds": 4300.4, "max_memory_gb": 19.17}
|
| 349 |
+
{"step": 349, "steps": 500, "lr": 5.987731284053239e-06, "loss": 7.631029256639398e-05, "grad_norm": 0.0038208451587706804, "loss_scale": 4.0, "retries": 0, "seconds": 4312.7, "max_memory_gb": 19.17}
|
| 350 |
+
{"step": 350, "steps": 500, "lr": 5.9393171126443e-06, "loss": 0.00023111804913740619, "grad_norm": 0.012280014343559742, "loss_scale": 4.0, "retries": 0, "seconds": 4325.2, "max_memory_gb": 19.17}
|
| 351 |
+
{"step": 351, "steps": 500, "lr": 5.89111615567911e-06, "loss": 5.303057568668961e-05, "grad_norm": 0.00257120537571609, "loss_scale": 4.0, "retries": 0, "seconds": 4337.5, "max_memory_gb": 19.17}
|
| 352 |
+
{"step": 352, "steps": 500, "lr": 5.843130443938982e-06, "loss": 0.04376973126453654, "grad_norm": 4.212891101837158, "loss_scale": 4.0, "retries": 0, "seconds": 4349.9, "max_memory_gb": 19.17}
|
| 353 |
+
{"step": 353, "steps": 500, "lr": 5.795361999136612e-06, "loss": 0.004904815478861608, "grad_norm": 0.22340500354766846, "loss_scale": 4.0, "retries": 0, "seconds": 4362.6, "max_memory_gb": 19.17}
|
| 354 |
+
{"step": 354, "steps": 500, "lr": 5.747812833830894e-06, "loss": 0.260514696069007, "grad_norm": 2.824404001235962, "loss_scale": 4.0, "retries": 0, "seconds": 4375.5, "max_memory_gb": 19.17}
|
| 355 |
+
{"step": 355, "steps": 500, "lr": 5.700484951342151e-06, "loss": 4.7368665031461376e-05, "grad_norm": 0.0019438171293586493, "loss_scale": 4.0, "retries": 0, "seconds": 4388.4, "max_memory_gb": 19.17}
|
| 356 |
+
{"step": 356, "steps": 500, "lr": 5.6533803456677035e-06, "loss": 0.00011896619195474821, "grad_norm": 0.011635109782218933, "loss_scale": 4.0, "retries": 0, "seconds": 4400.9, "max_memory_gb": 19.17}
|
| 357 |
+
{"step": 357, "steps": 500, "lr": 5.606501001397878e-06, "loss": 0.0016154664166127088, "grad_norm": 0.07291790097951889, "loss_scale": 4.0, "retries": 0, "seconds": 4413.6, "max_memory_gb": 19.17}
|
| 358 |
+
{"step": 358, "steps": 500, "lr": 5.559848893632384e-06, "loss": 0.0003252949594170218, "grad_norm": 0.022934565320611, "loss_scale": 4.0, "retries": 0, "seconds": 4426.1, "max_memory_gb": 19.17}
|
| 359 |
+
{"step": 359, "steps": 500, "lr": 5.513425987897107e-06, "loss": 0.00012984833816822382, "grad_norm": 0.007518356200307608, "loss_scale": 4.0, "retries": 0, "seconds": 4438.7, "max_memory_gb": 19.17}
|
| 360 |
+
{"step": 360, "steps": 500, "lr": 5.467234240061293e-06, "loss": 0.15884469468468804, "grad_norm": 8.55505084991455, "loss_scale": 4.0, "retries": 0, "seconds": 4450.9, "max_memory_gb": 19.17}
|
| 361 |
+
{"step": 361, "steps": 500, "lr": 5.421275596255145e-06, "loss": 0.053619838764368745, "grad_norm": 3.269211530685425, "loss_scale": 4.0, "retries": 0, "seconds": 4463.6, "max_memory_gb": 19.17}
|
| 362 |
+
{"step": 362, "steps": 500, "lr": 5.375551992787835e-06, "loss": 9.520176528677769e-05, "grad_norm": 0.007023792713880539, "loss_scale": 4.0, "retries": 0, "seconds": 4475.8, "max_memory_gb": 19.17}
|
| 363 |
+
{"step": 363, "steps": 500, "lr": 5.330065356065913e-06, "loss": 0.14381937227602684, "grad_norm": 2.425326347351074, "loss_scale": 4.0, "retries": 0, "seconds": 4488.3, "max_memory_gb": 19.17}
|
| 364 |
+
{"step": 364, "steps": 500, "lr": 5.284817602512153e-06, "loss": 0.06357772526573834, "grad_norm": 1.7965070009231567, "loss_scale": 4.0, "retries": 0, "seconds": 4501.5, "max_memory_gb": 19.17}
|
| 365 |
+
{"step": 365, "steps": 500, "lr": 5.239810638484819e-06, "loss": 0.04381445911390488, "grad_norm": 2.670266628265381, "loss_scale": 4.0, "retries": 0, "seconds": 4514.0, "max_memory_gb": 19.17}
|
| 366 |
+
{"step": 366, "steps": 500, "lr": 5.195046360197325e-06, "loss": 0.0213383124201879, "grad_norm": 0.8477484583854675, "loss_scale": 4.0, "retries": 0, "seconds": 4526.5, "max_memory_gb": 19.17}
|
| 367 |
+
{"step": 367, "steps": 500, "lr": 5.150526653638363e-06, "loss": 5.4658005123800734e-05, "grad_norm": 0.0032407999970018864, "loss_scale": 4.0, "retries": 0, "seconds": 4538.8, "max_memory_gb": 19.17}
|
| 368 |
+
{"step": 368, "steps": 500, "lr": 5.106253394492435e-06, "loss": 8.263453215368538e-05, "grad_norm": 0.004081536550074816, "loss_scale": 4.0, "retries": 0, "seconds": 4551.4, "max_memory_gb": 19.17}
|
| 369 |
+
{"step": 369, "steps": 500, "lr": 5.062228448060833e-06, "loss": 0.002260465312179928, "grad_norm": 0.25414910912513733, "loss_scale": 4.0, "retries": 0, "seconds": 4563.8, "max_memory_gb": 19.17}
|
| 370 |
+
{"step": 370, "steps": 500, "lr": 5.018453669183048e-06, "loss": 4.150918621803612e-05, "grad_norm": 0.0020287618972361088, "loss_scale": 4.0, "retries": 0, "seconds": 4576.3, "max_memory_gb": 19.17}
|
| 371 |
+
{"step": 371, "steps": 500, "lr": 4.974930902158616e-06, "loss": 0.006330140588405894, "grad_norm": 0.2712021768093109, "loss_scale": 4.0, "retries": 0, "seconds": 4588.5, "max_memory_gb": 19.17}
|
| 372 |
+
{"step": 372, "steps": 500, "lr": 4.931661980669439e-06, "loss": 0.0004554374429691745, "grad_norm": 0.022016579285264015, "loss_scale": 4.0, "retries": 0, "seconds": 4601.3, "max_memory_gb": 19.17}
|
| 373 |
+
{"step": 373, "steps": 500, "lr": 4.8886487277024854e-06, "loss": 0.0001958613336744913, "grad_norm": 0.01151500828564167, "loss_scale": 4.0, "retries": 0, "seconds": 4614.0, "max_memory_gb": 19.17}
|
| 374 |
+
{"step": 374, "steps": 500, "lr": 4.845892955473037e-06, "loss": 8.102025070916791e-05, "grad_norm": 0.006594622042030096, "loss_scale": 4.0, "retries": 0, "seconds": 4626.5, "max_memory_gb": 19.17}
|
| 375 |
+
{"step": 375, "steps": 500, "lr": 4.80339646534829e-06, "loss": 0.005262033253948381, "grad_norm": 0.24898651242256165, "loss_scale": 4.0, "retries": 0, "seconds": 4639.3, "max_memory_gb": 19.17}
|
| 376 |
+
{"step": 376, "steps": 500, "lr": 4.761161047771502e-06, "loss": 0.00519156746321931, "grad_norm": 0.6910143494606018, "loss_scale": 4.0, "retries": 0, "seconds": 4651.9, "max_memory_gb": 19.17}
|
| 377 |
+
{"step": 377, "steps": 500, "lr": 4.719188482186515e-06, "loss": 0.007182157150287533, "grad_norm": 0.38711515069007874, "loss_scale": 4.0, "retries": 0, "seconds": 4664.3, "max_memory_gb": 19.17}
|
| 378 |
+
{"step": 378, "steps": 500, "lr": 4.677480536962832e-06, "loss": 0.4087511156570995, "grad_norm": 6.351044178009033, "loss_scale": 4.0, "retries": 0, "seconds": 4676.9, "max_memory_gb": 19.17}
|
| 379 |
+
{"step": 379, "steps": 500, "lr": 4.6360389693210725e-06, "loss": 0.012840745952145613, "grad_norm": 1.2674638032913208, "loss_scale": 4.0, "retries": 0, "seconds": 4689.3, "max_memory_gb": 19.17}
|
| 380 |
+
{"step": 380, "steps": 500, "lr": 4.594865525258959e-06, "loss": 0.36315591348399456, "grad_norm": 3.1257548332214355, "loss_scale": 4.0, "retries": 0, "seconds": 4701.0, "max_memory_gb": 19.17}
|
| 381 |
+
{"step": 381, "steps": 500, "lr": 4.5539619394777495e-06, "loss": 0.00019644237700462827, "grad_norm": 0.008035489358007908, "loss_scale": 4.0, "retries": 0, "seconds": 4713.3, "max_memory_gb": 19.17}
|
| 382 |
+
{"step": 382, "steps": 500, "lr": 4.5133299353091496e-06, "loss": 0.0006813895854998526, "grad_norm": 0.04089701920747757, "loss_scale": 4.0, "retries": 0, "seconds": 4725.5, "max_memory_gb": 19.17}
|
| 383 |
+
{"step": 383, "steps": 500, "lr": 4.472971224642723e-06, "loss": 0.13863448725778552, "grad_norm": 4.958308696746826, "loss_scale": 4.0, "retries": 0, "seconds": 4738.1, "max_memory_gb": 19.17}
|
| 384 |
+
{"step": 384, "steps": 500, "lr": 4.43288750785373e-06, "loss": 0.00040811026797820727, "grad_norm": 0.018103409558534622, "loss_scale": 4.0, "retries": 0, "seconds": 4750.6, "max_memory_gb": 19.17}
|
| 385 |
+
{"step": 385, "steps": 500, "lr": 4.393080473731534e-06, "loss": 0.00015256390168164304, "grad_norm": 0.009235242381691933, "loss_scale": 4.0, "retries": 0, "seconds": 4763.4, "max_memory_gb": 19.17}
|
| 386 |
+
{"step": 386, "steps": 500, "lr": 4.353551799408406e-06, "loss": 0.0035289626587071284, "grad_norm": 0.5899673700332642, "loss_scale": 4.0, "retries": 0, "seconds": 4775.8, "max_memory_gb": 19.17}
|
| 387 |
+
{"step": 387, "steps": 500, "lr": 4.314303150288909e-06, "loss": 0.0001502169856786912, "grad_norm": 0.007569527253508568, "loss_scale": 4.0, "retries": 0, "seconds": 4788.3, "max_memory_gb": 19.17}
|
| 388 |
+
{"step": 388, "steps": 500, "lr": 4.275336179979683e-06, "loss": 0.026471046453924497, "grad_norm": 0.9251713156700134, "loss_scale": 4.0, "retries": 0, "seconds": 4800.9, "max_memory_gb": 19.17}
|
| 389 |
+
{"step": 389, "steps": 500, "lr": 4.236652530219822e-06, "loss": 0.00011437892092916711, "grad_norm": 0.007245059125125408, "loss_scale": 4.0, "retries": 0, "seconds": 4813.4, "max_memory_gb": 19.17}
|
| 390 |
+
{"step": 390, "steps": 500, "lr": 4.198253830811677e-06, "loss": 0.05295999058474944, "grad_norm": 3.46614408493042, "loss_scale": 4.0, "retries": 0, "seconds": 4825.9, "max_memory_gb": 19.17}
|
| 391 |
+
{"step": 391, "steps": 500, "lr": 4.160141699552193e-06, "loss": 0.00190088109626045, "grad_norm": 0.14735153317451477, "loss_scale": 4.0, "retries": 0, "seconds": 4838.6, "max_memory_gb": 19.17}
|
| 392 |
+
{"step": 392, "steps": 500, "lr": 4.122317742164759e-06, "loss": 0.0022191256778269644, "grad_norm": 0.11650993674993515, "loss_scale": 4.0, "retries": 0, "seconds": 4851.4, "max_memory_gb": 19.17}
|
| 393 |
+
{"step": 393, "steps": 500, "lr": 4.084783552231548e-06, "loss": 0.21345373788642522, "grad_norm": 5.747021675109863, "loss_scale": 4.0, "retries": 0, "seconds": 4864.1, "max_memory_gb": 19.17}
|
| 394 |
+
{"step": 394, "steps": 500, "lr": 4.047540711126381e-06, "loss": 0.00012360873432209536, "grad_norm": 0.004875665996223688, "loss_scale": 4.0, "retries": 0, "seconds": 4876.6, "max_memory_gb": 19.17}
|
| 395 |
+
{"step": 395, "steps": 500, "lr": 4.010590787948095e-06, "loss": 0.0008604981544728866, "grad_norm": 0.05772696062922478, "loss_scale": 4.0, "retries": 0, "seconds": 4889.2, "max_memory_gb": 19.17}
|
| 396 |
+
{"step": 396, "steps": 500, "lr": 3.973935339454447e-06, "loss": 0.00019625730078328985, "grad_norm": 0.00888798851519823, "loss_scale": 4.0, "retries": 0, "seconds": 4901.8, "max_memory_gb": 19.17}
|
| 397 |
+
{"step": 397, "steps": 500, "lr": 3.9375759099965105e-06, "loss": 0.0010573960525215916, "grad_norm": 0.04386509582400322, "loss_scale": 4.0, "retries": 0, "seconds": 4915.0, "max_memory_gb": 19.17}
|
| 398 |
+
{"step": 398, "steps": 500, "lr": 3.901514031453619e-06, "loss": 0.03534042177768077, "grad_norm": 1.5044738054275513, "loss_scale": 4.0, "retries": 0, "seconds": 4928.1, "max_memory_gb": 19.17}
|
| 399 |
+
{"step": 399, "steps": 500, "lr": 3.865751223168823e-06, "loss": 0.003644407909746407, "grad_norm": 0.19935546815395355, "loss_scale": 4.0, "retries": 0, "seconds": 4940.7, "max_memory_gb": 19.17}
|
| 400 |
+
{"step": 400, "steps": 500, "lr": 3.830288991884876e-06, "loss": 0.0001661983140066199, "grad_norm": 0.007087297737598419, "loss_scale": 4.0, "retries": 0, "seconds": 4953.5, "max_memory_gb": 19.17}
|
| 401 |
+
{"step": 401, "steps": 500, "lr": 3.7951288316807574e-06, "loss": 4.083842511254881e-05, "grad_norm": 0.0015015670796856284, "loss_scale": 4.0, "retries": 0, "seconds": 4966.7, "max_memory_gb": 19.17}
|
| 402 |
+
{"step": 402, "steps": 500, "lr": 3.76027222390872e-06, "loss": 0.006135537133835876, "grad_norm": 0.5081660747528076, "loss_scale": 4.0, "retries": 0, "seconds": 4979.6, "max_memory_gb": 19.17}
|
| 403 |
+
{"step": 403, "steps": 500, "lr": 3.725720637131883e-06, "loss": 0.01675548638728618, "grad_norm": 0.8181615471839905, "loss_scale": 4.0, "retries": 0, "seconds": 4992.0, "max_memory_gb": 19.17}
|
| 404 |
+
{"step": 404, "steps": 500, "lr": 3.6914755270623518e-06, "loss": 5.71586446175781e-05, "grad_norm": 0.0033938181586563587, "loss_scale": 4.0, "retries": 0, "seconds": 5004.5, "max_memory_gb": 19.17}
|
| 405 |
+
{"step": 405, "steps": 500, "lr": 3.6575383364998937e-06, "loss": 0.03312319981206067, "grad_norm": 2.304827928543091, "loss_scale": 4.0, "retries": 0, "seconds": 5016.9, "max_memory_gb": 19.17}
|
| 406 |
+
{"step": 406, "steps": 500, "lr": 3.623910495271154e-06, "loss": 0.00013156665763150696, "grad_norm": 0.005578307900577784, "loss_scale": 4.0, "retries": 0, "seconds": 5029.5, "max_memory_gb": 19.17}
|
| 407 |
+
{"step": 407, "steps": 500, "lr": 3.5905934201693992e-06, "loss": 5.000201769369348e-05, "grad_norm": 0.002667422406375408, "loss_scale": 4.0, "retries": 0, "seconds": 5041.6, "max_memory_gb": 19.17}
|
| 408 |
+
{"step": 408, "steps": 500, "lr": 3.557588514894842e-06, "loss": 0.006075347915658437, "grad_norm": 0.31043216586112976, "loss_scale": 4.0, "retries": 0, "seconds": 5054.0, "max_memory_gb": 19.17}
|
| 409 |
+
{"step": 409, "steps": 500, "lr": 3.524897169995492e-06, "loss": 6.458720671079475e-05, "grad_norm": 0.003121808171272278, "loss_scale": 4.0, "retries": 0, "seconds": 5066.8, "max_memory_gb": 19.17}
|
| 410 |
+
{"step": 410, "steps": 500, "lr": 3.492520762808572e-06, "loss": 0.00037170240963488066, "grad_norm": 0.013812028802931309, "loss_scale": 4.0, "retries": 0, "seconds": 5079.2, "max_memory_gb": 19.17}
|
| 411 |
+
{"step": 411, "steps": 500, "lr": 3.4604606574024906e-06, "loss": 8.440735762798631e-05, "grad_norm": 0.004225461278110743, "loss_scale": 4.0, "retries": 0, "seconds": 5092.0, "max_memory_gb": 19.17}
|
| 412 |
+
{"step": 412, "steps": 500, "lr": 3.428718204519368e-06, "loss": 0.006439617354079598, "grad_norm": 0.216554656624794, "loss_scale": 4.0, "retries": 0, "seconds": 5104.8, "max_memory_gb": 19.17}
|
| 413 |
+
{"step": 413, "steps": 500, "lr": 3.3972947415181376e-06, "loss": 0.0005761140403809475, "grad_norm": 0.030316047370433807, "loss_scale": 4.0, "retries": 0, "seconds": 5117.3, "max_memory_gb": 19.17}
|
| 414 |
+
{"step": 414, "steps": 500, "lr": 3.3661915923181757e-06, "loss": 0.04247296058866823, "grad_norm": 1.0728676319122314, "loss_scale": 4.0, "retries": 0, "seconds": 5129.8, "max_memory_gb": 19.17}
|
| 415 |
+
{"step": 415, "steps": 500, "lr": 3.335410067343559e-06, "loss": 0.005945553096619383, "grad_norm": 0.3586409091949463, "loss_scale": 4.0, "retries": 0, "seconds": 5142.2, "max_memory_gb": 19.17}
|
| 416 |
+
{"step": 416, "steps": 500, "lr": 3.3049514634678196e-06, "loss": 0.001130881861376503, "grad_norm": 0.15017768740653992, "loss_scale": 4.0, "retries": 0, "seconds": 5154.5, "max_memory_gb": 19.17}
|
| 417 |
+
{"step": 417, "steps": 500, "lr": 3.2748170639593357e-06, "loss": 0.0029816974619853395, "grad_norm": 0.17578285932540894, "loss_scale": 4.0, "retries": 0, "seconds": 5166.8, "max_memory_gb": 19.17}
|
| 418 |
+
{"step": 418, "steps": 500, "lr": 3.245008138427235e-06, "loss": 0.16062351218448612, "grad_norm": 8.150918006896973, "loss_scale": 4.0, "retries": 0, "seconds": 5179.1, "max_memory_gb": 19.17}
|
| 419 |
+
{"step": 419, "steps": 500, "lr": 3.215525942767934e-06, "loss": 0.009000644126982397, "grad_norm": 0.3600267767906189, "loss_scale": 4.0, "retries": 0, "seconds": 5191.6, "max_memory_gb": 19.17}
|
| 420 |
+
{"step": 420, "steps": 500, "lr": 3.1863717191122057e-06, "loss": 0.20285998306140776, "grad_norm": 1.638970971107483, "loss_scale": 4.0, "retries": 0, "seconds": 5203.8, "max_memory_gb": 19.17}
|
| 421 |
+
{"step": 421, "steps": 500, "lr": 3.1575466957728514e-06, "loss": 0.013110695037426012, "grad_norm": 0.5052224397659302, "loss_scale": 4.0, "retries": 0, "seconds": 5216.1, "max_memory_gb": 19.17}
|
| 422 |
+
{"step": 422, "steps": 500, "lr": 3.129052087192952e-06, "loss": 0.07472769943844201, "grad_norm": 1.9395318031311035, "loss_scale": 4.0, "retries": 0, "seconds": 5228.3, "max_memory_gb": 19.17}
|
| 423 |
+
{"step": 423, "steps": 500, "lr": 3.1008890938946993e-06, "loss": 0.0001224458937578923, "grad_norm": 0.007496996317058802, "loss_scale": 4.0, "retries": 0, "seconds": 5240.5, "max_memory_gb": 19.17}
|
| 424 |
+
{"step": 424, "steps": 500, "lr": 3.0730589024288223e-06, "loss": 0.027882858940936922, "grad_norm": 1.021713376045227, "loss_scale": 4.0, "retries": 0, "seconds": 5252.8, "max_memory_gb": 19.17}
|
| 425 |
+
{"step": 425, "steps": 500, "lr": 3.0455626853245828e-06, "loss": 0.020161831339413024, "grad_norm": 4.874251842498779, "loss_scale": 4.0, "retries": 0, "seconds": 5264.5, "max_memory_gb": 19.17}
|
| 426 |
+
{"step": 426, "steps": 500, "lr": 3.0184016010403933e-06, "loss": 5.024057992838752e-05, "grad_norm": 0.0019905122462660074, "loss_scale": 4.0, "retries": 0, "seconds": 5276.8, "max_memory_gb": 19.17}
|
| 427 |
+
{"step": 427, "steps": 500, "lr": 2.9915767939149864e-06, "loss": 0.00040708044538978976, "grad_norm": 0.030293680727481842, "loss_scale": 4.0, "retries": 0, "seconds": 5289.0, "max_memory_gb": 19.17}
|
| 428 |
+
{"step": 428, "steps": 500, "lr": 2.9650893941192267e-06, "loss": 0.0003819404879498672, "grad_norm": 0.020305423066020012, "loss_scale": 4.0, "retries": 0, "seconds": 5301.5, "max_memory_gb": 19.17}
|
| 429 |
+
{"step": 429, "steps": 500, "lr": 2.9389405176084782e-06, "loss": 0.0005632992212915931, "grad_norm": 0.03285671025514603, "loss_scale": 4.0, "retries": 0, "seconds": 5313.6, "max_memory_gb": 19.17}
|
| 430 |
+
{"step": 430, "steps": 500, "lr": 2.9131312660755903e-06, "loss": 0.00014197088473689624, "grad_norm": 0.008543761447072029, "loss_scale": 4.0, "retries": 0, "seconds": 5326.0, "max_memory_gb": 19.17}
|
| 431 |
+
{"step": 431, "steps": 500, "lr": 2.887662726904487e-06, "loss": 0.0008305038242223295, "grad_norm": 0.035685665905475616, "loss_scale": 4.0, "retries": 0, "seconds": 5338.3, "max_memory_gb": 19.17}
|
| 432 |
+
{"step": 432, "steps": 500, "lr": 2.8625359731243483e-06, "loss": 0.00011874086695616951, "grad_norm": 0.010592748410999775, "loss_scale": 4.0, "retries": 0, "seconds": 5350.5, "max_memory_gb": 19.17}
|
| 433 |
+
{"step": 433, "steps": 500, "lr": 2.837752063364405e-06, "loss": 0.02331068938476154, "grad_norm": 1.7703243494033813, "loss_scale": 4.0, "retries": 0, "seconds": 5362.6, "max_memory_gb": 19.17}
|
| 434 |
+
{"step": 434, "steps": 500, "lr": 2.813312041809334e-06, "loss": 0.00033723808615026104, "grad_norm": 0.03322066739201546, "loss_scale": 4.0, "retries": 0, "seconds": 5374.8, "max_memory_gb": 19.17}
|
| 435 |
+
{"step": 435, "steps": 500, "lr": 2.7892169381552727e-06, "loss": 0.004038744473310177, "grad_norm": 0.18931661546230316, "loss_scale": 4.0, "retries": 0, "seconds": 5386.7, "max_memory_gb": 19.17}
|
| 436 |
+
{"step": 436, "steps": 500, "lr": 2.7654677675664216e-06, "loss": 4.07834257704177e-05, "grad_norm": 0.002919039921835065, "loss_scale": 4.0, "retries": 0, "seconds": 5398.9, "max_memory_gb": 19.17}
|
| 437 |
+
{"step": 437, "steps": 500, "lr": 2.742065530632291e-06, "loss": 4.841398680872028e-05, "grad_norm": 0.002003242028877139, "loss_scale": 4.0, "retries": 0, "seconds": 5411.2, "max_memory_gb": 19.17}
|
| 438 |
+
{"step": 438, "steps": 500, "lr": 2.719011213325533e-06, "loss": 4.031863784865147e-05, "grad_norm": 0.002063480205833912, "loss_scale": 4.0, "retries": 0, "seconds": 5423.4, "max_memory_gb": 19.17}
|
| 439 |
+
{"step": 439, "steps": 500, "lr": 2.6963057869604053e-06, "loss": 0.00977690380578622, "grad_norm": 0.9493473172187805, "loss_scale": 4.0, "retries": 0, "seconds": 5435.5, "max_memory_gb": 19.17}
|
| 440 |
+
{"step": 440, "steps": 500, "lr": 2.6739502081518463e-06, "loss": 0.0004953732946533634, "grad_norm": 0.04580886289477348, "loss_scale": 4.0, "retries": 0, "seconds": 5447.7, "max_memory_gb": 19.17}
|
| 441 |
+
{"step": 441, "steps": 500, "lr": 2.651945418775174e-06, "loss": 0.0001492563727135183, "grad_norm": 0.0070538390427827835, "loss_scale": 4.0, "retries": 0, "seconds": 5459.8, "max_memory_gb": 19.17}
|
| 442 |
+
{"step": 442, "steps": 500, "lr": 2.6302923459264005e-06, "loss": 0.051891566709477525, "grad_norm": 1.9641982316970825, "loss_scale": 4.0, "retries": 0, "seconds": 5472.1, "max_memory_gb": 19.17}
|
| 443 |
+
{"step": 443, "steps": 500, "lr": 2.608991901883174e-06, "loss": 0.0006541647940836852, "grad_norm": 0.049304015934467316, "loss_scale": 4.0, "retries": 0, "seconds": 5484.4, "max_memory_gb": 19.17}
|
| 444 |
+
{"step": 444, "steps": 500, "lr": 2.588044984066341e-06, "loss": 0.009107369663154105, "grad_norm": 1.2912565469741821, "loss_scale": 4.0, "retries": 0, "seconds": 5496.6, "max_memory_gb": 19.17}
|
| 445 |
+
{"step": 445, "steps": 500, "lr": 2.5674524750021456e-06, "loss": 0.08997119134555347, "grad_norm": 1.3709532022476196, "loss_scale": 4.0, "retries": 0, "seconds": 5508.9, "max_memory_gb": 19.17}
|
| 446 |
+
{"step": 446, "steps": 500, "lr": 2.5472152422850257e-06, "loss": 0.09769768167701383, "grad_norm": 4.2948431968688965, "loss_scale": 4.0, "retries": 0, "seconds": 5521.0, "max_memory_gb": 19.17}
|
| 447 |
+
{"step": 447, "steps": 500, "lr": 2.5273341385410883e-06, "loss": 0.0004086265294424927, "grad_norm": 0.024789856746792793, "loss_scale": 4.0, "retries": 0, "seconds": 5533.5, "max_memory_gb": 19.17}
|
| 448 |
+
{"step": 448, "steps": 500, "lr": 2.507810001392163e-06, "loss": 0.0003209108037819678, "grad_norm": 0.017854448407888412, "loss_scale": 4.0, "retries": 0, "seconds": 5545.7, "max_memory_gb": 19.17}
|
| 449 |
+
{"step": 449, "steps": 500, "lr": 2.488643653420525e-06, "loss": 0.00035650014295253066, "grad_norm": 0.015480437316000462, "loss_scale": 4.0, "retries": 0, "seconds": 5558.0, "max_memory_gb": 19.17}
|
| 450 |
+
{"step": 450, "steps": 500, "lr": 2.4698359021342357e-06, "loss": 0.0176392603202496, "grad_norm": 1.3546510934829712, "loss_scale": 4.0, "retries": 0, "seconds": 5570.3, "max_memory_gb": 19.17}
|
| 451 |
+
{"step": 451, "steps": 500, "lr": 2.451387539933117e-06, "loss": 0.006693495756559804, "grad_norm": 0.25157630443573, "loss_scale": 4.0, "retries": 0, "seconds": 5582.6, "max_memory_gb": 19.17}
|
| 452 |
+
{"step": 452, "steps": 500, "lr": 2.4332993440753743e-06, "loss": 0.00017472784959757996, "grad_norm": 0.010994181968271732, "loss_scale": 4.0, "retries": 0, "seconds": 5594.9, "max_memory_gb": 19.17}
|
| 453 |
+
{"step": 453, "steps": 500, "lr": 2.415572076644838e-06, "loss": 0.0006925813913253814, "grad_norm": 0.06871061772108078, "loss_scale": 4.0, "retries": 0, "seconds": 5607.2, "max_memory_gb": 19.17}
|
| 454 |
+
{"step": 454, "steps": 500, "lr": 2.3982064845188708e-06, "loss": 0.00044230015681085376, "grad_norm": 0.02553018182516098, "loss_scale": 4.0, "retries": 0, "seconds": 5619.5, "max_memory_gb": 19.17}
|
| 455 |
+
{"step": 455, "steps": 500, "lr": 2.3812032993368828e-06, "loss": 4.628639264839504e-05, "grad_norm": 0.002344967797398567, "loss_scale": 4.0, "retries": 0, "seconds": 5631.6, "max_memory_gb": 19.17}
|
| 456 |
+
{"step": 456, "steps": 500, "lr": 2.3645632374695245e-06, "loss": 0.00028452439494489306, "grad_norm": 0.018305206671357155, "loss_scale": 4.0, "retries": 0, "seconds": 5643.7, "max_memory_gb": 19.17}
|
| 457 |
+
{"step": 457, "steps": 500, "lr": 2.3482869999884874e-06, "loss": 0.00016722808815798504, "grad_norm": 0.006496606860309839, "loss_scale": 4.0, "retries": 0, "seconds": 5655.9, "max_memory_gb": 19.17}
|
| 458 |
+
{"step": 458, "steps": 500, "lr": 2.332375272636983e-06, "loss": 0.10839487322105158, "grad_norm": 2.0768303871154785, "loss_scale": 4.0, "retries": 0, "seconds": 5668.1, "max_memory_gb": 19.17}
|
| 459 |
+
{"step": 459, "steps": 500, "lr": 2.316828725800841e-06, "loss": 0.04836432344738999, "grad_norm": 1.419457197189331, "loss_scale": 4.0, "retries": 0, "seconds": 5680.2, "max_memory_gb": 19.17}
|
| 460 |
+
{"step": 460, "steps": 500, "lr": 2.301648014480267e-06, "loss": 0.00019445398910859652, "grad_norm": 0.017869578674435616, "loss_scale": 4.0, "retries": 0, "seconds": 5692.2, "max_memory_gb": 19.17}
|
| 461 |
+
{"step": 461, "steps": 500, "lr": 2.2868337782622484e-06, "loss": 0.00013382617802645572, "grad_norm": 0.006245922762900591, "loss_scale": 4.0, "retries": 0, "seconds": 5704.5, "max_memory_gb": 19.17}
|
| 462 |
+
{"step": 462, "steps": 500, "lr": 2.272386641293606e-06, "loss": 9.75851546614463e-05, "grad_norm": 0.005519750062376261, "loss_scale": 4.0, "retries": 0, "seconds": 5716.8, "max_memory_gb": 19.17}
|
| 463 |
+
{"step": 463, "steps": 500, "lr": 2.258307212254698e-06, "loss": 0.00010321310457845297, "grad_norm": 0.005393592640757561, "loss_scale": 4.0, "retries": 0, "seconds": 5729.3, "max_memory_gb": 19.17}
|
| 464 |
+
{"step": 464, "steps": 500, "lr": 2.2445960843337744e-06, "loss": 0.003220317624757224, "grad_norm": 0.348457932472229, "loss_scale": 4.0, "retries": 0, "seconds": 5741.7, "max_memory_gb": 19.17}
|
| 465 |
+
{"step": 465, "steps": 500, "lr": 2.231253835201989e-06, "loss": 0.0015666195537788626, "grad_norm": 0.09082016348838806, "loss_scale": 4.0, "retries": 0, "seconds": 5754.0, "max_memory_gb": 19.17}
|
| 466 |
+
{"step": 466, "steps": 500, "lr": 2.218281026989053e-06, "loss": 0.19213598361462594, "grad_norm": 6.493363857269287, "loss_scale": 4.0, "retries": 0, "seconds": 5766.3, "max_memory_gb": 19.17}
|
| 467 |
+
{"step": 467, "steps": 500, "lr": 2.2056782062595655e-06, "loss": 3.502422411439842e-05, "grad_norm": 0.0018857986433431506, "loss_scale": 4.0, "retries": 0, "seconds": 5778.7, "max_memory_gb": 19.17}
|
| 468 |
+
{"step": 468, "steps": 500, "lr": 2.193445903989966e-06, "loss": 0.26040905250123497, "grad_norm": 8.987494468688965, "loss_scale": 4.0, "retries": 0, "seconds": 5791.1, "max_memory_gb": 19.17}
|
| 469 |
+
{"step": 469, "steps": 500, "lr": 2.1815846355461853e-06, "loss": 0.03401932699387607, "grad_norm": 1.1270427703857422, "loss_scale": 4.0, "retries": 0, "seconds": 5803.4, "max_memory_gb": 19.17}
|
| 470 |
+
{"step": 470, "steps": 500, "lr": 2.1700949006619163e-06, "loss": 2.8335702666672873e-05, "grad_norm": 0.001105878851376474, "loss_scale": 4.0, "retries": 0, "seconds": 5815.7, "max_memory_gb": 19.17}
|
| 471 |
+
{"step": 471, "steps": 500, "lr": 2.158977183417566e-06, "loss": 0.2990545488408145, "grad_norm": 9.064373970031738, "loss_scale": 4.0, "retries": 0, "seconds": 5828.1, "max_memory_gb": 19.17}
|
| 472 |
+
{"step": 472, "steps": 500, "lr": 2.1482319522198587e-06, "loss": 0.00012216664562846415, "grad_norm": 0.007071193307638168, "loss_scale": 4.0, "retries": 0, "seconds": 5840.7, "max_memory_gb": 19.17}
|
| 473 |
+
{"step": 473, "steps": 500, "lr": 2.137859659782103e-06, "loss": 0.33857974968243937, "grad_norm": 7.563459396362305, "loss_scale": 4.0, "retries": 0, "seconds": 5853.0, "max_memory_gb": 19.17}
|
| 474 |
+
{"step": 474, "steps": 500, "lr": 2.1278607431051185e-06, "loss": 0.0006770835277569114, "grad_norm": 0.10429906100034714, "loss_scale": 4.0, "retries": 0, "seconds": 5865.4, "max_memory_gb": 19.17}
|
| 475 |
+
{"step": 475, "steps": 500, "lr": 2.1182356234588196e-06, "loss": 0.00010850640880603765, "grad_norm": 0.0067459894344210625, "loss_scale": 4.0, "retries": 0, "seconds": 5877.9, "max_memory_gb": 19.17}
|
| 476 |
+
{"step": 476, "steps": 500, "lr": 2.1089847063644735e-06, "loss": 0.006919217177361568, "grad_norm": 0.49672865867614746, "loss_scale": 4.0, "retries": 0, "seconds": 5890.3, "max_memory_gb": 19.17}
|
| 477 |
+
{"step": 477, "steps": 500, "lr": 2.100108381577611e-06, "loss": 0.0338329010017188, "grad_norm": 2.0788114070892334, "loss_scale": 4.0, "retries": 0, "seconds": 5902.9, "max_memory_gb": 19.17}
|
| 478 |
+
{"step": 478, "steps": 500, "lr": 2.091607023071606e-06, "loss": 4.43022660011717e-05, "grad_norm": 0.002246302552521229, "loss_scale": 4.0, "retries": 0, "seconds": 5915.3, "max_memory_gb": 19.17}
|
| 479 |
+
{"step": 479, "steps": 500, "lr": 2.0834809890219166e-06, "loss": 0.006158807909805185, "grad_norm": 0.31262844800949097, "loss_scale": 4.0, "retries": 0, "seconds": 5927.6, "max_memory_gb": 19.17}
|
| 480 |
+
{"step": 480, "steps": 500, "lr": 2.0757306217910033e-06, "loss": 0.0002770505386499942, "grad_norm": 0.014388740062713623, "loss_scale": 4.0, "retries": 0, "seconds": 5940.0, "max_memory_gb": 19.17}
|
| 481 |
+
{"step": 481, "steps": 500, "lr": 2.068356247913894e-06, "loss": 0.00044134340926857174, "grad_norm": 0.02521849423646927, "loss_scale": 4.0, "retries": 0, "seconds": 5952.5, "max_memory_gb": 19.17}
|
| 482 |
+
{"step": 482, "steps": 500, "lr": 2.0613581780844348e-06, "loss": 8.499210310023129e-05, "grad_norm": 0.004430806264281273, "loss_scale": 4.0, "retries": 0, "seconds": 5964.9, "max_memory_gb": 19.17}
|
| 483 |
+
{"step": 483, "steps": 500, "lr": 2.0547367071421955e-06, "loss": 0.00022545559828140492, "grad_norm": 0.015385109931230545, "loss_scale": 4.0, "retries": 0, "seconds": 5977.2, "max_memory_gb": 19.17}
|
| 484 |
+
{"step": 484, "steps": 500, "lr": 2.048492114060048e-06, "loss": 0.015566607814646846, "grad_norm": 1.1825180053710938, "loss_scale": 4.0, "retries": 0, "seconds": 5989.5, "max_memory_gb": 19.17}
|
| 485 |
+
{"step": 485, "steps": 500, "lr": 2.0426246619324153e-06, "loss": 0.0006939320586551194, "grad_norm": 0.0438900925219059, "loss_scale": 4.0, "retries": 0, "seconds": 6001.8, "max_memory_gb": 19.17}
|
| 486 |
+
{"step": 486, "steps": 500, "lr": 2.0371345979641848e-06, "loss": 0.009279735050452844, "grad_norm": 0.27678123116493225, "loss_scale": 4.0, "retries": 0, "seconds": 6014.1, "max_memory_gb": 19.17}
|
| 487 |
+
{"step": 487, "steps": 500, "lr": 2.032022153460292e-06, "loss": 0.054767744515831156, "grad_norm": 1.3154027462005615, "loss_scale": 4.0, "retries": 0, "seconds": 6026.6, "max_memory_gb": 19.17}
|
| 488 |
+
{"step": 488, "steps": 500, "lr": 2.0272875438159797e-06, "loss": 0.03720996182861892, "grad_norm": 6.017664432525635, "loss_scale": 4.0, "retries": 0, "seconds": 6039.3, "max_memory_gb": 19.17}
|
| 489 |
+
{"step": 489, "steps": 500, "lr": 2.022930968507718e-06, "loss": 0.0187284242288735, "grad_norm": 1.1124653816223145, "loss_scale": 4.0, "retries": 0, "seconds": 6051.6, "max_memory_gb": 19.17}
|
| 490 |
+
{"step": 490, "steps": 500, "lr": 2.0189526110848026e-06, "loss": 0.23881834856746828, "grad_norm": 5.071252346038818, "loss_scale": 4.0, "retries": 0, "seconds": 6063.9, "max_memory_gb": 19.17}
|
| 491 |
+
{"step": 491, "steps": 500, "lr": 2.0153526391616207e-06, "loss": 8.083220465819481e-05, "grad_norm": 0.004373323172330856, "loss_scale": 4.0, "retries": 0, "seconds": 6076.3, "max_memory_gb": 19.17}
|
| 492 |
+
{"step": 492, "steps": 500, "lr": 2.0121312044105927e-06, "loss": 3.1128111121926274e-05, "grad_norm": 0.0012161321938037872, "loss_scale": 4.0, "retries": 0, "seconds": 6088.5, "max_memory_gb": 19.17}
|
| 493 |
+
{"step": 493, "steps": 500, "lr": 2.009288442555774e-06, "loss": 0.0030669401408700155, "grad_norm": 0.15313969552516937, "loss_scale": 4.0, "retries": 0, "seconds": 6100.8, "max_memory_gb": 19.17}
|
| 494 |
+
{"step": 494, "steps": 500, "lr": 2.0068244733671474e-06, "loss": 0.07815304925418332, "grad_norm": 1.821289300918579, "loss_scale": 4.0, "retries": 0, "seconds": 6113.3, "max_memory_gb": 19.17}
|
| 495 |
+
{"step": 495, "steps": 500, "lr": 2.004739400655566e-06, "loss": 0.0008578298300340686, "grad_norm": 0.04263243451714516, "loss_scale": 4.0, "retries": 0, "seconds": 6125.7, "max_memory_gb": 19.17}
|
| 496 |
+
{"step": 496, "steps": 500, "lr": 2.003033312268388e-06, "loss": 6.82631249989382e-05, "grad_norm": 0.003672684310004115, "loss_scale": 4.0, "retries": 0, "seconds": 6138.1, "max_memory_gb": 19.17}
|
| 497 |
+
{"step": 497, "steps": 500, "lr": 2.0017062800857716e-06, "loss": 0.018843024163494062, "grad_norm": 2.7229461669921875, "loss_scale": 4.0, "retries": 0, "seconds": 6150.0, "max_memory_gb": 19.17}
|
| 498 |
+
{"step": 498, "steps": 500, "lr": 2.000758360017648e-06, "loss": 0.025737829324353614, "grad_norm": 1.5621174573898315, "loss_scale": 4.0, "retries": 0, "seconds": 6162.3, "max_memory_gb": 19.17}
|
| 499 |
+
{"step": 499, "steps": 500, "lr": 2.000189592001363e-06, "loss": 0.0001492113796876476, "grad_norm": 0.009199470281600952, "loss_scale": 4.0, "retries": 0, "seconds": 6175.0, "max_memory_gb": 19.17}
|
| 500 |
+
{"step": 500, "steps": 500, "lr": 2e-06, "loss": 4.3416689900962793e-05, "grad_norm": 0.0016163547988981009, "loss_scale": 4.0, "retries": 0, "seconds": 6187.2, "max_memory_gb": 19.17}
|
runs/joint_qwen/joint/seed73/config.json
ADDED
|
@@ -0,0 +1,32 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"seed": 73,
|
| 3 |
+
"model": "/data/OPSD/assets/models/Qwen3.5-9B",
|
| 4 |
+
"backbone": "qwen3_5",
|
| 5 |
+
"train": "/data/OPSD/work/joint/joint.jsonl",
|
| 6 |
+
"records": 1000,
|
| 7 |
+
"steps": 500,
|
| 8 |
+
"steps_run": 500,
|
| 9 |
+
"category_weights": {
|
| 10 |
+
"original": 1.0,
|
| 11 |
+
"removal": 1.0,
|
| 12 |
+
"replacement": 1.0,
|
| 13 |
+
"context": 0.5,
|
| 14 |
+
"no_image": 0.02
|
| 15 |
+
},
|
| 16 |
+
"task_weight": 0.5,
|
| 17 |
+
"phases": [
|
| 18 |
+
{
|
| 19 |
+
"color": "b",
|
| 20 |
+
"count": "b"
|
| 21 |
+
}
|
| 22 |
+
],
|
| 23 |
+
"lora_rank": 16,
|
| 24 |
+
"lr": {
|
| 25 |
+
"peak": 2e-05,
|
| 26 |
+
"warmup": 16,
|
| 27 |
+
"final": 2e-06,
|
| 28 |
+
"decay": "cosine"
|
| 29 |
+
},
|
| 30 |
+
"clip": 1.0,
|
| 31 |
+
"initial_loss_scale": 4.0
|
| 32 |
+
}
|
runs/joint_qwen/joint/seed73/steps.jsonl
ADDED
|
@@ -0,0 +1,500 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{"step": 1, "steps": 500, "lr": 1.25e-06, "loss": 3.937962674281814, "grad_norm": 45.09823226928711, "loss_scale": 4.0, "retries": 0, "seconds": 16.4, "max_memory_gb": 18.84}
|
| 2 |
+
{"step": 2, "steps": 500, "lr": 2.5e-06, "loss": 3.902453004636548, "grad_norm": 41.589500427246094, "loss_scale": 4.0, "retries": 0, "seconds": 28.9, "max_memory_gb": 18.92}
|
| 3 |
+
{"step": 3, "steps": 500, "lr": 3.7500000000000005e-06, "loss": 3.721885292367502, "grad_norm": 44.06940841674805, "loss_scale": 4.0, "retries": 0, "seconds": 41.1, "max_memory_gb": 18.96}
|
| 4 |
+
{"step": 4, "steps": 500, "lr": 5e-06, "loss": 3.548557719723745, "grad_norm": 48.46406555175781, "loss_scale": 4.0, "retries": 0, "seconds": 53.5, "max_memory_gb": 18.96}
|
| 5 |
+
{"step": 5, "steps": 500, "lr": 6.25e-06, "loss": 2.894924033094536, "grad_norm": 40.01211929321289, "loss_scale": 4.0, "retries": 0, "seconds": 65.8, "max_memory_gb": 18.96}
|
| 6 |
+
{"step": 6, "steps": 500, "lr": 7.500000000000001e-06, "loss": 2.811823938719251, "grad_norm": 40.73472595214844, "loss_scale": 4.0, "retries": 0, "seconds": 78.2, "max_memory_gb": 19.16}
|
| 7 |
+
{"step": 7, "steps": 500, "lr": 8.750000000000001e-06, "loss": 2.3958460378714586, "grad_norm": 43.218406677246094, "loss_scale": 4.0, "retries": 0, "seconds": 90.4, "max_memory_gb": 19.16}
|
| 8 |
+
{"step": 8, "steps": 500, "lr": 1e-05, "loss": 1.8557859300212427, "grad_norm": 28.35599136352539, "loss_scale": 4.0, "retries": 0, "seconds": 102.4, "max_memory_gb": 19.16}
|
| 9 |
+
{"step": 9, "steps": 500, "lr": 1.125e-05, "loss": 1.4133896784518254, "grad_norm": 22.752063751220703, "loss_scale": 4.0, "retries": 0, "seconds": 114.7, "max_memory_gb": 19.16}
|
| 10 |
+
{"step": 10, "steps": 500, "lr": 1.25e-05, "loss": 1.6798100935041227, "grad_norm": 25.284154891967773, "loss_scale": 4.0, "retries": 0, "seconds": 127.0, "max_memory_gb": 19.16}
|
| 11 |
+
{"step": 11, "steps": 500, "lr": 1.375e-05, "loss": 0.705853993432935, "grad_norm": 14.1546630859375, "loss_scale": 4.0, "retries": 0, "seconds": 139.3, "max_memory_gb": 19.16}
|
| 12 |
+
{"step": 12, "steps": 500, "lr": 1.5000000000000002e-05, "loss": 0.8194626309400933, "grad_norm": 11.560633659362793, "loss_scale": 4.0, "retries": 0, "seconds": 151.6, "max_memory_gb": 19.16}
|
| 13 |
+
{"step": 13, "steps": 500, "lr": 1.6250000000000002e-05, "loss": 0.3831364937041971, "grad_norm": 4.333874225616455, "loss_scale": 4.0, "retries": 0, "seconds": 163.9, "max_memory_gb": 19.16}
|
| 14 |
+
{"step": 14, "steps": 500, "lr": 1.7500000000000002e-05, "loss": 0.3348661298196848, "grad_norm": 2.2110540866851807, "loss_scale": 4.0, "retries": 0, "seconds": 176.2, "max_memory_gb": 19.16}
|
| 15 |
+
{"step": 15, "steps": 500, "lr": 1.8750000000000002e-05, "loss": 0.40515751045578247, "grad_norm": 2.33944034576416, "loss_scale": 4.0, "retries": 0, "seconds": 188.1, "max_memory_gb": 19.16}
|
| 16 |
+
{"step": 16, "steps": 500, "lr": 2e-05, "loss": 0.3429822918895463, "grad_norm": 1.7948001623153687, "loss_scale": 4.0, "retries": 0, "seconds": 200.3, "max_memory_gb": 19.16}
|
| 17 |
+
{"step": 17, "steps": 500, "lr": 1.999981040799864e-05, "loss": 0.357461025250483, "grad_norm": 2.1488749980926514, "loss_scale": 4.0, "retries": 0, "seconds": 212.6, "max_memory_gb": 19.16}
|
| 18 |
+
{"step": 18, "steps": 500, "lr": 1.9999241639982356e-05, "loss": 0.22826169353687545, "grad_norm": 1.4782428741455078, "loss_scale": 4.0, "retries": 0, "seconds": 224.8, "max_memory_gb": 19.16}
|
| 19 |
+
{"step": 19, "steps": 500, "lr": 1.9998293719914233e-05, "loss": 0.30677780884699046, "grad_norm": 1.9481818675994873, "loss_scale": 4.0, "retries": 0, "seconds": 237.0, "max_memory_gb": 19.16}
|
| 20 |
+
{"step": 20, "steps": 500, "lr": 1.9996966687731615e-05, "loss": 0.24221952601113694, "grad_norm": 2.0277817249298096, "loss_scale": 4.0, "retries": 0, "seconds": 249.6, "max_memory_gb": 19.16}
|
| 21 |
+
{"step": 21, "steps": 500, "lr": 1.9995260599344435e-05, "loss": 0.2378379394861192, "grad_norm": 2.613340377807617, "loss_scale": 4.0, "retries": 0, "seconds": 261.8, "max_memory_gb": 19.16}
|
| 22 |
+
{"step": 22, "steps": 500, "lr": 1.9993175526632854e-05, "loss": 0.2944336176839319, "grad_norm": 3.9608428478240967, "loss_scale": 4.0, "retries": 0, "seconds": 274.1, "max_memory_gb": 19.16}
|
| 23 |
+
{"step": 23, "steps": 500, "lr": 1.9990711557444226e-05, "loss": 0.30202148899182, "grad_norm": 2.8532330989837646, "loss_scale": 4.0, "retries": 0, "seconds": 286.2, "max_memory_gb": 19.16}
|
| 24 |
+
{"step": 24, "steps": 500, "lr": 1.998786879558941e-05, "loss": 0.1552490545092786, "grad_norm": 3.5183849334716797, "loss_scale": 4.0, "retries": 0, "seconds": 298.6, "max_memory_gb": 19.16}
|
| 25 |
+
{"step": 25, "steps": 500, "lr": 1.998464736083838e-05, "loss": 0.12924687246347524, "grad_norm": 2.458832263946533, "loss_scale": 4.0, "retries": 0, "seconds": 310.9, "max_memory_gb": 19.16}
|
| 26 |
+
{"step": 26, "steps": 500, "lr": 1.99810473889152e-05, "loss": 0.21447264663980936, "grad_norm": 2.638190507888794, "loss_scale": 4.0, "retries": 0, "seconds": 323.1, "max_memory_gb": 19.16}
|
| 27 |
+
{"step": 27, "steps": 500, "lr": 1.9977069031492283e-05, "loss": 0.09663257765417009, "grad_norm": 1.513710618019104, "loss_scale": 4.0, "retries": 0, "seconds": 335.7, "max_memory_gb": 19.16}
|
| 28 |
+
{"step": 28, "steps": 500, "lr": 1.9972712456184023e-05, "loss": 0.12535415272974537, "grad_norm": 1.9420799016952515, "loss_scale": 4.0, "retries": 0, "seconds": 348.0, "max_memory_gb": 19.16}
|
| 29 |
+
{"step": 29, "steps": 500, "lr": 1.996797784653971e-05, "loss": 0.07887037931438748, "grad_norm": 1.1391682624816895, "loss_scale": 4.0, "retries": 0, "seconds": 360.6, "max_memory_gb": 19.16}
|
| 30 |
+
{"step": 30, "steps": 500, "lr": 1.9962865402035815e-05, "loss": 0.11482686354694041, "grad_norm": 3.771007776260376, "loss_scale": 4.0, "retries": 0, "seconds": 373.0, "max_memory_gb": 19.16}
|
| 31 |
+
{"step": 31, "steps": 500, "lr": 1.9957375338067585e-05, "loss": 0.03644144385543769, "grad_norm": 1.1859936714172363, "loss_scale": 4.0, "retries": 0, "seconds": 385.5, "max_memory_gb": 19.16}
|
| 32 |
+
{"step": 32, "steps": 500, "lr": 1.995150788593995e-05, "loss": 0.12561282219510098, "grad_norm": 2.5792927742004395, "loss_scale": 4.0, "retries": 0, "seconds": 398.0, "max_memory_gb": 19.16}
|
| 33 |
+
{"step": 33, "steps": 500, "lr": 1.9945263292857805e-05, "loss": 0.03637781073402122, "grad_norm": 1.874048113822937, "loss_scale": 4.0, "retries": 0, "seconds": 410.6, "max_memory_gb": 19.16}
|
| 34 |
+
{"step": 34, "steps": 500, "lr": 1.9938641821915566e-05, "loss": 0.023035750653269297, "grad_norm": 2.9694607257843018, "loss_scale": 4.0, "retries": 0, "seconds": 423.1, "max_memory_gb": 19.16}
|
| 35 |
+
{"step": 35, "steps": 500, "lr": 1.993164375208611e-05, "loss": 0.08356096329258766, "grad_norm": 5.016751289367676, "loss_scale": 4.0, "retries": 0, "seconds": 435.4, "max_memory_gb": 19.16}
|
| 36 |
+
{"step": 36, "steps": 500, "lr": 1.9924269378209e-05, "loss": 0.19963425054662698, "grad_norm": 2.8180973529815674, "loss_scale": 4.0, "retries": 0, "seconds": 447.8, "max_memory_gb": 19.16}
|
| 37 |
+
{"step": 37, "steps": 500, "lr": 1.9916519010978086e-05, "loss": 0.10769610243303099, "grad_norm": 2.3814663887023926, "loss_scale": 4.0, "retries": 0, "seconds": 460.2, "max_memory_gb": 19.16}
|
| 38 |
+
{"step": 38, "steps": 500, "lr": 1.9908392976928396e-05, "loss": 0.026370955225750302, "grad_norm": 2.5264670848846436, "loss_scale": 4.0, "retries": 0, "seconds": 472.7, "max_memory_gb": 19.16}
|
| 39 |
+
{"step": 39, "steps": 500, "lr": 1.9899891618422388e-05, "loss": 0.01769496669426603, "grad_norm": 1.01766037940979, "loss_scale": 4.0, "retries": 0, "seconds": 485.1, "max_memory_gb": 19.16}
|
| 40 |
+
{"step": 40, "steps": 500, "lr": 1.989101529363553e-05, "loss": 0.009222990607337457, "grad_norm": 0.3809492886066437, "loss_scale": 4.0, "retries": 0, "seconds": 497.5, "max_memory_gb": 19.16}
|
| 41 |
+
{"step": 41, "steps": 500, "lr": 1.9881764376541183e-05, "loss": 0.011445654274129167, "grad_norm": 0.7161529064178467, "loss_scale": 4.0, "retries": 0, "seconds": 509.9, "max_memory_gb": 19.16}
|
| 42 |
+
{"step": 42, "steps": 500, "lr": 1.9872139256894883e-05, "loss": 0.04517790666084859, "grad_norm": 2.089848518371582, "loss_scale": 4.0, "retries": 0, "seconds": 522.4, "max_memory_gb": 19.16}
|
| 43 |
+
{"step": 43, "steps": 500, "lr": 1.98621403402179e-05, "loss": 0.5271883074428843, "grad_norm": 16.08713722229004, "loss_scale": 4.0, "retries": 0, "seconds": 534.6, "max_memory_gb": 19.16}
|
| 44 |
+
{"step": 44, "steps": 500, "lr": 1.9851768047780144e-05, "loss": 0.006184657034943732, "grad_norm": 0.22708863019943237, "loss_scale": 4.0, "retries": 0, "seconds": 547.3, "max_memory_gb": 19.16}
|
| 45 |
+
{"step": 45, "steps": 500, "lr": 1.9841022816582437e-05, "loss": 0.008162071828213482, "grad_norm": 0.32443636655807495, "loss_scale": 4.0, "retries": 0, "seconds": 559.8, "max_memory_gb": 19.16}
|
| 46 |
+
{"step": 46, "steps": 500, "lr": 1.9829905099338088e-05, "loss": 0.004587129979140317, "grad_norm": 0.3080138564109802, "loss_scale": 4.0, "retries": 0, "seconds": 572.2, "max_memory_gb": 19.16}
|
| 47 |
+
{"step": 47, "steps": 500, "lr": 1.9818415364453816e-05, "loss": 0.07049361933048784, "grad_norm": 2.5799825191497803, "loss_scale": 4.0, "retries": 0, "seconds": 584.7, "max_memory_gb": 19.16}
|
| 48 |
+
{"step": 48, "steps": 500, "lr": 1.9806554096010038e-05, "loss": 0.12064811717771753, "grad_norm": 3.7254672050476074, "loss_scale": 4.0, "retries": 0, "seconds": 597.6, "max_memory_gb": 19.16}
|
| 49 |
+
{"step": 49, "steps": 500, "lr": 1.9794321793740438e-05, "loss": 0.0026239604988892943, "grad_norm": 0.09228042513132095, "loss_scale": 4.0, "retries": 0, "seconds": 609.7, "max_memory_gb": 19.16}
|
| 50 |
+
{"step": 50, "steps": 500, "lr": 1.9781718973010948e-05, "loss": 0.006076745723134991, "grad_norm": 0.6876630783081055, "loss_scale": 4.0, "retries": 0, "seconds": 622.6, "max_memory_gb": 19.16}
|
| 51 |
+
{"step": 51, "steps": 500, "lr": 1.9768746164798014e-05, "loss": 0.18126147886168073, "grad_norm": 3.284600019454956, "loss_scale": 4.0, "retries": 0, "seconds": 635.1, "max_memory_gb": 19.16}
|
| 52 |
+
{"step": 52, "steps": 500, "lr": 1.9755403915666227e-05, "loss": 0.0026798111696486803, "grad_norm": 0.1505034863948822, "loss_scale": 4.0, "retries": 0, "seconds": 647.3, "max_memory_gb": 19.16}
|
| 53 |
+
{"step": 53, "steps": 500, "lr": 1.9741692787745306e-05, "loss": 0.5388832367672298, "grad_norm": 42.68181228637695, "loss_scale": 4.0, "retries": 0, "seconds": 659.5, "max_memory_gb": 19.16}
|
| 54 |
+
{"step": 54, "steps": 500, "lr": 1.9727613358706398e-05, "loss": 0.0009175188518859366, "grad_norm": 0.08270559459924698, "loss_scale": 4.0, "retries": 0, "seconds": 672.2, "max_memory_gb": 19.16}
|
| 55 |
+
{"step": 55, "steps": 500, "lr": 1.9713166221737755e-05, "loss": 0.15463116096241628, "grad_norm": 6.862787246704102, "loss_scale": 4.0, "retries": 0, "seconds": 684.8, "max_memory_gb": 19.16}
|
| 56 |
+
{"step": 56, "steps": 500, "lr": 1.9698351985519734e-05, "loss": 0.0013927278900141005, "grad_norm": 0.26508861780166626, "loss_scale": 4.0, "retries": 0, "seconds": 697.6, "max_memory_gb": 19.16}
|
| 57 |
+
{"step": 57, "steps": 500, "lr": 1.968317127419916e-05, "loss": 0.0025437151538405733, "grad_norm": 0.15900775790214539, "loss_scale": 4.0, "retries": 0, "seconds": 709.9, "max_memory_gb": 19.16}
|
| 58 |
+
{"step": 58, "steps": 500, "lr": 1.9667624727363018e-05, "loss": 0.024316393188650472, "grad_norm": 3.066981315612793, "loss_scale": 4.0, "retries": 0, "seconds": 722.5, "max_memory_gb": 19.16}
|
| 59 |
+
{"step": 59, "steps": 500, "lr": 1.9651713000011512e-05, "loss": 0.04762154037595161, "grad_norm": 2.7689239978790283, "loss_scale": 4.0, "retries": 0, "seconds": 734.8, "max_memory_gb": 19.16}
|
| 60 |
+
{"step": 60, "steps": 500, "lr": 1.9635436762530477e-05, "loss": 0.0003224600139204045, "grad_norm": 0.03115033730864525, "loss_scale": 4.0, "retries": 0, "seconds": 747.0, "max_memory_gb": 19.16}
|
| 61 |
+
{"step": 61, "steps": 500, "lr": 1.9618796700663118e-05, "loss": 0.0688616782874321, "grad_norm": 3.630000114440918, "loss_scale": 4.0, "retries": 0, "seconds": 759.1, "max_memory_gb": 19.16}
|
| 62 |
+
{"step": 62, "steps": 500, "lr": 1.960179351548113e-05, "loss": 0.135928446566099, "grad_norm": 3.2474045753479004, "loss_scale": 4.0, "retries": 0, "seconds": 771.3, "max_memory_gb": 19.16}
|
| 63 |
+
{"step": 63, "steps": 500, "lr": 1.958442792335516e-05, "loss": 0.0031236960271879936, "grad_norm": 0.18157683312892914, "loss_scale": 4.0, "retries": 0, "seconds": 783.4, "max_memory_gb": 19.16}
|
| 64 |
+
{"step": 64, "steps": 500, "lr": 1.956670065592463e-05, "loss": 0.00019107353381771344, "grad_norm": 0.011817575432360172, "loss_scale": 4.0, "retries": 0, "seconds": 795.6, "max_memory_gb": 19.16}
|
| 65 |
+
{"step": 65, "steps": 500, "lr": 1.9548612460066884e-05, "loss": 0.0016046755100169528, "grad_norm": 0.17891326546669006, "loss_scale": 4.0, "retries": 0, "seconds": 807.8, "max_memory_gb": 19.16}
|
| 66 |
+
{"step": 66, "steps": 500, "lr": 1.9530164097865766e-05, "loss": 0.013380342309569025, "grad_norm": 1.1068801879882812, "loss_scale": 4.0, "retries": 0, "seconds": 820.0, "max_memory_gb": 19.16}
|
| 67 |
+
{"step": 67, "steps": 500, "lr": 1.9511356346579475e-05, "loss": 0.0003874506021617208, "grad_norm": 0.029773686081171036, "loss_scale": 4.0, "retries": 0, "seconds": 832.4, "max_memory_gb": 19.16}
|
| 68 |
+
{"step": 68, "steps": 500, "lr": 1.949218999860784e-05, "loss": 0.00012103609716456793, "grad_norm": 0.005857863463461399, "loss_scale": 4.0, "retries": 0, "seconds": 845.1, "max_memory_gb": 19.16}
|
| 69 |
+
{"step": 69, "steps": 500, "lr": 1.9472665861458913e-05, "loss": 0.08487564647859293, "grad_norm": 4.234208583831787, "loss_scale": 4.0, "retries": 0, "seconds": 857.6, "max_memory_gb": 19.16}
|
| 70 |
+
{"step": 70, "steps": 500, "lr": 1.9452784757714977e-05, "loss": 0.0039300093629270295, "grad_norm": 0.41977763175964355, "loss_scale": 4.0, "retries": 0, "seconds": 870.1, "max_memory_gb": 19.16}
|
| 71 |
+
{"step": 71, "steps": 500, "lr": 1.9432547524997857e-05, "loss": 0.00016123985072753882, "grad_norm": 0.009357220493257046, "loss_scale": 4.0, "retries": 0, "seconds": 882.5, "max_memory_gb": 19.16}
|
| 72 |
+
{"step": 72, "steps": 500, "lr": 1.941195501593366e-05, "loss": 0.01001710024820051, "grad_norm": 0.7233017683029175, "loss_scale": 4.0, "retries": 0, "seconds": 894.9, "max_memory_gb": 19.16}
|
| 73 |
+
{"step": 73, "steps": 500, "lr": 1.939100809811683e-05, "loss": 0.01863934136343398, "grad_norm": 3.3135464191436768, "loss_scale": 4.0, "retries": 0, "seconds": 907.0, "max_memory_gb": 19.16}
|
| 74 |
+
{"step": 74, "steps": 500, "lr": 1.93697076540736e-05, "loss": 0.00045674042036642447, "grad_norm": 0.07909339666366577, "loss_scale": 4.0, "retries": 0, "seconds": 919.4, "max_memory_gb": 19.16}
|
| 75 |
+
{"step": 75, "steps": 500, "lr": 1.934805458122483e-05, "loss": 0.05323649716321845, "grad_norm": 3.742339611053467, "loss_scale": 4.0, "retries": 0, "seconds": 932.0, "max_memory_gb": 19.16}
|
| 76 |
+
{"step": 76, "steps": 500, "lr": 1.9326049791848156e-05, "loss": 0.05242073321485143, "grad_norm": 4.86016845703125, "loss_scale": 4.0, "retries": 0, "seconds": 944.0, "max_memory_gb": 19.16}
|
| 77 |
+
{"step": 77, "steps": 500, "lr": 1.9303694213039598e-05, "loss": 0.00038062678844529393, "grad_norm": 0.04982038959860802, "loss_scale": 4.0, "retries": 0, "seconds": 956.4, "max_memory_gb": 19.16}
|
| 78 |
+
{"step": 78, "steps": 500, "lr": 1.9280988786674468e-05, "loss": 0.0019577835534092137, "grad_norm": 0.265474796295166, "loss_scale": 4.0, "retries": 0, "seconds": 968.7, "max_memory_gb": 19.16}
|
| 79 |
+
{"step": 79, "steps": 500, "lr": 1.925793446936771e-05, "loss": 0.00023526155565501872, "grad_norm": 0.03089662827551365, "loss_scale": 4.0, "retries": 0, "seconds": 980.9, "max_memory_gb": 19.16}
|
| 80 |
+
{"step": 80, "steps": 500, "lr": 1.923453223243358e-05, "loss": 0.7122101664758391, "grad_norm": 19.3371524810791, "loss_scale": 4.0, "retries": 0, "seconds": 993.2, "max_memory_gb": 19.16}
|
| 81 |
+
{"step": 81, "steps": 500, "lr": 1.9210783061844728e-05, "loss": 0.5578952376098001, "grad_norm": 18.24076271057129, "loss_scale": 4.0, "retries": 0, "seconds": 1005.4, "max_memory_gb": 19.16}
|
| 82 |
+
{"step": 82, "steps": 500, "lr": 1.9186687958190668e-05, "loss": 0.0013182461856176399, "grad_norm": 0.13428619503974915, "loss_scale": 4.0, "retries": 0, "seconds": 1018.2, "max_memory_gb": 19.16}
|
| 83 |
+
{"step": 83, "steps": 500, "lr": 1.91622479366356e-05, "loss": 0.0374841591082238, "grad_norm": 4.630455017089844, "loss_scale": 4.0, "retries": 0, "seconds": 1031.4, "max_memory_gb": 19.16}
|
| 84 |
+
{"step": 84, "steps": 500, "lr": 1.9137464026875653e-05, "loss": 0.15488203325697794, "grad_norm": 9.621185302734375, "loss_scale": 4.0, "retries": 0, "seconds": 1044.0, "max_memory_gb": 19.16}
|
| 85 |
+
{"step": 85, "steps": 500, "lr": 1.9112337273095516e-05, "loss": 0.00010378636366939729, "grad_norm": 0.006684581749141216, "loss_scale": 4.0, "retries": 0, "seconds": 1056.4, "max_memory_gb": 19.16}
|
| 86 |
+
{"step": 86, "steps": 500, "lr": 1.908686873392441e-05, "loss": 0.004066335515233278, "grad_norm": 0.431837797164917, "loss_scale": 4.0, "retries": 0, "seconds": 1068.9, "max_memory_gb": 19.16}
|
| 87 |
+
{"step": 87, "steps": 500, "lr": 1.9061059482391522e-05, "loss": 0.0009179214689641991, "grad_norm": 0.09083033353090286, "loss_scale": 4.0, "retries": 0, "seconds": 1081.5, "max_memory_gb": 19.16}
|
| 88 |
+
{"step": 88, "steps": 500, "lr": 1.9034910605880777e-05, "loss": 0.00036720928034386906, "grad_norm": 0.051428817212581635, "loss_scale": 4.0, "retries": 0, "seconds": 1093.9, "max_memory_gb": 19.16}
|
| 89 |
+
{"step": 89, "steps": 500, "lr": 1.9008423206085016e-05, "loss": 0.00040148351156042076, "grad_norm": 0.02886226959526539, "loss_scale": 4.0, "retries": 0, "seconds": 1106.5, "max_memory_gb": 19.16}
|
| 90 |
+
{"step": 90, "steps": 500, "lr": 1.898159839895961e-05, "loss": 0.000565486831237391, "grad_norm": 0.03413832560181618, "loss_scale": 4.0, "retries": 0, "seconds": 1118.7, "max_memory_gb": 19.16}
|
| 91 |
+
{"step": 91, "steps": 500, "lr": 1.895443731467542e-05, "loss": 8.899656934473074e-05, "grad_norm": 0.004374784883111715, "loss_scale": 4.0, "retries": 0, "seconds": 1131.1, "max_memory_gb": 19.16}
|
| 92 |
+
{"step": 92, "steps": 500, "lr": 1.892694109757118e-05, "loss": 8.807189865546083e-05, "grad_norm": 0.003908241167664528, "loss_scale": 4.0, "retries": 0, "seconds": 1143.6, "max_memory_gb": 19.16}
|
| 93 |
+
{"step": 93, "steps": 500, "lr": 1.8899110906105304e-05, "loss": 0.006773238088550734, "grad_norm": 1.2269067764282227, "loss_scale": 4.0, "retries": 0, "seconds": 1156.0, "max_memory_gb": 19.16}
|
| 94 |
+
{"step": 94, "steps": 500, "lr": 1.887094791280705e-05, "loss": 0.003786853066233417, "grad_norm": 0.22323040664196014, "loss_scale": 4.0, "retries": 0, "seconds": 1168.3, "max_memory_gb": 19.16}
|
| 95 |
+
{"step": 95, "steps": 500, "lr": 1.8842453304227153e-05, "loss": 0.043863321157382215, "grad_norm": 5.612365245819092, "loss_scale": 4.0, "retries": 0, "seconds": 1180.7, "max_memory_gb": 19.16}
|
| 96 |
+
{"step": 96, "steps": 500, "lr": 1.8813628280887796e-05, "loss": 0.0028242669616828794, "grad_norm": 0.2057361751794815, "loss_scale": 4.0, "retries": 0, "seconds": 1193.0, "max_memory_gb": 19.16}
|
| 97 |
+
{"step": 97, "steps": 500, "lr": 1.8784474057232066e-05, "loss": 0.0022623886109927457, "grad_norm": 0.14483234286308289, "loss_scale": 4.0, "retries": 0, "seconds": 1205.6, "max_memory_gb": 19.16}
|
| 98 |
+
{"step": 98, "steps": 500, "lr": 1.8754991861572767e-05, "loss": 0.0014549395799163408, "grad_norm": 0.11385591328144073, "loss_scale": 4.0, "retries": 0, "seconds": 1219.6, "max_memory_gb": 19.16}
|
| 99 |
+
{"step": 99, "steps": 500, "lr": 1.872518293604067e-05, "loss": 0.01134281041670058, "grad_norm": 1.138087511062622, "loss_scale": 4.0, "retries": 0, "seconds": 1232.9, "max_memory_gb": 19.16}
|
| 100 |
+
{"step": 100, "steps": 500, "lr": 1.8695048536532184e-05, "loss": 0.0018059812271321689, "grad_norm": 0.18906503915786743, "loss_scale": 4.0, "retries": 0, "seconds": 1246.3, "max_memory_gb": 19.16}
|
| 101 |
+
{"step": 101, "steps": 500, "lr": 1.8664589932656442e-05, "loss": 0.02876158849807098, "grad_norm": 1.5552310943603516, "loss_scale": 4.0, "retries": 0, "seconds": 1259.4, "max_memory_gb": 19.16}
|
| 102 |
+
{"step": 102, "steps": 500, "lr": 1.8633808407681826e-05, "loss": 0.2887438018471523, "grad_norm": 8.886519432067871, "loss_scale": 4.0, "retries": 0, "seconds": 1272.4, "max_memory_gb": 19.16}
|
| 103 |
+
{"step": 103, "steps": 500, "lr": 1.8602705258481866e-05, "loss": 0.00022300635201717839, "grad_norm": 0.015080842189490795, "loss_scale": 4.0, "retries": 0, "seconds": 1284.9, "max_memory_gb": 19.16}
|
| 104 |
+
{"step": 104, "steps": 500, "lr": 1.8571281795480632e-05, "loss": 0.0007284077123146786, "grad_norm": 0.04080623388290405, "loss_scale": 4.0, "retries": 0, "seconds": 1297.3, "max_memory_gb": 19.16}
|
| 105 |
+
{"step": 105, "steps": 500, "lr": 1.8539539342597513e-05, "loss": 6.157001062234714e-05, "grad_norm": 0.003194230142980814, "loss_scale": 4.0, "retries": 0, "seconds": 1309.7, "max_memory_gb": 19.16}
|
| 106 |
+
{"step": 106, "steps": 500, "lr": 1.8507479237191432e-05, "loss": 0.00039884858560800365, "grad_norm": 0.02823888696730137, "loss_scale": 4.0, "retries": 0, "seconds": 1321.9, "max_memory_gb": 19.16}
|
| 107 |
+
{"step": 107, "steps": 500, "lr": 1.847510283000451e-05, "loss": 0.05622028577386895, "grad_norm": 3.3068289756774902, "loss_scale": 4.0, "retries": 0, "seconds": 1334.3, "max_memory_gb": 19.16}
|
| 108 |
+
{"step": 108, "steps": 500, "lr": 1.844241148510516e-05, "loss": 0.00033448753695417764, "grad_norm": 0.024798566475510597, "loss_scale": 4.0, "retries": 0, "seconds": 1346.6, "max_memory_gb": 19.16}
|
| 109 |
+
{"step": 109, "steps": 500, "lr": 1.84094065798306e-05, "loss": 0.09215766244327286, "grad_norm": 6.484415531158447, "loss_scale": 4.0, "retries": 0, "seconds": 1359.3, "max_memory_gb": 19.16}
|
| 110 |
+
{"step": 110, "steps": 500, "lr": 1.8376089504728846e-05, "loss": 0.005746829963920225, "grad_norm": 0.6352929472923279, "loss_scale": 4.0, "retries": 0, "seconds": 1371.9, "max_memory_gb": 19.16}
|
| 111 |
+
{"step": 111, "steps": 500, "lr": 1.8342461663500107e-05, "loss": 0.009001563254762666, "grad_norm": 0.5234724283218384, "loss_scale": 4.0, "retries": 0, "seconds": 1385.5, "max_memory_gb": 19.16}
|
| 112 |
+
{"step": 112, "steps": 500, "lr": 1.8308524472937653e-05, "loss": 7.010171286868751e-05, "grad_norm": 0.0039367531426250935, "loss_scale": 4.0, "retries": 0, "seconds": 1398.7, "max_memory_gb": 19.16}
|
| 113 |
+
{"step": 113, "steps": 500, "lr": 1.827427936286812e-05, "loss": 9.569206603869547e-05, "grad_norm": 0.007573653012514114, "loss_scale": 4.0, "retries": 0, "seconds": 1411.5, "max_memory_gb": 19.16}
|
| 114 |
+
{"step": 114, "steps": 500, "lr": 1.8239727776091283e-05, "loss": 0.0035367565816179867, "grad_norm": 0.3879382014274597, "loss_scale": 4.0, "retries": 0, "seconds": 1424.1, "max_memory_gb": 19.16}
|
| 115 |
+
{"step": 115, "steps": 500, "lr": 1.8204871168319246e-05, "loss": 0.0010001140512010602, "grad_norm": 0.10388251394033432, "loss_scale": 4.0, "retries": 0, "seconds": 1436.6, "max_memory_gb": 19.16}
|
| 116 |
+
{"step": 116, "steps": 500, "lr": 1.8169711008115124e-05, "loss": 0.0010261092948166029, "grad_norm": 0.2307799756526947, "loss_scale": 4.0, "retries": 0, "seconds": 1449.1, "max_memory_gb": 19.16}
|
| 117 |
+
{"step": 117, "steps": 500, "lr": 1.813424877683118e-05, "loss": 0.003228139375954181, "grad_norm": 0.19496233761310577, "loss_scale": 4.0, "retries": 0, "seconds": 1461.5, "max_memory_gb": 19.16}
|
| 118 |
+
{"step": 118, "steps": 500, "lr": 1.8098485968546385e-05, "loss": 0.01025475323899369, "grad_norm": 0.9596688151359558, "loss_scale": 4.0, "retries": 0, "seconds": 1474.0, "max_memory_gb": 19.16}
|
| 119 |
+
{"step": 119, "steps": 500, "lr": 1.806242409000349e-05, "loss": 5.7178573045617184e-05, "grad_norm": 0.005452998913824558, "loss_scale": 4.0, "retries": 0, "seconds": 1486.4, "max_memory_gb": 19.16}
|
| 120 |
+
{"step": 120, "steps": 500, "lr": 1.8026064660545554e-05, "loss": 0.22115385573747728, "grad_norm": 3.7719943523406982, "loss_scale": 4.0, "retries": 0, "seconds": 1498.9, "max_memory_gb": 19.16}
|
| 121 |
+
{"step": 121, "steps": 500, "lr": 1.7989409212051905e-05, "loss": 4.445701244228737e-05, "grad_norm": 0.001942825852893293, "loss_scale": 4.0, "retries": 0, "seconds": 1511.2, "max_memory_gb": 19.16}
|
| 122 |
+
{"step": 122, "steps": 500, "lr": 1.7952459288873623e-05, "loss": 0.005547329936415612, "grad_norm": 1.7127381563186646, "loss_scale": 4.0, "retries": 0, "seconds": 1523.6, "max_memory_gb": 19.16}
|
| 123 |
+
{"step": 123, "steps": 500, "lr": 1.7915216447768455e-05, "loss": 4.475702546487396e-05, "grad_norm": 0.0020934510976076126, "loss_scale": 4.0, "retries": 0, "seconds": 1536.0, "max_memory_gb": 19.16}
|
| 124 |
+
{"step": 124, "steps": 500, "lr": 1.7877682257835244e-05, "loss": 0.04901760785742485, "grad_norm": 1.5948512554168701, "loss_scale": 4.0, "retries": 0, "seconds": 1548.7, "max_memory_gb": 19.16}
|
| 125 |
+
{"step": 125, "steps": 500, "lr": 1.783985830044781e-05, "loss": 0.00706339723017312, "grad_norm": 0.3198327124118805, "loss_scale": 4.0, "retries": 0, "seconds": 1561.6, "max_memory_gb": 19.16}
|
| 126 |
+
{"step": 126, "steps": 500, "lr": 1.7801746169188328e-05, "loss": 0.005727406753778879, "grad_norm": 0.4560946524143219, "loss_scale": 4.0, "retries": 0, "seconds": 1573.9, "max_memory_gb": 19.16}
|
| 127 |
+
{"step": 127, "steps": 500, "lr": 1.7763347469780178e-05, "loss": 0.00010904179954836351, "grad_norm": 0.007448344491422176, "loss_scale": 4.0, "retries": 0, "seconds": 1586.3, "max_memory_gb": 19.16}
|
| 128 |
+
{"step": 128, "steps": 500, "lr": 1.772466382002032e-05, "loss": 6.09326057640787e-05, "grad_norm": 0.005992976948618889, "loss_scale": 4.0, "retries": 0, "seconds": 1598.9, "max_memory_gb": 19.16}
|
| 129 |
+
{"step": 129, "steps": 500, "lr": 1.7685696849711094e-05, "loss": 0.007678193516720844, "grad_norm": 0.4535377621650696, "loss_scale": 4.0, "retries": 0, "seconds": 1611.3, "max_memory_gb": 19.16}
|
| 130 |
+
{"step": 130, "steps": 500, "lr": 1.7646448200591597e-05, "loss": 0.0037363267749374145, "grad_norm": 0.41243451833724976, "loss_scale": 4.0, "retries": 0, "seconds": 1623.6, "max_memory_gb": 19.16}
|
| 131 |
+
{"step": 131, "steps": 500, "lr": 1.760691952626847e-05, "loss": 0.06505798081888449, "grad_norm": 2.0486185550689697, "loss_scale": 4.0, "retries": 0, "seconds": 1635.9, "max_memory_gb": 19.16}
|
| 132 |
+
{"step": 132, "steps": 500, "lr": 1.756711249214627e-05, "loss": 0.002766088141681616, "grad_norm": 0.15225675702095032, "loss_scale": 4.0, "retries": 0, "seconds": 1648.1, "max_memory_gb": 19.16}
|
| 133 |
+
{"step": 133, "steps": 500, "lr": 1.752702877535728e-05, "loss": 0.15240897365849934, "grad_norm": 4.913930892944336, "loss_scale": 4.0, "retries": 0, "seconds": 1660.4, "max_memory_gb": 19.16}
|
| 134 |
+
{"step": 134, "steps": 500, "lr": 1.7486670064690848e-05, "loss": 8.171666550699734e-05, "grad_norm": 0.005142392590641975, "loss_scale": 4.0, "retries": 0, "seconds": 1672.9, "max_memory_gb": 19.16}
|
| 135 |
+
{"step": 135, "steps": 500, "lr": 1.7446038060522255e-05, "loss": 0.00012364882548803797, "grad_norm": 0.01379960123449564, "loss_scale": 4.0, "retries": 0, "seconds": 1685.6, "max_memory_gb": 19.16}
|
| 136 |
+
{"step": 136, "steps": 500, "lr": 1.7405134474741042e-05, "loss": 5.9346591051869595e-05, "grad_norm": 0.0027663891669362783, "loss_scale": 4.0, "retries": 0, "seconds": 1698.2, "max_memory_gb": 19.16}
|
| 137 |
+
{"step": 137, "steps": 500, "lr": 1.736396103067893e-05, "loss": 0.0001292191957130807, "grad_norm": 0.007053705397993326, "loss_scale": 4.0, "retries": 0, "seconds": 1711.4, "max_memory_gb": 19.16}
|
| 138 |
+
{"step": 138, "steps": 500, "lr": 1.732251946303717e-05, "loss": 0.00011699481955932097, "grad_norm": 0.006583182606846094, "loss_scale": 4.0, "retries": 0, "seconds": 1724.0, "max_memory_gb": 19.16}
|
| 139 |
+
{"step": 139, "steps": 500, "lr": 1.728081151781349e-05, "loss": 0.2166568809755283, "grad_norm": 4.091012477874756, "loss_scale": 4.0, "retries": 0, "seconds": 1736.9, "max_memory_gb": 19.16}
|
| 140 |
+
{"step": 140, "steps": 500, "lr": 1.7238838952228502e-05, "loss": 0.0039569173113569245, "grad_norm": 0.5881930589675903, "loss_scale": 4.0, "retries": 0, "seconds": 1749.7, "max_memory_gb": 19.16}
|
| 141 |
+
{"step": 141, "steps": 500, "lr": 1.719660353465171e-05, "loss": 0.010720013969621403, "grad_norm": 1.0916720628738403, "loss_scale": 4.0, "retries": 0, "seconds": 1762.2, "max_memory_gb": 19.16}
|
| 142 |
+
{"step": 142, "steps": 500, "lr": 1.7154107044526965e-05, "loss": 0.00031759783662463104, "grad_norm": 0.021731574088335037, "loss_scale": 4.0, "retries": 0, "seconds": 1774.6, "max_memory_gb": 19.16}
|
| 143 |
+
{"step": 143, "steps": 500, "lr": 1.7111351272297514e-05, "loss": 0.0778542185191911, "grad_norm": 3.717827320098877, "loss_scale": 4.0, "retries": 0, "seconds": 1787.1, "max_memory_gb": 19.16}
|
| 144 |
+
{"step": 144, "steps": 500, "lr": 1.7068338019330564e-05, "loss": 0.0026458051925986447, "grad_norm": 0.17396405339241028, "loss_scale": 4.0, "retries": 0, "seconds": 1799.6, "max_memory_gb": 19.16}
|
| 145 |
+
{"step": 145, "steps": 500, "lr": 1.702506909784138e-05, "loss": 0.00017305876025375255, "grad_norm": 0.015134356915950775, "loss_scale": 4.0, "retries": 0, "seconds": 1812.2, "max_memory_gb": 19.16}
|
| 146 |
+
{"step": 146, "steps": 500, "lr": 1.6981546330816953e-05, "loss": 0.0001228210928626471, "grad_norm": 0.007422479800879955, "loss_scale": 4.0, "retries": 0, "seconds": 1824.8, "max_memory_gb": 19.16}
|
| 147 |
+
{"step": 147, "steps": 500, "lr": 1.6937771551939167e-05, "loss": 7.470385674873177e-05, "grad_norm": 0.004431018605828285, "loss_scale": 4.0, "retries": 0, "seconds": 1838.1, "max_memory_gb": 19.16}
|
| 148 |
+
{"step": 148, "steps": 500, "lr": 1.6893746605507567e-05, "loss": 0.19796892105049133, "grad_norm": 7.51606559753418, "loss_scale": 4.0, "retries": 0, "seconds": 1850.6, "max_memory_gb": 19.16}
|
| 149 |
+
{"step": 149, "steps": 500, "lr": 1.684947334636164e-05, "loss": 0.011745184850538368, "grad_norm": 0.5827322602272034, "loss_scale": 4.0, "retries": 0, "seconds": 1863.2, "max_memory_gb": 19.16}
|
| 150 |
+
{"step": 150, "steps": 500, "lr": 1.6804953639802675e-05, "loss": 0.11331915589860841, "grad_norm": 2.5292882919311523, "loss_scale": 4.0, "retries": 0, "seconds": 1875.6, "max_memory_gb": 19.16}
|
| 151 |
+
{"step": 151, "steps": 500, "lr": 1.6760189361515182e-05, "loss": 8.296708735274065e-05, "grad_norm": 0.003969008103013039, "loss_scale": 4.0, "retries": 0, "seconds": 1888.4, "max_memory_gb": 19.16}
|
| 152 |
+
{"step": 152, "steps": 500, "lr": 1.6715182397487845e-05, "loss": 0.006613457255722014, "grad_norm": 0.28561046719551086, "loss_scale": 4.0, "retries": 0, "seconds": 1901.4, "max_memory_gb": 19.16}
|
| 153 |
+
{"step": 153, "steps": 500, "lr": 1.6669934643934088e-05, "loss": 0.00010350698201994589, "grad_norm": 0.006879589054733515, "loss_scale": 4.0, "retries": 0, "seconds": 1914.4, "max_memory_gb": 19.16}
|
| 154 |
+
{"step": 154, "steps": 500, "lr": 1.6624448007212168e-05, "loss": 0.0004961589428647399, "grad_norm": 0.029972517862915993, "loss_scale": 4.0, "retries": 0, "seconds": 1927.1, "max_memory_gb": 19.16}
|
| 155 |
+
{"step": 155, "steps": 500, "lr": 1.6578724403744854e-05, "loss": 0.00107577046225577, "grad_norm": 0.08721334487199783, "loss_scale": 4.0, "retries": 0, "seconds": 1939.8, "max_memory_gb": 19.16}
|
| 156 |
+
{"step": 156, "steps": 500, "lr": 1.653276575993871e-05, "loss": 0.013156649600094026, "grad_norm": 1.5990008115768433, "loss_scale": 4.0, "retries": 0, "seconds": 1952.4, "max_memory_gb": 19.16}
|
| 157 |
+
{"step": 157, "steps": 500, "lr": 1.6486574012102896e-05, "loss": 0.15005670221456124, "grad_norm": 2.4736573696136475, "loss_scale": 4.0, "retries": 0, "seconds": 1965.2, "max_memory_gb": 19.16}
|
| 158 |
+
{"step": 158, "steps": 500, "lr": 1.6440151106367617e-05, "loss": 0.020077338970785587, "grad_norm": 0.9571315050125122, "loss_scale": 4.0, "retries": 0, "seconds": 1978.1, "max_memory_gb": 19.16}
|
| 159 |
+
{"step": 159, "steps": 500, "lr": 1.6393498998602123e-05, "loss": 0.2954203728368777, "grad_norm": 2.490511894226074, "loss_scale": 4.0, "retries": 0, "seconds": 1990.8, "max_memory_gb": 19.16}
|
| 160 |
+
{"step": 160, "steps": 500, "lr": 1.6346619654332297e-05, "loss": 0.0004984020864950171, "grad_norm": 0.03394168242812157, "loss_scale": 4.0, "retries": 0, "seconds": 2003.7, "max_memory_gb": 19.16}
|
| 161 |
+
{"step": 161, "steps": 500, "lr": 1.6299515048657848e-05, "loss": 0.5891814767625917, "grad_norm": 15.127298355102539, "loss_scale": 4.0, "retries": 0, "seconds": 2016.2, "max_memory_gb": 19.16}
|
| 162 |
+
{"step": 162, "steps": 500, "lr": 1.6252187166169105e-05, "loss": 0.0037326589227704566, "grad_norm": 0.19703710079193115, "loss_scale": 4.0, "retries": 0, "seconds": 2028.9, "max_memory_gb": 19.16}
|
| 163 |
+
{"step": 163, "steps": 500, "lr": 1.6204638000863392e-05, "loss": 0.0010740316082030242, "grad_norm": 0.06957601755857468, "loss_scale": 4.0, "retries": 0, "seconds": 2041.5, "max_memory_gb": 19.16}
|
| 164 |
+
{"step": 164, "steps": 500, "lr": 1.615686955606102e-05, "loss": 0.03621848604432837, "grad_norm": 4.42715311050415, "loss_scale": 4.0, "retries": 0, "seconds": 2054.0, "max_memory_gb": 19.16}
|
| 165 |
+
{"step": 165, "steps": 500, "lr": 1.610888384432089e-05, "loss": 0.32620522838136495, "grad_norm": 6.767951488494873, "loss_scale": 4.0, "retries": 0, "seconds": 2066.6, "max_memory_gb": 19.16}
|
| 166 |
+
{"step": 166, "steps": 500, "lr": 1.60606828873557e-05, "loss": 0.1449463376536435, "grad_norm": 2.176162004470825, "loss_scale": 4.0, "retries": 0, "seconds": 2079.6, "max_memory_gb": 19.16}
|
| 167 |
+
{"step": 167, "steps": 500, "lr": 1.601226871594676e-05, "loss": 0.0009712095606201646, "grad_norm": 0.060767728835344315, "loss_scale": 4.0, "retries": 0, "seconds": 2092.2, "max_memory_gb": 19.16}
|
| 168 |
+
{"step": 168, "steps": 500, "lr": 1.5963643369858436e-05, "loss": 0.004835453754822084, "grad_norm": 0.5940470099449158, "loss_scale": 4.0, "retries": 0, "seconds": 2104.2, "max_memory_gb": 19.16}
|
| 169 |
+
{"step": 169, "steps": 500, "lr": 1.591480889775221e-05, "loss": 0.002560323318903102, "grad_norm": 0.16076427698135376, "loss_scale": 4.0, "retries": 0, "seconds": 2117.0, "max_memory_gb": 19.16}
|
| 170 |
+
{"step": 170, "steps": 500, "lr": 1.586576735710038e-05, "loss": 0.018513703768886364, "grad_norm": 1.0170692205429077, "loss_scale": 4.0, "retries": 0, "seconds": 2129.6, "max_memory_gb": 19.16}
|
| 171 |
+
{"step": 171, "steps": 500, "lr": 1.5816520814099346e-05, "loss": 0.003927388975832649, "grad_norm": 0.22579032182693481, "loss_scale": 4.0, "retries": 0, "seconds": 2142.2, "max_memory_gb": 19.16}
|
| 172 |
+
{"step": 172, "steps": 500, "lr": 1.576707134358259e-05, "loss": 0.0023971239211691264, "grad_norm": 0.10370494425296783, "loss_scale": 4.0, "retries": 0, "seconds": 2154.8, "max_memory_gb": 19.16}
|
| 173 |
+
{"step": 173, "steps": 500, "lr": 1.5717421028933236e-05, "loss": 0.0042926454405625755, "grad_norm": 0.15855689346790314, "loss_scale": 4.0, "retries": 0, "seconds": 2167.5, "max_memory_gb": 19.16}
|
| 174 |
+
{"step": 174, "steps": 500, "lr": 1.5667571961996297e-05, "loss": 0.00119648971750311, "grad_norm": 0.0684933215379715, "loss_scale": 4.0, "retries": 0, "seconds": 2179.9, "max_memory_gb": 19.16}
|
| 175 |
+
{"step": 175, "steps": 500, "lr": 1.5617526242990515e-05, "loss": 0.001566814792154632, "grad_norm": 0.0709943026304245, "loss_scale": 4.0, "retries": 0, "seconds": 2192.2, "max_memory_gb": 19.16}
|
| 176 |
+
{"step": 176, "steps": 500, "lr": 1.5567285980419903e-05, "loss": 0.001365076008369215, "grad_norm": 0.07757808268070221, "loss_scale": 4.0, "retries": 0, "seconds": 2205.0, "max_memory_gb": 19.16}
|
| 177 |
+
{"step": 177, "steps": 500, "lr": 1.551685329098488e-05, "loss": 0.16742371591325625, "grad_norm": 5.313250541687012, "loss_scale": 4.0, "retries": 0, "seconds": 2217.6, "max_memory_gb": 19.16}
|
| 178 |
+
{"step": 178, "steps": 500, "lr": 1.5466230299493127e-05, "loss": 0.0055290117623950545, "grad_norm": 0.29495665431022644, "loss_scale": 4.0, "retries": 0, "seconds": 2230.3, "max_memory_gb": 19.16}
|
| 179 |
+
{"step": 179, "steps": 500, "lr": 1.541541913877003e-05, "loss": 0.0013567925071914612, "grad_norm": 0.06588306277990341, "loss_scale": 4.0, "retries": 0, "seconds": 2243.0, "max_memory_gb": 19.16}
|
| 180 |
+
{"step": 180, "steps": 500, "lr": 1.5364421949568853e-05, "loss": 0.000517288378887315, "grad_norm": 0.027995094656944275, "loss_scale": 4.0, "retries": 0, "seconds": 2255.6, "max_memory_gb": 19.16}
|
| 181 |
+
{"step": 181, "steps": 500, "lr": 1.5313240880480514e-05, "loss": 0.0005022396814679682, "grad_norm": 0.02834111452102661, "loss_scale": 4.0, "retries": 0, "seconds": 2268.4, "max_memory_gb": 19.16}
|
| 182 |
+
{"step": 182, "steps": 500, "lr": 1.5261878087843087e-05, "loss": 0.004502766763421691, "grad_norm": 0.30753982067108154, "loss_scale": 4.0, "retries": 0, "seconds": 2281.0, "max_memory_gb": 19.16}
|
| 183 |
+
{"step": 183, "steps": 500, "lr": 1.521033573565094e-05, "loss": 0.002099600309334578, "grad_norm": 0.09736988693475723, "loss_scale": 4.0, "retries": 0, "seconds": 2293.7, "max_memory_gb": 19.16}
|
| 184 |
+
{"step": 184, "steps": 500, "lr": 1.5158615995463566e-05, "loss": 0.25663176452007974, "grad_norm": 11.091533660888672, "loss_scale": 4.0, "retries": 0, "seconds": 2306.3, "max_memory_gb": 19.16}
|
| 185 |
+
{"step": 185, "steps": 500, "lr": 1.510672104631408e-05, "loss": 0.15870168265374476, "grad_norm": 8.458619117736816, "loss_scale": 4.0, "retries": 0, "seconds": 2318.7, "max_memory_gb": 19.16}
|
| 186 |
+
{"step": 186, "steps": 500, "lr": 1.5054653074617441e-05, "loss": 0.0002862622581315836, "grad_norm": 0.013251623138785362, "loss_scale": 4.0, "retries": 0, "seconds": 2331.3, "max_memory_gb": 19.16}
|
| 187 |
+
{"step": 187, "steps": 500, "lr": 1.50024142740783e-05, "loss": 0.00028221519203890983, "grad_norm": 0.01485345046967268, "loss_scale": 4.0, "retries": 0, "seconds": 2344.1, "max_memory_gb": 19.16}
|
| 188 |
+
{"step": 188, "steps": 500, "lr": 1.4950006845598603e-05, "loss": 0.06960516120923123, "grad_norm": 3.4548964500427246, "loss_scale": 4.0, "retries": 0, "seconds": 2356.7, "max_memory_gb": 19.16}
|
| 189 |
+
{"step": 189, "steps": 500, "lr": 1.4897432997184845e-05, "loss": 0.016737217169934, "grad_norm": 2.4193179607391357, "loss_scale": 4.0, "retries": 0, "seconds": 2369.4, "max_memory_gb": 19.16}
|
| 190 |
+
{"step": 190, "steps": 500, "lr": 1.4844694943855065e-05, "loss": 0.00033576135018103577, "grad_norm": 0.015517300926148891, "loss_scale": 4.0, "retries": 0, "seconds": 2381.7, "max_memory_gb": 19.16}
|
| 191 |
+
{"step": 191, "steps": 500, "lr": 1.4791794907545496e-05, "loss": 0.001791032321689686, "grad_norm": 0.09485849738121033, "loss_scale": 4.0, "retries": 0, "seconds": 2394.4, "max_memory_gb": 19.16}
|
| 192 |
+
{"step": 192, "steps": 500, "lr": 1.4738735117016979e-05, "loss": 0.0001919557032269123, "grad_norm": 0.008382418192923069, "loss_scale": 4.0, "retries": 0, "seconds": 2407.2, "max_memory_gb": 19.16}
|
| 193 |
+
{"step": 193, "steps": 500, "lr": 1.4685517807761037e-05, "loss": 0.000207899364485539, "grad_norm": 0.011628544889390469, "loss_scale": 4.0, "retries": 0, "seconds": 2419.8, "max_memory_gb": 19.16}
|
| 194 |
+
{"step": 194, "steps": 500, "lr": 1.463214522190571e-05, "loss": 0.0008908388929181787, "grad_norm": 0.04332760348916054, "loss_scale": 4.0, "retries": 0, "seconds": 2432.4, "max_memory_gb": 19.16}
|
| 195 |
+
{"step": 195, "steps": 500, "lr": 1.4578619608121079e-05, "loss": 0.007032383951530557, "grad_norm": 0.6859220266342163, "loss_scale": 4.0, "retries": 0, "seconds": 2445.1, "max_memory_gb": 19.16}
|
| 196 |
+
{"step": 196, "steps": 500, "lr": 1.4524943221524533e-05, "loss": 0.0002776086543731667, "grad_norm": 0.014764529652893543, "loss_scale": 4.0, "retries": 0, "seconds": 2457.7, "max_memory_gb": 19.16}
|
| 197 |
+
{"step": 197, "steps": 500, "lr": 1.4471118323585751e-05, "loss": 0.03670429605307907, "grad_norm": 1.3997581005096436, "loss_scale": 4.0, "retries": 0, "seconds": 2470.5, "max_memory_gb": 19.16}
|
| 198 |
+
{"step": 198, "steps": 500, "lr": 1.4417147182031426e-05, "loss": 1.0764842509471775, "grad_norm": 12.229362487792969, "loss_scale": 4.0, "retries": 0, "seconds": 2483.2, "max_memory_gb": 19.16}
|
| 199 |
+
{"step": 199, "steps": 500, "lr": 1.4363032070749723e-05, "loss": 0.008071920519837893, "grad_norm": 0.29478734731674194, "loss_scale": 4.0, "retries": 0, "seconds": 2495.7, "max_memory_gb": 19.16}
|
| 200 |
+
{"step": 200, "steps": 500, "lr": 1.430877526969448e-05, "loss": 0.008791547492121897, "grad_norm": 0.3980095088481903, "loss_scale": 4.0, "retries": 0, "seconds": 2508.1, "max_memory_gb": 19.16}
|
| 201 |
+
{"step": 201, "steps": 500, "lr": 1.4254379064789138e-05, "loss": 0.005462351897862886, "grad_norm": 0.4863741099834442, "loss_scale": 4.0, "retries": 0, "seconds": 2520.8, "max_memory_gb": 19.16}
|
| 202 |
+
{"step": 202, "steps": 500, "lr": 1.4199845747830443e-05, "loss": 0.00041580583337525405, "grad_norm": 0.02131495624780655, "loss_scale": 4.0, "retries": 0, "seconds": 2537.7, "max_memory_gb": 19.16}
|
| 203 |
+
{"step": 203, "steps": 500, "lr": 1.4145177616391886e-05, "loss": 0.00044171233106350314, "grad_norm": 0.019782865419983864, "loss_scale": 4.0, "retries": 0, "seconds": 2551.7, "max_memory_gb": 19.16}
|
| 204 |
+
{"step": 204, "steps": 500, "lr": 1.4090376973726898e-05, "loss": 0.00031765265232376225, "grad_norm": 0.012770439498126507, "loss_scale": 4.0, "retries": 0, "seconds": 2564.6, "max_memory_gb": 19.16}
|
| 205 |
+
{"step": 205, "steps": 500, "lr": 1.4035446128671815e-05, "loss": 0.0001448996048390489, "grad_norm": 0.00648446986451745, "loss_scale": 4.0, "retries": 0, "seconds": 2577.4, "max_memory_gb": 19.16}
|
| 206 |
+
{"step": 206, "steps": 500, "lr": 1.3980387395548608e-05, "loss": 0.08343880551917045, "grad_norm": 6.9969305992126465, "loss_scale": 4.0, "retries": 0, "seconds": 2590.2, "max_memory_gb": 19.16}
|
| 207 |
+
{"step": 207, "steps": 500, "lr": 1.3925203094067364e-05, "loss": 0.0026008565836574153, "grad_norm": 0.10066260397434235, "loss_scale": 4.0, "retries": 0, "seconds": 2604.0, "max_memory_gb": 19.16}
|
| 208 |
+
{"step": 208, "steps": 500, "lr": 1.3869895549228569e-05, "loss": 0.00013131672659483232, "grad_norm": 0.005237441975623369, "loss_scale": 4.0, "retries": 0, "seconds": 2618.9, "max_memory_gb": 19.16}
|
| 209 |
+
{"step": 209, "steps": 500, "lr": 1.381446709122513e-05, "loss": 0.0005342572288265563, "grad_norm": 0.026137812063097954, "loss_scale": 4.0, "retries": 0, "seconds": 2631.7, "max_memory_gb": 19.16}
|
| 210 |
+
{"step": 210, "steps": 500, "lr": 1.3758920055344226e-05, "loss": 0.0002894719610563615, "grad_norm": 0.011848540045320988, "loss_scale": 4.0, "retries": 0, "seconds": 2644.3, "max_memory_gb": 19.16}
|
| 211 |
+
{"step": 211, "steps": 500, "lr": 1.3703256781868898e-05, "loss": 0.007441727389763485, "grad_norm": 0.29615113139152527, "loss_scale": 4.0, "retries": 0, "seconds": 2657.0, "max_memory_gb": 19.16}
|
| 212 |
+
{"step": 212, "steps": 500, "lr": 1.3647479615979466e-05, "loss": 0.0001622293029351402, "grad_norm": 0.008335969410836697, "loss_scale": 4.0, "retries": 0, "seconds": 2669.7, "max_memory_gb": 19.16}
|
| 213 |
+
{"step": 213, "steps": 500, "lr": 1.359159090765471e-05, "loss": 0.0003481462905380216, "grad_norm": 0.015240555629134178, "loss_scale": 4.0, "retries": 0, "seconds": 2682.4, "max_memory_gb": 19.16}
|
| 214 |
+
{"step": 214, "steps": 500, "lr": 1.3535593011572868e-05, "loss": 0.1197597094053035, "grad_norm": 3.312883138656616, "loss_scale": 4.0, "retries": 0, "seconds": 2695.1, "max_memory_gb": 19.16}
|
| 215 |
+
{"step": 215, "steps": 500, "lr": 1.347948828701242e-05, "loss": 0.34796570720508474, "grad_norm": 17.367321014404297, "loss_scale": 4.0, "retries": 0, "seconds": 2707.9, "max_memory_gb": 19.16}
|
| 216 |
+
{"step": 216, "steps": 500, "lr": 1.3423279097752711e-05, "loss": 0.03789487411970847, "grad_norm": 1.0933661460876465, "loss_scale": 4.0, "retries": 0, "seconds": 2721.0, "max_memory_gb": 19.16}
|
| 217 |
+
{"step": 217, "steps": 500, "lr": 1.3366967811974332e-05, "loss": 0.0002724848626588606, "grad_norm": 0.015745174139738083, "loss_scale": 4.0, "retries": 0, "seconds": 2734.7, "max_memory_gb": 19.16}
|
| 218 |
+
{"step": 218, "steps": 500, "lr": 1.3310556802159368e-05, "loss": 0.00027610361022115375, "grad_norm": 0.010211779735982418, "loss_scale": 4.0, "retries": 0, "seconds": 2747.4, "max_memory_gb": 19.16}
|
| 219 |
+
{"step": 219, "steps": 500, "lr": 1.325404844499143e-05, "loss": 0.08175654702661592, "grad_norm": 2.2950806617736816, "loss_scale": 4.0, "retries": 0, "seconds": 2760.0, "max_memory_gb": 19.16}
|
| 220 |
+
{"step": 220, "steps": 500, "lr": 1.3197445121255522e-05, "loss": 0.0005375740886393031, "grad_norm": 0.025781992822885513, "loss_scale": 4.0, "retries": 0, "seconds": 2772.6, "max_memory_gb": 19.16}
|
| 221 |
+
{"step": 221, "steps": 500, "lr": 1.3140749215737745e-05, "loss": 0.0002164840000773438, "grad_norm": 0.010444928891956806, "loss_scale": 4.0, "retries": 0, "seconds": 2785.2, "max_memory_gb": 19.16}
|
| 222 |
+
{"step": 222, "steps": 500, "lr": 1.3083963117124801e-05, "loss": 0.00021599248940550422, "grad_norm": 0.008891894482076168, "loss_scale": 4.0, "retries": 0, "seconds": 2797.8, "max_memory_gb": 19.16}
|
| 223 |
+
{"step": 223, "steps": 500, "lr": 1.3027089217903386e-05, "loss": 7.42778582889514e-05, "grad_norm": 0.0028295537922531366, "loss_scale": 4.0, "retries": 0, "seconds": 2810.4, "max_memory_gb": 19.16}
|
| 224 |
+
{"step": 224, "steps": 500, "lr": 1.2970129914259354e-05, "loss": 0.4238531878287198, "grad_norm": 3.0162160396575928, "loss_scale": 4.0, "retries": 0, "seconds": 2822.5, "max_memory_gb": 19.16}
|
| 225 |
+
{"step": 225, "steps": 500, "lr": 1.2913087605976789e-05, "loss": 0.00022615886443976373, "grad_norm": 0.00933234766125679, "loss_scale": 4.0, "retries": 0, "seconds": 2834.8, "max_memory_gb": 19.16}
|
| 226 |
+
{"step": 226, "steps": 500, "lr": 1.2855964696336894e-05, "loss": 0.01698133210471104, "grad_norm": 1.4324312210083008, "loss_scale": 4.0, "retries": 0, "seconds": 2847.3, "max_memory_gb": 19.16}
|
| 227 |
+
{"step": 227, "steps": 500, "lr": 1.2798763592016722e-05, "loss": 0.14421168226094366, "grad_norm": 3.232727527618408, "loss_scale": 4.0, "retries": 0, "seconds": 2859.9, "max_memory_gb": 19.16}
|
| 228 |
+
{"step": 228, "steps": 500, "lr": 1.2741486702987796e-05, "loss": 0.0028348259411979434, "grad_norm": 0.15486778318881989, "loss_scale": 4.0, "retries": 0, "seconds": 2872.2, "max_memory_gb": 19.16}
|
| 229 |
+
{"step": 229, "steps": 500, "lr": 1.268413644241457e-05, "loss": 0.9402760662094248, "grad_norm": 10.702475547790527, "loss_scale": 4.0, "retries": 0, "seconds": 2884.9, "max_memory_gb": 19.16}
|
| 230 |
+
{"step": 230, "steps": 500, "lr": 1.2626715226552741e-05, "loss": 0.0005519518559925597, "grad_norm": 0.020554179325699806, "loss_scale": 4.0, "retries": 0, "seconds": 2897.4, "max_memory_gb": 19.16}
|
| 231 |
+
{"step": 231, "steps": 500, "lr": 1.2569225474647482e-05, "loss": 0.0002957188567672182, "grad_norm": 0.010664981789886951, "loss_scale": 4.0, "retries": 0, "seconds": 2910.0, "max_memory_gb": 19.16}
|
| 232 |
+
{"step": 232, "steps": 500, "lr": 1.2511669608831483e-05, "loss": 0.00025081782266831647, "grad_norm": 0.009573732502758503, "loss_scale": 4.0, "retries": 0, "seconds": 2922.6, "max_memory_gb": 19.16}
|
| 233 |
+
{"step": 233, "steps": 500, "lr": 1.2454050054022921e-05, "loss": 0.0007000074856271319, "grad_norm": 0.02673402614891529, "loss_scale": 4.0, "retries": 0, "seconds": 2935.2, "max_memory_gb": 19.16}
|
| 234 |
+
{"step": 234, "steps": 500, "lr": 1.2396369237823284e-05, "loss": 0.00036541807437308967, "grad_norm": 0.015930907800793648, "loss_scale": 4.0, "retries": 0, "seconds": 2947.8, "max_memory_gb": 19.16}
|
| 235 |
+
{"step": 235, "steps": 500, "lr": 1.2338629590415109e-05, "loss": 0.0009826140602161683, "grad_norm": 0.055676691234111786, "loss_scale": 4.0, "retries": 0, "seconds": 2960.1, "max_memory_gb": 19.16}
|
| 236 |
+
{"step": 236, "steps": 500, "lr": 1.2280833544459568e-05, "loss": 0.0007424013093461988, "grad_norm": 0.0328722819685936, "loss_scale": 4.0, "retries": 0, "seconds": 2972.8, "max_memory_gb": 19.16}
|
| 237 |
+
{"step": 237, "steps": 500, "lr": 1.2222983534994002e-05, "loss": 0.1647205430273392, "grad_norm": 2.343611717224121, "loss_scale": 4.0, "retries": 0, "seconds": 2985.4, "max_memory_gb": 19.16}
|
| 238 |
+
{"step": 238, "steps": 500, "lr": 1.2165081999329318e-05, "loss": 0.13609561745823323, "grad_norm": 4.212459564208984, "loss_scale": 4.0, "retries": 0, "seconds": 2998.1, "max_memory_gb": 19.16}
|
| 239 |
+
{"step": 239, "steps": 500, "lr": 1.2107131376947301e-05, "loss": 0.035534400383436485, "grad_norm": 1.128331184387207, "loss_scale": 4.0, "retries": 0, "seconds": 3010.6, "max_memory_gb": 19.16}
|
| 240 |
+
{"step": 240, "steps": 500, "lr": 1.2049134109397833e-05, "loss": 0.2204519857250489, "grad_norm": 5.719948768615723, "loss_scale": 4.0, "retries": 0, "seconds": 3023.5, "max_memory_gb": 19.16}
|
| 241 |
+
{"step": 241, "steps": 500, "lr": 1.1991092640196032e-05, "loss": 0.06670198192209266, "grad_norm": 1.3551175594329834, "loss_scale": 4.0, "retries": 0, "seconds": 3036.1, "max_memory_gb": 19.16}
|
| 242 |
+
{"step": 242, "steps": 500, "lr": 1.1933009414719298e-05, "loss": 0.00024655006238431684, "grad_norm": 0.00876656360924244, "loss_scale": 4.0, "retries": 0, "seconds": 3049.1, "max_memory_gb": 19.16}
|
| 243 |
+
{"step": 243, "steps": 500, "lr": 1.1874886880104294e-05, "loss": 0.010783732643341698, "grad_norm": 0.2663790285587311, "loss_scale": 4.0, "retries": 0, "seconds": 3061.8, "max_memory_gb": 19.16}
|
| 244 |
+
{"step": 244, "steps": 500, "lr": 1.181672748514383e-05, "loss": 0.001065016643224226, "grad_norm": 0.039864420890808105, "loss_scale": 4.0, "retries": 0, "seconds": 3074.3, "max_memory_gb": 19.16}
|
| 245 |
+
{"step": 245, "steps": 500, "lr": 1.1758533680183702e-05, "loss": 0.0014829959621817002, "grad_norm": 0.06203922629356384, "loss_scale": 4.0, "retries": 0, "seconds": 3087.0, "max_memory_gb": 19.16}
|
| 246 |
+
{"step": 246, "steps": 500, "lr": 1.1700307917019457e-05, "loss": 0.0003531495115732303, "grad_norm": 0.012919925153255463, "loss_scale": 4.0, "retries": 0, "seconds": 3099.6, "max_memory_gb": 19.16}
|
| 247 |
+
{"step": 247, "steps": 500, "lr": 1.164205264879309e-05, "loss": 0.06076266340278381, "grad_norm": 3.562411069869995, "loss_scale": 4.0, "retries": 0, "seconds": 3112.3, "max_memory_gb": 19.16}
|
| 248 |
+
{"step": 248, "steps": 500, "lr": 1.1583770329889687e-05, "loss": 0.0005779959699959198, "grad_norm": 0.01964086852967739, "loss_scale": 4.0, "retries": 0, "seconds": 3125.0, "max_memory_gb": 19.16}
|
| 249 |
+
{"step": 249, "steps": 500, "lr": 1.152546341583402e-05, "loss": 0.001441277047888434, "grad_norm": 0.053593263030052185, "loss_scale": 4.0, "retries": 0, "seconds": 3137.6, "max_memory_gb": 19.16}
|
| 250 |
+
{"step": 250, "steps": 500, "lr": 1.1467134363187098e-05, "loss": 0.0031359501571056503, "grad_norm": 0.13909098505973816, "loss_scale": 4.0, "retries": 0, "seconds": 3150.2, "max_memory_gb": 19.16}
|
| 251 |
+
{"step": 251, "steps": 500, "lr": 1.1408785629442657e-05, "loss": 0.021904713957610514, "grad_norm": 1.5866868495941162, "loss_scale": 4.0, "retries": 0, "seconds": 3162.6, "max_memory_gb": 19.16}
|
| 252 |
+
{"step": 252, "steps": 500, "lr": 1.1350419672923639e-05, "loss": 0.05813657528390403, "grad_norm": 1.2439173460006714, "loss_scale": 4.0, "retries": 0, "seconds": 3174.9, "max_memory_gb": 19.16}
|
| 253 |
+
{"step": 253, "steps": 500, "lr": 1.1292038952678597e-05, "loss": 0.02276047660040025, "grad_norm": 0.8733367323875427, "loss_scale": 4.0, "retries": 0, "seconds": 3187.6, "max_memory_gb": 19.16}
|
| 254 |
+
{"step": 254, "steps": 500, "lr": 1.1233645928378104e-05, "loss": 0.0008990925899958961, "grad_norm": 0.04141373187303543, "loss_scale": 4.0, "retries": 0, "seconds": 3200.6, "max_memory_gb": 19.16}
|
| 255 |
+
{"step": 255, "steps": 500, "lr": 1.117524306021113e-05, "loss": 0.11959539564254969, "grad_norm": 2.0305917263031006, "loss_scale": 4.0, "retries": 0, "seconds": 3213.0, "max_memory_gb": 19.16}
|
| 256 |
+
{"step": 256, "steps": 500, "lr": 1.1116832808781377e-05, "loss": 0.00036754991762419115, "grad_norm": 0.013517336919903755, "loss_scale": 4.0, "retries": 0, "seconds": 3225.5, "max_memory_gb": 19.16}
|
| 257 |
+
{"step": 257, "steps": 500, "lr": 1.1058417635003616e-05, "loss": 0.0010068505280287354, "grad_norm": 0.04478692263364792, "loss_scale": 4.0, "retries": 0, "seconds": 3238.1, "max_memory_gb": 19.16}
|
| 258 |
+
{"step": 258, "steps": 500, "lr": 1.1e-05, "loss": 0.004750969352109926, "grad_norm": 0.34751060605049133, "loss_scale": 4.0, "retries": 0, "seconds": 3250.8, "max_memory_gb": 19.16}
|
| 259 |
+
{"step": 259, "steps": 500, "lr": 1.0941582364996383e-05, "loss": 0.05392479088886211, "grad_norm": 2.022554397583008, "loss_scale": 4.0, "retries": 0, "seconds": 3263.4, "max_memory_gb": 19.16}
|
| 260 |
+
{"step": 260, "steps": 500, "lr": 1.0883167191218624e-05, "loss": 0.03071391212465228, "grad_norm": 1.0369412899017334, "loss_scale": 4.0, "retries": 0, "seconds": 3276.0, "max_memory_gb": 19.16}
|
| 261 |
+
{"step": 261, "steps": 500, "lr": 1.0824756939788867e-05, "loss": 0.0017645334794865039, "grad_norm": 0.08773285150527954, "loss_scale": 4.0, "retries": 0, "seconds": 3289.2, "max_memory_gb": 19.16}
|
| 262 |
+
{"step": 262, "steps": 500, "lr": 1.0766354071621896e-05, "loss": 0.0011105325594888363, "grad_norm": 0.0539093054831028, "loss_scale": 4.0, "retries": 0, "seconds": 3301.7, "max_memory_gb": 19.16}
|
| 263 |
+
{"step": 263, "steps": 500, "lr": 1.0707961047321403e-05, "loss": 0.0011839448147838993, "grad_norm": 0.0473097562789917, "loss_scale": 4.0, "retries": 0, "seconds": 3314.3, "max_memory_gb": 19.16}
|
| 264 |
+
{"step": 264, "steps": 500, "lr": 1.064958032707636e-05, "loss": 0.0006484788871561747, "grad_norm": 0.026548797264695168, "loss_scale": 4.0, "retries": 0, "seconds": 3327.1, "max_memory_gb": 19.16}
|
| 265 |
+
{"step": 265, "steps": 500, "lr": 1.0591214370557342e-05, "loss": 0.01506292309994189, "grad_norm": 0.7748597860336304, "loss_scale": 4.0, "retries": 0, "seconds": 3339.4, "max_memory_gb": 19.16}
|
| 266 |
+
{"step": 266, "steps": 500, "lr": 1.0532865636812903e-05, "loss": 0.16255719535952684, "grad_norm": 5.113837242126465, "loss_scale": 4.0, "retries": 0, "seconds": 3351.9, "max_memory_gb": 19.16}
|
| 267 |
+
{"step": 267, "steps": 500, "lr": 1.047453658416598e-05, "loss": 0.0007140235646933104, "grad_norm": 0.027439549565315247, "loss_scale": 4.0, "retries": 0, "seconds": 3364.5, "max_memory_gb": 19.16}
|
| 268 |
+
{"step": 268, "steps": 500, "lr": 1.0416229670110312e-05, "loss": 0.000939288821097183, "grad_norm": 0.03479573875665665, "loss_scale": 4.0, "retries": 0, "seconds": 3377.1, "max_memory_gb": 19.16}
|
| 269 |
+
{"step": 269, "steps": 500, "lr": 1.0357947351206909e-05, "loss": 0.059324055101011595, "grad_norm": 1.0798722505569458, "loss_scale": 4.0, "retries": 0, "seconds": 3389.9, "max_memory_gb": 19.16}
|
| 270 |
+
{"step": 270, "steps": 500, "lr": 1.029969208298054e-05, "loss": 0.018478552407383508, "grad_norm": 0.6853156089782715, "loss_scale": 4.0, "retries": 0, "seconds": 3402.4, "max_memory_gb": 19.16}
|
| 271 |
+
{"step": 271, "steps": 500, "lr": 1.02414663198163e-05, "loss": 0.0012817167943859954, "grad_norm": 0.04608792066574097, "loss_scale": 4.0, "retries": 0, "seconds": 3414.6, "max_memory_gb": 19.16}
|
| 272 |
+
{"step": 272, "steps": 500, "lr": 1.0183272514856171e-05, "loss": 0.0007967445098725792, "grad_norm": 0.038885489106178284, "loss_scale": 4.0, "retries": 0, "seconds": 3427.2, "max_memory_gb": 19.16}
|
| 273 |
+
{"step": 273, "steps": 500, "lr": 1.0125113119895709e-05, "loss": 0.007776741186153852, "grad_norm": 0.4712148606777191, "loss_scale": 4.0, "retries": 0, "seconds": 3439.9, "max_memory_gb": 19.16}
|
| 274 |
+
{"step": 274, "steps": 500, "lr": 1.0066990585280703e-05, "loss": 0.0012956814889031939, "grad_norm": 0.05682696774601936, "loss_scale": 4.0, "retries": 0, "seconds": 3452.3, "max_memory_gb": 19.16}
|
| 275 |
+
{"step": 275, "steps": 500, "lr": 1.0008907359803967e-05, "loss": 0.056652042717794095, "grad_norm": 3.4593005180358887, "loss_scale": 4.0, "retries": 0, "seconds": 3464.9, "max_memory_gb": 19.16}
|
| 276 |
+
{"step": 276, "steps": 500, "lr": 9.950865890602168e-06, "loss": 0.00047169553561781703, "grad_norm": 0.020911937579512596, "loss_scale": 4.0, "retries": 0, "seconds": 3477.2, "max_memory_gb": 19.16}
|
| 277 |
+
{"step": 277, "steps": 500, "lr": 9.892868623052698e-06, "loss": 0.0010465722430436392, "grad_norm": 0.08909854292869568, "loss_scale": 4.0, "retries": 0, "seconds": 3489.7, "max_memory_gb": 19.16}
|
| 278 |
+
{"step": 278, "steps": 500, "lr": 9.834918000670681e-06, "loss": 0.06998033142674455, "grad_norm": 2.2065839767456055, "loss_scale": 4.0, "retries": 0, "seconds": 3502.4, "max_memory_gb": 19.16}
|
| 279 |
+
{"step": 279, "steps": 500, "lr": 9.777016465006e-06, "loss": 0.0001626044033648875, "grad_norm": 0.005732685327529907, "loss_scale": 4.0, "retries": 0, "seconds": 3515.0, "max_memory_gb": 19.16}
|
| 280 |
+
{"step": 280, "steps": 500, "lr": 9.719166455540433e-06, "loss": 0.13350524797098473, "grad_norm": 2.473573923110962, "loss_scale": 4.0, "retries": 0, "seconds": 3527.6, "max_memory_gb": 19.16}
|
| 281 |
+
{"step": 281, "steps": 500, "lr": 9.661370409584892e-06, "loss": 0.03322600937713105, "grad_norm": 1.329595923423767, "loss_scale": 4.0, "retries": 0, "seconds": 3540.2, "max_memory_gb": 19.16}
|
| 282 |
+
{"step": 282, "steps": 500, "lr": 9.603630762176718e-06, "loss": 0.08875684590950848, "grad_norm": 3.406735897064209, "loss_scale": 4.0, "retries": 0, "seconds": 3552.9, "max_memory_gb": 19.16}
|
| 283 |
+
{"step": 283, "steps": 500, "lr": 9.54594994597708e-06, "loss": 0.003574748422558762, "grad_norm": 0.13860517740249634, "loss_scale": 4.0, "retries": 0, "seconds": 3565.5, "max_memory_gb": 19.16}
|
| 284 |
+
{"step": 284, "steps": 500, "lr": 9.488330391168515e-06, "loss": 0.019122873558881218, "grad_norm": 1.24374258518219, "loss_scale": 4.0, "retries": 0, "seconds": 3578.2, "max_memory_gb": 19.16}
|
| 285 |
+
{"step": 285, "steps": 500, "lr": 9.430774525352518e-06, "loss": 0.0010922777560153918, "grad_norm": 0.04787156358361244, "loss_scale": 4.0, "retries": 0, "seconds": 3591.0, "max_memory_gb": 19.16}
|
| 286 |
+
{"step": 286, "steps": 500, "lr": 9.373284773447258e-06, "loss": 0.001197592116320514, "grad_norm": 0.0547746941447258, "loss_scale": 4.0, "retries": 0, "seconds": 3603.6, "max_memory_gb": 19.16}
|
| 287 |
+
{"step": 287, "steps": 500, "lr": 9.315863557585435e-06, "loss": 0.09986017688168598, "grad_norm": 3.546703338623047, "loss_scale": 4.0, "retries": 0, "seconds": 3616.2, "max_memory_gb": 19.16}
|
| 288 |
+
{"step": 288, "steps": 500, "lr": 9.258513297012203e-06, "loss": 0.0029356995373358463, "grad_norm": 0.11370750516653061, "loss_scale": 4.0, "retries": 0, "seconds": 3628.9, "max_memory_gb": 19.16}
|
| 289 |
+
{"step": 289, "steps": 500, "lr": 9.201236407983281e-06, "loss": 0.004941365535304696, "grad_norm": 0.3385595381259918, "loss_scale": 4.0, "retries": 0, "seconds": 3641.6, "max_memory_gb": 19.16}
|
| 290 |
+
{"step": 290, "steps": 500, "lr": 9.144035303663109e-06, "loss": 0.009090819119581863, "grad_norm": 1.0005338191986084, "loss_scale": 4.0, "retries": 0, "seconds": 3654.6, "max_memory_gb": 19.16}
|
| 291 |
+
{"step": 291, "steps": 500, "lr": 9.08691239402321e-06, "loss": 0.0003793743161731362, "grad_norm": 0.014053011313080788, "loss_scale": 4.0, "retries": 0, "seconds": 3667.2, "max_memory_gb": 19.16}
|
| 292 |
+
{"step": 292, "steps": 500, "lr": 9.029870085740647e-06, "loss": 0.0022058772358170904, "grad_norm": 0.09234262257814407, "loss_scale": 4.0, "retries": 0, "seconds": 3680.1, "max_memory_gb": 19.16}
|
| 293 |
+
{"step": 293, "steps": 500, "lr": 8.972910782096616e-06, "loss": 0.05826282243460604, "grad_norm": 4.093517780303955, "loss_scale": 4.0, "retries": 0, "seconds": 3693.0, "max_memory_gb": 19.16}
|
| 294 |
+
{"step": 294, "steps": 500, "lr": 8.916036882875198e-06, "loss": 0.00039438157109440334, "grad_norm": 0.01643233560025692, "loss_scale": 4.0, "retries": 0, "seconds": 3705.6, "max_memory_gb": 19.16}
|
| 295 |
+
{"step": 295, "steps": 500, "lr": 8.859250784262256e-06, "loss": 0.0011189305652537357, "grad_norm": 0.0632515400648117, "loss_scale": 4.0, "retries": 0, "seconds": 3718.2, "max_memory_gb": 19.16}
|
| 296 |
+
{"step": 296, "steps": 500, "lr": 8.802554878744479e-06, "loss": 0.14724339803438563, "grad_norm": 2.617766857147217, "loss_scale": 4.0, "retries": 0, "seconds": 3730.6, "max_memory_gb": 19.16}
|
| 297 |
+
{"step": 297, "steps": 500, "lr": 8.74595155500857e-06, "loss": 0.045030529020082935, "grad_norm": 1.2365998029708862, "loss_scale": 4.0, "retries": 0, "seconds": 3743.2, "max_memory_gb": 19.16}
|
| 298 |
+
{"step": 298, "steps": 500, "lr": 8.689443197840635e-06, "loss": 0.00015450206123804574, "grad_norm": 0.006193649955093861, "loss_scale": 4.0, "retries": 0, "seconds": 3756.2, "max_memory_gb": 19.16}
|
| 299 |
+
{"step": 299, "steps": 500, "lr": 8.633032188025669e-06, "loss": 0.0007754712136871109, "grad_norm": 0.06943564116954803, "loss_scale": 4.0, "retries": 0, "seconds": 3768.7, "max_memory_gb": 19.16}
|
| 300 |
+
{"step": 300, "steps": 500, "lr": 8.576720902247288e-06, "loss": 0.09398975484895479, "grad_norm": 4.781513690948486, "loss_scale": 4.0, "retries": 0, "seconds": 3781.1, "max_memory_gb": 19.16}
|
| 301 |
+
{"step": 301, "steps": 500, "lr": 8.52051171298758e-06, "loss": 0.019030062609793032, "grad_norm": 0.7006915211677551, "loss_scale": 4.0, "retries": 0, "seconds": 3793.5, "max_memory_gb": 19.16}
|
| 302 |
+
{"step": 302, "steps": 500, "lr": 8.464406988427132e-06, "loss": 0.03861162022977541, "grad_norm": 1.8240753412246704, "loss_scale": 4.0, "retries": 0, "seconds": 3806.0, "max_memory_gb": 19.16}
|
| 303 |
+
{"step": 303, "steps": 500, "lr": 8.40840909234529e-06, "loss": 0.0014955322551693162, "grad_norm": 0.0774783343076706, "loss_scale": 4.0, "retries": 0, "seconds": 3818.4, "max_memory_gb": 19.16}
|
| 304 |
+
{"step": 304, "steps": 500, "lr": 8.352520384020533e-06, "loss": 0.003683489004370958, "grad_norm": 0.1621408611536026, "loss_scale": 4.0, "retries": 0, "seconds": 3830.8, "max_memory_gb": 19.16}
|
| 305 |
+
{"step": 305, "steps": 500, "lr": 8.296743218131103e-06, "loss": 0.006149945137332742, "grad_norm": 0.6775586009025574, "loss_scale": 4.0, "retries": 0, "seconds": 3843.0, "max_memory_gb": 19.16}
|
| 306 |
+
{"step": 306, "steps": 500, "lr": 8.241079944655777e-06, "loss": 0.00022774764292163047, "grad_norm": 0.008926430717110634, "loss_scale": 4.0, "retries": 0, "seconds": 3855.0, "max_memory_gb": 19.16}
|
| 307 |
+
{"step": 307, "steps": 500, "lr": 8.185532908774872e-06, "loss": 0.0005580129834470428, "grad_norm": 0.024442050606012344, "loss_scale": 4.0, "retries": 0, "seconds": 3867.3, "max_memory_gb": 19.16}
|
| 308 |
+
{"step": 308, "steps": 500, "lr": 8.130104450771434e-06, "loss": 0.0011434059886141767, "grad_norm": 0.05166007578372955, "loss_scale": 4.0, "retries": 0, "seconds": 3879.9, "max_memory_gb": 19.16}
|
| 309 |
+
{"step": 309, "steps": 500, "lr": 8.074796905932634e-06, "loss": 0.00017973696970908685, "grad_norm": 0.009203780442476273, "loss_scale": 4.0, "retries": 0, "seconds": 3892.3, "max_memory_gb": 19.16}
|
| 310 |
+
{"step": 310, "steps": 500, "lr": 8.019612604451395e-06, "loss": 0.0016824830164833575, "grad_norm": 0.08425532281398773, "loss_scale": 4.0, "retries": 0, "seconds": 3904.5, "max_memory_gb": 19.16}
|
| 311 |
+
{"step": 311, "steps": 500, "lr": 7.964553871328185e-06, "loss": 0.2202799927746666, "grad_norm": 3.2226295471191406, "loss_scale": 4.0, "retries": 0, "seconds": 3916.7, "max_memory_gb": 19.16}
|
| 312 |
+
{"step": 312, "steps": 500, "lr": 7.909623026273106e-06, "loss": 0.0012258958640710596, "grad_norm": 0.08666382730007172, "loss_scale": 4.0, "retries": 0, "seconds": 3929.2, "max_memory_gb": 19.16}
|
| 313 |
+
{"step": 313, "steps": 500, "lr": 7.854822383608116e-06, "loss": 0.00045128865539564606, "grad_norm": 0.02203456684947014, "loss_scale": 4.0, "retries": 0, "seconds": 3941.0, "max_memory_gb": 19.16}
|
| 314 |
+
{"step": 314, "steps": 500, "lr": 7.800154252169558e-06, "loss": 0.007198288616192175, "grad_norm": 0.37400224804878235, "loss_scale": 4.0, "retries": 0, "seconds": 3953.3, "max_memory_gb": 19.16}
|
| 315 |
+
{"step": 315, "steps": 500, "lr": 7.745620935210861e-06, "loss": 0.0002851408128725036, "grad_norm": 0.010604113340377808, "loss_scale": 4.0, "retries": 0, "seconds": 3965.6, "max_memory_gb": 19.16}
|
| 316 |
+
{"step": 316, "steps": 500, "lr": 7.691224730305518e-06, "loss": 0.2320547811007559, "grad_norm": 6.883956432342529, "loss_scale": 4.0, "retries": 0, "seconds": 3977.6, "max_memory_gb": 19.16}
|
| 317 |
+
{"step": 317, "steps": 500, "lr": 7.636967929250276e-06, "loss": 0.18424554282637473, "grad_norm": 6.3328704833984375, "loss_scale": 4.0, "retries": 0, "seconds": 3990.0, "max_memory_gb": 19.16}
|
| 318 |
+
{"step": 318, "steps": 500, "lr": 7.5828528179685755e-06, "loss": 0.001462603066370734, "grad_norm": 0.059794869273900986, "loss_scale": 4.0, "retries": 0, "seconds": 4002.5, "max_memory_gb": 19.16}
|
| 319 |
+
{"step": 319, "steps": 500, "lr": 7.528881676414252e-06, "loss": 0.0007502905820873906, "grad_norm": 0.05321456119418144, "loss_scale": 4.0, "retries": 0, "seconds": 4014.7, "max_memory_gb": 19.16}
|
| 320 |
+
{"step": 320, "steps": 500, "lr": 7.475056778475468e-06, "loss": 0.019025936614711594, "grad_norm": 0.7035971879959106, "loss_scale": 4.0, "retries": 0, "seconds": 4026.5, "max_memory_gb": 19.16}
|
| 321 |
+
{"step": 321, "steps": 500, "lr": 7.421380391878924e-06, "loss": 0.0010172240829226873, "grad_norm": 0.0628204345703125, "loss_scale": 4.0, "retries": 0, "seconds": 4038.7, "max_memory_gb": 19.16}
|
| 322 |
+
{"step": 322, "steps": 500, "lr": 7.367854778094291e-06, "loss": 0.0008389166169319072, "grad_norm": 0.03872298449277878, "loss_scale": 4.0, "retries": 0, "seconds": 4050.7, "max_memory_gb": 19.16}
|
| 323 |
+
{"step": 323, "steps": 500, "lr": 7.314482192238962e-06, "loss": 0.00017715977565496195, "grad_norm": 0.0077464100904762745, "loss_scale": 4.0, "retries": 0, "seconds": 4063.1, "max_memory_gb": 19.16}
|
| 324 |
+
{"step": 324, "steps": 500, "lr": 7.261264882983024e-06, "loss": 8.798185745655993e-05, "grad_norm": 0.003915536683052778, "loss_scale": 4.0, "retries": 0, "seconds": 4075.4, "max_memory_gb": 19.16}
|
| 325 |
+
{"step": 325, "steps": 500, "lr": 7.208205092454502e-06, "loss": 0.0021580556715641505, "grad_norm": 0.10994229465723038, "loss_scale": 4.0, "retries": 0, "seconds": 4087.4, "max_memory_gb": 19.16}
|
| 326 |
+
{"step": 326, "steps": 500, "lr": 7.155305056144935e-06, "loss": 0.003624316547602162, "grad_norm": 0.28592929244041443, "loss_scale": 4.0, "retries": 0, "seconds": 4099.7, "max_memory_gb": 19.16}
|
| 327 |
+
{"step": 327, "steps": 500, "lr": 7.102567002815155e-06, "loss": 0.0003473411583482547, "grad_norm": 0.01586746610701084, "loss_scale": 4.0, "retries": 0, "seconds": 4112.1, "max_memory_gb": 19.16}
|
| 328 |
+
{"step": 328, "steps": 500, "lr": 7.0499931544013985e-06, "loss": 0.18094335572506925, "grad_norm": 2.4887235164642334, "loss_scale": 4.0, "retries": 0, "seconds": 4124.7, "max_memory_gb": 19.16}
|
| 329 |
+
{"step": 329, "steps": 500, "lr": 6.997585725921701e-06, "loss": 0.00046548144060083854, "grad_norm": 0.018354911357164383, "loss_scale": 4.0, "retries": 0, "seconds": 4137.0, "max_memory_gb": 19.16}
|
| 330 |
+
{"step": 330, "steps": 500, "lr": 6.94534692538256e-06, "loss": 0.003713219024534324, "grad_norm": 0.15043435990810394, "loss_scale": 4.0, "retries": 0, "seconds": 4149.6, "max_memory_gb": 19.16}
|
| 331 |
+
{"step": 331, "steps": 500, "lr": 6.893278953685919e-06, "loss": 0.009980189333909742, "grad_norm": 1.2808014154434204, "loss_scale": 4.0, "retries": 0, "seconds": 4162.1, "max_memory_gb": 19.16}
|
| 332 |
+
{"step": 332, "steps": 500, "lr": 6.841384004536434e-06, "loss": 0.0036494139512435386, "grad_norm": 0.16019704937934875, "loss_scale": 4.0, "retries": 0, "seconds": 4174.7, "max_memory_gb": 19.16}
|
| 333 |
+
{"step": 333, "steps": 500, "lr": 6.789664264349063e-06, "loss": 0.0007319736652623926, "grad_norm": 0.029109826311469078, "loss_scale": 4.0, "retries": 0, "seconds": 4187.0, "max_memory_gb": 19.16}
|
| 334 |
+
{"step": 334, "steps": 500, "lr": 6.738121912156913e-06, "loss": 0.0007069415632527539, "grad_norm": 0.031431276351213455, "loss_scale": 4.0, "retries": 0, "seconds": 4199.5, "max_memory_gb": 19.16}
|
| 335 |
+
{"step": 335, "steps": 500, "lr": 6.686759119519493e-06, "loss": 0.00015700635835649086, "grad_norm": 0.007923110388219357, "loss_scale": 4.0, "retries": 0, "seconds": 4211.8, "max_memory_gb": 19.16}
|
| 336 |
+
{"step": 336, "steps": 500, "lr": 6.63557805043115e-06, "loss": 0.0004278885191152685, "grad_norm": 0.015446571633219719, "loss_scale": 4.0, "retries": 0, "seconds": 4224.1, "max_memory_gb": 19.16}
|
| 337 |
+
{"step": 337, "steps": 500, "lr": 6.5845808612299715e-06, "loss": 0.20253631467426206, "grad_norm": 6.585240364074707, "loss_scale": 4.0, "retries": 0, "seconds": 4236.7, "max_memory_gb": 19.16}
|
| 338 |
+
{"step": 338, "steps": 500, "lr": 6.5337697005068745e-06, "loss": 0.01651747856325611, "grad_norm": 0.9524202346801758, "loss_scale": 4.0, "retries": 0, "seconds": 4249.0, "max_memory_gb": 19.16}
|
| 339 |
+
{"step": 339, "steps": 500, "lr": 6.48314670901512e-06, "loss": 0.09531598710083763, "grad_norm": 1.9820928573608398, "loss_scale": 4.0, "retries": 0, "seconds": 4261.3, "max_memory_gb": 19.16}
|
| 340 |
+
{"step": 340, "steps": 500, "lr": 6.4327140195801e-06, "loss": 0.0012395682531612033, "grad_norm": 0.0998372882604599, "loss_scale": 4.0, "retries": 0, "seconds": 4273.5, "max_memory_gb": 19.16}
|
| 341 |
+
{"step": 341, "steps": 500, "lr": 6.382473757009482e-06, "loss": 0.00014523615976713702, "grad_norm": 0.005677056964486837, "loss_scale": 4.0, "retries": 0, "seconds": 4285.8, "max_memory_gb": 19.16}
|
| 342 |
+
{"step": 342, "steps": 500, "lr": 6.332428038003706e-06, "loss": 0.00012452088625474806, "grad_norm": 0.005162009969353676, "loss_scale": 4.0, "retries": 0, "seconds": 4298.1, "max_memory_gb": 19.16}
|
| 343 |
+
{"step": 343, "steps": 500, "lr": 6.282578971066761e-06, "loss": 0.06932404581572303, "grad_norm": 2.3560328483581543, "loss_scale": 4.0, "retries": 0, "seconds": 4310.5, "max_memory_gb": 19.16}
|
| 344 |
+
{"step": 344, "steps": 500, "lr": 6.232928656417414e-06, "loss": 0.00011500940127007199, "grad_norm": 0.005542322993278503, "loss_scale": 4.0, "retries": 0, "seconds": 4322.8, "max_memory_gb": 19.16}
|
| 345 |
+
{"step": 345, "steps": 500, "lr": 6.183479185900656e-06, "loss": 0.11368039023173539, "grad_norm": 4.569474697113037, "loss_scale": 4.0, "retries": 0, "seconds": 4335.1, "max_memory_gb": 19.16}
|
| 346 |
+
{"step": 346, "steps": 500, "lr": 6.134232642899625e-06, "loss": 0.00806146514540532, "grad_norm": 0.49225693941116333, "loss_scale": 4.0, "retries": 0, "seconds": 4347.4, "max_memory_gb": 19.16}
|
| 347 |
+
{"step": 347, "steps": 500, "lr": 6.085191102247789e-06, "loss": 8.032101134796738e-05, "grad_norm": 0.0037922831252217293, "loss_scale": 4.0, "retries": 0, "seconds": 4359.9, "max_memory_gb": 19.16}
|
| 348 |
+
{"step": 348, "steps": 500, "lr": 6.036356630141565e-06, "loss": 0.0036527031810774956, "grad_norm": 0.1589166224002838, "loss_scale": 4.0, "retries": 0, "seconds": 4371.9, "max_memory_gb": 19.16}
|
| 349 |
+
{"step": 349, "steps": 500, "lr": 5.987731284053239e-06, "loss": 0.08615136777825683, "grad_norm": 2.948496103286743, "loss_scale": 4.0, "retries": 0, "seconds": 4384.2, "max_memory_gb": 19.16}
|
| 350 |
+
{"step": 350, "steps": 500, "lr": 5.9393171126443e-06, "loss": 9.148736993645139e-05, "grad_norm": 0.0034884114284068346, "loss_scale": 4.0, "retries": 0, "seconds": 4396.7, "max_memory_gb": 19.16}
|
| 351 |
+
{"step": 351, "steps": 500, "lr": 5.89111615567911e-06, "loss": 0.0009943516781525182, "grad_norm": 0.051536448299884796, "loss_scale": 4.0, "retries": 0, "seconds": 4409.0, "max_memory_gb": 19.16}
|
| 352 |
+
{"step": 352, "steps": 500, "lr": 5.843130443938982e-06, "loss": 7.414488665476298e-05, "grad_norm": 0.0028014786075800657, "loss_scale": 4.0, "retries": 0, "seconds": 4421.4, "max_memory_gb": 19.16}
|
| 353 |
+
{"step": 353, "steps": 500, "lr": 5.795361999136612e-06, "loss": 0.004085043504039216, "grad_norm": 0.13024139404296875, "loss_scale": 4.0, "retries": 0, "seconds": 4433.7, "max_memory_gb": 19.16}
|
| 354 |
+
{"step": 354, "steps": 500, "lr": 5.747812833830894e-06, "loss": 8.455638067061956e-05, "grad_norm": 0.003989525139331818, "loss_scale": 4.0, "retries": 0, "seconds": 4446.0, "max_memory_gb": 19.16}
|
| 355 |
+
{"step": 355, "steps": 500, "lr": 5.700484951342151e-06, "loss": 0.0028465278568525973, "grad_norm": 0.19685086607933044, "loss_scale": 4.0, "retries": 0, "seconds": 4458.5, "max_memory_gb": 19.16}
|
| 356 |
+
{"step": 356, "steps": 500, "lr": 5.6533803456677035e-06, "loss": 0.0014042308224175445, "grad_norm": 0.0613064244389534, "loss_scale": 4.0, "retries": 0, "seconds": 4470.9, "max_memory_gb": 19.16}
|
| 357 |
+
{"step": 357, "steps": 500, "lr": 5.606501001397878e-06, "loss": 0.0003466499336106856, "grad_norm": 0.015487834811210632, "loss_scale": 4.0, "retries": 0, "seconds": 4483.1, "max_memory_gb": 19.16}
|
| 358 |
+
{"step": 358, "steps": 500, "lr": 5.559848893632384e-06, "loss": 0.018684413964282692, "grad_norm": 0.5729737281799316, "loss_scale": 4.0, "retries": 0, "seconds": 4495.4, "max_memory_gb": 19.16}
|
| 359 |
+
{"step": 359, "steps": 500, "lr": 5.513425987897107e-06, "loss": 0.1621259255891387, "grad_norm": 4.466996192932129, "loss_scale": 4.0, "retries": 0, "seconds": 4508.0, "max_memory_gb": 19.16}
|
| 360 |
+
{"step": 360, "steps": 500, "lr": 5.467234240061293e-06, "loss": 0.04820351093235332, "grad_norm": 2.4621996879577637, "loss_scale": 4.0, "retries": 0, "seconds": 4520.3, "max_memory_gb": 19.16}
|
| 361 |
+
{"step": 361, "steps": 500, "lr": 5.421275596255145e-06, "loss": 0.01940415656583189, "grad_norm": 1.6296817064285278, "loss_scale": 4.0, "retries": 0, "seconds": 4532.8, "max_memory_gb": 19.16}
|
| 362 |
+
{"step": 362, "steps": 500, "lr": 5.375551992787835e-06, "loss": 0.0030985350165131176, "grad_norm": 0.1759425401687622, "loss_scale": 4.0, "retries": 0, "seconds": 4545.1, "max_memory_gb": 19.16}
|
| 363 |
+
{"step": 363, "steps": 500, "lr": 5.330065356065913e-06, "loss": 0.0008315693841247844, "grad_norm": 0.03893636167049408, "loss_scale": 4.0, "retries": 0, "seconds": 4557.4, "max_memory_gb": 19.16}
|
| 364 |
+
{"step": 364, "steps": 500, "lr": 5.284817602512153e-06, "loss": 0.1379382352527149, "grad_norm": 1.7052942514419556, "loss_scale": 4.0, "retries": 0, "seconds": 4569.6, "max_memory_gb": 19.16}
|
| 365 |
+
{"step": 365, "steps": 500, "lr": 5.239810638484819e-06, "loss": 0.19896103068472049, "grad_norm": 2.107457160949707, "loss_scale": 4.0, "retries": 0, "seconds": 4582.3, "max_memory_gb": 19.16}
|
| 366 |
+
{"step": 366, "steps": 500, "lr": 5.195046360197325e-06, "loss": 0.10417209255174179, "grad_norm": 11.286628723144531, "loss_scale": 4.0, "retries": 0, "seconds": 4594.6, "max_memory_gb": 19.16}
|
| 367 |
+
{"step": 367, "steps": 500, "lr": 5.150526653638363e-06, "loss": 0.0023698343239185633, "grad_norm": 0.12136298418045044, "loss_scale": 4.0, "retries": 0, "seconds": 4606.9, "max_memory_gb": 19.16}
|
| 368 |
+
{"step": 368, "steps": 500, "lr": 5.106253394492435e-06, "loss": 0.026221286689767774, "grad_norm": 0.7769099473953247, "loss_scale": 4.0, "retries": 0, "seconds": 4619.9, "max_memory_gb": 19.16}
|
| 369 |
+
{"step": 369, "steps": 500, "lr": 5.062228448060833e-06, "loss": 0.0005493059341729763, "grad_norm": 0.027811385691165924, "loss_scale": 4.0, "retries": 0, "seconds": 4632.6, "max_memory_gb": 19.16}
|
| 370 |
+
{"step": 370, "steps": 500, "lr": 5.018453669183048e-06, "loss": 0.00016384932630651087, "grad_norm": 0.0076027121394872665, "loss_scale": 4.0, "retries": 0, "seconds": 4645.0, "max_memory_gb": 19.16}
|
| 371 |
+
{"step": 371, "steps": 500, "lr": 4.974930902158616e-06, "loss": 0.29859274645749107, "grad_norm": 5.071730136871338, "loss_scale": 4.0, "retries": 0, "seconds": 4657.7, "max_memory_gb": 19.16}
|
| 372 |
+
{"step": 372, "steps": 500, "lr": 4.931661980669439e-06, "loss": 0.0009729199586093852, "grad_norm": 0.06940754503011703, "loss_scale": 4.0, "retries": 0, "seconds": 4670.4, "max_memory_gb": 19.16}
|
| 373 |
+
{"step": 373, "steps": 500, "lr": 4.8886487277024854e-06, "loss": 0.00024041520801306015, "grad_norm": 0.010912906378507614, "loss_scale": 4.0, "retries": 0, "seconds": 4683.7, "max_memory_gb": 19.16}
|
| 374 |
+
{"step": 374, "steps": 500, "lr": 4.845892955473037e-06, "loss": 0.002098004602213554, "grad_norm": 0.11801932007074356, "loss_scale": 4.0, "retries": 0, "seconds": 4696.1, "max_memory_gb": 19.16}
|
| 375 |
+
{"step": 375, "steps": 500, "lr": 4.80339646534829e-06, "loss": 0.013155777758573675, "grad_norm": 0.5251166820526123, "loss_scale": 4.0, "retries": 0, "seconds": 4708.5, "max_memory_gb": 19.16}
|
| 376 |
+
{"step": 376, "steps": 500, "lr": 4.761161047771502e-06, "loss": 0.017348555433169502, "grad_norm": 0.8344725370407104, "loss_scale": 4.0, "retries": 0, "seconds": 4720.9, "max_memory_gb": 19.16}
|
| 377 |
+
{"step": 377, "steps": 500, "lr": 4.719188482186515e-06, "loss": 0.00016326992135614916, "grad_norm": 0.007397943641990423, "loss_scale": 4.0, "retries": 0, "seconds": 4733.6, "max_memory_gb": 19.16}
|
| 378 |
+
{"step": 378, "steps": 500, "lr": 4.677480536962832e-06, "loss": 0.00011624985122890314, "grad_norm": 0.005277517717331648, "loss_scale": 4.0, "retries": 0, "seconds": 4746.1, "max_memory_gb": 19.16}
|
| 379 |
+
{"step": 379, "steps": 500, "lr": 4.6360389693210725e-06, "loss": 0.06605497611426678, "grad_norm": 1.9242485761642456, "loss_scale": 4.0, "retries": 0, "seconds": 4758.6, "max_memory_gb": 19.16}
|
| 380 |
+
{"step": 380, "steps": 500, "lr": 4.594865525258959e-06, "loss": 0.0007708051456347678, "grad_norm": 0.030460722744464874, "loss_scale": 4.0, "retries": 0, "seconds": 4771.4, "max_memory_gb": 19.16}
|
| 381 |
+
{"step": 381, "steps": 500, "lr": 4.5539619394777495e-06, "loss": 0.18886044278083736, "grad_norm": 5.047060012817383, "loss_scale": 4.0, "retries": 0, "seconds": 4784.0, "max_memory_gb": 19.16}
|
| 382 |
+
{"step": 382, "steps": 500, "lr": 4.5133299353091496e-06, "loss": 0.0006306594890371674, "grad_norm": 0.027732253074645996, "loss_scale": 4.0, "retries": 0, "seconds": 4796.6, "max_memory_gb": 19.16}
|
| 383 |
+
{"step": 383, "steps": 500, "lr": 4.472971224642723e-06, "loss": 0.0005488920471841333, "grad_norm": 0.027126001194119453, "loss_scale": 4.0, "retries": 0, "seconds": 4809.2, "max_memory_gb": 19.16}
|
| 384 |
+
{"step": 384, "steps": 500, "lr": 4.43288750785373e-06, "loss": 0.0014876006768893355, "grad_norm": 0.0634305328130722, "loss_scale": 4.0, "retries": 0, "seconds": 4821.7, "max_memory_gb": 19.16}
|
| 385 |
+
{"step": 385, "steps": 500, "lr": 4.393080473731534e-06, "loss": 0.00019422769335619705, "grad_norm": 0.008490351028740406, "loss_scale": 4.0, "retries": 0, "seconds": 4834.2, "max_memory_gb": 19.16}
|
| 386 |
+
{"step": 386, "steps": 500, "lr": 4.353551799408406e-06, "loss": 0.0005717329867862151, "grad_norm": 0.02161582186818123, "loss_scale": 4.0, "retries": 0, "seconds": 4847.2, "max_memory_gb": 19.16}
|
| 387 |
+
{"step": 387, "steps": 500, "lr": 4.314303150288909e-06, "loss": 0.028463635192524223, "grad_norm": 0.8400639891624451, "loss_scale": 4.0, "retries": 0, "seconds": 4859.7, "max_memory_gb": 19.16}
|
| 388 |
+
{"step": 388, "steps": 500, "lr": 4.275336179979683e-06, "loss": 0.005308518890741837, "grad_norm": 0.30515098571777344, "loss_scale": 4.0, "retries": 0, "seconds": 4872.2, "max_memory_gb": 19.16}
|
| 389 |
+
{"step": 389, "steps": 500, "lr": 4.236652530219822e-06, "loss": 0.00019270581091875144, "grad_norm": 0.009549099020659924, "loss_scale": 4.0, "retries": 0, "seconds": 4884.8, "max_memory_gb": 19.16}
|
| 390 |
+
{"step": 390, "steps": 500, "lr": 4.198253830811677e-06, "loss": 0.23647707214103203, "grad_norm": 5.206164360046387, "loss_scale": 4.0, "retries": 0, "seconds": 4897.3, "max_memory_gb": 19.16}
|
| 391 |
+
{"step": 391, "steps": 500, "lr": 4.160141699552193e-06, "loss": 0.0008029658780825147, "grad_norm": 0.04129781574010849, "loss_scale": 4.0, "retries": 0, "seconds": 4910.0, "max_memory_gb": 19.17}
|
| 392 |
+
{"step": 392, "steps": 500, "lr": 4.122317742164759e-06, "loss": 0.027866450005293787, "grad_norm": 5.771969318389893, "loss_scale": 4.0, "retries": 0, "seconds": 4922.5, "max_memory_gb": 19.17}
|
| 393 |
+
{"step": 393, "steps": 500, "lr": 4.084783552231548e-06, "loss": 0.011841413924938397, "grad_norm": 1.5458953380584717, "loss_scale": 4.0, "retries": 0, "seconds": 4935.1, "max_memory_gb": 19.17}
|
| 394 |
+
{"step": 394, "steps": 500, "lr": 4.047540711126381e-06, "loss": 0.0002282617293965152, "grad_norm": 0.010389527305960655, "loss_scale": 4.0, "retries": 0, "seconds": 4947.7, "max_memory_gb": 19.17}
|
| 395 |
+
{"step": 395, "steps": 500, "lr": 4.010590787948095e-06, "loss": 9.270541481111073e-05, "grad_norm": 0.0037239876110106707, "loss_scale": 4.0, "retries": 0, "seconds": 4960.3, "max_memory_gb": 19.17}
|
| 396 |
+
{"step": 396, "steps": 500, "lr": 3.973935339454447e-06, "loss": 0.0001341605516322695, "grad_norm": 0.006181060336530209, "loss_scale": 4.0, "retries": 0, "seconds": 4973.1, "max_memory_gb": 19.17}
|
| 397 |
+
{"step": 397, "steps": 500, "lr": 3.9375759099965105e-06, "loss": 0.0034997519393920647, "grad_norm": 0.16308438777923584, "loss_scale": 4.0, "retries": 0, "seconds": 4985.6, "max_memory_gb": 19.17}
|
| 398 |
+
{"step": 398, "steps": 500, "lr": 3.901514031453619e-06, "loss": 0.0004892917049956354, "grad_norm": 0.02331484854221344, "loss_scale": 4.0, "retries": 0, "seconds": 5000.8, "max_memory_gb": 19.17}
|
| 399 |
+
{"step": 399, "steps": 500, "lr": 3.865751223168823e-06, "loss": 0.002433067878453957, "grad_norm": 0.11774349212646484, "loss_scale": 4.0, "retries": 0, "seconds": 5014.5, "max_memory_gb": 19.17}
|
| 400 |
+
{"step": 400, "steps": 500, "lr": 3.830288991884876e-06, "loss": 0.00011524208991757364, "grad_norm": 0.005277537275105715, "loss_scale": 4.0, "retries": 0, "seconds": 5028.4, "max_memory_gb": 19.17}
|
| 401 |
+
{"step": 401, "steps": 500, "lr": 3.7951288316807574e-06, "loss": 0.00022670817142770005, "grad_norm": 0.013286967761814594, "loss_scale": 4.0, "retries": 0, "seconds": 5042.0, "max_memory_gb": 19.17}
|
| 402 |
+
{"step": 402, "steps": 500, "lr": 3.76027222390872e-06, "loss": 0.001694208281254429, "grad_norm": 0.06475505977869034, "loss_scale": 4.0, "retries": 0, "seconds": 5056.4, "max_memory_gb": 19.17}
|
| 403 |
+
{"step": 403, "steps": 500, "lr": 3.725720637131883e-06, "loss": 0.012246507792507477, "grad_norm": 0.40647801756858826, "loss_scale": 4.0, "retries": 0, "seconds": 5070.2, "max_memory_gb": 19.17}
|
| 404 |
+
{"step": 404, "steps": 500, "lr": 3.6914755270623518e-06, "loss": 0.04010583492814393, "grad_norm": 1.1061969995498657, "loss_scale": 4.0, "retries": 0, "seconds": 5083.9, "max_memory_gb": 19.17}
|
| 405 |
+
{"step": 405, "steps": 500, "lr": 3.6575383364998937e-06, "loss": 0.001972155537910939, "grad_norm": 0.07515816390514374, "loss_scale": 4.0, "retries": 0, "seconds": 5098.3, "max_memory_gb": 19.17}
|
| 406 |
+
{"step": 406, "steps": 500, "lr": 3.623910495271154e-06, "loss": 0.0010833552221485704, "grad_norm": 0.11062446236610413, "loss_scale": 4.0, "retries": 0, "seconds": 5112.5, "max_memory_gb": 19.17}
|
| 407 |
+
{"step": 407, "steps": 500, "lr": 3.5905934201693992e-06, "loss": 0.017165382288236615, "grad_norm": 0.7474635243415833, "loss_scale": 4.0, "retries": 0, "seconds": 5125.7, "max_memory_gb": 19.17}
|
| 408 |
+
{"step": 408, "steps": 500, "lr": 3.557588514894842e-06, "loss": 0.009050041756061299, "grad_norm": 0.42637190222740173, "loss_scale": 4.0, "retries": 0, "seconds": 5139.4, "max_memory_gb": 19.17}
|
| 409 |
+
{"step": 409, "steps": 500, "lr": 3.524897169995492e-06, "loss": 0.00044874873669507247, "grad_norm": 0.030392132699489594, "loss_scale": 4.0, "retries": 0, "seconds": 5153.0, "max_memory_gb": 19.17}
|
| 410 |
+
{"step": 410, "steps": 500, "lr": 3.492520762808572e-06, "loss": 0.0013048730473263445, "grad_norm": 0.12795132398605347, "loss_scale": 4.0, "retries": 0, "seconds": 5166.5, "max_memory_gb": 19.17}
|
| 411 |
+
{"step": 411, "steps": 500, "lr": 3.4604606574024906e-06, "loss": 0.00011642224662657762, "grad_norm": 0.004410674795508385, "loss_scale": 4.0, "retries": 0, "seconds": 5180.1, "max_memory_gb": 19.17}
|
| 412 |
+
{"step": 412, "steps": 500, "lr": 3.428718204519368e-06, "loss": 0.0003617908816202296, "grad_norm": 0.016240479424595833, "loss_scale": 4.0, "retries": 0, "seconds": 5193.8, "max_memory_gb": 19.17}
|
| 413 |
+
{"step": 413, "steps": 500, "lr": 3.3972947415181376e-06, "loss": 0.044698119511052556, "grad_norm": 1.1593976020812988, "loss_scale": 4.0, "retries": 0, "seconds": 5207.6, "max_memory_gb": 19.17}
|
| 414 |
+
{"step": 414, "steps": 500, "lr": 3.3661915923181757e-06, "loss": 0.0001120163664154461, "grad_norm": 0.008511999621987343, "loss_scale": 4.0, "retries": 0, "seconds": 5222.3, "max_memory_gb": 19.17}
|
| 415 |
+
{"step": 415, "steps": 500, "lr": 3.335410067343559e-06, "loss": 0.0034311091182633972, "grad_norm": 0.14297839999198914, "loss_scale": 4.0, "retries": 0, "seconds": 5236.0, "max_memory_gb": 19.17}
|
| 416 |
+
{"step": 416, "steps": 500, "lr": 3.3049514634678196e-06, "loss": 6.966215528336761e-05, "grad_norm": 0.002966104308143258, "loss_scale": 4.0, "retries": 0, "seconds": 5249.9, "max_memory_gb": 19.17}
|
| 417 |
+
{"step": 417, "steps": 500, "lr": 3.2748170639593357e-06, "loss": 0.0016891980731553635, "grad_norm": 0.19520126283168793, "loss_scale": 4.0, "retries": 0, "seconds": 5263.6, "max_memory_gb": 19.17}
|
| 418 |
+
{"step": 418, "steps": 500, "lr": 3.245008138427235e-06, "loss": 0.007831014622695422, "grad_norm": 0.5746603012084961, "loss_scale": 4.0, "retries": 0, "seconds": 5277.5, "max_memory_gb": 19.17}
|
| 419 |
+
{"step": 419, "steps": 500, "lr": 3.215525942767934e-06, "loss": 0.00019781491361021094, "grad_norm": 0.010579267516732216, "loss_scale": 4.0, "retries": 0, "seconds": 5291.2, "max_memory_gb": 19.17}
|
| 420 |
+
{"step": 420, "steps": 500, "lr": 3.1863717191122057e-06, "loss": 0.5400156412124689, "grad_norm": 6.8561859130859375, "loss_scale": 4.0, "retries": 0, "seconds": 5305.2, "max_memory_gb": 19.17}
|
| 421 |
+
{"step": 421, "steps": 500, "lr": 3.1575466957728514e-06, "loss": 0.023409382353778794, "grad_norm": 0.6690499782562256, "loss_scale": 4.0, "retries": 0, "seconds": 5319.4, "max_memory_gb": 19.17}
|
| 422 |
+
{"step": 422, "steps": 500, "lr": 3.129052087192952e-06, "loss": 9.05950747354919e-05, "grad_norm": 0.004311261232942343, "loss_scale": 4.0, "retries": 0, "seconds": 5333.1, "max_memory_gb": 19.17}
|
| 423 |
+
{"step": 423, "steps": 500, "lr": 3.1008890938946993e-06, "loss": 0.0005073301599364559, "grad_norm": 0.02941230684518814, "loss_scale": 4.0, "retries": 0, "seconds": 5346.8, "max_memory_gb": 19.17}
|
| 424 |
+
{"step": 424, "steps": 500, "lr": 3.0730589024288223e-06, "loss": 6.216656051740509e-05, "grad_norm": 0.0026414503809064627, "loss_scale": 4.0, "retries": 0, "seconds": 5360.4, "max_memory_gb": 19.17}
|
| 425 |
+
{"step": 425, "steps": 500, "lr": 3.0455626853245828e-06, "loss": 0.033580514593128635, "grad_norm": 1.3612784147262573, "loss_scale": 4.0, "retries": 0, "seconds": 5374.2, "max_memory_gb": 19.17}
|
| 426 |
+
{"step": 426, "steps": 500, "lr": 3.0184016010403933e-06, "loss": 0.37921819803909795, "grad_norm": 9.43320083618164, "loss_scale": 4.0, "retries": 0, "seconds": 5388.0, "max_memory_gb": 19.17}
|
| 427 |
+
{"step": 427, "steps": 500, "lr": 2.9915767939149864e-06, "loss": 4.102481042490457e-05, "grad_norm": 0.0018078451976180077, "loss_scale": 4.0, "retries": 0, "seconds": 5401.7, "max_memory_gb": 19.17}
|
| 428 |
+
{"step": 428, "steps": 500, "lr": 2.9650893941192267e-06, "loss": 0.00010783390810202451, "grad_norm": 0.004544247407466173, "loss_scale": 4.0, "retries": 0, "seconds": 5415.7, "max_memory_gb": 19.17}
|
| 429 |
+
{"step": 429, "steps": 500, "lr": 2.9389405176084782e-06, "loss": 0.06365214864131924, "grad_norm": 1.6526484489440918, "loss_scale": 4.0, "retries": 0, "seconds": 5429.4, "max_memory_gb": 19.17}
|
| 430 |
+
{"step": 430, "steps": 500, "lr": 2.9131312660755903e-06, "loss": 0.005882708576719284, "grad_norm": 0.231899693608284, "loss_scale": 4.0, "retries": 0, "seconds": 5443.6, "max_memory_gb": 19.17}
|
| 431 |
+
{"step": 431, "steps": 500, "lr": 2.887662726904487e-06, "loss": 0.0037401834821030825, "grad_norm": 0.2532614469528198, "loss_scale": 4.0, "retries": 0, "seconds": 5457.3, "max_memory_gb": 19.17}
|
| 432 |
+
{"step": 432, "steps": 500, "lr": 2.8625359731243483e-06, "loss": 9.048010379977067e-05, "grad_norm": 0.0042205494828522205, "loss_scale": 4.0, "retries": 0, "seconds": 5471.3, "max_memory_gb": 19.17}
|
| 433 |
+
{"step": 433, "steps": 500, "lr": 2.837752063364405e-06, "loss": 0.056018279360162376, "grad_norm": 1.3281782865524292, "loss_scale": 4.0, "retries": 0, "seconds": 5484.8, "max_memory_gb": 19.17}
|
| 434 |
+
{"step": 434, "steps": 500, "lr": 2.813312041809334e-06, "loss": 5.440794888045276e-05, "grad_norm": 0.002305725123733282, "loss_scale": 4.0, "retries": 0, "seconds": 5498.5, "max_memory_gb": 19.17}
|
| 435 |
+
{"step": 435, "steps": 500, "lr": 2.7892169381552727e-06, "loss": 0.0029307758605845243, "grad_norm": 0.10304230451583862, "loss_scale": 4.0, "retries": 0, "seconds": 5512.7, "max_memory_gb": 19.17}
|
| 436 |
+
{"step": 436, "steps": 500, "lr": 2.7654677675664216e-06, "loss": 0.0009059357177482692, "grad_norm": 0.054590433835983276, "loss_scale": 4.0, "retries": 0, "seconds": 5526.4, "max_memory_gb": 19.17}
|
| 437 |
+
{"step": 437, "steps": 500, "lr": 2.742065530632291e-06, "loss": 0.009878394489775972, "grad_norm": 0.6030979752540588, "loss_scale": 4.0, "retries": 0, "seconds": 5540.7, "max_memory_gb": 19.17}
|
| 438 |
+
{"step": 438, "steps": 500, "lr": 2.719011213325533e-06, "loss": 0.01061752410407218, "grad_norm": 0.481180340051651, "loss_scale": 4.0, "retries": 0, "seconds": 5554.5, "max_memory_gb": 19.17}
|
| 439 |
+
{"step": 439, "steps": 500, "lr": 2.6963057869604053e-06, "loss": 0.018015180835296914, "grad_norm": 2.6129438877105713, "loss_scale": 4.0, "retries": 0, "seconds": 5567.7, "max_memory_gb": 19.17}
|
| 440 |
+
{"step": 440, "steps": 500, "lr": 2.6739502081518463e-06, "loss": 0.0002118246470000817, "grad_norm": 0.010632632300257683, "loss_scale": 4.0, "retries": 0, "seconds": 5581.6, "max_memory_gb": 19.17}
|
| 441 |
+
{"step": 441, "steps": 500, "lr": 2.651945418775174e-06, "loss": 0.00786077602499931, "grad_norm": 0.27253010869026184, "loss_scale": 4.0, "retries": 0, "seconds": 5595.4, "max_memory_gb": 19.17}
|
| 442 |
+
{"step": 442, "steps": 500, "lr": 2.6302923459264005e-06, "loss": 0.03373281412397814, "grad_norm": 3.5118961334228516, "loss_scale": 4.0, "retries": 0, "seconds": 5609.1, "max_memory_gb": 19.17}
|
| 443 |
+
{"step": 443, "steps": 500, "lr": 2.608991901883174e-06, "loss": 0.009489734372979538, "grad_norm": 0.3265901803970337, "loss_scale": 4.0, "retries": 0, "seconds": 5622.9, "max_memory_gb": 19.17}
|
| 444 |
+
{"step": 444, "steps": 500, "lr": 2.588044984066341e-06, "loss": 0.18469125665370714, "grad_norm": 8.518656730651855, "loss_scale": 4.0, "retries": 0, "seconds": 5636.7, "max_memory_gb": 19.17}
|
| 445 |
+
{"step": 445, "steps": 500, "lr": 2.5674524750021456e-06, "loss": 0.0007159348560192444, "grad_norm": 0.02713143825531006, "loss_scale": 4.0, "retries": 0, "seconds": 5650.3, "max_memory_gb": 19.17}
|
| 446 |
+
{"step": 446, "steps": 500, "lr": 2.5472152422850257e-06, "loss": 0.0015022408956817594, "grad_norm": 0.19477081298828125, "loss_scale": 4.0, "retries": 0, "seconds": 5663.9, "max_memory_gb": 19.17}
|
| 447 |
+
{"step": 447, "steps": 500, "lr": 2.5273341385410883e-06, "loss": 0.08109403122385961, "grad_norm": 1.7336889505386353, "loss_scale": 4.0, "retries": 0, "seconds": 5677.9, "max_memory_gb": 19.17}
|
| 448 |
+
{"step": 448, "steps": 500, "lr": 2.507810001392163e-06, "loss": 9.241810371143155e-05, "grad_norm": 0.004546531941741705, "loss_scale": 4.0, "retries": 0, "seconds": 5691.8, "max_memory_gb": 19.17}
|
| 449 |
+
{"step": 449, "steps": 500, "lr": 2.488643653420525e-06, "loss": 0.00185516965591019, "grad_norm": 0.05984044447541237, "loss_scale": 4.0, "retries": 0, "seconds": 5705.9, "max_memory_gb": 19.17}
|
| 450 |
+
{"step": 450, "steps": 500, "lr": 2.4698359021342357e-06, "loss": 0.033190533303261625, "grad_norm": 2.5395562648773193, "loss_scale": 4.0, "retries": 0, "seconds": 5720.2, "max_memory_gb": 19.17}
|
| 451 |
+
{"step": 451, "steps": 500, "lr": 2.451387539933117e-06, "loss": 0.07104482047660875, "grad_norm": 6.272982120513916, "loss_scale": 4.0, "retries": 0, "seconds": 5734.1, "max_memory_gb": 19.17}
|
| 452 |
+
{"step": 452, "steps": 500, "lr": 2.4332993440753743e-06, "loss": 0.0001187254014832268, "grad_norm": 0.005750690121203661, "loss_scale": 4.0, "retries": 0, "seconds": 5748.0, "max_memory_gb": 19.17}
|
| 453 |
+
{"step": 453, "steps": 500, "lr": 2.415572076644838e-06, "loss": 0.005675338537276854, "grad_norm": 0.25066444277763367, "loss_scale": 4.0, "retries": 0, "seconds": 5761.9, "max_memory_gb": 19.17}
|
| 454 |
+
{"step": 454, "steps": 500, "lr": 2.3982064845188708e-06, "loss": 0.025823774191911238, "grad_norm": 0.8870373964309692, "loss_scale": 4.0, "retries": 0, "seconds": 5775.2, "max_memory_gb": 19.17}
|
| 455 |
+
{"step": 455, "steps": 500, "lr": 2.3812032993368828e-06, "loss": 0.00010309478970136547, "grad_norm": 0.004293735604733229, "loss_scale": 4.0, "retries": 0, "seconds": 5789.1, "max_memory_gb": 19.17}
|
| 456 |
+
{"step": 456, "steps": 500, "lr": 2.3645632374695245e-06, "loss": 0.0003700417415550626, "grad_norm": 0.019564151763916016, "loss_scale": 4.0, "retries": 0, "seconds": 5802.9, "max_memory_gb": 19.17}
|
| 457 |
+
{"step": 457, "steps": 500, "lr": 2.3482869999884874e-06, "loss": 0.12734309294021592, "grad_norm": 3.2487895488739014, "loss_scale": 4.0, "retries": 0, "seconds": 5817.0, "max_memory_gb": 19.17}
|
| 458 |
+
{"step": 458, "steps": 500, "lr": 2.332375272636983e-06, "loss": 0.0002442804404009538, "grad_norm": 0.013640064746141434, "loss_scale": 4.0, "retries": 0, "seconds": 5830.8, "max_memory_gb": 19.17}
|
| 459 |
+
{"step": 459, "steps": 500, "lr": 2.316828725800841e-06, "loss": 0.00040954975096693414, "grad_norm": 0.01889805495738983, "loss_scale": 4.0, "retries": 0, "seconds": 5844.4, "max_memory_gb": 19.17}
|
| 460 |
+
{"step": 460, "steps": 500, "lr": 2.301648014480267e-06, "loss": 0.010716437763200953, "grad_norm": 0.7851383090019226, "loss_scale": 4.0, "retries": 0, "seconds": 5858.3, "max_memory_gb": 19.17}
|
| 461 |
+
{"step": 461, "steps": 500, "lr": 2.2868337782622484e-06, "loss": 7.496963443868894e-05, "grad_norm": 0.003166107926517725, "loss_scale": 4.0, "retries": 0, "seconds": 5872.5, "max_memory_gb": 19.17}
|
| 462 |
+
{"step": 462, "steps": 500, "lr": 2.272386641293606e-06, "loss": 0.013121848369890318, "grad_norm": 1.6314365863800049, "loss_scale": 4.0, "retries": 0, "seconds": 5886.3, "max_memory_gb": 19.17}
|
| 463 |
+
{"step": 463, "steps": 500, "lr": 2.258307212254698e-06, "loss": 0.019347859030819918, "grad_norm": 1.0238255262374878, "loss_scale": 4.0, "retries": 0, "seconds": 5900.1, "max_memory_gb": 19.17}
|
| 464 |
+
{"step": 464, "steps": 500, "lr": 2.2445960843337744e-06, "loss": 0.05921736892204102, "grad_norm": 3.7313339710235596, "loss_scale": 4.0, "retries": 0, "seconds": 5913.9, "max_memory_gb": 19.17}
|
| 465 |
+
{"step": 465, "steps": 500, "lr": 2.231253835201989e-06, "loss": 0.0003258359668775897, "grad_norm": 0.012227877043187618, "loss_scale": 4.0, "retries": 0, "seconds": 5927.9, "max_memory_gb": 19.17}
|
| 466 |
+
{"step": 466, "steps": 500, "lr": 2.218281026989053e-06, "loss": 0.06329480615034362, "grad_norm": 2.8076770305633545, "loss_scale": 4.0, "retries": 0, "seconds": 5941.5, "max_memory_gb": 19.17}
|
| 467 |
+
{"step": 467, "steps": 500, "lr": 2.2056782062595655e-06, "loss": 0.00031920185380262086, "grad_norm": 0.016901683062314987, "loss_scale": 4.0, "retries": 0, "seconds": 5955.3, "max_memory_gb": 19.17}
|
| 468 |
+
{"step": 468, "steps": 500, "lr": 2.193445903989966e-06, "loss": 0.006601210228309769, "grad_norm": 0.7672933340072632, "loss_scale": 4.0, "retries": 0, "seconds": 5969.7, "max_memory_gb": 19.17}
|
| 469 |
+
{"step": 469, "steps": 500, "lr": 2.1815846355461853e-06, "loss": 5.178099814238315e-05, "grad_norm": 0.0020908950828015804, "loss_scale": 4.0, "retries": 0, "seconds": 5983.6, "max_memory_gb": 19.17}
|
| 470 |
+
{"step": 470, "steps": 500, "lr": 2.1700949006619163e-06, "loss": 0.00036359388225677827, "grad_norm": 0.015552155673503876, "loss_scale": 4.0, "retries": 0, "seconds": 5997.4, "max_memory_gb": 19.17}
|
| 471 |
+
{"step": 471, "steps": 500, "lr": 2.158977183417566e-06, "loss": 0.003056384985391958, "grad_norm": 0.3168199956417084, "loss_scale": 4.0, "retries": 0, "seconds": 6011.4, "max_memory_gb": 19.17}
|
| 472 |
+
{"step": 472, "steps": 500, "lr": 2.1482319522198587e-06, "loss": 0.001716745514706666, "grad_norm": 0.07894337177276611, "loss_scale": 4.0, "retries": 0, "seconds": 6025.3, "max_memory_gb": 19.17}
|
| 473 |
+
{"step": 473, "steps": 500, "lr": 2.137859659782103e-06, "loss": 0.012281389051206692, "grad_norm": 1.0563141107559204, "loss_scale": 4.0, "retries": 0, "seconds": 6039.2, "max_memory_gb": 19.17}
|
| 474 |
+
{"step": 474, "steps": 500, "lr": 2.1278607431051185e-06, "loss": 5.1627807532771526e-05, "grad_norm": 0.002051991643384099, "loss_scale": 4.0, "retries": 0, "seconds": 6053.3, "max_memory_gb": 19.17}
|
| 475 |
+
{"step": 475, "steps": 500, "lr": 2.1182356234588196e-06, "loss": 0.004307755174739792, "grad_norm": 0.410569965839386, "loss_scale": 4.0, "retries": 0, "seconds": 6067.0, "max_memory_gb": 19.17}
|
| 476 |
+
{"step": 476, "steps": 500, "lr": 2.1089847063644735e-06, "loss": 0.00012144272147261083, "grad_norm": 0.005147796124219894, "loss_scale": 4.0, "retries": 0, "seconds": 6080.9, "max_memory_gb": 19.17}
|
| 477 |
+
{"step": 477, "steps": 500, "lr": 2.100108381577611e-06, "loss": 0.04862067738816801, "grad_norm": 3.0412793159484863, "loss_scale": 4.0, "retries": 0, "seconds": 6094.8, "max_memory_gb": 19.17}
|
| 478 |
+
{"step": 478, "steps": 500, "lr": 2.091607023071606e-06, "loss": 0.0029428106268328106, "grad_norm": 0.41956159472465515, "loss_scale": 4.0, "retries": 0, "seconds": 6108.5, "max_memory_gb": 19.17}
|
| 479 |
+
{"step": 479, "steps": 500, "lr": 2.0834809890219166e-06, "loss": 0.16124056293520797, "grad_norm": 4.618929386138916, "loss_scale": 4.0, "retries": 0, "seconds": 6122.4, "max_memory_gb": 19.17}
|
| 480 |
+
{"step": 480, "steps": 500, "lr": 2.0757306217910033e-06, "loss": 7.988807369553218e-05, "grad_norm": 0.004087595734745264, "loss_scale": 4.0, "retries": 0, "seconds": 6136.2, "max_memory_gb": 19.17}
|
| 481 |
+
{"step": 481, "steps": 500, "lr": 2.068356247913894e-06, "loss": 0.00028909292219674916, "grad_norm": 0.013049381785094738, "loss_scale": 4.0, "retries": 0, "seconds": 6150.0, "max_memory_gb": 19.17}
|
| 482 |
+
{"step": 482, "steps": 500, "lr": 2.0613581780844348e-06, "loss": 0.012734770889478983, "grad_norm": 0.5663409233093262, "loss_scale": 4.0, "retries": 0, "seconds": 6163.8, "max_memory_gb": 19.17}
|
| 483 |
+
{"step": 483, "steps": 500, "lr": 2.0547367071421955e-06, "loss": 0.002607958436364946, "grad_norm": 0.12692683935165405, "loss_scale": 4.0, "retries": 0, "seconds": 6177.6, "max_memory_gb": 19.17}
|
| 484 |
+
{"step": 484, "steps": 500, "lr": 2.048492114060048e-06, "loss": 0.005404806627510084, "grad_norm": 0.9028825759887695, "loss_scale": 4.0, "retries": 0, "seconds": 6192.0, "max_memory_gb": 19.17}
|
| 485 |
+
{"step": 485, "steps": 500, "lr": 2.0426246619324153e-06, "loss": 0.00011519123607037976, "grad_norm": 0.0049295080825686455, "loss_scale": 4.0, "retries": 0, "seconds": 6205.9, "max_memory_gb": 19.17}
|
| 486 |
+
{"step": 486, "steps": 500, "lr": 2.0371345979641848e-06, "loss": 0.0001340481907499741, "grad_norm": 0.012550486251711845, "loss_scale": 4.0, "retries": 0, "seconds": 6219.8, "max_memory_gb": 19.17}
|
| 487 |
+
{"step": 487, "steps": 500, "lr": 2.032022153460292e-06, "loss": 0.06867050284016199, "grad_norm": 1.6918431520462036, "loss_scale": 4.0, "retries": 0, "seconds": 6233.7, "max_memory_gb": 19.17}
|
| 488 |
+
{"step": 488, "steps": 500, "lr": 2.0272875438159797e-06, "loss": 4.9853203653330275e-05, "grad_norm": 0.0020134535152465105, "loss_scale": 4.0, "retries": 0, "seconds": 6247.7, "max_memory_gb": 19.17}
|
| 489 |
+
{"step": 489, "steps": 500, "lr": 2.022930968507718e-06, "loss": 0.0019088071981782796, "grad_norm": 0.25720250606536865, "loss_scale": 4.0, "retries": 0, "seconds": 6261.6, "max_memory_gb": 19.17}
|
| 490 |
+
{"step": 490, "steps": 500, "lr": 2.0189526110848026e-06, "loss": 0.001489763116369018, "grad_norm": 0.09999098628759384, "loss_scale": 4.0, "retries": 0, "seconds": 6275.2, "max_memory_gb": 19.17}
|
| 491 |
+
{"step": 491, "steps": 500, "lr": 2.0153526391616207e-06, "loss": 0.021687409960272613, "grad_norm": 0.9772077798843384, "loss_scale": 4.0, "retries": 0, "seconds": 6289.3, "max_memory_gb": 19.17}
|
| 492 |
+
{"step": 492, "steps": 500, "lr": 2.0121312044105927e-06, "loss": 0.0009757266102806997, "grad_norm": 0.07047022134065628, "loss_scale": 4.0, "retries": 0, "seconds": 6302.8, "max_memory_gb": 19.17}
|
| 493 |
+
{"step": 493, "steps": 500, "lr": 2.009288442555774e-06, "loss": 8.376529127929189e-05, "grad_norm": 0.004029429517686367, "loss_scale": 4.0, "retries": 0, "seconds": 6316.1, "max_memory_gb": 19.17}
|
| 494 |
+
{"step": 494, "steps": 500, "lr": 2.0068244733671474e-06, "loss": 0.027377128009801903, "grad_norm": 2.9322948455810547, "loss_scale": 4.0, "retries": 0, "seconds": 6329.7, "max_memory_gb": 19.17}
|
| 495 |
+
{"step": 495, "steps": 500, "lr": 2.004739400655566e-06, "loss": 4.5031662701624306e-05, "grad_norm": 0.002280195476487279, "loss_scale": 4.0, "retries": 0, "seconds": 6343.4, "max_memory_gb": 19.17}
|
| 496 |
+
{"step": 496, "steps": 500, "lr": 2.003033312268388e-06, "loss": 6.781029387845841e-05, "grad_norm": 0.0032118570525199175, "loss_scale": 4.0, "retries": 0, "seconds": 6356.9, "max_memory_gb": 19.17}
|
| 497 |
+
{"step": 497, "steps": 500, "lr": 2.0017062800857716e-06, "loss": 0.0008377044396920301, "grad_norm": 0.07605088502168655, "loss_scale": 4.0, "retries": 0, "seconds": 6370.6, "max_memory_gb": 19.17}
|
| 498 |
+
{"step": 498, "steps": 500, "lr": 2.000758360017648e-06, "loss": 0.0005385675257992324, "grad_norm": 0.028991760686039925, "loss_scale": 4.0, "retries": 0, "seconds": 6384.5, "max_memory_gb": 19.17}
|
| 499 |
+
{"step": 499, "steps": 500, "lr": 2.000189592001363e-06, "loss": 4.445056483022695e-05, "grad_norm": 0.0021389720495790243, "loss_scale": 4.0, "retries": 0, "seconds": 6398.4, "max_memory_gb": 19.17}
|
| 500 |
+
{"step": 500, "steps": 500, "lr": 2e-06, "loss": 0.04979770332515314, "grad_norm": 3.097243070602417, "loss_scale": 4.0, "retries": 0, "seconds": 6412.0, "max_memory_gb": 19.17}
|